Method and system for eliminating impact sound of power amplifier based on deep learning

Abstract

The invention provides a power amplifier impact sound elimination method and system based on deep learning, and the method comprises the steps: monitoring an audio signal of each audio channel, and determining whether each audio channel is about to carry out sound source switching or not; when it is confirmed that the first audio channel is about to be subjected to audio source switching, obtaining to-be-switched target audio source data and audio data sets corresponding to the audio channels; and inputting each audio data set and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs an impact sound prediction result of each audio channel according to each audio data set, the impact sound prediction model is constructed based on a long short-term memory network and is obtained through training of a historical audio data set; determining a sound source switching strategy according to the impact sound prediction result of each audio channel; and switching the sound source according to the sound source switching strategy. The sound quality of the broadcasting system and the user experience are improved.

Description

Technical Field

[0001] The present application relates to the field of audio data processing and deep learning technology, and in particular to a method and system for eliminating impact sound of an amplifier based on deep learning. Background Art

[0002] Power amplifier, also known as PA, generally refers to one of the most basic devices in an audio system. Its main function is to amplify the weaker signal input from the sound source equipment and generate a large enough current to drive the speaker to reproduce the sound.

[0003] In large-scale audio playback occasions such as broadcasting systems, theater audio, conference centers, etc., the switching process of zone power amplifier sound sources is often accompanied by harsh impact sounds. These impact sounds are mainly caused by the sudden level change or phase difference between the new and old sound source signals, which seriously affects the sound quality experience and the comfort of the audience. In the prior art, impact sound prediction methods are usually based on fixed thresholds or simple rules to eliminate power amplifier impact sounds. Their accuracy is affected by multiple factors such as environmental noise and sound source characteristics, and they are difficult to adapt to complex and changeable actual application scenarios. At the same time, existing power amplifier impact sound elimination methods usually need to detect and process the impact sound after it occurs, and cannot predict and take measures in advance. They lack intelligent learning and adaptive capabilities and cannot automatically adjust the control strategy according to environmental changes. Summary of the invention

[0004] In response to the above technical problems, the present application provides a method and system for eliminating power amplifier impact sound based on deep learning, which effectively eliminates the power amplifier impact sound and improves the sound quality and user experience of the broadcasting system.

[0005] In a first aspect, an embodiment of the present application provides a method for eliminating power amplifier impact sound based on deep learning, comprising:

[0006] Monitor the audio signals of each audio channel to confirm whether each audio channel is about to switch audio sources;

[0007] When it is confirmed that there is a first audio channel that is about to switch the audio source, obtaining the target audio source data to be switched and the audio data sets corresponding to each audio channel, wherein the audio data sets include the audio source data currently played by each audio channel and the environmental noise data;

[0008] Inputting each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs an impact sound prediction result of each audio channel according to each of the audio data sets, wherein the impact sound prediction result includes the magnitude of the impact sound generated by the audio channel and the time when the impact sound is generated, and the impact sound prediction model is constructed based on a long short-term memory network and obtained through training of a historical audio data set;

[0009] Determining a sound source switching strategy according to the impact sound prediction results of each audio channel;

[0010] The audio source switching is performed according to the audio source switching strategy.

[0011] The embodiment of the present application provides a method for eliminating power amplifier impact sound based on deep learning. When it is detected that the first audio channel is about to switch the sound source, the impact sound prediction model constructed based on the long short-term memory network is used to predict the impact sound of each audio channel, determine the size of the impact sound generated by each audio channel and the time when the impact sound is generated, and achieve accurate prediction of the timing of switching the sound source signal, providing a decision-making basis for the elimination of the subsequent power amplifier impact sound. Finally, the sound source switching strategy is automatically selected according to the impact sound prediction result, so that the embodiment of the present application can adapt to different audio environments and needs. Compared with the existing power amplifier impact sound elimination method, the power amplifier impact sound elimination method based on deep learning provided by the embodiment of the present application can predict the impact sound in advance, and then perform relevant power amplifier impact sound elimination operations before the impact sound occurs, effectively eliminating the power amplifier impact sound in the sound source switching scenario, and improving the sound quality and user experience of the broadcasting system.

[0012] Furthermore, determining the sound source switching strategy according to the impact sound prediction results of each audio channel includes:

[0013] If it is predicted that the first audio channel will not generate an impact sound, determining the sound source switching strategy is: performing sound source switching normally in the first audio channel;

[0014] If it is predicted that the impact sound generated by the first audio channel is less than a preset threshold, determining the sound source switching strategy is: using an electronic volume adjustment method to switch the sound source in the first audio channel;

[0015] If it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, the sound source switching strategy is determined as: according to the impact sound prediction results of each audio channel, a second audio channel is selected from each audio channel, and the target sound source is switched to the second audio channel.

[0016] The embodiment of the present application provides three sound source switching strategies, namely, normal sound source switching, sound source switching using electronic volume adjustment, and sound source switching using other audio channels. The embodiment of the present application selects from the above three sound source switching strategies according to the prediction results output by the impact sound prediction model. When it is predicted that the first audio channel will not generate impact sound, there is no need to perform additional impact sound elimination operations, and the sound source switching can be performed normally in the first audio channel; when it is predicted that the impact sound generated by the first audio channel is less than the preset threshold, it means that the impact sound can be eliminated by electronic volume adjustment, and the sound source switching is still performed in the first audio channel; when it is predicted that the impact sound generated by the first audio channel is greater than the preset threshold, it means that the impact sound is difficult to eliminate in the current audio channel, and continuing to switch the sound source will seriously affect the audience's sound quality experience and comfort. Therefore, according to the impact sound prediction results of each audio channel, the second audio channel for sound source switching is selected, and then the audio switching of the target sound source is performed in the second audio channel. The embodiment of the present application sets three different audio source switching strategies, so that the power amplifier impact sound elimination method provided in the embodiment of the present application can adapt to different audio environments and needs. At the same time, the solution has the characteristics of high automation, reduces the operation process of manual intervention, reduces maintenance costs, and improves the flexibility and convenience of power amplifier impact sound elimination.

[0017] In a possible implementation, the switching of the audio source in the first audio channel by electronic volume adjustment includes:

[0018] Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel;

[0019] At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

[0020] In an embodiment of the present application, when the impact sound generated by the first audio channel is less than a preset threshold, the cross-fade technology is used to eliminate the impact sound during the sound source switching process. The key to the implementation of the cross-fade technology is how to determine the volume adjustment time. If the volume adjustment time is too early, the old sound source has not been played yet, which is equivalent to switching to the new sound source in advance, giving the audience a poor experience. If the volume adjustment time is too late, there is still a risk of impact sound. Therefore, the embodiment of the present application predicts the time when the impact sound is generated in the first audio channel through the impact sound prediction model, and then uses this as a reference to determine the volume adjustment time of the electronic volume adjustment. The cross-fade technology is used before the impact sound is expected to occur, which improves the accuracy of the implementation of the cross-fade technology and further improves the user experience.

[0021] In a possible implementation, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel includes:

[0022] According to the impact sound prediction results of each audio channel and the audio signal characteristics of each audio channel, evaluating the anti-impact sound level of each audio channel;

[0023] According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel;

[0024] Switch the target audio source to the second audio channel.

[0025] In the embodiment of the present application, the anti-impact sound level of each audio channel is first evaluated, and then the second audio channel is determined in combination with the priority of the target sound source and other sound sources. For example, for important speeches or music performances, a higher priority can be set in advance to give priority to the normal playback of the sound source. At the same time, based on the anti-impact sound level evaluation results of the audio channel, the second audio channel to be switched to the current sound source is intelligently selected. When switching the sound source, it is preferred to switch to a channel that is predicted to have no impact sound or a smaller impact sound and the audio channel has no sound source being played or the sound source has a lower priority.

[0026] In a possible implementation, the impact sound prediction model is constructed based on a long short-term memory network and obtained through training with a historical audio data set, including:

[0027] Obtain historical audio data sets in various audio source switching scenarios;

[0028] Extracting a number of Mel-frequency cepstral coefficient features from the historical audio data set;

[0029] Arrange the Mel-frequency cepstral coefficient features in chronological order to construct a plurality of feature data sets;

[0030] The preset long short-term memory network is trained using the various feature data sets to obtain the impact sound prediction model.

[0031] The embodiment of the present application provides a training method for an impact sound prediction model, which realizes accurate prediction of the switching timing of the sound source signal by constructing a multi-layer LSTM (Long Short-Term Memory Network) network and combining MFCC (Mel Frequency Cepstral Coefficient) feature extraction. The LSTM (Long Short-Term Memory Network) network structure includes an input layer, a hidden layer (which may contain multiple layers of LSTM units) and an output layer. Through its unique gating mechanism, it can effectively process long-term and short-term dependencies in time series data. Among them, the forget gate can determine which old audio features (which may represent background noise or previous sound source information) should be forgotten at the current time step so that the LSTM model can focus on the characteristics of the current impact sound. For memory cell state updates, through the interaction of the input gate and the memory cell, the LSTM model can selectively add new information to the memory cell state, and allow the LSTM model to learn and retain the key features of the impact sound, which are used to eliminate the impact sound in subsequent time steps. The forget gate and updating the memory cell state enable LSTM to capture long-term dependencies in audio signals. The hidden layer of LSTM not only contains the input information of the current time step, but also integrates the memory information of the previous time step. The hidden layer enables LSTM to capture short-term dependencies in audio signals, such as the association between impact sounds and the audio frames before and after them. The output gate controls which information is passed from the memory cell state to the hidden layer and ultimately serves as the output of the current time step. Therefore, the output gate can selectively output feature information related to the impact sound elimination as needed. The LSTM network can capture the unique timing patterns and frequency characteristics of impact sounds through its gating mechanism. These feature information may appear as sudden amplitude changes in the audio signal, an increase in high-frequency components, or a surge in energy in a specific frequency band. At the same time, these feature information plays a key role in the impact sound elimination process because they are used to guide how to modify or replace the impact sound part in the original audio frame after training the LSTM model. Therefore, the impact sound prediction model finally obtained can adapt to the changes in different environmental noises and sound source characteristics, and achieve more accurate impact sound prediction.

[0032] Furthermore, the extracting of a plurality of Mel-frequency cepstral coefficient features from the historical audio data set includes:

[0033] Performing noise removal, echo cancellation, pre-emphasis, framing and windowing operations on the historical audio data set to obtain a plurality of audio frames;

[0034] Performing Fourier transform (FFT) transformation, calculating energy spectrum and Mel filter bank conversion operation on each of the audio frames to obtain corresponding logarithmic energies;

[0035] The first N coefficients are extracted from each of the logarithmic energies by discrete cosine transform as each of the Mel-frequency cepstral coefficient features.

[0036] In the second aspect, accordingly, the present application provides a power amplifier impact sound elimination system based on deep learning, including a monitoring module, an acquisition module, a prediction module, a strategy determination module and a sound source switching module;

[0037] The monitoring module is used to monitor the audio signals of each audio channel to confirm whether each audio channel is about to switch the audio source;

[0038] The acquisition module is used to acquire the target sound source data to be switched and the audio data sets corresponding to each audio channel when it is confirmed that the first audio channel is about to switch the sound source, wherein the audio data sets include the sound source data and environmental noise data currently played by each audio channel;

[0039] The prediction module is used to input each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs the impact sound prediction results of each audio channel according to each of the audio data sets, and the impact sound prediction results include the size of the impact sound generated by the audio channel and the time when the impact sound is generated. The impact sound prediction model is constructed based on a long short-term memory network and is obtained through training of a historical audio data set;

[0040] The strategy determination module is used to determine the sound source switching strategy according to the impact sound prediction results of each audio channel;

[0041] The audio source switching module is used to switch the audio source according to the audio source switching strategy.

[0042] Further, the policy determination module includes a first policy determination unit, a second policy determination unit and a third policy determination unit;

[0043] Wherein, the first strategy determination unit is used to determine the sound source switching strategy as: performing sound source switching normally in the first audio channel if it is predicted that the first audio channel will not generate an impact sound;

[0044] The second strategy determination unit is used to determine the sound source switching strategy as: using electronic volume adjustment to switch the sound source in the first audio channel if it is predicted that the impact sound generated by the first audio channel is less than a preset threshold;

[0045] The third strategy determination unit is used to determine the sound source switching strategy as follows: if it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel.

[0046] In a possible implementation, the switching of the audio source in the first audio channel by electronic volume adjustment includes:

[0047] Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel;

[0048] At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

[0049] In a possible implementation, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel includes:

[0050] According to the impact sound prediction results of each audio channel and the audio signal characteristics of each audio channel, evaluating the anti-impact sound level of each audio channel;

[0051] According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel;

[0052] Switch the target audio source to the second audio channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 : A flow chart of a method for eliminating impact sound of an amplifier based on deep learning provided in an embodiment of the present application.

[0054] Figure 2 : A flow chart of determining the sound source switching strategy in a method for eliminating impact sound of an amplifier based on deep learning provided in an embodiment of the present application.

[0055] Figure 3 : A structural diagram of a power amplifier impact sound elimination system based on deep learning provided in an embodiment of the present application.

[0056] Figure 4 : A structural diagram of a strategy determination module in a power amplifier impact sound elimination system based on deep learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments and do not serve to limit the order in which the steps are executed. In the description of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0059] Embodiment 1:

[0060] like Figure 1 As shown, embodiment 1 provides a method for eliminating power amplifier impact sound based on deep learning, including steps S1-S5:

[0061] Step S1, monitoring the audio signals of each audio channel to confirm whether each audio channel is about to switch the audio source;

[0062] Step S2: when it is confirmed that there is a first audio channel that is about to switch the audio source, obtain the target audio source data to be switched and the audio data sets corresponding to each audio channel, wherein the audio data sets include the audio source data currently played by each audio channel and the environmental noise data;

[0063] Step S3, inputting each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs the impact sound prediction results of each audio channel according to each of the audio data sets, and the impact sound prediction results include the size of the impact sound generated by the audio channel and the time when the impact sound is generated, and the impact sound prediction model is constructed based on a long short-term memory network and obtained through training of a historical audio data set;

[0064] Step S4, determining a sound source switching strategy according to the impact sound prediction results of each audio channel;

[0065] Step S5: switching the audio source according to the audio source switching strategy.

[0066] The embodiment of the present application provides a method for eliminating power amplifier impact sound based on deep learning. When it is detected that the first audio channel is about to switch the sound source, the impact sound prediction model constructed based on the long short-term memory network is used to predict the impact sound of each audio channel, determine the size of the impact sound generated by each audio channel and the time when the impact sound is generated, and achieve accurate prediction of the timing of switching the sound source signal, providing a decision-making basis for the elimination of the subsequent power amplifier impact sound. Finally, the sound source switching strategy is automatically selected according to the impact sound prediction result, so that the embodiment of the present application can adapt to different audio environments and needs. Compared with the existing power amplifier impact sound elimination method, the power amplifier impact sound elimination method based on deep learning provided by the embodiment of the present application can predict the impact sound in advance, and then perform relevant power amplifier impact sound elimination operations before the impact sound occurs, effectively eliminating the power amplifier impact sound in the sound source switching scenario, and improving the sound quality and user experience of the broadcasting system.

[0067] Further, in step S4, the sound source switching strategy is determined according to the impact sound prediction results of each audio channel, such as Figure 2 As shown, steps S401-S403 are included:

[0068] S401: If it is predicted that the first audio channel will not generate an impact sound, determine the sound source switching strategy as: normally switching the sound source in the first audio channel;

[0069] S402: If it is predicted that the impact sound generated by the first audio channel is less than a preset threshold, determine that the sound source switching strategy is: use an electronic volume adjustment method to switch the sound source in the first audio channel;

[0070] S403: If it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, determine the sound source switching strategy as follows: select a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switch the target sound source to the second audio channel.

[0071] The embodiment of the present application provides three sound source switching strategies, namely, normal sound source switching, sound source switching using electronic volume adjustment, and sound source switching using other audio channels. The embodiment of the present application selects from the above three sound source switching strategies according to the prediction results output by the impact sound prediction model. When it is predicted that the first audio channel will not generate impact sound, there is no need to perform additional impact sound elimination operations, and the sound source switching can be performed normally in the first audio channel; when it is predicted that the impact sound generated by the first audio channel is less than the preset threshold, it means that the impact sound can be eliminated by electronic volume adjustment, and the sound source switching is still performed in the first audio channel; when it is predicted that the impact sound generated by the first audio channel is greater than the preset threshold, it means that the impact sound is difficult to eliminate in the current audio channel, and continuing to switch the sound source will seriously affect the audience's sound quality experience and comfort. Therefore, according to the impact sound prediction results of each audio channel, the second audio channel for sound source switching is selected, and then the audio switching of the target sound source is performed in the second audio channel. The embodiment of the present application sets three different audio source switching strategies, so that the power amplifier impact sound elimination method provided in the embodiment of the present application can adapt to different audio environments and needs. At the same time, the solution has the characteristics of high automation, reduces the operation process of manual intervention, reduces maintenance costs, and improves the flexibility and convenience of power amplifier impact sound elimination.

[0072] In a preferred embodiment, the impact sound can also be eliminated by using a partition output switch control strategy based on the impact sound prediction result. During the sound source switching process, the seamless connection of the new and old sound sources in time and space is ensured to avoid the impact sound caused by signal interruption or overlap. Specifically, the audio output device is divided into multiple areas or partitions, and each partition corresponds to a different playback environment and audience group. For example, the conference room can be divided into a speaker area, an audience area, and a remote video conference area. According to the combination of audio partitions, volume groups, and audio types, the audio routing strategy is configured. Determine which channels of audio signals should be received by each partition, and how to adjust parameters such as volume and sound quality. Real-time monitoring of the currently playing application or sound source, and dynamically adjust the audio routing strategy according to its type and importance. For example, in a video conference, the speaker's voice can be automatically transmitted to all partitions first, and the volume can be adjusted to ensure clarity. According to the audio routing strategy, the output switch of each partition is controlled. When the impact sound prediction model predicts that the impact sound may occur, the partition output switch control strategy will quickly switch the output signal of the partition to achieve a smooth transition and effective elimination of the impact sound.

[0073] In a possible implementation, in step S402, the switching of the audio source in the first audio channel by electronic volume adjustment includes:

[0074] Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel;

[0075] At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

[0076] In an embodiment of the present application, when the impact sound generated by the first audio channel is less than a preset threshold, the cross-fade technology is used to eliminate the impact sound during the sound source switching process. The key to the implementation of the cross-fade technology is how to determine the volume adjustment time. If the volume adjustment time is too early, the old sound source has not been played yet, which is equivalent to switching to the new sound source in advance, giving the audience a poor experience. If the volume adjustment time is too late, there is still a risk of impact sound. Therefore, the embodiment of the present application predicts the time when the impact sound is generated in the first audio channel through the impact sound prediction model, and then uses this as a reference to determine the volume adjustment time of the electronic volume adjustment. The cross-fade technology is used before the impact sound is expected to occur, which improves the accuracy of the implementation of the cross-fade technology and further improves the user experience.

[0077] In a possible implementation, in step S404, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel includes:

[0078] According to the impact sound prediction results of each audio channel and the audio signal characteristics of each audio channel, evaluating the anti-impact sound level of each audio channel;

[0079] According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel;

[0080] Switch the target audio source to the second audio channel.

[0081] In the embodiment of the present application, the anti-impact sound level of each audio channel is first evaluated, and then the second audio channel is determined in combination with the priority of the target sound source and other sound sources. For example, for important speeches or music performances, a higher priority can be set in advance to give priority to the normal playback of the sound source. At the same time, based on the anti-impact sound level evaluation results of the audio channel, the second audio channel to be switched to the current sound source is intelligently selected. When switching the sound source, it is preferred to switch to a channel that is predicted to have no impact sound or a smaller impact sound and the audio channel has no sound source being played or the sound source has a lower priority.

[0082] In a possible implementation, the impact sound prediction model is constructed based on a long short-term memory network and obtained through training with a historical audio data set, including:

[0083] Obtain historical audio data sets in various audio source switching scenarios;

[0084] Extracting a number of Mel-frequency cepstral coefficient features from the historical audio data set;

[0085] Arrange the Mel-frequency cepstral coefficient features in chronological order to construct a plurality of feature data sets;

[0086] The preset long short-term memory network is trained using the various feature data sets to obtain the impact sound prediction model.

[0087] The embodiment of the present application provides a training method for an impact sound prediction model, which realizes accurate prediction of the switching timing of the sound source signal by constructing a multi-layer LSTM (Long Short-Term Memory Network) network and combining MFCC (Mel Frequency Cepstral Coefficient) feature extraction. The LSTM (Long Short-Term Memory Network) network structure includes an input layer, a hidden layer (which may contain multiple layers of LSTM units) and an output layer. Through its unique gating mechanism, it can effectively process long-term and short-term dependencies in time series data. Among them, the forget gate can determine which old audio features (which may represent background noise or previous sound source information) should be forgotten at the current time step so that the LSTM model can focus on the characteristics of the current impact sound. For memory cell state updates, through the interaction of the input gate and the memory cell, the LSTM model can selectively add new information to the memory cell state, and allow the LSTM model to learn and retain the key features of the impact sound, which are used to eliminate the impact sound in subsequent time steps. The forget gate and updating the memory cell state enable LSTM to capture long-term dependencies in audio signals. The hidden layer of LSTM not only contains the input information of the current time step, but also integrates the memory information of the previous time step. The hidden layer enables LSTM to capture short-term dependencies in audio signals, such as the association between impact sounds and the audio frames before and after them. The output gate controls which information is passed from the memory cell state to the hidden layer and ultimately serves as the output of the current time step. Therefore, the output gate can selectively output feature information related to the impact sound elimination as needed. The LSTM network can capture the unique timing patterns and frequency characteristics of impact sounds through its gating mechanism. These feature information may appear as sudden amplitude changes in the audio signal, an increase in high-frequency components, or a surge in energy in a specific frequency band. At the same time, these feature information plays a key role in the impact sound elimination process because they are used to guide how to modify or replace the impact sound part in the original audio frame after training the LSTM model. Therefore, the impact sound prediction model finally obtained can adapt to the changes in different environmental noises and sound source characteristics, and achieve more accurate impact sound prediction.

[0088] In a preferred embodiment, the acquisition of historical audio data sets in multiple sound source switching scenarios; and the use of each feature data set to train a preset long short-term memory network to obtain the impact sound prediction model are specifically as follows:

[0089] A historical audio dataset containing sound source switching scenes is collected from each audio channel. The dataset contains target sound sources and environmental noise. The target sound sources include human voices, musical instruments, electronic sound effects, etc. The environmental noise includes background noise, echo, reverberation, etc. These audio datasets contain impact sounds that may be generated when the sound source is switched. In the historical audio dataset, the time point of the sound source switching needs to be clearly marked. These marks are used in the subsequent training process to help the LSTM model learn the changing characteristics of the audio signal when the sound source is switched. At the same time, audio clips containing impact sounds also need to be marked. These marks tell the model which parts of the audio are impact sounds, so that the model can learn the characteristics of impact sounds and identify impact sounds in future predictions. The audio dataset should contain data on multiple sound source characteristics so that the model can adapt to and accurately predict the impact sounds when different sound sources are switched.

[0090] Furthermore, the extracting of a plurality of Mel-frequency cepstral coefficient features from the historical audio data set includes:

[0091] Performing noise removal, echo cancellation, pre-emphasis, framing and windowing operations on the historical audio data set to obtain a plurality of audio frames;

[0092] Performing Fourier transform (FFT) transformation, calculating energy spectrum and Mel filter bank conversion operation on each of the audio frames to obtain corresponding logarithmic energies;

[0093] The first N coefficients are extracted from each of the logarithmic energies by discrete cosine transform as each of the Mel-frequency cepstral coefficient features.

[0094] In a preferred embodiment, the Mel-frequency cepstral coefficient features are arranged in chronological order to construct several feature data sets, specifically:

[0095] The previously extracted Mel Frequency Cepstral Coefficient (MFCC) features are arranged in chronological order to form a feature data set. Each feature data set is given a clear label to indicate whether the data set contains impact sound. At the same time, the labeled feature data set is divided into a training set, a validation set, and a test set. The training set is used to train the LSTM model, the validation set is used to adjust the model parameters and prevent overfitting, and the test set is used to evaluate the performance and generalization ability of the model. This data set is used to train the LSTM model so that it can learn and identify the impact sound characteristics that may be generated when the sound source is switched, and improve the prediction accuracy of the impact sound. Optimization strategies such as backpropagation algorithm and gradient descent method are used to continuously adjust the model parameters to improve the prediction accuracy and generalization ability of the model.

[0096] The embodiment of the present application uses the extracted Mel Frequency Cepstral Coefficient (MFCC) features as the input of the LSTM (Long Short-Term Memory Network) model, and uses the sequence learning ability of the LSTM (Long Short-Term Memory Network) to analyze the long-term and short-term dependencies in the audio signal. The LSTM model predicts whether the current sound source switching will produce an impact sound based on the input MFCC features.

[0097] Embodiment 2

[0098] like Figure 3 As shown, accordingly, embodiment 2 provides a power amplifier impact sound elimination system based on deep learning, including a monitoring module 10, an acquisition module 20, a prediction module 30, a strategy determination module 40 and a sound source switching module 50;

[0099] The monitoring module 10 is used to monitor the audio signals of each audio channel to confirm whether each audio channel is about to switch the audio source;

[0100] The acquisition module 20 is used to acquire the target sound source data to be switched and the audio data sets corresponding to each audio channel when it is confirmed that the first audio channel is about to switch the sound source, wherein the audio data sets include the sound source data and environmental noise data currently played by each audio channel;

[0101] The prediction module 30 is used to input each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs the impact sound prediction results of each audio channel according to each of the audio data sets, and the impact sound prediction results include the size of the impact sound generated by the audio channel and the time when the impact sound is generated. The impact sound prediction model is constructed based on a long short-term memory network and is obtained through training of a historical audio data set;

[0102] The strategy determination module 40 is used to determine the sound source switching strategy according to the impact sound prediction results of each audio channel;

[0103] The audio source switching module 50 is used to switch the audio source according to the audio source switching strategy.

[0104] Further, such as Figure 4 As shown, the policy determination module 40 includes a first policy determination unit 401, a second policy determination unit 402 and a third policy determination unit 403;

[0105] The first strategy determination unit 401 is used to determine the sound source switching strategy as: performing sound source switching normally in the first audio channel if it is predicted that the first audio channel will not generate an impact sound;

[0106] The second strategy determination unit 402 is used to determine the sound source switching strategy as: using electronic volume adjustment to switch the sound source in the first audio channel if it is predicted that the impact sound generated by the first audio channel is less than a preset threshold;

[0107] The third strategy determination unit 403 is used to determine the sound source switching strategy as follows: if it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel.

[0108] In a possible implementation, the switching of the audio source in the first audio channel by electronic volume adjustment includes:

[0109] Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel;

[0110] At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

[0111] In a possible implementation, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel includes:

[0112] According to the impact sound prediction results of each audio channel and the audio signal characteristics of each audio channel, evaluating the anti-impact sound level of each audio channel;

[0113] According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel;

[0114] Switch the target audio source to the second audio channel.

[0115] The embodiment of the present application provides a power amplifier impact sound elimination system based on deep learning. When it is detected that the first audio channel is about to switch the sound source, the impact sound prediction model constructed based on the long short-term memory network is used to predict the impact sound of each audio channel, determine the size of the impact sound generated by each audio channel and the time when the impact sound is generated, and achieve accurate prediction of the timing of switching the sound source signal, providing a decision-making basis for the elimination of the subsequent power amplifier impact sound. Finally, the sound source switching strategy is automatically selected according to the impact sound prediction result, so that the embodiment of the present application can adapt to different audio environments and needs. Compared with the existing power amplifier impact sound elimination method, the power amplifier impact sound elimination method based on deep learning provided by the embodiment of the present application can predict the impact sound in advance, and then perform relevant power amplifier impact sound elimination operations before the impact sound occurs, effectively eliminating the power amplifier impact sound in the sound source switching scenario, and improving the sound quality and user experience of the broadcasting system.

[0116] The more detailed working principle and step flow of this embodiment can refer to, but are not limited to, the relevant records of Embodiment 1.

[0117] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for eliminating power amplifier impact sound based on deep learning, characterized in that: include: Monitor the audio signals of each audio channel to confirm whether each audio channel is about to switch audio sources; When it is confirmed that there is a first audio channel that is about to switch the audio source, obtaining the target audio source data to be switched and the audio data sets corresponding to each audio channel, wherein the audio data sets include the audio source data currently played by each audio channel and the environmental noise data; Inputting each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs an impact sound prediction result of each audio channel according to each of the audio data sets, wherein the impact sound prediction result includes the magnitude of the impact sound generated by the audio channel and the time when the impact sound is generated, and the impact sound prediction model is constructed based on a long short-term memory network and obtained through training of a historical audio data set; Determining a sound source switching strategy according to the impact sound prediction results of each audio channel; The audio source switching is performed according to the audio source switching strategy.

2. A method for eliminating power amplifier impact sound based on deep learning as claimed in claim 1, characterized in that: The step of determining a sound source switching strategy according to the impact sound prediction results of each audio channel includes: If it is predicted that the first audio channel will not generate an impact sound, determining the sound source switching strategy is: performing sound source switching normally in the first audio channel; If it is predicted that the impact sound generated by the first audio channel is less than a preset threshold, determining the sound source switching strategy is: using an electronic volume adjustment method to switch the sound source in the first audio channel; If it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, the sound source switching strategy is determined as: according to the impact sound prediction results of each audio channel, a second audio channel is selected from each audio channel, and the target sound source is switched to the second audio channel.

3. A method for eliminating power amplifier impact sound based on deep learning as claimed in claim 2, characterized in that: The method of switching the audio source by electronic volume adjustment in the first audio channel includes: Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel; At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

4. A method for eliminating power amplifier impact sound based on deep learning as claimed in claim 2, characterized in that: The selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel, comprises: evaluating the anti-impact sound level of each of the audio channels according to the impact sound prediction results of each of the audio channels and the audio signal characteristics of each of the audio channels; According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel; Switch the target audio source to the second audio channel.

5. A method for eliminating power amplifier impact sound based on deep learning as described in any one of claims 1 to 4, characterized in that: The impact sound prediction model is constructed based on a long short-term memory network and obtained by training with a historical audio data set, including: Obtain historical audio data sets in various audio source switching scenarios; Extracting a number of Mel-frequency cepstral coefficient features from the historical audio data set; Arrange the Mel-frequency cepstral coefficient features in chronological order to construct a plurality of feature data sets; The preset long short-term memory network is trained using the various feature data sets to obtain the impact sound prediction model.

6. A method for eliminating power amplifier impact sound based on deep learning as claimed in claim 5, characterized in that: The step of extracting a plurality of Mel-frequency cepstral coefficient features from the historical audio data set includes: Performing noise removal, echo cancellation, pre-emphasis, framing and windowing operations on the historical audio data set to obtain a plurality of audio frames; Performing Fourier transform (FFT) transformation, calculating energy spectrum and Mel filter bank conversion operation on each of the audio frames to obtain corresponding logarithmic energies; The first N coefficients are extracted from each of the logarithmic energies by discrete cosine transform as each of the Mel-frequency cepstral coefficient features.

7. A power amplifier impact sound elimination system based on deep learning, characterized in that: It includes monitoring module, acquisition module, prediction module, strategy determination module and sound source switching module; The monitoring module is used to monitor the audio signals of each audio channel to confirm whether each audio channel is about to switch the audio source; The acquisition module is used to acquire the target sound source data to be switched and the audio data sets corresponding to each audio channel when it is confirmed that the first audio channel is about to switch the sound source, wherein the audio data sets include the sound source data and environmental noise data currently played by each audio channel; The prediction module is used to input each of the audio data sets and the target sound source data into a preset impact sound prediction model, so that the impact sound prediction model outputs the impact sound prediction results of each audio channel according to each of the audio data sets, and the impact sound prediction results include the size of the impact sound generated by the audio channel and the time when the impact sound is generated. The impact sound prediction model is constructed based on a long short-term memory network and is obtained through training of a historical audio data set; The strategy determination module is used to determine the sound source switching strategy according to the impact sound prediction results of each audio channel; The audio source switching module is used to switch the audio source according to the audio source switching strategy.

8. The power amplifier impact sound elimination system based on deep learning as claimed in claim 7, characterized in that: The policy determination module includes a first policy determination unit, a second policy determination unit and a third policy determination unit; Wherein, the first strategy determination unit is used to determine the sound source switching strategy as: performing sound source switching normally in the first audio channel if it is predicted that the first audio channel will not generate an impact sound; The second strategy determination unit is used to determine the sound source switching strategy as: using electronic volume adjustment to switch the sound source in the first audio channel if it is predicted that the impact sound generated by the first audio channel is less than a preset threshold; The third strategy determination unit is used to determine the sound source switching strategy as follows: if it is predicted that the impact sound generated by the first audio channel is greater than a preset threshold, selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel.

9. The power amplifier impact sound elimination system based on deep learning as claimed in claim 8, characterized in that: The method of switching the audio source by electronic volume adjustment in the first audio channel includes: Determining a volume adjustment time for performing the electronic volume adjustment according to a time when the impact sound is generated in the first audio channel; At the volume adjustment moment, a cross-fade technique is used for the first audio channel to gradually reduce the volume of the current sound source of the first audio channel and gradually increase the volume of the target sound source.

10. The power amplifier impact sound elimination system based on deep learning as claimed in claim 8, characterized in that: The selecting a second audio channel from each of the audio channels according to the impact sound prediction results of each of the audio channels, and switching the target sound source to the second audio channel, comprises: According to the impact sound prediction results of each audio channel and the audio signal characteristics of each audio channel, evaluating the anti-impact sound level of each audio channel; According to the anti-impact sound level of each of the audio channels, the priority of the currently playing sound source of each of the audio channels and the priority of the target sound source, audio channel selection is performed in each of the audio channels to determine the second audio channel; Switch the target audio source to the second audio channel.

Images