Low-latency audio signal processing method and system
By splitting the audio signal and optimizing the processing flow, the delay and interruption overhead problems in active noise reduction technology are solved, low latency and efficient CPU processing capabilities are achieved, and actual noise reduction needs are met.
Patent Information
- Application Number
- CN202510902555.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-01
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2045-07-01
Smart Images
Figure CN120416732B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an audio signal processing method and system, and in particular to a low-delay audio signal processing method and system. Background Art
[0002] Noise pollution has become a serious problem in industrial production and daily life, not only damaging human hearing health but also affecting work efficiency and comfort. Therefore, reducing noise levels in the environment through advanced active noise reduction methods has become particularly important. Active noise reduction generally includes audio acquisition, signal processing, and noise reduction calculation and execution. Audio acquisition primarily involves collecting a set amount of audio data to generate an audio signal of a predetermined frame length. Subsequently, noise reduction is achieved through signal processing and noise reduction calculation and execution.
[0003] However, to reduce the latency of active noise reduction, a common method is to reduce the frame length of the audio signal to shorten the processing time of active noise reduction. However, noise reduction algorithms usually require more data, and reducing the frame length will cause interrupts to be triggered frequently. The frequent recovery of these interrupts will bring huge overhead to the processor (CPU) performing noise reduction, and multiple interrupts are required before the noise reduction algorithm processing can begin. This limits the MIPS (Million Instructions Per Second) occupied by the active noise reduction algorithm.
[0004] To resolve the conflict between interruption overhead and the real-time performance of active noise reduction, two methods are currently available:
[0005] 1) Increase the audio sampling rate. It should be noted that increasing the audio sampling rate requires more professional audio acquisition equipment, and does not fundamentally solve the interruption overhead and real-time problems.
[0006] 2) Reducing the audio signal frame length. Although this can shorten the active noise reduction processing cycle, it will cause frequent interrupt triggering and increase CPU context switching overhead. Active noise reduction algorithms usually require longer frames to ensure active noise reduction performance, resulting in a "frame length-latency" contradiction and failing to meet actual active noise reduction needs. Summary of the Invention
[0007] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a low-latency audio signal processing method and system, which can achieve low latency for active noise reduction, reduce CPU interrupt overhead, and improve the MIPS upper limit of the CPU processing algorithm.
[0008] According to the technical solution provided by the present invention, a low-latency audio signal processing method includes:
[0009] Configuring noise reduction processing parameters for performing noise reduction processing on the audio signal, wherein the noise reduction processing parameters at least include a specified frame length for noise reduction processing when performing active noise reduction on the audio signal,
[0010] Based on the configured noise reduction processing parameters, selecting a set of audio information to be noise reduced corresponding to the specified frame length for noise reduction processing, wherein the audio information to be noise reduced includes at least the acquired first audio signal to be noise reduced and the yet-to-be-acquired second audio signal to be noise reduced, and in a timing sequence of audio signal processing, the second audio signal to be noise reduced is immediately adjacent to the first audio signal to be noise reduced;
[0011] When performing audio signal processing on any set of audio information to be de-noised, obtaining a second audio signal to be de-noised, wherein when obtaining the second audio signal to be de-noised, first obtaining a second audio first sub-signal within the second audio signal to be de-noised, and performing noise reduction calculation processing on the first audio signal to be de-noised during the process of obtaining the second audio first sub-signal, and generating a de-noised audio signal after performing the noise reduction calculation processing;
[0012] After acquiring the second audio first sub-signal, transmitting and outputting the noise-reduced audio signal, and while transmitting and outputting the noise-reduced audio signal, acquiring the second audio second sub-signal within the second audio signal to be de-noised, thereafter forming the second audio signal to be de-noised based on the acquired second audio second sub-signal and the second audio first sub-signal, and using the acquired second audio signal to be de-noised as the first audio signal to be de-noised within the next set of audio information to be de-noised;
[0013] The second audio first sub-signal has a first audio frame length, the second audio second sub-signal has a second audio frame length, and the sum of the first audio frame length and the second audio frame length is consistent with the specified frame length for noise reduction processing.
[0014] When sending the noise-reduced audio signal to the output, it includes:
[0015] dividing the noise-reduced audio signal into a noise-reduced first sub-signal and a noise-reduced second sub-signal, wherein a frame length of the noise-reduced first sub-signal is consistent with a frame length of the second audio second sub-signal;
[0016] When the reception of the second audio second sub-signal is completed, the sending and output of the noise-reduced first sub-signal is completed, and thereafter, the sending and output of the noise-reduced second sub-signal is completed.
[0017] For any group of audio information to be reduced in noise, when the second audio second sub-signal is received, the second audio signal to be reduced in noise forms the first audio signal to be reduced in noise in the next group of audio information to be reduced in noise, wherein:
[0018] When the acquisition of the second audio first sub-signal in the second audio signal to be de-noised in the next set of audio information to be de-noised is completed, the sending and output of the current de-noised second sub-signal is terminated.
[0019] For any group of audio information to be de-noised, the first audio frame length of the second audio first sub-signal in the current group of audio information to be de-noised is generated at least according to the noise reduction calculation processing time performed on the first audio signal to be de-noised in the previous group.
[0020] For any set of audio information to be de-noised, when the first audio signal to be de-noised is received, a noise reduction calculation execution interrupt is triggered to perform noise reduction calculation processing on the first audio signal to be de-noised;
[0021] When the acquisition of the second audio first sub-signal is completed, the noise reduction sending interrupt is triggered, so that the noise-reduced audio signal is sent and output after entering the noise reduction sending interrupt.
[0022] The method further includes a frame length fine-tuning step, wherein the frame length fine-tuning step is used to fine-tune the first audio frame length of the second audio first sub-signal, wherein:
[0023] The frame length fine-tuning steps include:
[0024] Configuring a frame length fine-tuning period for adaptively fine-tuning the first audio frame length;
[0025] After the noise reduction processing of the audio signal undergoes a frame length fine-tuning period, within the current frame length fine-tuning period, counting external interruption information occurring in the process of obtaining each second audio first sub-signal, and generating fine-tuning weight information based on the obtained external interruption information, wherein the fine-tuning weight information includes a plurality of fine-tuning weight items;
[0026] Based on the fine-tuning weight item in the fine-tuning weight information and the pre-built fine-tuning model, a fine-tuning weight is calculated. When the fine-tuning weight is greater than the fine-tuning threshold, the frame length fine-tuning determination process is executed, and the determination state is that the frame length can be fine-tuned.
[0027] When the frame length fine-tuning judgment processing is performed and the frame length fine-tuning condition is met, the first audio frame length of the second audio first sub-signal in the corresponding group of audio information to be noise reduced is fine-tuned.
[0028] During the frame length fine-tuning period, the statistical external interrupt information includes the number of interrupt triggering and the interrupt processing time corresponding to each interrupt triggering, wherein the type of the external interrupt includes at least UART interrupt;
[0029] The fine-tuning weight items in the fine-tuning weight information include an interruption distribution uniformity item and an interruption triggering intensity item;
[0030] The frame length fine-tuning condition includes at least the average noise reduction calculation processing time within the cycle and the average interruption time within the cycle. When the sum of the average noise reduction calculation processing time within the cycle and the average interruption time within the cycle is greater than the noise reduction calculation processing time corresponding to the current group of audio information to be reduced, the frame length fine-tuning condition is met; otherwise, the frame length fine-tuning condition is not met.
[0031] The fine-tuning threshold is 1. When the fine-tuning weight is greater than 1, the frame length fine-tuning judgment process is executed, and the judgment state is that the frame length can be fine-tuned. Otherwise, the frame length fine-tuning judgment process is executed, and the judgment state is that the frame length cannot be fine-tuned.
[0032] When the frame length is updated, the first audio frame length of the second audio first sub-signal is increased by at least one audio frame length.
[0033] For the constructed fine-tuning model, we have:
[0034]
[0035] in, To fine-tune the weights, is the distribution uniformity measurement sub-item within the interrupt distribution uniformity item, is the interruption trigger continuity factor within the interruption distribution uniformity term, is the average trigger strength within the interrupt trigger strength term, is the stability factor within the interrupt trigger strength term, is the weight coefficient of the distribution uniformity measurement sub-item, The weight coefficient of the interrupt trigger continuity factor.
[0036] A low-latency audio signal processing system includes at least an audio signal processing device for actively reducing noise on an audio signal. When actively reducing noise on an audio signal, the audio signal processing device adopts the audio signal processing method described above.
[0037] The advantages of the present invention are as follows: for any group of audio information to be reduced, when obtaining the second audio signal to be reduced, the second audio first sub-signal in the second audio signal to be reduced is first obtained, and in the process of obtaining the second audio first sub-signal, noise reduction calculation processing is performed on the first audio signal to be reduced, and a noise reduced audio signal is generated after the noise reduction calculation processing is performed; after obtaining the second audio first sub-signal, the noise reduced audio signal is sent and output, and in the process of sending and outputting the noise reduced audio signal, the second audio second sub-signal in the second audio signal to be reduced is obtained; thereafter, the second audio signal to be reduced is formed based on the obtained second audio second sub-signal and the second audio first sub-signal, and the obtained second audio signal to be reduced is used as the first audio signal to be reduced in the next group of audio information to be reduced, thereby achieving low latency of active noise reduction, reducing CPU interruption overhead, and improving the MIPS upper limit of the CPU processing algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 This is a flow chart of an embodiment of audio signal processing according to the present invention.
[0039] Figure 2 Schematic diagram of an embodiment of the present invention for receiving a second audio signal to be denoised and sending a noise-reduced audio signal. DETAILED DESCRIPTION
[0040] The present invention will be further described below with reference to specific drawings and embodiments.
[0041] In order to achieve low latency for active noise reduction, reduce CPU interrupt overhead, and increase the MIPS upper limit of the CPU processing algorithm, the present invention provides a low-latency audio signal processing method. Specifically, the audio signal processing method includes:
[0042] Configuring noise reduction processing parameters for performing noise reduction processing on the audio signal, wherein the noise reduction processing parameters at least include a specified frame length for noise reduction processing when performing active noise reduction on the audio signal,
[0043] Based on the configured noise reduction processing parameters, selecting a set of audio information to be noise reduced corresponding to the specified frame length for noise reduction processing, wherein the audio information to be noise reduced includes at least the acquired first audio signal to be noise reduced and the yet-to-be-acquired second audio signal to be noise reduced, and in a timing sequence of audio signal processing, the second audio signal to be noise reduced is immediately adjacent to the first audio signal to be noise reduced;
[0044] When performing audio signal processing on any set of audio information to be de-noised, obtaining a second audio signal to be de-noised, wherein when obtaining the second audio signal to be de-noised, first obtaining a second audio first sub-signal within the second audio signal to be de-noised, and performing noise reduction calculation processing on the first audio signal to be de-noised during the process of obtaining the second audio first sub-signal, and generating a de-noised audio signal after performing the noise reduction calculation processing;
[0045] After acquiring the second audio first sub-signal, transmitting and outputting the noise-reduced audio signal, and while transmitting and outputting the noise-reduced audio signal, acquiring the second audio second sub-signal within the second audio signal to be de-noised, thereafter forming the second audio signal to be de-noised based on the acquired second audio second sub-signal and the second audio first sub-signal, and using the acquired second audio signal to be de-noised as the first audio signal to be de-noised within the next set of audio information to be de-noised;
[0046] The second audio first sub-signal has a first audio frame length, the second audio second sub-signal has a second audio frame length, and the sum of the first audio frame length and the second audio frame length is consistent with the specified frame length for noise reduction processing.
[0047] It should be understood that the audio signal processing method of the present invention mainly refers to the processing operation when actively reducing the noise of the audio signal. Figure 1 A flow chart of an embodiment of the present invention for processing an audio signal is shown in FIG. Figure 1 It can be seen that when processing audio signals, noise reduction processing parameters should be configured. Specifically, the noise reduction processing parameters include at least a specified frame length for noise reduction processing. The specified frame length for noise reduction processing specifically refers to the fixed frame length used in the active noise reduction processing mentioned above; the specified frame length for noise reduction processing should be based on whether it can meet the requirements for noise reduction processing. Of course, the noise reduction processing parameters should also include other necessary parameters, such as the sampling frequency for audio signal acquisition. The noise reduction processing parameters can be selected as needed to meet the requirements for noise reduction processing of the audio signal.
[0048] After setting the sampling frequency of the audio signal, the noise reduction process specifies that the frame length specifically refers to the number of audio sampling points that should be included, and the corresponding time length is determined according to the number of corresponding audio sampling points, such as Figure 2 An embodiment of the noise reduction process with a specified frame length of 15 frames is shown in FIG. , that is, the specified frame length of the noise reduction process includes 15 audio sampling points. According to the sampling frequency of the audio sampling points, the time length corresponding to the specified frame length of the noise reduction process can be obtained. For other situations, please refer to Figure 2 As well as the explanation here, no more examples are given one by one.
[0049] In order to implement audio signal processing, after configuring the noise reduction processing parameters, a group of audio information to be noise reduced corresponding to the noise reduction processing frame length should be selected. In one embodiment of the present invention, each group of audio information to be noise reduced should include an acquired first audio signal to be noise reduced and a second audio signal to be noise reduced to be acquired. Specifically: the first audio signal to be noise reduced is an audio signal that has been sampled, and the second audio signal to be noise reduced is an audio signal to be sampled and generated. It should be understood that the corresponding frame lengths of the first audio signal to be noise reduced and the second audio signal to be noise reduced should both be the frame length specified for the noise reduction processing.
[0050] The timing of audio signal processing specifically refers to the corresponding time sequence of audio acquisition, audio reception, and noise reduction processing. The second audio signal to be denoised should be immediately adjacent to the first audio signal to be denoised. For example, after receiving and acquiring the first audio signal to be denoised, the second audio signal to be denoised can be acquired immediately afterwards. In other words, the first audio signal to be denoised and the second audio signal to be denoised are received sequentially. When performing audio signal denoising, the present invention treats the acquired first audio signal to be denoised and the subsequently acquired second audio signal with denoised audio as a set of audio information to be denoised. It should be understood that for the acquired first audio signal to be denoised, denoising has not yet been performed on it, that is, the noise reduction calculation processing has not been performed.
[0051] It should be noted that after selecting and forming a group of audio signals to be noise reduced, the same signal processing is performed on each group of audio signals to be noise reduced, that is, when processing the audio signal, the corresponding signal processing will be repeatedly performed, such as: a second audio signal to be noise reduced should be obtained, wherein obtaining the second audio signal to be noise reduced specifically refers to receiving the audio sampling points corresponding to the second audio signal to be noise reduced, and the method of obtaining the second audio signal to be noise reduced and the first audio signal to be noise reduced are consistent. If the audio sampling points are received in the same manner, the specific method of receiving the audio sampling points may be consistent with the existing technology, and will not be repeated here.
[0052] Depend on Figure 1 It can be seen that when obtaining the second audio signal to be denoised, the present invention divides the second audio signal to be denoised into two parts: the second audio first sub-signal and the second audio second sub-signal. That is, when receiving the second audio signal to be denoised, it includes two receiving stages, wherein the first receiving stage receives the second audio first sub-signal, and the second stage receives the second audio second sub-signal. The specific meaning of reception can be referred to the above description of obtaining the second audio signal to be denoised.
[0053] In a specific implementation, the second audio first sub-signal has a first audio frame length, the second audio second sub-signal has a second audio frame length, and the sum of the frame lengths of the first audio frame length and the second audio frame length is consistent with the specified frame length for noise reduction processing. Therefore, after dividing the second audio signal to be denoised into two parts, the number of audio sampling points contained in the second audio first sub-signal and the second audio second sub-signal is equal to the number of specified audio sampling points in the specified frame length for noise reduction processing. Therefore, the second audio signal to be denoised can be composed of the second audio first sub-signal and the second audio second sub-signal. Unlike the prior art, the audio sampling points in the second audio signal to be denoised in the present invention are not received all at once, but include two receiving stages. It should be understood that the two receiving stages are mainly to meet the requirements of performing noise reduction calculation processing on the first audio signal to be denoised and transmitting the output after the noise reduction calculation processing under low latency conditions.
[0054] In a specific implementation, a second audio first sub-signal can be formed based on the previously received audio sampling points, such as receiving audio sampling points with the same frame length as the first audio frame and serving as the second audio first sub-signal. In one embodiment of the present invention, during the process of receiving and forming the second audio first sub-signal, noise reduction computational processing is performed on the first audio signal to be de-noised. It should be understood that performing noise reduction computational processing on the first audio signal to be de-noised, i.e., performing active noise reduction on the first audio signal to be de-noised, can generate a de-noised audio signal after performing noise reduction computational processing on the first audio signal to be de-noised. The noise reduction computational processing performed on the first audio signal to be de-noised can be consistent with the prior art and can be selected as needed to satisfy the requirement of performing noise reduction processing on the first audio signal to be de-noised.
[0055] As can be seen from the above description, the frame length of the second audio first sub-signal should be no less than the time required to perform noise reduction calculations on the first audio signal to be de-noised. After receiving the second audio first signal, the de-noised audio signal should be transmitted and output. This transmission and output of the de-noised audio signal can be performed in a manner consistent with existing techniques, such as via DMA, and will not be further described here. Acquisition of the second audio first sub-signal specifically refers to the completion of receiving the audio sampling points required to form the second audio first sub-signal.
[0056] Generally, after acquiring the second audio first sub-signal and transmitting the noise-reduced audio signal, a second audio second sub-signal should be acquired. Specifically, during the transmission and output of the noise-reduced audio signal, the second audio second sub-signal within the second audio signal to be noise-reduced is acquired. The method for acquiring the second audio second sub-signal can be referred to above and will not be further described here. Once the second audio second sub-signal has been acquired, the second audio signal to be noise-reduced is obtained. This acquired second audio signal to be noise-reduced can then be used as the first audio signal to be noise-reduced within the next set of audio information to be noise-reduced. The same audio signal processing described above is then performed on the next set of audio information to be noise-reduced.
[0057] From the above description, it can be seen that the present invention can effectively solve the "frame length-delay" contradiction in the prior art, that is, when using noise reduction processing to specify the frame length, noise reduction calculation processing is performed on the noise-reduced first audio signal in the process of obtaining the second audio first sub-signal, and after obtaining the second audio first sub-signal, the noise-reduced audio signal is sent and output; and in the process of the second audio second sub-signal, the noise-reduced audio signal can be sent and output synchronously, which can effectively reduce the delay in the existing active noise reduction and achieve an extremely low-latency audio processing effect.
[0058] In one embodiment of the present invention, for any group of audio information to be de-noised, the first audio frame length of the second audio first sub-signal in the current group of audio information to be de-noised is generated at least according to the noise reduction calculation processing time performed on the first audio signal to be de-noised in the previous group.
[0059] To further reduce latency, the first audio frame length of the second audio first sub-signal can be generated based on the noise reduction calculation time performed on the previous group of first audio signals to be de-noised. For example, when performing audio signal processing on the third group of audio information to be de-noised according to the aforementioned audio signal processing sequence, the first audio frame length of the second audio first sub-signal within the third group of audio information to be de-noised can be determined by counting the corresponding times of the noise reduction calculation processing performed on the first audio signals to be de-noised within the second group of audio signals to be de-noised. That is, after counting the corresponding times of the noise reduction calculation processing performed on the first audio signals to be de-noised within the second group of audio signals to be de-noised, the first audio frame length of the second audio first sub-signal within the third group can be configured to correspond to the counted corresponding times of the noise reduction calculation processing. Of course, the first audio frame length should be equal to or slightly greater than the counted corresponding times of the noise reduction calculation processing, which is specifically related to the sampling frequency within the noise reduction processing parameters. It is understood that after determining the first audio frame length, it can be determined that the second audio first sub-signal has corresponding audio sampling points.
[0060] Initially, the first audio frame length and the second audio frame length are set according to the noise reduction calculation process, such as Figure 2In the initial case, the second audio first sub-signal may include 2 audio sampling points, and the second audio second sub-signal should include 13 audio sampling points. Figure 2 It can be seen that after setting the first audio frame length and the second audio frame length, the first audio signal is received. At this time, the received audio signal is only equivalent to the second audio signal to be denoised. At this time, there is no noise-reduced audio signal, and there is no sending or output operation of the noise-reduced audio signal. From the above description, it can be seen that after receiving the first audio signal, the first audio signal to be denoised within a group of audio information to be denoised can be formed. Thereafter, together with the next second audio signal to be denoised to be obtained, the first group of audio information to be denoised can be formed. Figure 2 In the figure, the green part is the first audio signal to be de-noised in the first group of audio information to be de-noised, and the orange part is the second audio signal to be de-noised. Figure 2 In the data receiving part, "frame 1" is the second audio first sub-signal of the present invention, and "frame 2" is the second audio second sub-signal of the present invention.
[0061] In one embodiment of the present invention, when the noise-reduced audio signal is sent and output, the method includes:
[0062] dividing the noise-reduced audio signal into a noise-reduced first sub-signal and a noise-reduced second sub-signal, wherein a frame length of the noise-reduced first sub-signal is consistent with a frame length of the second audio second sub-signal;
[0063] When the reception of the second audio second sub-signal is completed, the sending and output of the noise-reduced first sub-signal is completed, and thereafter, the sending and output of the noise-reduced second sub-signal is completed.
[0064] In order to reduce the interrupt triggering during transmission and output, the noise-reduced audio signal can be divided into two parts for transmission and output. For example, the noise-reduced audio signal can be divided into a first noise-reduced sub-signal and a second noise-reduced sub-signal. It can be understood that the division here is only the order in which the corresponding audio sampling points are sent, and will not change the content of the noise-reduced audio signal. In specific implementation, the frame length of the first noise-reduced sub-signal is consistent with the frame length of the second audio second sub-signal, such as Figure 2 In the data transmission part, the green part is the noise-reduced audio signal, and the noise-reduced audio signal is Figure 2 The data receiving part corresponds to the green audio signal. In the data sending part, frames 1-13 are the frame lengths of the first sub-signal with noise reduction. In the corresponding data receiving part, frames 2-13 are the frame lengths of the second sub-signal of the second audio. When the frame lengths of the two are equal, interrupt triggering can be reduced, further reducing latency.
[0065] In one embodiment of the present invention, for any group of audio information to be reduced in noise, when the second audio second sub-signal is received, the second audio signal to be reduced in noise forms the first audio signal to be reduced in noise in the next group of audio information to be reduced in noise, wherein:
[0066] When the acquisition of the second audio first sub-signal in the second audio signal to be de-noised in the next set of audio information to be de-noised is completed, the sending and output of the current de-noised second sub-signal is terminated.
[0067] Specifically, for each group of audio information to be noise reduced, when the second audio second sub-signal is received, the first audio signal to be noise reduced can be formed by the second audio second sub-signal and the above-mentioned second audio first sub-signal. At the same time, the second audio signal to be noise reduced forms the first audio signal to be noise reduced that has been obtained in the next group of audio information to be noise reduced.
[0068] Since the frame length of the noise-reduced first sub-signal is consistent with the frame length of the second audio second sub-signal, when the reception of the second audio second sub-signal is completed, the noise-reduced first sub-signal is sent and outputted. In the process of acquiring the second audio first sub-signal in the next set of audio information to be noise-reduced, the noise-reduced second sub-signal is synchronously sent and outputted. When the acquisition of the second audio first sub-signal in the next set of audio information to be noise-reduced is completed, the sending and outputting of the current noise-reduced second sub-signal should be terminated. It should be understood that the sending status of the noise-reduced second sub-signal should be related to the frame length of the current noise-reduced second sub-signal and the first audio frame length corresponding to the second audio first signal. For details, please refer to the corresponding description below, such as Figure 2 shown.
[0069] In one embodiment of the present invention, for any set of audio information to be de-noised, when the first audio signal to be de-noised is received, a noise reduction calculation execution interrupt is triggered to perform noise reduction calculation processing on the first audio signal to be de-noised;
[0070] When the acquisition of the second audio first sub-signal is completed, the noise reduction sending interrupt is triggered, so that the noise-reduced audio signal is sent and output after entering the noise reduction sending interrupt.
[0071] To accurately execute the aforementioned audio signal processing, when the first audio signal to be de-noised is received, a noise reduction calculation execution interrupt is triggered to execute noise reduction calculation processing on the first audio signal to be de-noised. Specifically, as can be seen from the above description, completion of reception of the first audio signal to be de-noised means completion of reception of the second audio sub-signal, and the second audio signal to be de-noised within the previous group of audio information to be de-noised is used to form the first audio signal to be de-noised within the current group of audio information to be de-noised. When the noise reduction calculation execution interrupt is triggered, an interrupt routine is entered, thereby executing noise reduction calculation processing on the first audio signal to be de-noised.
[0072] It should be noted that triggering and entering the noise reduction calculation execution interrupt primarily performs noise reduction calculation processing on the noise-reduced first audio signal and does not affect the reception of the second audio first sub-signal. After the second audio first sub-signal is received, the noise reduction transmission interrupt is triggered, and after entering the noise reduction transmission interrupt, the noise-reduced audio signal is transmitted and output.
[0073] As can be seen from the above description, when performing audio signal processing for each set of audio information to be reduced, two interrupts will be triggered. These two interrupts can respectively enter the noise reduction calculation processing and send the reduced audio signal for output. Therefore, the number of interrupt triggers when performing active noise reduction on each set of audio information to be reduced can be reduced, thereby reducing the interrupt overhead of the CPU (central processing unit) and improving the MIPS upper limit of the CPU processing algorithm.
[0074] As can be seen from the above description, the first audio frame length of the second audio first sub-signal within each set of audio information to be noise reduced can be determined based on the time it takes to perform noise reduction calculations on the first audio signal to be noise reduced within the previous set of audio information to be noise reduced. However, during the audio information processing process, external interruptions may occur, such as during the acquisition of the second audio first sub-signal. In particular, if these interruptions occur multiple times, this can lead to deviations in the statistical execution time of the noise reduction calculations, thereby affecting the accuracy of the configuration of the first audio frame length. It should be understood that if the configuration of the first audio frame length is not precise, it will result in new delays.
[0075] In order to improve the accuracy of the first audio frame length configuration and reduce the delay of audio processing, an embodiment of the present invention further includes a frame length fine-tuning step, so as to fine-tune the first audio frame length of the second audio first sub-signal by using the frame length fine-tuning step, wherein:
[0076] The frame length fine-tuning steps include:
[0077] Configuring a frame length fine-tuning period for adaptively fine-tuning the first audio frame length;
[0078] After the noise reduction processing of the audio signal undergoes a frame length fine-tuning period, within the current frame length fine-tuning period, counting external interruption information occurring in the process of obtaining each second audio first sub-signal, and generating fine-tuning weight information based on the obtained external interruption information, wherein the fine-tuning weight information includes a plurality of fine-tuning weight items;
[0079] Based on the fine-tuning weight item in the fine-tuning weight information and the pre-built fine-tuning model, a fine-tuning weight is calculated. When the fine-tuning weight is greater than the fine-tuning threshold, the frame length fine-tuning determination process is executed, and the determination state is that the frame length can be fine-tuned.
[0080] When the frame length fine-tuning judgment processing is performed and the frame length fine-tuning condition is met, the first audio frame length of the second audio first sub-signal in the corresponding group of audio information to be noise reduced is fine-tuned.
[0081] Specifically, the frame length fine-tuning step can be used to fine-tune the first audio frame length to avoid deviations in the first audio frame length setting due to external interruptions. When performing adaptive fine-tuning, a frame length fine-tuning period should be set. The frame length fine-tuning period is the number of times the second audio first sub-signal is received, starting from the second audio signal to be de-noised within the first set of audio information to be de-noised. For example, the frame length fine-tuning period can be 10. In this case, after 10 sets of audio information to be de-noised, that is, after performing the above-mentioned noise reduction calculation process 10 times, the frame length fine-tuning step of the present invention is executed. Of course, the frame length fine-tuning period can also be other values, which can be selected according to actual needs and will not be further described here.
[0082] In a specific implementation, after setting the frame length fine-tuning period, a sliding window approach is used to determine the frame length fine-tuning step to be executed. For example, starting from the initial sliding window, when the frame length fine-tuning period is satisfied, the corresponding frame length fine-tuning step is executed. For example, if the frame length fine-tuning period is 10, the frame length fine-tuning step is executed once from the first group of audio information to be de-noised to the tenth group of audio information to be de-noised. Furthermore, the frame length fine-tuning step is also executed once from the second group of audio information to be de-noised to the eleventh group of audio information to be de-noised, and so on for other situations. Of course, the frame length fine-tuning step can also be executed in other ways, such as executing the frame length fine-tuning step once every ten groups of audio information to be de-noised. In this case, the frame length fine-tuning step is executed once from the first group of audio information to be de-noised to the tenth group of audio information to be de-noised, and once from the eleventh group of audio information to be de-noised to the twentieth group of audio information to be de-noised. The specific method for executing the frame length fine-tuning step can be selected as needed and will not be listed here.
[0083] After setting the above-mentioned frame length fine-tuning period, when the signal processing performed on the audio signal undergoes the set frame length fine-tuning period, the external interruption information within the current frame length fine-tuning period should be counted, wherein the external interruption information is at least the external interruption that occurs in the process of obtaining the second audio first sub-signal. From the above description, it can be seen that the external interruption should be other interruptions except the noise reduction calculation execution interruption and the noise reduction sending interruption; in addition, the external interruption to be counted should be the external interruption that occurs in the process of obtaining the second audio first sub-signal in each group of audio information to be noise reduced within the frame length fine-tuning period. Therefore, the number of statistics should be consistent with the frame length fine-tuning period, and the method and process of counting the external interruption information can be consistent with the existing technology.
[0084] Based on the statistically collected external interrupt information, fine-tuning weight information can be generated. The fine-tuning weight information can include several fine-tuning weight items. In specific implementation, during the frame length fine-tuning period, the statistically collected external interrupt information includes the number of interrupt triggers and the interrupt processing time corresponding to each interrupt trigger, where the type of external interrupt includes at least UART (Universal Asynchronous Receiver / Transmitter) interrupts. The fine-tuning weight items in the fine-tuning weight information include an interrupt distribution uniformity item and an interrupt trigger intensity item. Of course, the fine-tuning weight information and fine-tuning weight items can also be other situations, which can be selected as needed to meet the fine-tuning requirements of the present invention for the first audio frame length.
[0085] After obtaining the fine-tuning weight item, the fine-tuning weight item is substituted into the pre-built fine-tuning model, thereby calculating the fine-tuning weight of the current frame length fine-tuning period. Thereafter, the fine-tuning weight is compared with the fine-tuning threshold. When the calculated fine-tuning weight is greater than the fine-tuning threshold, the first audio frame length should be fine-tuned. Otherwise, the first audio frame length may not be fine-tuned. Among them, when the fine-tuning weight is compared with the fine-tuning threshold, the frame length fine-tuning judgment processing is executed. When the fine-tuning weight is greater than the fine-tuning threshold, the judgment state of the frame length fine-tuning judgment processing is that the frame length can be fine-tuned.
[0086] During specific implementation, when the judgment state of the frame length fine-tuning judgment processing is that the frame length can be fine-tuned and the frame length fine-tuning conditions are met, the first audio frame length of the second audio first sub-signal in the corresponding group of audio information to be noise reduced is fine-tuned, wherein the audio information to be noise reduced in the corresponding group specifically refers to the next group of audio information to be noise reduced of the group of audio information to be noise reduced corresponding to the frame length fine-tuning period. For example, when the above-mentioned frame length fine-tuning period is 10, the audio information to be noise reduced in the corresponding group may be the next group of audio information to be noise reduced after 10 groups of audio information to be noise reduced. If the frame length fine-tuning period experienced is: from the first group of audio information to be noise reduced to the tenth group of audio information to be noise reduced, then the audio information to be noise reduced in the corresponding group is the eleventh group of audio information to be noise reduced.
[0087] In one embodiment of the present invention, the constructed fine-tuning model has:
[0088]
[0089] in, To fine-tune the weights, is the distribution uniformity measurement sub-item within the interrupt distribution uniformity item, is the interruption trigger continuity factor within the interruption distribution uniformity term, is the average trigger strength within the interrupt trigger strength term, is the stability factor within the interrupt trigger strength term, is the weight coefficient of the distribution uniformity measurement sub-item, The weight coefficient of the interrupt trigger continuity factor.
[0090] From the above description, it can be seen that the interrupt distribution uniformity item may include the distribution uniformity metric sub-item and interrupt trigger continuity factor , the interrupt trigger strength term includes the average trigger strength and stability factor , weight coefficient and weight coefficient The sum should be 1. Generally, the weight coefficient The value can be 0.6, the weight coefficient The acceptable value is 0.4.
[0091] When determining the distribution uniformity metric , interrupt trigger continuity factor , average trigger strength and stability factor After that, substitute the above fine-tuning model to get the corresponding fine-tuning weight Q, and then calculate the fine-tuning weight Q. Next, we will determine the distribution uniformity metric sub-item. , interrupt trigger continuity factor , average trigger strength and stability factor Explain the situation.
[0092] If the frame length fine-tuning period is 10, the statistical external interrupt information may include 10 interrupt triggering frequency sequences, and thus: ,in, is the number of external interruptions that occur during the process of obtaining the first sub-signal of the second audio in the first group of audio information to be noise reduced, For the The number of external interruptions that occur during the process of obtaining the second audio first sub-signal from the set of audio information to be noise reduced, and so on for other cases. If 5 external interruptions occur during the process of obtaining the first sub-signal of the second audio in the first group of audio information to be reduced, the number of Should be 5. For other values of times, please refer to the instructions here.
[0093] For the distribution uniformity metric , then:
[0094]
[0095] From the above distribution uniformity measurement sub-item From the calculation expression, we can see that the distribution uniformity metric sub-item The value range is [0, 1], where the distribution uniformity metric When it is 0, it indicates that the external interrupt trigger is completely unevenly distributed, and the distribution uniformity measurement sub-item When it is 1, it indicates that the external interrupt trigger is completely evenly distributed. At this time, the number of ~Number of times The values of are all equal, and n is the value of the frame length fine-tuning period.
[0096] For interrupt trigger continuity factor (Trigger continuity factor, TCF), then:
[0097] =Number of non-zero trigger frames / n-length of the longest zero trigger sequence / n;
[0098] In specific implementation, for triggering non-zero frame number, there are: ~Number of times For the longest zero trigger sequence length, that is, the length of consecutive non-zero values, such as the number of times ~Number of times If both are not 0, then the longest zero trigger sequence length is 5. ~Number of times In the example, the corresponding sequence is: 2, 0, 3, 0, 4, 0, 1, 0, 2, 0, so the longest zero trigger sequence length is 1. The other cases are similar and will not be given examples here.
[0099] Continuity factor triggered by the above interruption The calculation method shows that the trigger coverage of external interrupt triggering can be characterized by the number of non-zero trigger frames / n, and the longest zero trigger sequence length / n can be used to punish long periods without external interrupt triggering.
[0100] For average trigger strength (Average trigger intensity, ATI), then:
[0101] .
[0102] For the stability factor (Stability factor, TSF), then:
[0103]
[0104] in, (Coefficient of Variation, CV) is the coefficient of variation, stability factor The value range is: (0,1], the stability factor The larger the value of , the more stable the representation.
[0105] In one embodiment of the present invention, the fine-tuning threshold is 1. When the fine-tuning weight is greater than 1, the frame length fine-tuning determination process is executed if the frame length can be fine-tuned; otherwise, the frame length fine-tuning determination process is executed if the frame length cannot be fine-tuned.
[0106] When the frame length is updated, the first audio frame length of the second audio first sub-signal is increased by at least one audio frame length.
[0107] When the fine-tuning model uses the aforementioned expression, the fine-tuning threshold should be 1. Therefore, when the fine-tuning weight is greater than 1, the frame length fine-tuning determination process is executed, indicating that the frame length can be fine-tuned. Otherwise, the frame length fine-tuning determination process is executed, indicating that the frame length cannot be fine-tuned. However, when the fine-tuning model uses other expressions, the corresponding fine-tuning threshold can be set accordingly, and these examples are not provided here.
[0108] In one embodiment of the present invention, the frame length fine-tuning condition includes at least the average noise reduction calculation processing time within the period and the average interruption time within the period, wherein when the sum of the average noise reduction calculation processing time within the period and the average interruption time within the period is greater than the noise reduction calculation processing time corresponding to the current group of audio information to be noise reduced, the frame length fine-tuning condition is met; otherwise, the frame length fine-tuning condition is not met.
[0109] In specific implementations, the above-mentioned external interrupt information can be counted using commonly used technical means in this technical field to obtain the average noise reduction calculation processing time and the average interruption time within the cycle. For example, when the frame length fine-tuning period is n, the time for each noise reduction calculation execution within the frame length fine-tuning period is counted. Thereafter, the n times of the noise reduction calculation execution are averaged to obtain the average noise reduction calculation processing time within the cycle. Similarly, the interruption processing time of each external interruption in the process of obtaining the first sub-signal of the second audio is counted. Thereafter, the interruption processing time corresponding to all external interruptions is added and multiplied by the total number of external interruptions to calculate the average interruption time within the cycle.
[0110] The sum of the average noise reduction processing time and the average interruption time within a period. Specifically, this refers to the cumulative sum of the average noise reduction processing time and the average interruption time within a period. The noise reduction processing time for the current group of audio information to be noise reduced refers to the noise reduction processing time for the nth group of audio information to be noise reduced.
[0111] In one embodiment of the present invention, when the sum of the average noise reduction calculation processing time within a period and the average interruption time within the period is greater than the noise reduction calculation processing time corresponding to the current group of audio information to be reduced, it is considered that the frame length fine-tuning condition is met; otherwise, it is considered that the frame length fine-tuning condition is not met.
[0112] When updating the frame length, the first audio frame length of the second audio first sub-signal is increased by at least one audio frame length. For example, in the above description, if the first audio frame length includes two audio sampling points, then after increasing the audio frame length by one, the first audio frame length should now include three audio sampling points. It should be noted that when the frame length fine-tuning conditions are not met, the first audio frame length within the aforementioned n+1 sets of audio information to be de-noised should be determined based on the noise reduction calculation processing time for the first audio signal to be de-noised within the n sets of audio information to be de-noised. The specific determination method can be found in the above description and is not further elaborated here.
[0113] Figure 2 In the figure, n1 and n3 are the adjusted first audio frame lengths. From the above description, it can be seen that n1+n2=n3+n4, and the cumulative sum of n1+n2 is consistent with the value corresponding to the specified frame length of the above-mentioned noise reduction processing.
[0114] It is understandable that when the first audio frame length is configured as described above or the first audio frame length is fine-tuned by the frame length fine-tuning step, the first audio frame length in the next set of audio information to be noise-reduced may be unequal to the frame length of the second sub-signal in the currently transmitted output that has been noise-reduced. Figure 2 In the figure, n1 is not equal to the frame length corresponding to the green part frame 1-2, which will be explained below in conjunction with the corresponding frame length of the first audio frame and the corresponding frame length of the noise-reduced second sub-signal.
[0115] Specifically, the noise-reduced audio signal is generally transmitted via DMA (Direct Memory Access). When the first audio frame length is greater than the corresponding frame length of the noise-reduced second sub-signal, after the noise-reduced second sub-signal is transmitted, the corresponding second audio first sub-signal is still being received. Since DMA generally uses cyclic transmission, the DMA cyclic transmission method can be maintained at this time. For example, the corresponding number of audio points in the noise-reduced first sub-signal can be transmitted. The number of audio points should be the difference between the first audio frame length and the corresponding frame length of the noise-reduced second sub-signal. In this case, the frame length of the audio information transmitted outward must be greater than the frame length specified for the noise reduction processing.
[0116] When the first audio frame length is less than the corresponding frame length of the noise-reduced second sub-signal, when the corresponding second audio first sub-signal is received, the noise-reduced second sub-signal has not been sent. At this time, the sending of the noise-reduced second sub-signal will be terminated, and the noise-reduced audio information in the current group of audio information to be noise-reduced will be sent and output. Figure 2When n1 is less than the frame length corresponding to frame 1-2, the transmission of the green portion frame 1-2 is terminated, and the noise-reduced audio signal formed in the second group of audio signals to be noise-reduced in the orange portion is transmitted and output.
[0117] Therefore, it can be seen from the above description that in the process of sending the noise-reduced audio signal outward, there may be a situation where the frame length of the sent noise-reduced audio signal is inconsistent with the specified frame length for noise reduction processing. At this time, the specified processing can be performed through subsequent audio processing steps, such as directly playing the corresponding audio quotation output. It can be understood that although in some cases, the frame length of the noise-reduced audio signal is inconsistent with the specified frame length for noise reduction processing, since the difference between the first audio frame length and the corresponding frame length of the noise-reduced second sub-signal is generally small, it generally does not affect the subsequent specified processing of the audio signal, that is, it does not affect the subsequent processing of active noise reduction.
[0118] From the above description, a low-latency audio signal processing system can be obtained. In one embodiment of the present invention, it includes at least an audio signal processing device for actively reducing noise on an audio signal, wherein when actively reducing noise on an audio signal, the audio signal processing device adopts the audio signal processing method described above.
[0119] Specifically, the audio signal processing device may adopt a computer terminal in the prior art that can perform the above-mentioned noise reduction processing. The manner and process of the audio signal processing device actively reducing the noise of the audio signal may refer to the above description and will not be repeated here.
Claims
1. A low-latency audio signal processing method, characterized in that: The audio signal processing method comprises: Configuring noise reduction processing parameters for performing noise reduction processing on the audio signal, wherein the noise reduction processing parameters at least include a specified frame length for noise reduction processing when performing active noise reduction on the audio signal, Based on the configured noise reduction processing parameters, selecting a set of audio information to be noise reduced corresponding to the specified frame length for noise reduction processing, wherein the audio information to be noise reduced includes at least the acquired first audio signal to be noise reduced and the yet-to-be-acquired second audio signal to be noise reduced, and in a timing sequence of audio signal processing, the second audio signal to be noise reduced is immediately adjacent to the first audio signal to be noise reduced; When performing audio signal processing on any set of audio information to be de-noised, obtaining a second audio signal to be de-noised, wherein when obtaining the second audio signal to be de-noised, first obtaining a second audio first sub-signal within the second audio signal to be de-noised, and performing noise reduction calculation processing on the first audio signal to be de-noised during the process of obtaining the second audio first sub-signal, and generating a de-noised audio signal after performing the noise reduction calculation processing; After acquiring the second audio first sub-signal, transmitting and outputting the noise-reduced audio signal, and while transmitting and outputting the noise-reduced audio signal, acquiring the second audio second sub-signal within the second audio signal to be de-noised, thereafter forming the second audio signal to be de-noised based on the acquired second audio second sub-signal and the second audio first sub-signal, and using the acquired second audio signal to be de-noised as the first audio signal to be de-noised within the next set of audio information to be de-noised; The second audio first sub-signal has a first audio frame length, the second audio second sub-signal has a second audio frame length, and a sum of the first audio frame length and the second audio frame length is consistent with a specified frame length for noise reduction processing; For any group of audio information to be de-noised, the first audio frame length of the second audio first sub-signal in the current group of audio information to be de-noised is generated at least according to the noise reduction calculation processing time performed on the first audio signal to be de-noised in the previous group; The method further includes a frame length fine-tuning step, wherein the frame length fine-tuning step is used to fine-tune the first audio frame length of the second audio first sub-signal, wherein: The frame length fine-tuning steps include: Configuring a frame length fine-tuning period for adaptively fine-tuning the first audio frame length; After the noise reduction processing of the audio signal undergoes a frame length fine-tuning period, within the current frame length fine-tuning period, counting external interruption information occurring in the process of obtaining each second audio first sub-signal, and generating fine-tuning weight information based on the obtained external interruption information, wherein the fine-tuning weight information includes a plurality of fine-tuning weight items; Based on the fine-tuning weight item in the fine-tuning weight information and the pre-built fine-tuning model, a fine-tuning weight is calculated. When the fine-tuning weight is greater than the fine-tuning threshold, the frame length fine-tuning determination process is executed, and the determination state is that the frame length can be fine-tuned. When the frame length fine-tuning judgment processing is performed and the frame length fine-tuning condition is met, the first audio frame length of the second audio first sub-signal in the corresponding group of audio information to be noise reduced is fine-tuned.
2. The low-latency audio signal processing method according to claim 1, wherein: When sending the noise-reduced audio signal to the output, it includes: dividing the noise-reduced audio signal into a noise-reduced first sub-signal and a noise-reduced second sub-signal, wherein a frame length of the noise-reduced first sub-signal is consistent with a frame length of the second audio second sub-signal; When the reception of the second audio second sub-signal is completed, the sending and output of the noise-reduced first sub-signal is completed, and thereafter, the sending and output of the noise-reduced second sub-signal is completed.
3. The low-latency audio signal processing method according to claim 2, wherein: For any group of audio information to be reduced in noise, when the second audio second sub-signal is received, the second audio signal to be reduced in noise forms the first audio signal to be reduced in noise in the next group of audio information to be reduced in noise, wherein: When the acquisition of the second audio first sub-signal in the second audio signal to be de-noised in the next set of audio information to be de-noised is completed, the sending and output of the current de-noised second sub-signal is terminated.
4. The low-latency audio signal processing method according to claim 1, wherein: For any set of audio information to be de-noised, when the first audio signal to be de-noised is received, a noise reduction calculation execution interrupt is triggered to perform noise reduction calculation processing on the first audio signal to be de-noised; When the acquisition of the second audio first sub-signal is completed, the noise reduction sending interrupt is triggered, so that the noise-reduced audio signal is sent and output after entering the noise reduction sending interrupt.
5. The low-latency audio signal processing method according to claim 1, wherein: During the frame length fine-tuning period, the statistical external interrupt information includes the number of interrupt triggering and the interrupt processing time corresponding to each interrupt triggering, wherein the type of the external interrupt includes at least UART interrupt; The fine-tuning weight items in the fine-tuning weight information include an interruption distribution uniformity item and an interruption triggering intensity item; The frame length fine-tuning condition includes at least the average noise reduction calculation processing time within the cycle and the average interruption time within the cycle. When the sum of the average noise reduction calculation processing time within the cycle and the average interruption time within the cycle is greater than the noise reduction calculation processing time corresponding to the current group of audio information to be reduced, the frame length fine-tuning condition is met; otherwise, the frame length fine-tuning condition is not met.
6. The low-latency audio signal processing method according to claim 5, wherein: The fine-tuning threshold is 1. When the fine-tuning weight is greater than 1, the frame length fine-tuning judgment process is executed, and the judgment state is that the frame length can be fine-tuned. Otherwise, the frame length fine-tuning judgment process is executed, and the judgment state is that the frame length cannot be fine-tuned. When the frame length is updated, the first audio frame length of the second audio first sub-signal is increased by at least one audio frame length.
7. The low-delay audio signal processing method according to claim 5, wherein: For the constructed fine-tuning model, we have: in, To fine-tune the weights, is the distribution uniformity measurement sub-item within the interrupt distribution uniformity item, is the interruption trigger continuity factor within the interruption distribution uniformity term, is the average trigger strength within the interrupt trigger strength term, is the stability factor within the interrupt trigger strength term, is the weight coefficient of the distribution uniformity measurement sub-item, The weight coefficient of the interrupt trigger continuity factor.
8. A low-latency audio signal processing system, characterized in that: At least includes an audio signal processing device for actively reducing noise on an audio signal, wherein when actively reducing noise on an audio signal, the audio signal processing device adopts the audio signal processing method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Sound collection method, system and apparatus
CN107071647A
Coal mine dispatching room audio automatic understanding method based on deep learning
CN117437920A