Local sound reinforcement and remote interaction method, system and equipment based on DSP (Digital Signal Processor)

Abstract

The invention relates to the technical field of intelligent audio processing, and discloses a local sound amplification and remote interaction method, system and device based on a DSP processor, and the method comprises the steps: inputting a plurality of microphone devices, collecting microphone data, and converting the microphone data into digital signals; the DSP processor synchronously performs data cutting after acquiring the microphone data, and sequentially performs data processing related to local sound amplification and remote interaction, including: performing howling suppression on the acquired data, and performing echo cancellation by adopting a digital recovery form to generate pure audio; carrying out noise reduction processing on the pure audio, after noise reduction, carrying out automatic gain and human voice equalizer adjustment, and outputting high-quality audio data; and outputting the high-quality audio data to the local through an I2S interface, sending the high-quality audio data to a far end through a network, converting the high-quality audio data into analog audio, and playing the analog audio through a sound box. By means of the method, a system is achieved, the current redundant equipment stacking form is solved, and the corresponding processing effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent audio processing, and in particular to a local sound reinforcement and remote interaction method, system and equipment based on a DSP processor. Background Art

[0002] Generally speaking, building a set of converged equipment requires a conference host, a howling suppressor, an audio processor, a noise reducer, a mixer and remote interactive equipment. These devices form a large cabinet, which decomposes all the technical links required for local sound reinforcement and remote interaction into multiple devices. The devices are connected through audio cables according to the input and output of the device interface, which are mainly used to realize audio mixing, howling suppression, EQ adjustment, environmental noise reduction, far-field pickup and echo cancellation. All technologies correspond to a functional device. To realize the functions of local sound reinforcement and remote interaction, multiple devices need to be superimposed in the entire system. Each link will go through a round of encoding and decoding, resulting in serious distortion of the processed audio and human voice, and the superposition of multiple devices leads to a large delay in the overall audio link. Since it is the superposition processing of multiple devices, the interfaces between devices must be matched, and the system debugging is complicated and cumbersome. On the device side, the cascade of multiple devices will increase the delay of the audio link while adding devices and processing the audio streams of multiple devices. Continuous collection and conversion will cause multiple losses of the frequency components of the human voice, affecting the end user experience.

[0003] In the prior art, the solution is to streamline the types of equipment, such as not purchasing a noise reducer, but this will affect the final audio processing effect, resulting in the processed audio always containing a lot of noise, or purchasing multiple devices to build a system, including a howling suppressor, an audio processor, an amplifier, a mixer, etc., and each device is connected by multiple audio cables to build a set of complex equipment links. At the same time, most of them are based on traditional signal processing algorithms, so they have poor adaptability to complex links, and the debugging time is long. Whether it is using notch or frequency shift technology, the debugging personnel are required to test the frequency points at each location on site to achieve the howling suppression effect; the noise reduction equipment based on traditional signals filters out a single type of noise, and cannot filter out sudden noise in the environment. Although it is a system of multiple devices, the overall effect of the use experience is very poor. The most unacceptable thing about the cascade of multiple devices for local sound reinforcement is the continuous superposition of delays, resulting in a very poor on-site experience, and you can always hear your own echo. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art and to provide a local sound reinforcement and remote interaction method, system and device based on a DSP processor. Based on a high-performance DSP processor, the functions of sound reinforcement and sound pickup interaction in complex scenes can be realized, the current redundant equipment stacking form can be solved, the loss of human voice quality can be reduced, and complex engineering debugging is not required. In addition, based on the high performance and strong computing power of DSP, the corresponding howling suppression, noise suppression and echo cancellation processing effects can be improved.

[0005] On the one hand, a local sound reinforcement and remote interaction method based on a DSP processor is provided, comprising the following steps: S1: Input several microphone devices, collect microphone data and convert them into digital signals; S2: After acquiring the microphone data, the DSP processor synchronously cuts the data and sequentially performs data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; S3: The high-quality audio data is output to the local through the I2S interface, and sent to the remote end through the network, converted into analog audio and played through the speaker.

[0006] Furthermore, in step S1, a plurality of input devices are cascaded through the DSP processor, and then the input microphone data is collected by the collection unit in the DSP processor, and converted into digital signals and sent to the core of the DSP processor for calculation.

[0007] Further, in step S2, after acquiring the microphone data, the DSP processor synchronously cuts the data through a short frame shift algorithm, further comprising: For one of the microphone data sequences, set the frame length and frame shift of the cut short frames; Starting from the starting position of the original continuous microphone data sequence, the data range of the first short frame is determined according to the set frame length; After the first short frame is determined, the subsequent short frames are divided according to the set frame shift until the entire microphone data sequence is completely cut; When dividing to the end of the microphone data, choose to discard or pad the boundary.

[0008] Further, in step S2, the DSP processor includes an algorithm module required for local sound reinforcement and remote interaction, wherein the howling suppression of the collected data includes: The sound feedback signal is obtained by hardware collection, and the obtained feedback signal is sent to the collection unit to be converted into a digital signal At the same time, the speaker sound source signal is simulated and played through the speaker, and after playing, the audio signal transmitted in the environment where the speaker is located is obtained. ; The audio with howling suppression is obtained after the automatic frequency control AFC processing of the adaptive howling suppression algorithm , the principle is as follows: in, Indicates adaptive filtering operation, hardware circuit transmission Faster than space transmission Therefore, the howling point is located according to the echo signal, and the howling point is suppressed when the energy of the howling frequency point has not reached saturation.

[0009] Further, in step S2, the echo cancellation in the form of digital echo collection includes: The sound source played by the speaker includes the remote sound , Local Audio As well as external input background audio, the signal obtained by the microphone ,in, Related to the path, For local near-end vocals; Through digital capture, the remote sound It is directly sent to the DSP processor to process the collected data L2, and avoid the self-cancellation of the local near-end audio information on the basis of eliminating the far-end sound, which is expressed as: .

[0010] Furthermore, in step S2, a neural network self-learning model is introduced to perform noise reduction processing on the clean audio, including: Collect various unstable noise data in life, and collect batches of labeled human voice audio data, and pre-process the collected human voice audio data and noise data; The pre-processed human voice audio data and noise data are grouped in a certain proportion and input into the neural network self-learning model in batches. Each batch of data includes mixed audio of noise and human voice in different combinations. The model performs noise reduction on the input audio data based on the set parameters. Define the loss function Train the model and reduce the denoised audio output by the model by reducing the loss value Pure target vocals with annotations The differences that exist are as follows: in, The amount of audio data input for the batch; According to the calculated loss value, the back propagation algorithm is used to calculate the gradient of each parameter with respect to the loss layer by layer along the structure of the neural network starting from the output layer. Based on the gradient, the optimization algorithm is used to update and adjust the model parameters, which is expressed as follows: in, is the learning rate, Represents the loss function About weight The gradient of Input new audio data and use the trained and optimized model to perform noise reduction on it.

[0011] Preferably, in step S2, after noise reduction, automatic gain and vocal equalizer adjustment are performed, and outputting high-quality audio data further comprises: The audio data after noise reduction processing At a certain threshold The range of amplification or reduction ensures the stability of the human voice. Flexible adjustment according to different environments; According to the weight Smooth the human voice, adjust the curve of the human voice frequency, and adjust the frequency point in different environments and according to different human voice frequency characteristics; Output final audio data , which is expressed as follows: Among them, the weight .

[0012] Furthermore, step S3 further includes: The DSP processor outputs data through the I2S interface. After receiving the output audio data, the MCU further processes the local near-end audio, including audio encoding and protocol encapsulation, to make it meet the network transmission requirements. The processed local near-end audio is sent to the far end through network communication, thereby realizing the transmission of audio data between different areas. When the MCU is connected to an external audio source, its audio data is transmitted to the digital-to-analog converter through the I2S interface, which integrates the audio digital signals from different sources, converts them into analog signals, and then plays them through the speakers.

[0013] On the other hand, a local sound reinforcement and remote interaction system based on a DSP processor is provided, comprising: The audio acquisition module is used to input several microphone devices, collect microphone data and convert them into digital signals; The audio processing module is used for the DSP processor to synchronously cut the data after acquiring the microphone data, and sequentially perform data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; The audio output module is used to output the high-quality audio data to the local through the I2S interface, and send it to the remote end through the network, convert it into analog audio and play it through the speaker.

[0014] In addition, a local sound reinforcement and remote interaction device based on a DSP processor for processing the above method is also disclosed. The multi-core algorithm module built into the DSP processor includes the algorithm modules required for local sound reinforcement and remote interaction, which specifically include: An analog-to-digital conversion module, used to collect input microphone data and convert it into a digital signal; The digital-to-analog conversion module is used to receive digital audio signals from different sources, convert them into analog signals and play them through the speakers; Howling suppression module, used when the device is in different environments, adopts adaptive howling suppression algorithm, detects the acoustic feedback signal in real time through adaptive filter and performs howling suppression processing; The echo cancellation module is used for remote interaction. It eliminates the audio sound at the far end through digital echo collection, and after processing, retains the pure near-end audio and sends it to the far end. The noise reduction module is used to introduce a neural network self-learning model, add noise types and calibrate the target human voice, build a stable model through continuous self-learning, and perform noise reduction on the samples; Automatic gain module, used to automatically perform gain processing on pure audio based on howling suppression, echo cancellation and noise reduction to ensure the stability of human voice; The EQ adjustment module is used to adjust the human voice frequency curve after being processed by the above modules, and to adjust the frequency points according to different environments and different human voice frequency characteristics.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention is based on a high-performance DSP microprocessor, which integrates howling suppression, echo cancellation, noise reduction, automatic gain, and EQ adjustment to achieve the functions of local sound reinforcement and remote interaction. At the same time, it also uses the high computing power of DSP to greatly improve the overall sound reinforcement gain, echo cancellation effect, and noise reduction performance. The present invention adopts a single digital signal as an echo signal during the echo cancellation process, thereby avoiding the self-cancellation of local near-end audio information on the basis of eliminating the far-end sound, and solving the problem of far-end audio distortion. Aiming at the scene requirements of local sound reinforcement, the present invention proposes a short frame shift algorithm data cutting method to minimize the link delay and meet the low delay requirements of local sound reinforcement. In the device that integrates echo cancellation and local sound reinforcement, the audio link is more flexible because the device processes the audio inside. The echo cancellation process can be clear and clean in the form of digital acquisition, ensuring the fluency of remote interaction and solving the bottleneck of remote interaction in the current system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a local sound reinforcement and remote interaction method based on a DSP processor of the present invention; Figure 2 A schematic diagram of a howling suppression module of the present invention; Figure 3 A schematic diagram of an echo cancellation module of the present invention; Figure 4 The present invention is a flow chart of a local sound reinforcement and remote interaction method based on a DSP processor. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] The new device has no limit on the number of microphones (microphone ≥ 1), and collects microphone data through the DSP AD module, marked as ,Will The digital signal is sent to the DSP core for calculation. The built-in multi-core algorithm module includes all the algorithm modules required for local sound reinforcement and remote interaction, including howling suppression module (AFC), echo cancellation module (AEC), noise reduction module (linear processing + neural network NN), automatic gain control (AGC) and EQ adjustment processing module. The processed audio is marked as .

[0019] The specific implementation of the present invention is described below with reference to the accompanying drawings and embodiments.

[0020] Example 1 like Figure 1 As shown, a local sound reinforcement and remote interaction method based on a DSP processor is provided in this embodiment, and the technical solution includes the following steps: S1: Input several microphone devices, collect microphone data and convert them into digital signals; S2: After acquiring the microphone data, the DSP processor synchronously cuts the data and sequentially performs data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; S3: The high-quality audio data is output to the local through the I2S interface, and sent to the remote end through the network, converted into analog audio and played through the speaker.

[0021] Among them, in step S1, multiple input devices are cascaded through the DSP processor, and then the input microphone data is collected by the collection unit in the DSP processor, and converted into digital signals and sent to the core of the DSP processor for calculation. This step is based on the analog-to-digital conversion module AD in the DSP processor, which is used to collect the input microphone data and convert it into digital signals.

[0022] In step S2, after acquiring the microphone data, the DSP processor synchronously cuts the data through a short frame shift algorithm, further comprising: For one of the microphone data sequences, set the frame length and frame shift of the cut short frames; Starting from the starting position of the original continuous microphone data sequence, the data range of the first short frame is determined according to the set frame length; After the first short frame is determined, the subsequent short frames are divided according to the set frame shift until the entire microphone data sequence is completely cut; When dividing to the end of the microphone data, choose to discard or pad the boundary.

[0023] In step S2, the DSP processor includes algorithm modules required for local sound reinforcement and remote interaction, wherein the howling suppression of the collected data includes: The sound feedback signal is obtained by hardware collection, and the obtained feedback signal is sent to the collection unit to be converted into a digital signal At the same time, the speaker sound source signal is simulated and played through the speaker, and after playing, the audio signal transmitted in the environment where the speaker is located is obtained. ; The audio with howling suppression is obtained after the automatic frequency control AFC processing of the adaptive howling suppression algorithm , the principle is as follows: in, Indicates adaptive filtering operation, hardware circuit transmission Faster than space transmission Therefore, the howling point is located according to the echo signal, and the howling point is suppressed when the energy of the howling frequency point has not reached saturation.

[0024] This method corresponds to the howling suppression module in the DSP processor, and is used when the device is in different environments. It adopts an adaptive howling suppression algorithm, detects the acoustic feedback signal in real time through an adaptive filter, and performs howling suppression processing.

[0025] like Figure 2 As shown, the basic principle is cancellation. When the device is in different environments, the acoustic feedback signal is detected and processed in real time through an adaptive filter. The feedback signal is collected by hardware, which is marked as follows: , will collect the sound The analog signal is sent to the AD module built into the DSP, which converts the analog signal into a digital signal and sends it to the DSP. The speaker sound source signal is marked as , the audio transmitted in the environment through the speaker simulation is marked as , the audio after AFC processing by the adaptive howling suppression algorithm is marked as , according to the principle of howling suppression, , where Ref1 transmitted by hardware circuit is faster than that transmitted by space Therefore, in DSP, the AFC module locates the howling point according to the echo signal, and the suppression of the howling point is at the stage where the energy of the howling frequency has not reached saturation, so the suppression amount of the human voice frequency is small. The entire implementation process is real-time detection and real-time suppression, which will not damage the human voice too much, and there is no need for debugging personnel to test the howling frequency on site and suppress it, which improves the debugging time of the overall project.

[0026] In step S2 of this embodiment, the echo cancellation in the form of digital echo further includes: The sound source played by the speaker includes the remote sound , Local Audio And external input background audio ,Right now: , the signal picked up by the microphone ,in, Related to the path, For local near-end vocals; Through digital capture, the remote sound It is directly sent to the DSP processor to process the collected data L2, and avoid the self-cancellation of the local near-end audio information on the basis of eliminating the far-end sound, which is expressed as: .

[0027] This step method corresponds to the echo cancellation module in the DSP processor, which is used for remote interaction to eliminate the audio sound at the far end through digital echo collection, and retain the pure near-end audio after processing and send it to the far end.

[0028] like Figure 3 As shown in the figure, this module is the most critical part of the fusion system. The previous remote interaction did not include the local sound reinforcement function, and the speaker playback only included the remote sound. In the remote interactive device, the audio of the speaker end can be directly returned to the AD module of the DSP through hardware. However, this scenario also includes local sound reinforcement, and they are all independent devices, and it is impossible to provide other feedback signals. However, the sound source played by the speaker in the fusion scenario includes the remote sound. , local audio And external input background audio , the audio played by the speaker is marked as Mix, where If Mix is ​​directly provided to DSP as a reference signal, the local near-end human voice will be eliminated, and the AEC work process will be Figure 3 shown.

[0029] If you follow Figure 3 Path 1 is marked as L1 sampling mode. All the sound sources played by the speaker will be sent to the AEC module. At this time, the speaker plays the sound source: , , where n represents the hardware voltage divider parameter; the signal picked up by the microphone is: , after passing through the AEC module, the processed audio is marked as , , you can see the signal picked up by the microphone and reference signal The local audio signal is included in both. Although the far-end audio can be attenuated according to path 1, the local near-end audio is attenuated after processing because the microphone signal and the reference signal both contain the local near-end audio. As a result, the near-end human voice is missing or the human voice is intermittent when the audio is sent to the far-end, resulting in invalid audio. Path 2 is to use digital capture to collect the sound from the far-end. Directly sent to DSP, the retrieved data is marked as , directly retrieve the digital The audio is sent to the DSP for processing. It does not contain local near-end audio information and does not have self-cancellation. It ensures that the far-end audio is clean and has no residual echo. It is marked as: , Therefore, this embodiment adopts path 2 as the echo cancellation method.

[0030] In step S2, a neural network self-learning model is introduced to perform noise reduction processing on the clean audio, further comprising: Collect various unstable noise data in life, and collect batches of labeled human voice audio data, and pre-process the collected human voice audio data and noise data; The pre-processed human voice audio data and noise data are grouped in a certain proportion and input into the neural network self-learning model in batches. Each batch of data includes mixed audio of noise and human voice in different combinations. The model performs noise reduction on the input audio data based on the set parameters. Define the loss function Train the model and reduce the denoised audio output by the model by reducing the loss value Pure target vocals with annotations The differences that exist are as follows: in, The amount of audio data input for the batch; According to the calculated loss value, the back propagation algorithm is used to calculate the gradient of each parameter with respect to the loss layer by layer along the structure of the neural network starting from the output layer. Based on the gradient, the optimization algorithm is used to update and adjust the model parameters, which is expressed as follows: in, is the learning rate, Represents the loss function About weight The gradient of Input new audio data and use the trained and optimized model to perform noise reduction on it.

[0031] This step method corresponds to the noise reduction module of the DSP processor, which is used to introduce a neural network self-learning model, increase noise types and calibrate the target human voice, build a stable model through continuous self-learning, and perform noise reduction on the sample. In traditional equipment, noise reduction uses a linear processing method, which can filter out some stable noises such as environmental background noise and fans. However, in actual use, in addition to this stable noise, there are also some unstable noises, such as door closing, applause, keyboard sound, mouse button sound, etc. These noises exist in daily life. For these noises, we have introduced a neural network self-learning model through the collection and training of various noise data sets. By continuously increasing our noise types and calibrating our target human voice, we have gradually formed a stable model through continuous self-learning. At present, it can solve nearly 100 types of daily non-steady-state noises.

[0032] In step S2, after noise reduction, automatic gain and vocal equalizer adjustment are performed, and outputting high-quality audio data further includes: The audio data after noise reduction processing At a certain threshold The range of amplification or reduction ensures the stability of the human voice. Flexible adjustment according to different environments; According to the weight Smooth the human voice, adjust the curve of the human voice frequency, and adjust the frequency point in different environments and according to different human voice frequency characteristics; Output final audio data , which is expressed as follows: Among them, the weight .

[0033] This step method corresponds to the automatic gain module and EQ adjustment module of the DSP processor. In this embodiment, the automatic gain module is used to perform automatic gain processing on the pure audio based on howling suppression, echo cancellation and noise reduction processing. The human voice at this time is the last step before sound amplification and remote interaction. This step is to perform automatic gain processing on the clean noise after the previous processing. The purpose is to amplify or reduce the processed human voice audio within a certain threshold range to ensure the stability of the human voice. The threshold here is a settable value, which can be flexibly adjusted for different environments; and the EQ adjustment module is used for the last step of the DSP algorithm chain after being processed by the above modules. Different from the front EQ in the original system, our EQ is the last link of the chain. After processing by all algorithm modules, the human voice frequency can be adjusted according to the curve. In the face of different environments and for different human voice frequency characteristics, the frequency point can be adjusted. Here, it can support three-stage settings of low, medium and high.

[0034] In addition, step S3 further comprises: The DSP processor outputs data through the I2S interface. After receiving the output audio data, the MCU further processes the local near-end audio, including audio encoding and protocol encapsulation, to make it meet the network transmission requirements. The processed local near-end audio is sent to the far end through network communication, thereby realizing the transmission of audio data between different areas. When the MCU is connected to an external audio source, its audio data is transmitted to the digital-to-analog converter through the I2S interface, which integrates the audio digital signals from different sources, converts them into analog signals, and then plays them through the speakers.

[0035] This involves the digital-to-analog conversion module DAC in the DSP processor, which is used to receive audio digital signals from different sources, convert them into analog signals and play them through speakers.

[0036] In this embodiment, the MCU can be connected to other audio M1 and M1 can be played to the speaker through the DAC.

[0037] All of the above algorithm modules are implemented based on high-performance DSP. Because sound reinforcement has extremely high requirements for the delay of the overall link, the DSP must use a low frame shift data module to ensure that the running time of the algorithm module in each link meets the requirements. The bypass link time of the overall DSP is controlled within 1ms to meet the equipment link requirements.

[0038] The above is the working process of the entire fusion device. Figure 4 As shown in the figure, howling suppression, echo cancellation, noise reduction, automatic gain, and EQ adjustment are integrated together through DSP, and DSP has high-performance computing speed, which can realize low frame shift calculation. The built-in AD module realizes the collection of microphone data and has an I2S interface to realize the output of processed audio. At the same time, DSP has a uart / I2C interface to realize the interaction with the control function of the host computer, thereby realizing low latency and high-performance processing effect of the entire device.

[0039] On the other hand, this embodiment provides a local sound reinforcement and remote interaction system based on a DSP processor, including: The audio acquisition module is used to input several microphone devices, collect microphone data and convert them into digital signals; The audio processing module is used for the DSP processor to synchronously cut the data after acquiring the microphone data, and sequentially perform data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; The audio output module is used to output the high-quality audio data to the local through the I2S interface, and send it to the remote end through the network, convert it into analog audio and play it through the speaker.

[0040] The functions of the modules and units in the local sound reinforcement and remote interaction system based on the DSP processor correspond to the steps in the above-mentioned local sound reinforcement and remote interaction method embodiment based on the DSP processor, see Figures 1 to 4 Its functions and implementation process will not be described here one by one.

[0041] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0042] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A local sound reinforcement and remote interaction method based on a DSP processor, characterized in that: The steps include: S1: Input several microphone devices, collect microphone data and convert them into digital signals; S2: After acquiring the microphone data, the DSP processor synchronously cuts the data and sequentially performs data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; S3: The high-quality audio data is output to the local through the I2S interface, and sent to the remote end through the network, converted into analog audio and played through the speaker.

2. The local sound reinforcement and remote interaction method based on DSP processor according to claim 1 is characterized in that: In step S1, multiple input devices are cascaded through the DSP processor, and then the input microphone data is collected by the collection unit in the DSP processor, and converted into digital signals and sent to the core of the DSP processor for calculation.

3. The local sound reinforcement and remote interaction method based on DSP processor according to claim 1 is characterized in that: In step S2, after acquiring the microphone data, the DSP processor synchronously cuts the data through a short frame shift algorithm, further comprising: For one of the microphone data sequences, set the frame length and frame shift of the cut short frames; Starting from the starting position of the original continuous microphone data sequence, the data range of the first short frame is determined according to the set frame length; After the first short frame is determined, the subsequent short frames are divided according to the set frame shift until the entire microphone data sequence is completely cut; When dividing to the end of the microphone data, choose to discard or pad the boundary.

4. The local sound reinforcement and remote interaction method based on DSP processor according to claim 2 is characterized in that: In step S2, the DSP processor includes algorithm modules required for local sound reinforcement and remote interaction, wherein the howling suppression of the collected data further includes: The sound feedback signal is obtained by hardware collection, and the obtained feedback signal is sent to the collection unit to be converted into a digital signal At the same time, the speaker sound source signal is simulated and played through the speaker, and after playing, the audio signal transmitted in the environment where the speaker is located is obtained. ; The audio with howling suppression is obtained after the automatic frequency control AFC processing of the adaptive howling suppression algorithm , the principle is as follows: in, Indicates adaptive filtering operation, hardware circuit transmission Faster than space transmission Therefore, the howling point is located according to the echo signal, and the howling point is suppressed when the energy of the howling frequency point has not reached saturation.

5. The local sound reinforcement and remote interaction method based on DSP processor according to claim 1 is characterized in that: In step S2, the echo cancellation in the form of digital echo further includes: The sound source played by the speaker includes the remote sound , Local Audio As well as external input background audio, the signal obtained by the microphone ,in, Related to the path, For local near-end vocals; Through digital capture, the remote sound It is directly sent to the DSP processor to process the collected data L2, and avoid the self-cancellation of the local near-end audio information on the basis of eliminating the far-end sound, which is expressed as: .

6. The local sound reinforcement and remote interaction method based on DSP processor according to claim 1 is characterized in that: In step S2, a neural network self-learning model is introduced to perform noise reduction processing on the clean audio, further comprising: Collect various unstable noise data in life, and collect batches of labeled human voice audio data, and pre-process the collected human voice audio data and noise data; The pre-processed human voice audio data and noise data are grouped in a certain proportion and input into the neural network self-learning model in batches. Each batch of data includes mixed audio of noise and human voice in different combinations. The model performs noise reduction on the input audio data based on the set parameters. Defining the loss function Train the model and reduce the denoised audio output by the model by reducing the loss value Pure target vocals with annotations The differences that exist are as follows: in, The amount of audio data input for the batch; According to the calculated loss value, the back propagation algorithm is used to calculate the gradient of each parameter with respect to the loss layer by layer along the structure of the neural network starting from the output layer. Based on the gradient, the optimization algorithm is used to update and adjust the model parameters, which is expressed as follows: in, is the learning rate, Represents the loss function About weight The gradient of Input new audio data and use the trained and optimized model to perform noise reduction on it.

7. The local sound reinforcement and remote interaction method based on a DSP processor according to any one of claims 4 to 6, characterized in that: In step S2, after noise reduction, automatic gain and vocal equalizer adjustment are performed, and outputting high-quality audio data further includes: The audio data after noise reduction processing At a certain threshold The range of amplification or reduction ensures the stability of the human voice. Flexible adjustment according to different environments; According to the weight Smooth the human voice, adjust the curve of the human voice frequency, and adjust the frequency point in different environments and according to different human voice frequency characteristics; Output final audio data , which is expressed as follows: Among them, the weight .

8. The local sound reinforcement and remote interaction method based on DSP processor according to any one of claim 1, characterized in that: Step S3 further comprises: The DSP processor outputs data through the I2S interface. After receiving the output audio data, the MCU further processes the local near-end audio, including audio encoding and protocol encapsulation, to make it meet the network transmission requirements. The processed local near-end audio is sent to the far end through network communication, thereby realizing the transmission of audio data between different areas. When the MCU is connected to an external audio source, its audio data is transmitted to the digital-to-analog converter through the I2S interface, which integrates the audio digital signals from different sources, converts them into analog signals, and then plays them through the speakers.

9. A local sound reinforcement and remote interaction system based on a DSP processor, characterized in that: include: The audio acquisition module is used to input several microphone devices, collect microphone data and convert them into digital signals; The audio processing module is used for the DSP processor to synchronously cut the data after acquiring the microphone data, and sequentially perform data processing related to local sound reinforcement and remote interaction, including: The collected data is subjected to howling suppression and echo cancellation in the form of digital acquisition to generate pure audio; The pure audio is subjected to noise reduction processing, and after the noise reduction, it is adjusted by automatic gain and vocal equalizer to output high-quality audio data; The audio output module is used to output the high-quality audio data to the local through the I2S interface, and send it to the remote end through the network, convert it into analog audio and play it through the speaker.

10. A local sound reinforcement and remote interaction device based on a DSP processor for processing the method described in claims 1 to 7, characterized in that: The multi-core algorithm modules built into the DSP processor include the algorithm modules required for local sound reinforcement and remote interaction, including: An analog-to-digital conversion module, used to collect input microphone data and convert it into a digital signal; The digital-to-analog conversion module is used to receive digital audio signals from different sources, convert them into analog signals and play them through the speakers; Howling suppression module, used when the device is in different environments, adopts adaptive howling suppression algorithm, detects the acoustic feedback signal in real time through adaptive filter and performs howling suppression processing; The echo cancellation module is used for remote interaction. It eliminates the audio sound at the far end through digital echo collection, and after processing, retains the pure near-end audio and sends it to the far end. The noise reduction module is used to introduce a neural network self-learning model, add noise types and calibrate the target human voice, build a stable model through continuous self-learning, and perform noise reduction on the samples; Automatic gain module, used to automatically process the pure audio based on howling suppression, echo cancellation and noise reduction to ensure the stability of human voice; The EQ adjustment module is used to adjust the human voice frequency curve after being processed by the above modules, and to adjust the frequency points according to different environments and different human voice frequency characteristics.

Images