Audio processing method, system and device, storage medium and program product

Abstract

The invention discloses an audio processing method, system and device, a storage medium and a program product, and relates to the technical field of audio processing, the audio processing method is applied to the audio processing system, and the audio processing system comprises an audio topology design platform and an audio processing module. The audio processing method comprises the steps that an audio topology design platform responds to modification operation for an algorithm component of preset audio topology engineering in a display interface to obtain a modified algorithm component, the modification operation comprises modification of the type of the algorithm component and / or modification of parameters of the algorithm component, and the modification operation comprises modification of the type of the algorithm component and / or modification of the parameters of the algorithm component. An audio processing module processes an audio signal based on the modified algorithm component. Therefore, the audio processing efficiency can be improved, the audio is processed by adding the AI algorithm component, and the sound effect processing effect is improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio processing method, system, device, storage medium and program product. Background Art

[0002] Audio processing technology is mainly used in conferences, command centers, stadiums and other scenarios. However, current audio processors can only use fixed audio topology engineering. Fixed audio topology engineering cannot be applied to audio processing in different scenarios, reducing audio processing efficiency. Summary of the invention

[0003] The main purpose of this application is to provide an audio processing method, system, device, storage medium and program product, aiming to improve audio processing efficiency and sound effect processing effect.

[0004] To achieve the above object, the present application proposes an audio processing method, which is applied to an audio processing system. The audio processing system includes an audio topology design platform and an audio processing module. The method includes: The audio topology design platform obtains a modified algorithm component in response to a modification operation on an algorithm component of a preset audio topology project in a display interface, wherein the modification operation includes modifying a type of the algorithm component and / or modifying a parameter of the algorithm component; The audio processing module processes the audio signal based on the modified algorithm component.

[0005] In one embodiment, the step of processing the audio signal by the audio processing module based on the modified algorithm component comprises: The audio processing module loads the modified audio topology project and processes the audio signal based on the loaded first audio topology project, wherein the modified audio topology project is obtained by the audio topology design platform in response to a modification operation on the type of the algorithm component, and is sent to the audio processing module in response to a confirmation upload operation on the modified audio topology project; And / or, the audio processing module modifies the parameters of the algorithm component of the loaded second audio topology project based on the parameter modification instruction to obtain a third audio topology project, and processes the audio signal based on the third audio topology project, wherein the second audio topology project includes a preset audio topology project or a first audio topology project, and the parameter modification instruction is sent to the audio processing module by the audio topology design platform in response to the parameter modification operation on the algorithm component of the second audio topology project.

[0006] In one embodiment, the audio processing module is provided with a DSP processor and an ARM processor, and the step of loading the modified audio topology engineering into the audio processing module includes: The ARM processor obtains the algorithm type identifier of the algorithm component in the modified audio topology project; If the algorithm type identifier includes an AI algorithm, the ARM processor splits the modified audio topology project into a first audio topology sub-project corresponding to the AI ​​algorithm and a second audio topology sub-project corresponding to the traditional algorithm, and sends the second audio topology sub-project to the DSP processor; The ARM processor loads the first audio topology sub-project, and the DSP processor synchronously loads the second audio topology sub-project.

[0007] In one embodiment, the algorithm components of the modified audio topology project exist in a preset order, the loaded first audio topology project includes the loaded first audio topology sub-project and the loaded second audio topology sub-project, and the step of processing the audio signal based on the loaded first audio topology project includes: The DSP processor obtains a first audio signal, and determines whether the first audio signal needs to be processed locally based on a preset order of algorithm components and an algorithm type identifier in the first audio topology project, wherein the first audio signal includes a second audio signal or a third audio signal processed by the ARM processor, and the second audio signal is an audio signal obtained by processing based on the loaded second audio topology sub-project when local processing is required, or is an initial audio signal that has not been processed by the algorithm component; If it is necessary to process the first audio signal locally on the DSP processor, the DSP processor processes the first audio signal based on the loaded second audio topology sub-project, and returns the step of determining whether the first audio signal needs to be processed locally based on the preset order and algorithm type identifier of the algorithm components in the first audio topology project, until it is no longer necessary to process the first audio signal locally; If the DSP processor does not need to process locally, judging whether the ARM processor needs to process based on the preset order of the algorithm components and the algorithm type identifier in the first audio topology project, and if the ARM processor needs to process, sending the second audio signal to the ARM processor, otherwise, outputting the first audio signal; When receiving the second audio signal, the ARM processor processes the second audio signal based on the loaded first audio topology sub-project to obtain a third audio signal, and determines whether the third audio signal needs to be processed locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology sub-project; If it is necessary to process the third audio signal locally on the ARM processor, the ARM processor processes the third audio signal based on the loaded first audio topology sub-project, and returns to the step of determining whether it is necessary to process the third audio signal locally based on the preset order and algorithm type identifier of the algorithm components in the first audio topology project, until it is no longer necessary to process the third audio signal locally, at which point the ARM processor sends the third audio signal to the DSP processor.

[0008] In one embodiment, the step of the audio processing module modifying the parameters of the algorithm component of the loaded second audio topology engineering based on the parameter modification instruction includes: If the algorithm type identifier is an AI algorithm identifier, the ARM processor modifies the parameters of the algorithm component according to the algorithm component parameter modification instruction; If the algorithm type identifier is not an AI algorithm identifier, the ARM processor sends the algorithm component parameter modification instruction to the DSP processor, and the DSP processor modifies the parameters of the algorithm component according to the algorithm component parameter modification instruction.

[0009] In one embodiment, the audio processing module further comprises, after the step of processing the audio signal based on the modified algorithm component: The audio topology design platform sends a monitoring instruction to the ARM processor in response to a monitoring operation on the audio signal, wherein the monitoring instruction includes a channel number to be monitored and a target component to be monitored; If the target component is a traditional algorithm component, the monitoring instruction is sent to the DSP processor, the DSP processor sends the fourth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the traditional algorithm component to the ARM processor, and the ARM processor sends the fourth audio signal to the audio topology design platform; If the target component is an AI algorithm component, the ARM processor sends the fifth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the AI ​​algorithm component to the audio topology design platform; The audio topology design platform plays the audio corresponding to the fourth audio signal or the fifth audio signal to complete monitoring.

[0010] In addition, to achieve the above objectives, the present application also proposes an audio processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio processing method described above.

[0011] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio processing method described above are implemented.

[0012] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the audio processing method described above are implemented.

[0013] One or more technical solutions proposed in this application have at least the following technical effects: The audio processing system of the present application includes an audio topology design platform and an audio processing module. The user can modify the type and / or parameters of the algorithm components of the audio topology engineering preset in the audio topology design platform based on different application scenarios. The audio topology design platform responds to the modification operation of the algorithm components of the audio topology engineering to obtain a modified algorithm component suitable for the current application scenario. The audio processing module can process the audio signal based on the modified algorithm component. Therefore, the present application can improve the audio processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0016] Figure 1 A flowchart of the first embodiment of the audio processing method of the present application is provided; Figure 2 A schematic diagram of the audio topology design platform for this application; Figure 3 A flowchart of the third embodiment of the audio processing method of the present application is provided; Figure 4 A brief schematic diagram of the audio signal processing process of this application; Figure 5 A flowchart of the fourth embodiment of the audio processing method of the present application is provided; Figure 6 This is a schematic diagram of the module structure of the audio processing system according to an embodiment of the present application; Figure 7Schematic diagram of the device structure of the hardware operating environment involved in the audio processing method in the embodiment of the present application.

[0017] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0019] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0020] Based on this, the present application embodiment provides an audio processing method, referring to Figure 1 In this embodiment, the audio processing method includes steps S10 to S20: Step S10, the audio topology design platform responds to the modification operation of the algorithm component of the preset audio topology project in the display interface to obtain a modified algorithm component, wherein the modification operation includes modifying the type of the algorithm component and / or modifying the parameters of the algorithm component; It should be noted that the execution subject of this embodiment is an audio processing system, and the audio processing system includes an audio topology design platform.

[0021] The audio topology design platform is implemented based on an open management software for designing and modifying audio topology projects, and is presented to users in the form of a graphical interface. The interface of the audio topology design platform displays the structure of the audio topology project designed by the user. The structure of the audio topology project is composed of multiple components and signal transmission channels. The multiple components can be audio signal input components (refer to Figure 2 Audio Process MIC / Line in components), audio signal algorithm components (refer to Figure 2 AI (Artificial Intelligence) denoising, AI dereverberation, echo canceller, limited feedback suppressor, parametric equalizer, level meter and other components), audio signal output components (refer to Figure 2 Audio Process Line out component), other components (control components), etc.

[0022] In addition, it should be noted that the audio topology engineering is used to process the input audio signal through the above-mentioned multiple components and output it, referring to Figure 2 .

[0023] The input audio signal is collected by an audio collection device such as a microphone, and the collected audio is processed by an analog-to-digital converter inside the audio collection device. The function of the analog-to-digital converter is to convert the analog signal (audio) into a digital signal (audio signal).

[0024] The audio signal input component is a component used to receive the audio signal to be processed during the audio signal processing process, and the audio signal output component is a component used to output the processed audio signal during the audio signal processing process. The channel number is used to identify each independent audio signal channel in a multi-channel audio system. Figure 2 The 8 input channels and 6 output channels of the matrix mixer on the left are numbered by channel numbers. The audio signal algorithm component is a software or hardware unit that processes audio signals, including one or more algorithms such as echo canceller, parametric equalizer, AI denoising, and AI dereverberation.

[0025] Among them, the types of algorithm components include AI algorithm components (used for audio signal processing based on artificial intelligence algorithms) and traditional algorithm components. AI algorithm components are algorithm components obtained after artificial intelligence is trained based on a large amount of data, such as the above-mentioned AI denoising and AI dereverberation. Traditional algorithm components are algorithm components based on signal processing theory and have been used in the field of audio processing for a long time, such as the above-mentioned echo canceller and volume equalizer.

[0026] It is understandable that the user can modify the audio topology project through the audio topology design platform. The specific operation is that the user can add or change components by operating the interface of the audio topology design platform, and the audio topology design platform will modify the components in the interface in response to the user's modification operation. The user can also double-click the algorithm component in the audio topology project, and the audio topology design platform will display the parameter setting interface in response to the user's operation. The user can set the parameters of the algorithm component in the parameter setting interface, and the audio topology design platform will modify the parameters of the algorithm component in the audio topology design platform interface in response to the user's setting operation.

[0027] Specifically, the motivation for modifying the above audio topology project may be that, assuming that an audio topology project is designed for a conference room scenario, when the scenario changes to a gymnasium, the user needs to modify the audio topology project. The user removes the volume equalizer component in the audio topology project through the audio topology design platform, and adds the AI ​​denoising algorithm component and the echo canceller algorithm component. In response to the user's modification operation on the algorithm component of the platform interface, the audio topology design platform displays the algorithm component of the audio topology project in the interface as the algorithm component after the user's modification.

[0028] In order to improve the sound processing effect in the stadium scene, the user can modify the parameters of the algorithm component. The user double-clicks the algorithm component in the audio topology, and the audio topology design platform displays the setting interface of the algorithm component parameters in response to the user's double-click operation. The user enters the parameters he wants to set, and the audio topology design platform changes the parameters of the corresponding algorithm component to the parameters set by the user in response to the user's setting completion operation.

[0029] Step S20: the audio processing module processes the audio signal based on the modified algorithm component.

[0030] It should be noted that the audio processing system also includes an audio processing module.

[0031] The audio processing module used in this embodiment has one or more core processors. The core processor is used to receive the modified audio topology project sent by the audio topology design platform, receive the audio signal, process the audio signal based on the received modified audio topology project, and output the audio signal.

[0032] It is understandable that the user can upload the modified audio topology project displayed in the current interface of the audio topology design platform by triggering the upload operation, and the audio topology design platform uploads the modified audio topology project displayed in the current interface of the audio topology design platform to the audio processing module in response to the user's upload operation. The audio processing module can obtain the audio signal converted by the audio collection device such as a microphone in real time, or obtain the existing audio signal through a computer and other devices, and process the audio signal based on the algorithm component in the modified audio topology project uploaded by the user.

[0033] Specifically, the user uploads the modified audio topology project through the audio topology design platform. The audio topology project is modified by the user on the audio topology design platform based on the gymnasium scene, and has an audio signal input module, an audio signal output module, a matrix mixer, an AI denoising algorithm component, an AI dereverberation algorithm component, and an echo canceller algorithm component.

[0034] The audio signal processed by the audio processing module can be obtained by the user inputting the existing audio signal of the gymnasium host into the audio processing module for processing through a computer or other equipment. After the audio processing module receives the audio signal, the matrix mixer, AI denoising algorithm component, AI dereverberation algorithm component, echo canceller and other components in the modified audio topology project uploaded on the audio topology design platform process the audio signal.

[0035] In summary, the audio processing system of this embodiment includes an audio topology design platform and an audio processing module. The user can modify the type of algorithm components and / or parameters of the algorithm components of the audio topology engineering preset in the audio topology design platform based on different application scenarios. The audio topology design platform responds to the modification operation of the algorithm components of the audio topology engineering to obtain a modified algorithm component suitable for the current application scenario. The audio processing module can process the audio signal based on the modified algorithm component. Therefore, this embodiment can improve the audio processing efficiency, and by adding AI algorithm components to process audio, the sound processing effect is improved.

[0036] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. The specific implementation of processing the audio signal based on the modified algorithm component includes at least one of the following: First, the audio processing module loads the modified audio topology project and processes the audio signal based on the loaded first audio topology project, wherein the modified audio topology project is obtained by the audio topology design platform in response to a modification operation on the type of the algorithm component, and is sent to the audio processing module in response to a confirmation upload operation on the modified audio topology project; Second, the audio processing module modifies the parameters of the algorithm component of the loaded second audio topology project based on the parameter modification instruction to obtain a third audio topology project, and processes the audio signal based on the third audio topology project, wherein the second audio topology project includes a preset audio topology project or a first audio topology project, and the parameter modification instruction is sent to the audio processing module by the audio topology design platform in response to the parameter modification operation of the algorithm component of the second audio topology project.

[0037] It should be noted that the modified audio topology project is loaded by the core processor of the audio processing module. The first audio topology project is the loaded modified audio topology project, and the first audio topology project is in the core processor. The modification operation of the type of the algorithm component includes adding a new algorithm component and reducing an existing algorithm component.

[0038] It is understandable that the core processor of the audio processing module responds to the upload operation of the audio topology design platform, receives the modified audio topology project and parses it, and the parts that need to be parsed include the type and quantity of components of the modified audio processing module, the parameters of the components, the channel numbers of the components, the order and connection methods of the components. The core processor completes the parsing operation of the modified audio topology project and loads the modified audio topology project. After loading, the audio topology project in the core processor is the first audio topology project, and the core processor processes the audio signal based on the first audio topology project.

[0039] Specifically, the core processor of the audio processing module receives the modified audio topology project uploaded by the audio topology design platform, and the core processor parses the modified audio topology project to obtain an audio signal input component, two matrix mixer components, an AI denoising component, an AI dereverberation component, an echo canceller component, and an audio signal output component in the modified audio topology project, and obtains information such as parameters of the components in the modified audio topology project, channel numbers, and connection methods between components. The core processor loads the modified audio topology project based on the above-obtained information, obtains a first audio topology project, and processes the audio signal based on the first audio topology project.

[0040] It should also be noted that the parameters of the algorithm component are internal parameters of the algorithm component, and the processing effect of the algorithm component on the audio signal can be affected by changing the parameters of the algorithm component. The preset audio topology project is an audio topology project that has not been modified by the user in the audio topology design platform interface. The parameter modification instruction can modify the parameters of the algorithm component in the unmodified audio topology project, and can also modify the parameters of the algorithm component in the loaded modified audio topology project.

[0041] It is understandable that the audio topology design platform generates a corresponding parameter modification instruction in response to the user's parameter setting completion operation and sends it to the audio processing module. The core processor of the audio processing module responds to the parameter modification instruction sending operation of the audio topology design platform, receives the parameter modification instruction and modifies the parameters of the corresponding algorithm component based on the parameter modification instruction, and the algorithm component is the algorithm component of the audio topology engineering loaded in the core processor. The core processor processes the audio signal based on the modified algorithm component of the audio topology engineering.

[0042] Specifically, the user modifies the parameters of the AI ​​denoising algorithm component, and the audio topology design platform generates a corresponding parameter modification instruction of the AI ​​denoising algorithm component in response to the user's setting of the parameters of the AI ​​denoising algorithm component and sends it to the audio processing module. The core processor of the audio processing module receives the parameter modification instruction and modifies the parameters of the AI ​​denoising algorithm component based on the parameter modification instruction. The core processor processes the audio signal based on the modified AI denoising algorithm component and other components.

[0043] In a feasible implementation, a DSP (Digital Signal Processor) processor and an ARM (Advanced RISC Machines) processor are provided in the audio processing module, and a specific implementation of the audio processing module loading the modified audio topology engineering may be: The ARM processor obtains the algorithm type identifier of the algorithm component in the modified audio topology project; if the algorithm type identifier includes an AI algorithm, the ARM processor splits the modified audio topology project into a first audio topology sub-project corresponding to the AI ​​algorithm and a second audio topology sub-project corresponding to the traditional algorithm, and sends the second audio topology sub-project to the DSP processor; The ARM processor loads the first audio topology sub-project, and the DSP processor synchronously loads the second audio topology sub-project.

[0044] It should be noted that the ARM processor is a microprocessor based on a reduced instruction set computing architecture that can operate using a smaller number of basic instructions, thereby improving processing efficiency and speed. The DSP processor is a digital signal processor, which is a microprocessor specially designed to efficiently perform digital signal processing tasks, and is used to process audio signals in this embodiment.

[0045] The algorithm type identification of the algorithm component includes the AI ​​algorithm identification and the non-AI algorithm identification. The algorithm component whose algorithm type identification is the AI ​​algorithm is the AI ​​algorithm component, and the algorithm component whose algorithm type identification is not the AI ​​algorithm is the traditional algorithm component. The algorithm type identification is part of the algorithm component attribute and is used to determine the algorithm type of the algorithm component.

[0046] It should also be noted that the first audio topology sub-project and the second audio topology sub-project are split from the modified audio topology sub-project, and the algorithm components in the first audio sub-project are different from the algorithm components in the second audio sub-project.

[0047] The first audio topology sub-project is an audio topology project loaded on the ARM processor and consists of an AI algorithm component and an input / output interface, and the input / output interface is used to input and output audio signals. The second audio topology project is an audio topology project loaded on the DSP processor and consists of an audio signal input component, an audio signal output component, a traditional algorithm component and other components. In addition, there is an audio signal reading interface between the ARM processor and the DSP processor, which is used to input and output audio signals between the ARM processor and the DSP processor.

[0048] It can be understood that the specific splitting process of the ARM processor splitting the modified audio topology project into two audio topology sub-projects is that the ARM processor creates a first audio topology sub-project and a second audio topology sub-project, reads the modified audio topology project and identifies the algorithm component types of the algorithm components in the audio topology project one by one.

[0049] If the algorithm component type of the algorithm component is an AI algorithm component, the ARM processor splits the algorithm component into a first audio topology sub-project. If the algorithm component type is not an AI algorithm component, the ARM processor splits the algorithm component into a second audio topology sub-project and sends the second audio topology sub-project to the DSP processor.

[0050] Specifically, the modified audio topology project has an audio input component, a matrix mixer component, an AI denoising component, an AI dereverberation component, an echo canceller component, and an audio output component. The ARM processor creates a first audio topology sub-project and a second audio topology sub-project and identifies the algorithm type of the modified algorithm components one by one.

[0051] The ARM processor splits the AI ​​denoising component and the AI ​​dereverberation component into the first audio topology sub-project, and splits the audio signal input component, the matrix mixer component, the echo canceller component, and the audio signal output component into the second audio topology sub-project. After the split is completed, the ARM processor sends the second audio topology sub-project to the DSP processor.

[0052] It should be noted that the loading operation conditions for the first audio topology sub-project and the second audio topology sub-project are that the modified audio topology project has been completely split and the ARM processor has sent the second audio topology sub-project to the DSP processor. The loading operation of the first audio topology sub-project and the second audio topology sub-project is performed synchronously by the ARM processor and the DSP processor.

[0053] In a feasible embodiment, the specific implementation of the audio processing module modifying the algorithm component parameters based on the algorithm component parameter modification instruction may be: If the algorithm type identifier is an AI algorithm identifier, the ARM processor modifies the parameters of the algorithm component according to the algorithm component parameter modification instruction; if the algorithm type identifier is not an AI algorithm identifier, the ARM processor sends the algorithm component parameter modification instruction to the DSP processor, and the audio processing module modifies the parameters of the algorithm component of the loaded second audio topology project based on the parameter modification instruction.

[0054] It is understandable that the ARM processor in the audio processing system receives the algorithm component parameter modification instruction, and by parsing the algorithm component parameter modification instruction, the ARM processor can confirm the algorithm component that needs to be modified in parameters. By reading the algorithm type identifier of the algorithm component that needs to be modified in parameters, the ARM processor can confirm whether the algorithm component that needs to be modified in parameters is an AI algorithm component or a traditional algorithm component. If the algorithm component that needs to be modified in parameters is an AI algorithm component, then the algorithm component that needs to be modified in parameters is in the ARM processor, and the ARM processor modifies the parameters of the algorithm component that needs to be modified according to the algorithm component parameter modification instruction.

[0055] Specifically, the user modifies the parameters of the AI ​​denoising algorithm component, and the generated algorithm component parameter modification instruction is sent to the ARM processor of the audio processing module through the audio topology design platform. The ARM processor confirms that the algorithm component to be modified is the AI ​​denoising algorithm component based on the received algorithm component parameter modification instruction, and determines that the AI ​​denoising algorithm component is the algorithm component loaded on the ARM processor through the algorithm type identifier of the AI ​​denoising algorithm component. The ARM processor modifies the parameters of the AI ​​denoising algorithm component based on the received algorithm component parameter modification instruction.

[0056] It is understandable that if the type of the algorithm component that needs to be modified parameters confirmed by the ARM processor is not an AI algorithm, the algorithm component that needs to be modified parameters is in the DSP processor, and the ARM processor cannot modify the parameters of the algorithm component that needs to be modified parameters. The ARM processor sends the parameter modification instruction of the algorithm component to the DSP processor, and the DSP processor receives and modifies the algorithm component parameters on the DSP processor based on the parameter modification instruction of the algorithm component.

[0057] Specifically, the user modifies the parameters of the echo canceller algorithm component, and the generated algorithm component parameter modification instruction is sent to the ARM processor of the audio processing module through the audio topology design platform. The ARM processor confirms that the algorithm component to be modified is the echo canceller algorithm component based on the received algorithm component parameter modification instruction, and determines that the echo canceller algorithm component is the algorithm component loaded on the DSP processor through the algorithm type identifier of the echo canceller algorithm component. The ARM processor sends the received algorithm component parameter modification instruction to the DSP processor. The DSP processor receives and modifies the parameters of the echo canceller algorithm component based on the algorithm component parameter modification instruction.

[0058] In summary, the audio processing module of this embodiment determines the processor location of the algorithm component that needs to be loaded and / or modified based on the user's modification of the algorithm component and / or the algorithm component parameter operation, and loads and / or modifies the algorithm component on the corresponding processor, and the user does not need to determine which core processor the algorithm component is located in. Therefore, this embodiment can improve the convenience of users in modifying the audio topology.

[0059] Based on the first and second embodiments of the present application, in the third embodiment of the present application, refer to Figure 3 , the same or similar contents as those in the above-mentioned embodiment 1 and embodiment 2 can be referred to the above introduction, and will not be described in detail later. In the third embodiment of the present application, the algorithm components of the modified audio topology engineering have a preset order, and the processing of the audio signal based on the loaded first audio topology engineering includes steps S30~S34: Step S30, the DSP processor acquires a first audio signal, and determines whether the first audio signal needs to be processed locally based on a preset order of algorithm components and an algorithm type identifier in the first audio topology project, wherein the first audio signal includes a second audio signal or a third audio signal processed by the ARM processor, and the second audio signal is an audio signal obtained by processing based on the loaded second audio topology sub-project when the audio signal needs to be processed locally by the DSP processor, or is an initial audio signal that has not been processed by the algorithm component; It should be noted that operations such as obtaining audio signals and determining the audio signal processing position are performed by the DSP processing thread. The DSP processing thread is an audio signal stream processing thread running on the DSP processor, and is used to execute the task of processing continuous audio signals on the DSP processor.

[0060] It should also be noted that there is a preset order for the algorithm components of the modified audio topology project; specifically, when the user changes the type of the algorithm components of the audio topology project, the order between the algorithm components will change. Therefore, after uploading the designed audio topology project to the audio processing module in the audio topology design platform, the audio processing module receives the audio topology project and the ARM processor parses it to obtain the preset order between the algorithm components.

[0061] It is understandable that the DSP processor may obtain an audio signal that has not been processed by any algorithm component (i.e., the initial audio signal), or an audio signal that was last processed by the DSP processor, or an audio signal that was last processed by the ARM processor (i.e., the third audio signal). After obtaining one of the above audio signals, the DSP processor determines the next algorithm component to be used based on the preset order of the algorithm components, and determines whether the algorithm component type is an AI algorithm component based on the algorithm type identifier of the algorithm component to be used. If the algorithm type is an AI algorithm component, it does not need to be processed locally on the DSP processor. If the algorithm component type is not an AI algorithm component, it needs to be processed locally on the DSP processor.

[0062] Specifically, the algorithm component sequence of the first audio topology project is the AI ​​denoising algorithm component. The DSP processor obtains the initial audio signal, and determines that the next algorithm component to be used is the AI ​​denoising algorithm component according to the preset sequence of the algorithm components, and the algorithm type identifier of the AI ​​denoising algorithm component is the AI ​​algorithm component. The DSP processor determines that the AI ​​denoising algorithm component does not need to be processed locally on the DSP processor.

[0063] Step S31, if it is necessary to process the first audio signal locally on the DSP processor, the DSP processor processes the first audio signal based on the loaded second audio topology sub-project, and returns to the step of determining whether the first audio signal needs to be processed locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology project, until it is no longer necessary to process the first audio signal locally; It is understandable that when the DSP processor determines that the audio signal needs to be processed locally on the DSP processor, the DSP processor processes the audio signal based on the loaded second audio topology sub-project. After the audio signal is processed by the current algorithm component, the DSP processor obtains the processed audio signal, and confirms the next algorithm component to be used according to the preset order of the algorithm components of the first audio topology project, and determines whether the audio signal needs to be processed locally on the DSP processor. If the next algorithm component to be used is still loaded on the DSP processor, the DSP processor uses the next algorithm component to process the audio signal. The DSP processor repeats the steps of determining the algorithm component currently needed to be used, identifying the type of the algorithm component, determining whether it needs to be processed locally on the DSP processor, and processing the audio signal until the audio signal no longer needs to be processed locally on the DSP processor.

[0064] Specifically, the order of algorithm components of the first audio topology project is echo canceller algorithm component, parametric equalizer algorithm component, and AI denoising algorithm component. The DSP processor obtains the initial audio signal, and determines that the next algorithm component to be used is the echo canceller algorithm component according to the preset order of the algorithm components. The algorithm type identifier of the echo canceller algorithm component is not the AI ​​algorithm component. The DSP processor determines that the audio signal needs to be processed locally on the DSP processor through the echo canceller algorithm component, and the DSP processor processes the audio signal through the echo canceller algorithm component.

[0065] After processing the audio signal through the echo canceller, the DSP processor obtains the audio signal processed locally by the DSP processor, and determines that the next algorithm component to be used is the parametric equalizer algorithm component according to the preset order of the algorithm components. The algorithm type identifier of the parametric equalizer algorithm component is not an AI algorithm component. The DSP processor determines that the parametric equalizer algorithm component needs to be used to process the audio signal locally on the DSP processor, and the DSP processor processes the audio signal through the parametric equalizer algorithm component.

[0066] After processing the audio signal through the parametric equalizer, the DSP processor obtains the audio signal processed locally by the DSP processor, and determines that the next algorithm component to be used is the AI ​​denoising algorithm component according to the preset order of the algorithm components. The algorithm type identifier of the AI ​​denoising algorithm component is the AI ​​algorithm component. The DSP processor determines that the audio signal does not need to be processed locally on the DSP processor through the AI ​​denoising algorithm component, and the DSP processor no longer repeats the above steps.

[0067] Step S32: if the DSP processor does not need to process locally, then based on the preset order of the algorithm components in the first audio topology project and the algorithm type identifier, determine whether the ARM processor needs to process; if the ARM processor needs to process, send the second audio signal to the ARM processor; otherwise, output the first audio signal; It should be noted that the DSP processor sends the audio signal to the ARM processor through IIS (Inter-IC Sound), which is used to transmit audio signals between digital audio devices. The ARM processor reads the audio signal through the ALSA (Advanced Linux Sound Architecture) interface, which is used to control audio hardware and transmit data. Linux is a free and open source operating system.

[0068] It is understandable that when the DSP processor determines that there is no need to process the audio signal locally on the DSP processor, the DSP processor determines the algorithm component that needs to be used currently based on the preset order of the algorithm components of the first audio topology project, and determines the algorithm type identifier of the algorithm component. If the algorithm component is an AI algorithm component, then the audio signal needs to be processed on the ARM processor next, and the DSP processor sends the audio signal to the ARM processor through IIS. If the algorithm component is not an AI algorithm component, and since there is no need to process the audio signal locally on the DSP processor, the DSP processor can confirm that there is no algorithm component that needs to be used next, the audio signal processing is completed, and the DSP processor outputs the processed audio signal. For details, please refer to Figure 4 .

[0069] Specifically, the next algorithm component to be used in the audio topology project is the AI ​​denoising algorithm component. The DSP processor determines that the audio signal needs to be processed on the ARM processor based on the algorithm type identifier of the AI ​​denoising algorithm component, and the DSP processor sends the audio signal to the ARM processor.

[0070] Or specifically, there is no next algorithm component to be used in the audio topology project, and the DSP processor determines that the audio signal does not need to be processed on the ARM processor and does not need to be processed on the DSP processor. Then the audio signal has been processed and the DSP processor outputs the audio signal.

[0071] Step S33, when receiving the second audio signal, the ARM processor processes the second audio signal based on the loaded first audio topology sub-project to obtain a third audio signal, and determines whether the third audio signal needs to be processed locally based on the preset order of the algorithm components and the algorithm type identifier in the first audio topology sub-project; It can be understood that, like the DSP processor, the operations such as reading, processing, and writing of audio signals on the ARM processor are performed by the ARM processing thread created by the ARM processor. The audio signal received by the ARM is the audio signal sent by the DSP processor, that is, the DSP processor determines that the audio signal needs to be processed by the algorithm component loaded on the ARM processor, and the ARM processor reads the audio signal through the ALSA interface and processes the audio signal through the algorithm component currently needed.

[0072] The ARM processor obtains the audio signal sent by the DSP processor, indicating that the audio signal needs to be processed by the AI ​​algorithm component loaded on the ARM processor. The ARM processor processes the audio signal through the AI ​​algorithm component according to the preset order of the algorithm components in the first audio topology project, and determines the next algorithm component to be used. According to the algorithm type identifier of the algorithm component, it is determined whether the audio signal needs to be processed locally on the ARM processor. If the algorithm type identifier of the algorithm component is an AI algorithm component, the audio signal needs to be processed locally on the ARM processor. If the algorithm type identifier of the algorithm component is not an AI algorithm component, it means that the ARM processor has processed part of the AI ​​algorithm and does not need to be processed locally on the ARM processor.

[0073] Specifically, the algorithm components that need to be used next in the audio topology project are the AI ​​dereverberation algorithm component and the parameter equalizer algorithm component. The processing thread in the ARM processor reads the audio signal sent by the DSP processor from the ALSA interface, and processes the audio signal through the AI ​​dereverberation algorithm component. The ARM processor obtains the processed audio signal, and determines that the next algorithm component to be used is the parameter equalizer algorithm component according to the preset order of the algorithm components in the audio topology project. The ARM processor determines that the parameter equalizer algorithm component is not an AI algorithm component based on the algorithm component type of the parameter equalizer algorithm component, which means that the ARM processor has processed part of the AI ​​algorithm and does not need to continue to process the audio signal locally on the ARM processor.

[0074] Step S34: if it is necessary to process the third audio signal locally on the ARM processor, the ARM processor processes the third audio signal based on the loaded first audio topology sub-project, and returns to the step of determining whether it is necessary to process the third audio signal locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology project, until local processing is no longer necessary, and the ARM processor sends the third audio signal to the DSP processor.

[0075] It is understandable that after the ARM processor processes the audio signal through the algorithm component loaded on the ARM processor, the ARM processor determines that the audio signal needs to be processed locally on the ARM processor. The ARM processor processes the audio signal through the algorithm component currently required to be used, and obtains the processed audio signal. The ARM processor confirms the next algorithm component to be used based on the preset order of the algorithm components in the first audio topology project, and determines whether the audio signal needs to be processed locally on the ARM processor next through the algorithm type identifier of the algorithm component.

[0076] If the audio signal needs to be processed locally on the ARM processor, the ARM processor processes the audio signal through the algorithm component currently needed, and repeats the steps of obtaining the audio signal, determining the algorithm component to be used, determining the type of the algorithm component, determining whether the audio signal needs to be processed locally on the ARM processor, and processing the audio signal until the audio signal does not need to be processed locally on the ARM processor. If the audio signal does not need to be processed locally on the ARM processor, the ARM processor sends the audio signal to the DSP processor.

[0077] Specifically, the algorithm components that need to be used in the audio topology project are AI denoising algorithm component, AI dereverberation algorithm component, and echo canceller algorithm component. The ARM processor reads the audio signal sent by the DSP processor through the ALSA interface. Since the DSP processor has determined that the audio signal needs to be processed on the ARM processor, the ARM processor processes the audio signal through the AI ​​denoising algorithm component.

[0078] The ARM processor determines that the next algorithm component to be used is the AI ​​dereverberation algorithm component according to the preset order of the algorithm components in the audio topology project, and determines that the algorithm component is an AI algorithm component through the algorithm type identifier of the AI ​​dereverberation algorithm component, which needs to be processed locally on the ARM processor. The ARM processor processes the audio signal through the AI ​​dereverberation algorithm component and obtains the processed audio signal.

[0079] The ARM processor confirms that the next algorithm component to be used is the echo canceller algorithm component according to the order of the algorithm components in the audio topology project, and confirms through the algorithm component type identifier of the echo canceller algorithm component that the algorithm component is not an AI algorithm component and does not need to be processed locally on the ARM processor. The ARM processor writes the audio signal to the ALSA interface.

[0080] In summary, the audio processing module in this embodiment determines the processor that needs to perform audio signal processing operations through the order of algorithm components and algorithm type identification in the audio topology project, and automatically processes audio signals and inputs and outputs audio signals, solving the problem of manually controlling the input and output of audio signals when multiple processors process audio signals, thereby reducing the processing efficiency of audio signals. Therefore, this embodiment can improve the processing efficiency of audio signals, and improve the sound effect processing effect by adding AI algorithm components to process audio.

[0081] Based on the first to third embodiments of the present application, in the fourth embodiment of the present application, refer to Figure 5 The same or similar contents as those in the above-mentioned embodiments 1 to 3 can be referred to the above description and will not be described in detail later. In the fourth embodiment of the present application, the audio processing module processes the audio signal based on the modified algorithm component, including steps S40 to S43: Step S40, the audio topology design platform sends a monitoring instruction to the ARM processor in response to the monitoring operation on the audio signal, wherein the monitoring instruction includes the channel number to be monitored and the target component to be monitored; It should be noted that audio monitoring refers to the process of listening to audio output in real time during audio signal processing. The target component to be monitored is determined by the name of the target component, the channel number to be monitored is the output channel number of the target component, and the monitoring instruction is sent by the audio topology design platform to the ARM processor of the audio processing module.

[0082] It is understandable that the specific operation required for the user to monitor the audio is: the user double-clicks any output PIN pin of any component. The PIN pin refers to the pin in an electronic component or integrated circuit, which is used for electrical connection and signal transmission. The PIN pin in this embodiment belongs to the PIN pin at the software level, which is similar to the PIN pin in an electronic component and is used for connection between components. The PIN pin will change color as the user double-clicks. When the color of the PIN pin is red, it means that the user's monitoring operation is successful. Figure 2 In the figure, the PIN color of the output channel number 1 of the left matrix mixer turns red, indicating that the user has successfully performed the audio monitoring operation on the channel number 1 passing through the matrix mixer.

[0083] Specifically, the user wants to monitor the audio of output channel 1 of the AI ​​denoising algorithm component. Therefore, the user finds the AI ​​denoising algorithm component of the audio topology project in the audio topology design platform interface and double-clicks the PIN corresponding to output channel 1 of the AI ​​denoising algorithm component. The PIN turns red, indicating that the monitoring operation is successful. The audio topology design platform sends the generated monitoring instruction to the ARM processor of the audio processing module. The information contained in the monitoring instruction is the AI ​​denoising algorithm component and output channel 1.

[0084] Step S41, if the target component is a traditional algorithm component, the monitoring instruction is sent to the DSP processor, the DSP processor sends the fourth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the traditional algorithm component to the ARM processor, and the ARM processor sends the fourth audio signal to the audio topology design platform; It should be noted that the monitoring operation on the ARM processor is performed through the ARM monitoring thread created by the ARM processor. Traditional components include parametric equalizers, echo cancellers, input and output components, level meters and other components. In addition to traditional algorithm components and AI algorithm components, the audio processing module also includes non-algorithm components, such as control components and other components used for logical control. The control component is mainly used to trigger another action after a specified parameter is modified to achieve various control effects.

[0085] The fourth audio signal is an audio signal obtained after being processed by the algorithm component and finally processed by the traditional component. The audio signal is sent to the audio topology design platform through Websocket by the ARM processing monitoring thread in PCM (Pulse Code Modulation) format. The PCM format is a digital audio format that converts analog audio signals into digital form for storage or transmission. By using the PCM format, it can be played without decoding when output. Websocket is a protocol for establishing a two-way communication channel between a server and a client. In this embodiment, it is mainly used for data transmission between the ARM processor and the audio topology design platform.

[0086] It is understandable that the ARM processor receives the monitoring instruction, confirms the target component and target component channel number to be monitored through the information in the monitoring instruction, and confirms that the algorithm component is not an AI algorithm component through the algorithm type identifier of the target component, that is, the audio signal to be monitored is not in the ARM processor, and needs to be sent to the ARM processor by the DSP processing thread first, and then sent to the audio topology design platform by the ARM monitoring thread. The ARM processor sends the audio signal to the audio topology design platform in PCM format through Websocket, and the audio topology design platform receives the audio signal through Websocke.

[0087] Specifically, the information contained in the monitoring instruction received by the ARM processor is that the component is a matrix mixer and the channel number is output channel No. 2. The ARM processor determines that the component is not an AI algorithm component through the algorithm type identifier of the component, and sends the monitoring instruction to the DSP processor. The DSP processor sends the audio signal of output channel No. 2 of the matrix mixer to the ARM processor based on the received instruction. The ARM monitoring thread sends the audio signal to the audio topology design platform in PCM format through Websocket, and the audio topology design platform receives the audio signal through Websocket.

[0088] Step S42: if the target component is an AI algorithm component, the ARM processor sends the fifth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the AI ​​algorithm component to the audio topology design platform; It is understandable that the ARM processor receives the monitoring instruction, confirms the target component and the channel number of the target component to be monitored through the information in the monitoring instruction, and confirms that the algorithm component is an AI algorithm through the algorithm type identifier of the target component, that is, the audio signal to be monitored is in the ARM processor and can be sent to the audio topology design platform by the ARM processor. The ARM processor sends the audio signal to the audio topology design platform through Websocket in PCM format, and the audio topology design platform receives the audio signal through Websocket.

[0089] Specifically, the monitoring instruction received by the ARM processor contains the component AI denoising algorithm and the channel number output channel 1. The ARM processor determines that the component is an AI algorithm component through the algorithm type identifier of the component, and the audio signal to be monitored is on the ARM processor. The ARM monitoring thread sends the audio signal of output channel 1 of the AI ​​denoising algorithm component to the audio topology design platform through Websocket in PCM format, and the audio topology design platform receives the audio signal through Websocket.

[0090] Step S43: the audio topology design platform plays the audio corresponding to the fourth audio signal or the fifth audio signal to complete monitoring.

[0091] It should be noted that the fourth audio signal or the fifth audio signal is in PCM format. The audio topology design platform plays the fourth audio signal or the fifth audio signal through a PCM format player. At this time, the PCM format player converts the audio signal into audio, and the user listens to the audio played by the audio topology design platform in real time to complete the monitoring of the audio signal.

[0092] In summary, in this embodiment, the user selects the output channel of the algorithm component that he wants to monitor in the audio topology design platform, and the audio processing module determines the location of the audio signal that needs to be monitored based on the received monitoring instruction, and sends the audio signal that needs to be monitored to the audio topology design platform for playback by the ARM processor, and completes the monitoring of the audio signal. The problem of difficulty in monitoring the audio signal of any algorithm of any channel is solved, and the convenience of audio signal monitoring is improved.

[0093] It should be noted that the above examples are only used to assist in understanding the present application and do not constitute a limitation on the audio processing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0094] This application also provides an audio processing system, please refer to Figure 6 , the audio processing system comprises: The audio topology design platform 10 is used to obtain a modified algorithm component in response to a modification operation on an algorithm component of a preset audio topology project in a display interface, wherein the modification operation includes modifying a type of the algorithm component and / or modifying a parameter of the algorithm component; The audio processing module 20 is used to process the audio signal based on the modified algorithm component.

[0095] The audio processing system provided by the present application adopts the audio processing method in the above embodiment, which can solve the technical problem of low audio processing efficiency. Compared with the prior art, the beneficial effects of the audio processing system provided by the present application are the same as the beneficial effects of the audio processing method provided by the above embodiment, and other technical features of the audio processing system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0096] The present application provides an audio processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio processing method in the above-mentioned embodiment 1.

[0097] Reference below Figure 7 , which shows a schematic diagram of the structure of an audio processing device suitable for implementing the embodiment of the present application. The audio processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, tablet computers, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The audio processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0098] like Figure 7 As shown, the audio processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the audio processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the audio processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an audio processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0099] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0100] The audio processing device provided by the present application adopts the audio processing method in the above embodiment to solve the technical problem of low audio processing efficiency. Compared with the prior art, the beneficial effects of the audio processing device provided by the present application are the same as the beneficial effects of the audio processing method provided by the above embodiment, and other technical features in the audio processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0101] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0102] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0103] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the audio processing method in the above-mentioned embodiment.

[0104] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0105] The computer-readable storage medium may be included in the audio processing device; or may exist independently without being assembled into the audio processing device.

[0106] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the audio processing device, the audio processing device executes the audio processing method.

[0107] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0110] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned audio processing method, and can solve the technical problem of low audio processing efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the audio processing method provided in the above-mentioned embodiment, and will not be described in detail here.

[0111] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned audio processing method when executed by a processor.

[0112] The computer program product provided by this application can solve the technical problem of low audio processing efficiency. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the audio processing method provided by the above embodiment, which will not be repeated here.

[0113] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An audio processing method, It is characterized in that Applied to an audio processing system, the audio processing system includes an audio topology design platform and an audio processing module, and the method includes: The audio topology design platform obtains a modified algorithm component in response to a modification operation on an algorithm component of a preset audio topology project in a display interface, wherein the modification operation includes modifying a type of the algorithm component and / or modifying a parameter of the algorithm component; The audio processing module processes the audio signal based on the modified algorithm component.

2. The method according to claim 1, It is characterized in that The audio processing module processes the audio signal based on the modified algorithm component, comprising: The audio processing module loads the modified audio topology project and processes the audio signal based on the loaded first audio topology project, wherein the modified audio topology project is obtained by the audio topology design platform in response to a modification operation on the type of the algorithm component, and is sent to the audio processing module in response to a confirmation upload operation on the modified audio topology project; And / or, the audio processing module modifies the parameters of the algorithm component of the loaded second audio topology project based on the parameter modification instruction to obtain a third audio topology project, and processes the audio signal based on the third audio topology project, wherein the second audio topology project includes a preset audio topology project or a first audio topology project, and the parameter modification instruction is sent to the audio processing module by the audio topology design platform in response to the parameter modification operation on the algorithm component of the second audio topology project.

3. The method according to claim 2, It is characterized in that The audio processing module is provided with a DSP processor and an ARM processor, and the steps of loading the modified audio topology engineering into the audio processing module include: The ARM processor obtains the algorithm type identifier of the algorithm component in the modified audio topology project; If the algorithm type identifier includes an AI algorithm, the ARM processor splits the modified audio topology project into a first audio topology sub-project corresponding to the AI ​​algorithm and a second audio topology sub-project corresponding to the traditional algorithm, and sends the second audio topology sub-project to the DSP processor; The ARM processor loads the first audio topology sub-project, and the DSP processor synchronously loads the second audio topology sub-project.

4. The method according to claim 3, It is characterized in that The algorithm components of the modified audio topology project are in a preset order, the loaded first audio topology project includes the loaded first audio topology sub-project and the loaded second audio topology sub-project, and the step of processing the audio signal based on the loaded first audio topology project includes: The DSP processor obtains a first audio signal, and determines whether the first audio signal needs to be processed locally based on a preset order of algorithm components and an algorithm type identifier in the first audio topology project, wherein the first audio signal includes a second audio signal or a third audio signal processed by the ARM processor, and the second audio signal is an audio signal obtained by processing based on the loaded second audio topology sub-project when the audio signal needs to be processed locally by the DSP processor, or is an initial audio signal that has not been processed by the algorithm component; If it is necessary to process the first audio signal locally on the DSP processor, the DSP processor processes the first audio signal based on the loaded second audio topology sub-project, and returns to the step of determining whether the first audio signal needs to be processed locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology project, until it is no longer necessary to process the first audio signal locally; If the DSP processor does not need to process locally, judging whether the ARM processor needs to process based on the preset order of the algorithm components and the algorithm type identifier in the first audio topology project, and if the ARM processor needs to process, sending the second audio signal to the ARM processor, otherwise, outputting the first audio signal; When receiving the second audio signal, the ARM processor processes the second audio signal based on the loaded first audio topology sub-project to obtain a third audio signal, and determines whether the third audio signal needs to be processed locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology sub-project; If it is necessary to process the third audio signal locally on the ARM processor, the ARM processor processes the third audio signal based on the loaded first audio topology sub-project, and returns to the step of determining whether it is necessary to process the third audio signal locally based on the preset order of algorithm components and algorithm type identifiers in the first audio topology project, until it is no longer necessary to process the third audio signal locally, and the ARM processor sends the third audio signal to the DSP processor.

5. The method according to claim 3, It is characterized in that The step of the audio processing module modifying the parameters of the algorithm component of the loaded second audio topology project based on the parameter modification instruction includes: If the algorithm type identifier is an AI algorithm identifier, the ARM processor modifies the parameters of the algorithm component according to the algorithm component parameter modification instruction; If the algorithm type identifier is not an AI algorithm identifier, the ARM processor sends the algorithm component parameter modification instruction to the DSP processor, and the DSP processor modifies the parameters of the algorithm component according to the algorithm component parameter modification instruction.

6. The method according to claim 4, It is characterized in that The audio processing module is based on a modified algorithm component, and after the audio signal processing step, it also includes: The audio topology design platform sends a monitoring instruction to the ARM processor in response to the monitoring operation on the audio signal, wherein the monitoring command includes the channel number to be monitored and the target component to be monitored; If the target component is a traditional algorithm component, the monitoring command is sent to the DSP processor, the DSP processor sends the fourth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the traditional algorithm component to the ARM processor, and the ARM processor sends the fourth audio signal to the audio topology design platform; If the target component is an AI algorithm component, the ARM processor sends the fifth audio signal which is in the channel corresponding to the channel number and obtained after being processed by the AI ​​algorithm component to the audio topology design platform; The audio topology design platform plays the audio corresponding to the fourth audio signal or the fifth audio signal to complete monitoring.

7. An audio processing system, It is characterized in that The audio processing system includes an audio topology design platform and an audio processing module; The audio topology design platform is used to respond to a modification operation on an algorithm component of a preset audio topology project in a display interface to obtain a modified algorithm component, wherein the modification operation includes modifying a type of the algorithm component and / or modifying a parameter of the algorithm component; The audio processing module is used to process the audio signal based on the modified algorithm component.

8. An audio processing device, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio processing method according to any one of claims 1 to 6.

9. A storage medium, It is characterized in that The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the audio processing method according to any one of claims 1 to 6 are implemented.

Images