Audio noise reduction accelerator system and method based on RISC v custom instruction set expansion
By customizing the instruction set expansion on the RISCv instruction set architecture, combined with the collaborative work of E203_CORE and NICE_CORE, the hardware acceleration of the audio noise reduction algorithm is achieved, solving the difficulties in the hardware implementation of the audio noise reduction algorithm in the existing technology, and achieving flexibility Fast and dedicated audio noise reduction system.
Patent Information
- Application Number
- CN202111037629.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-06
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2041-09-06
AI Technical Summary
The prior art is difficult to effectively realize hardware acceleration of audio noise reduction algorithms, especially the traditional hardware FxLMS algorithm implementation has the challenges of multiplier and adder, and does not have the flexibility and flexibility of co-processing of software and hardware.
The audio noise reduction accelerator system based on RISCv custom instruction set expansion is adopted. Through the coordinated work of E203_CORE and NICE_CORE, custom instructions such as Mac and Conv instruction circuits are used to accelerate multiplication and accumulation operations and convolution operations.
It realizes the flexible and mutable algorithm, fast computing and specialized audio noise reduction system, and can be widely used in the fields of noise reduction of sound sources such as automobiles, headphones, and aircraft.
Smart Images
Figure CN113851103B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of integrated circuits, and in particular relates to an audio noise reduction accelerator system and method based on RISC v custom instruction set expansion. Background Art
[0002] With the development of economy and the continuous advancement of technology, the application scenarios of audio noise reduction systems are increasing, such as in-car noise reduction, noise reduction in gas stations, and noise reduction in headphones. However, passive noise reduction methods such as physical isolation of noise sources are often adopted. Active noise reduction methods that allow sound source signals to superimpose and cancel each other often have hardware implementation difficulties. Improved excellent algorithms are difficult to implement in hardware using FPGAs. Therefore, the use of software implementation will enable the improved algorithms to be implemented, which will be an important means to solve the difficulties in implementing audio noise reduction algorithms.
[0003] At the same time, RISC v, as an emerging instruction set architecture, has the advantages of open source and latecomer. For the embedded field, the soft core using this instruction set architecture can customize instructions, and can customize dedicated acceleration unit circuits for the multiplication and accumulation structure and convolution structure in the algorithm, realizing the dedicated design field of audio noise reduction. With the enthusiastic promotion of RISC v instruction set architecture in China, such dedicated field SOC chip design will be able to achieve greater development.
[0004] Then, with the rise of AIOT in China, many different types of accelerators have emerged to address issues such as the running speed of processor cores. Common ones include neural network accelerators for images, of which there are many types. However, the traditional hardware FxLMS algorithm implementation does not have a suitable accelerator to accelerate its specific operations. As the complexity of the algorithm increases, the multipliers and adders make the hardware implementation of the FxLMS algorithm more challenging. Therefore, the use of FPGA+MCU architecture for hardware and software collaboration, using software to implement the algorithm and hardware to accelerate specific operations, will show new directions and possibilities in audio noise reduction systems.
[0005] Traditional noise reduction methods, such as extracting the amplitude and angle values of multiple noise audio signals, and then using a neural network for training to determine the complex spectrum of a pure audio signal, and then inverse transforming the obtained complex spectrum, and finally performing audio noise reduction based on this inverse transformation. Such methods are often not flexible, and require retraining for different noise sources. At the same time, such methods do not have the flexibility of using software and hardware collaborative processing. Therefore, the audio noise reduction accelerator system based on the RISCv custom instruction set expansion of the present invention will provide a new method for audio noise reduction. Summary of the invention
[0006] The present invention aims to solve the above problems of the prior art. An audio noise reduction accelerator system and method based on the expansion of the RISC v custom instruction set are proposed. The present invention can realize the flexible and modifiable algorithm, the stability of transmission, the rapidity of operation, and the audio noise reduction system with strong specialization. The system can be widely used in the fields of noise reduction of automobile noise reduction, headphone noise reduction, high-speed rail and airplane noise reduction, etc. The technical solution of the present invention is as follows:
[0007] An audio noise reduction accelerator system based on RISC v custom instruction set expansion includes: E203_CORE (Hummingbird E203 open source processor core), NICE_CORE (custom coprocessor), NICE_Interface (interface circuit between main processor and coprocessor), E203_SOC (system on chip built with E203 as core), audio codec WM8731 module, audio noise reduction FxLMS algorithm, among which,
[0008] The E203_CORE is connected to the NICE_CORE through the NICE_Interface, and the E203_CORE, NICE_CORE and related peripheral ports together form the E203_SOC, which is connected to the audio codec WM8731 module. The audio noise reduction FxLMS algorithm is downloaded to the RISC through software programming. v processor core; the executable file is saved in the off-chip Flash memory. After the system starts, E203_CORE is initialized first, and then the IIC interface circuit on the peripheral bus is accessed according to the instruction sequence. According to the instruction, the audio codec module WM8731 is first configured through the IIC interface circuit. After that, the sound signal and the sound source signal after the superposition of the target noise are collected through the reference microphone, and the residual noise is collected through the error microphone. The collected sound source signal and residual noise are passed through the Audio_data_rx of the WM8731 module, and the analog signal is converted into a digital signal through the built-in ADC conversion module of the WM8731 module. After that, the digital filter module built in the WM8731 module is used for filtering, and the filtered digital signal is transmitted to the peripheral bus through the IIS interface circuit. Then, according to the instruction, the filtered digital signal is obtained from the peripheral bus and transmitted to E203_CORE according to the FxLMS algorithm. The software implementation process is processed. When processing the multiplication and accumulation operation and the convolution operation, they will be detected as custom instructions by the decoding part in E230_CORE. E203_CORE passes the custom instructions and source operands to NICE_CORE through NICE_Interface to accelerate the specific operation part. The multiplication and accumulation operation will call the Mac instruction circuit, and the convolution operation will call the Conv instruction circuit. Then, according to the instruction execution, the processed digital signal is transmitted to the peripheral bus, and the digital signal processed by the noise reduction FxLMS algorithm is output, and then transmitted to the WM8731 audio codec module through the IIS peripheral interface circuit for DAC conversion, and the processed digital signal is converted into an analog signal, which is transmitted to the secondary sound source through Audio_data_tx to obtain the anti-phase noise. The anti-phase noise finally obtained interferes with the target noise, thereby performing active noise reduction of the audio and realizing the audio noise reduction system.
[0009] Furthermore, the E203_SOC is composed of E203_CORE, NICE_CORE, ICB_APB, UART serial port, IIC interface and IIS interface; wherein, E203_CORE is a processor core for running instructions and executing instructions in sequence, and controlling corresponding modules to process data information according to the content of the instructions, and the general register Reg of the execution unit EXU in E203_CORE saves the processed data returned by the coprocessor, and the data tight coupler DTCM in the memory access control unit LSU in E203_CORE provides data corresponding to the source register index to the coprocessor, so that the custom instruction can read and write data, NICE_Interface is used for communication and transmission between E203_CORE and NICE_CORE, NICE_CORE is used to accelerate the processing of data transmitted from the processor core, obtain the custom instruction through NICE_Interface, and then find the corresponding custom instruction through decoding Decode, the instruction is Lbuf, Sbuf, Conv, Mac, and then the data is processed according to the function of the corresponding instruction. The data is accessed for storage or data processing and written back. The Lbuf instruction is used for data loading, which obtains data from the DTCM in the LSU and stores it in Buf. The Sbuf instruction is used for data storage, which returns the data in Buf to the DTCM in the LSU. The Conv instruction obtains the weight coefficient and the input signal from Buf for convolution operation, and writes the result back to Reg in EXU. The Mac instruction obtains the weight coefficient and the error component from Buf for multiplication and accumulation operation, and temporarily stores the result in Buf. After the operation is completed, the entire result is written back to Reg in EXU. The whole process is controlled by a finite state machine FSM for state transfer. ICB_APB is used for communication and transmission between the processor core and various peripheral interfaces. The UART serial port is used to transmit ADCDATA and DACDATA to the host computer for storage, which is convenient for data analysis using MATLAB. The IIC interface is used by the processor core to configure the registers of the audio codec WM8731 module, and the IIS interface is used for data transmission between the processor core and the audio codec WM8731 module.
[0010] Furthermore, the NICE_CORE decodes the signals including the custom instruction and source operand register address transmitted from E203_CORE through NICE_Interface, decodes the corresponding custom instruction by comparing the specific format part, and then executes the corresponding custom instruction under the control of the state machine.
[0011] Furthermore, the audio codec module WM8731 is internally provided with a built-in ADC, a DAC and a digital filter. Through the MIC transmission acquisition mode, the external sound source to be processed is collected, and the analog signal is converted into a digital signal through its built-in ADC, and then the obtained digital signal is filtered through the built-in digital filter to obtain the output data ADCDATA, and then the data DACDATA processed by the RISC v soft core is input into the DAC in the audio codec module to convert the digital signal into an analog signal, and the converted analog signal is output through the MIC.
[0012] Furthermore, the audio codec module WM8731 is internally provided with an IIC interface circuit, which can perform operations including configuration of master-slave mode, selection of ADC and DAC enable, volume in MIC and LINE modes, and whether to turn on digital filtering. Since the principle of the ANC audio noise reduction system is to utilize the destructive interference of sound waves, the WM8731 module will be configured as slave mode, MIC mode, ADC, and DAC enable, and the register configuration information will be configured by E203_CORE through the IIC interface bus.
[0013] Furthermore, the audio noise reduction FxLMS algorithm is written in C language as the description language and inline assembly language as the auxiliary language. It is compiled and assembled through IDE software to link and download the executable file to the RISC v soft core, and the FxLMS audio noise reduction algorithm is implemented in software through the execution of instructions.
[0014] Furthermore, in the software implementation of the audio noise reduction FxLMS algorithm, the digital signal obtained after the noise reduction processing of the digital signal to be processed is subjected to noise reduction processing using the following formula:
[0015] y(n)=y(n)+w(n)(k-2)x(n)
[0016] e(n)=d(n)-y s (n)
[0017] d(n)=p(n)*x(n)
[0018] y s (n) = s(n) * y(n)
[0019]
[0020]
[0021] Where y(n): secondary sound source, w(n)(k): weight coefficient, x(n): sound source signal, e(n): error signal, d(n): residual noise, ys (n): secondary sound source through the secondary path, p(n): reference signal from the sound source signal to the error microphone, s(n): signal generated by the secondary sound source to the error microphone secondary path, The estimated value of the sound source signal, where the constraint on the step factor is:
[0022]
[0023] Where μ: step size of FxLMS algorithm; λ max : The maximum eigenvalue of the autocorrelation matrix.
[0024] An audio noise reduction method based on the system comprises the following steps:
[0025] First, the executable file generated after the C language program is compiled, assembled and linked is saved in the off-chip Flash memory. After the system starts, E203_CORE is initialized first, and then the IIC interface circuit on the peripheral bus is accessed according to the instruction sequence. According to the instruction, the audio codec module WM8731 is first configured through the IIC interface circuit;
[0026] Secondly, the reference microphone collects the sound source signal after the sound signal and the target noise are superimposed, and the error microphone collects the residual noise. The collected sound source signal and residual noise are passed through the Audio_data_rx of the WM8731 module, and the built-in ADC conversion module of the WM8731 module converts the analog signal into a digital signal. After that, the built-in digital filter module of the WM8731 module performs filtering processing, and the filtered digital signal is transmitted to the peripheral bus through the IIS interface circuit;
[0027] Next, according to the instruction, the filtered digital signal is obtained from the peripheral bus and transmitted to E203_CORE for processing according to the software implementation process of the FxLMS algorithm. When processing the multiplication and accumulation operation and the convolution operation, the decoding part in E203_CORE will detect it as a custom instruction. E203_CORE will pass the custom instruction and source operand to NICE_CORE through NICE_Interface to accelerate the specific operation part. The multiplication and accumulation operation will call the Mac instruction circuit, and the convolution operation will call the Conv instruction circuit.
[0028] Then, according to the instruction execution, the processed digital signal is transmitted back to the peripheral bus, the digital signal processed by the noise reduction FxLMS algorithm is output, and then transmitted to the WM8731 audio codec module through the IIS peripheral interface circuit for DAC conversion, the processed digital signal is converted into an analog signal, and transmitted to the secondary sound source through the audio data sending end to obtain the anti-phase noise;
[0029] Finally, the anti-phase noise and the target noise interfere with each other, thereby performing active audio noise reduction and realizing an audio noise reduction system.
[0030] Furthermore, a custom instruction set encoding format is defined in the coprocessor, involving a total of 4 custom instructions, namely LBUF instruction, SBUF instruction, CONV instruction, and MAC instruction. Since the LBUF instruction and the SBUF instruction operate on the BUF of the same address, they need to be distinguished by their opcode fields, which are determined by custom0 and custom1, respectively, namely 7`b0001011 and 7`b0101011. Since the SBUF and LBUF instructions only need to read and write from DTCM and do not need to write data back, only the source operand 1, namely rs1, needs to be used. Therefore, the funct3 field is defined as 3`b010 format, which means that only the source operand register rs1 is operated. Since rd and rs2 are not used, this part can be defined as the length of the data to be read or written. The funct7 field is defined as the starting address of loading and storing in BUF, wherein the CONV instruction only needs the target register to return data, so it needs to operate on the target register it returns, i.e., rd. The funct3\funct7\rs1\rs2 and other parts are defined as the data X[9:0] and weight coefficient W[9:0] it processes, which are all read by the pseudo dual-port SRAM and passed to the CONV circuit part for calculation. The MAC instruction needs to call the target register and source operation for its calculation, so its funct3 field is defined as 3`b111 format, and the funct7 field has no definition requirements. The opcodes of CONV and MAC instructions are defined as any of four custom formats. The present invention uses custom3 and custom4, i.e., 7`b1011011 and 7`b1111011.
[0031] Furthermore, the data information to be processed can be loaded through the LBUF instruction, and then stored in the two pseudo dual-port SRAMs XBUF and QBUF, and WBUF defines an empty pseudo dual-port SRAM in the form of initial values. Then, the data of XBUF and WBUF are read through the CONV instruction. This instruction is a convolution operation unit circuit using an addition tree structure, and its data length is determined by the filter order, that is, when the filter order is J, its signal to be processed X[J-1:0] and weight coefficient W[J-1:0] are obtained. After the operation result is obtained, it is saved in YBUF for FIFO storage, and its width and depth are defined as the data width and the order length of the filter. Afterwards, the MAC instruction is used to multiply QBUF by the error coefficient and then add the resultant to the weight coefficient WBUF in the previous column to obtain a new weight coefficient WBUF. The new weight coefficient is written to it through the MAC custom instruction to update the weight. The product of the error coefficient is obtained by subtracting the primary noise signal Dn from the data stored in YBUF and then multiplying it by the step factor. The above steps are then repeated until all outputs fill the depth of the FIFO.
[0032] The advantages and beneficial effects of the present invention are as follows:
[0033] 1. The present invention adopts RISC v soft-core SOC, which is highly configurable and can configure different peripheral interface circuits according to different functional requirements. At the same time, some interface peripherals that are not used in the circuit are deleted to reduce the waste of resources.
[0034] 2. The present invention adopts RISC v instruction set architecture, can define instructions according to the key steps of the algorithm, and then use the inline assembly format to design the hardware acceleration unit to accelerate the specific key steps of the algorithm, making the algorithm faster and reducing the number of instructions and execution cycles.
[0035] 3. The present invention adopts a hardware IIS interface circuit, which can stably transmit audio signals, and the configurable FIFO width and depth and clock unit, etc. can meet the selection of different sampling rates and sampling byte numbers, and its application scope is wider.
[0036] 4. The present invention adopts four custom instructions to accelerate the operation, among which the CONV instruction only uses the multiplier and the adder to form a convolution operation unit with an addition tree structure, and can reuse the instruction circuit for multiple convolution operations in the algorithm. At the same time, the MAC instruction can also reuse the multiplication and accumulation part of the algorithm multiple times, reducing the hardware resources consumed by the entire coprocessor development and reducing its granularity.
[0037] 5. The present invention uses software to implement the FxLMS algorithm, uses the Jacobi formula to find the maximum eigenvalue of the matrix, and then finds the inverse to get the step size factor. Compared with pure hardware implementation, software implementation is more feasible and not as difficult as hardware implementation. At the same time, software implementation has better flexibility. Under the RISC v instruction set system, hardware acceleration units can also be used to partially accelerate it. The software and hardware collaborative processing method is adopted to bring together the advantages of hardware and software development. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 This is a system architecture diagram of an audio noise reduction accelerator according to a preferred embodiment of the present invention;
[0039] Figure 2 NICE_Interface interface circuit diagram;
[0040] Figure 3 Schematic diagram of custom instruction set encoding;
[0041] Figure 4 The circuit structure diagram of convolution operation and multiplication and accumulation operation is shown in FIG.
[0042] Figure 5 The hardware model structure diagram of FxLMS algorithm is shown in Figure 2.
[0043] Figure 6 This is the C language implementation flow chart of the FxLMS algorithm;
[0044] Figure 7 This is the VIVADO joint simulation diagram of MAC custom instruction set. DETAILED DESCRIPTION
[0045] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.
[0046] The technical solution of the present invention to solve the above technical problems is:
[0047] The present invention designs an audio noise reduction accelerator system based on RISC v custom instruction set expansion, as shown in the attached Figure 1 As shown, its structure includes: E203_CORE, NICE_CORE, NICE_Interface, E203_SOC, audio codec WM8731 module, audio noise reduction FxLMS algorithm; among which:
[0048] The E203_SOC is composed of E203_CORE, NICE_CORE, ICB_APB, UART serial port, IIC interface and IIS interface; wherein, an executable file generated after C language program compilation and assembly linking is stored through an off-chip Flash memory. After the system is started, E203_CORE is initialized first, and then the IIC interface circuit on the peripheral bus is accessed according to the instruction sequence. According to the instruction, the audio codec module WM8731 is first configured through the IIC interface circuit. Thereafter, the sound source signal after the sound signal and the target noise are superimposed is collected through the reference microphone, and the residual noise is collected through the error microphone. The collected sound source signal and the residual noise are passed through the Audio_data_rx of the WM8731 module, and the analog signal is converted into a digital signal through the built-in ADC conversion module of the WM8731 module. Then, the digital filter module built in the WM8731 module is used for filtering, and the filtered digital signal is transmitted to the peripheral bus through the IIS interface circuit, and then the filtered digital signal is obtained from the peripheral bus according to the instruction. And transmitted to E203_CORE for processing according to the software implementation process of the FxLMS algorithm. When processing the multiplication and accumulation operation and the convolution operation, since the custom instructions in the form of inline assembly are used in the C language, they will be detected as custom instructions by the decoding part of E203_CORE. E203_CORE passes the custom instructions and source operands to NICE_CORE through NICE_Interface for acceleration of specific operation parts. The multiplication and accumulation operation will call the Mac instruction circuit, and the convolution operation will call the Conv instruction circuit. Then, according to the instruction execution, the processed digital signal is transmitted to the peripheral bus, and the digital signal processed by the noise reduction FxLMS algorithm is output, and then transmitted to the WM8731 audio codec module through the IIS peripheral interface circuit for DAC conversion, and the processed digital signal is converted into an analog signal, which is transmitted to the secondary sound source through Audio_data_tx to obtain the anti-phase noise. Finally, the anti-phase noise obtained interferes with the target noise, thereby performing active noise reduction of the audio and realizing the audio noise reduction system.
[0049] In order to ensure good timing stability between NICE_CORE and E203_CORE, the present invention adopts the following Figure 2The interface circuit NICE_Interface shown, when the EXU execution module detects that the instruction is a custom instruction, by judging the encoding of its opcode and funct3 parts, the instruction and source operand information are passed to NICE_CORE. NICE_CORE accelerates the operation of the coprocessor part under the control of the internal FSM state machine, specifically the convolution operation and the multiplication and accumulation operation part. When it needs to access the data in the DTCM in the LSU, it sends the address read and write request and write data information to the LSU, and waits for the LSU to return the read result for NICE_CORE to process. After the processing is completed, it judges whether it needs to return to the target register according to the custom instruction encoding rules. If necessary, the result is returned to the general register in the EXU.
[0050] The present invention adopts the custom instruction set definition officially reserved by RISC v International, such as Figure 3As shown, the opcode field, funct3 field, and funct7 field determine the difference in the custom instruction set, and rd, rs1, rs2, etc. respectively represent the write-back target register, source operand register 1, and source operand register 2. In the present invention, a total of 4 custom instructions are involved, namely, the LBUF instruction, the SBUF instruction, the CONV instruction, and the MAC instruction. Since the LBUF instruction and the SBUF instruction operate on the BUF of the same address, they need to be distinguished by their opcode fields, which are determined by custom0 and custom1, respectively, i.e., 7`b0001011 and 7`b0101011. Since the SBUF and LBUF instructions only need to read and write from the DTCM and do not need to write back data, they only need to use their source operand 1, i.e., rs1. Therefore, the funct3 field is defined as the 3`b010 format, which means that only the source operand register rs1 is operated. Since rd and rs2 are not used, this part can be defined as the length selection of the data to be read or written, and the funct7 word The segment is defined as the starting address of loading and storing in BUF, wherein the CONV instruction only needs the target register to return data, so it needs to operate on the target register it returns, i.e., rd. The funct3\funct7\rs1\rs2 and other parts are defined as the data X[9:0] and weight coefficient W[9:0] it processes, which are all read by the pseudo dual-port SRAM and passed to the CONV circuit part for calculation. The MAC instruction needs to call the target register and the source operand for its calculation, so its funct3 field is defined as 3`b111 format, and the funct7 field has no definition requirements. The opcodes of the CONV and MAC instructions are defined as any of the four custom ones. The present invention uses custom3 and custom4, i.e., 7`b1011011 and 7`b1111011.
[0051] The custom instruction operation part involved in the present invention is as follows Figure 4As shown, the data information to be processed can be loaded through the LBUF instruction, and then stored in the two pseudo dual-port SRAMs XBUF and QBUF, and WBUF defines an empty pseudo dual-port SRAM in the form of initial values. Then, the data of XBUF and WBUF are read through the CONV instruction. Under this instruction is a convolution operation unit circuit using an addition tree structure, and its data length is determined by the filter order, that is, when the filter order is J, its signal to be processed X[J-1:0] and weight coefficient W[J-1:0] are obtained. After the operation result is obtained, it is saved in YBUF for FIFO storage, and its width and depth are defined as the data width and the order length of the filter. Afterwards, the product of QBUF and the error coefficient is multiplied by the MAC instruction and then added to the weight coefficient WBUF in the previous column to obtain a new weight coefficient WBUF. The new weight coefficient is written to it through the MAC custom instruction to update the weight. The product part of the error coefficient is obtained by subtracting the primary noise signal Dn from the data stored in YBUF and then multiplying it by the step factor. The above steps are then repeated until the depth of all output FIFOs is filled.
[0052] The present invention analyzes the hardware model of the typical FxLMS algorithm as shown in the attached Figure 5 As shown in the figure, the input sequence passes through the delayer and is multiplied by the weight coefficient and then accumulated to obtain the output sequence. At the same time, the weight coefficient enters the LMS and is updated and iterated. This step is repeated many times to update the input sequence many times to obtain the sequence after the noise reduction algorithm is processed. At the same time, through the hardware model of the FxLMS algorithm, the following calculation formula can be extracted:
[0053] y(n)=y(n)+w(n)(k-2)x(n)
[0054] e(n)=d(n)-y s (n)
[0055] d(n)=p(n)*x(n)
[0056] y s (n) = s(n) * y(n)
[0057]
[0058]
[0059] Then the formula is as follows Figure 6 The C language program shown in the figure also adds our specific hardware acceleration unit circuit for the specific operation part of the algorithm, adds convolution operation unit for the digital filter part, and adds multiplication and accumulation circuit for the part of weight coefficient update to accelerate the operation, so as to reflect the advantages of RISC v instruction set architecture.
[0060] Finally, as attached Figure 7 As shown in the figure, it is the result of VIVADO joint simulation using MAC custom instructions. From the figure, we can see the specific instruction format, the data to be processed loaded, and the data obtained by the final calculation, etc., which shows that the hardware acceleration part of the whole system is complete.
[0061] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.
[0062] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0063] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.
Claims
1. An audio noise reduction accelerator system based on RISC v custom instruction set expansion, characterized in that: include: E203_CORE, NICE_CORE, NICE_Interface, E203_SOC, audio codec WM8731 module, audio noise reduction FxLMS algorithm, E203_CORE is the core of the Hummingbird E203 open source processor, NICE_CORE is a custom coprocessor, NICE_Interface is the interface circuit between the main processor and the coprocessor, E203_SOC is a system on chip built with E203 as the core, wherein the E203_CORE is connected to NICE_CORE through NICE_Interface, E203_CORE, NICE_CORE and related peripheral ports together form E203_SOC, E203_SOC is connected to the audio codec WM8731 module, and the audio noise reduction FxLMS algorithm is downloaded to RISC through software programming v runs in the processor core; the executable file is saved in the off-chip Flash memory. After the system starts, E203_CORE is initialized first, and then the IIC interface circuit on the peripheral bus is accessed according to the instruction sequence. First, the audio codec module WM8731 is configured through the IIC interface circuit. Then, the sound signal and the sound source signal after the superposition of the target noise are collected through the reference microphone, and the residual noise is collected through the error microphone. The collected sound source signal and residual noise are passed through the Audio_data_rx of the WM8731 module, and the analog signal is converted into a digital signal through the built-in ADC conversion module of the WM8731 module. After that, the digital filter module built in the WM8731 module is used for filtering, and the filtered digital signal is transmitted to the peripheral bus through the IIS interface circuit. Then, according to the instruction, the filtered digital signal is obtained from the peripheral bus and transmitted to the software in E203_CORE according to the FxLMS algorithm. The implementation process is processed, wherein when processing the multiplication and accumulation operation and the convolution operation, they will be detected as custom instructions by the decoding part in E230_CORE, and E203_CORE will pass the custom instructions and source operands to NICE_CORE through NICE_Interface to accelerate the specific operation part, wherein the multiplication and accumulation operation will call the Mac instruction circuit, and the convolution operation will call the Conv instruction circuit, and then according to the instruction execution, the processed digital signal will be transmitted to the peripheral bus, and the digital signal after the FxLMS algorithm noise reduction processing will be output, and then transmitted to the WM8731 audio codec module through the IIS peripheral interface circuit for DAC conversion, and the processed digital signal will be converted into an analog signal, and transmitted to the secondary sound source through Audio_data_tx to obtain the anti-phase noise, and finally the anti-phase noise obtained will interfere with the target noise, thereby performing active noise reduction of the audio and realizing the audio noise reduction system.
2. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 1 is characterized in that: The E203_SOC is composed of E203_CORE, NICE_CORE, ICB_APB, UART serial port, IIC interface and IIS interface; wherein, E203_CORE is a processor core for running instructions and executing instructions in sequence, and controlling corresponding modules to process data information according to the content of the instructions, and the general register Reg of the execution unit EXU in E203_CORE saves the processed data returned by the coprocessor, and the data tight coupler DTCM in the memory access control unit LSU in E203_CORE provides data corresponding to the source register index to the coprocessor, so that the custom instruction can read and write data, and NICE_Interface is used for communication and transmission between E203_CORE and NICE_CORE, and NICE_CORE is used to accelerate the processing of data transmitted from the processor core, obtain custom instructions through NICE_Interface, and then find the corresponding custom instructions through decoding Decode, the instructions are Lbuf, Sbuf, Conv, Mac, and then perform data access according to the function of the corresponding instruction. The Lbuf instruction is used to load data, obtain data from the DTCM in the LSU and store it in Buf. The Sbuf instruction is used to store data, and the data in Buf is returned to the DTCM in the LSU. The Conv instruction obtains the weight coefficient and the input signal from Buf for convolution operation, and writes the result back to Reg in EXU. The Mac instruction obtains the weight coefficient and the error component from Buf for multiplication and accumulation operation, and temporarily stores the result in Buf. After the operation is completed, the entire result is written back to Reg in EXU. The whole process is controlled by a finite state machine FSM for state transfer. ICB_APB is used for communication and transmission between the processor core and various peripheral interfaces. The UART serial port is used to transmit ADCDATA and DACDATA to the host computer for storage, which is convenient for data analysis using MATLAB. The IIC interface is used by the processor core to configure the registers of the audio codec WM8731 module, and the IIS interface is used for data transmission between the processor core and the audio codec WM8731 module.
3. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 1 or 2, characterized in that: The NICE_CORE decodes the signals including the custom instruction and source operand register address transmitted by E203_CORE through NICE_Interface, decodes the corresponding custom instruction by comparing the specific format part, and then executes the corresponding custom instruction under the control of the state machine.
4. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 1 or 2, characterized in that: The audio codec module WM8731 is internally provided with a built-in ADC, a DAC and a digital filter. Through the MIC transmission acquisition mode, the external sound source to be processed is collected, and the analog signal is converted into a digital signal through its built-in ADC, and then the obtained digital signal is filtered through the built-in digital filter to obtain the output data ADCDATA, and then the data DACDATA processed by the RISC v soft core is input into the DAC in the audio codec module to convert the digital signal into an analog signal, and the converted analog signal is output through the MIC.
5. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 4 is characterized in that: The audio codec module WM8731 is internally provided with an IIC interface circuit, which can perform operations including configuration of master-slave mode, selection of ADC and DAC enable, volume size in MIC and LINE modes, and whether to turn on digital filtering. Since the principle of the ANC audio noise reduction system is to use the destructive interference of sound waves, the WM8731 module will be configured as slave mode, MIC mode, ADC, and DAC enable, and the register configuration information will be configured by E203_CORE through the IIC interface bus.
6. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 5, characterized in that: The audio noise reduction FxLMS algorithm is written in C language as the description language and inline assembly language as the auxiliary language. It is compiled and assembled through IDE software to link and download the executable file to the RISC v soft core, and the FxLMS audio noise reduction algorithm is implemented in software through the execution of instructions.
7. The audio noise reduction accelerator system based on RISC v custom instruction set expansion according to claim 6, characterized in that: In the software implementation of the audio noise reduction FxLMS algorithm, the digital signal obtained after the noise reduction of the digital signal to be processed is subjected to noise reduction processing using the following formula: y(n)=y(n)+w(n)(k-2)x(n) e(n)=d(n)-y s (n) d(n)=p(n)*x(n) y s (n)=s(n)*y(n) Where y(n): secondary sound source, w(n)(k): weight coefficient, x(n): sound source signal, e(n): error signal, d(n): residual noise, y s (n): secondary sound source through the secondary path, p(n): reference signal from the sound source signal to the error microphone, s(n): signal generated by the secondary sound source to the error microphone secondary path, The estimated value of the sound source signal, where the constraint on the step factor is: Where μ: step size of FxLMS algorithm; λ max : The maximum eigenvalue of the autocorrelation matrix.
8. An audio noise reduction method based on the system according to any one of claims 1 to 7, characterized in that: The following steps are involved: First, the executable file generated after the C language program is compiled, assembled and linked is saved in the off-chip Flash memory. After the system starts, E203_CORE is initialized first, and then the IIC interface circuit on the peripheral bus is accessed according to the instruction sequence. According to the instruction, the audio codec module WM8731 is first configured through the IIC interface circuit; Secondly, the reference microphone collects the sound source signal after the sound signal and the target noise are superimposed, and the error microphone collects the residual noise. The collected sound source signal and residual noise are passed through the Audio_data_rx of the WM8731 module, and the built-in ADC conversion module of the WM8731 module converts the analog signal into a digital signal. After that, the built-in digital filter module of the WM8731 module performs filtering processing, and the filtered digital signal is transmitted to the peripheral bus through the IIS interface circuit; Next, according to the instruction, the filtered digital signal is obtained from the peripheral bus and transmitted to E203_CORE for processing according to the software implementation process of the FxLMS algorithm. When processing the multiplication and accumulation operation and the convolution operation, the decoding part in E203_CORE will detect it as a custom instruction. E203_CORE will pass the custom instruction and source operand to NICE_CORE through NICE_Interface to accelerate the specific operation part. The multiplication and accumulation operation will call the Mac instruction circuit, and the convolution operation will call the Conv instruction circuit. Then, according to the instruction execution, the processed digital signal is transmitted back to the peripheral bus, the digital signal processed by the noise reduction FxLMS algorithm is output, and then transmitted to the WM8731 audio codec module through the IIS peripheral interface circuit for DAC conversion, the processed digital signal is converted into an analog signal, and transmitted to the secondary sound source through the audio data sending end to obtain the anti-phase noise; Finally, the anti-phase noise and the target noise interfere with each other, thereby performing active audio noise reduction and realizing an audio noise reduction system.
9. The audio noise reduction method according to claim 8, characterized in that: There are a total of 4 custom instructions involved, namely LBUF, SBUF, CONV and MAC. Since LBUF and SBUF operate on the BUF at the same address, they need to be distinguished by their opcode fields, which are determined by custom0 and custom1, i.e. 7`b0001011 and 7`b0101011, respectively. Since SBUF and LBUF instructions only need to read and write from DTCM and do not need to write data back, they only need to use their source operand 1, i.e. rs1, so the funct3 field is defined as 3`b 010 format, represents that only the source operand register rs1 is operated. Since rd and rs2 are not used, this part can be defined as the length selection of the data to be read or written. The funct7 field is defined as the starting address of the load storage in the BUF. The CONV instruction only needs the target register to return the data, so it is necessary to operate the target register returned, that is, rd. The funct3\funct7\rs1\rs2 and other parts are combined and defined as the data X[9:0] and weight coefficient W[9:0] processed by it, which are all read by the pseudo dual-port SRAM and passed to the CONV circuit part for calculation. The MAC instruction needs to call the target register and the source operand for its calculation, so its funct3 field is defined as 3`b111 format, and the funct7 field has no definition requirements. The opcodes of the CONV and MAC instructions are defined as any of the four custom ones. The present invention uses custom3 and custom4, that is, 7`b1011011 and 7`b1111011.
10. The audio noise reduction method according to claim 8, characterized in that: The LBUF instruction can load the data information to be processed, and then store it in the two pseudo-dual-port SRAMs XBUF and QBUF. WBUF defines an empty pseudo-dual-port SRAM in the form of initial value, and then reads the data from XBUF and WBUF through the CONV instruction. This instruction is a convolution operation unit circuit using an addition tree structure. Its data length is determined by the filter order. That is, when the filter order is J, its signal to be processed X[J-1:0] and weight coefficient W[J-1:0] are obtained and stored in YBU after the operation result is obtained. F is used for FIFO storage, and its width and depth are defined as the data width and the order length of the filter; afterwards, the product of QBUF and the error coefficient is multiplied by the MAC instruction and then added to the weight coefficient WBUF in the previous column to obtain a new weight coefficient WBUF, and the new weight coefficient is written to it through the MAC custom instruction to update the weight, wherein the product part of the error coefficient is obtained by subtracting the primary noise signal Dn from the data stored in YBUF and then multiplying it by the step size factor, and then the above steps are repeated until all outputs fill the depth of the FIFO.