Synchronous multi-channel loopback within embedded architecture

Abstract

In at least one embodiment, an embedded Linux system is provided. The Linux system includes a memory, a system on chip (SoC) device, and a first circuit. The SoC device includes the memory and is programmed to at least process a reference signal indicating undesired audio content and a measurement signal indicating measured audio data in a listening environment. The first circuit is programmed to receive the reference signal and the measurement signal. The first circuit is also programmed to merge the reference signal with the measurement signal to provide a combined system input to the SoC device, thereby preventing a time misalignment caused by one or more software layers of the Linux system from occurring between the reference signal and the measurement signal.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 799,338, filed January 31, 2019, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] Various aspects disclosed herein generally relate to synchronous multi-channel loopback with embedded architecture. These and other aspects will be discussed in more detail below. Background Art

[0004] Typically, processing in embedded systems (e.g., Linux operating systems) is not real-time because the buffers applied to the data system are not constant. Such buffers may vary over time due to memory usage and system load. This behavior makes those popular and common system-on-chip (SoC) widely used in mobile phones almost unusable for advanced algorithms, such as acoustic echo cancellation (AEC) algorithms. Summary of the invention

[0005] In at least one embodiment, an embedded Linux system is provided. The Linux system includes a memory, a system on chip (SoC) device, and a first circuit. The SoC device includes the memory and is programmed to at least process a reference signal indicating undesired audio content and a measurement signal indicating measured audio data in a listening environment. The first circuit is programmed to receive the reference signal and the measurement signal. The first circuit is also programmed to merge the reference signal with the measurement signal to provide a combined system input to the SoC device, thereby preventing a time misalignment caused by one or more software layers of the Linux system from occurring between the reference signal and the measurement signal.

[0006] In at least another embodiment, a computer program product embodied in a non-transitory computer-readable medium is provided, the computer program product being programmed to prevent time misalignment between a reference signal and a measurement signal of an embedded Linux system. The computer program product includes instructions for: receiving the reference signal indicating undesired audio content; and receiving the measurement signal indicating measured audio data in a listening environment. The computer program product includes instructions for: processing the reference signal and the measurement signal at a system-on-chip (SoC) device; and merging the reference signal with the measurement signal to provide a combined system data stream to the SoC device, thereby preventing time misalignment between the reference signal and the measurement signal caused by one or more software layers of the Linux system.

[0007] In at least another embodiment, a computer program product embodied in a non-transitory computer readable medium is provided, the computer program product being programmed to prevent time misalignment between a reference signal and a measurement signal of an embedded Linux system. The computer program product includes instructions for receiving the reference signal indicative of output data of an adaptive control system. The computer program product includes instructions for: processing the reference signal and the measurement signal at a system on chip (SoC) device; and merging the reference signal with the measurement signal to provide a combined system data stream to the SoC device, thereby preventing time misalignment between the reference signal and the measurement signal caused by one or more software layers of the Linux system. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The embodiments of the present disclosure are particularly pointed out in the appended claims. However, other features of the various embodiments will become more apparent and best understood by referring to the following detailed description in conjunction with the accompanying drawings:

[0009] Figure 1 An example of an embedded adaptive control system is depicted;

[0010] Figure 2 depicts a high-level implementation of a closed-loop control implementation according to one embodiment;

[0011] Figure 3 depicts an embedded adaptive control system according to one embodiment;

[0012] Figure 4 depicts an example of a detailed implementation of an adaptive control system for use in conjunction with an acoustic echo canceller / cancellation according to one embodiment; and

[0013] Figure 5 An example of a detailed implementation of an adaptive control system as used in conjunction with active noise cancellation according to one embodiment is depicted. DETAILED DESCRIPTION

[0014] As required, detailed embodiments of the present invention are disclosed herein; however, it should be understood that the disclosed embodiments are merely examples of the present invention, which may be embodied in various alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to use the present invention in various ways.

[0015] It should be recognized that various electrical devices such as servers, controllers, and clients disclosed herein may include various microprocessors, integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other suitable variants thereof) and software that cooperate with each other to perform the operations disclosed herein. In addition, these electrical devices utilize one or more microprocessors to execute a computer program embodied in a non-temporary computer-readable medium, the computer program being programmed to perform any number of functions disclosed. In addition, various electronic devices as provided herein include a housing and various numbers of microprocessors, integrated circuits, and memory devices (e.g., FLASH, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) located within the housing. The electrical device also includes hardware-based inputs and outputs for receiving data from and transmitting data to other hardware-based devices as disclosed herein, respectively.

[0016] The embodiments disclosed herein generally provide, among other things, a synchronized reference and measurement embedded input signal architecture that can be used in conjunction with, for example, a LINUX operating system that enables many advanced algorithms. The architecture can be reliable and can be implemented for any number of adaptive control strategies. The Linux operating system can be used for any number of audio processing devices. For example, the architecture can utilize synchronized reference and measurement signals in conjunction with, but not limited to, for example, acoustic echo cancellers or cancellation (AEC) applications, active noise cancellation (ANC) systems, any other suitable audio-based applications, or generally any control application that requires relatively synchronized reference and measurement signals. The depicted enhancement (or architecture) can utilize a signal conditioning circuit that can be simple and obtain a low-cost method that provides a relatively synchronized design between a reference signal and a measurement signal, while the end-to-end delay can be dynamically changed. It should be appreciated that the embodiments as described herein can be applied to any system that does not include an audio system, whereby it is desired to achieve a control goal by, for example, minimizing errors using relatively synchronized reference and measurement signals.

[0017] Figure 1An example of an embedded adaptive control system 100 is depicted. The adaptive control system 100 may be implemented, for example, with an embedded Linux operating system (OS) 101. It should be appreciated that the system 100 may include a Linux kernel (e.g., the core of the OS) and associated supporting tools and libraries. The system 100 generally includes an application layer 102, a server (or server layer) 104, a sound subsystem layer 106 or, for example, an Advanced Linux Sound Architecture (ALSA) library layer 106 or an Open Sound System (OSS) library layer 106, a kernel (or driver) layer (e.g., an ALSA kernel / driver or an OSS kernel / driver) 108, and a hardware layer 110. The application layer 102, the server 104, the sound subsystem layer 106, the kernel layer 108, and the hardware layer 110 may form at least a system-on-chip (SoC) device 111. The SoC device 111 may be an integrated circuit in which all associated components are integrated. It should be appreciated that the SoC device 111 may include a central processing unit (CPU), a memory 113, input / output ports, and / or a secondary storage device that may be packaged on a single substrate or microchip. The SoC device 111 may also include a digital device, an analog device and / or a mixed signal processing device. The SoC device 111 may be part of a Linux embedded system.

[0018] It should be appreciated that the system 100 also includes at least one controller 112 (hereinafter referred to as controller 112) for performing any and all tasks performed by the application layer 102, the server layer 104, the sound subsystem layer 106, and the kernel (or driver) layer 108. In addition, the controller 112 can interface with the hardware layer 110 to process data as received from the hardware layer 110 or transmit data to the hardware layer 110.

[0019] It should be appreciated that the control system 100 may be used in any adaptive control implementation (or closed-loop control strategy). Figure 2A control implementation 200 employing a closed-loop strategy is depicted. The implementation 200 includes a signal conditioning circuit 201 (or first circuit 201), a controller 202, a system or plant 204 (hereinafter referred to as "plant 204") under the control of the controller 202, and a sensor 206. The signal conditioning circuit 201 may include: a mixer 203 for separating the signal "reference" from the system / plant input signal; and a pack circuit 209 for combining the signal "reference" with the signal "measurement" to provide the signal "controller_input". It should be recognized that all references to any signal herein may be singular or plural (e.g., each corresponding input and output has any number of signals). The signal "reference" may be provided to the signal conditioning circuit 201 via the mixer 203 and may be separated by the system / plant input signal or a separate input of the signal conditioning circuit 201. The signal "reference" may correspond directly to, for example, a combination of Figure 1The audio output data previously output from the hardware layer 110 shown. This can also be referred to as a loopback mechanism. The signal "reference" can include data provided on any number (M) of audio channels. The signal "reference" can correspond to audio data determined to be undesirable by the user. The signal "measurement" can also be provided to the signal conditioning circuit 201. The signal "measurement" can, for example, correspond to audio data transmitted from a sensor 206 (e.g., a microphone). The sensor 206 generates a signal "measurement", which corresponds to the output measured from the device 204, also defined as a signal "factory_output" from the factory 204. In this case, the signal "measurement" can correspond to the audio on the signal "factory_input" or "controller_output". The signal "measurement" can include the actual audio heard by the user in the listening environment. The signal "measurement" can include any number (N) of audio channels. The controller 202 receives the signal "controller_input", which is a synchronous combination of the signal "reference" and the signal "measurement" from, for example, a sensor 206 (or a microphone). It should be appreciated that the analog-to-digital converter can be located between the sensor 206 and the signal conditioning circuit 201. In general, the grouping circuit 209 can merge data between the signal measurement "measurement" and the signal "reference". In one example, the merging operation can correspond to the grouping circuit 209 adding or subtracting the signal "measurement" relative to the signal "reference" to provide the combined signal "controller_input" to the controller 202. The signal "controller_input" can be in the form of M channels because the signal "reference" is superimposed on the signal "measurement". Alternatively, the merging operation can correspond to the grouping circuit 209 combining (or grouping) the signal "measurement" with the signal "reference" to provide the signal "controller_input". However, due to the grouping operation, the signal "controller_input" can be in the form of M+N channels because the signals "reference" and "measurement" are grouped into respective time slots of the signal "controller_input". For example, the signal "controller_input" can form M+N audio-based channels that provide a time division multiplexed (TDM) data stream and grouped with M audio-based channels from the signal "measurement" and N audio-based channels from the signal "reference". Generally speaking, the signal conditioning circuit 201 is configured to merge the data on the signal "reference" with the data on the signal "measurement" to provide the combined system data stream to the SoC device 111, thereby preventing time misalignment between the data on the signal "reference" and the data on the signal "measurement". Such time misalignment may be caused by one or more of the layers 102, 104, 106 and 108 during the operation of the control implementation 200. This aspect will be discussed in more detail below.

[0020] As noted above, the embodiments of the present invention can be applied to any system that does not include an audio system, thereby expecting to reduce the time misalignment between two signals propagated through one or more software layers of a Linux system. In an adaptive control system, the signal "reference" may correspond only to input data to a plant that utilizes a loopback mechanism. The signal "measurement" may correspond to data such as provided by a sensor in an adaptive control system, where the sensor provides some form of feedback information. Therefore, in this case, the signal conditioning circuit 201 can merge the data on the signal "reference" with the data on the signal "measurement". After merging the data between these signals, the signal conditioning circuit 201 provides the combined system data stream to the SoC device 111 to prevent time misalignment between the data on the signal "reference" and the data on the "measurement". Time misalignment can generally be caused by one or more of the software layers 102, 104, 106, and 108. This aspect can, for example, achieve control objectives by minimizing errors using relatively synchronized signals "reference and measurement signals". It should be appreciated that implementations as set forth herein may not only be used for audio-related purposes, but may be used by any system attempting to resolve or reduce time misalignment between two signals propagating through the software layers of a Linux system.

[0021] The controller 202 employs an adaptive control strategy to repeatedly perform this process based on the signal "measurement" and the signal "reference" so that the control implementation 200 adapts to the plant 204 in a desired manner. However, it should be recognized that due to inherent system errors or inconsistencies, the controller 202 may not be able to fully control the plant 204 to achieve the desired results. The adaptive control implementation 200 noted above can be executed any number of times to provide control actions in an optimal manner, which may require at least relatively stable delays and timings between the signals "reference" and "measurement" to ensure control stability and optimal performance of the plant 204 under the control of the controller 202. Any dynamic time / delay misalignment issues between the signal "measurement" and the signal "reference" (such as, for example, distributed in, for example, a LINUX system 101, such as Figure 1 The examples shown) may cause the controller 202 (or Figure 1 As shown, the controller 202 may degrade performance or even cause instability. However, it should be appreciated that due to inherent system (or plant) errors or inconsistencies, the controller 202 may not be fully capable of controlling the plant 204 to achieve the desired results.

[0022] Reference again Figure 1, application layer 102 generally includes, for example, audio clients and related user programs that enable music playback. These audio clients can be connected to sound servers via audio streams. Server layer 104 (or sound server) can be software, and manages the use and access of audio devices (e.g., sound cards). Server layer 104 can usually run as a background process for sink and source processing. Server layer 104 can be implemented as PulseAudio or an audio connection toolbox such as JACK. Generally speaking, PulseAudio is a sound system for various operating systems. PulseAudio is generally used as an agent for sound applications. If server layer 104 is implemented for PulseAudio, it should be recognized that PulseAudio can be a sound server used in UBUNTU distribution, which is connected to a lower software layer, such as kernel layer 108 (or ASLA layer). PulseAudio provides server / client connections within application layer 102. JACK can be included in any number of LINUX applications.

[0023] The sound system layer 106 generally includes software for implementing based on whether ALSA or OSS is used to provide libraries for ALSA and OSS. Kernel or driver layer 108 can include sound driver and sound device. If kernel layer 108 is used in combination with ALSA, then ALSA kernel layer 108 can support multi-channel audio interface and provide application program interface (API) access on management hardware control. ALSA may be a standard software layer in the embedded system based on multiprocessor LINUX, for managing all audio streams and stream mixing, including buffer processing between audio hardware peripherals and upper application layer. If kernel layer 108 is used in combination with OSS, then OSS kernel layer 108 includes LINUX kernel sound driver and sound device and application layer 102 is connected with real audio hardware peripherals.

[0024] The system 100 also includes a hardware-based input device 120 that provides input signals to the hardware layer 110. The input device 120 may include any number of sensors, such as microphones, accelerometers, etc. The system 100 also includes a hardware-based output device 122 that receives output signals from the hardware layer 110. In one example, the hardware-based output device 122 may include at least one controller for an audio system, such as acoustic echo cancellation (AEC), an active noise cancellation system (ANC), etc. In general, as the system 100 receives input signals from the input device 120 and the input signals propagate through the hardware layer 110, the kernel layer 108, the sound subsystem layer 106, and the server layer 104, the processing performed by the controller 112 to perform such layers 102, 104, 106 and 108, the sound subsystem layer 106, and the server layer 104 may cause latency issues with respect to the different and separate input streams of data processed by the system 100.

[0025] The application layer 102 generally includes a controller unit 124 that can utilize the signals "reference" and "measurement" independently of each other. For example, the signals "reference" and "measurement" each include independent and separate data from each other (e.g., the data between each signal is grouped or added together (i.e., merged together)). For example, the inputs of the controller unit 124 generally correspond to the signals "measurement" as provided by the sensor 206 and the signals "measurement" as provided by the sensor 206. Figure 2 However, in this case, the system 100 does not use the signal "reference" signal shown. Figure 2 2. In this case, the system 100 buffers the signals "reference" and "measurement" as separate data streams. Therefore, the signals are conditioned at the various layers 102, 104, 106, and 108. Figure 1 Separate buffering of the signals "measurement" and "reference" in the MCU (i.e., dynamic buffer size modification) may cause these separate signals to be misaligned or out of sync with each other. Although layer 110 may be hardware-based, the layer 110 may not exacerbate the delay problem between the signals "reference" and "measurement".

[0026] However, it is noted that layers 102 or 104, 106, 108 and 110 process the data stream on the signal "controller output" or "factory input" in a downstream manner, and this signal may be fully or partially looped back for upstream processing as the signal "reference". In addition, it is noted that layers 102, 104, 106 and 108 process the signals "measurement" and "reference" in an upstream manner. In view of this, the signal "reference" may be based on the signal "controller output" or "factory input", which may exacerbate the misalignment or asynchrony as described above. Generally speaking, the hardware layer 110 may include programmable sub-units and is generally defined as hardware in a LINUX system. The delays mentioned above can make the hardware layer 110 (although programmable) deterministic and static. The kernel layer 108 and all layers above layer 106 may be software-based layers, and the delays attributed to such layers 102, 104, 106, 108 may be considered dynamic rather than static (or deterministic due to operating system (OS) settings within such layers 102, 104, 106, 108).

[0027] Generally speaking, the data provided by the hardware-based input device 120 can be digital. The hardware-based input device 120 can include any number of analog-to-digital converters to provide digital data on the signal "measurement" and within certain controller applications, just as, for example, the ANC will also provide a signal "reference", because here the signal "reference" will not deviate from the signal "controller output" or "factory input", but still requires a signal conditioner circuit 201 to add or group the signal "reference" and "measurement" together into a combined signal "controller input" and forward the signal to the upstream layers 110, 108, 106 and 104 for processing (see, for example, Figure 5 ). Likewise, the data transmitted from the hardware-based output device 122 may be digital. The hardware-based output device 122 may include any number of digital-to-analog converters (DACs) for converting the digital data back into analog data.

[0028] Figure 3 An embedded adaptive control system 100' is depicted according to one embodiment. It should be appreciated that the system 100' can be implemented in a mobile device (not shown), such as a cellular phone (or any other device capable of cellular communication), a laptop computer, a tablet computer, etc. The system 100' can be implemented in an audio processing device of the mobile device. The system 100 generally includes a signal conditioning circuit 201, which is located intermediate the hardware layer 110 and the hardware-based input device 120 and the hardware-based output device 122. As described above, the input device 120 can correspond to a microphone or an accelerometer that provides measured audio data from the listening environment (e.g., see Figure 2) Also as described above, the hardware-based output device 122 may include a power amplifier and any number of D / A converters for converting digital data on the signal "controller output" or "factory input" into analog data and providing it to speakers in the room or listening environment. As noted above, the signal "reference" may correspond in whole or in part to the signal "controller output" or "factory input". In addition, the hardware-based output device 122 may also include at least one controller for an audio system, such as an acoustic echo cancellation (AEC) system, an active noise cancellation (ANC) system, or the like. The signal conditioning circuit 201 may be a field programmable gate array (FGPA), an application specific integrated circuit (ASIC), a digital signal processor (DSP), or the like. Figure 2 The signal conditioning circuit 201 is similar to that of Figure 3 The signal conditioning circuit 201 shown may also include a mixer 203 and a grouping circuit 209 for combining the signals "reference" and "measurement" to provide a combined "controller input" data stream. Figure 3 As shown, the combined system data stream may be referred to as the signal "combined reference and measurement signal". As noted above, the signal "reference" may be fully utilized, or may be partially deviated from (or derived from) the signal "controller_output" or "plant_input", but in Figure 2 Not shown in FIG.

[0029] As noted above, layers 102, 104, 106, and 108 may be software-based layers and may be considered non-deterministic (or time-varying). Thus, when executed by controller 112, such layers 102, 104, 106, and 108 may generally result in misalignment between signals "reference" and "measurement" due to dynamic buffer size modifications and / or processing delays. However, given that signal conditioning circuitry 201 (e.g., mixer 203 and / or grouping circuitry 209) is hardware-based and time-deterministic, signal conditioning circuitry 201 is generally applied and positioned before all of the non-time-deterministic (or time-varying) layers 102, 104, 106, and 108, which may mitigate any misalignment between signals "reference" and "measurement" because both signals are combined with one another via signal conditioning circuitry 201 to provide signal "combined reference and measurement," referred to as signal "controller_input."

[0030] For example, typical dynamic buffer size modifications between layers 102, 104, 106, and 108 may change the overall end-to-end delay. However, signal conditioning circuit 201 may synchronize signals "reference and measurement" because signal conditioning circuit 201 is positioned beyond potential dynamic buffer delay modifications associated with SW layer processing, and the delay of signal "combined reference and measurement" may be considered constant. For example, any delay applied to signal "combined reference and measurement" may affect signals "reference" and "measurement" in the same manner, and thus the relative delays may be considered constant. When present in signal "combined reference and measurement", signals "reference" and "measurement" may correspond to, for example, Figure 2 The signals "reference" and "measurement" are shown. Figure 2 The same applies to the signal “combined reference and measurement” of the signal “controller_input” shown. The signal conditioning circuit 201 (via a grouping circuit or mixer) merges or combines the signals “reference” and “measurement” with each other to provide a combined system input (or a combined system data stream) to the SoC device 111, thereby preventing any time misalignment within one or more of the non-deterministic software layers 102, 104, 106 and 108 during device operation.

[0031] Figure 4 An example of an adaptive control system 100" for use in conjunction with an AEC system according to one embodiment is depicted. The server layer 104 (or "PulseAudio daemon") generally includes a sink layer 300 and a source layer 302. The adaptive control system 100" generally illustrates the case of independent sink and source processing embedded in a typical PulseAudio daemon framework. The PulseAudio I / O buffers of the server layer 104 cannot be assumed to be synchronous.

[0032] The hardware-based input device 120 may provide audio data on a signal "measurement" as captured from a microphone or accelerometer (not shown) used in conjunction with the AEC system. The captured audio data may correspond to desired speech data, but include undesired echo data captured in the room or listening environment (e.g., data on signal "measurement"). The signal conditioning circuit 201 may forward the audio data from the signal "system_input" and provide it to the hardware-based output device 122 for broadcast to an acoustic path, which represents the acoustic path as shown. Figure 2plant 204 shown. As described above, the signal "reference" can correspond to the signal "system_input". In the case of AEC, the audio data on the signal "system_input" can correspond to the following music data, which is played back independently of the controller operation but once forwarded to the system or plant during playback, it is considered an undesirable echo. The signal conditioning circuit 201 can fully or partially mix the signal "system_input" as the signal "reference" and combine it with the signal "measurement" to allow echo cancellation to occur within the controller. It should be appreciated that the hardware-based output device 122 may include a power amplifier and any number of D / A converters for converting the digital data on the signal "system_input" into analog data and providing it to speakers in the room or listening environment.

[0033] The signal conditioning circuit 201 may group or add the data on the signal REFERENCE with the measured microphone data (i.e., desired speech with undesired echo) on the signal MEASUREMENT in real time to provide a signal COMBINED REFERENCE AND MEASUREMENT or signal CONTROLLER_INPUT. The signal COMBINED REFERENCE AND MEASUREMENT or signal CONTROLLER_INPUT may be considered robust to any dynamic system behavior because the delay between the microphone data and the reference data may remain relatively constant on the signal COMBINED REFERENCE AND MEASUREMENT or signal CONTROLLER_INPUT. Although the input / output (I / O) end-to-end delay may vary dynamically, AEC convergence may be guaranteed due to the stable / constant relative delay between the microphone data and the reference data on the signal COMBINED REFERENCE AND MEASUREMENT.

[0034] Figure 5 An example of an adaptive control system 100'' is depicted for use in conjunction with an ANC system according to one embodiment. The server layer 104 (or "PulseAudio daemon") generally includes a sink layer 300 and a source layer 302. The adaptive control system 100'' generally illustrates the case of independent sink and source processing embedded in a typical PulseAudio daemon framework. The PulseAudio I / O buffers of the server layer 104 cannot be assumed to be synchronous.

[0035] The hardware-based input device 120 may provide audio data on a signal "measurement" as captured from a microphone or accelerometer (not shown) used in conjunction with the ANC system. The captured audio data may correspond to undesirable noise data as well as desired music or speech data (e.g., data on the signal "measurement") captured in a room or listening environment. The signal conditioning circuit 201 may forward the audio data from the signal "controller_output" and provide it to the hardware-based output device 122 to be broadcast to the acoustic path for noise cancellation, such as Figure 2As mentioned above, the signal "reference" may not correspond to the signal "controller_output", but may only correspond to the undesired noise provided by another sensor that only provides undesired noise, in Figure 2 1. In the case of ANC, the audio data on the signal "CONTROLLER_OUTPUT" may correspond to anti-noise data that is looped back to the audio source via the controller operation within the application layer 102 for playback via the plant 204 for anti-noise signal to cancel noise. The signal conditioning circuit 201 merges the signal "REFERENCE" with the signal "MEASUREMENT" to allow the controller to cancel noise via the plant 204. It should be appreciated that the hardware-based output device 122 may include a power amplifier and any number of D / A converters for converting the digital data on the signal "CONTROLLER_OUTPUT" into analog data and providing it to speakers in the room or listening environment.

[0036] The signal conditioning circuit 201 can group or add (i.e., merge the data) data on the signal "reference" with the measured microphone data (i.e., desired music / speech with undesired noise) on the signal "measurement" in real time to provide a signal "combined reference and measurement" or signal "controller_input". The signal "combined reference and measurement" or signal "controller_input" can be considered robust to any dynamic system behavior because the delay between the microphone data and the reference data can remain relatively constant on the signal "combined reference and measurement" or signal "controller_input". Although the input / output (I / O) end-to-end delay may change dynamically, ANC convergence can be guaranteed due to the stable / constant relative delay on the microphone data and the signal "combined reference and measurement".

[0037] The data streams corresponding to the signals "measurement", "reference", "combined reference and measurement", "controller_input", "factory_input", and "controller_output" may be implemented as time division multiplexed (TDM) data streams or as internal linear or circular buffers between software layers 102, 104, 106. Figure 4In the example described, the TDM data stream is partially populated with input data (eg, audio data from the microphone under test), output data (eg, data from signal "controller_output"), and data from signal "reference". In the example described in conjunction with system 100", a TDM-based data stream may be grouped with 1 to N+M channels (or data bits), where M corresponds to audio data from the microphone under test (or from a hardware-based input device 120) and N corresponds to reference data. In the case of AEC, the microphone content may contain desired speech and undesired echo after the AEC controller processes the echo (ideally completely canceled) and the speech is considered a "sink" for further application services. Generally speaking, music may be used as a "source" (device transceiver music). Speech may be used as a "sink" because the device receives speech for a communication proposal (e.g., a phone call). In the case of ANC, the microphone content may contain desired music / speech and undesired noise after the ANC controller processes the noise (ideally completely canceled) and the anti-noise signal is considered a "sink" for audio "source" applications, so the controller anti-noise signal may be looped back to the device output and perform noise cancellation within the acoustic path.

[0038] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the present invention. On the contrary, the words used in the specification are descriptive rather than restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present invention. In addition, the features of the various embodiments can be combined to form other embodiments of the present invention.

Claims

1. An embedded Linux system for an audio processing device, the system comprising: Memory; a system-on-chip (SoC) device comprising the memory and programmed to process at least a reference signal indicative of undesired audio content and a measurement signal indicative of measured audio data in a listening environment; as well as A first circuit, wherein the first circuit is programmed to: receiving the reference signal; receiving a measurement signal; as well as The reference signal is combined with the measurement signal to provide the combined system input to the hardware layer of the system-on-chip (SoC) device before the combined system input is received at the multiple software layers of the Linux system, thereby preventing time misalignment between the reference signal and the measurement signal caused by the multiple software layers of the Linux system.

2. The system of claim 1, further comprising a mixer programmed to transmit the reference signal to the first circuit.

3. The system of claim 1, further comprising an input sensor programmed to transmit the reference signal to the first circuit.

4. The system of claim 1 further comprising an input sensor programmed to transmit the measurement signal to the first circuit.

5. The system of claim 4, wherein the input sensor is one of a microphone or an acceleration sensor.

6. The system of claim 1, wherein the first circuit is further programmed to combine the reference signal with the measurement signal and then transmit the combined system input to the one or more software layers of the Linux system.

7. The system of claim 1, wherein: A first software layer of the one or more software layers includes at least one audio-based kernel, The at least one audio-based kernel includes a sound driver; A second software layer of the one or more software layers comprises an audio-based library; and A third software layer of the one or more software layers includes a sound server to manage a sound card.

8. The system of claim 1, wherein the first circuit is further programmed to receive the reference signal as a plurality of N audio-based channels.

9. The system of claim 8, wherein the first circuit is further programmed to receive the measurement signal as a plurality M of audio-based channels.

10. The system of claim 9, wherein the first circuit is further programmed to combine the reference signal with the measurement signal to generate M+N audio-based channels to form a time division multiplexed (TDM) data stream.

11. A non-transitory computer readable medium comprising a computer program product programmed to and executed by one or more controllers to prevent time misalignment between a reference signal and a measurement signal in an embedded Linux system, the computer program product comprising instructions for: receiving in a first circuit the reference signal indicative of undesirable audio content; receiving said measurement signal indicative of measured audio data in a listening environment; processing the reference signal and the measurement signal in a system on a system-on-chip device; as well as The reference signal is combined with the measurement signal to provide the combined system input to the hardware layer of the system-on-chip (SoC) device before the combined system input is received at the multiple software layers of the Linux system, thereby preventing time misalignment between the reference signal and the measurement signal caused by the multiple software layers of the Linux system.

12. The non-transitory computer-readable medium comprising a computer program product of claim 11, further comprising instructions for transmitting the reference signal to the first circuit via a mixer.

13. The non-transitory computer readable medium comprising a computer program product of claim 11, further comprising instructions for transmitting the reference signal to the first circuit via an input sensor.

14. The non-transitory computer readable medium comprising a computer program product of claim 11, further comprising instructions for transmitting the measurement signal to the first circuit.

15. The non-transitory computer readable medium comprising a computer program product of claim 11, further comprising instructions for combining the reference signal with the measurement signal and then transmitting the combined system input to the plurality of software layers of the Linux system.

16. The non-transitory computer readable medium comprising a computer program product of claim 11, wherein A first software layer of the plurality of software layers includes at least one audio-based kernel, The at least one audio-based kernel includes a sound driver; A second software layer of the plurality of software layers comprises an audio-based library; and A third software layer among the plurality of software layers includes a sound server to manage a sound card.

17. The non-transitory computer-readable medium comprising a computer program product of claim 11, further comprising instructions for receiving the reference signal as a plurality of N audio-based channels.

18. The non-transitory computer-readable medium comprising a computer program product of claim 17, further comprising instructions for receiving the measurement signal as a plurality M of audio-based channels.

19. The non-transitory computer-readable medium comprising a computer program product of claim 18, further comprising instructions for grouping or adding / combining the reference signal with the measurement signal to generate M+N audio-based channels to form a time division multiplexed (TDM) data stream.

20. A computer program product embodied in a non-transitory computer readable medium, the computer program product being programmed and executed by one or more controllers to prevent time misalignment between a reference signal and a measurement signal in an embedded Linux system, the computer program product comprising instructions for: receiving said reference signal indicative of output data of an adaptive control system; receiving the measurement signal; processing the reference signal and the measurement signal in a system on a system-on-chip device; as well as The reference signal is combined with the measurement signal to provide the combined system input to the hardware layer of the system-on-chip (SoC) device before the combined system input is received at the multiple software layers of the Linux system, thereby preventing time misalignment between the reference signal and the measurement signal caused by the multiple software layers of the Linux system.

Images