Method and system for extracting features

Abstract

A device for extracting features from a time series signal, the device comprising: a signal generator configured for generating a periodic signal; a signal mixer connected to the signal generator and configured for mixing together the time series signal and the periodic signal to obtain a mixed signal; and a Chua circuit having a circuit frequency, the Chua circuit being connected to the signal mixer for receiving the mixed signal therefrom and configured for: generating a feature signal indicative of features of time series data contained in the time series signal when operated to have a limit cycle behavior; and outputting the feature signal, wherein a given frequency of the periodic signal substantially corresponds to the circuit frequency.

Description

METHOD AND SYSTEM FOR EXTRACTING FEATURESCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority on US Provisional Patent Application No. 63 / 513,135 filed on July 12, 2023, the content of which is hereby incorporated by reference.FIELD

[0002] The present technology pertains to the field of reservoir computing, and more particularly to methods and systems for extracting features from a time series signal.BACKGROUND

[0003] A physical reservoir computer provides a powerful approach to conduct machine learning for time series data processing. One of the biggest challenges for such technology is that the existing reservoirs do not usually possess sufficient non-linearities necessary for the complex machine learning tasks such as audio recognition.

[0004] A conventional approach to solve this problem is to use a time-delayed feedback to conduct time-multiplexing (i.e., to combine the previous results processed from physical reservoir with the new data) to feed more information to the reservoir for computation. However, the delayed feedback needs digital-to-analog conversion, which usually complicates the physical reservoir setup, adds power consumption and sacrifices the processing speed. Therefore, physical reservoir computing is usually built with optical setups and quantum circuits for higher processing which usually renders the physical reservoir computing cumbersome and economically unviable for the deployment to edge devices.

[0005] Additionally, when it is applied to audio, reservoir computing usually uses nonlinearity that is not associated with the oscillatory nature of audio. As such, almost all physical reservoir computers require the users to preprocess the audio data using a spectrogram before feeding the audio data to the physical reservoir. Such an approach usually requires the use of processing chips and a digital-to-analog convertor, which significantly reduces the processing speed and increases the computational power envelope. The benefits brought by the analog computing, such as high speed and low computational power, may be canceled out by the added components to the physical reservoirs. More powerful machine learning models have been employed to solve this problem. However, such machine learning models, such as convolutional neural networks (CNN), require specialized processing hardware (e.g., graphic processing unit or neural chip) or cloud to process signals. As such, the cost of deploying an audio recognition system will be inevitably high in addition to causing privacy concerns.

[0006] Therefore, there is a need for an improved method and system for extracting features in the context of reservoir computing.SUMMARY

[0007] In accordance with a first broad aspect, there is provided a device for extracting features from a time series signal, the device comprising: a signal generator configured for generating a periodic signal; a signal mixer connected to the signal generator and configured for mixing together the time series signal and the periodic signal to obtain a mixed signal; and a Chua circuit having a circuit frequency, the Chua circuit being connected to the signal mixer for receiving the mixed signal therefrom and configured for: generating a feature signal indicative of features of time series data contained in the time series signal when operated to have a limit cycle behavior; and outputting the feature signal, wherein a given frequency of the periodic signal substantially corresponds to the circuit frequency.

[0008] In some non-limiting embodiments, the signal generator comprises one of a Wien bridge circuit, a phase shift oscillator and a digital processor configured for direct digital synthesis.

[0009] In some non-limiting embodiments, the given frequency is equal to the circuit frequency.

[0001] In some non-limiting embodiments, the periodic signal comprises a square wave signal.[OOH] In some non-limiting embodiments, the signal generator comprises a timer integrated circuit.

[0012] In other non-limiting embodiments, the periodic signal comprises a triangular wave signal.

[0013] In some non-limiting embodiments, the signal generator comprises a Schmitt trigger and an Op-amp integrator circuit.

[0014] In some non-limiting embodiments, the device further comprises an amplifier for amplifying and offsetting the time series signal to obtain an amplified and offset time series signal, the amplifier being connected to the signal mixer for providing the amplified and offset time series signal thereto.

[0015] In some non-limiting embodiments, the Chua circuit comprises at least a nonlinear component, properties of the non-linear component being chosen so that the Chua circuit has the limit cycle behavior.

[0016] In some non-limiting embodiments, the non-linear component comprises a nonlinear resistor, a resistance of the non-linear resistor being chosen so that the Chua circuit has the limit cycle behavior.

[0017] In some non-limiting embodiments, the time series signal comprises an audio signal.

[0018] In accordance with a second broad aspect, there is provided a method for extracting features from a time series signal, the method comprising: generating a periodic signal; mixing together the time series signal and the periodic signal, thereby obtaining a mixed signal; operating a Chua circuit so that the Chua circuit has a limit cycle behavior and propagating the mixed signal into the Chua circuit, thereby obtaining a feature signal indicative of features of time series data contained in the time series signal, a given frequency of the periodic signal substantially corresponding to a circuit frequency of the Chua circuit; and outputting the feature signal.

[0019] In some non-limiting embodiments, the step of generating the periodic signal is performed using one of a Wien bridge circuit, a phase shift oscillator and a digital processor configured for direct digital synthesis.

[0020] In some non-limiting embodiments, the given frequency is equal to the circuit frequency.

[0021] In some non-limiting embodiments, the periodic signal comprises a square wave signal.

[0022] In some non-limiting embodiments, the step of generating the periodic signal is performed using a timer integrated circuit.

[0023] In other non-limiting embodiments, the periodic signal comprises a triangular wave signal.

[0024] In some non-limiting embodiments, the step of generating the periodic signal is performed using a Schmitt trigger and an Op-amp integrator circuit.

[0025] In some non-limiting embodiments, the method further comprises amplifying and offsetting the time series signal prior to said mixing.

[0026] In some non-limiting embodiments, the Chua circuit comprises a non-linear component, said operating the Chua circuit comprises setting properties of the non-linear component to a target property value chosen so that the Chua circuit has the limit cycle behavior.

[0027] In some non-limiting embodiments, the non-linear component comprises a nonlinear resistor, said operating the Chua circuit comprises setting a resistance of the nonlinear resistor to a target resistance value chosen so that the Chua circuit has the limit cycle behavior.

[0028] In some non-limiting embodiments, the time series signal comprises an audio signal.

[0029] Implementations of the present technology each have at least one of the above- mentioned objects and / or aspects, but do not necessarily have all of them. It should be understood that some aspects of the present technology that have resulted from attempting to attain the above-mentioned object may not satisfy this object and / or may satisfy other objects not specifically recited herein.

[0030] Additional and / or alternative features, aspects and advantages of implementations of the present technology will become apparent from the following description, the accompanying drawings and the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0031] For a better understanding of the present technology, as well as other aspects and further features thereof, reference is made to the following description which is to be used in conjunction with the accompanying drawings, where:

[0002] Fig. 1 is a block diagram illustrating a system for analysing an audio signal, in accordance with an embodiment;

[0033] Fig. 2 illustrates an exemplary Chua circuit that may be used in the system of Fig. 1;

[0034] Fig. 3 is an exemplary graph illustrating a limit cycle produced by the Chua circuit of Fig. 2, in which the inner circles represent the initial transient period of the oscillation;

[0035] Figs. 4a and 4b illustrate an exemplary comparison of the spectrum of the 1 kHz single tone after signal mixer and after the Chua circuit for the whole audible (20 Hz to 20 kHz range) (a) and the range with lower frequency (b) up to 2 kHz;

[0036] Figs. 5a and 5b illustrate an exemplary Fourier transform of the Chua circuit output from a single tone of 1,000 Hz playing at 67 dB and 106 dB, respectively;

[0037] Fig. 6 illustrates an exemplary preprocessing flow for signal generated the system of Fig. 10 using reservoir computing;

[0038] Fig. 7 illustrates exemplary audio features corresponding to high, mid, and low SNRs for gunshot audio pieces labeled in the white boxes, wherein a slight mismatching of different audio pieces is present due to the different starting instants of different experiments;

[0039] Figs. 8a and 8b illustrate an exemplary preprocessing operated on the signal after the signal mixer and operated on the signal after Chua circuit, respectively;

[0040] Fig. 9 illustrates an exemplary comparison of 10-class urban sound recognition accuracy using proposed hardware and signal preprocessing technology, where except for CNN, other machine learning approaches uses less than 200 KB of both dynamic and static memory and are capable of being deployed to the generic edge devices (e.g., STM32 microcontroller unit with 384 KB memory);

[0041] Fig. 10 illustrates an example increasing amplitude of activation signal (i.e., increasing Vpp) that suppresses the effects of random noise on the audio features generated from a Chua circuit and enhances the contrast of audio features when the input audio tested is the 1,000 Hz single tone with background noise; and

[0042] Fig. 11 is a flow chart illustrating a method for extracting features form an analog audio signal, in accordance with an embodiment.DETAILED DESCRIPTION

[0043] The examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the present technology and not to limit its scope to such specifically recited examples and conditions. It will be appreciated that those skilled in the art may devise various arrangements which, although not explicitly described or shown herein, nonetheless embody the principles of the present technology and are included within its spirit and scope.

[0044] Furthermore, as an aid to understanding, the following description may describe relatively simplified implementations of the present technology. As persons skilled in the art would understand, various implementations of the present technology may be of a greater complexity.

[0045] In some cases, what are believed to be helpful examples of modifications to the present technology may also be set forth. This is done merely as an aid to understanding, and, again, not to define the scope or set forth the bounds of the present technology. These modifications are not an exhaustive list, and a person skilled in the art may make other modifications while nonetheless remaining within the scope of the present technology. Further, where no examples of modifications have been set forth, it should not be interpreted that no modifications are possible and / or that what is described is the sole manner of implementing that element of the present technology.

[0046] Fig. 1 illustrates one embodiment of a system 10 for analysing an analog audio signal such as identifying audio events within the analog audio signal. In some non-limiting implementations, an audio event corresponds to an audio signature of an event. For example, an audio event may correspond to the audio signature of an action such as a vehicle passing-by and wake words.

[0047] The system 10 comprises a signal generator 12, a signal mixer 14, a Chua circuit 16 and a processor 18. It should be understood that the signals generated and / or outputted by the signal generator 12, the signal mixer 14 and the Chua circuit 16 are analog signals while the signals received and outputted by the processor 18 are digital signals. The personskilled in the art will understand that an analog-to-digital (AD) converter (not shown) is present between the Chua circuit 16 and the processor 18 to convert the analog signal outputted by the Chua circuit 16 into a digital signal suitable for the processor 18.

[0048] It should also be understood that the signal generator 12, the signal mixer 14 and the Chua circuit 16 form together a feature extractor as better described below.

[0049] While the system 10 is described herein in relation with the identification of audio events, the person skilled in the art will understand that the system 10 may be configured for performing another task depending on how the processor 18 is configured. For example, the processor 18 may be configured for performing a task other than the identification of audio events from an audio time series signal. For example, the processor 18 may be configured to identify features, characteristics or elements of an audio signal such as voice biometrics, audio signatures to predict anomalies of industrial equipment, etc. while using the same feature extractor as the one contained in the system 10.

[0050] The signal mixer 14 is operatively connected to the signal generator 12 and the Chua circuit 16 while the Chua circuit is operatively connected to the processor 18 via an AD converter (not shown). Optionally, the system 10 may comprise an output device 20 operatively connected to the processor 18. For example, the output device 20 may be a memory for storing thereon the signal outputted by the processor 18. In another example, the output device 20 may be a display unit for displaying the output of the processor 18 to a user.

[0051] It should be understood that the signal mixer 14 is further connected to an audio input device such as a microphone for receiving the analog audio signal to be processed therefrom.

[0052] In operation, the signal mixer 14 receives an analog audio times series signal to be processed while the signal generator 12 generates an analog periodic audio signal. The signal mixer 14 combines together the received audio time series signal and the generated periodic audio signal to obtain a combined analog audio signal which propagates up to the Chua circuit 16. The Chua circuit 16 is configured for converting the received combinedaudio signal into an analog feature signal indicative of features contained in the combined audio signal. Then, the feature signal is converted into a digital signal which is fed into the processor 18. The processor 18 is configured for extracting events using the features identified by the Chua circuit 16.

[0053] The signal generator 12 is configured for generating a periodic signal of which the frequency corresponds to the frequency of the Chua circuit 16. In some implementations, the frequency of the periodic signal is equal to the frequency of the Chua circuit 16. In other implementations, the frequency of the periodic signal is similar to the frequency of the Chua circuit 16. For example, the frequency of the periodic signal may be comprised within + / - 5% of the frequency of the Chua circuit 16. In another example, the frequency of the periodic signal may be within + / - 1% of the frequency of the Chua circuit 16.

[0054] The periodic signal generated by the signal generator 12 may have any adequate periodic shape. For example, periodic signal generated by the signal generator 12 may be a sinusoidal signal, a square wave signal, a triangular wave signal, or the like.

[0055] In some implementations, the signal generator 12 comprises a Wien bridge circuit configured for generating a periodic signal (such as a sinusoidal signal) of which the frequency corresponds to that of the Chua circuit 16.

[0056] In other implementations, the signal generator 12 comprises an oscillator such as a phase shift oscillator configured for generating a periodic signal (such as a sinusoidal signal) of which the frequency corresponds to that of the Chua circuit 16.

[0057] In further implementations, the signal generator 12 comprises a digital processor configured for generating a digital periodic signal of which the frequency corresponds to that of the Chua circuit 16 by direct synthesis, and a digital-to-analog (DA) converter configured for converting the digital periodic signal into an analog periodic signal.

[0058] In some implementations in which the periodic signal is a square wave signal, the signal generator 12 may comprise a timer integrated signal such as a CMOS 555 integrated circuit or an equivalent and adequate integrated circuit.

[0059] In some implementations in which the periodic signal comprises a triangular wave, the signal generator 12 may comprise a Schmitt trigger and an Op-amp integrator circuit, or any adequate and equivalent integrated circuits.

[0060] In some implementations, the amplitude of the periodic signal generated by the signal generator 12 is greater than that of the audio time series signal to be processed and received by the signal mixer 14. For example, the amplitude of the periodic signal may be chosen to be about 25 times greater than that of the audio time series signal received by the signal mixer 14.

[0061] In some implementations, the amplitude of the generated periodic signal is chosen so as to fulfil both sufficient ratio of the periodic signal amplitude to the amplitude of the audio time series signal to be processed considering supressing background audio and random noise, as well as sufficient features of the audio time series signal to be captured by the processor 18.

[0062] Referring back to Fig. 1, the signal mixer 14 is configured for receiving the audio time series signal to be processed from an audio input device such as a microphone and the periodic audio signal from the signal generator 12. The signal mixer 14 is further configured for mixing together the audio time series signal and the periodic audio signal to output a mixed audio signal. In some implementations, the signal mixer 14 is configured for multiplying together the audio time series signal and the periodic audio signal to obtain the mixed signal. In other implementations, the signal mixer 14 is configured for adding together the audio time series signal and the periodic audio signal to obtain the mixed signal. In further implementations, the signal mixer 14 is configured for modulating the audio time series signal and the periodic audio signal to obtain the mixed signal.

[0063] The thus-obtained mixed signal is then received by the Chua circuit 16 which is operated to exhibit a limit cycle behavior so as to generate a feature signal indicative of features of the audio time series signal. The feature signal outputted by the Chua circuit 16 is converted into a digital signal and transmitted to the processor 18.

[0064] The processor 18 is configured for identifying at least one audio event from the signal received from the Chua circuit 16.

[0065] In some implementations, the processor 18 is configured for executing a machine learning model for analysing the audio time series signal. Examples of adequate machine learning methods includes linear regression, decision tree, k-nearest neighboring, recurrent neural network, etc. In some implementations, the machine learning model is trained prior to be deployed to the processor 18. In this case, training audio segments or clips are fed into the feature extractor and the output signals of the feature extractor are used to train the machine learning model on a computer other than the processor 18, such as a server located in the cloud.

[0066] In some implementations, the processor 18 is a lightweight processor such as a microcontroller.

[0067] In one embodiment, the system 10 further comprises an amplifier for amplifying and offsetting the audio time series signal to obtain an amplified audio signal. The amplifier is operatively connected to the signal mixer 14 so that the signal mixer 14 mixes together the amplified audio signal and the periodic signal to avoid negative values.

[0068] Fig. 2 illustrates one embodiment of an adequate Chua circuit that may be used in the system 10. The Chua circuit further comprises a coil LI , a first capacitor Cl , a second capacitor C2, a first diode DI, a second diode D2 and a Chua diode (comprising an amplifier and two resistors), which are all connected in parallel. The Chua circuit further comprises a resistor R1 connected in series with the Chua diode and two resistors R2 each mounted in series with a respective diode DI, D2. The Chua circuit further comprises a resistor R connected between the two capacitors Cl and C2. The resistor R as an adjustable potentiometer to control the operation cycle of the Chua circuit.

[0069] In some implementations, the resistance of the resistor R connected between the two capacitors Cl and C2 is adjusted to be around 1 ,500 ohms. Around this value, the Chua circuit produces a limit cycle system as illustrated in Fig. 3, of which the radius is constantly changing non-linearly based on the mixed signal.

[0070] The limit cycle occurring in the Chua circuit is defined as the cycle that the two parameters in the non-linear system (i.e., Vcl and Vc2 in the Chua circuit case) create a trajectory (Vc2 vs. Vcl in the Chua circuit case) that will not diverge as the time approaches infinity, where Vcl and Vc2 correspond to the electrical voltage across the capacitors Cl and C2, respectively. In other words, the signals Vcl and Vc2 containing audio information evolve around a fixed and closed trajectory over time with variation of the instantaneous curvature on the trajectory due to audio signals, encoding and regularizing audio signals on the oscillation of the Chua audio processor, without chaotic signals unperceivable by the machine learning readouts.

[0071] The non-linear resistor, i.e., the Chua diode comprising an amplifier and two resistors, operates as an equivalence to the activation function in machine learning built in the digital platform to extract the key features such as volume and pitch change of the audio.

[0072] While in the illustrated example the non-linear resistor comprises an amplifier and two resistors, the person skilled in the art will understand that the non-linear resistor may be constructed using other electrical components with high non-linearity such as a Gunn diode, a memristor, a pre-programmed FPGA, a gap-closing actuator MEMS (Microelectromechanical systems) etc.

[0073] The person skilled in the art will understand that the natural frequency of the Chua circuit may be adjusted by the capacitors Cl and C2. In some implementations, the natural frequency of the Chua circuit is around 1,000 Hz. The linear and non-linear harmonics produced by a Chua circuit having a natural frequency of about 1,000 Hz cover from about 200 Hz to about 10,000 Hz in one embodiment, or almost all the frequencies associated with human voice and urban and natural sound events.

[0074] Two tests have been conducted to demonstrate the non-linear effects brought by the above-described Chua circuit to the audio signal for processing. First, a comparison is made on the Fourier transform before and after the Chua circuit, as illustrated in Figs. 4a and 4b. It clearly shows more and wider peaks over the whole spectrum (about 200 Hz to over 16,000 Hz). Second, a single tone audio with 1,000 Hz with different volumes wasplayed to the sound recognition hardware. As shown in Figs. 5a and 5b, with increasing volume, new peaks in the spectrum (especially in the frequency range lower than audio frequency) are created following the Farey sequence, providing evidence of strong nonlinearity.

[0075] A typical signal processing flow of using a Chua circuit to carry out audio recognition is shown in Fig. 6. The signal processing procedure uses the oscillatory nature of the Chua circuit to reduce the amount of data for the learning model to process.

[0076] As mentioned above, the non-linear oscillator, i.e., the Chua circuit, uses the limit cycle radius variation of the signal from the oscillator to encode and enhance the features from the audio signal. To capture this variation, the processor 18 first reads one oscillation cycle (based on the activation frequency) of the signal from the Chua circuit 16. Though the oscillation is highly non-linear, the most prominent oscillation besides the one corresponding to the input audio is the one with the frequency same to the natural frequency of the non-linear oscillator.

[0077] Followed by a Z-score normalization expressed in Eq. 1 below, the signal is processed to show the variations of signals over different cycles of the oscillation over time. In Eq. 1, x corresponds to the original value of the signal, subtracted by the average of one oscillation cycle. This value is then divided by the standard deviation. The inverse tanh activation as expressed in Eq. 2 below is operated afterwards using the normalized values of signal such that the granularities of the audio signal (e.g., the abrupt change of the volume and pitch in the audio) are further enhanced and random noises (usually low values around zero) are supressed. After inverse tanh activation, a delay is set to build to sample the data. After a sufficiently long period of time, the signals are ensemble averaged based on the cycle reading to further reduce the amount of data for machine learning readout. Last, the ensemble average reading of the oscillator signal is shifted to have universal zero-crossings (i.e., where the value of the reading changes from positive to negative) over different ensemble average readings of the signal to reduce the randomness in the processing. As shown in Fig. 7 which illustrates the rendering of audio features generated from the system of Fig. 1, the signal corresponding to a specific sound eventshows a clearly distinguishable feature even with very low signal to noise ratio. Additionally, compared to the feature maps generated by the present method and system, commonly used Mel spectrogram (Fig. 7b and d) loses important features and inevitably loses accuracy while the proposed reservoir computing method still has clear feature map under very low signal to noise ratio.>x xcycle -tnorm 7T7A 7 std(cycie)■^feature ~ Ee(ai ctanh(%Iionn))(Eq. 2)

[0078] Noteworthily, without non-linear components in the system, the oscillatory signal mixed from the original audio and activation signal could not generate sustained limited cycle system. As such, as shown in Fig. 7, the feature map illustrated in Fig. 4 does not show clear and perceivable patterns for sound recognition in comparison to the one with the effect of Chua circuit. This demonstration further underscores the importance of the role of Chua circuit in the process of extracting useful audio information for machine learning model to perceive and recognize.

[0079] Figs. 8a and 8b illustrate an exemplary preprocessing operated on the signal after the signal mixer and operated on the signal after Chua circuit, respectively. In this case, the audio piece in this test is a 1,000 Hz single tone audio.

[0080] This ensemble reading is subsequently fed into the processor (both static and dynamic memory usages < 100 KB) for audio event recognition. The overall accuracy running a 10-class urban sound recognition dataset could reach as high as >90% with a machine learning model capable of being deployed to the edge devices, as illustrated in Fig. 9. In one embodiment, the whole audio recognition system including microphone, processing circuit, and readout device only consumes less than 10 mW power (i.e., always- on running relaying on Li-Po battery for over one month) and is reconfigurable on the edge by swapping software readouts or different hardware settings for different recognition tasks. An example of hardware reconfiguration is shown in Fig. 10, with increasing amplitude of activation signal, the intrinsic non-linearity of the Chua circuit becomes moredominant to the background noise of the input audio, generating more diverse and clearer audio feature patterns, suppressing noises. One could imagine a future sound detection system running at different activation amplitude based on the background noise, to achieve an on-the-fly audio detection optimization.

[0081] In some implementations, the system 10 can be implemented on a 2 cm x 3 cm area with less than 10 mW power consumption.

[0082] In some implementations, the system 10 can implement a 10-class audio recognition on edge device reaches over 95% accuracy with <100 KB dynamic memory.

[0083] In some implementations, the system 10 can implement compression of output signals from the oscillator-based physical reservoir computer by over 99% for high-fidelity audio recognition with low computational load.

[0084] In v, the system 10 allows for on-the-fly reconfiguration via swapping software readouts and change activation signals.

[0085] In some implementations, the system 10 eliminates the needs of constructing reservoir computer using time multiplexing and delay feedback on hardware, and audio preprocessing on software, significantly reduces the complexity of reservoir computer operations.

[0086] In some implementations, the system 10 allows for substantially reducing the number of the electrical components and power consumption of the entire hardware. At the same time, by carefully following the design parameters and operational principle of the oscillator-based physical reservoir computing, the preprocessing technology not only reduces the amount of data for processing, but also keeps the most important features to achieve high fidelity audio recognition on edge with limited computational resource and bandwidth.

[0087] In some implementations, the present technology creates an on-sensor sound signal recognition using the interaction of the audio signal with the non-linear oscillation to extract audio features for machine learning recognition. As such, most of the machinelearning operations happens on the edge devices while taking the data, eliminating the need of separate data storage and massive processing scheme which is usually needed for automatic sound event detection. The audio detection pipeline including inference and training is fully deployable on the edge devices, saving bandwidth and storage of edgecloud communication, and closing potential security and privacy loopholes. For end users, the bandwidth reduction could be accounted for at least 50% of operational cost on data transfer and storage, allowing massive scale-up of automatic audio detection in underserved sectors such as natural sound detection and classification (internal estimation based on available public data). At the same time, the closure of the security loophole could prevent the material loss of 4.24 million dollars per potential security breach. This removes one major hurdle of the wide spread of automatic audio (especially for human voice) recognition of the private data leakage. And the new use cases including payment over voice recognition can be explored owing to the highly secure audio detection the invention provides. For original equipment manufacturers (OEMs), the computational load of the audio data detection on central processors can be drastically reduced since that most of the feature extraction of the audio is achieved by the non-linear oscillator attached to the microphone sensor. The requirement of the amount of the data for audio recognition model development is also reduced for the same reason, paving the way of deploying sound recognition for the scarce dataset scenario. It is worth noting that the proposed system is directly related to the early-stage revenue as the backbone technology for the development of audio recognition kit for the use cases with scarce datasets and special requirements such as unreliable power source and zero internet connection.

[0088] In some implementations, the intended use case of the system 10 is for audio recognition deployed on low-end edge devices for both inference and machine learning training. However, the general idea behind the invention is extendable to the use cases for processing non-audio time series data such as the uses cases with vibration in predictive maintenance and accelerometer data. Meanwhile, by detecting the key features of the oscillatory signal such as rising and falling edge, envelope of the signal, and zero crossing, some unique audio event like glass breaking could be detected without machine learning model.

[0089] In some implementations and compared to at least some of the prior art of edge audio processing using reservoir computing technology, the system 10 has much smaller form factor and could be attached with much lower end processing hardware since the use of non-linear oscillation as the processing method is directly related to the audio signal, without the needs of preprocessing of the data usually applied. Second, the activation signal triggering the oscillator oscillation creates a clear and strong reference for timemultiplexing (i.e., combine signal from different time instants for better processing results) on software eliminating hardware-based time-multiplexing which is usually bulky and increases the power consumption. Third, the use of analog circuit as the physical reservoir drastically reduces the manufacturing complexity and configuration cost compared to the usual methods like optical based. This allows wide dissemination of the automatic audio event detection technology in underserved areas such as environmental protection and predictive maintenance.

[0090] In the same or other implementations, the data acquisition and preprocessing follow the design parameters of the oscillator-reservoir and reduces the amount of data for machine learning recognition. This naturally reduces the requirement of software and hardware for audio signal recognition. It may be noted that though the preprocessing eventually reduced the signal to the signal vector to fulfil audio recognition fully on edge with very stringent resource, the machine-learning-based audio recognition can be done on 2-D feature maps as exemplary shown in Fig. 7, and with different sampling method for different lengths of audios, showing great versality which previous method does not possess.

[0091] In some implementations, the periodic signal may be reconfigured by its amplitude for different signal to noise ratio and by its frequency for specific use cases (e.g., glass breaking) allowing no-learning audio detection and hardware-based reservoir computer reconfiguration which was substantially not possible previously.

[0092] While in the above description, the system 10 is used for analysing audio signals, the person skilled in the art will understand the feature extractor of the system 10 can be used for extracting features from time series signals other than audio time series signals,such as pressure time series signals, vibration time series signals, acceleration time series signals, any analog time series signal outputted by an adequate sensor, or the like.

[0093] The present technology may be embodied as a method 200 for extracting features from an analog audio time series signal, as illustrated in Fig. 11.

[0094] At step 202, an analog periodic signal is generated, as described above.

[0095] At step 204, the analog periodic signal is mixed with the analog time series signal from which features are to be extracted, as described above, thereby obtaining an analog mixed signal.

[0096] At step 206, an electronic Chua circuit is operated so as to have a limit cycle behavior, as described above.

[0097] At step 208, the analog mixed signal is propagated into the Chua circuit, thereby obtaining an analog feature signal indicative of features of time series data contained in the time series signal, as described above. The frequency of the analog periodic signal substantially corresponds to the frequency of the Chua circuit.

[0098] At step 210, the analog feature signal is outputted. As described above, the feature signal may propagate to an analog-to-digital converter being transmitted to a processor.

[0099] In some non-limiting embodiments, the generation of the periodic signal is performed using a Wien bridge circuit, a phase shift oscillator, a digital processor configured for direct digital synthesis, or the like.

[0100] In some non-limiting embodiments, the frequency of the periodic signal is equal to the circuit frequency.

[0011] In some non-limiting embodiments, the periodic signal comprises a square wave signal.

[0102] In some non-limiting embodiments, the generation of the periodic signal is performed using a timer integrated circuit.

[0103] In some non-limiting embodiments, the periodic signal comprises a triangular wave signal.

[0104] In some non-limiting embodiments, generation of the periodic signal is performed using a Schmitt trigger and an Op-amp integrator circuit.

[0105] In some non-limiting embodiments, the method 200 further comprises amplifying and offsetting the time series signal prior to the mixing step 204.

[0106] In some non-limiting embodiments, the Chua circuit comprises a non-linear component. In this case, the operation of the Chua circuit comprises setting properties of the non-linear component to a target property value chosen so that the Chua circuit has the limit cycle behavior.[0.107] In some non-limiting embodiments, the non-linear component comprises a nonlinear resistor. In this case, the operation of the Chua circuit comprises setting a resistance of the non-linear resistor to a target resistance value chosen so that the Chua circuit has the limit cycle behavior.

[0108] In some non-limiting embodiments, the time series signal comprises an audio or sound signal.[0.109] Modifications and improvements to the above-described implementations of the present technology may become apparent to those skilled in the art. The foregoing description is intended to be exemplary rather than limiting.

Claims

CLAIMSWhat is claimed is:

1. A device for extracting features from a time series signal, the device comprising: a signal generator configured for generating a periodic signal; a signal mixer connected to the signal generator and configured for mixing together the time series signal and the periodic signal to obtain a mixed signal; and a Chua circuit having a circuit frequency, the Chua circuit being connected to the signal mixer for receiving the mixed signal therefrom and configured for:- generating a feature signal indicative of features of time series data contained in the time series signal when operated to have a limit cycle behavior; and- outputting the feature signal, wherein a given frequency of the periodic signal substantially corresponds to the circuit frequency.

2. The device of claim 1 , wherein the signal generator comprises one of a Wien bridge circuit, a phase shift oscillator and a digital processor configured for direct digital synthesis.

3. The device of claim 1 or 2, wherein the given frequency is equal to the circuit frequency.

4. The device of any one of claims 1 to 3, wherein the periodic signal comprises a square wave signal.

5. The device of claim 4, wherein the signal generator comprises a timer integrated circuit.

6. The device of any one of claims 1 to 3, wherein the periodic signal comprises a triangular wave signal.

7. The device of claim 6, wherein the signal generator comprises a Schmitt trigger and an Op-amp integrator circuit.

8. The device of any one of claims 1 to 7, further comprising an amplifier for amplifying and offsetting the time series signal to obtain an amplified and offset time series signal, the amplifier being connected to the signal mixer for providing the amplified and offset time series signal thereto.

9. The device of any one of claims 1 to 8, wherein the Chua circuit comprises at least a non-linear component, properties of the non-linear component being chosen so that the Chua circuit has the limit cycle behavior.

10. The device of claim 9, wherein the non-linear component comprises a non-linear resistor, a resistance of the non-linear resistor being chosen so that the Chua circuit has the limit cycle behavior.

11. The device of any one of claims 1 to 10, wherein the time series signal comprises an audio signal.

12. A method for extracting features from a time series signal, the method comprising: generating a periodic signal; mixing together the time series signal and the periodic signal, thereby obtaining a mixed signal; operating a Chua circuit so that the Chua circuit has a limit cycle behavior; propagating the mixed signal into the Chua circuit, thereby obtaining a feature signal indicative of features of time series data contained in the time series signal, a given frequency of the periodic signal substantially corresponding to a circuit frequency of the Chua circuit; and outputting the feature signal.

113. The method of claim 12, wherein said generating the periodic signal is performed using one of a Wien bridge circuit, a phase shift oscillator and a digital processor configured for direct digital synthesis.

14. The method of claim 12 or 13, wherein the given frequency is equal to the circuit frequency.

15. The method of claim 12, wherein the periodic signal comprises a square wave signal.

16. The method of claim 15, wherein said generating the periodic signal is performed using a timer integrated circuit.

17. The method of claim 12, wherein the periodic signal comprises a triangular wave signal.

18. The method of claim 17, wherein said generating the periodic signal is performed using a Schmitt trigger and an Op-amp integrator circuit.

19. The method of any one of claims 12 to 18, further comprising amplifying and offsetting the time series signal prior to said mixing.

20. The method of any one of claims 12 to 19, wherein the Chua circuit comprises a non-linear component, said operating the Chua circuit comprises setting properties of the non-linear component to a target property value chosen so that the Chua circuit has the limit cycle behavior.

21. The method of claim 20, wherein the non-linear component comprises a non-linear resistor, said operating the Chua circuit comprises setting a resistance of the non-linear resistor to a target resistance value chosen so that the Chua circuit has the limit cycle behavior22. The method of any one of claims 12 to 21, wherein the time series signal comprises an audio signal.

Images