Machine learning model training

Abstract

The embodiment of the invention relates to machine learning model training. The proposed concepts are directed to providing schemes, solutions, concepts, designs, methods and systems relating to training a machine learning model to detect audio of a target classification. Specifically, embodiments are directed to providing a method for training a machine learning model to detect a target classification of audio by performing quantitative perceptual training on the pre-trained audio classification machine learning model using training data including at least one audio signal, the at least one audio signal includes audio satisfying the target classification.

Description

Technical Field

[0001] The present invention relates to the field of training machine learning models, and more particularly to the field of training machine learning models to detect audio of a target category. Background Art

[0002] More than 130 million babies are born worldwide each year. The baby monitor market is a multi-billion dollar industry and is driven by growing child safety concerns, innovations in video and internet technology, and an increasing number of working parents living in nuclear families. These monitors typically include a baby unit, which is usually placed in or near the baby's crib, and a parent unit, which receives an audio and / or video feed from the baby unit. Some of the features available in baby monitors include infrared vision, temperature sensors, recording capabilities, etc. Some monitors can also play lullabies on the baby unit and have two-way communication capabilities for parents to talk or soothe the baby.

[0003] Most baby monitors only provide audio and video that parents must constantly listen and watch. Automatically identifying and alerting parents when a baby is crying in real time would be an enhancement over existing monitors.

[0004] Classical methods for crying detection involve extracting different features from audio signals and using these features to train machine learning models to detect crying. Time domain and frequency domain features are commonly used for crying detection. Frequency domain features are generally more robust than time domain features and are therefore widely used. In addition to time domain and frequency domain features, prosody features are also used for crying detection.

[0005] Cepstral analysis is widely used in speech processing because it perfectly symbolizes speech waveforms and characteristics with features of limited size. Mel-frequency cepstral coefficients (MFCC) are cepstral representations of audio and are the most commonly used features in speech recognition systems. Linear prediction cepstral coefficients (LPCC) involve predicting the next value by a linear combination of previous coefficients. Linear frequency cepstral coefficients (LFCC) use linear rather than Mel filter banks. LFCCs perform better than MFCCs in distinguishing high-frequency audio signals such as female voices and baby crying signals.

[0006] Using such features, classifying whether the baby is crying can be done by training on a traditional machine learning classifier or a neural network-based model. Traditional machine learning models include support vector machines (SVM), K-nearest neighbors (KNN), random forests, XGBoost, etc., which can all be used for classification. On the other hand, neural network-based models include feedforward neural networks (FFNN), convolutional neural networks (CNN), recurrent neural networks (RNN), and long short-term memory (LSTM), etc.

[0007] However, these existing methods are not optimized to work well on low processing power, such as on a baby monitor.

[0008] “Transfer Learning Based Automatic Model Creation Tool For Resource Constraint Devices” by Karthik Bhat et al. describes an automatic model creation tool and the CNN models created by it using pre-trained models such as YAMNet and MobileNetV2 as feature extractors. Summary of the invention

[0009] The invention is defined by the claims.

[0010] According to an example of one aspect of the invention, a method for training a machine learning model to detect audio of a target classification is provided.

[0011] The method includes: training a pre-trained audio classification machine learning model on a training data set to predict whether an input audio signal includes audio that satisfies a target classification, wherein the training data set includes at least one audio signal, and the at least one audio signal includes audio that satisfies the target classification; and is characterized in that the training includes quantization-perceptual training, wherein at least one learnable value or activation of the model is quantized.

[0012] Therefore, the proposed concepts are intended to provide schemes, solutions, concepts, designs, methods and systems related to training a machine learning model to detect audio of a target classification. Specifically, an embodiment is intended to provide a method for training a machine learning model to detect audio of a target classification, the method performing quantized perceptual training on a pre-trained audio classification machine learning model using training data including at least one audio signal, wherein the at least one audio signal includes audio that satisfies the target classification.

[0013] In other words, it is proposed that by adopting a pre-trained audio classification machine learning model and further training it using quantized perceptual training, a machine learning model capable of classifying target classified audio can be provided, which is lighter than the original pre-trained model but can still accurately perform target classification.

[0014] By using quantization to reduce the computational load of determining the links between layers in an AI model (i.e., by limiting the representation space required to represent numbers, such as by truncating digits after the first decimal or limiting the number of gray values ​​in a grayscale), it is possible to provide a more computationally lightweight and smaller machine learning model that is still capable of classifying audio of the target category.

[0015] Thus, the present invention can be particularly useful in training models that can be deployed in computationally simple devices, such as baby monitors. In this way, the device can run a further trained machine learning model to detect audio of a particular target classification without requiring excessive computing resources or connectivity to the cloud. For example, a baby monitor can be provided that includes a model trained according to the present invention to detect audio of a baby / infant crying.

[0016] Since audio classification is a computationally demanding task, models that can classify audio into multiple categories must be large and computationally expensive. By further training the pre-trained audio classification machine learning model to essentially reduce it to only classify audio of the target category, the size and computational requirements of the model can be reduced accordingly without significantly sacrificing its accuracy in classifying audio of the target category.

[0017] Quantization-aware training differs from post-training quantization in that post-training quantization involves training the model traditionally (or further training a pre-trained model) and then quantizing the weights only after training (other learnable parameters cannot be quantized). Post-training quantization results in a loss of accuracy because the model has been trained and optimized for non-quantized parameters and is then simply blindly quantized afterwards. On the other hand, quantization-aware training builds quantization into the training itself so that the model can be more accurate at (and optimized for) the quantization level that is ultimately used. Furthermore, during quantization-aware training, learnable parameters other than weights, such as biases and thresholds, and even the training data itself can be quantized. Quantization-aware training therefore results in more accurately quantized models, and generally more compressed models, than post-training quantization because other learnable parameters can be quantized in addition to weights.

[0018] In summary, by taking a part of an already trained audio classification model and reusing it in a new model, we can avoid the large amount of computing power and labeled training data required to train a new model from scratch. In addition, by using quantization-aware training to train the "new model", we can reduce the size and computational requirements of the new model, making it implementable in computationally simple devices.

[0019] Ultimately, the present invention provides an improved method for training a machine learning model to detect audio of a target class.

[0020] In some embodiments, the at least one learnable value may include at least one of: a node weight; a bias; a batch normalization parameter; a gating or masking parameter; and a threshold. These are examples of learnable values ​​that can be quantized to reduce their numerical precision. Each of these parameters contributes to the size, memory footprint, and computational efficiency of the model during deployment.

[0021] In some embodiments, the target classification may include a baby crying. This helps train a model to detect the audio of a baby crying. The model can then be subsequently deployed in a baby monitor.

[0022] In some embodiments, the training data set may include at least one audio signal that does not satisfy the target classification. It has been found that this provides a more robust model because the model has been trained on a wider range of sounds and can therefore more effectively distinguish between a baby's cry and other sounds. In particular, it is useful to include sounds similar to a baby's cry (such as a cat meowing, babies and adults laughing, doors creaking, etc.) and more obviously different sounds (such as washing machines, alarm clocks, etc.) in the training data. In other words, the training data set may include at least one audio signal that does not satisfy the target classification but is substantially similar to the target classification.

[0023] In some embodiments, before training, the method may further include converting the audio signal of the training dataset into a floating point tensor. This facilitates the use of a wider range of machine learning models.

[0024] In some embodiments, before training, the method may further include resampling the audio signal of the training data set to a 16kHz mono audio signal. This improves the efficiency and / or effectiveness of the training because it evens out the audio signal in terms of sampling frequency and number of channels.

[0025] In some embodiments, the method may further include obtaining a training data set.

[0026] According to another aspect of the present invention, there is provided a computer program comprising code means for implementing any of the methods disclosed herein when the program is run on a processing system.

[0027] According to another aspect of the present invention, there is provided a computer program comprising code means for implementing a machine learning model trained according to any of the methods disclosed herein.

[0028] According to another aspect of the present invention, a device is provided, the device comprising a machine learning model trained according to any method disclosed herein for training a machine learning model to detect audio of a target classification. Thus, a device capable of detecting audio of a target classification is provided.

[0029] In some embodiments, the device can be configured to output a control signal in response to detecting audio of a target classification. In this way, the device can perform a specific action in response to detecting audio of a target classification. The control signal can be used to control the device itself and / or an external device / system.

[0030] In some embodiments, the control signal may include a signal for outputting an alarm. In this way, in response to detecting audio of a target classification (e.g., a baby crying), an alarm may be output on the device itself and / or may be sent to an external device / system (such as a parent's smartphone).

[0031] In some embodiments, the device can be configured to classify audio signals in real time. For example, a machine learning model can be run on the device, enabling the device to detect audio of a target classification in real time, and thereby potentially react to the detection in real time.

[0032] In some embodiments, the device may include a baby monitor.This is a computationally simple device that would benefit from including a lightweight model trained according to any of the embodiments disclosed herein in accordance with the above methods.

[0033] In some embodiments, the device may include a processor. In this way, a processor may be provided that includes a machine learning model trained according to any of the embodiments disclosed herein of the above method, which may then be utilized in various systems / devices.

[0034] Therefore, a concept for training a machine learning model to detect audio of a target classification can be proposed, and this can be achieved based on training a pre-trained audio classification machine learning model using quantization-aware training.

[0035] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] For a better understanding of the invention, and to show more clearly how it may be put into practice, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0037] Figure 1 is a simplified flow chart of a method for training a machine learning model to detect audio of a target classification according to a proposed embodiment;

[0038] Figure 2 is a flowchart of a method for training a machine learning model to detect audio of a target classification according to a proposed embodiment;

[0039] Figure 3 is a diagram for explaining a method for training a machine learning model to detect audio of a target classification according to a proposed embodiment;

[0040] Figure 4is a diagram for explaining a method for deploying a machine learning model to detect audio of a crying baby according to a proposed embodiment;

[0041] Figure 5A is a simplified block diagram of a device including a machine learning model trained according to the proposed embodiments;

[0042] Figure 5B is a simplified block diagram of a baby monitor according to the proposed embodiment, the baby monitor including a machine learning model and configured to output a control signal; and

[0043] Figure 6 An example of a computer is shown in which one or more portions of the embodiments may be employed. DETAILED DESCRIPTION

[0044] The present invention will be described with reference to the accompanying drawings.

[0045] It should be understood that the specific embodiments and specific examples, although indicating exemplary embodiments of the apparatus, system and method, are for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the apparatus, system and method of the invention will be better understood through the following description, the appended claims and the accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0046] By studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and implement variations to the disclosed embodiments in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0047] Implementations according to the present disclosure involve various techniques, methods, schemes, and / or solutions related to training a machine learning model to detect audio of a target classification. Based on the proposed concepts, multiple possible solutions can be implemented individually or in combination. That is, although these possible solutions may be described separately below, two or more of these possible solutions may be implemented in one combination or another.

[0048] Embodiments of the present invention are intended to provide a method for training a machine learning model to detect audio of a target classification. This can be achieved by training a pre-trained audio classification machine learning model using quantized perceptual training.

[0049] Therefore, the proposed concepts are intended to provide schemes, solutions, concepts, designs, methods and systems related to training a machine learning model to detect audio of a target classification. Specifically, an embodiment is intended to provide a method for training a machine learning model to detect audio of a target classification by performing quantized perceptual training on a pre-trained audio classification machine learning model using training data including at least one audio signal, wherein the at least one audio signal includes audio that satisfies the target classification.

[0050] In other words, it is proposed that by adopting a pre-trained audio classification machine learning model and further training it using quantization-perceptual training, a machine learning model capable of classifying target classified audio can be provided, which is lighter than the original pre-trained model but still capable of accurately performing target classification.

[0051] Reference now Figure 1 , depicts a simplified flow chart of a method 100 for training a machine learning model to detect audio of a target classification according to the proposed embodiment.

[0052] Method 100 includes step 110: training a pre-trained audio classification machine learning model on a training data set to predict whether an input audio signal includes audio that satisfies a target classification, wherein the training data set includes at least one audio signal, the at least one audio signal including audio that satisfies the target classification. The training includes quantized perceptual training, wherein at least one learnable value or activation of the model is quantized during training. An audio signal including audio that satisfies the target classification may also be understood as an audio signal of the target classification or an audio signal within the target classification. An audio signal including audio that satisfies the target classification may be understood as an audio signal including at least some audio that matches / satisfies the target classification, and the audio signal is not prohibited from also including audio that does not satisfy the target classification (such as background noise).

[0053] Quantization-aware training can be understood as a method for making machine learning models (or neural networks) ready for deployment on hardware with limited numerical precision. The goal of quantization-aware training is to make machine learning models more robust to the reduced computational precision involved in these hardware environments while minimizing the loss in model accuracy.

[0054] Machine learning models are typically trained using 32-bit floating point numbers (single precision) or even 64-bit (double precision), which provide high accuracy but require a lot of memory and computing resources. However, many hardware platforms, especially those in embedded systems or mobile devices, use lower precision data types (such as 8-bit integers) for calculations due to their reduced memory requirements and faster computing speed.

[0055] Quantization refers to the process of mapping a series of continuous values ​​to a smaller set of discrete values. In the context of machine learning models, quantization involves converting the network's high-precision floating-point weights and activations to lower-precision integer representations.

[0056] In other words, in quantization-aware training, the quantization process is integrated into the training itself. During training, the model learns to adjust its parameters taking into account the impact of quantization. This means that quantization is simulated during the forward and backward passes of the network. QAT ensures that the weights of the model are more suitable for quantization, resulting in minimal accuracy loss compared to simply quantizing the model after traditional training.

[0057] On the other hand, post-quantization training involves first training the model at full precision using standard techniques. After training, the model is quantized to reduce precision. This approach does not modify the training process, but rather applies quantization afterwards. It can result in a more significant loss of accuracy compared to QAT, especially if the original model architecture was not designed with quantization in mind.

[0058] Essentially, the difference is when quantization is applied during the training process. QAT incorporates the effects of quantization directly into the optimization, while post-quantization training applies quantization after the model has been trained at full precision. QAT generally produces better results in terms of maintaining accuracy under quantization, but it requires more expertise to implement effectively.

[0059] Learnable parameters (also called trainable parameters or parameters to be learned) are values ​​within a machine learning model that are adjusted during the training process. These parameters are core components that allow machine learning models, especially neural networks, to adapt and improve their performance on specific tasks.

[0060] In a machine learning model, learnable parameters are the values ​​that are iteratively updated as the model learns from the input data. The model learns to adjust these parameters so as to minimize the difference between its predicted values ​​and the actual target values ​​in the training data. The process of updating these parameters is usually achieved through an optimization algorithm, such as gradient descent and its variants.

[0061] In this embodiment, the at least one learnable value includes at least one of the following: node weight; bias; batch normalization parameter; gating or masking parameter; and threshold. These are examples of learnable values ​​that can be quantized to reduce their numerical precision. Each of these parameters contributes to the size, memory usage, and computational efficiency of the model during deployment. In other embodiments, the at least one learnable value may include any suitable learnable parameter.

[0062] The activation of the model can be understood as the output of each layer (i.e., intermediate and final layers) in the neural network. Therefore, the final output of the machine learning model is included in the term "activation". Quantization of activations can significantly reduce memory and computation requirements during inference.

[0063] In this way, an optimized and lightweight machine learning model can be provided that can accurately detect audio of a target category while being robust to various background noises such as television, door opening or closing, knocking, etc. In other words, even if the audio that satisfies the target category is relatively quiet, the model can still accurately detect the audio of the target category.

[0064] Reference now Figure 2 , a flowchart of a method 200 for training a machine learning model to detect audio of a target classification according to the proposed embodiment is shown.

[0065] Step 110 and Figure 1 In step 210, a training data set is obtained. This makes the method more independent without having to rely on a third party to provide a training data set. In this embodiment, the training data set includes at least one audio signal that does not meet the target classification. It has been found that this provides a more robust model because the model has been trained on a wider range of sounds, so the crying of babies and other sounds can be distinguished more effectively. In particular, it is useful to include sounds similar to the crying of babies (such as cats meowing, babies and adults laughing, doors creaking, etc.) and more obviously different sounds (such as washing machines, alarm clocks, etc.) in the training data. In other words, the training data set may include at least one audio signal that does not meet the target classification but is substantially similar to the target classification. However, this feature is not necessary, and in some other embodiments, the training data set may only include audio signals that meet the target classification.

[0066] Furthermore, in this embodiment, the target classification includes a baby crying. This helps train a model to detect the audio of a baby crying. This model can then be subsequently deployed in a baby monitor. Using such a baby monitor can have a significant impact on the well-being of the baby as well as the parent / guardian, as the parent will no longer need to actively listen and be aware of the audio / video feed of the baby monitor to manually detect whether the baby is crying. Therefore, the parent / guardian is free to shift their attention elsewhere, and furthermore, the machine learning model can detect crying that the parent / guardian might otherwise miss.

[0067] In step 220, the audio signal of the training data set is converted into a floating point tensor. This facilitates the use of a wider range of machine learning models. In step 230, the audio signal of the training data set is resampled to a 16kHz single-channel audio signal, i.e., a 16kHz mono audio signal. This improves the efficiency and / or effectiveness of training because it homogenizes the audio signal in terms of sampling frequency and number of channels. In some other embodiments, either or both of steps 220 and 230 may be skipped. In addition, in some other embodiments, the order of steps 220 and 230 may be reversed.

[0068] Reference now Figure 3 , showing a schematic diagram of a method 300 for training a machine learning model to detect audio of a target classification according to the proposed embodiment.

[0069] For example, an example of a pre-trained audio classification machine learning model is YAMNet. YAMNet is a pre-trained deep network based on the AudioSet-YouTube TM corpus and uses Mobilenet_v1 deep separable convolutional architecture to predict 521 audio event classes.

[0070] The audio signal input in step 310 can be an audio signal from an open source data set. For example, a donation data set contains the crying of a baby (infant) due to various reasons such as hunger, tiredness, hiccups, abdominal pain and discomfort. Another data set is ESC-50, which is composed of 50 classes. This data set is composed of 2000 audio signals, and each audio signal duration is 5 seconds. For current training purposes, the sample of the ESC-50 data set can be used so that the audio files from the target classification are more, and the audio files from other classes are less. In some cases, the audio files in the target classification can be used only. In this case, the target classification is a baby crying.

[0071] In step 320, audio pre-processing is performed on the .wav audio files (signals) by first converting each of the audio files (signals) into a floating point tensor, and then resampling each of them to 16kHz mono audio.

[0072] Feature extraction is an important step for analyzing data 330. Audio data is a 3D signal representing time, amplitude, and frequency. Since AI models cannot directly process 3D formats, feature extraction methods are used. YAMNet is trained with audio features calculated as follows:

[0073] - Resample all audio to 16kHz mono.

[0074] - Spectrograms were computed using the magnitude of the short-time Fourier transform with a window size of 25ms, a window hop of 10ms, and a periodic Hann window.

[0075] - Mel-spectrogram is calculated by mapping the spectrogram into 64 Mel bins covering the range of 125-7500 Hz.

[0076] - compute the stable log-mel-spectrum by applying log(mel-spectrum+0.001),

[0077] The offset is used to avoid taking the logarithm around zero.

[0078] The features are then framed into 0.96 second 50% overlapping examples, where each example covers 64 mel bands and 96 10ms frames. These 96×64 patches are then fed into the Mobilenet_v1 model to produce a 3×2 array of activations for 1024 kernels on top of the convolution. These are averaged to give a 1024-dimensional embedding, which is then passed through a single logistic layer to get 521 output scores per class corresponding to a 960ms segment of the input waveform. Due to window framing, at least 975ms of the input waveform is required to get the output score for the first frame.

[0079] Each audio signal includes multiple features. However, it is necessary to extract the features that are relevant to the problem at hand. Using these audio features, an intelligent audio classification model can be built.

[0080] Training a new model from scratch requires a lot of labeled training data and a lot of computing power. Transfer learning is a technique that greatly shortens this process by taking parts of a model that has already been trained on a related task and essentially reusing it as a new model. For example, YAMNet can be used as a high-level feature extractor, and the 1024-D embedding output of YAMNet can be used as input features for another shallower model, which can then be trained on a small amount of data for a specific task. This helps to quickly create specialized audio classifiers without requiring large amounts of labeled data or having to train large models end-to-end.

[0081] To this end, the YAMNet model can essentially be used to train a new model. The YAMNet model is used to extract embeddings (features) from the audio file (step 330), and then the model is built in step 350 using a portion of the embeddings in the audio file (the embeddings can be split into training data (step 340) and test data (step 345)).

[0082] This new model can then be used to score test data (e.g., which can be audio files from a donated database) to verify the performance of the model in step 360. Thus, the resulting model is lightweight and can be deployed on-chip and in the cloud.

[0083] Reference now Figure 4 , a diagram is shown for explaining a method 400 for deploying a machine learning model to detect audio of a baby crying according to the proposed embodiment. Essentially, the flow from deployment to production is shown in method 400. A machine learning model trained according to any of the methods disclosed herein can be considered a base model, which can subsequently be retrained / fine-tuned in step 420 based on the new labeled audio files provided in step 410. In step 430, fine-tuning the model using a new audio dataset and its efficacy allows the best AI model to be saved. Model updating is an iterative process in which the model is continuously retrained and fine-tuned, and in step 440, the best model can be continuously deployed to edge devices.

[0084] In this example, when the model is used to detect a baby / infant crying, the baby monitor audio can be input into the deployed model in step 435. In other words, in the production stage, the baby monitor's microphone can take real-time audio as input and pass it to the latest iteration of the model, which can then process the audio to detect whether the baby is crying in step 450. In some cases, one or more similar models can be deployed on an edge device (e.g., a baby monitor), but for classifying video files and / or other environmental information (such as temperature, lighting, etc.). They can work together to produce a single indication of the detection of a certain classification, or they can each provide their own detection.

[0085] Reference now Figure 5A , depicts a device 500 including a machine learning model 510 trained according to the proposed embodiment. The device 500 includes a model 510 trained according to any method disclosed herein for training a machine learning model to detect audio of a target classification (e.g., method 100 or 200).

[0086] For example, the device 500 may include a computer program including code means for implementing a machine learning model 510 trained according to any of the methods disclosed herein.

[0087] In this embodiment, the device 500 is configured to classify audio signals in real time. For example, the machine learning model 510 can be run on the device, enabling the device to detect audio of the target classification in real time, and thereby potentially react to the detection in real time. However, this is not necessary for the operation of the device 500, for example, the device can analyze any received audio signal with the training model 510 at a later time or with a slight delay.

[0088] In this embodiment, the device 500 includes a processor. The processor 500 can then be utilized in various systems / devices.

[0089] Reference now Figure 5B , depicts a simplified block diagram of a device 505 according to the proposed embodiment, the device including a machine learning model 515 and configured to output a control signal 520. In this embodiment, the device 505 includes a baby monitor. This is a computationally simple device that would benefit from including a lightweight model 515 trained according to any of the embodiments disclosed herein in accordance with the above methods. Thus, the machine learning model 515 has been trained to detect audio of a target classification, where the target classification is a baby crying.

[0090] In response to detecting the audio of a crying baby, the device 505 is configured to output a control signal 520. In this way, the device 505 can perform a specific action in response to detecting a crying baby. The control signal can be used to control the device 505 itself and / or an external device / system.

[0091] Furthermore, in this embodiment, the control signal 520 includes a signal for outputting an alarm. In other words, the control signal 520 includes an alarm signal. In this way, in response to detecting the audio of a crying baby, an alarm can be output on the device itself and / or can be sent to an external device / system (such as a parent's smartphone).

[0092] Of course, in other embodiments, in response to detecting audio of other target categories, the same or different control signal 520 may also be output. In addition, as described above, the control signal does not necessarily include an alarm, but may be used to control ambient lighting, audio, equipment, etc.

[0093] In some embodiments, the baby monitor 505 may keep a historical track of the time and duration of the baby's crying.

[0094] Figure 6An example of a computer 600 is shown in which one or more portions of the embodiments may be employed. The various operations described above may utilize the capabilities of the computer 600. In this regard, it should be understood that the system functional blocks may be executed on a single computer or may be distributed across multiple computers and locations (e.g., connected via the Internet).

[0095] Computer 600 includes, but is not limited to, PCs, workstations, laptops, PDAs, handheld devices, servers, storage devices, and the like. Typically, in terms of hardware architecture, computer 600 may include one or more processors 610, memory 620, and one or more I / O devices 630, which are communicatively coupled via a local interface (not shown). The local interface may be, for example, but not limited to, one or more buses or other wired or wireless connections, as known in the art. The local interface may have additional elements, such as controllers, buffers (cache), drivers, repeaters, and receivers, to enable communication. In addition, the local interface may include address, control, and / or data connections to enable appropriate communication between the above components.

[0096] Processor 610 is a hardware device for executing software that may be stored in memory 620. Processor 610 may be virtually any custom or commercial processor, a central processing unit (CPU), a digital signal processor (DSP), or an auxiliary processor among several processors associated with computer 600, and processor 610 may be a semiconductor-based microprocessor (in the form of a microchip) or a microprocessor.

[0097] The memory 620 may include any one or combination of volatile memory elements (e.g., random access memory (RAM), such as dynamic random access memory, static random access memory, etc.) and non-volatile memory elements (e.g., ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic tape, compact disk read-only memory (CD-ROM), magnetic disk, floppy disk, cassette, cassette, etc.). In addition, the memory 620 may incorporate electronic, magnetic, optical, and / or other types of storage media. Note that the memory 620 may have a distributed architecture in which various components are remote from each other but can be accessed by the processor 610.

[0098] The software in the memory 620 may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. The software in the memory 620 includes a suitable operating system (O / S) 650, a compiler 660, a source code 670, and one or more applications 680 according to an exemplary embodiment. As shown, the application 680 includes many functional components for implementing the features and operations of the exemplary embodiment. According to an exemplary embodiment, the application 680 of the computer 600 can represent various applications, computing units, logics, functional units, processes, operations, virtual entities and / or modules, but the application 680 does not represent a limitation.

[0099] The operating system 650 controls the execution of other computer programs and provides scheduling, input-output control, file and data management, memory management, and communication control and related services. The inventors contemplate that the application 680 for implementing the exemplary embodiments may be applicable to all commercially available operating systems.

[0100] Application 680 may be a source program, an executable program (object code), a script, or any other entity including a set of instructions to be executed. When it is a source program, the program is usually translated via a compiler (such as compiler 660), an assembler, an interpreter, etc., which may or may not be included in memory 620 to operate normally in conjunction with O / S 650. In addition, application 680 may be written in an object-oriented programming language with data and method classes, or a procedural programming language with routines, subroutines and / or functions, such as but not limited to C, C++, C#, Pascal, Python, BASIC, API calls, HTML, XHTML, XML, ASP scripts, JavaScript, FORTRAN, COBOL, Perl, Java, ADA, .NET, etc.

[0101] I / O devices 630 may include input devices such as, but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. In addition, I / O devices 630 may also include output devices such as, but not limited to, a printer, display, etc. Finally, I / O devices 630 may also include devices that transmit both input and output such as, but not limited to, a NIC or modulator / demodulator (for accessing remote devices, other files, devices, systems, or networks), a radio frequency (RF) or other transceiver, a telephone interface, a bridge, a router, etc. I / O devices 630 also include components for communicating over various networks such as the Internet or an intranet.

[0102] If the computer 600 is a PC, workstation, intelligent device, etc., the software in the memory 620 may also include a basic input and output system (BIOS) (omitted for simplicity). The BIOS is a set of basic software routines that are used to initialize and test the hardware at startup, start the O / S 650, and support data transfer between hardware devices. The BIOS is stored in some type of read-only memory, such as ROM, PROM, EPROM, EEPROM, etc., so that when the computer 600 is activated, the BIOS can be executed.

[0103] When the computer 600 is operating, the processor 610 is configured to execute software stored in the memory 620, to transfer data to and from the memory 620, and to generally control the operation of the computer 600 according to the software. The applications 680 and the O / S 650 are read in whole or in part by the processor 610, possibly buffered within the processor 610, and then executed.

[0104] When the application 680 is implemented in software, it should be noted that the application 680 can be stored on substantially any computer-readable medium for use by or in conjunction with any computer-related system or method. In the context of this document, a computer-readable medium can be an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in conjunction with a computer-related system or method.

[0105] Application 680 may be embodied in any computer-readable medium for use by or in conjunction with an instruction execution system, apparatus, or device, such as a computer-based system, a system containing a processor, or other system that can obtain instructions from an instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" may be any device that can store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.

[0106] Figure 1-Figure 2 Methods and Figure 5A and Figure 5BThe device can be implemented in hardware or software, or in a mixture of the two (for example, as firmware running on a hardware device). In the case where the embodiment is partially or entirely implemented in software, the functional steps shown in the process flow chart can be performed by a suitably programmed physical computing device, such as one or more central processing units (CPUs) or graphics processing units (GPUs). Each process and the individual component steps shown in its flow chart can be performed by the same or different computing devices. According to an embodiment, a computer readable storage medium stores a computer program including a computer program code, which is configured to cause the one or more physical computing devices to perform the encoding or decoding method as described above when the program is run on one or more physical computing devices.

[0107] Storage media may include volatile and non-volatile computer memory, such as RAM, PROM, EPROM and EEPROM, optical disks (such as CD, DVD, BD), magnetic storage media (such as hard disk and magnetic tape). Various storage media may be fixed within the computing device or may be removable so that one or more programs stored thereon can be loaded into the processor.

[0108] In the case where the embodiments are partially or completely implemented in hardware, Figure 6 The blocks shown in the block diagrams may be separate physical components, logical subdivisions of a single physical component, or may all be implemented in an integrated manner in one physical component. The functionality of a block shown in the drawings may be divided among multiple components in one implementation, or the functionality of multiple blocks shown in the drawings may be combined in a single component in one implementation. Hardware components suitable for embodiments of the present invention include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs). One or more blocks may be implemented as a combination of dedicated hardware that performs some functions and one or more programmed microprocessors and associated circuitry that performs other functions.

[0109] A single processor or other unit may fulfill the functions of several items recited in the claims. The fact that certain measures are referred to in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. If a computer program is discussed above, it may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. If the term "adapted to" is used in the claims or description, it should be noted that "adapted to" is intended to be equivalent to "configured to". Any reference signs in the claims should not be construed as limiting the scope.

[0110] The flow chart and block diagram in the figure show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each box in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing (multiple) specified logical functions.In some alternative implementations, the functions described in the box may not appear in the order shown in the figure.For example, according to the functions involved, the two boxes shown in succession can actually be executed substantially at the same time, or these boxes can sometimes be executed in reverse order.It will also be noted that each box of the block diagram and / or flow chart diagram and the combination of the boxes in the block diagram and / or flow chart diagram can be implemented by a system based on special hardware, which performs a specified function or action, or performs a combination of computer instructions, architecture, function and operation that may be realized by special hardware and the system, method and computer program product according to various embodiments of the present invention.

Claims

1. A method for training a machine learning model to detect audio of a target classification, the method comprising: Training a pre-trained audio classification machine learning model on a training data set to predict whether an input audio signal includes audio that satisfies the target classification, wherein the training data set includes at least one audio signal, and the at least one audio signal includes audio that satisfies the target classification; and Characterized in that the training comprises quantization-aware training, wherein at least one learnable value or activation of the model is quantized during training.

2. The method of claim 1, wherein the at least one learnable value comprises at least one of: a node weight; a bias; a batch normalization parameter; a gating or masking parameter; and a threshold.

3. A method according to any one of the preceding claims, wherein the target classification includes a crying baby.

4. The method according to any of the preceding claims, wherein the training data set comprises at least one audio signal that does not satisfy the target classification.

5. The method according to any one of the preceding claims, wherein before training, the method further comprises converting the audio signals of the training dataset into floating point tensors.

6. The method according to any one of the preceding claims, wherein before training, the method further comprises resampling the audio signal of the training data set to a 16 kHz mono audio signal.

7. The method according to any one of the preceding claims, wherein the method further comprises acquiring the training data set.

8. A computer program comprising code means for implementing a method according to any one of the preceding claims when said program is run on a processing system.

9. A computer program comprising code means for implementing a machine learning model trained according to the method of any one of claims 1 to 7.

10. A device comprising a machine learning model trained according to the method of any one of claims 1 to 7.

11. The device of claim 10, wherein the device is configured to output a control signal in response to detecting audio of the target category.

12. The apparatus according to claim 11, wherein the control signal comprises a signal for outputting an alarm.

13. The device according to any one of claims 10 to 12, wherein the device is configured to classify the audio signal in real time.

14. A device according to any one of claims 10 to 13, wherein the device comprises a baby monitor.

15. The device of any one of claims 10 to 13, wherein the device comprises a processor.

Images