System and method for using spectrogram to monitor eating activity

Abstract

A system for monitoring a user's eating activity includes a wearable device having at least one audio input unit, the at least one audio input unit being configured to record an audio sample corresponding to audio from the user's neck. The system further includes a processor configured to execute programming instructions stored in a memory to obtain an audio sample from the audio input unit of the wearable device, determine a segmented feature value of a selected feature set from the audio sample by extracting short-term features in the selected feature set from the audio sample and determining the segmented feature value of the selected feature set from the extracted short-term features. The processor is further configured to use a classifier to classify eating activities based on the determined segmented feature values ​​of the audio sample, and generate an output corresponding to the classified eating activities.

Description

[0001] Priority claim

[0002] This application claims priority to U.S. Provisional Application Serial No. 62 / 637,800, filed on March 2, 2018, entitled “Wearable Neckband Dietary Activity Monitoring Apparatus and Method,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to dietary monitoring, and more particularly to wearable systems and methods for monitoring and classifying dietary activity. Background Art

[0004] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

[0005] Known wearable dietary activity monitoring systems have attracted increasing attention because the systems can track an individual's daily intake, food amounts, associated calories, and nutrient data when dietary tracking is active. Wearable dietary activity monitoring systems include: (a) image / camera-based systems, (b) wrist-worn systems, (c) neck-worn systems, and (d) earbud-based systems.

[0006] Image / camera based systems analyze photos of food that an individual is consuming and aim to automatically detect the number of calories consumed. However, image / camera based systems cannot confirm whether an individual has fully or partially consumed the food without human feedback.

[0007] In another example, a wristband-based system focuses on the individual's gestures and hand movements to detect eating / drinking activities. Again, such a system cannot confirm whether the individual has fully or partially consumed food without manual feedback (e.g., self-reported food consumption) and is therefore highly inaccurate. On the other hand, throat-worn and earplug-based systems are based on jaw muscle movement and voice activity detection. However, conventional systems cannot accurately monitor the amount of food ingested and calories consumed.

[0008] Generally speaking, head-mounted systems include a stethoscope or a vibration sensor mounted within a housing. A stethoscope is able to capture breathing sounds, but is unable to detect the high-frequency characteristics of captured sounds. As such, the accuracy of stethoscope-based systems is low. Conventional vibration sensor-based systems have low sensitivity and may generate signals when the user moves. As such, the recorded vibrations are inaccurate and cannot be used to accurately classify captured activity.

[0009] Therefore, a wearable eating activity monitoring system that can accurately classify chewing and swallowing sounds and monitor food intake and calories would be beneficial. Summary of the invention

[0010] In one embodiment, a system for monitoring a user's eating activity includes a wearable device having at least one audio input unit, the at least one audio input unit being configured to record an audio sample corresponding to audio from the user's neck. The system further includes a processor configured to execute programming instructions stored in a memory to obtain an audio sample from the audio input unit of the wearable device, determine a segmented feature value of a selected feature set from the audio sample by extracting short-term features in the selected feature set from the audio sample and determining the segmented feature value of the selected feature set from the extracted short-term features. The processor is further configured to use a classifier to classify the eating activity based on the determined segmented feature value of the audio sample, and generate an output corresponding to the classified eating activity.

[0011] In some embodiments, the audio input unit includes one or more of an electret condenser microphone, a silicon microphone, a micro-electromechanical system (MEMS) microphone, a dynamic microphone, a microphone array, and a vibration sensor.

[0012] In some embodiments, the system includes an external device including a processor and a memory, and the wearable device further includes a communication module. The processor is further configured to obtain the audio sample via wireless communication with the communication module.

[0013] In another embodiment, the processor is disposed in a wearable device.

[0014] In yet another embodiment, the wearable device is a neckband configured to be worn around the neck of a user.

[0015] In further embodiments, the selected feature set and classifier parameters of the classifier stored in the memory are determined during a machine learning training process.

[0016] In one embodiment of the system, the classifier is one of a random forest, a Gaussian mixture model, a linear discriminant analysis, a naive Bayes, a decision tree, and a k-nearest neighbor classifier. The determination of the segment feature value may further include segmenting the extracted short-term features into segments of predetermined duration, and applying a statistical function to the extracted short-term features in each segment.

[0017] In some embodiments, the short-term features include one or more of the following: at least one MFCC coefficient, delta of at least one MFCC coefficient, delta-delta of at least one MFCC coefficient, energy entropy, short-term energy, zero crossing rate, spectral flux, spectral peak, spectral skewness, spectral centroid, spectral slope, spectral drop, spectral spread, spectral roll-off, and spectral flatness. The statistical function may include one or more of mean, median, standard deviation, skewness, kurtosis, maximum, and minimum.

[0018] An embodiment of the system further includes an output device configured to generate a perceptible output, wherein the processor is configured to communicate with the output device to generate the output as the perceptible output.

[0019] In another embodiment, the generating of the output includes storing the output in a memory.

[0020] In some embodiments, the processor is further configured to detect whether activity occurs in each frame of the audio sample by comparing the energy of the frame to a predetermined energy threshold before determining the segment feature value of the selected feature set.

[0021] In another embodiment, a method for monitoring eating activities includes: obtaining an audio sample from an audio input unit of a wearable device using a processor, the audio sample corresponding to audio from the user's neck; determining a segmented feature value of a selected feature set from the audio sample using the processor, the determination of the segmented feature value including: extracting short-term features in the selected feature set from the audio sample; and determining the segmented feature value of the selected feature set from the extracted short-term features; using a classifier using the processor to classify eating activities based on the determined segmented feature values ​​of the audio sample; and generating an output corresponding to the classified eating activities.

[0022] In some embodiments of the method, the audio input unit includes one or more of an electret condenser microphone, a silicon microphone, a micro-electromechanical system (MEMS) microphone, a dynamic microphone, a microphone array, and a vibration sensor.

[0023] In another embodiment, the obtaining of the audio samples includes obtaining the audio samples by wireless communication via a communication module of the wearable device, and the processor is disposed in the external device.

[0024] In further embodiments, the processor is mounted in a wearable device.

[0025] In some embodiments of the method, the wearable device is a neckband.

[0026] The method may be performed such that the selected feature set and classifier parameters of the classifier are determined during a machine learning training process.

[0027] The classifier of the method can be one of a random forest, a Gaussian mixture model, a linear discriminant analysis, a naive Bayes, a decision tree, and a k-nearest neighbor classifier. The determination of the segmented features can include segmenting the extracted short-term features into segments of a predetermined duration, and applying a statistical function to the extracted short-term features in each segment. The short-term features can include one or more of the following: at least one MFCC coefficient, a delta of at least one MFCC coefficient, a delta-delta of at least one MFCC coefficient, energy entropy, short-term energy, zero crossing rate, spectral flux, spectral peak, spectral skewness, spectral centroid, spectral slope, spectral drop, spectral spread, spectral roll-off, and spectral flatness. The statistical function can include one or more of a mean, a median, a standard deviation, a skewness, a kurtosis, a maximum, and a minimum.

[0028] In another embodiment of the method, the generating of the output includes generating a perceptible output via an output device.

[0029] A wearable neckband according to the present disclosure includes: at least one audio input unit, which includes one or more of an electret condenser microphone, a silicon microphone, a microelectromechanical system (MEMS) microphone, a dynamic microphone, a microphone array, and a vibration sensor configured to record audio samples corresponding to audio from a user's neck; a communication module, which is configured to wirelessly transmit the recorded audio samples to an external device; an energy storage unit, which is configured to store power for operation of the at least one audio input unit and the communication module; and a shell, in which the at least one audio input unit, the communication module, and the energy storage unit are arranged, the shell being elastically deformable and including a fastening arrangement at each end of the shell. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] These and other features, aspects and advantages of the present disclosure will become better understood when the following detailed description of certain exemplary embodiments is read with reference to the accompanying drawings, wherein like symbols refer to like technology throughout the drawings, and wherein:

[0031] Figure 1 is a schematic illustration of a dietary activity monitoring system in which a wearable device communicates with an external device.

[0032] Figure 2is a schematic illustration of a dietary activity monitoring system in which a wearable device includes a processor and a memory.

[0033] Figure 3 is a simplified perspective view of an exemplary wearable neckband dietary activity monitoring device according to an embodiment of the present disclosure.

[0034] Figure 4 is a simplified perspective view of an exemplary wearable neckband dietary activity monitoring device according to an embodiment of the present disclosure.

[0035] Figure 5 Is training and / or calibration Figure 1 Process diagram of a method of a dietary activity monitoring system.

[0036] Figure 6 is used Figure 1 Process diagram of a method for classifying eating activities by an eating activity monitoring system.

[0037] Figure 7a is a perspective view of an experimental embodiment of an eating activity monitoring system.

[0038] Figure 7b yes Figure 7a Schematic block diagram of a dietary activity monitoring system.

[0039] Figure 8a Before any parts are attached to the elastic band, Figure 7a Schematic view of an experimental embodiment of .

[0040] Figure 8b After the electret condenser microphone, Bluetooth® module and battery are attached to the elastic strip, Figure 8a Schematic view of an elastic strip.

[0041] Figure 8c is attached to Figure 8b Schematic view of a housing component of an elastic strip.

[0042] Fig. 9 Shown are spectrograms of water drinking and biscuit chewing recordings acquired using the experimental device.

[0043] Fig.10 is a simplified schematic diagram illustrating the relationship between the neck strap and the training and assessment process.

[0044] Fig.11 is a bar chart showing the performance of different machine learning algorithms from experiments used in a 5-class classification task.

[0045] Fig.12 is a diagram depicting one form of computing shifted delta cepstral coefficients. DETAILED DESCRIPTION

[0046] For the purpose of promoting an understanding of the principles of the embodiments described herein, reference is now made to the drawings and the description in the following written specification. The reference is not intended to impose any limitation on the scope of the subject matter. The present disclosure also includes any changes and modifications to the illustrated embodiments, and includes further applications of the principles of the described embodiments as would normally occur to a person skilled in the art to which this document belongs.

[0047] In a manner that is most helpful for understanding the claimed subject matter, various operations may be described in turn as multiple discrete actions or operations. However, the order of description should not be interpreted as implying that these operations must rely on the order. In particular, these operations may not be performed in the order presented. The described operations may be performed in an order different from the described embodiments. In additional embodiments, various additional operations may be performed and / or the described operations may be omitted.

[0048] The terms “comprising,” “including,” “having,” and the like as used with respect to embodiments of the present disclosure are synonymous. As used herein, the term “approximately” refers to a value within ±20% of a reference value.

[0049] Figure 1 A schematic illustration of an eating activity monitoring system 100 having a wearable device is illustrated, which in the illustrated embodiment is a neckband monitoring device 120 configured to monitor and track a user's dietary intake. As described herein, the monitoring device 120 is configured to be worn around the neck of a human being to monitor the human's food and drink consumption. In some embodiments, the monitoring device 120 can be worn at various locations on the user's body where chewing and swallowing noises can be sensed, such as on the chin, head, mouth, cheeks, chest, or other desired locations. Furthermore, the reader should appreciate that in some embodiments, the wearable device can be configured to be worn on an animal (e.g., around the animal's neck) to track the animal's food and drink consumption. Additionally, while Figure 1 The monitoring device 120 is depicted as a neckband, but the reader should appreciate that the monitoring device may be configured as another wearable device, such as, for example, a necklace, a pendant, etc.

[0050] The monitoring device 120 includes a housing 124 having first and second ends 128, 130. In some embodiments, the housing 124 can be a single continuous main body housing, while in other embodiments, the housing 124 can include multiple layers of continuous main body housing, multiple joint sections attached together to form a continuous main body housing, or multiple housing parts forming a discontinuous housing. The housing 124 is formed of an elastic stretchable material that can be formed to the neck of an individual, and in some embodiments can be an elastic strip covered in velvet or another comfortable and wearable fabric. In one embodiment, the housing 124 can be formed of a biomaterial or a biocompatible material. In some embodiments, the housing 124 is formed of a shaped polymer that is elastic, stretchable and configured to return to its original shape and length. In another embodiment, the housing 124 is made of a bendable and foldable material when the monitoring device 120 is placed, so that the monitoring device 120 can be easily stored when the user is not wearing the monitoring device 120.

[0051] The locking or fastening assembly 132 is incorporated into the ends 128, 130 of the housing 124. As illustrated, the locking or fastening assembly 132 is configured as a hook and loop type fastener system (e.g., Velcro®), wherein one end (e.g., end 128) has a hook portion and the other end (e.g., end 130) has a loop portion. In other embodiments, the locking or fastening assembly 132 is another desired fastening arrangement, such as a magnetic connector, a button fastener, a knot, a clasp such as a lobster clasp, a spring ring, a bayonet clasp, or a barrel clasp, etc. In another embodiment, the locking or fastening assembly 132 is formed as a permanent connection between the ends 128, 130, so that the user puts on and takes off the monitoring device 120 by elastically stretching the monitoring device 120 to fit the monitoring device 120 over the user's head.

[0052] The communication module 136, the audio input unit 140, and the energy source 144 are mounted to or within the housing 124. Other computer-implemented modules, such as computer-readable media, such as the memory 152 (e.g. Figure 2 ); Input and / or output interface 156 (eg Figure 2 ), such as a display, a touch screen, a track pad, a switch, a button, an indicator, and / or one or more LEDs; a digital signal processing unit (DSP) 148 (e.g. Figure 2 ); and any other desired modules may be incorporated into or onto the housing 124.

[0053] The communication module 136 is configured to communicate with one or more external devices 160, such as machine equipment, networks, servers, and / or other devices, to transmit parameters and data collected by the monitoring device 120. As used herein, a smart device refers to a cellular phone or smart phone, a tablet computer, a smart watch, etc. The communication module 136 can be configured for any suitable wireless communication protocol, such as, for example, Bluetooth®, near field communication, Internet data communication, wireless telephone network, Wi-Fi®, ZigBee®, Z-Wave®, Thread®, ultrasonic protocol, etc. In some embodiments, the communication module 136 can be configured for wired data transmission via, for example, USB, Ethernet, or another suitable wired data connection protocol. As illustrated, the communication module 136 is a Bluetooth® transmitter or transceiver.

[0054] The machine device may be one or more of the following: a cellular device, such as a smart phone; a portable device; a wearable device, such as a watch, glasses, or goggles; a laptop computer; a desktop computer, a tablet computer; an entertainment system, such as a television; a voice-activated device with or without a display, such as a digital personal assistant (e.g., Alexa®, Google Assistant®, Siri®, Cortana®, etc.), and any suitable machine device capable of receiving, storing, and / or displaying information collected by the monitoring device 120. The network may be one or more communication networks of any suitable type in any combination, including a wireless network, a wired network, a local area network, a wide area network, a cellular data network, the Internet, a cloud network, and the like. The server may be implemented by any configuration or any configuration of one or more computer machines, such as a remote server computer, and the like.

[0055] In the illustrated embodiment, the audio input unit 140 includes a single electret capacitor microphone (ECM) capable of capturing swallowing and chewing sound activity during food intake events. In other embodiments, the audio input unit 140 may include one or more of a silicon microphone, a micro-electromechanical system (MEMS) microphone, a dynamic microphone, a microphone array, a vibration sensor (e.g., a piezoelectric sensor), etc. In further embodiments, the audio input unit 140 may include multiple electret microphones, multiple other types of audio sensors, or a combination of both. In some embodiments, the audio input unit 140 or a separate input unit may include a motion sensor, an accelerometer, a camera, and / or any other desired input unit.

[0056] The energy source 144 may be, for example, a rechargeable battery. In one particular embodiment, the energy source is a 100 mAh Li-based battery. The energy source 144 may be charged, for example, by a wired connection (e.g., USB® or a dedicated AC or DC power source), inductive charging (e.g., Qi charging), RF charging, an energy harvesting system, etc.

[0057] The eating activity monitoring system 100 further includes a processor 168 operably connected to a memory 172. Figure 1 In the embodiment of the present invention, the processor 168 and the memory 172 are arranged in the external device 160, and the external device 160 includes a transceiver 164 that communicates with the communication module 136 of the monitoring device 120. Figure 2 In another embodiment schematically illustrated in FIG. 1 , the processor 168 and the memory 172 are integrated in or on the housing 124 of the monitoring device 120. In some embodiments, the processor 168 may be integrated in the communication module 136 and / or the audio input unit 140. In other embodiments, the monitoring device 120 and the external device 160 each have a processor, which may jointly or individually perform data processing on the data collected by the audio input unit 140.

[0058] Those skilled in the art will recognize that a "processor" includes any hardware system, hardware mechanism, or hardware component that processes data, signals, or other information. Processor 168 may include a system with a central processing unit, multiple processing units, dedicated circuits for implementing functionality, or other systems.

[0059] The memory 172 may be any type of device capable of storing information accessible by the processor 168, such as a writable memory, a read-only memory, a memory card, a ROM, a RAM, a hard drive, a disk, a flash memory, or other computer-readable medium. The memory 172 is configured to store program instructions that, when executed by the processor 168, enable the processor 168 to perform one or more of ingestion activity detection, feature extraction, and classification, as described in further detail below.

[0060] In particular, the memory 172 is configured to store program instructions corresponding to at least one machine learning model, in particular an intake classification model and its classification parameters. The processor 144 is configured to extract features from one or more audio signals using the intake classification model and classify the user's consumption based on the audio signals. As used herein, the term "machine learning model" refers to a system or set of program instructions and / or data configured to implement an algorithm or mathematical model that predicts and provides a desired output based on a given input. It will be appreciated that the parameters of the machine learning model are not explicitly programmed, or that the machine learning model is not necessarily designed to follow specific rules in order to provide the desired output for a given input. Instead, the machine learning model is provided with a corpus of training data from which the processor identifies or "learns" patterns and statistical relationships or structures in the data that are generalized to make predictions relative to new data inputs. The classification parameters include multiple values ​​of the parameters of the intake activity classification model learned during the training process.

[0061] Figure 3 and 4 A simplified perspective view of an exemplary wearable neckband activity monitoring device 120a, 120b is shown. Figure 3 In the embodiment of the present invention, the housing 124a includes a woven elastic fabric. The communication module 136, the audio input unit 140 and the energy source 144 are attached to and encapsulated in the woven elastic fabric to protect the communication module 136, the audio input unit 140 and the energy source 144.

[0062] exist Figure 4 In an embodiment, the housing 124b includes a fabric tube covered with an elastic band. The fabric tube may be, for example, velvet, cloth, or other comfortable fabric. Figure 4 In the embodiment of FIG. 1 , the audio input unit 140 and the energy source 144 are arranged in the fabric tube, while the communication module 136 is mounted to the outside of the elastic tube.

[0063] System training

[0064] Figure 5 A machine learning process 200 is illustrated for training a machine learning ingestion classification system such as Figure 1 and Figure 2The dietary activity monitoring system 100 of the present invention is used to classify the user's intake. In the description of the method, the statement that the method is performing a certain task or function refers to a controller or a general-purpose processor executing programming instructions stored in a non-transitory computer-readable storage medium operatively connected to the controller or processor to manipulate data or operate one or more components in the dietary activity monitoring system 100 to perform the task or function. In particular, the processor 168 of the dietary activity monitoring system 100 described above can be such a controller or processor. Alternatively, the controller or processor can be implemented using more than one processor and associated circuits and components, each of which is configured to form one or more tasks or functions described herein. It will be appreciated that some or all of the operations of the method can also be performed by a remote server or cloud processing infrastructure. Additionally, the steps of the method can be performed in any feasible time order, regardless of the order shown in the figures or the order used to describe the steps.

[0065] Process 200 begins by obtaining an audio sample (block 204). In one embodiment, the audio sample is sensed by a single audio sensor (e.g., an audio input unit or an electret microphone 140). The processor communicates with the one or more audio sensors to receive a time series of acoustic values ​​corresponding to the detected audio. The audio sample may include a data set of a plurality of known swallowing and speaking activities from a plurality of subjects detected by the audio input unit 140 and stored in a memory associated with the processor (e.g., memory 172).

[0066] In some embodiments, the extraction of audio features begins with an activity detection process (block 206). The audio signal is divided into frames, for example, between approximately 10 ms and approximately 100 ms, or in another embodiment, approximately 40 ms. The processor then determines the audio energy of the frame (i.e., the sum of the second powers of the signal) according to the following equation:

[0067]

[0068] If the energy is below a predetermined threshold, the audio frame is discarded as not containing relevant eating activity. If the energy is above a predetermined threshold, it is assumed that there is some activity detected in the frame, and the audio sample is retained for further processing.

[0069] When the method 200 proceeds with extracting features from the audio samples (block 208), the processor begins by separating the audio samples into short-term frames. In one embodiment, the frames are between approximately 5 ms and approximately 500 ms. In another embodiment, the frames are between approximately 20 ms and approximately 80 ms. In a specific embodiment, the frames are approximately or exactly 40 ms. In one embodiment, the frames may be non-overlapping. In another embodiment, the frames may overlap between approximately 25% and approximately 90%. In a specific embodiment, the frames overlap by approximately or exactly 75%.

[0070] Feature extraction continues by determining short-term features for each frame. Table 1 illustrates a list of short-term features that may be extracted in one embodiment of process 200.

[0071] Table 1: Overall short-term characteristics

[0072]

[0073] Mel-frequency cepstral coefficients (MFCCs) and their derivative (also called "delta" or "d") and acceleration (also called "delta-delta" or "dd") coefficients are commonly used in the art to enable automated frequency detection to interpret frequency differences more like the human ear. In one embodiment, 30 filter banks are used to calculate the MFCCs, and the static coefficients of the lower 21 filter banks are retained, along with their associated delta and acceleration coefficients, although the reader should appreciate that other numbers of filter banks may be calculated and retained in other embodiments.

[0074] Using the 63 MFCC coefficients listed in Table 1 and the features based on spectrum, waveform and energy, each frame includes a 76-dimensional feature vector. The reader should appreciate that in other embodiments, other features and / or fewer features can be extracted from the frame. Additionally, in some embodiments, shifted delta cepstral (SDC) coefficients are used instead of delta coefficients. The shifted delta coefficients are a generalized version of the delta coefficients and are extracted by subtracting the feature vector (MFCC) from a longer duration span (d). Fig.12 Illustration of SDC characteristics The parameter d determines the spread within which the delta is calculated, and the parameter P determines the gap between successive delta calculations. For unshifted delta coefficients, d=1 and P=1 are used, while for SDC, the larger value of d and P is used. In one embodiment, the SDC for a diet monitoring system may be d=2 and P=2.

[0075] The process 200 continues by partitioning the features into segments of fixed time length (block 212). When the features are segmented, the extracted feature vectors are grouped together over the segment length to enable statistical functions to be performed for each segment. In some embodiments, the segments completely overlap, i.e., the segments have an offset of only one frame from adjacent segments. In other embodiments, the segments may only partially overlap, such that each segment is displaced from a temporally adjacent segment by 5, 10, 25, or 50 frames, or by any other desired number of frames. In further embodiments, the segments may not overlap each other.

[0076] The segments may have a length, for example, between 0.1 seconds and 60 seconds. In one embodiment, the segments have a length between 1 second and 10 seconds. In some embodiments, features are partitioned into segments of approximately or exactly 3 seconds. In other embodiments, features may be partitioned into different segment lengths depending on the statistical function applied to the segmentation, the desired accuracy, available computing resources, and other parameters. Further, in some embodiments, the processor analyzes different segment lengths of the features to investigate an optimal time window for a particular ingest determination. The optimal time window may be based on features in the audio signal and / or features of the individual or task being analyzed.

[0077] Next, process 200 proceeds with the processor applying a statistical function to the segment features determined for each segment (block 216). Table 2 illustrates the statistical function for one exemplary embodiment. However, the reader should appreciate that other functions and / or fewer functions may be applied to the segment features.

[0078] Table 2: Statistical functions used for segmentation features

[0079]

[0080] The processor may be configured to determine each function of each segment as a determined piecewise function.Alternatively, the processor may be configured to determine a limited number of piecewise functions to reduce computational resources necessary for determining the piecewise functions.

[0081] The results of applying the seven statistical functions of Table 2 on the 76-dimensional feature vector generated using the features of Table 1 result in a 532-dimensional segmented feature vector. After the segmented features are determined, a feature selection step is performed to reduce the number of features to a selected feature set (block 220). In one embodiment, the features are selected according to the ReliefF method. In other embodiments, other known feature selection methods may be used to select a feature set of coefficients that are most relevant to the determination of the ingestion classification. In one embodiment, the ReliefF method is used to select a feature set of approximately or exactly 76 features, although other numbers of features may be selected in other embodiments.

[0082] The process ends by classifying the selected features (block 224). In one embodiment, the processor is configured to implement a random forest classifier on the selected features. The random forest classifier can be generated using approximately or exactly 76 bags per tree and uses a classification decision tree approach. Bagging is an ensemble learning method that can be used with any machine learning (ML) algorithm. In the illustrated embodiment, the ML algorithm is a random forest and the ensemble learning method is bagging. In particular, the training data is randomly sampled with replacement, and the classification algorithm (e.g., random forest) is run on each subset or bag of the samples, and the bags are then combined by averaging the results from each subset.

[0083] In another embodiment, a Gaussian mixture model (GMM) may be used to determine the classifier, which has, for example, approximately 16 mixture components and approximately 50 iterations. In another embodiment, a linear discriminant analysis with a linear kernel and Bayesian optimization may be used to determine the classifier. In yet another embodiment, a naive Bayes classifier with a normal distribution kernel and a maximum objective evaluation of approximately 30 may be used to determine the classifier. In some embodiments, the processor determines a decision tree classifier with a maximum grid division of 10. In another embodiment, a k-nearest neighbor classifier with approximately 5 neighbors is used and a kd tree neighbor search method is used to determine the classifier. The reader should appreciate that other classifier models may also be used in other embodiments.

[0084] In some embodiments, the training process 200 is performed on a computer or server, and the resulting features and classifiers are stored in the memory 172 of the eating activity monitoring system 100. In such embodiments, the eating activity monitoring system 100 can be easily adapted to a variety of different uses with reduced expense and installation costs because the results of the training process can be pre-installed in the eating activity monitoring system 100.

[0085] In another embodiment, when the eating activity monitoring system 100 is used for the first time, the training process is performed as a system calibration. The training data can then be stored in the memory 172 for subsequent use. In addition, the training process can be based on both pre-installed data and the training data obtained during the calibration step. Therefore, the machine learning algorithm of the eating activity monitoring system 100 can be customized for individual users while also including sufficient data to produce accurate results. In such an embodiment, because the features and classification parameters are based on the specific characteristics of the user who is to use the eating activity monitoring system 100, a high degree of accuracy can be obtained.

[0086] Ingest activity detection and monitoring

[0087] Figure 4 A flowchart of a process 300 for monitoring dietary activity is illustrated. The process 300 refers to a processor (e.g., processor 168) executing programming instructions stored in a memory (e.g., memory 172) to perform the functions described below to classify dietary activity. In the description of the method, the statement that the method is performing a certain task or function refers to a controller or a general processor executing programming instructions stored in a non-transitory computer-readable storage medium operatively connected to the controller or processor to manipulate data or operate one or more components in the dietary activity monitoring system 100 to perform the task or function. In particular, the processor 168 of the above-mentioned dietary activity monitoring system 100 can be such a controller or processor. Alternatively, the controller or processor can be implemented using more than one processor and associated circuits and components, each of which is configured to form one or more tasks or functions described herein. It will be appreciated that some or all of the operations of the method can also be performed by a remote server or cloud processing infrastructure. Additionally, the steps of the method can be performed in any feasible time order, regardless of the order shown in the figures or the order used to describe the steps. As an example, in some embodiments, process 300 may be performed by external device 160 , while in other embodiments, process 300 may be performed by wearable neckband monitoring device 120 .

[0088] In some embodiments, process 300 begins by calibrating or training the system (block 304). For example, Figure 5 The training is performed by process 200 of the machine learning diet classification system. The short-term features, segmentation features, and / or classifier parameters (e.g., random forest) selected from the calibration or training of the system are stored in a memory (e.g., memory 172) associated with the processor 168. In other embodiments, the system can be pre-programmed with selected short-term features, selected segmentation feature sets, and / or classification algorithms so that some or all of the calibration or training of the machine learning diet classification system is not necessary.

[0089] The process then proceeds with the processor obtaining audio samples using an audio input unit, such as an electret microphone (block 308). In one embodiment, the audio samples are sensed by a single audio sensor, such as the audio input unit 140 of the wearable monitoring device 120. In another embodiment, the audio samples are sensed by multiple audio sensors of one audio input unit or multiple audio input units. The processor communicates with the one or more audio sensors to receive a time series of acoustic values ​​corresponding to the audio detected by the wearable monitoring device 120. The processor obtains the sensed audio samples from the (one or more) audio sensors via a direct connection or via communication through a wired or wireless network. When the wearable monitoring device 120 is powered on, the processor's acquisition of audio samples may occur continuously, and the processor may be configured to obtain multiple audio samples in batches. In a specific embodiment, the audio samples are obtained in real time using three second buffers. In other embodiments, the audio data may be stored in a memory, such as the memory 172, and the processor may obtain the stored audio data, such as in a "synchronous" process, when the processor is physically connected to the audio input unit and / or when prompted by a user.

[0090] Since the audio input unit can continuously collect audio data, in some embodiments, the extraction of audio features begins with an activity detection process (box 310). Similar to the above-described activity detection process (box 206), the processor is configured to divide the audio samples into frames, for example, between approximately 10 ms and approximately 100 ms, or in another embodiment, approximately 40 ms. The processor then determines the audio energy of the frame, and if the energy is below a predetermined threshold, the audio frame is discarded because it does not contain relevant eating activities. If the energy is above the predetermined threshold, it is assumed that there is some activity detected in the frame, and the audio sample is retained for further processing. The activity detection process reduces the computing resources used by the system 100 because audio samples during which no activity occurs are discarded with minimal processing.

[0091] Next, the processor determines the selected short-term features from the audio samples (box 312). Short-term feature extraction is performed in a similar manner as in the short-term feature extraction step 208 described above, except that only the short-term features selected during the calibration and training process, such as in the above-mentioned step 304 or training process 200, are extracted. The feature frames can have a length similar to the feature frames used during the above-mentioned feature extraction step, or the feature frames can have a different length. In one embodiment, the frame is between approximately 5 ms and approximately 500 ms. In another embodiment, the frame is between approximately 20 ms and approximately 80 ms. In a specific embodiment, the frame is approximately or exactly 40 ms. In one embodiment, the frames may not overlap. In another embodiment, the frames may overlap between approximately 25% and approximately 90%. In a specific embodiment, the frames overlap by approximately or exactly 75%.

[0092] In contrast to the training process 200 described above, the process 300 for classifying eating activities is limited in the number of short-term features determined. In particular, in the process 300, only the short-term features required to calculate the segmentation features of the selected feature set are determined. However, the reader should appreciate that any desired number of short-term features may be used, depending on the desired accuracy and available computing resources. Additionally, the selected short-term features may vary based on the data received from executing the machine learning model. The selected short-term features may be those short-term features that provide the greatest amount of information relevant to the classification of the ingestion activity.

[0093] The process continues with segmenting and extracting segment features from the extracted short-term features (block 316). Again, the processor is configured to segment the extracted short-term feature vectors into groups of predetermined time lengths so that a statistical function can be performed for each segment. In some embodiments, the segments overlap completely, i.e., the segments have only one frame offset from adjacent segments. In other embodiments, the segments may only partially overlap, such that each segment is shifted from a temporally adjacent segment by 5, 10, 25, or 50 frames, or by any other desired number of frames. In further embodiments, the segments may have no overlap with each other.

[0094] The segments can have a length of, for example, between 0.1 seconds and 60 seconds. In one embodiment, the segments have a length of between 1 second and 10 seconds. In some embodiments, features are partitioned into segments of approximately or exactly 3 seconds. In other embodiments, features can be partitioned into different segment lengths depending on the statistical function applied to the segmentation, the desired accuracy, the available computing resources, and other parameters. In addition, in some embodiments, the processor analyzes different segment lengths of the features to study the optimal time window determined for a particular ingestion. The optimal time window can be based on features in the audio signal and / or features of the individual or task being analyzed.

[0095] Segment features are then determined from the segments by applying statistical functions to the segment feature vectors in a similar manner as above in step 216. However, only those segment features stored as a selected feature set are determined, thereby limiting the computational resources necessary for the calculation.

[0096] Finally, the process ends by classifying the audio sample (block 320). As discussed above, a machine learning model (such as Figure 5 A classifier is developed using a machine learning training process (a machine learning training process). In one embodiment, the classifier is a random forest classifier, although other classifiers may be used in other embodiments, such as Gaussian mixture models, linear discriminant analysis, naive Bayes, decision trees, and / or k-nearest neighbor classifiers. The extracted selected segment features are input into a classifier, which determines the ingestion activities that may be recorded in the recorded audio sample based on the segment features. In one embodiment, the classifier output may be a probability of a certain ingestion activity based on the recorded audio. In another embodiment, the classifier output may be the most likely specific activity.

[0097] The method 300 continues by generating an output based on the determination of the occupancy (block 324). In some embodiments, the output is based on matching the audio signal with known food or drink types and amounts in a database stored in the memory 172 or in a remote memory or server based on the classifier results. In one embodiment, the output may be a perceptible output depicted on a user interface regarding the ingestion activity detected using the classifier, such as a screen on the external device 160 or the wearable monitoring device 120. The perceptible output may include an indication of the ingestion activity detected and / or an estimated amount of calories or nutritional facts (e.g., macronutrients and / or micronutrients, etc.) in the ingested food or drink. In other embodiments, the perceptible output may be an audible indicator, such as an audio alert specifying the detected ingestion activity and / or the estimated amount of calories or nutritional facts in the activity.

[0098] In other embodiments, the output is an electronic signal that is transmitted to another electronic device or stored in a memory or storage 172. For example, the output can be an electronic signal that is output to external device 160 or a different external device. In some embodiments, the output is stored in memory 172, and multiple outputs are aggregated over a specified time period (e.g., a day). In another embodiment, the output is stored in memory 172 and can be recalled by the user when desired, so that the user can track his or her nutritional input over a specified time period.

[0099] The disclosed eating activity monitoring system 100 provides many improvements to computer and nutrition or eating activity tracking and monitoring technology by providing an efficient and cost-effective way to track eating and nutritional activities. In one embodiment, the neckband monitoring device 120 requires only readily available and inexpensive components, such as a microphone (particularly an electret microphone), a battery, and a transmitter (particularly a Bluetooth® transmitter). Therefore, the eating activity monitoring system 100 can be produced inexpensively, thereby enabling low-cost tracking of eating activities, which is an improvement over conventional eating activity tracking technology.

[0100] In addition, because the dietary activity monitoring system 100 automatically detects nutritional activity, nutritional tracking is accomplished with minimal user input. Because many conventional diet and nutritional tracking systems require user input, nutritional profiles are often incomplete because users may forget or neglect to record some activities, or may stop tracking nutritional activities altogether due to the time required to enter activities. The automatic recording of nutrition and dietary activity by the disclosed dietary activity monitoring system 100 thus provides a more complete nutritional record for the user. As such, the disclosed dietary activity monitoring system 100 is a further improvement in dietary tracking technology.

[0101] Furthermore, in many conventional automated nutrition tracking systems, the system is not able to confirm the amount of food or drink consumed by the user. Since the disclosed eating activity monitoring system 100 detects nutrition activity based on chewing and swallowing sounds, the eating activity monitoring system 100 can more accurately detect the amount of food or drink consumed. Therefore, for this additional reason, the eating activity monitoring system 100 is a further improvement over conventional diet tracking technology.

[0102] Other conventional nutrition tracking systems operate based on limited frequencies or sensitivities. In particular, the use of an electret microphone provides reduced sensitivity to contact with the user's skin as compared to a stacked piezoelectric plate microphone. Since the sensitivity to the user's skin results in undesirable readings and sound distortion during movement of the user's head, the electret microphone provides improved audio signal capture. Additionally, it is difficult or impossible to accurately capture swallowing signals using a piezoelectric microphone sensor, and the disclosed electret sensor thereby provides improved audio detection. As such, the disclosed eating activity monitoring system 100 is an improvement over conventional nutrition tracking systems because the eating activity monitoring system 100 sensitively detects a wide range of frequencies, thereby providing improved accuracy over conventional systems.

[0103] Experimental Results

[0104] Experiments were performed using an embodiment of the eating activity monitoring system 100. Figure 7a, the experimental system includes a neckband 420 designed using a housing 424 that includes an elastic band covered with velvet fabric. The housing 424 further includes adjustable hook and loop strips 428 on both ends. The fabric-based design is inexpensive to implement and provides comfort to the user to ensure that the user can wear the device comfortably all day.

[0105] An exemplary embodiment of the neckband 420 is manufactured by first cutting the elastic strip into the desired length, such as Figure 8a In one embodiment, the elastic strip 422 is cut to a length of approximately 38 cm without elasticity, which can stretch to approximately 45 cm under strain. The velvet fabric 425 is cut to a length of approximately 26 cm, while the hook and loop ends are cut to a length of approximately 6 cm per end (i.e., a hook portion 426 at one end and a loop portion 428 at the opposite end).

[0106] Adhere the electret microphone 440 to the elastic strip (eg, using glue) at approximately or exactly the center of the elastic strip. Figure 8b ). The electret microphone 440 is connected to a Bluetooth® 4.0 module 436, which in the illustrated embodiment is a F-6888V1 Bluetooth 4.0 module powered by a rechargeable 100 mAh lithium-ion battery 444. The Bluetooth® module 436 and battery 444 are adhered to an elastic band inside the fabric neckband with at least 2 cm of separation between each element 440, 436, 444. For example, the electret microphone may have a radius of, for example, 2.2 mm, with a sensitivity range of ~44 dB to ~66 dB and a maximum current rating of 0.5 mA. A power button is attached and glued to the Bluetooth module to enable the user to power the neckband on and off.

[0107] The hook and loop ends 426, 428 are attached to the elastic strip 422, and the velvet fabric 435 is wrapped over a central portion of the elastic strip 422 adjacent to the hook and loop ends 426, 428, thereby forming the shell 424 of the neckband 420, as shown in FIG. Figure 8c As shown in the figure.

[0108] The experimental eating activity data collection was performed in a soundproof facility at the Department of Biomedical Engineering, Bangladesh University of Engineering and Technology (BUET). The facility consisted of a fully soundproofed and anechoic recording room along with an adjacent control room with a soundproof window between them. Each subject performed various nutritional and other tasks in a comfortable sitting position inside the recording room.

[0109] The eating activity dataset was collected from 20 healthy subjects aged between 20 and 25 years of age with gender balance. For drinking activity, each subject consumed 100 ml of water on four different occasions. For solid food, subjects were given 15 g of potato chips (crunchy), 1 club sandwich (soft), and 2 chocolate chip cookies (hard). In order to train the algorithm to be robust to other vocal activities, other non-nutritional activities such as speaking, coughing, laughing, breathing, and dry swallowing were also recorded. The activities recorded in the dataset are summarized in Table 3 below. Fig. 9 4 shows exemplary spectrograms of water drinking and biscuit chewing records acquired using the experimental device 420.

[0110] Table 3: Summary of activities for each subject

[0111]

[0112] To identify frames containing eating activities, the audio signal is first divided into 40 ms frames and the energy within each frame is calculated. If the short-term energy of the frame is above a predefined threshold, it is assumed that there is a related eating activity. The energy threshold is set empirically by observing the energy histogram obtained from a subset of the data.

[0113] Sixty three-dimensional Mel frequency cepstral coefficients (MFCC) were used as the first short-term feature set. A 30-channel Mel filter bank was used, in which 21 static coefficients (including C0 and log energy) were retained. Next, the delta and acceleration coefficients of the MFCC were attached. As summarized in Table 1 above, in addition to the MFCC, several spectrum-, waveform-, and energy-based features were extracted. All short-term features were calculated from 40 ms frames with 75% overlap. The features were then concatenated to the selected feature set to obtain a 76-dimensional vector.

[0114] To detect long-term eating activities (e.g., chewing), feature parameters are observed within a window of several seconds. In the experimental wearable system 420, segmental features are extracted by calculating several statistical functions from short-term features within a 3-second window or segment. From each of the 76-dimensional features in Table 1, the 7 statistical functions summarized in Table 2 (mean, median, standard deviation, skewness, kurtosis, maximum, and minimum) are calculated to obtain segmental features of dimension 532 from each 3-second window. The experimental real-time implementation also utilizes a 3-second audio buffer and provides classification decisions within a 3-second time frame.

[0115] The ReliefF method is used to select a feature subset from the 532-dimensional segmented feature vector to reduce the dimension of the 532-dimensional segmented feature vector to a segmented feature vector containing a set of 76 selected features, each of which is a coefficient determined by applying a statistical function to a segment of the short-term features.

[0116] Experimental data were tested using several different machine learning algorithms for eating activity detection from segmented features extracted from 3 second windows. The classifiers evaluated included Gaussian mixture models (GMM), linear discriminant analysis (LDA), naive Bayes (NB), decision tree (DT), k-nearest neighbor (kNN), and random forest (RF). The classifier parameters used in the experimental examples are summarized in Table 4 below.

[0117] Table 4: Parameters used for the experimental classifiers

[0118]

[0119] Fig.10 The relationship between the neckband 420 and the training and evaluation process is illustrated. To monitor eating activity in real time, the neckband 420 is connected to the PC as a Bluetooth® device. The MATLAB application on the computer starts the device as an audio input, and when eating monitoring starts, the application obtains the audio stream from the device over a 3 second buffer and notifies the user of detected eating / non-eating activities.

[0120] In order to evaluate the effectiveness of the wearable eating activity detection system 420, an experimental setting with a 5-class classification task was designed. The 5 classes included: (i) drinking water (liquid), (ii) eating cookies (hardness), (iii) eating chips (crispyness), (iv) eating sandwiches (softness), and (v) other non-ingestive activities. A 3-class experimental protocol was also designed for additional experiments, in which the three classes included: (i) drinking (liquid), (ii) chewing (solid food), and (iii) other non-ingestive activities.

[0121] In the recorded dataset, the total duration of eating activities is much smaller than the total duration of non-eating activities, which creates an imbalance in the training set. To address this issue, the eating activity features of the eating class are oversampled several times to balance the training dataset.

[0122] The experiments were set up in the form of leave-one-subject-out cross-validation on the data of 20 subjects. For performance evaluation, class-wise precision, recall, F-measure and also the average of these metrics were determined over different classes.

[0123] First, the effect of feature selection is observed for the 5-class classification task. Table 5 shows the average classification results with and without feature selection when using the RF classifier.

[0124] Table 5: Effect of feature selection on a 5-class task using the RF classifier

[0125]

[0126] Fig.11 The performance of different machine learning algorithms from the experiment used in the 5-class classification task is illustrated. The comparison of performance metrics shows that the RF classifier outperforms other classifiers in the experimental dataset. Table 6 shows the detailed RF classifier experimental results, which shows that the RF classifier has an overall accuracy of 86.53% in the 5-class classification experiment.

[0127] Table 6: 5-class experimental results of random forest classifier

[0128]

[0129] The embodiments described above have been shown as examples, and it should be understood that these embodiments may allow various modifications and alternative forms. It should be further understood that the claims are not intended to be limited to the specific forms disclosed, but to cover all modifications, equivalents and substitutes that fall within the spirit and scope of the present disclosure.

[0130] Another experiment was evaluated on the experimental system by using a simpler task of classifying between drinking, solid food, and other activities. The results of the 3-class experiment using the RF classifier are shown in Table 7, which illustrates an overall precision, recall, and F-measure of more than 80% and an accuracy of 92.49% for this experiment.

[0131] Table 7: Results for 3-class classification using random forest classifier

[0132]

[0133] The experimental results of both 3-class and 5-class experiments demonstrate that the wearable neckband eating activity monitoring system accurately detects different types of nutritional activities and further fully distinguishes from other non-ingestive activities (e.g., talking, coughing, laughing). Additionally, the use of a more detailed training data set enables detection of not only the classification of food but also specific foods so as to enable tracking of ingested nutritional content based on chewing and swallowing ingestion sounds.

[0134] It will be appreciated that variations of the above and other features and functions or alternatives thereof may be desirably combined into many other different systems, applications or methods. Various currently unforeseen or unanticipated alternatives, modifications, variations or improvements may be subsequently made by those skilled in the art, which alternatives, modifications, variations or improvements are also intended to be included in the foregoing disclosure.

Claims

1. A system for monitoring a user's eating activities, comprising: A wearable device comprising at least one audio input unit configured to record audio samples corresponding to audio from a neck of a user; A processor configured to execute programmed instructions stored in the memory for: Obtaining an audio sample from an audio input unit of the wearable device; Determining a segmented feature value of a selected feature set from the audio sample, the segmented feature value indicating chewing and swallowing sounds, the determining of the segmented feature value comprising: Extracting short-term features from a selected feature set from the audio sample; and determining segmented feature values ​​of the selected feature set from the extracted short-term features; Using a classifier, classifying the eating activity based on the determined segment feature values ​​of the audio sample, the classification including classification of ingestion of a specific food; and Output corresponding to the classified eating activities is generated.

2. The system of claim 1, wherein the audio input unit comprises an electret condenser microphone.

3. The system according to claim 1, further comprising: External devices including processors and memory, The wearable device further comprises a communication module, and Wherein the processor is further configured to obtain the audio samples via wireless communication with the communication module.

4. The system according to claim 1, wherein: The processor is arranged in the wearable device.

5. The system according to claim 1, wherein: The wearable device is a neckband configured to be worn around the neck of a user.

6. The system according to claim 1, wherein: The selected feature set and classifier parameters of the classifier stored in the memory are determined during the machine learning training process.

7. The system of claim 1, wherein: The classifier is one of the following: random forest, Gaussian mixture model, linear discriminant analysis, naive Bayes, decision tree, and k-nearest neighbor classifier; The determination of the segment characteristic value further includes: segmenting the extracted short-term features into segments of predetermined duration; and A statistical function is applied to the extracted short-term features in each segment.

8. The system of claim 7, wherein: The short-term features include one or more of: at least one MFCC coefficient, a delta of at least one MFCC coefficient, a delta-delta of at least one MFCC coefficient, energy entropy, short-term energy, zero crossing rate, spectral flux, spectral peak, spectral skewness, spectral centroid, spectral slope, spectral drop, spectral spread, spectral roll-off, and spectral flatness; and The statistical functions include one or more of mean, median, standard deviation, skewness, kurtosis, maximum and minimum.

9. The system of claim 1, further comprising: An output device configured to generate a perceptible output, wherein the processor is configured to communicate with the output device to generate the output as the perceptible output.

10. The system of claim 1, wherein the generating of the output comprises storing the output in a memory.

11. The system of claim 1, wherein the processor is further configured to detect whether activity occurs in each frame of the audio sample by comparing the energy of the frame to a predetermined energy threshold before determining the segment feature value of the selected feature set.

12. A method for monitoring eating activities, comprising: Obtaining, using a processor, an audio sample from an audio input unit of the wearable device, the audio sample corresponding to audio from a neck of the user; Determining, using a processor, segmented feature values ​​of a selected feature set from the audio sample, the segmented feature values ​​being indicative of chewing and swallowing sounds, the determining of the segmented feature values ​​comprising: Extracting short-term features from a selected feature set from the audio sample; and determining segmented feature values ​​of the selected feature set from the extracted short-term features; using a classifier with a processor to classify eating activities based on the determined segment feature values ​​of the audio samples, the classification including classification of ingestion of specific foods; and Output corresponding to the classified eating activities is generated.

13. The method according to claim 12, wherein: The audio input unit includes an electret condenser microphone.

14. The method according to claim 12, wherein: The obtaining of the audio samples includes obtaining the audio samples by wirelessly communicating via a communication module of the wearable device, and the processor is disposed in the external device.

15. The method according to claim 12, wherein: The processor is installed in the wearable device.

16. The method according to claim 12, wherein: The wearable device is a neckband.

17. The method according to claim 12, wherein: The selected feature set and classifier parameters of the classifier are determined during the machine learning training process.

18. The method of claim 17, wherein: The classifier is one of the following: random forest, Gaussian mixture model, linear discriminant analysis, naive Bayes, decision tree, and k-nearest neighbor classifier; The determination of the segment characteristic value further includes: Segmenting the extracted short-term features into segments of predetermined duration; and applying a statistical function to the extracted short-term features in each segment; The short-term features include one or more of: at least one MFCC coefficient, a delta of at least one MFCC coefficient, a delta-delta of at least one MFCC coefficient, energy entropy, short-term energy, zero crossing rate, spectral flux, spectral peak, spectral skewness, spectral centroid, spectral slope, spectral drop, spectral spread, spectral roll-off, and spectral flatness; and The statistical functions include one or more of mean, median, standard deviation, skewness, kurtosis, maximum and minimum.

19. The method of claim 12, wherein the generating of the output comprises generating a perceptible output via an output device.

20. A system for monitoring a user's eating activities, comprising: Wearable neck strap, including: at least one audio input unit configured to record audio samples corresponding to audio from a neck of a user, the audio input unit comprising an electret condenser microphone; a communication module configured to wirelessly transmit the recorded audio samples to an external device; an energy storage unit configured to store power for operation of the at least one audio input unit and the communication module; as well as a housing in which the at least one audio input unit, the communication module and the energy storage unit are arranged, the housing being elastically deformable and comprising a fastening arrangement at each end of the housing; and an external device including a processor configured to execute programmed instructions stored in a memory for: obtaining recorded audio samples from a wearable neckband; Determining segmented feature values ​​of a selected feature set from the recorded audio sample, the segmented feature values ​​indicating chewing and swallowing sounds, the determining of the segmented feature values ​​comprising: extracting short-term features from a selected feature set from the recorded audio sample; and determining segmented feature values ​​of the selected feature set from the extracted short-term features; Using a classifier, classifying the eating activity based on the determined segment feature values ​​of the audio sample, the classification including classification of ingestion of a specific food; and Output corresponding to the classified eating activities is generated.

Images