System for real-time tracking of nutritional events through acoustic chewing motion analysis in a portable device

Abstract

A system (100) for tracking nutritional events in real time, comprising the following: a portable device (1) with sound sensors (11) for recording acoustic data of chewing sounds; a Bi-LSTM network (2) configured to classify sequential chewing data into chewing states; a Hidden Markov Model (HMM) (3) to analyze the state transitions between the identified chewing events; a sliding window segmentation module (4) for segmenting continuous audio data for real-time event tracking; and a module of the Viterbi algorithm (5) that is integrated into the HMM (3) to enable optimal sequence estimation of chewing states.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to a wearable eating behavior monitoring system that classifies food intake by analyzing chewing sounds. It utilizes advanced machine learning techniques to enable non-invasive, real-time tracking of eating behavior.

[0002] In particular, the present invention relates to a system for tracking real-time nutritional events using acoustic mastication analysis in a portable device. BACKGROUND OF THE INVENTION

[0003] The subject matter discussed in the "Background" section should not be assumed to be prior art merely by virtue of its mention in the "Background" section. Likewise, it should not be assumed that a problem mentioned in the Background section or related to the subject matter of the Background section has already been recognized in the prior art. The subject matter of the Background section merely presents various approaches, which may also be inventions in their own right.

[0004] Existing dietary monitoring methods rely heavily on self-reporting or complex sensor arrays, making them prone to inaccuracies or user inconvenience. The state of the art has explored various machine learning techniques for dietary monitoring; however, many are limited by ambient noise, require multiple sensor types, or do not provide real-time monitoring. Traditional models using convolutional neural networks (CNNs), for example, are limited in accurately detecting different eating events under noisy conditions. Furthermore, complex multi-sensor models are cumbersome, limiting the feasibility of long-term dietary monitoring in real-world environments.

[0005] The present invention solves these problems by providing a system that uses a hybrid Bi-LSTM (Bidirectional Long Short-Term Memory) and HMM (Hidden Markov Model) to classify chewing events from acoustic data. This system is embedded in a wearable, earphone-like device that is lightweight, non-invasive, and immune to background noise, making it ideal for monitoring food intake in the wild.

[0006] The information disclosed in this Background section is provided solely for the purpose of facilitating an understanding of the background of the invention and may therefore contain information that is not prior art and that is already known to a person skilled in the art in this country. SUMMARY

[0007] Before describing the present systems and methods, it should be noted that this application is not limited to the systems and methods described, as there may be several possible embodiments not expressly shown in this disclosure. It should also be understood that the terminology used in the description is for the purpose of describing the particular versions or embodiments only and is not intended to limit the scope of the present application.

[0008] The invention is a wearable system that can monitor food intake in real time by analyzing chewing sounds. It features a hybrid Bi-LSTM-HMM model that enables sequential data interpretation and dynamic state tracking. The system segments audio signals using a sliding window approach and leverages the Viterbi algorithm for precise state transitions, thereby improving classification accuracy. This model is implemented in a wearable, headphone-like device equipped with sound sensors, enabling non-invasive tracking of food intake even in noisy environments.

[0009] The system aims to improve the accuracy and convenience of dietary monitoring by non-invasively capturing, analyzing, and classifying food intake behavior. BRIEF DESCRIPTION OF THE DRAWING

[0010] To clarify various aspects of some embodiments of the present invention, a more detailed description of the invention will be given by reference to specific embodiments illustrated in the accompanying drawings. It is understood that these drawings show only illustrative embodiments of the invention and are therefore not to be considered limiting its scope. The invention will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0011] In order that the advantages of the present invention may be readily understood, a detailed description of the invention is discussed below in conjunction with the accompanying drawings, which, however, should not be construed as limiting the scope of the invention to the accompanying drawings, in which: Fig.shows a block diagram of the system (100) for tracking feeding events in real time using acoustic chewing assessment in a wearable device. DETAILED DESCRIPTION

[0012] The present invention relates to a system (100) for tracking nutritional events in real time using acoustic mastication analysis in a portable device

[0013] The present invention relates to a system (100) developed for real-time tracking of feeding events through the analysis of acoustic chewing sounds. The system (100) comprises various components, including a wearable device (1), a Bi-LSTM network (2), a hidden Markov model (HMM) (3), a sliding window segmentation module (4), and a Viterbi algorithm module (5) integrated into the HMM (3). Together, these components enable the system (100) to accurately detect, classify, and track feeding events such as biting, chewing, and swallowing in real time, even in noisy environments.

[0014] The wearable device (1) is a headphone-like device designed for non-invasive, real-time monitoring of feeding processes. Equipped with sound sensors (11) strategically positioned near the user's mouth, the device (1) captures acoustic signals associated with chewing sounds. The headphone-like design ensures that the device can be worn comfortably for extended periods, making it suitable for use outdoors.

[0015] The noise sensors (11) are configured to filter out background noise and detect sounds primarily within the frequency range associated with chewing, optimizing the device (1) for noise immunity. This configuration enables the system (100) to function accurately in everyday environments where ambient noise might otherwise interfere with the acquisition of acoustic data.

[0016] The Bi-LSTM (Bidirectional Long Short-Term Memory) network (2) is a core component responsible for analyzing sequential data from chewing sounds. The Bi-LSTM network (2) operates on sequential data and is particularly well-suited for capturing temporal dependencies within the audio signals of chewing events. Each sound event is processed in both forward and reverse directions, allowing the network (2) to make precise classifications based on the complete context of the surrounding data points.

[0017] After receiving audio data segmented by the sliding-window segmentation module (4), the Bi-LSTM network (2) classifies each segment into masticatory states, such as biting, chewing, or swallowing. By incorporating bidirectional data flow, the Bi-LSTM network (2) increases the system's accuracy in identifying specific feeding events, regardless of their duration or timing within a sequence.

[0018] The Hidden Markov Model (HMM) (3) is integrated into the system (100) to analyze and map transitions between identified chewing states. During the chewing process, the HMM (3) predicts likely state transitions, such as the transition from biting to chewing or from chewing to swallowing. This mapping of transitions is crucial for accurately tracking feeding events in real time, as it refines the classification process performed by the Bi-LSTM network (2).

[0019] By applying probabilistic state transition modeling, the HMM (3) ensures that successive events follow logical sequences, reducing the likelihood of classification errors. This dynamic approach to state tracking allows the system (100) to operate effectively under real-world conditions where chewing events can vary in length and intensity.

[0020] The sliding window segmentation module (4) processes continuous audio data captured by the sound sensors (11) in the wearable device (1). This module segments the audio stream into discrete windows, each representing a fixed time frame of audio data. These segmented frames provide manageable data units for classifying chewing events in real time.

[0021] Each segment generated by the sliding window segmentation module (4) contains relevant features associated with a single chewing event or part of an event. This segmentation enables the Bi-LSTM network (2) to accurately classify chewing events within each window, while the HMM (3) can effectively process and map the transitions between windows. By splitting continuous audio data into smaller, analyzable frames, the sliding window segmentation module (4) facilitates high-resolution tracking of feeding behavior.

[0022] Module (5) of the Viterbi algorithm is embedded in the HMM (3) and is used to optimize the estimation of the state sequence of chewing events. While the HMM (3) analyzes the transitions between chewing states, the module of the Viterbi algorithm (5) calculates the most probable sequence of states, thus improving the accuracy of tracking feeding events.

[0023] This algorithmic integration ensures that each sequence of identified chewing events matches the most likely trajectory based on the observed data. The Viterbi algorithm (5) minimizes misclassifications by ensuring logical consistency in the sequence of chewing events, e.g., that a bite event is followed by a chew rather than an unconnected state. This optimization significantly improves the reliability and precision of real-time food tracking, even under conditions of fluctuating signal quality.

[0024] Functional sequence of the system (100): The following steps describe the operation of the system (100): Sound data capture: The wearable device (1) collects acoustic data from chewing sounds via its sound sensors (11). These sensors focus on the frequencies associated with chewing and filter out distracting background noise to optimize data quality for classification.

[0025] Segmentation of audio data: The sliding-window segmentation module (4) divides the continuous stream of audio data into fixed time frames. Each segmented window serves as a data unit that the Bi-LSTM network (2) processes individually, enabling high-resolution tracking of feeding events in real time.

[0026] Classification of chewing events: Each segmented window is fed into the Bi-LSTM network (2), where it is analyzed and classified according to masticatory states (e.g., biting, chewing, swallowing). The Bi-LSTM network (2) uses bidirectional processing to ensure that the classification of each segment takes temporal dependencies into account, increasing accuracy.

[0027] Mapping of state transitions: After classification by the Bi-LSTM network (2), the HMM (3) assigns probabilities to the transitions between the detected chewing states. This assignment of state transitions helps ensure logical consistency in the sequence of detected chewing events.

[0028] Optimal estimation of the sequence: The Viterbi algorithm module (5) processes the sequence of chewing states predicted by the HMM (3) and estimates the most likely sequence of events. This step refines the tracking accuracy of the system, reduces classification errors, and increases the reliability of dietary monitoring in diverse environments.

[0029] Data collection and analysis: Each identified dietary event and sequence is logged in the system (100) so that users or healthcare professionals can review and analyze dietary patterns over time. This data can be used to promote healthier eating habits or support nutritional research and monitoring. Advantages of the system (100)

[0030] High accuracy: The integration of the Bi-LSTM network (2) with the HMM (3) and the Viterbi algorithm (5) improves the classification accuracy and enables reliable identification of feeding events.

[0031] Noise immunity: The portable device (1) is noise insensitive, so it works accurately even in real environments with background noise.

[0032] Real-time monitoring: The sliding window segmentation module (4) enables continuous tracking of chewing events in real time, providing users with a comprehensive overview of their eating behavior without delay.

[0033] User-friendly design: The earphone-like form factor of the wearable device (1) ensures comfort and ease of use, making the system (100) suitable for extended wear and daily use. Example of implementation

[0034] In one embodiment, the wearable device (1) records chewing sounds via a microphone positioned near the user's mouth. The sound data is segmented into segments by the sliding-window segmentation module (4), each of which is analyzed by the Bi-LSTM network (2) to classify the chewing states. The HMM (3) then refines these classifications by mapping the transitions between states using the Viterbi algorithm module (5) to determine the most likely sequence of events. For example, after detecting a bite, the system predicts chewing or swallowing based on likely state sequences, thus providing a highly accurate log of feeding events.

Claims

[1] A system (100) for tracking real-time nutritional events, comprising: a portable device (1) with sound sensors (11) for recording acoustic data of chewing sounds; a Bi-LSTM network (2) configured to classify sequential chewing data into chewing states; a Hidden Markov Model (HMM) (3) to analyze the state transitions between the identified chewing events; a sliding window segmentation module (4) for segmenting continuous audio data for real-time event tracking; and a module of the Viterbi algorithm (5) integrated into the HMM (3) to enable optimal sequence estimation of the chewing states. [2] The system (100) of claim 1, wherein the Bi-LSTM network (2) classifies chewing events in segmented audio frames, identifying individual chewing states such as biting, chewing, and swallowing. [3] The system (100) of claim 1, wherein the portable device (1) is a headphone-like device designed for non-invasive monitoring and optimized for insensitivity to background noise.

Images