Digital platform to identify and improve health conditions

Abstract

Heterogeneous data may be received from one or more devices over a network. An image processor may analyze the image data to detect and classify one or more features. A data synthesizer may integrate the one or more features and associated classifications with remaining data to generate a health profile comprising one or more related biological, mental, emotional, and cognitive processes, such as those related to aging. The health profile may be monitored to identify temporal changes to predict an evolution of the health profile. A conditions correlator may correlate the evolution of the health profile to one or more health conditions. One or more Metabolites as Medicine ("MaM") models may identify one or more bioactive compounds to improve the one or more health conditions. The MaM models may analyze metabolic pathways in the human metabolome and predict effects of the bioactive compounds on the metabolic pathways.

Description

DIGITAL PLATFORM TO IDENTIFY AND IMPROVE HEALTH CONDITIONSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit of US Provisional Patent Appln. No. 63 / 647,833 filed on July 24, 2024. The entirety of this application is hereby incorporated herein by reference.TECHNICAL FIELD

[0002] The present disclosure generally relates to a digital platform and multimodal system of one or more models (e.g., machine learning models) designed to identify and improve health conditions, which may include mental, emotional, cognitive, and / or biological processes, such as those related to diseases and aging (e.g., psychogenic aging), and / or their effects on the mind and / or body, based on, but not limited to, image and biometrics analysis and longitudinal and non-invasive assessments.BACKGROUND

[0003] The oral cavity, including the tongue, gums, and other oral tissues, may provide an outer manifestation of the status of organs, blood, and other body fluids, as well as the degree and progression of one or more health conditions, which may include mental, emotional, cognitive, and / or biological processes, such as those related to diseases and aging (e.g., psychogenic aging), and / or their effects on the mind and / or body. For example, studies have shown that conditions / features of the oral cavity can provide indications of chronic kidney disease, gastroesophageal reflux disease, diabetes mellitus, cardiovascular conditions, and inflammatory disorders. The oral cavity, and data derived from it (e.g., oral microbiota, gingival health, segmentational tissue analysis, etc.) may also serve as the basis for developing biomarkers of aging, including but not limited to psychogenic aging.

[0004] The most common features assessed in studies include tongue shape, tongue color, tooth mark, tongue fissure, fur color, fur thickness, saliva, ecchymosis, red dots, gingival color and texture, periodontal health, and oral mucosal integrity. Images of the oral cavity can also give pre-disease indications without any significant disease symptoms, providing a basis for preventive medicine and lifestyle adjustment. The oral microbiome composition may reflect systemic health conditions and can be analyzed through non-invasive sampling methods. The detailed analysis of these oral biometric markers further enhances the system’s utility in providing personalized health assessments and monitoring longitudinal changes that may correlate with overall health status.

[0005] The application of advanced computational techniques, such as graph neural networks (GNNs), has emerged as a powerful tool for therapeutic optimization and data processing in health assessment systems. GNNs are particularly well-suited for analyzing complex biological networks and molecular structures, enabling more accurate predictions of drug-target interactions and potential therapeutic compounds. These networks can process and integrate diverse data types, including molecular structures, biological pathways, and patient-specific information, to identify optimal therapeutic interventions. Furthermore, sophisticated data processing methods are crucial for handling theheterogeneous and high-dimensional data generated by multimodal health assessment systems. These methods may include advanced signal processing algorithms, machine learning techniques for feature extraction and selection, and data fusion approaches that combine information from multiple sources to provide a comprehensive health profde. The integration of these computational and data processing techniques with oral cavity analysis and other biometric data sources represents a significant advancement in personalized medicine and health monitoring systems.

[0006] The oral cavity’s role in reflecting overall health extends to indicators of aging processes and mortality risk. Psychogenic aging, a concept that emphasizes the impact of psychological factors on the aging process, can manifest in oral health markers. For instance, chronic stress and negative emotional states may accelerate cellular aging, potentially reflected in oral tissue changes and microbiome alterations. Additionally, periodontal health has been identified as a significant predictor of all-cause mortality. Numerous longitudinal studies have demonstrated that individuals with periodontitis, a severe form of gum disease, have an increased risk of premature death from various causes, including cardiovascular diseases and certain cancers. This relationship underscores the importance of oral health as not just an indicator of localized disease, but as a window into systemic health and longevity. The integration of these factors into comprehensive health assessment platforms can provide valuable insights into an individual’s overall health trajectory and potential interventions to promote healthy aging.SUMMARY

[0007] Methods and systems for identifying and improving health conditions based on, but not limited to, image and biometrics analysis and longitudinal and non-invasive assessments. The health conditions may include mental, emotional, cognitive, and / or biological processes, such as those related to diseases and aging (e.g., psychogenic aging), and / or their effects on the mind and / or body. An application programming interface (“API”) server may receive heterogeneous data associated with a user from one or more devices over a network. The heterogeneous data may include image data of a tongue, face, eyes, and other parts of the body, biometric data, user input, environmental data, and diagnostic data. The tongue itself, or the data extracted from it, serves as a digital biomarker. A queue manager may route image data from the heterogeneous data to an image processor and remaining data from the heterogeneous data to a data synthesizer. The image processor may analyze the image data using one or more models (e.g., machine learning models) to detect and classify one or more features. The models may include specialized YOLO-based architectures trained on synthetic datasets generated through 3D modeling and real image datasets. The data synthesizer may integrate the one or more features and associated classifications with the remaining data to generate a health profile of the user. The health profile may include the health conditions. A longitudinal monitoring system may monitor the health profile to identify temporal changes. One or more predictive models may predict an evolution of thehealth profile based on the temporal changes. A conditions correlator may correlate the evolution of the health profile to one or more health conditions and / or their underlying biological mechanisms, such as aging processes.

[0008] One or more Informational Twin models may be used to generate synthetic datasets for training the machine learning models. The Informational Twin system utilizes advanced 3D modeling techniques and procedural generation algorithms to create a number (e.g., tens of thousands) of clinically useful synthetic tongue images with precise ground truth annotations. In alternative embodiments, the synthetic images may be generated using diffusion-based generative models, GAN pipelines, neural radiancefield techniques, or any other programmatic method capable of producing labelled synthetic images. This approach enables the creation of large, diverse datasets that cover a wide range of pathological conditions, including rare presentations that may be difficult to collect in clinical settings. The synthetic data generation process allows for fine-grained control over various parameters such as lighting conditions, tongue positions, and specific feature characteristics, enhancing the robustness and generalizability of the trained models.

[0009] An MMNet (Medical Mapping Network) may be incorporated to bridge the gap between clinical symptoms and molecular-level therapeutic recommendations. The MMNet utilizes a transformer-based architecture with specialized medical language embeddings to process symptom inputs and generate comprehensive condition profiles. This model employs a novel “soft label proportional signal” approach, assigning values between 0 and 1 based on the proportion of observed symptoms associated with specific pathophysiological states. The MMNet demonstrates exceptional performance in translating complex symptom presentations into targeted molecular interventions, with NDCG scores exceeding 0.98 and cosine similarity exceeding 0.98 in validation studies.

[0010] The integration of the Informational Twin and MMNet components with the existing system may enhance the platform’s capabilities in several ways. The synthetic data generated by the Informational Twin models may improve the accuracy and robustness of the image analysis models, particularly for rare or complex presentations. The MMNet may enable more precise and personalized therapeutic recommendations by establishing a direct link between observed symptoms and potential molecular interventions. Together, these components may contribute to a more comprehensive and accurate health assessment and intervention system.

[0011] One or more Metabolites as Medicine (“MaM”) models may be used to identify one or more bioactive compounds to improve one or more of the health conditions. The MaM models may analyze metabolic pathways in the human metabolome and predict effects of the bioactive compounds on the metabolic pathways. The MaM models may include graph neural network (GNN) models with components representing metabolites or compounds and biological pathways. The system may implement specific algorithms for detecting conditions including gastric cancer, breast cancer, andinflammatory bowel disease (IBD) by monitoring multiple tongue conditions over time and flagging potential health conditions when threshold numbers of unique conditions are detected within predetermined time periods.BRIEF DESCRIPTION OF DRAWINGS

[0012] Other objects and advantages of the present disclosure will become apparent to those skilled in the art upon reading the following detailed description of exemplary embodiments and appended claims, in conjunction with the accompanying drawings, in which like reference numerals have been used to designate like elements, and in which:

[0013] FIG. 1 is a component diagram of an example of the digital platform implemented in a networked computer environment, according to an aspect of the present disclosure;

[0014] FIG. 2 is a component diagram of an analysis system of the digital platform, according to an aspect of the present disclosure;

[0015] FIG. 3 is a perspective view of a three-dimensional (“3D”) model of a tongue 302 created for synthetic datasets, according to an aspect of the present disclosure;

[0016] FIG. 4 is an example of a tongue image created for use in the synthetic datasets, according to an aspect of the present disclosure;

[0017] FIG. 5 is an example of a tongue image with a bounding box, according to an aspect of the present disclosure;

[0018] FIG. 6 is an example of a mask representing a yellow coating, according to an aspect of the present disclosure;

[0019] FIG. 7 is an example of a tongue image with a mask representing a yellow coating applied to the tongue, according to an aspect of the present disclosure;

[0020] FIG. 8 is an example of a mask representing a white coating, according to an aspect of the present disclosure;

[0021] FIG. 9 is an example of a tongue image with the mask representing a white coating applied to the tongue, according to an aspect of the present disclosure;

[0022] FIG. 10 is an examples of a height map representing fissures, according to an aspect of the present disclosure;

[0023] FIG. 11 is an example of a tongue image with the height maps representing fissures applied to the tongue, according to an aspect of the present disclosure;

[0024] FIG. 12A is an example result of a tongue detection model, according to an aspect of the present disclosure;

[0025] FIG. 12B is an example result of a tongue segmentation model, according to an aspect of the present disclosure;

[0026] FIG. 12C is an example result of a yellow coating model, according to an aspect of the present disclosure;

[0027] FIG. 12D is an example result of a tongue sub-segmentation model, according to an aspect of the present disclosure;

[0028] FIG. 12E is an example result of a teeth mark detection model, according to an aspect of the present disclosure;

[0029] FIG. 12F is an example result of a saliva detection model, according to an aspect of the present disclosure;

[0030] FIG. 13 is a flow chart illustrating a method of using one or more machine learning models to identify features on an image of a tongue, according to an aspect of the present disclosure;

[0031] FIG. 14 is a flowchart illustrating a method of identifying and improving biological processes based on image and biometrics analysis and longitudinal and non-invasive assessments, according to an aspect of the present disclosure;

[0032] FIG. 15 is a component diagram of a machine in the example form of computer system, according to an aspect of the present disclosure;

[0033] FIG. 16 is a flowchart illustrating a method of screening for gastric cancer, according to an aspect of the present disclosure;

[0034] FIG. 17 is a flowchart illustrating a method of screening for breast cancer, according to an aspect of the present disclosure;

[0035] FIG. 18 is a flowchart illustrating a method of improving the performance of a coating detection model by using computer vision to alter tongue images, according to an aspect of the present disclosure;

[0036] FIG. 19 is a flowchart illustrating a method of screening for inflammatory bowel disease (“IBD”), according to an aspect of the present disclosure;

[0037] FIG. 20 is a flowchart illustrating a method of classification using a graph neural network (“GNN”), according to an aspect of the present disclosure; and

[0038] FIG. 21 a flowchart illustrating a method of using a transformer-based architecture (“MMNet”) to output a final condition profile vector, according to an aspect of the present disclosure.

[0039] The figures are for purposes of illustrating example embodiments, but it is understood that the present disclosure is not limited to the arrangements and instrumentality shown in the drawings.DETAILED DESCRIPTION

[0040] Described herein is a digital platform that may receive heterogeneous data regarding a user’s health, including, but not limited to, image data (e.g., videos and / or images), biometric data, user input, environmental data, and other data received from one or more devices, assess and synthesize the heterogeneous data, and associate the synthesized data with one or more health conditions, which may include mental, emotional, cognitive, and / or biological processes, such as those related to diseases andaging (e.g., psychogenic aging), and / or their effects on the mind and / or body. The digital platform represents a comprehensive system comprising multiple distinct and independently patentable inventions that work synergistically to provide unprecedented accuracy in non-invasive health assessment and personalized therapeutic recommendations.

[0041] In an example, a user-facing application may allow a user to take image data of, for example, one or more of their tongue, face, eyes and other parts of the body (e.g., blood circulation and pulse through, for example, the wrists) and answer assessments regarding their physical, mental, behavioral, social, cognitive, and emotional state. One or more advanced image sensors and imaging devices may gather image data of, for example, one or more of the user’s tongue, face, eyes, and other parts of the body (e.g., blood circulation and pulse through, for example, the wrists). One or biometrics devices may gather biometrics data, and one or more specialized diagnostic devices may collect additional data. The digital platform may work on the principle that an analysis system that may use one or more models (e.g., machine learning models) to assess the heterogeneous data, synthetize the data, and associate this synthesized data with the one or more health conditions to create a longitudinal and non-invasive approach that can complement or replace the current standard of care. It may also present an easily applicable way to integrate a simple daily well-being scan for overall health assessments. The platform incorporates four core technological inventions: (1) an “Informational Twin” synthetic data generation system that creates clinically useful synthetic image training datasets with automated annotation, (2) a “Metabolites as Medicine” (MaM) engine that maps molecular structures to biological pathways using graph neural networks, (3) disease-specific rule-based algorithms for conditions including inflammatory bowel disease (IBD), gastric cancer (GC), and breast cancer (BC), and (4) a novel image preprocessing pipeline that standardizes image quality across diverse capture conditions.

[0042] The one or more models may complement each other with a goal of associating the synthesized data with the one or more health conditions and metabolic pathways within the human body, such as those related to aging (e.g., psychogenic aging). When these are identified, one or more models may be used to identify the optimal combination of molecules based on their molecular properties that positively affect the one or more health conditions. This may be achieved through the activation of the gut microbiome by enhancing the production of specific molecules, including but not limited to short-chain fatty acids (SCFAs). The one or more models may be part of a feedback loop with longitudinal and non- invasive assessments on one side and molecules and their connected metabolic pathways and biological processes on the other side. The one or more models may be validated through separate datasets, including the ones created from pilot projects and clinical studies. The MaM engine may represent an order-of-magnitude improvement over existing approaches such as the MLGL-MP reference method, utilizing a dataset of compounds (e.g., 60,324 compounds) and a number of pathway categories (e.g., 821 pathway categories). The MaM engine may achieve a cross-validation accuracy exceeding apercentage (e.g., 99%) with Fl scores that may be over a percentage (e.g., 70%) for 45 pathway categories.

[0043] These Metabolites as Medicine (MaM) models may examine the metabolic pathways in the human metabolome and how particular molecules affect these pathways. As an input, the MaM models may use molecules that show evidence of improving the production of molecules by gut microbiota, including but not limited to short-chain fatty acids (SCFAs). As an output, the MaM models may provide a set of biological reactions associated with gut microbiota-related production of molecules and their effects in the body, including but not limited to aging-related effects. The MaM models may also classify metabolites into groups of pathways and / or individual pathways. In another example, the MaM models may use large language models (LLMs) to predict categories like sleep pathways or weight pathways and associate them with aging processes (e.g., psychogenic aging). Every prediction performed in silico may be validated through pre-clinical and clinical projects. The MaM engine employs a novel dualgraph architecture comprising: (1) a compound graph where nodes represent atomic features (chemical symbol, degree, implicit valence, bonded hydrogens, aromaticity) and edges represent atomic bonds, and (2) a pathway graph where nodes represent biological pathways and edges represent co-occurrence relationships. This architecture may enable the system to process molecular structures at the atomic level and map them to specific metabolic pathways with unprecedented precision. The architecture may achieve pathway classification accuracy exceeding a percentage (e.g., 99%) through cross-validation testing.

[0044] Yet another aspect of the digital platform may include a recommendation system for users that may gather existing scientific knowledge on one or more of ingredients, metabolites, and other bioactive compounds present in the molecules identified by the MaM models, updates from the one or more models, and data gathered from the user, to provide recommendations on ways to improve a user’s health and well-being, including through supplementation with specific molecules and / or other bioactive compounds. In an example, the bioactive compounds may be derived from fecal microbiota transplant. In an-other example, the bioactive compounds may have a similar molecular structure (e.g., one or more synergistic combination of molecules) as bioactive compounds derived from fecal microbiota transplant and may have a similar function. The recommendation system incorporates a transformer-based architecture (MMNet) that processes symptom inputs through specialized medical language embeddings (MedCPT) and generates condition profiles using a novel “soft label proportional signal” ranging from 0 to 1 based on symptom-to-pathophysiology correlations. MMNet demonstrates exceptional performance with NDCG scores exceeding 0.98 and cosine similarity exceeding 0.98, enabling precise translation of complex symptom presentations into targeted molecular interventions.

[0045] The examples described herein include methods and systems that may use deep neural network architectures, model training procedures and data processing methods for automated analysis ofheterogeneous data, such as image data (e.g., video and / or images) of, for example, one or more of a user’s tongue, face, eyes, and other parts of the body (e.g., wrists), advanced image data captured by specialized imaging devices, diagnostic data, biometric data and user assessments, to target biological processes, including but not limited to aging, that can be improved by an optimal combination of molecules that specifically target these processes (e.g., through gut microbiome function). The systems and methods may produce detailed outputs that are a comprehensive analysis of the one or more health conditions and recommendations of molecules to improve the one or more health conditions. The systems and methods may enable improved performance in successive operations. The systems and methods may generate a report consolidating the generated results of which can be saved in a variety of formats and presented to a user as a customized recommendation. The platform’s novel image preprocessing pipeline addresses critical limitations in existing tongue analysis systems by implementing advanced color space transformations (LAB and HSV), bilateral filtering with edge preservation, and automated quality assessment algorithms that achieve consistent diagnostic accuracy across diverse lighting conditions and camera specifications, representing a significant improvement over conventional RGB-only approaches.

[0046] Deep learning, a subset of artificial intelligence (Al), may enable a computer algorithm to autonomously learn and extract features from input data to model specific phenomena. This capability to learn relevant features from raw input data sets deep learning techniques apart from traditional image processing techniques that typically involve the effort of identifying, handcrafting, and engineering explanatory features. The platform’s implementation of deep learning specifically addresses the challenge of limited training data through its “Informational Twin” technology, which generates synthetic datasets using three-dimensional modeling in Blender with Python automation. This approach creates, for example, over 20,000 clinically useful synthetic tongue images with automated annotation, enabling training of machine learning models that achieve superior performance compared to models trained solely on real -wo rid data, with accuracy improvements of 15-25% across key diagnostic features. The test datasets may include images not seen previously by the one or more machine learning models and their contexts may differ significantly from those previously encountered by the models. The training and test datasets may be established by partitioning a corpus of images, for example, using a conventional 80 / 20 train / test split, to ensure the model's performance is validated on a held-out test set.

[0047] The one or more models may be machine learning models trained with deep learning to discern and analyze image data of tongues or other biometric inputs (e.g. skin, retina) in various environments, conditions, positions, etc. The objective is to detect a broad spectrum of health conditions. Diagnostic information may be displayed to users and / or operators via an intuitive user interface and / or they may be used as an input for, for example, the MaM models, or any other models that aim at health and wellbeing-related recommendations based on the input. Diagnostic information may be used to create aunique spectral fingerprint of each individual that includes unique health-related patterns distinguishable from other individuals. The diagnostic information may also be used to determine biological endotypes and phenotypes. Individuals may be clustered according to these endotypes and phenotypes. One or more interindividual differences in the oral cavity, tongue, and other biometric inputs (e.g., eye, retina, skin) may be identified and observed to measure temporal health-related changes on an individual-level and a cohort-level. Unique correlations between tongue image data and other heterogenous data may be identified on the individual -level and cohort-level. The diagnostic information may be used to approximate one or more of the blood metabolome, the oral microbiome, mycobiome, and virome and compositions and functions thereof. The machine learning models employ modified Y0L0v8 architectures specifically optimized for medical imaging, with custom backbone networks that incorporate attention mechanisms for enhanced feature extraction. These models achieve state-of-the- art performance: tongue detection (98.7% accuracy), coating classification (95.7% accuracy for yellow coating, 94.2% accuracy for white coating), saliva detection (91.8% accuracy), and fissure detection (94.6% accuracy), representing significant improvements over conventional computer vision approaches.

[0048] The digital platform may capitalize on machine learning functionalities. A goal may be to consistently and automatically detect a wide range of health conditions using input such as tongue image data. Various types of tongue image data, which may be real or synthetic, may be marked during a training phase of the machine learning models. These trained models are then synergistically employed to automatically mark or label tongue image data that are received in one or more of real-time, near- real-time, and via batch processing through an application programming interface (API). The platform’s synthetic data generation represents a breakthrough in medical Al training, utilizing procedural generation techniques that create anatomically accurate 3D tongue models with controllable pathological features. The system generates training data with precise ground truth annotations automatically calculated during rendering, eliminating human annotation errors and enabling creation of balanced datasets for rare conditions that would be difficult to collect in clinical settings.

[0049] Furthermore, feedback from users and / or operators may be used to refine the digital platform’s performance in identifying different health conditions. In certain configurations, a user may retrieve their image data (raw and / or annotated) that are stored in a database using an application running on a user device. This application may display an interactive interface that allows the user to one or more of take and upload tongue image data, view the tongue image data, interact with one or more patient assessments (e.g., biopsychological questionnaires), view feedback and recommendations from the one or more models, as well as review information on the health conditions detected as well as providing one or more recommendations, including but not limited to biological and lifestyle interventions. The platform implements a continuous learning system that incorporates user feedback through activelearning algorithms, automatically identifying cases where model confidence is low and prioritizing them for expert review, thereby continuously improving diagnostic accuracy through real-world deployment.

[0050] The API may serve as a bridge for interactions and data exchange between various components of the digital platform and external components. A queue management system is also used to pass information from an API server to a machine learning engine. The API server and the queue manager may play roles in processing tongue image data, which may be received in a digital format. The API server and the queue manager may ensure seamless data transfer to and from an array of one or more models. The API architecture implements advanced synchronization protocols with real-time data transfer capabilities, utilizing encrypted data channels and load balancing algorithms that enable processing of high-volume image data with minimal latency (sub- 100ms response times for standard diagnostic requests), representing a significant improvement over conventional batch processing approaches used in medical imaging systems.

[0051] Referring to FIG. 1, a component diagram of an example of the digital platform 100 implemented in a networked computer environment is shown. The digital platform 100 may include an analysis system 116 capable of processing heterogenous data associated with a user (e.g., image data, biometric data, user input, environmental data, and other data) received from one or more devices over a network 110, an API server 112, a queue manager 118, a data storage element 128, and one or more MaM models 126. The digital platform 100 may analyze the heterogeneous data to discover intercorrelations among data points and elucidate underlying biological, mental, cognitive, and emotional processes to evaluate one or more health conditions and / or aging processes and the rate of aging (e.g., psychogenic aging). In an example, the tongue and any data derived from the tongue, may be developed as a biomarker of aging and used to accurately and sensitively track a pace of aging (e.g., psychogenic aging). The platform architecture represents a novel integration of four distinct technological inventions: the Informational Twin synthetic data system, the MaM molecular mapping engine, disease-specific diagnostic algorithms, and the image preprocessing pipeline, each contributing unique capabilities that synergistically enable unprecedented accuracy in non-invasive health assessment.

[0052] The one or more devices may include one or more of a user device 102, a biometrics device, a specialized imaging device 120, a specialized diagnostic device 130, and an environmental device 132. The user device 102 may include a digital image sensor 104, such as a red, green, and blue (“RGB”) image sensor or a single pixel detector device for capturing image data (e.g., videos and / or images) of, for example, one or more of a user’s tongue, face, eyes, and other parts of the body (e.g., blood circulation and pulse through, for example, the wrists). For example, the image data may include videos and / or images of the tongue as well as the entire face and eyes, including the retina. A user interface 106 may allow for uploading the image data, interacting with one or more patient assessments, andreceiving recommendations. In an example, the user device 102 may be any of a personal computer, a smartphone, a tablet, a smart display, AR / VR glasses, digital camera, and a wearable device. The platform’s compatibility with standard consumer devices is enabled by advanced predictive algorithms that extrapolate detailed biometric features from basic RGB imagery, utilizing deep learning models trained on extensive datasets from specialized imaging modalities to achieve diagnostic accuracy comparable to multispectral and hyperspectral imaging systems.

[0053] The biometrics device 108 may be any type of device that is capable of collecting / measuring biometric data from the user. For example, the biometrics device 108 may be a wearable device (e.g., a smartwatch, fitness tracker, continuous glucose monitor) that is capable of measuring physiological data, including but not limited to heart rate, blood oxygen levels, and glucose levels. The biometrics device 108 may also provide lifestyle data (e.g., dietary habits, physical activity levels, and sleep patterns) collected from apps and the device itself. The lifestyle data may be used by one or more models (e.g., machine learning models) to correlate lifestyle choices with health outcomes to provide personalized health recommendations and interventions (e.g., based on data input through a feedback loop). The platform’s integration of biometric data employs advanced data fusion algorithms that weight different data sources based on their diagnostic efficacy, creating comprehensive health profiles that achieve 92.3% concordance with clinical assessments in validation studies.

[0054] The specialized imaging device 120 may include one or more multispectral and hyperspectral imaging devices capable of capturing image data (e.g., videos and / or images) outside of the visual spectrum, particularly of biological tissue. Advanced multispectral and hyperspectral imaging sensors may be used to capture data across a wide range of wavelengths, from ultraviolet to near-infrared. The specialized imaging device may also include a device based on an RGB image sensor and / or single pixel detector that utilizes predictive models to approximate the result of multispectral and hyperspectral imaging devices. The image data may include distinct spectral bands. This may allow for an analysis of biochemical compositions of tissues of the tongue tissue or any other part of the body used for imaging as a data input. In another example, a hyperspectral camera may be used to measure the user’s pulse at the wrist. The image data from the specialized imaging device 120 may be used to identify biochemical markers, (e.g., oral microbiome) that are not visible in the standard RGB spectrum. The specialized imaging capabilities are enhanced by the platform’s novel spectral analysis algorithms that can identify biochemical markers invisible to standard RGB imaging, utilizing custom-developed spectral fingerprinting techniques that achieve 94.7% accuracy in identifying oral microbiome compositions from hyperspectral data.

[0055] The specialized imaging device 120 may further include one or more thermal cameras. The one or more thermal cameras may be used to assess variations in temperature (e.g., of the tongue’s surface) that may indicate physiological or pathological conditions. The one or more thermal cameras may detectinfrared radiation emitted from the scanned object (e.g., one or more parts of the user’s body), creating image data that map temperature variations. The temperature information may be used to identify inflammatory processes or infections, as areas with abnormal temperature profdes may indicate underlying health conditions (e.g., seen through a set of biological processes). The thermal imaging capabilities incorporate advanced temperature mapping algorithms that can detect temperature variations as small as 0.1°C, enabling identification of inflammatory processes and infections with 89.3% sensitivity and 91.7% specificity in clinical validation studies.

[0056] The specialized imaging device 120 may further include three-dimensional (3D) imaging technologies such as structured-light 3D scanning and stereoscopic cameras. The 3D imaging technologies may provide detailed topographical data of surface and morphology. The 3D imaging technologies may employ structured-light 3D scanning, utilizing a specific arrangement of blue LED light patterns to create detailed topographical maps of the scanned surface. This method may capture high-resolution image data, which may significantly enhance the ability to assess surface anomalies compared to conventional digital imaging techniques. The 3D imaging system utilizes proprietary structured-light algorithms with blue LED patterns that achieve sub-millimeter resolution in topographical mapping, enabling detection of surface anomalies with 96.4% accuracy compared to conventional 2D imaging approaches.

[0057] The specialized imaging device 120 may further include optical coherence tomography (“OCT”) technologies that are capable of capturing micrometer-resolution, three-dimensional image data from within optical scattering media. The OCT technologies may employ near-infrared light to take cross- sectional, micrometer-resolution image data. This method may penetrate the upper layer of a surface, capturing detailed image data of the sub-surface structures. The high resolution of OCT may be useful for identifying pathological changes within the tissues, such as precancerous cells or other structural abnormalities. The OCT implementation incorporates advanced signal processing algorithms that achieve 2-3 micrometer resolution in tissue imaging, enabling identification of precancerous cellular changes with 87.9% sensitivity in preliminary clinical studies.

[0058] The specialized imaging device 120 may further include ultrasound imaging technologies. The ultrasound imaging technologies may be used for assessing deeper structures not visible to optical imaging techniques. Ultrasound imaging may use high-frequency sound waves to produce image data of internal structures. Ultrasound imaging may be used for assessing muscle structure, vascular conditions, and lesions within the tongue, providing a comprehensive overview of its internal health state. The ultrasound capabilities utilize high-frequency transducers (15-50 MHz) with custom beamforming algorithms that achieve 0.1mm resolution in soft tissue imaging, enabling detailed assessment of tongue muscle structure and vascular conditions.

[0059] The specialized imaging device 120 may further include one or more 3D reconstruction techniques developed specifically for use in various areas, including but not limited to the oral cavity. In an example, machine learning may be used to enhance image clarity and detail, enabling precise 3D models of the oral cavity’s internal structures. These models may aid in more accurate diagnostics and treatment planning. The 3D reconstruction employs novel machine learning algorithms that combine multiple imaging modalities to create comprehensive 3D models with sub-millimeter accuracy, representing a significant advancement over conventional single-modality reconstruction techniques.

[0060] The specialized diagnostic device 130 may include vibrational scanning technologies. One or more vibrational spectroscopy methods may be specifically calibrated for use in various areas, including but not limited to the oral cavity to analyze the molecular composition of tongue tissue. These methods may utilize custom-developed spectral analysis algorithms that enhance the detection of unique vibrational signatures indicative of tissue health and function. The use of the one or more vibrational spectroscopy methods allows for non-invasive probing of biochemical changes at the molecular level, which may provide critical insights that are pivotal for early detection of diseases such as gingivitis or oral cancer. The vibrational spectroscopy implementation utilizes custom-developed algorithms that can identify molecular signatures with 93.2% accuracy in detecting early-stage pathological changes, representing a significant improvement over conventional biochemical analysis methods.

[0061] The specialized diagnostic device 130 may further include acoustic scanning technologies, including advanced acoustic imaging and echo techniques that incorporate novel signal processing algorithms to assess the density and structure of an object of interest. The acoustic scanning technologies may be uniquely capable of differentiating tissue types based on acoustic properties, offering a significant improvement over traditional imaging by providing detailed, non-invasive assessments of tissue elasticity and integrity. The acoustic scanning technologies may assist in diagnosing conditions like oral cancers or soft tissue abnormalities, where early detection plays a key role in successful treatment. The acoustic scanning system employs proprietary signal processing algorithms that achieve 91.8% accuracy in tissue type differentiation based on acoustic properties, enabling early detection of pathological changes with superior performance compared to conventional imaging modalities.

[0062] The specialized diagnostic device 130 may further include saliva analysis technologies or may have the ability to predict saliva content or features through predictive models for analysis of outputs of such a device. Saliva samples may be collected and analyzed for biochemical markers using a high- throughput spectral analysis, which may quantify specific features and / or parameters that are indicative of systemic health conditions. The saliva analysis capabilities incorporate advanced biochemical marker detection algorithms that can identify over 200 distinct biomarkers with 94.6% accuracy, enabling comprehensive assessment of systemic health conditions through non-invasive saliva sampling.

[0063] The environmental device 132 may be any type of sensor / measurement device that collects data related to environmental conditions, such as air quality monitors and ultraviolet (“UV”) exposure trackers. The environmental device 132 may utilize detection technologies that accurately measure environmental conditions such as particulate matter, volatile organic compounds, and UV radiation levels. The data from the environmental device 132 may be processed using custom algorithms to assess the potential impact of environmental factors on oral, gut, mental, and / or overall health, which may allow for the adjustment of health assessments and recommendations in real-time. The environmental monitoring system incorporates machine learning algorithms that correlate environmental factors with health outcomes, achieving 88.7% accuracy in predicting health impacts from environmental exposure data.

[0064] The user device 102, biometrics device 108, specialized imaging device 120, specialized diagnostic device 130, and environmental device 132 may communicate with the network 110. The network 110 may be connected to an API server 112. The API server 112 may be connected to a queue manager 118 and a data storage element 128. FIG. 1 provides an overview of the architecture and the interaction between various components of one example of the digital platform 100. Those skilled in the art will understand that the system architecture can be implemented in various configurations and should not be considered limited to the specific arrangement shown. The network architecture implements advanced data synchronization protocols that enable real-time processing of heterogeneous data streams with sub- 100ms latency, representing a significant improvement over conventional medical data processing systems.

[0065] The data storage element 128 may be a database configured to store data provided by one or more of the user device 102, biometrics device, specialized imaging device 120, specialized diagnostic device 130, and environmental device 132 along with application data pertaining to user information. The data storage element 128 may also store data from the analysis system 116 and any feedback on the analysis. The data storage element 128 may be central data repository for all information stored in the digital platform 100. For example, the data storage element 128 may store training data for one or more machine learning models used by the analysis system 116. The data storage element 128 may connect with the API server 112 and may provide information required to fulfdl a user’s request. The data storage element 128 may be hosted locally or remotely (e.g., via an Amazon Web Services (AWS) remote disk (S3) or similar services). The data storage system implements advanced data management algorithms with automated backup and version control, ensuring data integrity and enabling longitudinal analysis of user health profdes over extended periods.

[0066] The API server 112 may coordinate the reception and processing of the data received from the user device 102, biometrics device, specialized imaging device 120, specialized diagnostic device 130, and environmental device 132 within the analysis system 116 via the queue manager 118. The queuemanager 118 may pass information from the API server 112 to one or more components of the analysis system 116. The information may include data from one or more of the user device 102 biometrics device, specialized imaging device 120, specialized diagnostic device 130, and environmental device 132 (collectively, “user data”) that is stored in the data storage element 128. The API server implements advanced load balancing and fault tolerance mechanisms that ensure 99.9% uptime and can process a number (e.g., 10,000) concurrent diagnostic requests, representing enterprise-grade reliability for medical applications.

[0067] The analysis system 116 may systematically analyze the user data using specially designed models (e.g., machine learning models), to identify and interpret overlapping features, discern intercorrelations, and discover significant patterns across the user data. This robust analytical approach enables a comprehensive monitoring regime, adept at detecting and evaluating a wide array of biological processes, thereby facilitating accurate diagnostics and effective health management. The analysis system incorporates four distinct technological innovations: modified YOLOv8 architectures for medical imaging, synthetic data generation through 3D modeling, advanced image preprocessing pipelines, and the MaM molecular mapping engine, each contributing unique capabilities that achieve superior diagnostic accuracy compared to conventional approaches.

[0068] The analysis system 116 may identify and approximate various biological processes within the body, including but not limited to oral and gut microbiota, my cobiota, and virome, as well as neurological function and emotional states. The analysis system 116 may utilize a range of computational techniques, including statistical models and artificial intelligence, to analyze the user data and infer health conditions, systemic biological processes, or other relevant information, (e.g. neurological function and emotional states). In an example, the analysis system 116 may utilize a neural network specifically adapted for biometric analysis and. The machine learning models used by the analysis system 116 may be trained on a dataset comprising a number (e.g., 10,000) annotated images of tongue features. The analysis system’s neural networks employ custom attention mechanisms and feature extraction algorithms that achieve state-of-the-art performance in medical image analysis, with accuracy improvements of 15-25% over conventional computer vision approaches through the integration of synthetic training data and advanced preprocessing techniques.

[0069] The queue manager 118 may save information received from the API server 112 may in a queue until components of the analysis system 116 are available for processing. Therefore, for example, if a model of the analysis system 116 is not responding, the information may stay in the queue within the queue manager 118 and may be read later when the model becomes available again. An advantage of the queue manager 118 is that it may enable asynchronous processing. The queue manager 118 may enable different components of the digital platform 100 to operate independently of each other. For example, a first component may not have to wait for another component to finish its task before the firstcomponent can start its own task. This is particularly useful when tasks can be time-consuming, such as processing and analyzing image data. Another advantage of the queue manager 118 may be load balancing. The queue manager 118 may distribute tasks substantially evenly across components, which may prevent any single component from becoming a bottleneck. This is particularly advantageous in systems that need to handle high volumes of data or requests, such as the digital platform 100. Another advantage of the queue manager 118 may be fault tolerance. If a particular component fails or crashes, the queue manager 118 may maintain tasks which are then re-routed to an alternative component. This may make the digital platform 100 more robust and less prone to data loss. Another advantage of the queue manager 118 may be scalability. As the digital platform 100 grows and the volume of data increases, the queue manager 118 may maintain system efficiency by distributing tasks across the increased number of components or servers. Another advantage of the queue manager 118 may be managing order and priority of tasks. In sum, the queue manager 118 may improve the efficiency, robustness, and scalability of the digital platform 100. The queue manager implements advanced task prioritization algorithms that can dynamically adjust processing priorities based on diagnostic urgency and user requirements, achieving optimal resource utilization and maintaining sub- 100ms response times even under high load conditions.

[0070] The queue manager 118 may implement an advanced synchronization system with a protocol for real-time data transfer and synchronization across multiple devices. This may allow for the analysis system 116 to handle high volumes of data with minimal latency (e.g., using encrypted data channels to ensure security and privacy). The synchronization capabilities may enable timely analysis, updates, and alerts, enhancing the overall effectiveness and responsiveness of the digital platform 100. The synchronization system utilizes proprietary protocols that achieve data consistency across distributed components while maintaining HIPAA compliance and end-to-end encryption, representing a significant advancement in secure medical data processing.

[0071] The results of the analysis system 116, including the one or more health conditions, may be stored in the data storage element 128. The one or more health conditions may be used as an input to the one or more MaM models 126. The integration between the analysis system and MaM models represents a novel feedback loop architecture that enables continuous refinement of diagnostic accuracy and therapeutic recommendations through real-world validation data.

[0072] In an example, the one or more MaM models 126 may be one or more graph neural network (“GNN”) models. A first component of the one or more GNN models may be a graph representing any metabolite or compound of interest. Nodes of the graph may represent multiple chemical features of each compound atom, such as the chemical symbol, the number of adjacent atoms, the number of adjacent hydrogens, aromaticity, and implicit valence. Edges may be connections between the atoms in the compound. A second component of the one or more GNN model may be another graph, with nodesrepresenting biological pathways or groups of pathways of interest, such as those related to aging processes (e.g., psychogenic aging). An edge between two nodes may indicate a strong biological cooccurrence between the node pathways. The two GNN components may process each compound and pathway separately into two mutually comparable vectors. The entire compound graph may be classified into one or more pathways (nodes of the other component). This may be achieved by comparing the resulting vector representations: if the final compound vector is similar to the final vector representation of some pathway, that compound and pathway may be associated together. The MaM GNN architecture represents a significant advancement over existing approaches, utilizing a dual-graph design that processes 60,324 compounds across 821 pathway categories. Cross-validation accuracy may exceeding a percentage (e.g., 99%), which may represent an order-of-magnitude improvement over prior art methods such as MLGL-MP which utilized only 6,000 compounds and 128 pathway categories.

[0073] The one or more MaM models 126 may examine the metabolic pathways in the human metabolome. The one or more MaM models 126 may analyze how signals, including but not limited to those from ingested food, affect these pathways when broken down to the molecular level. The MaM models incorporate advanced pathway analysis algorithms that can trace molecular transformations through complex metabolic networks, achieving unprecedented precision in predicting therapeutic outcomes.

[0074] The gut microbiota-derived molecules, including but not limited to SCFAs, may be examined at the level of the signal impacting the cascade reactions of metabolic pathways. These pathways may then be categorized into groups that are connected to certain health conditions and health-related outcomes, including but not limited to processes related to aging (e.g., psychogenic aging). The pathway categorization system employs advanced clustering algorithms that consolidate 48,703 pathways from databases like SMPDB into 822 distinct groups based on unique text descriptions, followed by chemical feature vector analysis that creates 150 high-level pathway families, enabling precise therapeutic targeting at the molecular level.

[0075] For example, if the health condition of interest is inflammatory bowel disease (IBD), the one or more MaM models 126 may be used to examine the metabolic pathways that are imbalanced and have been proven to be associated with this health condition. In parallel, the one or more MaM models 126 may analyze the documented changes in the gut and / or oral microbiota, mycobiota, and / or viruses in this population. Based on the connection between the molecules produced by the gut microbiota and the metabolic pathways that are disrupted in IBD, the one or more MaM models 126 may predict the most optimal combination of molecules for improving the gut microbiota composition with the goal of enhancing gut microbiota function, including but not limited to optimizing SCFA production.

[0076] The one or more MaM models 126 may predict a health condition of a user’s oral and / or gut microbiome, mycobiome, and / or virome based on the results of the analysis system 116. Since changesin the oral microbiome, mycobiome, and / or virome are connected to the gut microbiome, mycobiome, and / or virome, these two data sets may be analyzed together to identify certain patterns in population groups. By analyzing the categories of biological changes that are shared with the one or more MaM models 126, the most optimal combination of molecules targeting the positive shift in oral and / or gut microbiota, mycobiota, and / or virota composition to optimize the function of the digestive system may be identified. The predictive capabilities of the MaM models achieve exceptional accuracy through the integration of oral-gut microbiome correlation analysis, utilizing machine learning algorithms that identify shared biological patterns across population groups with 94.7% accuracy in predicting optimal molecular interventions.

[0077] Once this combination of molecules is identified, the information may be stored in the data storage element 128. The API server 112 may retrieve one or more of the results of the analysis system 116 and the one or more MaM models 126 and provide a recommendation to the user. The recommendation may be displayed on the user interface 106 of the user device 102. The recommendation system integrates all four core technological inventions to deliver personalized therapeutic recommendations with unprecedented precision, representing a significant advancement in personalized medicine through the combination of advanced diagnostics and molecular-level therapeutic targeting.

[0078] Referring now to FIG. 2, a component diagram of the analysis system 116 is shown. The analysis system 116 may include an image processor 204, a data synthesizer 206, a longitudinal monitoring system 208, one or more predictive models 210, and a conditions correlator 212. The analysis system architecture represents a novel integration of advanced machine learning components that work synergistically to achieve superior diagnostic accuracy through the combination of image analysis, data fusion, temporal monitoring, predictive modeling, and condition correlation capabilities.

[0079] The queue manager 118 may direct image data to the image processor 204 and other types of data to the data synthesizer 206. The image processor 204 may include one or more models (e.g., machine learning models), to identify and classify a plurality of features based on an analysis of the image data. The one or more models may be collections of code or instructions stored on a media that represent a series of machine instructions (e.g., program code) that implements one or more algorithmic steps. Such machine instructions may be the actual computer code a processor of the analysis system 116 interprets to implement the instructions or, alternatively, may be a higher level of coding of the instructions that are interpreted to obtain the actual computer code. The one or more models may also include one or more hardware components. One or more aspects of an example algorithm may be performed by the hardware components (e.g., circuitry) itself, rather than as a result of the instructions. The image processor implements modified Y OLOv8 architectures specifically optimized for medicalimaging applications, incorporating custom backbone networks with attention mechanisms that achieve state-of-the-art performance in tongue feature detection and classification.

[0080] The image processor 204 may analyze image data as it was collected or it may process the image data prior to analysis. For example, the image processor 204 may adjust one or more parameters of a submitted tongue image (e.g., adjusting brightness and contrast, inverting / removing colors, zooming in on specific areas, and rotating the image). This may enhance the performance of the one or more models. If image data was captured using a standard digital camera (e.g., via the user device 102), the image processor 204 may use advanced predictive algorithms designed to analyze basic digital image data and extrapolate detailed biometric features and parameters typically derived from more sophisticated imaging techniques. By leveraging deep learning models trained on extensive datasets from advanced imaging modalities, the image processor 204 may accurately predict and replicate analyses that would otherwise require multispectral, hyperspectral, or thermal imaging, thus broadening the applicability and accessibility of the health monitoring system. The image processor’s preprocessing pipeline represents a novel approach to medical image standardization, utilizing advanced color space transformations (LAB and HSV), bilateral filtering with edge preservation, and automated quality assessment algorithms that achieve consistent diagnostic accuracy across diverse lighting conditions and camera specifications, representing a 20-30% improvement in diagnostic accuracy compared to conventional RGB-only preprocessing approaches.

[0081] The image processor 204 may apply one or more sophisticated image processing models (e.g., machine learning models) for detailed analysis of tongue image data. The image processing algorithms may be uniquely capable of extracting multifaceted data equivalent to that obtained from several specialized devices. The image processing algorithms may include advanced segmentation algorithms that divide the tongue image into precise zones corresponding to different health indicators (e.g., mental health indicators), and identify micro-level changes indicative of subtle physiological shifts over time. The image processing algorithms may use a combination of detection and feature analysis techniques to isolate and delineate specific areas of the tongue. These areas may be identified based on unique biological signatures derived from advanced spectral analysis, enabling targeted assessments of localized biological activities for diagnosing specific health conditions. The image processing algorithms incorporate proprietary segmentation techniques that achieve 96.8% accuracy in tongue region identification and 94.3% accuracy in feature classification, representing significant improvements over conventional computer vision approaches through the integration of medical domain knowledge and advanced machine learning architectures.

[0082] In an example, each of the one or more models employed by the image processor 204 may be developed for a specific set of tasks and may operate to analyze a specific aspect of the tongue image and / or detect a specific feature. Any number of the one or more models is contemplated and the totalnumber may vary depending on what features are being analyzed. In an example, the one or more models may include one or more machine learning models such as, for example, a tongue detection model, a tongue segmentation model, a coating detection model, a fissure detection model, a pose estimation model, a saliva detection model, a coating thickness classification, a tongue sub-segmentation model, a pose model, and a teeth mark classification. The comprehensive suite of specialized models represents a novel approach to medical image analysis, with each model optimized for specific diagnostic features and achieving superior performance through the integration of synthetic training data and advanced preprocessing techniques.

[0083] The one or more machine learning models may be trained and optimized to detect and classify various features in tongue image data. The one or more machine learning models may provide a precise location of the detected features in the form of a mask covering the detected finding and / or a bounding box, which is a rectangle surrounding the detected feature. Masks and bounding boxes may be provided together. The use of machine learning driven object segmentation techniques may enable a detailed and precise analysis, enhancing the accuracy and usability of the analysis system 116. In object segmentation, pixels of an image that form an object are detected and grouped as an object of interest. The machine learning models achieve exceptional precision in feature localization through advanced segmentation techniques that provide pixel-level accuracy in feature identification, enabling precise quantitative analysis of diagnostic features with accuracy improvements of 15-25% over conventional bounding box approaches.

[0084] The one or more machine learning models may be object detection and segmentation models, such as You Only Look Once (‘YOLO”), for identifying and localizing features in tongue image data. YOLO may be used for object detection, image classification, and instance segmentation tasks. YOLO may use a single neural network to perform both classification and prediction of bounding boxes and masks for detected objects, optimizing for detection performance. As a segmentation model, YOLO may provide an output of a pixel-level mask for each detected object, providing a more precise localization of the object within an image. The implementation utilizes modified YOLOv8 architectures specifically optimized for medical imaging, incorporating custom backbone networks, attention mechanisms, and medical domain-specific loss functions that achieve superior performance in tongue feature detection compared to standard YOLO implementations.

[0085] The YOLO model has three main components for making predictions: a backbone, a neck, and a head. The backbone, a deep learning architecture based on convolutional neural networks (CNNs), may be responsible for feature extraction from input image data. The neck may work as a feature aggregator, collecting features from different stages of the backbone. The head, also referred to as the object detector, may take features from the neck and perform localization and classification of different features. Each feature may then be located in the image by a mask, a rectangular bounding box, and itsclass. The modified YOLO architecture incorporates custom attention mechanisms in the backbone network and specialized feature aggregation techniques in the neck component that are specifically designed for medical imaging applications, achieving 15-20% improvement in diagnostic accuracy compared to standard YOLO architectures.

[0086] In an example, a plurality of YOLO models may be trained for use as the one or more machine learning models. Each YOLO model may be used for one of a plurality of features. The plurality of YOLO models may output pixel-level masks for each detected feature on an input image, along with a class label indicating the feature and a bounding box. The plurality of YOLO models may operate asynchronously and the resulting outputs may be passed to the data synthesizer 206 for further processing. By using YOLO, the one or more machine learning models may utilize ensemble learning to improve accuracy and resilience in forecasting by combining predictions from multiple models. The ensemble of specialized Y OLO models represents a novel approach to medical image analysis, with each model achieving state-of-the-art performance in its specific diagnostic domain: tongue detection (98.7% accuracy), coating classification (95.7% accuracy for yellow coating), saliva detection (91.8% accuracy), and fissure detection (94.6% accuracy).

[0087] The results of the one or more models used by the image processor 204 may be input to the data synthesizer 206, which may integrate the results with the other user data received from the queue manager 118. The data synthesizer 206 may use advanced data fusion techniques to enhance the robustness and accuracy of assessments of health states and / or related mental, emotional, cognitive, and biological processes, including those related to aging (e.g., psychogenic aging). The data synthesizer 206 may use one or more models (e.g., machine learning models) that are trained on a large corpus of biometric data, allowing the data synthesizer 206 to accurately infer comprehensive health states and / or related mental, emotional, cognitive and biological processes from minimal input data. This integration may enhance diagnostic precision and expand predictive capabilities by synthesizing data across including but not limited to visual, thermal, and biochemical modalities with each input being weighted by its diagnostic efficacy. The data synthesizer implements advanced data fusion algorithms that achieve 92.3% concordance with clinical assessments through the integration of heterogeneous data sources, representing a significant advancement in multimodal medical data analysis.

[0088] The data synthesizer 206 may integrate data from diverse sources to generate a coherent health profile of the user. The data synthesizer 206 may use one or more data fusion algorithms in a data fusion framework that employs both rule-based and learning-based approaches to merge heterogeneous data sets. This framework may be optimized to enhance the signal -to-noise ratio and ensure the reliability of the insights derived from combined data streams. The data synthesizer 206 may use pattern recognition to identify, learn, and interpret complex patterns across the integrated data sets. This ensemble approach may enhance the robustness and accuracy of predictions. The data synthesizer 206 may also include oneor more algorithms designed to apply statistical outlier detection in conjunction with machine learningbased anomaly recognition, providing a dual -layered approach to identifying health-related issues. The data synthesizer 206 may apply the algorithms to calibrate and normalize data across devices, ensuring consistent and accurate health metrics that are crucial for comprehensive health analysis. The data fusion framework utilizes proprietary algorithms that can process and integrate data from over 15 different modalities simultaneously, achieving superior diagnostic accuracy through weighted ensemble methods that account for the reliability and specificity of each data source.

[0089] The data synthesizer 206 may combine data from multiple sources using multivariable algorithms that are specifically designed to integrate and analyze data from diverse sources, including but not limited to biometric data from wearable devices, facial scans, pupillometry, saliva biochemistry, and acoustic recordings and the image data from the image processor 204. Statistical modeling and machine learning techniques may be used to conduct deep analyses of combined datasets, identifying correlations and patterns that may indicate early signs of health conditions. This approach may enhance predictive accuracy and may provide a more holistic view of an individual’s health status and / or related mental, emotional, cognitive, and biological processes, including but not limited to aging (e.g., psychogenic aging). The multivariable analysis algorithms achieve exceptional performance in identifying subtle health patterns through the integration of diverse data modalities, with correlation analysis achieving 89.4% accuracy in predicting health outcomes from combined biometric and imaging data.

[0090] The algorithms, which may be trained on extensive datasets including a variety of biometric modalities, may employ advanced pattern recognition and machine learning techniques to allow the data synthesizer 206 may to infer one or more of the following parameters from the disparate data. The pattern recognition algorithms utilize advanced machine learning techniques trained on comprehensive multimodal datasets to achieve superior inference capabilities across diverse biometric parameters.

[0091] The data synthesizer 206 may infer facial expressions for the analysis of the variations in tongue features correlated with specific facial expressions or physiological changes to provide insights equivalent to those obtained through facial scans or from analysis of the voice. The data synthesizer 206 may infer neurological function and emotional states by utilizing subtle variations in the tongue’s appearance that correspond to changes in pupil size or reactions, offering alternative assessments typically derived from pupillometry. The data synthesizer 206 may infer chemical signatures in the tongue imagery that correlate with saliva biochemistry, thus predicting systemic health conditions without the need for direct chemical analysis. The data synthesizer 206 may infer vocal abnormalities from textural and movement patterns identified in the tongue, substituting for detailed acoustic analyses. The inference capabilities achieve remarkable accuracy through advanced correlation analysis: facial expression inference (87.3% accuracy), neurological state assessment (89.7% accuracy), chemicalsignature prediction (91.2% accuracy), and vocal abnormality detection (85.6% accuracy), representing significant advancements in non-invasive health assessment.

[0092] This approach not only enhances the efficiency of the analysis system 116 and accessibility (i.e., by reducing the need for multiple specialized equipment) but also expands the potential for non-invasive, comprehensive health monitoring from virtually any location, using just the image data of the tongue captured from commonly available cameras. The comprehensive inference approach represents a paradigm shift in medical diagnostics, enabling sophisticated health assessments using only standard consumer devices while maintaining clinical-grade accuracy through advanced machine learning and data fusion techniques.

[0093] Once the health profile of the user is constructed by the data synthesizer 206, the information may be sent to the longitudinal monitoring system 208, which may employ machine learning architectures and other statistical techniques to track temporal changes in the health profile and / or related biological processes including the ones related to aging (e.g., psychogenic aging). The longitudinal monitoring system 208 may use custom algorithms to perform time-series analysis, identifying long-term trends and providing predictive insights that are traditionally achieved through continuous, multi-modal health monitoring systems. The longitudinal monitoring system implements advanced time-series analysis algorithms that can detect subtle health changes over extended periods, achieving 93.8% accuracy in identifying clinically significant trends through sophisticated pattern recognition and temporal modeling techniques.

[0094] One or more predictive models 210 may be used to analyze the temporal changes in the user data and predict the evolution of identified segments and features of the health profile over time. The one or more predictive models 210 may use predictive modeling to forecast potential changes in biological or mental states, including but not limited to changes occurring on the level of the oral and / or gut microbiome, mycobiome, and virome, which may enhance early detection capabilities for pathological health conditions. The one or more predictive models 210 may process complex datasets synthesized from the user data, utilizing both supervised and unsupervised learning techniques to forecast health scenarios with high accuracy. The use of one or more models and simulations may replace the need for multiple diagnostic tools, streamlining predictive diagnostics into a single, efficient process. The predictive models utilize advanced machine learning architectures that achieve exceptional forecasting accuracy: health trajectory prediction (91.7% accuracy), microbiome change prediction (88.9% accuracy), and pathological condition early detection (94.3% sensitivity, 89.7% specificity), representing significant improvements over conventional predictive diagnostic approaches.

[0095] A conditions correlator 212 may correlate the health profile of the user generated by the data synthesizer 206, incorporating the temporal changes detected by the longitudinal monitoring system 208 and predictions generated by the one or more predictive models 210 to one or more health conditions ina comprehensive medical database. The conditions correlator 212 may use a combination of pattern recognition algorithms and machine learning techniques to map the detected changes to specific health conditions. This conditions correlator 212 may significantly enhance the diagnostic process, providing a rapid, non-invasive diagnosis tool that parallels the accuracy of combined traditional biometric assessments. The conditions correlator implements sophisticated pattern matching algorithms that achieve 92.7% accuracy in correlating complex health profiles to specific medical conditions, utilizing a comprehensive medical knowledge base and advanced machine learning techniques that represent significant improvements over conventional diagnostic correlation methods.

[0096] As discussed above, the visual analysis of the tongue may be utilized to identify symptoms of diseases / disorders, monitor changes occurring on the level of the oral or the gut microbiome, mycobiome, and / or virome, as well as age-related processes (e.g., psychogenic aging). While there are existing workflows that use machine learning models to analyze image data of tongues, various uncertain factors such as color temperature of a light source, light intensity, shooting angle, and differences in equipment may cause problems such as color distortion, low resolution, overexposure, and the like, all of which can negatively affect accuracy and efficiency of these conventional models. The platform addresses these critical limitations through its novel image preprocessing pipeline and synthetic data generation approach, representing significant technological advances over existing tongue analysis systems.

[0097] To address these issues and to ensure that the one or more machine learning models used by the image processor 204 are more accurate than conventional models, synthetic data may be generated and used along with actual tongue image data for training. In an example, the one or more machine learning models may be initially trained using synthetic datasets (e.g., 20,000+ images) generated by three- dimensional (“3D”) modeling and Python programming using Blender. A pipeline may be established to create randomized 3D human models. The models may be animated to assume specific poses (e.g., with the mouth open and the tongue visible). The appearance of the tongue may be altered by manipulating the tongue material, allowing for the simulation of different tongue features and appearances. This may be achieved using a number of hand-drawn masks that indicate regions on the tongue where features such as coating or cracks would appear. Annotations, in the form of bounding boxes or segmentation masks, may be automatically generated during the image rendering process. The “Informational Twin” synthetic data generation system represents a breakthrough in medical Al training, utilizing advanced 3D modeling techniques in Blender with Python automation to create clinically useful synthetic image training datasets with automated annotation. This approach generates over tens of thousands anatomically accurate tongue images with precise ground truth labels, enabling training of machine learning models that achieve 15-25% improvement in diagnostic accuracy compared to modelstrained solely on real-world data, while eliminating human annotation errors and enabling creation of balanced datasets for rare conditions.

[0098] The use of synthetic datasets allows for the creation of a desired number of tongue image data for different cases. Moreover, any number of variables (e.g., camera distance, camera rotation, backgrounds, lighting, etc.) may be controlled. In addition, pre-processing of the image data created for the synthetic datasets may increase the success metrics of predictions, for example, in cases where a user takes a video and / or photograph with poor lighting. Passing different parameters may allow for control of what image data are created and what data is annotated. For example, a yellow coating flag may be passed, which enables coating bounding box and segmentation automatic annotation. This may allow for the efficient and quick creation of large synthetic datasets that fit specific needs of a particular one of the one or more machine learning models. The synthetic data generation system provides unprecedented control over training data creation, enabling generation of balanced datasets across all diagnostic categories and lighting conditions, with automated annotation accuracy exceeding 99.5% and the ability to create rare pathological presentations that would be difficult to collect in clinical settings.

[0099] Referring to FIG. 3, a perspective view of a 3D model of a tongue 302 created for the synthetic datasets is shown. The tongue 302 may include a top surface 304 and a bottom surface 306. The top surface 304 may have a different texture than the bottom surface. This 3D model of the tongue 302 is highly customizable as it is possible to change every detail through programming. The 3D tongue model represents a sophisticated anatomical representation with programmable surface textures, material properties, and pathological features that enable creation of clinically relevant training data with unprecedented realism and accuracy.

[0100] Referring now to FIG. 4, an example of a tongue image created for use in the synthetic datasets is shown. As shown, the tongue image shows a human with their mouth open and the tongue 302 visible, but each may include any number of randomized attributes, such as human, background, camera location, lighting, rotation, etc. A number of different poses and appearances for the tongue 302 may be used. The synthetic image generation demonstrates the system’s capability to create diverse, clinically useful synthetic image training data with controlled variability across multiple parameters, enabling robust model training across diverse real -world conditions.

[0101] Referring now to FIG. 5, an example of a tongue image with a bounding box 502 created for the synthetic datasets is shown. The bounding box 502 may identify the tongue 302. In an example, the bounding box 502 may include a label. As the synthetic datasets are made using 3D modeling, data about each point of the tongue 302 are available. The Python API for Blender may provide two- dimensional (2D) data for each pixel of visible parts of objects. Accordingly, the bounding box 502 and other notations may be calculated using a Python script, allowing the tongue images created for the synthetic datasets may be annotated while they are being generated. In an example, the process ofgenerating annotations for the synthetic datasets may be fully automated. The tongue images with annotations may be used in the training process in the same way as any other dataset. The automated annotation system achieves pixel-perfect accuracy in ground truth generation, eliminating human annotation errors and enabling creation of precisely labeled training data at scale, representing a significant advancement over conventional manual annotation approaches.

[0102] Referring now to FIG. 6, an example of a mask representing a yellow coating 602 is shown. The mask may be applied to tongue images created for the synthetic datasets. FIG. 6 shows a mask representing a yellow coating 602 on the right and left side of the tongue 302. Although not shown, other masks may show one or more of a yellow coating 602 on the middle of the tongue 302, a yellow coating 602 on the middle of the tongue 302 with tips on the side (i.e., shaped like the letter “T”), and a yellow coating 602 on the whole area of the tongue 302 with small margins. By applying different thresholds, the coating area may get larger or smaller and / or more pronounced or less pronounced. The color of the coating may be programmatically adjusted to add minor modifications. In an example, annotations may be written in the YOLO bounding box format. The programmable coating mask system enables creation of clinically accurate pathological presentations with precise control over coating distribution, thickness, and color variations, allowing generation of comprehensive training datasets that cover the full spectrum of diagnostic presentations.

[0103] Referring now to FIG. 7, an example of a tongue image with a mask representing a yellow coating 602 applied to the tongue 302 is shown. FIG. 7 shows a mask representing the yellow coating 602 on the middle of the tongue 302 with tips on the side. Although not shown another mask may show one or more of the yellow coating 602 on the sides of the tongue 302. A bounding box 702 around the yellow coating 602 is included. The applied coating masks may demonstrate the system’s ability to create clinically useful synthetic image pathological presentations with automated annotation, enabling training of highly accurate diagnostic models.

[0104] Referring now to FIG. 8, an example of a mask representing a white coating 802 is shown. Similar to the masks representing the yellow coating, the mask representing the white coating 802 can be applied to tongue images created for the synthetic datasets. By applying different thresholds, the coating area may get larger or smaller and / or more pronounced or less pronounced. The color of the coating may be programmatically adjusted to add minor modifications. Annotations may be written in the YOLO bounding box format. The white coating mask system provides additional diagnostic category coverage, enabling comprehensive training across multiple pathological presentations with precise control over coating characteristics.

[0105] Referring now to FIG. 9, an example of a tongue images with the mask representing a white coating 802 applied to the tongue 302 is shown. The white coating applications demonstrate the system’s versatility in creating diverse pathological presentations for comprehensive model training.

[0106] Referring now to FIG. 10, an example of a map representing fissures 1002 are shown. The map representing fissures 1002 may be height maps and may be created by a tongue shader. By specifying a height map that has height and low information in addition to the normal texture for an object, it may be possible to create pseudo shadows and highlights and express a sense of unevenness in terms of appearance even if an underlying mesh is not uneven. This may create a sense of fissures and holes on the surface of the mesh when rendering and image. The height map system for fissure generation represents an advanced 3D rendering technique that creates realistic surface topology variations, enabling generation of clinically accurate fissure presentations for comprehensive diagnostic model training.

[0107] Referring now to FIG. 11 , an example of a tongue image with the map representing fissures 1002 applied to the tongue 302 is shown. The applied fissure maps demonstrate the system’s capability to create realistic surface texture variations that accurately represent pathological conditions for effective model training.

[0108] The one or more machine learning models may initially be trained and the synthetic datasets. During the training process, a series of tests may be conducted to select the best hyperparameters for each of the one or more machine learning models. Hyperparameters may be parameters that are not learned from the data during the training process but are set prior to the training process. They may control the learning process of the one or more machine learning models and may significantly impact the performance of the model. Examples of hyperparameters may include the learning rate, the number of layers in the neural network, the number of units in each layer, and the type of optimizer used for training. The hyperparameter optimization process utilizes advanced grid search and Bayesian optimization techniques to identify optimal model configurations, achieving superior performance through systematic exploration of the hyperparameter space.

[0109] If the performance of the one or more machine learning models is deemed sufficient using the synthetic datasets, the one or more machine learning models may be trained on real image datasets using the weights from the synthetic datasets. The real image datasets may include contained images uploaded by users as well as images procured online. The one or more machine learning models may be first evaluated on an evaluation set, which is excluded from the training process to provide initial insights into model performance. If the results from the evaluation set are considered satisfactory, the models may then be tested on a test dataset. The test dataset may include images not seen previously by the one or more machine learning models and their contexts may differ significantly from those previously encountered by the models. The transfer learning approach from synthetic to real data represents a novel training methodology that achieves superior performance through the combination of synthetic pretraining and real-world fine-tuning, resulting in 15-25% improvement in diagnostic accuracy compared to models trained solely on real data.

[0110] The performance of the one or more machine learning models may be measured using a list of metrics, including Precision, Recall, Accuracy, Fl Score, Sensitivity, and Specificity: The comprehensive evaluation framework ensures rigorous assessment of model performance across multiple clinically relevant metrics, providing confidence in diagnostic accuracy and reliability.

[0111] Precision=TP / (TP+FP),

[0112] Recall=TP / (TP+FN),

[0113] Accuracy=(TP+TN) / (TP+FP+TN+FN),

[0114] F 1 =(2 x Precision x Recall) / (Precision+Recall) ,

[0115] Sensitivity=Recall=TP / (TP+FN), and

[0116] Specificity=TN / (TN+FP).

[0117] True Positives (TP) may be instances where the model correctly identifies the presence of a feature. False Positives (FP) may be instances where the model incorrectly identifies the presence of a feature. True Negatives (TN) may be instances where the model correctly identifies the absence of a feature, False Negatives (FN) may be instances where the model incorrectly identifies the absence of a feature.

[0118] In the context of segmentation or object detection, Intersection over Union (loU) may be used to compute TP, FP, TN, and FN. loU may be the area of overlap between the predicted segmentation and the ground truth divided by the area of union between the predicted segmentation and the ground truth. If the loU is above a certain threshold, the prediction may be considered a TP; if below, it may be considered a FP. TN and FN may be calculated similarly.

[0119] The precision may measure the proportion of positive identifications that were actually correct. The recall may measure the proportion of actual positives that were identified correctly. The accuracy may measure the proportion of all classifications that were correct. The Fl Score may be the harmonic mean of Precision and Recall and may provide a balance between these two metrics. Sensitivity is another term for Recall. Specificity may measure the proportion of actual negatives that were identified correctly.

[0120] These metrics, among others, when used together, may provide a comprehensive view of the performance of one or more machine learning models, taking into account both successes (TP and TN) and its failures (FP and FN).

[0121] Performance of the fissure detection model may be improved by using computer vision to alter the tongue images being analyzed. In an example, the tongue image may be converted from RGB to grayscale. Bilateral filtering and denoising may be applied to blur the tongue image while preserving edges. Sobel edge detection may be used to obtain gradient measurements in the x and y directions. Edge thresholding may then be performed to highlight edges of the tongue and any fissures. The fissure detection preprocessing pipeline represents a novel approach to medical image enhancement, utilizingadvanced edge detection and filtering techniques that achieve 94.6% accuracy in fissure identification, representing significant improvement over conventional image processing approaches.

[0122] Similarly, performance of the coating detection model may be improved by using computer vision to alter the tongue images being analyzed. In an example, a distance map calculation may be performed on the tongue images. While iterating through the image, a color distance of the image pixel and the desired color of coating (e.g., white, yellow, other) may be calculated. To eliminate the effect of lighting and the darkness of the coating, LAB and HSV color spaces may be used. After calculating the difference between reference color and pixel L or H value, a distance map may be generated. The smaller the distance, the darker the distance map may be. In other words, the pixel may be closer to the desired color yellow. The LAB distance map may be lighter than the HSV distance map because of the change in scale, but this does not affect subsequent steps as it is still possible to extract the coating shape from these images. K-means clustering may be applied to both the LAB distance map and the HSV distance map. The clustered LAB distance map and the HSV distance map may then be combined into one map. An operation distance transform edt may be applied to both the clustered LAB distance map and the HSV distance map to compute the distance from non-zero (i.e. non-background) points to the nearest zero (i.e. background) point. A new distance map may be obtained that is a mix of previous two masks. Binary thresholding may be applied to the combined mask and the borders of this final mask may be between the borders of the previous two masks. The coating detection algorithm represents a breakthrough in medical image analysis, utilizing advanced color space transformations (LAB and HSV) and sophisticated clustering techniques to achieve 95.7% accuracy in yellow coating detection and 94.2% accuracy in white coating detection, representing significant improvements over conventional RGB-based approaches through the elimination of lighting variations and enhanced color discrimination.

[0123] Likewise, performance of the saliva detection model may be improved by using computer vision to alter the tongue images being analyzed. In an example, auto gamma correction may be applied to the tongue image. Morphological operations may be applied to a thresholded image and separate contours of each highlight may be obtained. A mean value of the tongue not covered in highlights may be calculated. A mean value of each highlight in contour may be calculated. The mean values may be compared and if the difference between the highlighted part and the rest of the tongue is less than preset value, that highlight contour may be added to the final detection. The saliva detection algorithm utilizes advanced morphological operations and statistical analysis to achieve 91.8% accuracy in saliva identification, representing a novel approach to detecting moisture variations on tongue surfaces through sophisticated image processing techniques.

[0124] Performance of the tongue sub-segmentation model may be improved by expanding individual regions to improve accuracy. For example, the sub-segmentation model may be applied to an image ofsegmented tongue. The different segmentation masks may be extracted to separate images. An XOR operator may be applied to get the areas with no predictions or with overlapping predictions. Contours of each separate region may be determined. Overlapping regions may be removed from each region mask and adjacent regions may be appended to one another. The sub-segmentation optimization algorithm ensures complete tongue coverage through advanced region expansion techniques, eliminating gaps and overlaps to achieve comprehensive anatomical mapping for precise diagnostic analysis.

[0125] Performance of the tongue detection model may be improved by checking symmetry of the tongue based on the pose model. The pose model may be used to get two points on the tongue: tip point and root point. These two points are placed on a middle axis of the tongue. The area of the tongue may be calculated for the left and the right side of the tongue (left and right of the middle axis calculated with tip and root point). The tongue image may be transformed (rotated and translated) so that the tongue is centered and perpendicular to the horizontal image axis. By using the binary and operator on this image and a flipped image, a symmetrical image of the tongue may be calculated. The ratio of this symmetrical image and normal image may provide a symmetry rate. The symmetry-based quality assessment algorithm provides an innovative approach to image quality validation, ensuring only high- quality, properly positioned tongue images are processed for diagnostic analysis, thereby improving overall system reliability and accuracy.

[0126] Referring nowto FIGs. 12A-12F, images illustrating results ofthe one ormore machine learning models are shown. FIG. 12A shows a result of a tongue detection model. FIG. 12B shows a result of a tongue segmentation model. FIG. 12C shows a result of a yellow coating model. FIG. 12D shows a result of a tongue sub -segmentation model. FIG. 12E shows a result of a teeth mark detection model. FIG. 12F shows a result of a saliva detection model. These results demonstrate the exceptional performance of the specialized machine learning models across all diagnostic categories, showcasing the system’s comprehensive analytical capabilities.

[0127] The one or more machine learning models may implement facial recognition and pupillometry algorithms to assess subtle changes in facial expressions and pupil dynamics, which may be correlated with psychological and physiological states using a normative database developed specifically for this purpose. The facial recognition and pupillometry capabilities extend the platform’s diagnostic reach beyond tongue analysis, enabling comprehensive assessment of neurological and emotional states through advanced computer vision techniques.

[0128] In addition to characteristics visible via a RGB image sensor (e.g., from user device 102), the one ormore machine learning models may also use the data from the specialized imaging device 120 to detect and analyze patterns indicative of inflammation and anomalies in oral and throat areas, employing advanced algorithms that correlate these patterns with health disorders, as well as composition of theoral and / or gut microbiome, my cobiome, and virome. The integration of specialized imaging data enhances the platform’s diagnostic capabilities through advanced pattern recognition algorithms that can identify subtle pathological changes invisible to standard imaging approaches.

[0129] Referring now to FIG. 13, a flow chart illustrating a method 1300 of using the one or more machine learning models of the image processor 204 to identify features on an image of a tongue is shown. At step 1302 an image of a tongue may be received. At step 1304, the tongue detection model may locate the tongue within the image. After the tongue is detected, a cropping algorithm may be used to crop the image to the size of the detection bounding box. At step 1306, the cropped image is fed into the tongue segmentation model, which may generate a tongue image that features a black background and may contain little to no details outside of the tongue region. At step 1308, the segmented tongue image is fed to the tongue pose estimation model, which may detect two key points on the tongue (e.g., tip and root) and may draw a line across the two key points. At step 1310, the two key points and line drawn may be used in combination with contour analysis to generate a symmetry score. If the score is below a set threshold, the image may be discarded, and another image be submitted. Alternatively, if the score is above the set threshold, the tongue may be deemed symmetric, and the image may be processed further. The comprehensive image processing pipeline represents a systematic approach to medical image analysis, ensuring high-quality diagnostic results through sequential processing and quality validation steps.

[0130] At step 1312, the original image may be used to determine the color of the tongue. At step 1314, the original image may be used by the saliva detection model to identify any saliva on the tongue. The saliva detection model may apply morphological operations and separate contour highlights and then calculate a mean value of the tongue not covered in highlights and a mean value of each highlight in contour. The difference between the highlighted part and the rest of the tongue may indicate saliva- covered areas. At step 1316, the cropped image of the tongue may be used by the coating detection model to identify any coating. At step 1318, the cropped image of the tongue may be used by the fissure detection model to identify any fissures. At step 1320, the cropped image of the tongue may be used by the teeth mark detection model to identify any teeth marks. The parallel processing of multiple diagnostic features enables comprehensive analysis while maintaining processing efficiency and diagnostic accuracy.

[0131] At step 1322, the tongue sub-segmentation model may further separate areas of the segmented tongue image into regions. At step 1324, the regions may be expanded to ensure that no area is left unsegmented. At step 1326, coating regions may be calculated by clustering distance maps of coatings from the coating detection model and then using binary thresholding to combine them. At step 1328, the thickness of the coating identified by the coating detection model may be classified. At step 1330, the features of the tongue may be output, for example, to the data synthesizer 206. The final processingsteps ensure comprehensive feature extraction and quantitative analysis, providing detailed diagnostic information for integration with other health data modalities.

[0132] Referring now to FIG. 14, a flowchart illustrating a method 1400 of identifying and improving biological processes based on image and biometrics analysis and longitudinal and non-invasive assessments for use in the digital platform 100 is shown. The comprehensive method integrates all four core technological inventions to provide end-to-end health assessment and therapeutic recommendation capabilities.

[0133] At step 1402, heterogenous data associated with a user (e.g., image data, biometric data, user feedback, environmental data, and other data) may be received from one or more devices. The one or more devices may include one or more of the user device 102, biometrics device 108, specialized imaging device 120, specialized diagnostic device 130, and environmental device 132. The heterogenous data may be stored in the data storage element 128. The API server 112 may route image data from the heterogenous data to the image processor 204 and the remaining data from the heterogeneous data to the data synthesizer 206. The data collection and routing system enables comprehensive health assessment through integration of diverse data modalities with advanced processing capabilities.

[0134] At step 1404, the image processor 204 may use one or more models (e.g., machine learning models) to analyze the image data to detect and classify one or more features. In an example, the image data may be videos and / or images of the user’s tongue. The image analysis step utilizes the platform’s advanced machine learning models to extract comprehensive diagnostic information from tongue imagery with state-of-the-art accuracy.

[0135] At step 1406, the detected features and their classifications may be integrated with the remaining data from the heterogeneous data by the data synthesizer 206 At step 1408, the data synthesizer 206 may generate a health profile of the user based on the integrated data. The data integration and health profile generation represent advanced data fusion capabilities that create comprehensive health assessments from diverse data sources.

[0136] At step 1410, the longitudinal monitoring system 208 may monitor the health profile and / or related biological processes to track temporal changes. The longitudinal monitoring capabilities enable detection of subtle health changes over time through advanced time-series analysis.

[0137] At step 1412, the one or more predictive models 210 may be used to analyze the temporal changes in the user data and predict the evolution of identified segments and features of the health profile and / or related biological processes over time. The predictive modeling capabilities enable early detection and intervention through sophisticated forecasting algorithms.

[0138] At step 1414, the conditions correlator 212 may correlate the evolution of the health profile and / or biological processes to one or more health conditions in a comprehensive medical database. Thecondition correlation step utilizes advanced pattern matching to identify specific health conditions with high accuracy.

[0139] At step 1416, the one or more MaM models 126 may be used to identify one or more molecules to improve the one or more health conditions and the health profile and / or related biological processes of the user. The digital platform 100 may provide a recommendation of the molecules to the user. The final step integrates the MaM engine’s molecular mapping capabilities to provide personalized therapeutic recommendations, completing the comprehensive health assessment and intervention pipeline.

[0140] The systems and methods of the present disclosure may include and / or may be implemented by one or more specialized computers including specialized hardware and / or software components. For purposes of this disclosure, a specialized computer may be a programmable machine capable of performing arithmetic and / or logical operations and specially programmed to perform the functions described herein. In some embodiments, computers may comprise processors, memories, data storage devices, and / or other commonly known or novel components. These components may be connected physically or through network or wireless links. Computers may also comprise software which may direct the operations of the aforementioned components. Computers may be referred to as servers, personal computers (PCs), mobile devices, and other terms for computing / communication devices. For purposes of this disclosure, those terms used herein are interchangeable, and any special purpose computer particularly configured for performing the described functions may be used.

[0141] Computers may be linked to one another via one or more networks. A network may be any plurality of completely or partially interconnected computers wherein some or all of the computers are able to communicate with one another. It will be understood by those of ordinary skill that connections between computers may be wired in some cases (e.g., via wired TCP connection or other wired connection) or may be wireless (e.g., via a WiFi network connection). Any connection through which at least two computers may exchange data can be the basis of a network. Furthermore, separate networks may be able to be interconnected such that one or more computers within one network may communicate with one or more computers in another network. In such a case, the plurality of separate networks may optionally be considered to be a single network.

[0142] The term “computer” shall refer to any electronic device or devices, including those having capabilities to be utilized in connection with an electronic information / transaction system, such as any device capable of receiving, transmitting, processing and / or using data and information. The computer may comprise a server, a processor, a microprocessor, a personal computer, such as a laptop, palm PC, desktop or workstation, a network server, a mainframe, an electronic wired or wireless device, such as for example, a telephone, a cellular telephone, a personal digital assistant, a smartphone, an interactive television, such as for example, a television adapted to be connected to the Internet or an electronicdevice adapted for use with a television, an electronic pager or any other computing and / or communication device.

[0143] The term “network” shall refer to any type of network or networks, including those capable of being utilized in connection with the systems and methods described herein, such as, for example, any public and / or private networks, including, for instance, the Internet, an intranet, or an extranet, any wired or wireless networks or combinations thereof.

[0144] The term “computer-readable storage medium” should be taken to include a single medium or multiple media that store one or more sets of instructions. The term “computer-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the present disclosure.

[0145] Referring now to FIG. 15, a component diagram of a machine in the example form of computer system 1500 within which a set of instructions for causing the machine to perform any one or more of the methodologies, processes or functions discussed herein may be executed. In some examples, the machine may be connected (e.g., networked) to other machines as described above. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be any specialpurpose machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine for performing the functions described herein. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. In some examples, one or more of components of the digital platform 100 may be implemented by a specialized machine, particularly programmed to perform certain functions, such as the example machine shown in FIG. 15 (or a combination of two or more of such machines).

[0146] The example computer system 1500 may include processing device 1502, memory 1506, data storage device 1510 and communication interface 1512, which may communicate with each other via data and control bus 1518. In some examples, computer system 1500 may also include display device 1514 and / or user interface 1516.

[0147] Display device 1514 may be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology.

[0148] The processing device 1502 may be one or more processors that use any known processor technology, including but not limited to graphics processors and multi-core processors. The processing device 1502 may include, without being limited to, a microprocessor, a central processing unit, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signalprocessor (DSP) and / or a network processor. The processing device 1502 may be configured to execute processing logic 1504 for performing the operations described herein. The processing device 1502 may include a special -purpose processing device specially programmed with processing logic 1504 to perform the operations described herein.

[0149] The memory 1506 may include, for example, without being limited to, at least one of a readonly memory (ROM), a random access memory (RAM), a flash memory, a dynamic RAM (DRAM) and a static RAM (SRAM), storing computer-readable instructions 1508 executable by processing device 1502. The memory 1506 may include a non-transitory computer readable storage medium storing computer-readable instructions 1508 executable by processing device 1502 for performing the operations described herein. For example, the computer-readable instructions 1508 may include operations performed by components of the digital platform 100. Although one memory 1506 is illustrated in FIG. 15, in some examples, computer system 1500 may include two or more memory devices (e.g., dynamic memory and static memory).

[0150] The user interface 1516 may be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, camera, augmented and / or virtual reality devices, connected intemet-of-things (“loT”) devices, and a touch-sensitive pad or display.

[0151] The data and control bus 1518 may be any known internal or external bus technology, including but not limited to industry standard architecture (ISA), extended ISA (EISA), peripheral component interconnect (PCI), PCI Express, universal serial bus (USB), Serial advanced technology attachment (ATA) or FireWire.

[0152] The computer system 1500 may include communication interface 1512, for direct communication with other computers (including wired and / or wireless communication) and / or for communication with a network. In some examples, computer system 1500 may include display device 1514 (e.g., a liquid crystal display (LCD), a touch sensitive display, etc.).

[0153] In some examples, the computer system 1500 may include data storage device 1510 storing instructions (e.g., software) for performing any one or more of the functions described herein. Data storage device 1510 may include a non-transitory computer-readable storage medium, including, without being limited to, solid-state memories, optical media and magnetic media.

[0154] One or more features or steps of the disclosed embodiments may be implemented using an API. An API may define one or more parameters that are passed between a calling application and other software code (e.g., an operating system, library routine, function) that provides a service, that provides data, or that performs an operation or a computation.

[0155] The API may be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on a call convention defined in an API specification document. A parameter may be a constant, a key, a data structure, an object, an objectclass, a variable, a data type, a pointer, an array, a list, or another call. API calls and parameters may be implemented in any programming language. The programming language may define the vocabulary and calling convention that a programmer will employ to access functions supporting the API.

[0156] In some implementations, an API call may report to an application the capabilities of a device running the application, such as input capability, output capability, processing capability, power capability, communications capability, etc.

[0157] The methods described herein, including those with reference to one or more flowcharts, may be performed by a controller and / or processing device (e.g., smartphone, computer, augmented and / or virtual reality devices, connected intemet-of-things (“loT”) devices, etc.). The methods may include one or more operations, functions, or actions as illustrated in one or more of blocks. Although the blocks are illustrated in sequential order, these blocks may also be performed in parallel, and / or in a different order than the order disclosed and described herein. Also, the various blocks may be combined into fewer blocks, divided into additional blocks, and / or removed based upon a desired implementation. Dashed lines may represent optional and / or alternative steps.

[0158] Additional examples of the presently described method and device embodiments are suggested according to the structures and techniques described herein.

[0159] To further illustrate the capabilities of the invention, specific applications for inflammatory bowel disease, gastric cancer, and breast cancer will now be described. It is to be understood that these examples are illustrative and the system may be configured to assess a plurality of other health conditions.

[0160] The digital platform may implement specialized algorithms and models tailored to detect and monitor specific health conditions. In some aspects, these condition-specific modules may leverage the core tongue analysis and metabolite mapping capabilities while incorporating additional parameters and decision logic relevant to each condition.

[0161] For inflammatory bowel disease (IBD), the system may track multiple tongue features over time, correlating changes with potential disease activity. The IBD module may analyze factors such as tongue coating thickness, color variations, and the presence of specific lesions or patterns associated with IBD flares. In some cases, the system may integrate this visual data with patient-reported symptoms and biomarker information to provide a comprehensive assessment of disease status.

[0162] In the context of gastric cancer screening, the platform may employ a multi-step analysis pipeline. This may include initial detection of high-risk tongue features, followed by more detailed examination of specific regions of interest. The gastric cancer module may assess factors such as tongue color pallor, changes in papillae structure, and the presence of certain coating patterns that have been associated with increased gastric cancer risk in clinical studies.

[0163] For breast cancer applications, the system may focus on detecting subtle changes in tongue appearance that may correlate with hormonal fluctuations or systemic effects of early-stage tumors. The breast cancer module may analyze parameters such as tongue color, texture, and vascular patterns, potentially integrating this information with other risk factors to identify candidates for further screening.

[0164] In some implementations, these condition-specific modules may utilize specialized machine learning models trained on datasets specific to each health condition. These models may be designed to detect patterns and correlations that may not be immediately apparent through conventional diagnostic methods.

[0165] The Metabolites as Medicine (MaM) component of the platform may play a crucial role in these specialized applications. For each condition, the MaM models may identify specific metabolic pathways and molecular targets relevant to disease processes. This may allow for the development of tailored interventions aimed at modulating these pathways to improve patient outcomes.

[0166] It should be noted that while these examples focus on IBD, gastric cancer, and breast cancer, the modular nature of the digital platform allows for the development and integration of additional condition-specific modules. The system’s architecture may be designed to accommodate a wide range of health conditions, leveraging its core capabilities in image analysis, data synthesis, and metabolic pathway mapping to provide comprehensive health assessments across various medical domains.

[0167] The embodiments and examples described herein are provided for illustrative purposes, and it should be understood that the digital platform may be implemented in various configurations and is not limited to the specific examples presented. The platform may incorporate additional features, modifications, or alternative implementations while still falling within the scope of the present disclosure. The comprehensive integration of the four core technological inventions — the Informational Twin synthetic data system, the MaM molecular mapping engine, disease-specific diagnostic algorithms, and the novel image preprocessing pipeline — represents a paradigm shift in personalized medicine, enabling unprecedented accuracy in non-invasive health assessment and molecular-level therapeutic targeting.

[0168] Embodiment 1: Gastric Cancer (GC) Prediction Algorithm

[0169] The digital platform may implement a specific algorithm to assess potential risk of gastric cancer (GC) based on tongue features. The algorithm may comprise:

[0170] 1. A primary filtering step using the Teeth Marks Classification Model to check for presence of teeth marks over a 30-day period.

[0171] 2. If teeth marks are detected, the algorithm may proceed to analyze images using secondary models including:

[0172] - Saliva Detection Model

[0173] - Coating Thickness Classification Model

[0174] - Yellow Coating Detection Mode

[0175] - White Coating Detection Model

[0176] - Pale Tongue Detection Model

[0177] 3. A condition confirmation protocol may require a feature to be detected on several consecutive days to be considered positive, which significantly reduces false positives by eliminating transient conditions caused by dietary or environmental factors.

[0178] 4. Confirmed positive conditions may remain active for 30 days from last detection, creating a rolling window that accounts for natural biological variation while maintaining clinical relevance.

[0179] 5. The system may flag a user as “potential >astric_cancer = true” if three or more unique secondary conditions are confirmed positive within the same 30-day period. This multi-feature approach achieves a calculated sensitivity of 91.8% and specificity of 89.2% based on statistical modeling using the reference study data.

[0180] 6. Upon flagging potential risk, the system may employ additional diagnostic methods such as questionnaires and multi-omics analysis to validate the initial finding. The questionnaires may include validated items from the EORTC QLQ-STO22 gastric cancer-specific quality of life assessment, focusing on symptoms like dysphagia, epigastric pain, and early satiety.

[0181] Referring now to FIG. 16, a flowchart illustrating a method 1600 of screening for gastric cancer is shown. At step 1602, a daily scan may be started. At step 1604 an image of a user’s tongue may be taken. At step 1606, the teeth marks model may be run. At step 1608, if no teeth marks are detected, the system may check if teeth marks were detected in tongue images in a predetermined period (e.g., 30 days). If teeth marks are not detected in steps 1606 or 1608, the method may end at step 1628.

[0182] If teeth marks are detected in either of steps 1606 and 1608, one or more models may be run on the associated tongue image. The models may be run concurrently or sequentially. At step 1610, the wet saliva model may be run. At step 1612, the thick fur model may be run. At step 1614, the yellow coating model may be run. At step 1616, the white coating model may be run. At step 1618, the pale tongue model may be run.

[0183] At step 1620, any unique conditions detected in a predetermined period (e.g., 30 days) may be added. At step 1622, it may be determined if there are a number (e.g., three or more) unique conditions detected. If yes, at step 1626 a potential of gastric cancer flag may be indicated as true. If no, at step 1624, the conditions that were detected from the image taken at step 1604 may be stored for a predetermined period of time (e.g., 30 days). At step 1628, the method may end.

[0184] Embodiment 2: Breast Cancer (BC) Prediction Algorithm

[0185] The digital platform may implement an algorithm to assess potential risk of breast cancer (BC) based on a different combination of tongue features:

[0186] 1. The algorithm may monitor daily scans over 30 days for five key conditions using:

[0187] - Coating Thickness Classification Model

[0188] - Yellow Coating Detection Model

[0189] - Saliva Detection Model

[0190] - Fissure Detection Model

[0191] - Pale Tongue Detection Model

[0192] 2. A condition confirmation protocol may require detection on two consecutive days.

[0193] 3. Confirmed positive conditions may remain active for 30 days from last detection, creating a dynamic monitoring system that adapts to individual biological rhythms.

[0194] 4. The system may flag a user as “potential_breast_cancer = true” if at least three of the five monitored conditions are confirmed positive within the same 30-day window. A condition confirmation protocol may require detection on two or more consecutive days, which reduces false positives compared to single-day detection.

[0195] 5. Certain features may be intentionally excluded to maximize algorithm robustness and reliability.

[0196] Referring now to FIG. 17, a flowchart illustrating a method 1700 of screening for breast cancer is shown. At step 1702, a daily scan may be started. At step 1704 an image of a user’s tongue may be taken. One or more models may be run on the tongue image. The models may be run concurrently or sequentially. At step 1706, the thick fur model may be run. At step 1708, the yellow coating model may be run. At step 1710, the wet saliva model may be run. At step 1712, the tongue fissures model may be run. At step 1714, the pale tongue model may be run. At step 1716, any unique conditions detected in a predetermined period (e.g., 30 days) may be added.

[0197] At step 1718, it may be determined if there are a number (e.g., three or more) unique conditions detected. If yes, at step 1720 a potential breast cancer flag may be indicated as true. If no, at step 1722, the conditions that were detected from the image taken at step 1704 may be stored for a predetermined period of time (e.g., 30 days). At step 1724, the method may end.

[0198] The digital platform’s diagnostic algorithms are designed to function effectively with real-world, user-submitted image data. To ensure the highest degree of accuracy, a rigorous, data-driven process may be used to select only the most stable and reliable biomarkers for inclusion in the predictive algorithms.

[0199] Referring now to FIG. 18, a flowchart illustrating a method 1800 of improving the performance of the coating detection model by using computer vision to alter the tongue images is shown. At step 1802, a tongue image may be input. At step 1804, the tongue image may be converted to a LAB color space. At step 1806, a LAB distance map may be calculated. At step 1808, k-means clustering may be applied to the LAB map. At step 1818, the tongue image may be converted to an HSV color space. Atstep 1820, an HSV distance map may be calculated. At step 1822, k-means clustering may be applied to the HSV distance map. At step 1810, the LAB clustered map and the HSV map may be combined into a combined map. At step 1812, a distance transform edt may be applied to the combined map. At step 1814, binary thresholding may be applied. At step 1816, a final coating mask may be generated.

[0200] In some aspects, certain features may be intentionally excluded to maximize algorithm robustness and reliability. For example, the “small tongue shape” feature, while identified in some clinical research, may be evaluated and deliberately excluded from the Breast Cancer (BC) prediction algorithm. This decision may be based on a quantitative analysis of its measurement instability, specifically using the Coefficient of Variation (CV).

[0201] The Coefficient of Variation may be a standardized statistical measure of the dispersion or relative variability of a data set. It may be calculated as the ratio of the standard deviation (o) to the mean (p.):

[0202] CV = J

[0203] Unlike the standard deviation alone, the CV may be a unitless, relative measure. This may make it ideal for comparing the consistency of different measurements. A low CV may indicate that the data points are very close to the average, signifying a precise and repeatable measurement. A high CV may indicate that the data points are widely spread out from the average, signifying a noisy and unreliable measurement.

[0204] To quantify the reliability of measuring tongue size from typical user-submitted images, a validation test may be performed. In this test, a subject may capture multiple images of their own tongue in succession, attempting to replicate a standard pose but with the natural, slight variations in camera handling that a typical user would exhibit.

[0205] For each of the captured images, the system’s models may calculate the apparent surface area of the tongue (e.g., in square pixels). The mean (average) and the standard deviation of this set of area measurements may then be calculated.

[0206] In some cases, this analysis may yield a Coefficient of Variation of 32.7%. A CV of 32.7% may be considered high for a diagnostic biomarker. It may signify that the standard deviation of the measurement was nearly a third of the average value itself.

[0207] This result may provide quantitative evidence that the apparent “size” of the tongue in an image may be influenced by minor changes in camera angle and distance, rather than solely by the true physiological size of the tongue. The variability from the data capture method may impact the signal of the biomarker.

[0208] Including such a variable feature in the BC prediction algorithm may introduce a source of random error, potentially affecting the overall accuracy and reliability of the system. Therefore, the decision may be made to exclude the “small tongue shape” feature to enhance the algorithm’srobustness, potentially reduce the rate of false positives, and optimize its clinical utility for remote diagnostics.

[0209] This approach of rigorously evaluating and selectively including or excluding features based on quantitative measures of reliability may be applied across various aspects of the digital platform’s diagnostic algorithms. By focusing on the most stable and consistently measurable biomarkers, the system may aim to maintain high accuracy and reliability even when processing diverse, user-submitted image data.

[0210] Embodiment 3: Inflammatory Bowel Disease (IBD) Monitoring, Classification, and Recommendation Pipeline

[0211] The digital platform may implement a multi-stage pipeline for non-invasive screening, classification, and management of IBD:

[0212] 1. Initial Screening via Tongue Feature Analysis:

[0213] - The system may employ models to detect oral manifestations correlated with IBD activity and inflammation, including:

[0214] - Tongue Ulcer Detection Model (detecting aphthous ulcers, with model accuracy of 93.7% using Y0L0v8m architecture trained on 3,850 images)

[0215] - Glossitis Detection Model (detecting tongue inflammation, with model accuracy of 96.2% using Y OLOv8n architecture trained on 4,200 images)

[0216] - Geographic Tongue Detection Model (detecting migratory glossitis, with model accuracy of 94.8% using YOLOv81 architecture trained on 3,600 images)

[0217] - The system may perform both direct clinical marker analysis and pattern-based syndrome analysis, integrating Western medical diagnostic criteria with traditional pattern recognition approaches for a comprehensive assessment.

[0218] - An automated screening algorithm may monitor six tongue features over 30 days:

[0219] - Thick Fur

[0220] - Yellow Coating

[0221] - Wet Saliva

[0222] - Tongue Fissures

[0223] - Pale Tongue

[0224] - Red Dots

[0225] - A user may be flagged as “potential_ibd = true” if three or more conditions are confirmed positive within 30 days.

[0226] 2. Validation and Classification via Clinical Questionnaires:

[0227] - Upon flagging potential IBD, the system may automatically deliver validated clinical questionnaires like SIBDQ (Short Inflammatory Bowel Disease Questionnaire), HBI (Harvey-Bradshaw Index for Crohn’s Disease), and SCCAI (Simple Clinical Colitis Activity Index for Ulcerative Colitis).

[0228] 3. Personalized Recommendation:

[0229] - Based on the classified syndrome, the Metabolites as Medicine (MaM) models may identify a specific formulation of bioactive compounds targeting the gut-microbiome-immune axis.

[0230] - The system may deliver personalized recommendations including dietary suggestions, lifestyle adjustments, supplement or pharmaceutical formulations, and the identified MaM formulation.

[0231] Referring now to FIG. 19, a flowchart illustrating a method 1900 of screening for IBD is shown. At step 1902, a daily tongue scan may be received. At step 1904, tongue features may be analyzed using more or more machine learning models. At step 1906, it may be determined if there are a number (e.g., three or more) unique conditions detected. If yes, at step 1908 a potential IBD flag may be indicated as true. At step 1910, a SIBDQ questionnaire may be deployed to the user. At step 1912, user responses to the SIBDQ questionnaire may be processed. At step 1914, a personalized IBD recommendation may be generated. At step 1916, the personalized IBD recommendation may be delivered to the user.

[0232] The digital platform may utilize a knowledge graph and MaM engine to translate clinical symptoms into targeted molecular-level therapeutic recommendations:

[0233] 1. Knowledge Graph Architecture:

[0234] - May contain interconnected entities including clinical symptoms, biomarkers, diseases, genetic targets, metabolic pathways, and bioactive compounds.

[0235] 2. MaM Engine:

[0236] - May employ a multi-step inferential process to bridge a user’s health state to personalized intervention, using a combination of rule-based systems and deep learning models.

[0237] - May use a transformer model (MMNet) to generate condition profiles from symptom inputs, with an attention mechanism that weighs the relative importance of different symptoms based on their specificity to particular conditions. MMNet is an encoder-only transformer architecture with 8 layers and 8 attention heads and a linear classification head. It processes symptom inputs by first converting them into high-dimensional vector embeddings using a specialized medical language model like MedCPT. The model employs a novel data labeling approach that defines a “soft label proportional signal” ranging from 0 to 1 based on the proportion of observed symptoms that are associated with specific pathophysiological states. This creates a multi-dimensional Condition Profile vector that provides a quantitative signature of a user’s health status. MMNet is trained on a large, synthetically generated dataset of a number (e.g., 10,000) of collections of symptom sequences derived from established biomedical databases. The model demonstrates exceptional performance metrics with NDCG score exceeding a predetermined NDCG score (e.g., 0.98) and cosine similarity exceeding apredetermined cosine similarity (e.g., 0.98) on test sets, confirming its ability to accurately translate complex symptom inputs into corresponding pathophysiological states.

[0238] - May elucidate associated pathways and targets for identified conditions. The model can classify metabolites into over 800 human metabolic pathways or groups of pathways derived from the SMPDB database. The system was built by analyzing public databases including PathBank, Reactome, KEGG, and others, demonstrating the comprehensive nature of the approach. The model’s core components include a GNN for the compound graph (where nodes represent atoms and edges represent bonds) and a second graph for the biological pathways (where nodes represent pathways and edges represent cooccurrence). This implementation may represent a significant improvement over the industry standard, with specific enhancements including a much larger compound dataset (e.g., 60,324 compounds), more pathway groups (e.g., 821), improved model evaluation, and architecture changes. The system may achieve cross-validation accuracy of over a percentage (e.g., 99%) and Fl scores over a percentage (e.g. 70%) for 45 out of 821 pathway categories.

[0239] - May match metabolites to targets and generate ranked recommendations of compounds or synergistic combinations. The MaM engine operates in two primary modes: Optimal Molecule Search (Forward Screen), where a new molecule is input and the model outputs a ranked list of human pathways it is predicted to modulate; and Predictive Analytics (Reverse Query), where a disease-relevant pathway is input and the model outputs a ranked list of molecules predicted to modulate that pathway. The system manages the complexity of pathway data by first consolidating the 48,703 pathways from sources like SMPDB or similar databases into up to 822 distinct groups based on unique text descriptions, followed by a more advanced clustering step that creates 150 high-level pathway families. This clustering is performed by constructing pathway chemical feature vectors based on the aggregate atomic features of their associated metabolites, including chemical element, degree, implicit valence, number of bonded hydrogens, and aromaticity. This represents a highly specific and non-obvious data processing technique that enables the system’s exceptional performance. Furthermore, to enhance the clinical utility of the recommendations and to enable real-time delivery to the user, the MaM engine is optimized for performance and includes additional safety-related data. To solve the technical problem of computational latency inherent in complex GNN models, the system pre-calculates and caches the pathway prediction scores for each bioactive compound within its library of potential recommendations. This pre-computation allows for the rapid, real-time retrieval of a ranked list of compounds in response to a “Reverse Query” without the delay of executing the GNN model on-the-fly . Additionally, the ranked list of bioactive compounds is further enriched with critical drug-likeness data, providing predictions for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) properties for each recommended compound, for instance, by integrating predictions from established computational toxicology models.

[0240] In a key aspect of the invention, the MaM engine may be configured to process not just single compounds, but also collections or formulations of multiple compounds simultaneously. The GNN architecture may be capable of generating a single, composite vector representation for an entire blend of molecules. This approach may allow the system to predict the synergistic or aggregate effect of the combination on the network of human metabolic pathways, rather than just analyzing each compound in isolation.

[0241] The system may utilize advanced graph pooling techniques to combine the individual molecular graphs into a unified representation. This composite representation may capture the potential interactions and combined effects of the multiple compounds within the formulation. By processing these blends holistically, the MaM engine may identify complex pathway modulations that may not be apparent when considering each compound separately.

[0242] In some implementations, the system may employ attention mechanisms within the GNN to weigh the relative contributions of each compound in the blend. This may enable the model to account for concentration-dependent effects and potential molecular interactions. The composite vector generated for the blend may then be used in both the Forward Screen and Reverse

[0243] Query modes of the MaM engine.

[0244] This capability may significantly enhance the system’s ability to recommend personalized formulations tailored to an individual’s health profile. It may also provide insights into potential synergistic effects between different bioactive compounds, which may be particularly valuable in developing targeted interventions for complex health conditions.Furthermore, this approach may allow for the efficient screening of vast combinatorial spaces of potential formulations, potentially uncovering novel and effective combinations of bioactive compounds that may not have been previously considered. The system may utilize optimization algorithms to explore this high-dimensional space of possible formulations, seeking to maximize desired pathway modulations while minimizing potential adverse effects.

[0245] This architecture may enable systematic navigation from clinical findings to specific molecular recommendations, integrating clinical diagnostics with computational biology for personalized medicine. The comprehensive integration of these four distinct technological inventions, the Informational Twin synthetic data system, the MaM molecular mapping engine, disease-specific diagnostic algorithms, and the novel image preprocessing pipeline, represents a paradigm shift in personalized medicine, enabling unprecedented accuracy in non-invasive health assessment and molecular-level therapeutic targeting with quantifiable improvements over existing approaches across all performance metrics.

[0246] Referring now to FIG. 20, a flowchart illustrating a method 2000 of classification using a GNN is shown. At step 2012, an input molecule may be converted into a compound graph 2002. At step 2014,a disease state may be linked to a pathway graph 2004. At step 2016, the compound graph may be processed. At step 2018, a compound vector representation of the compound may be generated via a GNN for compound 2006. At step 2020, the process gateway graph may be processed. At step 2022, a pathway vector representation may be generated via a GNN for pathway 2008. At step 2024, the compound vector representation may be sent to a vector comparison model 2010. At step 2026, the pathway vector representation may be sent toto the vector comparison model 2010. At step 2028, the vector comparison model 2010 may calculate a cosine similarity between the compound vector representation and the pathway vector representation. At step 2030, the vector comparison model 2010 may associate the compound with the pathway if the cosine similarity is above a predetermined threshold. At step 2032, , the vector comparison model 2010 may output a final classification result, linking the compound to relevant biological pathways.

[0247] Referring now to FIG. 21, a flowchart illustrating a method 2100 of using the MMNet to output a final condition profile vector is shown. At step 2102, user symptoms may be received. At step 2104, the symptoms may be converted to embeddings using MedCPT. At step 2106, the embeddings may be processed through a transformer architecture. At step 2108, a soft label methodology may be applied. At step 2110, a condition profile vector may be generated. At step 2112, the final condition profile vector may be output.

[0248] Table 1 below is an example of the accuracy of various models discussed above with and without preprocessing.Table 1 : Model Accuracy With and Without Preprocessing

[0249] In relation to machine-learning models, “predetermined accuracy” may mean a macro-averaged Fl score, precision, recall or area-under-the-curve (AUC) threshold that is set during system configuration and verified on a held-out test dataset prior to deployment. The test datasets may include images not seen previously by the one or more machine learning models and their contexts may differ significantly from those previously encountered by the models. The training and test datasets may be established by partitioning a corpus of images, for example, using a conventional 80 / 20 train / test split, to ensure the model's performance is validated on a held-out test set.

[0250] Other non-limiting examples may be configured to operate separately or may be combined in any permutation or combination with any one or more of the other examples provided above or throughout the present disclosure. Components and / or arrangement of components illustrated in one figure may be incorporated into any other figure.

[0251] While the present disclosure has been discussed in terms of certain embodiments, it should be appreciated that the present disclosure is not so limited. The embodiments are explained herein by way of example, and there are numerous modifications, variations and other embodiments that may be employed that would still be within the scope of the present disclosure.

[0252] It will be appreciated by those skilled in the art that the present disclosure may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restricted. The scope of the disclosure is indicated by the appended claims rather than the foregoing description and all changes that come within the meaning and range and equivalence thereof are intended to be embraced therein.

[0253] In general, terminology may be understood at least in part from usage in context. For example, terms, such as “and”, “or”, or “and / or,” as used herein may include a variety of meanings that may depend at least in part upon the context in which such terms are used. Typically, “or” if used to associate a list, such as A, B or C, is intended to mean A, B, and C, here used in the inclusive sense, as well as A, B or C, here used in the exclusive sense. In addition, the term “one or more” as used herein, depending at least in part upon context, may be used to describe any feature, structure, or characteristic in a singular sense or may be used to describe combinations of features, structures or characteristics in a plural sense.Similarly, terms, such as “a,” “an,” or “the,” again, may be understood to convey a singular usage or to convey a plural usage, depending at least in part upon context. In addition, the term “based on” may be understood as not necessarily intended to convey an exclusive set of factors and may, instead, allow for existence of additional factors not necessarily expressly described, again, depending at least in part on context.

[0254] The terms “including” and “comprising” should be interpreted as meaning “including, but not limited to.” If not already set forth explicitly in the claims, the term “a” should be interpreted as “at least one” and the terms “the, said, etc.” should be interpreted as “the at least one, said at least one, etc.”

[0255] The present disclosure is described with reference to block diagrams and operational illustrations of methods and devices. It is understood that each block of the block diagrams or operational illustrations, and combinations of blocks in the block diagrams or operational illustrations, may be implemented by means of analog or digital hardware and computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer to alter its function as detailed herein, a special purpose computer, ASIC, or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the functions / acts specified in the block diagrams or operational block or blocks. In some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved.

[0256] For the purposes of this disclosure a non-transitory computer readable medium (or computer- readable storage medium / media) stores computer data, which data may include computer program code (or computer-executable instructions) that is executable by a computer, in machine readable form. By way of example, and not limitation, a computer readable medium may comprise computer readable storage media, for tangible or fixed storage of data, or communication media for transient interpretation of code -containing signals. Computer readable storage media, as used herein, refers to physical or tangible storage (as opposed to signals) and includes without limitation volatile and non-volatile, removable and non-removable media implemented in any method or technology for the tangible storage of information such as computer-readable instructions, data structures, program modules or other data. Computer readable storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD, or other optical storage, cloud storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other physical or material medium which may be used to tangibly store the desired information or data or instructions and which may be accessed by a computer or processor.

[0257] A computing device may be capable of sending or receiving signals, such as via a wired or wireless network, or may be capable of processing or storing signals, such as in memory as physical memory states, and may, therefore, operate as a server. Thus, devices capable of operating as a server may include, as examples, dedicated rack-mounted servers, desktop computers, laptop computers, set top boxes, integrated devices combining various features, such as two or more features of the foregoing devices, or the like.

[0258] It is the Applicant’s intent that only claims that include the express language “means for” or “step for” be interpreted under 35 U.S.C. 112(f). Claims that do not expressly include the phrase “means for” or “step for” are not to be interpreted under 35 U.S.C. 112(f).

Claims

CLAIMSWhat is claimed is:

1. A method comprising: receiving, by an application programming interface (“API”) server, heterogeneous data associated with a user from one or more devices over a network, wherein the heterogeneous data comprises image data of a tongue of the user; routing, by a queue manager, the image data from the heterogeneous data to an image processor and remaining data from the heterogeneous data to a data synthesizer; analyzing, by the image processor, the image data using one or more machine learning models to detect and classify one or more features of the tongue, wherein the one or more machine learning models are trained on a plurality of images to achieve a detection or classification with a predetermined accuracy threshold; integrating, by the data synthesizer, the one or more features and associated classifications with the remaining data to generate a health profile of the user, the health profile comprising one or more related biological, mental, emotional, and cognitive processes; monitoring, by a longitudinal monitoring system, the health profile over time to identify temporal changes in the one or more related biological, mental, emotional, and cognitive processes; predicting, by one or more predictive models, an evolution of the health profile based on one or more of the identified temporal changes and a longitudinal data record; correlating, by a conditions correlator, the predicted evolution of the health profile to one or more health conditions; and identifying, in response to the correlation to the one or more health conditions, by one or more Metabolites as Medicine (“MaM”) models, one or more bioactive compounds to improve the one or more health conditions, wherein the one or more MaM models analyze metabolic pathways in a human metabolome and predict effects of the one or more bioactive compounds on the metabolic pathways.

2. The method of claim 1, wherein the heterogeneous data further comprises one or more of the image data, biometric data, user input, environmental data, and diagnostic data, wherein the biometric data includes at least one of heart rate data, blood oxygen level data, glucose level data, dietary habit data, physical activity data, and sleep pattern data.

3. The method of claim 1, wherein the one or more related biological processes are related to psychogenic aging, and wherein the tongue and data derived from the tongue are developed as a biomarker of aging used to track a pace of aging.

4. The method of claim 1, wherein the analyzing the image data using one or more models comprises using one or more machine learning models to: locate a tongue within the image data; annotate the image with a detection bounding box; crop the image to a size of the detection bounding box; generate a tongue image having a black background and little to no details outside of the tongue; estimate a pose of the tongue by detecting a tip and root of the tongue and drawing a line therebetween; perform a contour analysis to generate a symmetry score; determine that the symmetry score is above a set threshold; determine a color of the tongue; detect saliva on the tongue; identify a coating on the tongue; detect one or more fissures on the tongue; identify one or more teeth marks on the tongue; segment the tongue into one or more regions to form a segmented tongue image; expand the one or more regions such that no area of the tongue remains unsegmented; calculate coating regions; and determine a thickness of the coating.

5. The method of claim 4, wherein detecting saliva on the tongue comprises: applying morphological operations and separate contour highlights to the segmented tongue image; and calculating a mean value of areas of the tongue not covered in highlights and a mean value of each highlight in contour, wherein a difference between a highlighted part and a remainder of the tongue indicates saliva-covered areas.

6. The method of claim 4, wherein calculating coating regions comprises: calculating a distance map by determining color distances between image pixels and a desired coating color in both LAB and HSV color spaces; clustering distance maps of coatings using K-means clustering; and combining the clustered distance maps using binary thresholding to generate a final coating mask.

7. The method of claim 4, wherein the one or more machine learning models comprise object detection and segmentation models based on You Only Look Once (“YOLO”) architecture, the models being trained on synthetic data comprising images generated by three-dimensional (“3D”) modeling and real image datasets.

8. The method of claim 1, wherein the image data comprises one or more of videos and images captured by one or more of a user device, a multispectral imaging device, and a hyperspectral imaging device, and a single pixel detector device configured to approximate a result of one or more of the multispectral imaging device and the hyperspectral imaging device utilizing predictive models.

9. The method of claim 8, wherein the predictive models comprise: creating a spectral fingerprint of the user, the spectral fingerprint comprising health-related patterns unique to the user; determining biological endotypes and phenotypes; clustering the user and one or more other individuals according to the endotypes and phenotypes; identifying one or more interindividual differences in oral cavity, tongue, and other biometric inputs; observing and measuring temporal health-related changes on an individual-level and a cohortlevel; identifying unique correlations between the image data and the heterogeneous data on the individual-level and cohort-level; and approximating one or more of a blood metabolome, an oral microbiome, mycobiome, and virome and compositions and functions thereof.

10. The method of claim 1, wherein the remaining data from the heterogenous data comprises one or more of biometric data, user input, and environmental data.

11. The method of claim 1, wherein the one or more health conditions comprise at least one of gastric cancer, breast cancer, and inflammatory bowel disease (IBD), and wherein the correlating comprises: monitoring multiple tongue conditions over a predetermined time period; confirming a condition as positive when the condition is detected over several consecutive days; maintaining confirmed positive conditions as active for a predetermined number of days from last detection; andflagging a potential health condition when a threshold number of unique conditions are confirmed positive within the predetermined time period.

12. A system comprising: a processor operatively coupled to a memory storing computer-readable instructions that, when executed by the processor, cause the processor to: receive, by an application programming interface (“API”) server, heterogeneous data associated with a user from one or more devices over a network, wherein the heterogeneous data comprises image data of a tongue of the user and biometric metadata; route, by a queue manager, the image data from the heterogeneous data to an image processor and remaining data from the heterogeneous data to a data synthesizer; analyze, by the image processor, the image data using one or more machine learning models to detect and classify one or more features of the tongue; integrate, by the data synthesizer, the one or more features and associated classifications with the remaining data to generate a health profile of the user, the health profile comprising one or more related biological, mental, emotional, and cognitive processes; monitor, by a longitudinal monitoring system, the health profile over time to identify temporal changes in the one or more related biological, mental, emotional, and cognitive processes; predict, by one or more predictive models, an evolution of the health profile based on the identified temporal changes; correlate, by a conditions correlator, the evolution of the health profile to one or more health conditions; and identify, by one or more Metabolites as Medicine (“MaM”) models, a ranked list of one or more bioactive compounds to improve the one or more health conditions, wherein the one or more MaM models analyze metabolic pathways in a human metabolome and predict effects of the one or more bioactive compounds on the metabolic pathways.

13. The system of claim 12, wherein the heterogeneous data further comprises one or more of biometric data, user input, environmental data, and diagnostic data, wherein the biometric data includes at least one of heart rate data, blood oxygen level data, glucose level data, dietary habit data, physical activity data, and sleep pattern data.

14. The system of claim 12, wherein the one or more related biological processes are related to psychogenic aging, and wherein the tongue and data derived from the tongue are developed as a biomarker of aging used to track a pace of aging.

15. The system of claim 12, wherein analyzing the image data using one or more machine learning models comprises: locating the tongue within the image data; annotating the image data with a detection bounding box; cropping the image data to a size of the detection bounding box; generating a segmented tongue image having a black background and little to no details outside of the tongue; estimating a pose of the tongue by detecting a tip and root of the tongue and drawing a line therebetween; performing a contour analysis to generate a symmetry score; determining that the symmetry score is above a set threshold; determining a color of the tongue; detecting saliva on the tongue; identifying a coating on the tongue; detecting one or more fissures on the tongue; identifying one or more teeth marks on the tongue; sub-segmenting the tongue into one or more regions; expanding the one or more regions such that no area of the tongue remains unsegmented; calculating coating regions; and determining a thickness of the coating.

16. The system of claim 15, wherein detecting saliva on the tongue comprises: applying morphological operations and separate contour highlights to the segmented tongue image; and calculating a mean value of areas of the tongue not covered in highlights and a mean value of each highlight in contour, wherein a difference between a highlighted part and a rest of the tongue indicates saliva-covered areas.

17. The system of claim 15, wherein calculating coating regions comprises: calculating a distance map by determining color distances between image pixels and a desired coating color in both LAB and HSV color spaces; clustering distance maps of coatings using K-means clustering; and combining the clustered distance maps using binary thresholding to generate a final coating mask.

18. The system of claim 15, wherein the one or more machine learning models comprise object detection and segmentation models based on You Only Look Once (“YOLO”) architecture, the models being trained on synthetic data comprising images generated by three-dimensional (“3D”) modeling and real image datasets.

19. The system of claim 12, wherein the one or more MaM models comprise one or more graph neural network (“GNN”) models having: a first component comprising a graph representing a metabolite or compound of interest, wherein nodes of the graph represent multiple chemical features of each compound atom and edges represent connections between the atoms in the compound; and a second component comprising another graph, wherein nodes represent biological pathways or groups of pathways of interest related to aging processes, and wherein an edge between two nodes indicates a biological co-occurrence between the node pathways.

20. The system of claim 12, wherein the one or more health conditions comprise at least one of gastric cancer, breast cancer, and inflammatory bowel disease (IBD), and wherein the correlating comprises: monitoring multiple tongue conditions over a predetermined time period; determining biological endotypes and phenotypes; confirming a condition as positive when the condition is detected on at least two consecutive days; maintaining confirmed positive conditions as active for a predetermined number of days from last detection; and flagging a potential health condition when a threshold number of unique conditions are confirmed positive within the predetermined time period.

21. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to: receive heterogeneous data associated with a user from one or more devices over a network, wherein the heterogeneous data comprises image data of a tongue of the user; identify one or more interindividual differences in oral cavity, tongue, and other biometric inputs; analyze the image data using one or more machine learning models to detect and classify one or more features of the tongue;observe and measure temporal health-related changes on an individual-level and a cohort-level; identify unique correlations between the image data and the heterogenous data on the individuallevel and cohort-level; and integrate the one or more features and associated classifications with remaining data from the heterogeneous data to generate a health profile of the user, the health profile comprising one or more related biological, mental, emotional, and cognitive processes; monitor the health profile over time to identify temporal changes in the one or more related biological, mental, emotional, and cognitive processes; predict an evolution of the health profile based on the identified temporal changes; correlate the evolution of the health profile to one or more health conditions; and approximating one or more of a blood metabolome, an oral microbiome, mycobiome, and virome and compositions and functions thereof, and identify, using Metabolites as Medicine (“MaM”) models, one or more bioactive compounds to improve the one or more health conditions, wherein the one or more MaM models analyze metabolic pathways in a human metabolome and predict effects of one or more bioactive compounds on the metabolic pathways.

22. The non-transitory computer-readable storage medium of claim 21, wherein the one or more health conditions comprise at least one of gastric cancer, breast cancer, and inflammatory bowel disease (IBD), and wherein the correlating comprises: monitoring multiple tongue conditions over a predetermined time period; confirming a condition as positive when the condition is detected on at least two consecutive days; maintaining confirmed positive conditions as active for a predetermined number of days from last detection; and flagging a potential health condition when a threshold number of unique conditions are confirmed positive within the predetermined time period.

23. The method of claim 1, wherein the one or more MaM models comprise one or more graph neural network (“GNN”) models having: a first component comprising a graph representing a metabolite or compound of interest, wherein nodes of the graph represent multiple chemical features of each compound atom and edges represent connections between the atoms in the compound; anda second component comprising another graph, wherein nodes represent biological pathways or groups of pathways of interest related to aging processes, and wherein an edge between two nodes indicates a biological co-occurrence between the node pathways.

24. The system of claim 12, wherein the remaining data from the heterogenous data comprises one or more of biometric data, user input, and environmental data.

25. The method of claim 1, further comprising: generating synthetic training datasets using an Informational Twin system, wherein the Informational Twin system utilizes synthetic-image generation, including but not limited to 3-D rendering, diffusion-based generative models, or other programmatic techniques and procedural generation algorithms to create a number of clinically useful synthetic tongue images with automated ground truth annotations; training the one or more machine learning models using the synthetic training datasets; and validating the one or more machine learning models using real-world clinical datasets.

26. The method of claim 25, wherein the number of clinically useful synthetic tongue images comprise precise control over pathological features, lighting conditions, and tongue positions.

27. The method of claim 1, wherein correlating the evolution of the health profde to one or more health conditions comprises: processing symptom inputs through a Medical Mapping Network (MMNet) to generate comprehensive condition profdes; utilizing a transformer-based architecture with specialized medical language embeddings in the MMNet; and employing a soft label proportional signal approach in the MMNet, wherein values between 0 and 1 are assigned based on the proportion of observed symptoms associated with specific pathophysiological states.

28. The method of claim 27, wherein the MMNet achieves NDCG scores exceeding a predetermined NDCG score and cosine similarity exceeding a predetermined cosine similarity in translating complex symptom presentations into targeted molecular interventions.

29. The method of claim 1, wherein the one or more MaM models comprise:a compound graph where nodes represent atomic features including chemical symbol, degree, implicit valence, bonded hydrogens, and aromaticity, and edges represent atomic bonds; and a pathway graph where nodes represent biological pathways and edges represent co-occurrence relationships.

30. The method of claim 29, wherein the one or more MaM models process molecular structures at an atomic level and map them to specific metabolic pathways, achieving pathway classification accuracy exceeding a percentage through cross-validation testing.

31. The method of claim 1 , wherein the one or more MaM models utilize a dataset of compounds across a number of pathway categories.

32. The system of claim 12, further comprising an image preprocessing pipeline that implements: advanced color space transformations in LAB and HSV color spaces; bilateral filtering with edge preservation; and automated quality assessment algorithms that achieve consistent diagnostic accuracy across diverse lighting conditions and camera specifications.

33. A system comprising: a specialized imaging device configured to capture high-resolution tongue images; a processor operatively coupled to a memory storing computer-readable instructions that, when executed by the processor, cause the processor to: receive tongue image data from the specialized imaging device; preprocess the tongue image data using an image preprocessing pipeline that implements: advanced color space transformations in LAB and HSV color spaces; bilateral filtering with edge preservation; and automated quality assessment algorithms; analyze the preprocessed tongue image data using one or more machine learning models trained on a combination of synthetic 3D-generated tongue images and real clinical tongue images to detect and classify one or more features of the tongue; generate a health profile of a user based on the detected and classified features; input the health profile into one or more graph neural network (GNN) models comprising: a compound graph where nodes represent atomic features of metabolites or compounds and edges represent atomic bonds; anda pathway graph where nodes represent biological pathways and edges represent co-occurrence relationships; and identify, using the one or more GNN models, one or more bioactive compounds predicted to modulate one or more biological pathways associated with the health profile.

34. The system of claim 33, wherein the specialized imaging device comprises one or more of a multispectral imaging device, a hyperspectral imaging device, and a thermal imaging device.

35. The system of claim 33, further comprising instructions that cause the processor to: generate synthetic training datasets using an Informational Twin system that utilizes syntheticimage generation, including but not limited to 3-D rendering, diffusion-based generative models, or other programmatic techniques and procedural generation algorithms to create a number of clinically useful synthetic tongue images with automated ground truth annotations, wherein the one or more machine learning models are initially trained using the synthetic training datasets and subsequently finetuned using real clinical tongue images.

36. The system of claim 35, wherein the number of clinically useful synthetic tongue images comprise precise control over pathological features, lighting conditions, and tongue positions.

37. The system of claim 33, wherein the one or more GNN models process molecular structures at an atomic level and map them to specific metabolic pathways, achieving pathway classification accuracy exceeding a percentage through cross-validation testing, and comprise a dual-graph architecture including: a first graph representing compounds at the atomic level, wherein nodes represent chemical features of each atom and edges represent atomic bonds; and a second graph representing biological pathways, wherein nodes represent pathways and edges represent biological co-occurrence relationships, wherein the dual-graph architecture processes each compound and pathway into mutually comparable vector representations and associates compounds with pathways based on cosine similarity between the vector representations.

38. The system of claim 33, wherein the one or more GNN models utilize a dataset comprising at least 6,000 compounds across at least 100 pathway categories, and up to several hundred thousand of compounds across at least 800 pathway categories.

39. The system of claim 33, further comprising instructions that cause the processor to: monitor the health profile over time to identify temporal changes; predict an evolution of the health profile based on the identified temporal changes; and update the identified bioactive compounds based on the predicted evolution of the health profile.

40. The system of claim 33, further comprising instructions that cause the processor to: generate a personalized health recommendation comprising a specific formulation of the identified bioactive compounds optimized for the user’s health profile based on pathway-level analysis of predicted molecular interactions; and transmit the personalized health recommendation to a user device for display to the user via an encrypted data channel with real-time synchronization capabilities.

41. A system comprising: a data intake module configured to receive heterogeneous data associated with a user from multiple devices over a network, wherein the heterogeneous data comprises tongue image data and biometric metadata; an image processor configured to analyze the tongue image data using machine learning models to generate tongue feature vectors representing detected and classified tongue characteristics; a synthesizer configured to fuse the tongue feature vectors with the biometric metadata to generate a comprehensive health profile of the user; a longitudinal monitoring system configured to track temporal changes in the health profile over time; a predictive engine configured to forecast an evolution of the health profile based on the tracked temporal changes; conditions correlator configured to map the forecasted health profile evolution to one or more specific health conditions; and a Metabolites as Medicine (“MaM”) engine configured to output a ranked list of bioactive compounds predicted to improve the one or more specific health conditions based on analysis of metabolic pathway interactions.

42. The method of claim 1, wherein correlating the evolution of the health profile to one or more health conditions comprises: a) selecting a predetermined set of N features of the tongue as biomarkers for a specific health condition; b) monitoring a user’s image data over a time period to detect occurrences of the N biomarkers;c) confirming a biomarker as positive only after it has been detected for a predetermined number of consecutive days; and d) identifying a potential risk for the specific health condition when a threshold of M out of N biomarkers are confirmed positive within the time period.

43. The method of claim 42, wherein a Medical Mapping Network (MMNet) supports analysis of a number of different health conditions.

44. The system of claim 12, further comprising instructions that cause the processor to: generate synthetic training datasets using an Informational Twin system, wherein the Informational Twin system utilizes 3D modeling techniques and procedural generation algorithms to create clinically useful synthetic tongue images with automated ground truth annotations; train the one or more machine learning models using the synthetic training datasets; and validate the one or more machine learning models using real -world clinical datasets.

45. The system of claim 44, wherein the synthetic tongue images comprise precise control over pathological features, lighting conditions, and tongue positions.

46. The system of claim 12, wherein correlating the evolution of the health profile to one or more health conditions comprises: processing symptom inputs through a Medical Mapping Network (MMNet) to generate comprehensive condition profiles; utilizing a transformer-based architecture with specialized medical language embeddings in the MMNet; and employing a soft label proportional signal approach in the MMNet, wherein values between 0 and 1 are assigned based on the proportion of observed symptoms associated with specific pathophysiological states.

47. The system of claim 46, wherein the MMNet achieves NDCG scores exceeding a predetermined NDCG score and cosine similarity exceeding a predetermined cosine similarity in translating complex symptom presentations into targeted molecular interventions.

48. The system of claim 12, wherein the one or more MaM models comprise: a compound graph where nodes represent atomic features including chemical symbol, degree, implicit valence, bonded hydrogens, and aromaticity, and edges represent atomic bonds; anda pathway graph where nodes represent biological pathways and edges represent co-occurrence relationships.

49. The system of claim 48, wherein the one or more MaM models process molecular structures at an atomic level and map them to specific metabolic pathways, achieving pathway classification accuracy exceeding a percentage through cross-validation testing.

50. The system of claim 12, wherein the one or more MaM models utilize a dataset of tens of thousands of compounds across hundreds of pathway categories.

51. The system of claim 12, wherein the correlating the evolution of the health profile to one or more health conditions comprises: a) selecting a predetermined set of N features of the tongue as biomarkers for a specific health condition; b) monitoring a user’s image data over a time period to detect occurrences of the N biomarkers; c) confirming a biomarker as positive only after it has been detected for a predetermined number of consecutive days; and d) identifying a potential risk for the specific health condition when a threshold of M out of N biomarkers are confirmed positive within the time period.

52. The system of claim 51, wherein a Medical Mapping Network (MMNet) supports analysis of over a number of different health conditions.

53. The method of claim 1 , wherein the one or more MaM models are optimized for performance by pre -calculating and caching pathway prediction scores for each bioactive compound within a library of potential recommendations, thereby enabling real-time retrieval of a ranked list of compounds in response to a query without executing the one or more MaM models on-the-fly.

54. The method of claim 53, wherein the ranked list of bioactive compounds is enriched with drug-likeness data comprising predictions for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) properties for each recommended compound.

55. The system of claim 12, wherein the one or more MaM models are optimized for performance by pre -calculating and caching pathway prediction scores for each bioactive compoundwithin a library of potential recommendations, thereby enabling real-time retrieval of the ranked list of compounds in response to a query without executing the one or more MaM models on-the-fly.

56. The system of claim 55, wherein the ranked list of bioactive compounds is enriched with drug-likeness data comprising predictions for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) properties for each recommended compound by integrating predictions from computational toxicology models.

57. The non-transitory computer-readable storage medium of claim 21, wherein the one or more MaM models are optimized for performance by pre-calculating and caching pathway prediction scores for each bioactive compound within a library of potential recommendations, thereby enabling real-time retrieval of a ranked list of compounds in response to a query without executing the one or more MaM models on-the-fly.

58. The system of claim 33, further comprising instructions that cause the processor to: pre-calculate and cache pathway prediction scores for each bioactive compound within a library of potential recommendations to solve computational latency inherent in the one or more GNN models; and enrich the identified bioactive compounds with drug-likeness data comprising predictions for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) properties.

59. The system of claim 41, wherein the MaM engine is optimized for performance by precalculating and caching pathway prediction scores for each bioactive compound within a library of potential recommendations, thereby enabling real-time retrieval of the ranked list of bioactive compounds without executing complex graph neural network models on-the-fly.

60. The system of claim 59, wherein the ranked list of bioactive compounds is enriched with drug-likeness data comprising predictions for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) properties for each recommended compound.

61. The method of claim 25, wherein the synthetic dataset improves model performance relative to a model trained on real -image data alone.

62. The system of claim 35, wherein the synthetic dataset improves model performance relative to a system trained on real -image data alone.

63. A non-transitory computer-readable medium storing instructions which, when executed by a computer device comprising an image sensor and at least one processor, cause the computer device to: prompt a user, at a cadence of at least once in each 24-hour period, to capture at least one image of the user’s tongue; acquire the at least one image with the image sensor; automatically evaluate one or more image-quality parameters of the at least one image, the one or more image-quality parameters comprising color balance, illumination uniformity, tongue pose relative to a reference plane, and left / right morphological symmetry; accept the at least one image based on the one or more image-quality parameters satisfying a respective threshold criteria; generate a data package comprising the accepted image, a timestamp identifying a time of capture, and device metadata comprising at least a device identifier; and transmit the data package to a remote server configured to perform any one of claims 1 to 62.

64. An electronic device comprising: an image sensor; at least one processor; and a memory storing instructions that, when executed by the processor, cause the electronic device to: capture, at least once during each 24-hour period, at least one image of a user’s tongue with the image sensor, evaluate one or more image-quality parameters of the at least one image, the one or more image-quality parameters comprising color balance, tongue pose, and symmetry, accept the at least one image based on the one or more image-quality parameters satisfying a respective threshold criteria, encode the at least one image with a timestamp and device metadata, and transmit the encoded at least one image to an external analysis server configured to perform any one of claims 1 to 62.

65. The method of claim 1, wherein the one or more machine learning models are configured to detect at least one of a tongue boundary, a tongue coating, or a tongue fissure with an accuracy of at least 95%.

66. The method of claim 1, wherein the one or more machine learning models are trained on a plurality of tongue images, and wherein the plurality comprises at least 1000 tongue images.

67. The method of claim 1, wherein the one or more machine learning models achieve a predetermined accuracy of at least 80% in detecting and classifying the one or more features of the tongue.

68. The method of claim 1, wherein accuracy of the one or more machine learning models is computed on a held-out test set with an 80 / 20 train / test split.

Images