Device, system, and method for multi-modal physiological sensing with automated medical record generation

Abstract

There is provided a system, method and device for physiological sensing for automated medical record generation. The method including: receiving a bioacoustics signal from a physiological sensing device, the physiological sensing device receiving and subtracting contemporaneous signals from a primary microphone and a secondary microphone to generate the bioacoustics signal, the primary microphone measuring sound through a membrane that is in contact with a subject, the secondary microphone measuring sound at a separate location than the membrane; using a trained machine learning classifier to determine health-related medical data from the bioacoustics signal, the machine learning classifier trained on a dataset of labeled bioacoustics signals; generating a medical report with a large language machine learning model taking the health-related medical data as input; and outputting the medical report.

Description

DEVICE, SYSTEM, AND METHOD FOR MULTI-MODAL PHYSIOLOGICAL SENSING WITH AUTOMATED MEDICAL RECORD GENERATIONTECHNICAL FIELD

[0001] The following relates, generally, to devices for physiological sensing and health-data processing; and more particularly, to a device, system, and method for non-invasive physiological sensing and a method and system for automated patient record-keeping.BACKGROUND

[0002] Various types of approaches can be used to assess physiological aspects of a person or animal. For example, the use of auscultation, which is the act of listening to the sounds made by internal organs. Traditional auscultation approaches for assessing physiological aspects of humans and animals rely heavily on manual techniques, which are prone to errors due to ambient noise, motion artifacts and subjective interpretation. Such limitations are often amplified in a veterinary setting where the animal under observation is unlikely to remain calm and still for accurate auscultation observations.

[0003] Healthcare professionals traditionally rely on separate tools and manual processes to assess patients and document findings. For example, auscultation with a stethoscope is typically used for physical exams, but conventional acoustic stethoscopes simply transmit sound to the clinician’s ears via a resonator and hollow tubes. Clinicians must interpret sounds (heart beats, lung sounds, etc.) in real time and later manually enter notes into electronic medical records (EM Rs). This manual documentation is time-consuming and prone to subjectivity.

[0004] Additionally, various types of approaches can be used to enhance patient care and operational efficiency of healthcare. Patient record intake and record keeping is one of the pivotal components of every healthcare system. Traditional patient record intake and record-keeping approaches are often manual, time-consuming, and prone to human error, impacting clinical efficiency and accuracy.SUMMARY

[0005] In an aspect, there is provided a system for automated medical record generation, the system comprising one or more processors in communication with a data storage, the one or more processors configured to execute: a communications module to receive a bioacousticssignal from a physiological sensing device, the physiological sensing device receiving and subtracting contemporaneous signals from a primary microphone and a secondary microphone to generate the bioacoustics signal, the primary microphone measuring sound through a membrane that is in contact with a subject, the secondary microphone measuring sound at a separate location than the membrane; a machine learning module to use a trained machine learning classifier to determine health-related medical data from the bioacoustics signal, the machine learning classifier trained on a dataset of labeled bioacoustics signals; a document generation module to generate a medical report with a large language machine learning model taking the health-related medical data as input; and an output module to output the medical report.

[0006] In a particular case of the system, the health-related medical data comprises quantitative measures or categorical results.

[0007] In another case of the system, input to the large language model comprises a time of capture of the bioacoustics signal and patient context.

[0008] In yet another case of the system, input to the large language model further comprises previously captured bioacoustics signals.

[0009] In yet another case of the system, the bioacoustics signal comprises a pulse rate, a respiratory rate, or both.

[0010] In yet another case of the system, the communications module further receives orientation and movement signals from an inertial measurement unit, the bioacoustics signal weighted based on the received orientation and movement signals.

[0011] In yet another case of the system, the communications module further receives a temperature reading from a non-contact far-infrared temperature sensor associated with the physiological sensing device, the temperature reading provided as further input to the trained machine learning classifier.

[0012] In yet another case of the system, the communications module further receives captured voice data, and the machine learning module determines a text-to-speech transcription using the captured voice data, the transcription forming a part of the health-related medical data.

[0013] In yet another case of the system, the membrane conforms to a body surface to form an acoustic seal, an acoustic aperture behind the membrane defines a chamber leading to the primary microphone to attenuate external noise.

[0014] In yet another case of the system, the medical report is formatted as a structured medical report or a note in a subjective, objective, assessment, and plan format.

[0015] In another aspect, there is provided a computer-implemented method for automated medical record generation, the method comprising: receiving a bioacoustics signal from a physiological sensing device, the physiological sensing device receiving and subtracting contemporaneous signals from a primary microphone and a secondary microphone to generate the bioacoustics signal, the primary microphone measuring sound through a membrane that is in contact with a subject, the secondary microphone measuring sound at a separate location than the membrane; using a trained machine learning classifier to determine health-related medical data from the bioacoustics signal, the machine learning classifier trained on a dataset of labeled bioacoustics signals; generating a medical report with a large language machine learning model taking the health-related medical data as input; and outputting the medical report.

[0016] In a particular case of the method, the health-related medical data comprises quantitative measures or categorical results.

[0017] In another case of the method, input to the large language model comprises a time of capture of the bioacoustics signal and patient context.

[0018] In yet another case of the method, input to the large language model further comprises previously captured bioacoustics signals.

[0019] In yet another case of the method, the bioacoustics signal comprises a pulse rate, a respiratory rate, or both.

[0020] In yet another case of the method, the method further comprising receiving orientation and movement signals from an inertial measurement unit, the bioacoustics signal weighted based on the received orientation and movement signals.

[0021] In yet another case of the method, the method further comprising receiving a temperature reading from a non-contact far-infrared temperature sensor associated with the physiologicalsensing device, the temperature reading provided as further input to the trained machine learning classifier.

[0022] In yet another case of the method, the method further comprising receiving captured voice data and determining a text-to-speech transcription using the captured voice data, the transcription forming a part of the health-related medical data.

[0023] In yet another case of the method, the membrane conforms to a body surface to form an acoustic seal, an acoustic aperture behind the membrane defines a chamber leading to the primary microphone to attenuate external noise.

[0024] In yet another case of the method, the medical report is formatted as a structured medical report or a note in a subjective, objective, assessment, and plan format.

[0025] These and other aspects are contemplated and described herein. It will be appreciated that the foregoing summary sets out representative aspects of the system, method, and device to assist skilled readers in understanding the following detailed description.DESCRIPTION OF THE DRAWINGS

[0026] A greater understanding of the embodiments will be had with reference to the Figures, in which:

[0027] FIG. 1 is an exploded perspective view of a physiological sensing device 100, in accordance with an embodiment;

[0028] FIG. 2 is a picture of the physiological sensing device of FIG. 1 in a hand-mounted orientation;

[0029] FIG. 3 is a picture of the physiological sensing device of FIG. 1 showing a strap attachment;

[0030] FIG. 4A is a front-view line-drawing, and FIG. 4B is a side-view line-drawing, of the physiological sensing device of FIG. 1 in its assembled form;

[0031] FIG. 5 is a diagram of a system for non-invasive physiological sensing, in accordance with an embodiment;

[0032] FIG. 6 is a flowchart of a method for non-invasive physiological sensing, in accordance with an embodiment;

[0033] FIG. 7 is a diagram of an example implementation of the system of FIG. 5;

[0034] FIG. 8 is a flowchart illustrating a method for determining heart rate (HR) and respiratory rate (RR) using stethoscope analysis, in accordance with an example embodiment;

[0035] FIG. 9 is a flowchart illustrating a method for determining heart rate (HR) and respiratory rate (RR) using IMU-based analysis, across each IMU channel, in accordance with an example embodiment;

[0036] FIG. 10 is a graph of an example experiment showing inertial measurement unit (IMU) data to illustrate heart rate and respiratory rate;

[0037] FIG. 11A shows measurements for a primary microphone and a secondary microphone in the example experiments of FIG. 10, where the top subplot is a measurement from the primary microphone, the second subplot showing a Fast Fourier Transform (FFT) of the primary microphone measurement, the third subplot is a measurement from the secondary microphone, and the fourth subplot shows a FFT of the secondary microphone measurement;

[0038] FIG. 11 B shows gyroscope measurements in each direction in the example experiments of FIG. 10;

[0039] FIG. 11C shows accelerometer measurements in each direction in the example experiments of FIG. 10;

[0040] FIG. 12 shows spectrogram outputs in the example experiments of FIG. 10, where the top output graph shows an example spectrogram from the primary microphone, the middle output graph shows an example spectrogram from the secondary microphone, and the bottom output graph shows the subtracted output;

[0041] FIG. 13 illustrates an example user interface showing data outputted in the example experiments of FIG. 10;

[0042] FIGS. 14A to 14C illustrates three graphs, for the example experiments of FIG. 10, showing accelerometer data in the Z-direction in FIG. 14A, determined confidence measure in FIG. 14B, and calculated heart rate in FIG. 14C;

[0043] FIG. 15 is a chart showing measurements versus measurement confidence for the example experiments of FIG. 10;

[0044] FIG. 16A illustrates a stethoscope signal for the example experiments of FIG. 10;

[0045] FIG. 16B illustrates a gyroscope signal in the X direction for the example experiments of FIG. 10;

[0046] FIG. 16C illustrates a calculated confidence for the extracted heart rate (HR) determination for the example experiments of FIG. 10;

[0047] FIG. 16D illustrates the determination of the HR signal for the example experiments of FIG. 10;

[0048] FIG. 17 is a diagram illustrating extensible modules over universal-serial-bus for the device of FIG. 1;

[0049] FIG. 18 shows a general diagram for a system for automated medical record generation, in accordance with an example embodiment;

[0050] FIG. 19 shows a high-level functional diagram of the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0051] FIG. 20 shows a high-level diagram of data transmission pathway in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0052] FIG. 21 shows four screenshots of graphical user interfaces in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0053] FIG. 22 shows another four screenshots of graphical user interfaces in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0054] FIG. 23 shows yet another screenshot of a graphical user interface in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0055] FIG. 24 shows yet another screenshot of a graphical user interface in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0056] FIG. 25 shows a visual flowchart of an example data processing flow in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0057] FIG. 26 shows a flowchart diagram illustrating an example feedback loop mechanism for a large language model used in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0058] FIG. 27 shows a visual flowchart of an example data processing flow in the automated medical record system of FIG. 18, in accordance with an example embodiment;

[0059] FIG. 28 shows an example of a stethoscope-attached device capable of recording clinical conversations, along with an example of recorded audio, and the generated medical record;

[0060] FIG. 29 shows multiple contexts for the recording of data and data examples, including recorded conversations when used as a stethoscope attachment, body sounds when used as an independent stethoscope, and cardioseismography data when used as a wearable device;

[0061] FIG. 30 is a flowchart for a computer-implemented method for automated medical record generation, in accordance with an embodiment;

[0062] FIGS. 31 A and 31 B show an average heartbeat plot, segmented signal plot, and spectrogram outputted by the system of FIG. 18 as part of an example medical report for a patient with a murmur, where the average S2 sound is abnormal; and

[0063] FIGS. 32A and 32B show an average heartbeat plot, segmented signal plot, and spectrogram outputted by the system of FIG. 18 as part of another example medical report, for a healthy patient, where the average S2 sound is normal.DETAILED DESCRIPTION

[0064] For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the Figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practised without these specific details. In other instances, well-known methods, procedures and components have not been described in detailso as not to obscure the embodiments described herein. Also, the description is not to be considered as limiting the scope of the embodiments described herein.

[0065] Various terms used throughout the present description may be read and understood as follows, unless the context indicates otherwise: “or” as used throughout is inclusive, as though written “and / or”; singular articles and pronouns as used throughout include their plural forms, and vice versa; similarly, gendered pronouns include their counterpart pronouns so that pronouns should not be understood as limiting anything described herein to use, implementation, performance, etc. by a single gender. Further definitions for terms may be set out herein; these may apply to prior and subsequent instances of those terms, as will be understood from a reading of the present description.

[0066] Any module, unit, component, server, computer, terminal or device exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and / or nonremovable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the device or accessible or connectable thereto. Further, unless the context clearly indicates otherwise, any processor or controller set out herein may be implemented as a singular processor or as a plurality of processors. The plurality of processors may be arrayed or distributed, and any processing function referred to herein may be carried out by one or by a plurality of processors, even though a single processor may be exemplified. Any method, application or module herein described may be implemented using computer readable / executable instructions that may be stored or otherwise held by such computer readable media and executed by the one or more processors.

[0067] Embodiments of the present invention provide a physiological sensing device, system, and method for physiological sensing; and in a particular case, physiological sensing for veterinary use. In a particular case, a sensory device is capable of being strapped or otherwisemounted in front of a palm of the medical practitioner or can be worn by the patient; such as worn by an animal as a collar. Embodiments of the present invention can advantageously include one or more of noise cancellation, body-conduction microphone(s), an acoustically-transparent membrane, and remote analysis using artificial intelligence.

[0068] Electronic stethoscopes can amplify internal body sounds and sometimes record audio for later analysis. However, many electronic stethoscopes use a single diaphragm-mounted microphone in the chest piece and generally suffers significant interference from ambient noise. Ambient room noise or movement can obscure important physiological sounds, making it difficult to diagnose conditions accurately. To address this, some stethoscopes have employed multiple microphones and noise-cancellation.

[0069] In addition to audio, in some cases, motion sensors such as inertial measurement units (IM Us), have been used for physiological monitoring. For example, to track the movement of the stethoscope or the patient’s chest, which can help distinguish between sounds caused by patient movement and true physiological signals. For example, an IMU attached to a stethoscope can detect respiratory motions or posture changes, allowing correlation of breath sounds with chest movements. Temperature sensors have also been used separately to measure patient body temperature during exams. However, these approaches only address one aspect of the examination at a time.

[0070] Generally, clinicians either rely on memory or handwritten notes to later update the EMR, or use voice dictation systems which still require them to verbalize and review documentation. In veterinary practice, similar challenges exist as veterinarians manually auscultate animal patients and must transcribe their findings into records, often without the benefit of specialized devices or assistants.

[0071] Advantageously, embodiments of the present invention can not only use multi-modal physiological data but also interpret such data to, for example, generate a structured medical report (such as a SOAP note) automatically. The device and system of the present emboidments can streamline the work of the clinician by capturing high-fidelity physiological signals in a noisy environment to producing accurate, real-time documentation; thereby improving efficiency, consistency, and diagnostic insight in both human and animal healthcare.

[0072] Turning to FIG. 1, a physiological sensing device 100 is shown, in accordance with an embodiment. The physiological sensing device 100 includes a first enclosure portion 102 that canbe attached to a second enclosure portion 112 to form a cavity therebetween. Such attachment can include screws, heat molding, or any other suitable approach. The first enclosure portion 102 includes an elevated bridge 114 that defines slots to receive a strap 120 therethrough, as illustrated in FIG. 3. For example, the slots permit threading of a 15mm wide strap 120 from left to right, or top to bottom. When threaded left to right, the strap can also perform the function of covering a waterproof universal-serial-bus (USB) connector, which provides additional dust proofing. The first enclosure portion 102 and the second enclosure portion 112 can be formed out of any suitable material, for example, from a 3D-printed photopolymer. In an example, the second enclosure portion 112 can be formed from 316L Stainless steel or AISilOMg Aluminum-alloy manufactured using SLS technology.

[0073] When in use as a handheld device, the device 100 reduces anxiety when, for example, palpating the subject; and thus, reduces the “white-coat” effect.

[0074] FIG. 2 is a picture of the physiological sensing device 100 showing a hand-mounted orientation. FIG. 3 is a picture of the physiological sensing device 100 showing the strap 120 attachments. In some cases, the straps can be fastened around the neck of an animal to be worn in the manner of an animal collar with the membrane 116 pressed against the neck of the animal.

[0075] In further cases, the device 100 can be a compact attachment or adapter mountable on, or associated with, a clinician’s existing stethoscope. The device 100 can use an attachment mechanism designed to fit securely around or onto the tube of standard stethoscopes, facilitating rapid installation and removal without interfering with conventional auscultation usage. This compatibility with existing stethoscopes can ensure that clinicians can readily adopt the device 100 without needing to alter existing clinical practices.

[0076] Positioned within the cavity formed by the first enclosure portion 102 and the second enclosure portion 112 is a main circuit board 104 including, at least, a processing unit and a data storage. In a particular example, the main circuit board 104 can include a wireless transceiver, a temperature-controlled battery charging circuit, a processor, a flash memory, a supplementary microphone 134, an inertial measurement unit (IMU) 128, a magnetometer, and a real-time clock. The IMU can include, for example, an accelerometer and a gyroscope The supplementary microphone 134, the accelerometer, and the gyroscope can be used for integrated activity detection. Also positioned within the cavity is a battery 106 to power the main circuit board 104. In an example, the battery 106 can be a 190 mAh Lithium-Polymer battery.

[0077] Also located within the cavity formed by the first enclosure portion 102 and the second enclosure portion 112 is a sensing circuit board 108 that includes one or more physiological sensors and is in communication with the processing circuit board 104. In an example embodiment, the sensing circuit board 108 can include a primary microphone 130, a secondary microphone 132, and a non-contact far-infrared temperature sensor 136. Also located within the cavity is a membrane 110. A protruding portion 116 of the membrane 110 extends through an aperture 118 in the second enclosure portion 112.

[0078] FIGS. 4A and 4B are line-drawing illustrations of the device 100, in its assembled form, showing an example layout of the primary microphone 130, the secondary microphone 132, the non-contact far-infrared temperature sensor 136, and the supplementary microphone 134. In this case, the primary microphone 130 is located underneath the membrane 110 such that there is an air gap between the deformable protruding membrane 110 and the primary microphone 130 located on the sensing circuit board 108 directly beneath the membrane 110. In further cases, the primary microphone can be directly coupled to the membrane 110 such that there is no air gap in between the membrane 110 and the primary microphone 130; for example, where a silicone membrane 110 is directly- cured to a MEMS primary microphone 130.

[0079] Advantageously, use of the acoustically-transparent membrane 110, which is placed in contact with the subject, allows for attenuation of the physiological signal captured by the primary microphone 130 located underneath the membrane 110; especially when noise is removed by comparison to the secondary microphone 132, which is sampling air and not in such contact with the subject.

[0080] In this way, the device 100 can include a contact surface featuring a flexible diaphragm membrane, as the membrane 110, which covers an acoustic aperture. This membrane 110 serves a similar purpose to a stethoscope diaphragm. When the device is placed against a subject’s body (human or animal), internal sounds (such as heartbeats or breath sounds) cause the membrane to vibrate. These vibrations pass through the aperture to the internal primary microphone 130 (auscultation microphone) located behind the membrane, converting the acoustic signals into electrical audio signals. The flexible membrane 110 can be made of a medical-grade material (such as silicone or polyurethane) that conforms to the body’s contours to form a good acoustic seal, ensuring that internal sounds are transmitted efficiently while external noise is physically dampened.

[0081] The secondary microphone 132 is positioned or oriented to pick up ambient environmental sounds. This microphone 132 is not coupled to the membrane, but instead captures background noise in the vicinity (e.g., room noise, conversations, airflow from HVAC systems, etc.). The dualmicrophone arrangement permits active noise cancellation on the bioacoustic signals by comparing the primary and secondary microphone inputs. The two microphones may be matched in sensitivity and frequency response and can be spaced such that the primary microphone mainly receives sounds from the subject’s body and the secondary microphone receives external ambient sounds. The enclosure can have openings or grilles positioned for the secondary microphone 132 to sample ambient sound effectively. By having a separate secondary microphone 132, noise can be algorithmically subtracted or cancelled by determining which sounds are common to both microphones; and in this way, allowing for the isolation of sounds emanating from the body. In this way, the device 100 can be used to extract a clean representation of, for example, heart sounds, lung sounds, and any other acoustic phenomena (e.g., bowel sounds if the device is used on the abdomen) from the noisy environment. After noise cancellation, feature extraction can be performed on the audio; for example, identifying heart sound peaks (S1, S2), calculating heart rate by measuring the intervals between beats, and analyzing sound patterns and timing (such as recognizing the timing of inhalation vs exhalation in the lung sounds).

[0082] Referring now to FIG. 5, a system 200 for non-invasive physiological sensing, in accordance with an embodiment, is shown. As understood by a person skilled in the art, in some cases, some components of the system 200 can be executed on separate hardware implementations. In other cases, some components of the system 200 can be implemented on one or more general purpose processors that may be locally or remotely distributed, or processors / hardware on the image sensor itself. It is understood that one or more of the components of the system 200 can be implemented on processors located on the physiological sensing device 100. In further cases, components of the system 200 can be implemented on processors located remote from the physiological sensing device 100, in which case, the physiological sensing device 100 is in communication with the remote components over a suitable communication channel (for example, Bluetooth™, WiFi, or the like).

[0083] FIG. 5 shows various physical and logical components of an embodiment of the system 200. As shown, the system 200 has a number of physical and logical components, including one or more processors 202, a data storage 204, a communication interface 206, and a local bus 208 enabling the components to communicate each other. The one or more processors 202 caninclude one or more central processing units, one or more graphical processing units, microprocessors, dedicated hardware, logic arrays or other integrated processing circuits. The one or more processors 202 can be the processor located on the main circuit board 104, or can be located externally from the physiological sensing device 100 and communicate with the processor on the main circuit board 104. The data storage 204 can store programs, instructions, and / or an operating system, including computer-executable instructions for implementing the methods described herein, as well as any derivative or related data. The data storage 204 can be the data storage located on the main circuit board 104, or can be located externally from the physiological sensing device 100, as appropriate.

[0084] The device 100 can include any suitable wireless communication module to transmit data externally. This module could support Bluetooth Low Energy (for pairing with a smartphone or tablet application), Wi-Fi (for connecting directly to a hospital network or router), or other wireless protocols (e.g., cellular). In an example use case, the device 100 might pair with a clinician’s mobile device via Bluetooth, and the mobile device can then relay data to a cloud storage over the Internet. Alternatively, the device 100 could send data directly to a cloud server when in range of a wireless network. In some cases, the sensor data can be packaged and sent securely (for example, with encryption) for remote analysis.

[0085] In some cases, the device 100 can be powered by an internal power source, for example, a rechargeable battery to allow portability. The enclosure of the device 100 can be ergonomically shaped for handheld use, resembling a slightly larger stethoscope head or a small puck-like gadget that can be easily placed on the chest or other body parts. In other cases, the device could be formed as suitable to be a wearable; for instance, a patch or strap that holds it against the body (useful for continuous monitoring or for use on animals who might not stay still for a handheld exam). For veterinary use, the device 100 enclosure could be made robust and perhaps have interchangeable diaphragm membranes or attachment accessories to accommodate different species (e.g., a larger membrane for large animals like horses, or a smaller profile for small pets). The versatility of the device hardware allows it to be used in a variety of settings, from a physician’s office or a hospital bedside to a farm or veterinary clinic, without invasive procedures.

[0086] While FIG. 5 illustrates the system 200 implemented on a single computing device, it is understood that the processing, or any of the functions undertaken by the system 200, can be distributed over multiple devices; for example, in a cloud or distributed computing environment.

[0087] The one or more processors 202 can be configured to execute a number of conceptual elements; for example, a sensing element 212, a signal element 214, a pulse element 216, a respiratory element 218, and an output element 220. In further cases, functions of the above can be combined or executed on other elements. In some cases, functions of the above elements can be executed on remote computing devices, such as centralized servers and cloud computing resources communicating over the communication interface 206.

[0088] The communication interface 206 enables sensors, devices, and / or computing devices to transmit data or receive the outputs from the system 200. In some embodiments, the communication interface 206 enables users to view such outputs, via for example, a display or monitor. In some cases, the outputs from the system 200 can also be stored in the data storage 204. In other cases, the outputs from the system 200 can undergo further processing by the system 200 or other computing devices; for example, for determination of physiological information.

[0089] FIG. 6 illustrates a method for non-invasive physiological sensing 600, in accordance with an embodiment.

[0090] At block 602, the sensing element 212, via the communication interface 206, receives signals from the primary microphone 130 and the secondary microphone 132. In this case, the secondary microphone 132 measures noise in the ambient air close to the membrane 110 and the primary microphone 130 measures body sounds transmitted and / or absorbed through the membrane 110 and into the internal chamber located beneath the membrane 110. In this way, the secondary microphone 132 measures noise and the primary microphone 130 measures noise in addition to a desired bioacoustic signal. It is understood that bioacoustic signal, as used herein, includes electronic representations of the sounds recorded by the primary microphone 130 and the secondary microphone 132; which are sounds generated by, for example, the circulatory, respiratory, and / or digestive systems of a patient (whether that be human or animal).

[0091] At block 603, in some cases, the sensing element 212 performs normalizing of the the primary microphone 130 and the secondary microphone 132 with respect to one another. In an example, after manufacturing, each device can be calibrated using an external sound source of a known amplitude and frequency spaced equally between the primary microphone 130 and the secondary microphone 132. A frequency sweep can be performed and a response for both microphones can be measured. Given that the primary microphone 130 is located beneathsilicone, and the secondary microphone 132 is exposed to air, the measured value will be different. A normalization model can be generated for this difference, such that when the output of one microphone is subtracted from the other, the output should be as close to 0 as possible (the common-mode noise is removed). Accordingly, the normalization model can be used to normalize the signals from the primary microphone 130 and the secondary microphone 132 with respect to one another prior to the processing of the signals described herein.

[0092] At block 604, in some cases, the sensing element 212 performs filtering on the received signals from the primary microphone 130 and the secondary microphone 132 to remove signals outside of the range of the desired bioacoustic signal; for example, performing low-pass filtering below 300 Hz when the desired bioacoustic signal is a heartbeat signal. In further cases, other suitable digital signal processing can be performed on the received signals.

[0093] At block 606, the signal element 214 determines the bioacoustic signal in a band-of-interest by subtracting the noise measured in the filtered signal by the secondary microphone 132 from the total filtered signal measured by the primary microphone 130. FIG. 11 illustrates an example determination of the desired bioacoustic signal; where the top graph illustrates a recording from the secondary microphone 132, the middle graph illustrates a recording from the primary microphone 130, and the bottom graph illustrates an output of the desired bioacoustic signal where the noise has been cancelled out due to the above subtraction. In this example, each vertical line represents a heartbeat. Advantageously, the dual microphone arrangement enables effective acoustic differential amplification for advanced noise cancellation due to the secondary microphone.

[0094] The noise cancellation uses the above techniques to isolate and amplify body sounds of interest. The above differential approach is enhanced by the acoustic properties of the membrane 116, which is engineered to transmit the body sounds efficiently while attenuating external noise. In some cases, a calibration can be performed to compensate for the difference in sensitivity.

[0095] In some cases, there may be a slight difference in noise measured by the secondary microphone 132 due to incident signal reflectance from different transmission mediums. In such cases, the sensing element 212, via the communication interface 206, can further receive measurements from the supplementary microphone 134, from which noise can be further measured and subtracted from the signal received from the primary microphone 130. In further cases, the supplementary microphone 134 can be located on another device in the vicinity of thephysiological sensing device 100, for example, located on a smartphone that is in communication with the communication interface 206. This is particularly useful where the ambient noise profile is complex and variable.

[0096] In further cases, an inertial measurement unit (IMU) 230 associated with the system 200 can provide weighting for further noise reduction, such as providing greater weighting during periods when the subject is not substantially moving. The IMU 230 provides real-time data to adjust the noise cancellation parameters dynamically, accounting for the device's orientation and any movement artifacts. This IMU-assisted weighting is particularly useful for maintaining the integrity of the auscultated signals, especially in noisy environments such as a veterinary clinic. In such cases, IMU-assisted weighting can dynamically adjust noise cancellation parameters based on the device's orientation and movement artifacts in order to maintains signal integrity. The weighting can be adjusted in real-time, with greater weight given during periods of minimal subject movement. In some cases, this weighting can be learned using sensor fusion and machine learning.

[0097] The IMU 230, which can include a multi-axis accelerometer and gyroscope, and optionally, a magnetometer. The IMU 230 can be mounted on the device’s 100 circuit board to sense motion of the device itself or vibrations of the subject. During use, the IMU 230 generates motion data that can indicate if the device 100 is moving or shaking, if the patient shifts position, or even subtle chest movements from breathing. For example, the accelerometer data may show a periodic motion corresponding to respiratory cycles or a sudden spike corresponding to a cough or the device being repositioned. This motion information can be used in conjunction with the audio signals to distinguish physiological sounds from artifacts. It is particularly advantageous to have the IMU, in conjunction with the primary and secondary microphones, find periods with minimal motion artifacts as the signal generated from such periods will have substantially less noise. This concept is demonstrated in FIG. 16C where there is reduced confidence in the period where there is elevated gyroscope activity.

[0098] The IMU 230 can be used by the system to determine whether the device 100 was stable or moving at various times and label portions of the acoustic data as potentially corrupted by movement. It can detect events such as a cough or a shift in device position (since those produce distinctive motion signatures) and either discard or down-weight the corresponding audio segments to avoid false interpretation. Additionally, for respiratory analysis, the motion data (especially from an accelerometer) can indicate the timing of chest wall movement(inhale / exhale), which helps confirm and supplement the breathing rate and pattern detected in the audio.

[0099] At block 608, in some cases, the sensing element 212 receives a temperature signal from the non-contact far-infrared (FIR) temperature sensor 136 to measure the subject’s body temperature without requiring direct contact. Non-contact temperature sensors enhance ease of use and reduce stress on the subject during the measurement process, especially where the subject is an animal. The FIR temperature sensor 136 captures the thermal radiation emitted by the subject’s body, allowing for an accurate temperature reading for diagnosing, for example, fevers and infections.

[0100] The non-contact far-infrared (IR) temperature sensor 136 can be positioned to measure the patient’s surface temperature without requiring direct contact beyond placing the device on the skin. For instance, the IR sensor 136 may be situated near the center of the device’s contact surface or in a rim around the membrane, with a clear line-of-sight to the skin through the aperture or a small window. It detects infrared radiation emanating from the body’s surface and calculates an approximate body temperature. Using the IR sensor 136 allows quick temperature readings concurrently with auscultation, eliminating the need for a separate thermometer. In a veterinary scenario, this is particularly useful as it can measure an animal’s temperature from the skin (for example, inside the ear or on the abdomen) without causing the stress of a rectal thermometer.

[0101] The temperature reading by the temperature sensor 136 can be calibrated to account for ambient conditions in order to reflect core body temperature or skin temperature, as needed. The temperature data can be packaged as part of the overall analysis with the other sensed data.

[0102] At block 610, in some cases, the pulse element 216 determines a pulse rate using the digital bioacoustic signal determined by the signal element 214, where such bioacoustic signal includes isolated heart sounds. In some cases, the pulse element 216 can use a combination of the determined digital bioacoustic signal and ballistocardiography to determine the pulse. The I MU 128 detects subtle mechanical vibrations of the body, which are associated with the heartbeat, known as ballistocardiographic signals. The pulse element 216 correlates the ballistocardiographic signals with the digital bioacoustic signal to arrive at a reliable and precise pulse measurement.

[0103] In some cases, the IMU can detect mechanical vibrations (ballistocardiographic signals), which, when correlated with digital bioacoustic signals (heart sounds), provide a reliable pulsemeasurement. For example, if the confidence measure for both I MU and stethoscope-based metrics are high, an average can be taken. If one signal is much lower than the other, then the weighting can be higher for the sensor with the stronger signal. In some cases, this weighted integration can be learned using sensor fusion and machine learning.

[0104] At block 612, in some cases, the respiratory element 218 determines a respiratory rate by integrating the digital bioacoustic signal with the ballistocardiographic signals. In this case, the digital bioacoustic signal captures breathing sounds and the IMU determines thoracic motions indicative of inhalations and exhalations. This combination of digital bioacoustic signal and ballistocardiographic signal provides an accurate respiratory rate assessment, effective even under conditions where the subject might not be stationary.

[0105] Determining the respiratory rate can include combining digital bioacoustic signals (capturing breathing sounds) with ballistocardiographic signals (indicative of thoracic motions) to assess the respiratory rate. In some cases, similar with determining the pulse measure, if the confidence measure for both IMU and stethoscope-based metrics are high, the average can be used. If one measure is significantly lower than the other, a higher weighting can be used for the signal with the higher associated confidence measure. In some cases, this weighted integration can be learned using sensor fusion and machine learning.

[0106] In some cases, a confidence determination can be determined for the primary microphone 130 and the secondary microphone 132 measurements (i.e. , stethoscope-based) by, for example, determining the confidence of a determined ACF for the stethoscope-based approach by comparing a max of the peak to a mean (i.e., peak prominence); and generating a confidence rating from the IMU by determining 1 over the standard-deviation (motion data in the window being inspected).

[0107] In some cases, a confidence determination can be determined for the IMU-based measurements by, for example, identifying peaks within a determined ACF for the IMU-based approach that fall within an expected period of a heart rate signal; determining the prominence of each peak, defined as the vertical gap between the peak and its highest adjacent minimum; selecting the peak with the highest prominence and determining its periodicity to determine the heart rate value; and determining a confidence value based on the ratio of the highest prominence to the mean over the identified interval.

[0108] In some cases, the system 200 can be configured to only perform measurements when the animal is stationary and a higher measurement confidence is more likely; for example, programming an interrupt to be triggered by the I MU to perform measurements when measured activity is low. When the animal is stationary, stethoscope measurements are more straightforward to analyze. However, movement can introduce noise and artifacts that obscure the true signals. Ballistocardiography measures the mechanical activity of the heart and blood flow and is less susceptible to noise from external movements because it focuses on internal bodily functions (like the recoil of the body with each heartbeat) and remains more consistent. By combining bioacoustic signals with ballistocardiographic signals (referred to herein as ‘sensor fusion’), such as in a weighted combination, the system 200 can use the measurement approach that has the highest confidence, even during movement.

[0109] In order to distinguish the breathing sounds from the heartbeat sounds, signal processing can be performed to identify peaks in an autocorrelation function that correspond to time lags between events (e.g. a heartbeat or an inhalation). Physiologically plausible ranges can be used to differentiate between heart rate (HR) and respiratory rate (RR). A typical HR of 60 to 140 BPM would have an expected autocorrelation function (ACF) peak lag of 1 second to 0.43 seconds. A typical RR of 15 to 30 breaths per minute would have an ACF peak lag of 2 to 4 seconds.

[0110] At block 614, the output element 220 outputs one or more of the determined pulse rate, respiratory rate, and temperature to, for example, the data storage 204 or to other databases, systems, or devices via the communication interface 206. For example, to a cloud-based system providing a user interface as illustrated in FIG. 11, or an application on a mobile phone. In some cases, the output element 220 also outputs raw data such as the IMU data and the recorded microphone signals. In some cases, the outputted data can be fed into a machine learning model for analysis of the collected sounds, allowing for sophisticated interpretation and diagnostic insights, as described herein.

[0111] In some cases, the system 200 can continuously, or periodically, sample the sensors, such as recording the primary microphone’s 132 audio signal (and the secondary microphone’s 134 signal) at a suitable frequency (e.g., audio sampling at 4 kHz to 44 kHz, capturing the range of heart and lung sounds), reading the IMU 230 data at, for example, 100 Hz or more, and reading temperature from the IR sensor 136 as needed (e.g., once per second or on demand). In some cases, initial filtering of these samples can be performed; for example, band-pass filtering the audio to the range of interest (roughly 20 Hz to 1000 Hz for heart and lung sounds). The noisecancellation, as described herein, can be performed by subtracting ambient microphone signals from the primary microphone signals.

[0112] By analyzing both the acoustic signals and the movement data from the IMU, the system 200 can discern various respiratory patterns, thereby detecting signs of respiratory distress or irregularities. The concurrent processing of these signals enriches the respiratory rate data, for example, aligning it with the detected heart sounds for a comprehensive cardiopulmonary evaluation. This integrated approach empowers veterinarians with a nuanced understanding of the animal's respiratory health, augmenting the overall diagnostic process. By integrating these multi-modal biosensing capabilities, the system 200 can be used as a comprehensive diagnostic tool, enabling health care professionals, such as veterinarians, to conduct a thorough and efficient assessment of the subject’s health.

[0113] FIG. 7 illustrates a diagram of an example implementation of the device 100. As illustrated, the primary microphone 130 is located underneath the silicone membrane 116 while the secondary microphone 132 is located adjacent to the membrane 116. With the dual microphone arrangement, the signal from the secondary microphone 132 will include environmental noise and a potential faint quantity of auscultation sounds and the signal from the primary microphone 130 will include auscultation sounds from the body and the environmental noise. In some cases, if the secondary microphone 132 picks up some auscultation sounds from the body, the subtraction to arrive at the desired bioacoustic signal will not be perfect; potentially leading to partial cancellation of the body sounds. Advantageously, the membrane 116 of the present embodiments provides attenuation of the auscultation sounds from the body to the primary microphone 130 compared to the signal received by the secondary microphone 132 which is physically separate from the membrane 116.

[0114] In the particular case illustrated in FIGS. 1, 2, and 7, the silicone membrane has a bulbous hemispherical shape. The membrane is flexible and its shape permits deformation when pressed against the skin (or fur in the case of an animal) by the health care professional. In this way, the sounds can be thoroughly transmitted through the membrane from the skin to the primary microphone 130. Advantageously, the primary microphone 130 located underneath the membrane 116 receives a greater quantity of the bioacoustics signals (from the tissue) due to the tissue having a similar impedance to the membrane 116 (e.g., silicone) while the noise of the air is reflected due to the acoustic impedance mismatch with the membrane 116.

[0115] The membrane can be selected for both acoustic transmission and resistance to degradation over time. For optimal acoustic transmission of body sounds to primary microphone 130, an ideal signal path would minimize acoustic reflection back into the body caused by impedance mismatches between the tissue and external medium (i.e. the body-device interface). This transmission coefficient Tis defined by the equation:Where Tis the transmission coefficient, representing the ratio of the sound intensity transmitted into medium B to the sound intensity arriving from medium A; Zais the acoustic impedance of medium A; and Zb is the acoustic impedance of medium B.

[0116] The acoustic impedance (Z) of a medium is given by the product of the medium's density (p) and the speed of sound in that medium (c), that is:Z — p ■ cWhere p is the density of the medium (in kilograms per cubic meter, kg / m3), and c is the speed of sound in the medium (in meters per second, m / s).

[0117] The closer the values of Za and Zb are to each other, the higher the transmission coefficient, meaning more sound is transmitted between the media. If Zaequals Zb, the transmission coefficient is 1, which means no sound is reflected and all the sound energy is transmitted.

[0118] The acoustic impedance of body tissue can vary depending on the specific type of tissue, as different tissues have different densities and sound speeds. As tissue is largely composed of water, the density p is about 1,000 kg / m3, and the speed of sound is roughly 1550 m / s. For soft tissues in the human body, such as muscle, the acoustic impedance is typically in the range of 1.3 to 1.7 xio6kg / (m / s).

[0119] For air at room temperature (approximately 20°C), the density p is about 1.21 kg / m3, and the speed of sound c is roughly 343 m / s. Therefore, the acoustic impedance Z of air can be calculated as approximately 415.03 kg / (m / s).

[0120] Given Za / ras 415.03 kg / (m / s), and Ztissue is 1.638 x106kg / (m / s), the acoustic transmission coefficient is approximately 0.00101, indicating that only a very small fraction of sound energy is transmitted from the body tissue to the air. This low value reflects the significant acoustic impedance mismatch between body tissues and air, resulting in most of the sound being reflected rather than transmitted.

[0121] In the case of silicone rubber with a durometer rating of 37A, the density p is about 1,100 kg / m3, and the speed of sound is roughly 950 m / s, therefore the estimated acoustic impedance Z of silicone is approximately 1.045 x106kg / (m / s).

[0122] Given Zsmcone as 1.045 xio6kg / (m / s), and Ztissue is 1.638 xio6kg / (m / s) the acoustic transmission coefficient T is approximately 0.951, indicating a high efficiency of sound transmission from the silicone to body tissue.

[0123] In addition, given that the external noise is transmitted through air, the silicone membrane also serves to reflect the unwanted acoustic signals while transmitting the desired signal to the microphone. Specifically, given Zsmcone as 1.045 X106 kg / (m / s)., and Zairas 415.03 kg / (m / s), the acoustic transmission coefficient Tis approximately 0.00159, indicating that a very small portion of noise in the air is transmitted through the silicone. This assumes that the silicone is directly coupled to the membrane of a MEMS microphone which was selected to enable direct-curing of the silicone without impacting the devices functionality. There may exist a further internal silicone-air interface between the membrane and the microphone, which reduces the efficiency of transmission of body sounds; however, the signal is still substantially improved because the signal comes from direct coupling to the body with external noise having been reflected.

[0124] Accordingly, the acoustic impedance of silicone is closer to that of body tissues when compared to air, as evidenced by the transmission coefficients. A higher transmission coefficient between silicone and body tissue (approximately 0.951) allows silicone to transmit the sounds of the body (like heartbeats and lung sounds) more efficiently than air. This is particularly advantageous for capturing clearer body sounds with less reflection and loss. Further advantageously, silicone is a flexible and durable that creates an airtight seal against the skin. Further, silicone is resistant to temperature variations, resistant to ultraviolet light, and is generally inert, non-reactive, and can be easily sterilized, making it suitable for repeated medical use. Further, Silicone is naturally waterproof, which gives the additional benefit of acting as a gasketto the internals of the device. This is particularly advantageous for animal subjects which are exposed to wet weather, swimming in water, and harsh weather conditions.

[0125] In some cases, the system 200 can enter a sleep state to conserve battery power and storage. In some of such cases, the system 200 can use the IMU to detect when motion exceeds a given threshold and then enter an operating state. In further cases, specific types of movement can be detected to enter the operating state, for example, recognizing when gait is occurring, recognizing when respiratory rate is exceeding a given threshold, or the like. In further cases, specific types of sounds can be detected to enter the operating state, for example, when measured noises are above a given threshold volume. In further cases, the system 200 can enter the operating state at certain periodic intervals, or during certain programmed times, based on the real-time clock.

[0126] FIG. 8 is a flowchart illustrating a method for determining heart rate (HR) and respiratory rate (RR) using stethoscope analysis, in accordance with an example embodiment. The signal element 214 performs preprocessing by performing one or more of:• At block 802, determining a noise-canceled stethoscope signal by filtering and subtracting the secondary microphone 132 reference signal from primary microphone 130 signal to enhance signal quality, as described herein.• At block 804, segmenting data into windows (e.g. a 3 second window would contain between 3-7 heartbeats for an adult dog).• At block 806, removing a direct-current (DC) component by subtracting a mean.• At block 808, performing high and low pass spectral filtering to remove noise and artifacts.• At block 810, removing spurious spikes in the signal using Schmidt's spike removal algorithm, by:o Detecting spikes where the amplitude exceeded a multiple (spike threshold factor) of the median of the max amplitude across all windows.o Determining the nearest zero-crossings around the spike, defining the boundary of the spike, then setting the values within that boundary to a small value.

[0127] After pre-processing, at block 812, the signal element 214 performs feature extraction by extracting the Hilbert-homomorphic envelope of the signal and determining the autocorrelation function (ACF).

[0128] After feature extraction, at block 814, the signal element 214 extracts the heart rate and the respiratory rate by extracting peaks in the ACF in an interval where a periodic heart or respiratory signal would be expected.

[0129] After heart rate and respiratory rate were extracted, at block 816, the signal element 214 determines a confidence interval for each of the HR data points by determining the confidence of the stethoscope ACF by comparing the max of the peak to the mean (peak prominence) and generating a confidence rating from the I MU by taking 1 over the standard-deviation (motion data in the window being inspected).

[0130] After the confidence interval was determined, at block 818, the signal element 214 selects a single value to be output as the HR and RR by taking the extracted metrics for the window with the highest confidence interval.

[0131] FIG. 9 is a flowchart illustrating a method for determining heart rate (HR) and respiratory rate (RR) using IMU-based analysis, across each IMU channel, in accordance with an example embodiment. The signal element 214 performs preprocessing by performing one or more of:• At block 902, segmenting the IMU signal into overlapping windows for processing.• At block 904, eliminating the DC component of the IMU signal by subtracting the mean of the signal from the signal itself.• At block 906, applying high and low pass filters to reduce noise and artifacts.

[0132] At block 908, the signal element 214 performs envelope extraction by obtaining a Hilbert-homomorphic or exponential moving average envelope for the IMU signal.

[0133] At block 910, the signal element 214 determines the autocorrelation function (ACF) of the envelope. At block 912, the signal element 214 identifies peaks within the ACF that fall within the expected period of a heart rate signal. At block 914, the signal element 214 determines the prominence of each peak, defined as the vertical gap between the peak and its highest adjacent minimum. At block 916, the signal element 214 selects the peak with the highest prominence anddetermines its periodicity to determine the heart rate value. At block 918, the signal element 214 determines a confidence value based on the ratio of the highest prominence to the mean over the identified interval.

[0134] With a determined heart rate and confidence level for each signal segment across all IMU channels, at block 920, the signal element 214 determines which of the IMU channels exhibited the highest maximum confidence factor and generates a weighted average of all HR or RR values for that channel using its corresponding confidence factors as the weight.

[0135] FIGS. 10 to 13 illustrate an example experiment conducted by the present inventors to illustrate visible respiratory rate signatures acquired by the device 100, as used as a collar worn around a dog’s neck. In this example, the subject held their breath and exhaled at the end of the experimental period; which is illustrated by the drop in the accelerometer X-direction at the end of the period. FIG. 10 is a graph that illustrates IMU data clearly illustrating heart rate and respiratory rate; where at point 1) showing an inhale, at point 2) showing an exhale, at point 3) showing an inhale, at point 4) showing an exhale, at point 5) showing an inhale, and at point 6) showing an exhale. As illustrated in FIG. 10, the heartbeat ballistocardiographic signal is primarily located on the slow / low frequency breathing component of the signal.

[0136] In FIG. 11A, the top subplot is a measurement from the primary microphone 130, the second subplot showing a Fast Fourier Transform (FFT) of the primary microphone 130 measurement, the third subplot is a measurement from the secondary microphone 132, and the fourth subplot shows a FFT of the secondary microphone 132 measurement. The subplots in FIG.11B show the gyroscope measurements in each direction and the subplots in FIG. 11C show the accelerometer measurements in each direction. The top output graph in FIG. 12 shows an example spectrogram from the primary microphone 130, the middle output graph in FIG. 12 shows an example spectrogram from the secondary microphone 132; and the bottom output graph in FIG. 12 shows the subtracted output; clearly illustrating the desired bioacoustic signal. As can be evidenced in the example experiments, the present embodiments are able to effectively filter out environmental noise, which provides substantially enhanced diagnostic capabilities. FIG. 13 illustrates an example user interface showing data outputted by the output element 220 for assessment by the health care professional.

[0137] FIGS. 14A to 14C are three graphs, for the example experiments, showing use of the IMU data for the determination of pulse rate (heart rate). In this example, a 30 second recording isdivided into 5 second intervals (chunks). For each interval, the pulse rate is determined using the bioacoustic measurements from the dual microphones and a confidence measure of the signal quality is also determined. In this case, the accelerometer data in the Z-direction is used to find time points with appropriate movement and higher confidence. FIG. 14A shows accelerometer recordings from a collar worn on a dog’s neck, FIG. 14B shows a calculated confidence for the extracted heart rate (HR), and FIG, 14C shows the determined HR. As seen, in the presence of moderately high motion periods, the HR estimates are less accurate and the confidence is reduced. When motion is reduced, confidence climbs and estimates are more accurate.

[0138] Where the device is strapped around the neck of an animal as a collar, the collar can rotate around the neck, the dog can be upside down, or the like. The axis to be used for ballistocardiography can likewise vary. The measurements can be repeated for each axes, and the segment with the highest confidence can be used; as illustrated in the example shown in the chart of FIG. 15. FIG. 15 shows that extracted HRs and confidence measures for different IMU dimensions. Given that the collar can rotate around the neck and the dog can be upside down or the like, the axis to be used for ballistocardiography (aka cardioseismography) can vary. In this way, in some cases, extraction can be performed for all axes, and the segment which has the highest confidence can be used; for example, in FIG. 15, the dot to the right at approximately 75 BPM can be the final measure that is determined by the device for pulse rate. Due to this high confidence, in some cases, the system 200 can only store this data point to save power and storage capacity; and can also improve reliability in cases with low transmission bandwidth.

[0139] FIGS. 16A to 16D illustrate experimental data of example experiments conducted by the present inventors. FIG. 16A illustrates a stethoscope signal, FIG. 16B illustrates a gyroscope signal in the X direction, FIG. 16C illustrates a calculated confidence for the extracted heart rate (HR) determination, and FIG. 16D illustrates the determination of the HR signal. In the presence of moderately high motion periods in the first half of the illustrated capture, there are less accurate estimates. When motion subsides, confidence measurements climb and estimates are more accurate. In the later captured period, motion again is experienced and confidence measures again recede.

[0140] As evidenced in the example experiments, the HR and RR determination can involve identifying peaks in the stethoscope data that correspond to heart beats and lung sounds. Specifically, an autocorrelation function (ACF) of the Hilbert envelope of a short stethoscope recording (e.g., 30 seconds) can be used to find these peaks. By looking for pronounced peaksin the ACF within the ranges HR and RR would be expected to fall under, the system 200 can infer the HR and RR. A typical HR of 60 to 140 BPM would have an expected ACF peak lag of 1 second to 0.43 seconds. A typical RR of 15-30 breaths per minute would have an ACF peak lag of 2 to 4 seconds.

[0141] To find periods of the data that are cleanest for RR and HR extraction, the signal can be windowed and the system 200 can operate, for example, on 3 second blocks with 50% overlap; e.g. for a 30 second window, heart rate (HR) metrics would be extracted at 1.5 seconds with a + / -1.5 second window, up to 28.5 + / - 3 seconds. This would result in a total of 19 windows, each with an HR datapoint for a 30 second recording.

[0142] In some cases, as illustrated in FIG. 17, the device 100 can be extensible over universal-serial-bus (USB), such that peripheral devices can be added to enable new sensing modalities. For example, an electrocardiogram (ECG) peripheral that's illustrated in the top trace in FIG. 13 can be recorded using a module connected via USB.

[0143] The system 200 and device 100 provide substantial advantages in the veterinary setting. For animals with fur, unlike humans, the medical professional cannot get direct contact with the skin and typically relies on rectal thermometers to determine an internal temperature; which can be distressing for the animal. Other conventional surface temperature sensors do not work sufficiently as they require direct contact with the skin. In contrast, the non-contact far-infrared temperature sensor of the present embodiments provides particular usefulness when monitoring animals with fur. Furthermore, animals generally find traditional auscultation measurement tools, such as stethoscopes, distressing due to the uncomfortable and unfamiliar nature of their use. In contrast, the palm mounted implementation illustrated in FIG. 2 allows the veterinarian to receive bioacoustic signals from a comfortable touch against the body of the animal; much in the same manner as caressing the animal. In this way, as evidenced by example experiments conducted by the present inventors, veterinarians find such auscultation measurements much more comfortable for the animal, and in this way, are much less prone to be subject to the white-coat effect.

[0144] In further embodiments of the present invention, there is provided a system and method for automated patient record-keeping that leverages data from one or mode multi-modal physiological sensing devices, such as the device 100; and in a particular case integrates it with one or more computing devices and applications such as a smartphone application and a cloud-based server. In a particular case, the system uses audio recording capabilities of devices, such as the device 100 or a smartphone, to record patient intakes, transcribe these recordings using advanced text-to-speech, and integrate the transcribed data with physiological data captured by the device.

[0145] In some cases, captured data can be enhanced by automatically tagging it with timestamps and processing it through a language model aware of the current time, thereby contextualizing the information. This automatic data tagging not only streamlines the recordkeeping process but also provides veterinarians with timely, accurate, and searchable records.

[0146] Additionally, in some cases, SOAP (Subjective, Objective, Assessment, and Plan) notes can be automatically generated and sent to a health care professional; such as to a veterinarian or a 3rdparty system; which improves the efficiency of the veterinary practice. In some cases, a cloud-based user interface allows for the visualization of the recorded data, and supports outsourcing of data labeling to a telemedicine board certified specialist (e.g. cardiologist or neurologist).

[0147] In some cases, a physiological sensing device can be integrated with the device 100 or can be integrated with a smartphone application which can be installed on a suitable device of a veneration in addition to other users. The sensing device captures physiological data from patients, which is then sent to the smartphone application via a secure wireless connection. Concurrently, the health care professional uses the smartphone to record a verbal summary of the patient intake. This verbal summary recording is transcribed in real-time using advanced text-to-speech technology, creating a textual record of the subjective and objective observations made during the intake.

[0148] To enhance accuracy and contextual relevance, a machine learning algorithm can be used that analyzes the transcribed data against a vast database of veterinary medical records to create an Al-enhance analysis which can identify patterns, anomalies, or correlations that may not be immediately apparent to the practitioner. This Al-enhanced analysis can provide predictive insights, such as potential diagnoses or treatment recommendations, based on the combined data set of transcribed notes and physiological data.

[0149] A language model is employed, such as a Large Language Model (LLM), which receives as input, or is otherwise aware of, the current time and context, to process both the transcribed text and the physiological data. The collected information is correlated, organized it into acoherent record that reflects the chronological sequence of the patient visits. This coherent recording approach also allows clinical judgment to be applied to the collected data. For example, the system can identify an elevated heart rate recorded multiple-days prior using an at-home monitoring device which may no longer be accurate and should be weighted in the clinical record accordingly. Additionally, the system incorporates a feedback mechanism where the veterinarian can validate or correct the Al-generated insights, continuously improving the accuracy and relevance of the model over time.

[0150] Once collected and processed, the processed data can be formatted into a standard SOAP note, which is automatically saved in the digital record of the patient and can be emailed to the veterinarian or other healthcare providers as needed.

[0151] Furthermore, the system can automatically recommend additional tests or insights based on the Al enhanced analysis, further enhancing the care process and ensuring timely interventions.

[0152] Turning to FIG. 18, a general diagram is shown for an operating environment 900 for a system for automated medical record generation 1000, also referred to as automated medical record system, in accordance with an embodiment. FIG. 18 shows various physical and logical components of an embodiment of the system 1000. As shown, the system 1000 has a number of physical and logical components, including one or more processors 1002, a non-transitory data storage 1004, a communication interface 1006, and a local bus 1008 enabling the components to communicate each other. The one or more processors 202 can include one or more central processing units, one or more graphical processing units, microprocessors, dedicated hardware, logic arrays or other integrated processing circuits. The one or more processors 1002 are generally located externally from the physiological sensing device 100 and communicates with the processor on the main circuit board 104. The data storage 1004 can store programs, instructions, and / or an operating system, including computer-executable instructions for implementing the methods described herein, as well as any derivative or related data. The data storage 1004 can include instructions for executing a number of conceptual modules on the one or more processors 1002, such as a communication module 1012, a machine learning module 1014, a document generation module 1016, and an output module 1018. It is understood that the functions of these conceptual modules can be combined or executed on other conceptual modules, as appropriate.

[0153] The operating environment 900 includes the system (such as a server) 1010 that is in communication with other components of the operating environment 900 and can maintain and process information received from them. The in-clinic device 1020 can be one or more devices used in clinics to generate or collect medical data and may include the system 200 or other devices, such as medical testing devices, medical diagnostics devices, or other computing devices and systems used in a medical context. In-clinic sensor device 1030 and at-home sensor device 1040 can each include one or more multi-modal physiological sensing devices, such as device 100, or any other device capable of measuring one or more physiological factors of a patient. User device 1050 is a computing device such as a smartphone, typically used by a user such a physician or patient, or the device 100, and is used to interact with the system 1000.

[0154] Once the raw data streams have been processed and key features or parameters extracted, as provided in the method 600 (e.g., clean audio, event markers, vital signs, detected anomalies, etc.), automated interpretation and documentation can be performed by the system 1000. It should be understood that the system for for automated medical record intake, record generation and record keeping 1000 and the system for non-invasive physiological sensing 200 can be run on the same device(s) and share one or more components. The system 1000 can reside on the same computing system or server as system 200 or on a dedicated computing system or server. In a particular case, predictive models can be used for diagnosis support. For example, a deep neural network classifier can take the processed heart sound waveform or its spectral features and determine whether a heart murmur is present (and if so, classify its type and severity). In another case, a machine learning model can analyze lung sound patterns to detect wheezing, crackles (rales), or other adventitious sounds. These models can be trained on large datasets of labeled medical audio and can output quantitative measures (e.g., murmur intensity, probability of abnormality) or categorical results (e.g., “suspected atrial fibrillation” if an irregular rhythm is detected). The outputs of these machine learning models can be used for diagnostic insights; for instance, an indication that “a mild systolic murmur is present,” or “lung sounds consistent with mild wheezing in the left lower lobe,” or simply that all examined sounds are within normal limits. The system 1000 may also incorporate patient-specific context in analysis, such as known pre-existing conditions (for example, if the patient has a history of asthma, the threshold for flagging a wheeze might be adjusted to reduce false positives).

[0155] FIG. 19 shows a high-level functional diagram 2000 of the automated medical record system 1000 and shows the functional interactions between the first server 1010 and the in-clinic device 1020, a first sensor device 1030 used in clinic, and a second sensor device 1040 used athome. Vet-provided records 2010 include all forms of manually collected patient data such as manually typed-in observations, notes, assessments, treatment plans, test results, etc. In some embodiments, vet provided-records 2010 include recorded audio and / or transcription of recorded audio of conversations with patients or their care providers and guardians. The vet-provided records 2010 are used in a style summarization 2020 function to be adapted into a proper format of data to be sent to the large language model 2030. The LLM 2030 can first transcribe the recorded audio of the vet-provided records 2010, and then integrate them with the physiological data received from in-clinic devices 1020 and the sensor devices 1030 and 1040. In some embodiments the LLM 2030 may add timestamps and other tagging and finally further analyze the vet-provided records 2030 against a vast database of veterinary medical records to create an Al-enhanced analysis to identify patterns, anomalies, or correlations that may not be immediately apparent to the user of the user device 1050 or a veterinarian. The LLM 2030 then generate medical records 2040, as well as patient support docs 2050 and suggestions of actions 2060 to be stored by server 1010 and used for user reviews or further processing in the next rounds of patient intake or LLM training cycles or the like. The suggestions of actions 2050 may include instructions for further collection and processing of physiological data from the in-clinic sensor device 1030, and the at-home sensor device 1040.

[0156] FIG. 20 shows a high-level diagram of data transmission pathway from the first sensor device 1030 to user device 1050 and subsequently to a cloud server 1010. The diagram details the various stages of secure and automatic transfer of collected data flow, starting with the collection of physiological data by the first sensor device 1030, which is then transmitted to a smartphone app 3010 executed on a user device 150 via a wireless connection. The user device 150 can additionally collect manual data entries or additional details such as audio records and / or transcription of conversations with patients, their care givers, or gaurdians. The user device 150 sends the collected data to the server 110 using a WiFi or cellular network which is then further processed and securely stored by the server 110. The diagram also identifies an example embodiment of the connections protocols used at each communication stage, however, any other suitable communication means and protocols may be used. In some embodiments, sensor device 150 may directly communicate with the server 110 without involving the user device 150.

[0157] FIG. 21 shows four screenshots of an example checkup procedure performed by a user of an example embodiment of the system 1000 such as a veterinarian. Screen 4010 shows a sample of the smartphone app 3010 executed on a user device 1050. The user can select the checkup option to open a checkup page such as a heart measurement page 4020. Aftercompletion of monitoring the results can be sent to the server 1010 followed by a submission confirmation 4030. A snippet of the graphical user interface 4040 is also included which shows a sample of results on the server 1010.

[0158] FIG. 22 shows four screenshots of an example patient intake procedure performed by a user of an example embodiment of the system 1000 such as a veterinarian. Screen 5010 shows a sample of the smartphone app 3010 executed on a user device 1050. The user can select the intake option to open an audio recording page 5020 for a specific patient. The user can view, playback, or synchronize previous recordings with the server 1010. A snippet of the graphical user interface 5040 is also included which shows a sample of the synchronized records on the server 1010. The sample 5040 shows how veterinarians can access and review patient records, SOAP notes, and Al-driven insights among other details. The sample 5040 also shows interface for the feedback mechanism, where practitioners can validate Al suggestions.

[0159] FIG. 23 shows a screenshots 6000 of a sample the graphical user interface of the server 1010 which displays menu options and a sample of collected patient information logs for a specific patient which is listed in a chronological order.

[0160] FIG. 24 shows a screenshot 7000 of the graphical user interface for the server 1010 which displays menu options and a sample of collected device data for a selected period of time. The sample shows a highlighted period of data which is marked with a label of “Unusual activity” by the system 1000. The highlighted period is further accompanied with user annotations of a veterinarian.

[0161] FIG. 25 shows a visual flowchart 8000 of an example data processing flow in system 1000. The demonstrated steps involve recording audio 8010 in a patient intake session, transcribing the recorded audio 8020 using an LLM 2030 which is sent for further LLM processing 8030 and analysis along with the physiological data 8040 for further analysis. The results can be recorded as a SOAP note 8050 or emailed 8060 to the veterinarian or sent to a 3rdparty system.

[0162] FIG. 26 shows a flowchart diagram illustrating the feedback loop mechanism 9000 for LLM 2030. The vet-provided records 2010 are sent to a preference learning 9020 along with multimodal inputs 9030 received from sensor devices 1030 and 1040 and in-clinic devices 1020. The output of the preference learning 9020 is passed to the LLM 2030 which generates medical records 9050. The medical records 9050 then goes through a clinical review 9060 stage which may include a manual review by a technician or a veterinarian who may add additional data tothe medical records 9050. The medical records are then passed to the preference learning step 9020 again for further processing in the next round of execution of the same feedback loop mechanism 9000. Advantageously, the feedback loop mechanism 9000 can refine and improve the predictive models and recommendations over time.

[0163] FIG. 27 shows a visual flowchart 1009 of an example data processing flow in system 1000. The demonstrated steps involve recording audio 1019 in a patient intake session, transcribing the recorded audio 1029 using an LLM 2030 which is sent for further LLM processing 8030 and analysis along with the physiological data 1049, medicine database 1079, prescription and specialist referral system 1089. The results can be recorded as a SOAP note 1059 or emailed or displayed in a web page 1099 to the veterinarian or sent to a 3rdparty system 1109.

[0164] In some embodiments, the visual flowchart 1009 of FIG. 27 can be considered as a visual representation of two integrated healthcare systems: an Automated Prescription and Specialist Referral Systems as the prescription and specialist referral system 1089, and a Real-Time Medication Suggestion System functioning in place of the medicine database 1079.

[0165] The Automated Prescription and Specialist Referral Systems may include a Prescription Generation and Pharmacy Stock Check component and a Specialist Referral component. When a medication is mentioned within the automated medical record system 100, an automated process is triggered to generate a prescription in the Prescription Generation and Pharmacy Stock Check component. Concurrently, the Prescription Generation and Pharmacy Stock Check component, verifies the availability of the prescribed medication by checking the stock levels at connected pharmacies. Additionally, based on Al-recommended diagnoses, including but not limited to the output of LLM 2030, the Specialist Referral component identifies and contacts relevant medical specialists. It also checks the availability of the identified specialists to facilitate timely appointments.

[0166] The Real-Time Medication Suggestion System may include a Symptom Analysis and Medication Suggestion component which is capable of analyzes the symptoms discussed during patient intake, using real-time data processing techniques. It then suggests possible medications immediately, aiming to enhance the responsiveness and accuracy of the care provided.

[0167] The following is a sample SOAP data record for a patient:**Subjective:**- Patient Name: Iggy- Species: Feline- Age: Approx. 9 months- History: Iggy started showing reluctance to jump and decreased activity approximately two weeks ago. She stopped jumping onto the bed and couch, which was unusual behavior for her. No more "zoomies" observed. Today, after being placed on the couch, she had difficulty standing straight after jumping off, with an inability to get up for about 30-40 seconds. No vocalization of pain but is generally a quiet cat. Appetite has decreased slightly; now eats wet food in two sittings rather than one. Fully vaccinated, adopted at eight weeks old. Possible hearing issues noted; does not respond to sounds as expected. No witnessed injuries. Indoor cat.- Current Diet: Dry food available at all times, wet food (goat milk-based, brand possibly "Kit Kat") once daily.**Objective:**- Physical Examination Findings: Iggy shows a lack of proprioception in her hind legs, not fully aware of their placement. When testing reflexes and responses, delayed reactions observed. Stands with feet upside down when manipulated. Pain response noted in the lower back upon palpation. No other visible signs of injury or illness.- Neurological Deficits: Yes, primarily affecting hind legs and lower back.- Pain: Yes, localized to the lower back.**Assessment:**- Iggy presents with neurological deficits and pain in her lower back, suggesting a spinal issue. Differential diagnoses include intervertebral disc disease (although rare in young cats) or an infection affecting the spinal cord (discospondylitis). Lack of proprioception and pain upon palpation of the lower back are significant findings. Hearing issues noted but not currently the primary concern.**Plan:**1. **lmmediate Care:**- Start on pain medication and antibiotics to cover for potential infection. This approach is chosen due to financial constraints and the possibility of infection being a contributing factor.- Confine Iggy to a small area with easy access to food, water, and litter. Avoid feeding tomorrow morning in case sedation is needed for further diagnostics.2. **Diagnostics:**- Recommend blood tests and X-rays to be conducted by the primary care veterinarian due to cost considerations. These tests can help identify infections (elevated white cell count) and may show changes in the spine indicative of discospondylitis or other abnormalities.- Advanced imaging (CT scan or MRI) discussed as an option for a more definitive diagnosis but is cost-prohibitive at this time.3. **Long-term Management:**- Monitor Iggy's response to medication closely. If no improvement or worsening of symptoms, re-evaluation and possibly advanced diagnostics will be necessary.- Discuss with roommate and consider financial options for further care, including the possibility of referral to a neurologist for advanced imaging and potential surgery.4. **Euthanasia:**- Discussed as a last resort if Iggy's condition worsens significantly and advanced care is not financially feasible. The owner is advised to consider this possibility but focus on current treatment and diagnostics.**Notes:**- Financial constraints significantly influence the diagnostic and treatment plan.- Owner to consult with roommate and follow up with primary care veterinarian for further diagnostics and treatment initiation.- Full records, including this conversation and recommendations, will be sent to the owner's email for sharing with the roommate and primary care veterinarian.**Follow-Up:**- Advise owner to monitor Iggy's condition closely, especially her ability to urinate and defecate, and to keep the follow-up appointment with the primary care veterinarian. - Reassess the need for referral to a neurologist based on primary care veterinarian's findings and financial situation.

[0168] The following is another sample SOAP data record for a patient:**Subjective:**Patient: Canine, Male, Age: Approximately 13.5 years, Adopted from Toronto Humane Society in August 2011, estimated to be 1-2 years old at the time of adoption, but owner believes he was closer to 6 months.History: Patient lives with three other dogs in the home, no current problems reported with the other dogs. Patient is up to date on vaccines. Diet consists of home-cooked meals including beef, chicken, sweet potatoes, mixed vegetables, and various additives. Previous medical history includes luxating patella in both back knees, a slight case of degenerative back disease, and a history of a mass thought to be cancerous but was treated holistically and resolved. The patient was scheduled for dental extraction due to dental disease. Blood work performed last week was unremarkable. Current medications include minerals, a capsule from a holistic vet believed to contain green mussels, collagen, and another unspecified pill recommended for his age.**Objective:**Physical Exam Findings: Patient is reactive in various places, indicating possible arthritis. Lame in one leg, with shoulder and neck pain on one side. Possible disc disease in the upper spine suggested by symptoms. Back legs appear okay despite known knee issues. Repeatable discomfort in the neck / shoulder area. Patient was given methadone for pain and tolerated small treats without vomiting. Lung sounds normal on auscultation today, despite previous vet noting something unusual.**Assessment:**1. Luxating patella - Known history2. Degenerative back disease - Slight case, known history3. Dental disease - Scheduled for extraction4. Possible disc disease - Indicated by physical exam5. Arthritis - Suspected due to reactivity and history6. Soft tissue injury vs. Fracture vs. Disc disease - Differential for current lameness and pain7. Previous mass - Treated holistically, believed to be related to dental disease by one hospital,but diagnosed as cancer by University of Guelph. No current evidence of mass.8. Pain management - Methadone administered**Plan:**1. Sedation and X-rays to further investigate the cause of lameness and pain.2. Monitor response to methadone.3. Dental extraction as planned, pending stabilization of current condition.4. Consider neurology consult for potential disc disease.5. Discuss hospitalization for overnight pain management and possible neurology consult tomorrow.6. Cost estimate provided for emergency exam, sedation, X-rays, etc., is approximately $1000 to $1200.**Notes:**- Owner reports holistic treatments for previous conditions. <Review: source audio unclear>- Specifics of supplements and medications need clarification for complete medical record. <Review: source audio unclear>- Follow-up required to obtain a copy of recent blood work.- Owner concerned about patient's mobility and bathroom needs pending diagnosis and treatment plan.

[0169] In some embodiments, as illustrated in FIGS. 28 and 29, clinical conversations can be recorded using a stethoscope attachment, which captures conversations ambiently using on-device processing without the need to use a smartphone. This attachment exists alongside a clinicians existing stethoscope, and can be simultaneously held in the hand while the clinician auscultates with their existing stethoscope, with the added advantage that body sounds are digitized and can be analyzed for diagnostic biomarkers (e.g. murmurs), and also provides a temperature measurement unlike a standard stethoscope, which is used to enhance the medical record.

[0170] FIG. 30 illustrates a method 3000 for automated medical record generation, in accordance with an embodiment. At block 3002, the communication module 1012 uses the communication interface 1006 to receive the one or more of respiratory rate, pulse rate, and temperature signal from the output element 220. In some cases, this receipt can be performed on the same computing system, and thus, retrieved from a shared storage device.

[0171] At block 3004, the machine learning module 1014 uses one or more machine learning classifiers to determine health related medical data from the inputted signals. For example, feeding a heart sound waveform or its spectral features into a model to determine whether a heart murmur is present. In another example, feeding lung sound patterns to a model to detect wheezing, crackles (rales), or other adventitious sounds. These classifier models can be trained on large datasets of labeled medical audio and can output quantitative measures (e.g., murmur intensity, probability of abnormality) or categorical results (e.g., “suspected atrial fibrillation” if an irregular rhythm is detected). The outputs of these classifier models can be used for diagnostic insights. In some cases, the communication module 1012 can also receive patient-specific context data to be used in the classifier analysis; such as known pre-existing conditions, age, weight, or the like.

[0172] At block 3006, in some cases, the machine learning module 1014 pre-processes the health-related medical data to extract one or more appropriate metrics. For example, the metric can be a standard deviation of NN Intervals (SDNN), which is a time-domain heart rate variability (HRV) metric measuring the standard deviation of normal heartbeat intervals (NN intervals) over a recording period. In another example, the metric can be root mean square of successive differences (RMSSD), which is a time-domain HRV metric calculating the root mean square of differences between consecutive heartbeat intervals. In another example, the metric can be an S1 / S2 amplitude ratio, which is the ratio of amplitudes between the first (S1) and second (S2) heart sounds. Other suitable metrics can be used as appropriate.

[0173] At block 3008, the document generation module 1016, in parallel with, or after block 3006, uses the output of the classifier model(s), and / or the derived metrics, to generate a human-readable medical report. In a particular case, the document generation module 1016 can use a large language machine learning model (or a set of templates supplemented by artificial intelligence) that generates text akin to a clinician’s report; for example, formatted as a SOAP (Subjective, Objective, Assessment, and Plan) note or exam report. Input to the large language model can include the time of the assessment and patient context allowing the output of the model to be time-aware and context-aware. In some cases, input to the model can include past data (for example, retrieved from storage 1004 or retrieved from an electronic medical record storage) in order to output relevant comparisons to the past data. In some cases, input to the model can include situational contexts (such as the fact that this exam is a follow-up, or the time of day, etc.). The large language model can be trained on medical terminology in order to structure the note clearly.

[0174] In an example where the document generation module 1016 structures the human-readable report in a SOAP note, the report can be structured as:• Subjective: In many cases, this section can be derived from patient input or clinician input (e.g., patient’s chief complaint or symptoms). The document generation module 1016 could optionally integrate with a voice recording or a questionnaire to capture subjective data. If the patient or clinician provides a brief voice description of symptoms, one of the device’s 100 microphones could record such description for transcription via speech-to- text. However, in other cases, this section can be left for user input or not automatically generated by the document generation module 1016.• Objective: This section can be automatically filled in with generated objective findings. For example: “Vital signs: Temperature 37.0 °C (via temporal scan), Heart rate ~75bpm regular, Respiratory rate ~16 / min. Physical exam: Heart -S1 and S2 present, no murmurs or gallops detected. Lungs - clear breath sounds bilaterally, no wheezes or crackles. No significant motion artifacts during exam.” This section may also mention the quality of data if relevant.• Assessment: In this section, the document generation module 1016 can generate an initial assessment based on the objective findings. If everything is normal, this section can state, for example, “Normal cardiac and pulmonary exam.” If an abnormality was detected, this section can state, for example, “Possible systolic murmur detected - consistent with trace mitral regurgitation (to be clinically correlated)” or “Wheezing noted - consistent with patient’s known asthma.” In some cases, the user can direct the document generation module 1016 to be as detailed or as brief as specified.• Plan: In this section, the document generation module 1016 can generate suggestions for plan elements where certain findings are present. For example, if an irregular heart rhythm was detected, the document generation module 1016 can output: “Plan: recommend ECG for further evaluation of arrhythmia.” Or if wheezes are detected in an asthmatic patient: “Plan: consider bronchodilator therapy adjustment.” The system 1000 could also integrate with clinical decision support databases to provide such recommendations.

[0175] At block 3010, after generating the human-readable report, the output module 1018 can output the report to the clinician for review via the communication interface 1006 (for example, displayed on the clinician’s smartphone or computer) or output the report for storage on the datastorage 1004. The clinician can verify the accuracy of the report and, in some cases, make any edits if necessary. In some cases, the output module 1018 can transmits the report to an Electronic Medical Record (EMR) depository. Integration with the EMR can be achieved via standard health information exchange formats (like HL7 FHIR or HL7 v2 messages) or via the EMR’s API. The report can either be inserted as a narrative clinical note entry, or the structured data (vitals, exam findings) can populate discrete fields in the patient’s record. For cloud-based or online EMRs, the output module 1018 can use, for example, a cloud server to communicate directly with the EMR cloud. For on-premises EMRs, the communication might route through the clinician’s device or a local network interface with appropriate credentials.

[0176] This method 300 can occur rapidly, in a matter of seconds to minutes; enabling real-time clinical documentation. For example, by the time a clinician finishes examining a patient with the device, a draft of the exam report can be already waiting for them, containing objective findings and initial assessment. This not only saves time but also standardizes documentation quality (using consistent structure and terminology). Moreover, the digital data (such as the recorded heart / lung sounds) can be stored alongside the report, allowing playback or re-analysis if needed, which is a significant improvement over the ephemeral nature of traditional auscultation.

[0177] Advantageously, the device 100 is able to capture clean physiological signals and such that the system 1000 can correctly interpret them. The device’s 100 dual-microphone bioacoustic sensing obtains clear heart and lung sounds. When the device is placed on the body, the primary microphone receives sounds transmitted through the membrane from inside the body (e.g., valve closures in the heart, breath sounds in the lungs). Simultaneously, the secondary microphone picks up environmental sounds that are not coming from the patient’s body. Because the secondary microphone also picks up some muffled body sounds (through the air, albeit much weaker), the noise cancellation algorithm can use techniques (such as adaptive filtering or spectral subtraction) to minimize only the noise components that are correlated between the two inputs. The result is a much higher signal-to-noise ratio for the internal body sounds. This technique allows the device to be used even in fairly noisy clinical environments or in the field (for example, outdoors in a farm setting for an animal exam) with minimal loss of fidelity. The device also mitigates mechanical noise; the flexible membrane and device housing help dampen vibrations from handling.

[0178] The IMU further assists by providing data to identify segments of audio that coincide with significant motion; if a large motion is detected by the IMU at a certain moment, the system canmark or discard the audio segment during that moment to avoid misinterpreting a bump or slide as a physiological sound. In this way, the IMU can be used to determine a real-time confidence metric that reflects the level of motion artifacts present during physiological signal acquisition. The IMU can be used to continuously capture motion and orientation data of the device 100 and the patient. The system 200 can segment this data into time windows and determine a motion confidence score for each window by computing the inverse of the standard deviation of the IMU signal within that window. Lower variability in the IMU signal corresponds to minimal movement, producing a higher confidence score. This confidence score can be dynamically weighted against, for example, the acoustic stethoscope data quality, allowing the system 200 to prioritize or extract heart rate (HR) and respiratory rate (RR) measurements during periods of reduced motion. As shown in FIG. 16C, higher confidence scores coincide with periods of reduced movement, enabling the system 200 to automatically select these segments for physiological parameter extraction. This approach reduces the impact of motion artifacts on vital sign determination, ensuring that the determined values are based on the most reliable data windows.

[0179] Advantageously, the present embodiments allow for multi-modal data to be integrated in a time-synchronized manner. Sensor data can be timestamped using a common clock in the device before transmission. This allows the system 1000 or the system 200 to align the data streams precisely. For example, suppose a faint heart sound anomaly is detected in the audio stream at a certain time; by checking the IMU stream at that same time, the system 1000 or the system 200 can determine if the device was in motion (in which case the sound might be artifact) or if it was steady (in which case the sound is more likely genuine). Likewise, a periodic pattern in the accelerometer data might correspond to respiratory motion, which can help confirm respiratory rate and phases detected in the audio. The time-aware analysis ensures that it can describe events or findings in chronological context, and even compare them to previous sessions from the same patient.

[0180] The machine learning models can be trained on diverse datasets of human and animal physiological signals. For heart and lung sound analysis, the system 200 may utilize convolutional neural networks or other pattern recognition algorithms that have been trained to recognize specific acoustic signatures of pathology. For example, in heart sounds, a model can be trained to detect the spectral and temporal patterns of a murmur or an arrhythmia. In lung sounds, a model can be trained to recognize the waveform shape of a crackle or the frequency content of wheezing. These models can output either a classification (normal / abnormal, type of abnormality) or regression (e.g., probability of a specific finding) that feeds into the document generationmodule 1016. The predictive analytics can also extend beyond immediate interpretation to forecasting or trend analysis. For instance, if over multiple visits the patient’s resting heart rate has been rising or a slight wheeze has gradually worsened, the system 1000 could detect that trend and flag it in the report (e.g., “wheezing has increased compared to prior exam last month”), potentially prompting earlier intervention.

[0181] Advantageously, the present embodiments can be performed with oversight; for example, a clinician can interact with the generated report. The system’s outputs (analysis and documentation) are transparent and can be reviewed, and the underlying data (like recorded sounds) can be audited if needed. The machine learning models can be advantageously cognizant of temporal information and patient history when generating outputs. This can be achieved by providing the model with relevant prior data (such as last visit’s key findings or trends) along with the current data. The result is that the generated documentation can read as if a clinician who knows the patient’s recent history wrote it. For example, rather than just stating current vitals, the generated report might say “Blood pressure slightly higher than at last visit” if that context is available and relevant. This contextual nuance is generally a technical feature of how the model was trained or programmed to incorporate external information (like EMR data) into the generation process.

[0182] In some cases, while sensing to documentation is automated, the present embodiments can accommodate user input or control at various points. For example, the clinician could start or stop the recording via a button on the device or app, tag certain audio segments (e.g., “lung sounds upper right”) via an input device, or even speak a note that gets transcribed. These features ensure that the technology adapts to clinical workflow rather than forcing the clinician to adapt to the technology.

[0183] Generally, the present embodiments comply with data security and regulatory compliance requirements. The communication with EMR systems can use secure, standardized communications. For example, it might employ the hospital’s VPN or secure HL7 messaging. The integration process will adhere to the healthcare provider’s IT policies. Additionally, measures like role-based access can ensure that only the treating veterinarian or physician sees the generated notes unless they choose to share them with colleagues.

[0184] The present embodiments are advantageously versatile, serving both human medical practice and veterinary practice with minimal modifications. In a typical human clinical scenario,a physician or nurse can use the device 100 during a routine physical exam or a focused examination (such as a cardiopulmonary check). The device 100 can be placed at various standard auscultation sites on the body (e.g., different locations on the chest for heart sounds, the back for lung sounds, the neck for carotid bruits, the abdomen for bowel sounds). The clinician might use a companion smartphone or tablet app that interfaces with the device 100. Through the companion, they can start a new recording session, enter patient identifier information (which could be pulled from the scheduling system or EMR), and see a live readout (like waveforms or a simple indicator that sound is being recorded properly). They perform the exam by moving the device to the required locations, possibly pressing a mark on the companion each time they switch location (such that the system 200 knows that, for example, “Segment 1 = heart apex, Segment 2 = lung base,” etc.).

[0185] In some cases, the clinician can see interim results in seconds; for example, a heart rate and respiratory rate calculation could be displayed almost immediately. By the time the clinician finishes listening to the heart and lungs, the system 100 (for example, using cloud resources) can have already processed the data and generated a draft report. The clinician can then review this draft on their device. If the note says “normal exam” and that matches their impression, they can accept it and maybe add a comment like “Patient reports mild cough for 2 days” (if not already captured). If the system 100 flagged something (say a “possible murmur”), the clinician can pay special attention to that; they could listen again manually or order further tests, or if they determine it was a false alarm, they might edit the note to remove that or mark it as insignificant. The final verified report can be uploaded to the EMR.

[0186] In another example use case, such as in an emergency or critical care setting, a paramedic could use the device 100 on a patient in an ambulance, and the data along with an auto-generated report could be sent ahead to the emergency department (ED). For example, “Breath sounds: diminished on right side, possible crackles; Heart rate 120, irregular” might alert the ED to prepare for a certain condition (like pneumothorax or heart failure). In the intensive care unit, a wearable form of the device could continuously monitor a patient and feed data to the system 1000, which could in turn update the medical record hourly and alert staff if trends go out of range.

[0187] In another use case in veterinary practice, a veterinarian could hold the device 100 against the pet’s chest or abdomen. The pet’s fur might dampen sound slightly, but the device’s sensitivity and noise cancellation can compensate (the vet might also wet the fur slightly or use a bit of gel to improve acoustic coupling if needed, similar to ultrasound gel usage). The machine learningmodels of system 1000, when in “veterinary mode,” would use models trained on animal data. Such models would know, for example, that a normal cat heart rate is much higher than a human’s, and that certain heart murmurs common in older cats have different acoustic profiles than human murmurs. The generated report might say, “HR -180 bpm (high-normal for clinic stress), no obvious murmur. Lung sounds clear. Temperature 38.6 °C (within normal range for feline).” For large animals like horses or cattle, the device 100 could be placed at the appropriate spots (with large animals, multiple devices or sequential recordings at different sites might be useful).

[0188] In some cases, the audio recorded by the device 100 can be compressed and encrypted locally; and in this way, the system 200 is not exposed to phone or web interface vulnerabilities. For example, if the data were unencrypted in transit, a third-party interloper could potentially listen to sensitive conversations, such as where they are stored on a personal cell phone. In an example, the system 200 can capture a 16-bit PCM audio sampled at 16 kHz from the secondary microphone positioned on the device 100 for clinical conversation recordings. The audio data can be processed in real-time by, for example, an Opus codec operating in voice-optimized mode, configured with constrained variable bitrate (CVBR) targeting 16 to 32 kbps and utilizing discontinuous transmission (DTX) to reduce bandwidth during silent periods. The compressed Opus audio frames can then be encrypted at the application layer, for example, using AES in Counter (AES-CTR) or Galois / Counter Mode (AES-GCM), with a 128-bit session key generated on-device using a hardware-based random number generator. This encryption secures the compressed data for local storage and end-to-end transmission. The resulting encrypted audio packets can be transmitted, for example, over a Bluetooth Low Energy (BLE) connection, where BLE link-layer AES-CCM encryption is applied to protect the data over the radio interface. This dual-layer encryption approach, first at the application level and then at the transport level, ensures that sensitive conversations remain confidential and tamper-resistant, whether stored on the device 100, transmitted wirelessly, or forwarded to a web application.

[0189] In some cases, the measurements from the device 100 can be stored with an associated timestamp. This is advantageous for auditing and regulatory compliance by the clinician or third-parties.

[0190] In an example, the system 1000 produced the following veterinary medical report using the device 100 outputs for a healthy patient with a normal sinus arrhythmia (common in thisbreed). Note that notes from the clinical conversation are integrated into the report alongside the objective measurements from the device 100:Analysis Summary:This 11-year-old neutered male Shih Tzu presents with several borderline and abnormal cardiac parameters warranting further investigation. While the heart rate falls within the normal range, the low heart rate variability (HRV) suggests potential autonomic nervous system imbalance. The accentuated S2 heart sound, prolonged systolic duration, and presence of extra heart sounds raise concerns about possible underlying cardiac abnormalities. Further diagnostics are recommended to determine the cause and severity of these findings.Cardiac Rhythm:Heart Rate: 105.1 BPM (Normal Range: 60-140 BPM) [OK]SDNN: 19.6 ms (Normal Range: 20-400 ms) [I]RMSSD: 10.5 ms (Normal Range: 20-300 ms) [X]Rhythm Regularity Score: 0.965 (Normal Range: >0.85) [OK]Heart Sounds:S1 / S2 Amplitude Ratio: 8.55 (Expected Range at standard apical position: <2) [X] Systolic Duration: 171.3 ms (Expected Range at this HR: 150-220 ms) [OK]Diastolic Duration: 399.8 ms (Expected Range at this HR: 400-500 ms) [OK]Patient Profile:This patient is an 11-year-old neutered male Shih Tzu.Breed-Specific Health Insights: Shih Tzus are predisposed to various cardiac conditions, most notably mitral valve disease (MVD). MVD can lead to murmurs, arrhythmias, and heart failure. They are also prone to brachycephalic airway syndrome, which can impact respiratory and cardiac function.Age-Related Considerations: At 11 years old, this Shih Tzu is considered senior and more likely to develop age-related cardiac changes, including degenerative valve disease and cardiomyopathies.Gender-Specific Trends: Neutered males might be at slightly higher risk for certain hormonal imbalances affecting heart function.Impact of Spay / Neuter Status: Neutering can influence metabolism and potentially increase the risk of obesity, indirectly impacting cardiovascular health.Actionable Insights: Given the breed, age, and clinical findings, prioritizing further cardiac evaluation is recommended.Recommendations:Based on the clinical discussion and findings, the following diagnostics are recommended to further evaluate cardiac health and clarify these findings:1. Echocardiogram: To assess heart chamber size, valve structure and function, and rule out structural heart disease.2. Electrocardiogram (ECG): To evaluate heart rhythm in detail and identify any underlying electrical abnormalities.3. Blood Pressure Measurement: To screen for hypertension, which can be associated with an accentuated S2.4. Thoracic Radiographs: To evaluate for concurrent respiratory conditions, such as brachycephalic airway syndrome, and assess lung size for signs of heart disease.5. Bloodwork (including thyroid testing): To rule out underlying metabolic conditions that might contribute to the observed findings.Detailed Analysis:1. Heart Rate and Rhythm Assessment:The heart rate is within normal range.However, low SDNN and RMSSD values indicate reduced HRV, suggesting decreased parasympathetic nervous system activity and increased cardiac risk.Rhythm regularity score indicates overall regular rhythm but does not rule out subtle irregularities.2. Heart Sound Characteristics and Timing Relationships:The significantly elevated S1 / S2 amplitude ratio (8.55) indicates an accentuated S2, potentially consistent with pulmonary hypertension or mitral valve disease.Prolonged systolic duration may indicate delayed ventricular ejection linked to muscle dysfunction or outflow obstruction.Diastolic duration is within normal range.3. Murmurs and Additional Sounds:Potential extra heart sounds were noted in the clinical conversation, raising concern for arrhythmia or abnormal blood flow. Further investigation is warranted.4. Heart Valve Function Assessment:The accentuated S2 sound raises suspicion of potential mitral valve abnormalities, common in Shih Tzus.Echocardiography is recommended to assess valve structure and function.5. Overall Cardiac Function Assessment:Multiple concerning cardiac findings warrant further investigation. The combination of abnormal HRV, elevated S1 / S2 ratio, prolonged systolic duration, and possible extra sounds supports the need for comprehensive cardiac workup.FIGS. 31 A and 31 B show an average heartbeat plot, segmented signal plot, and spectrogram outputted by the system 1000 as part of the above example medical report.

[0191] In another example, the system 1000 produced the following veterinary medical report using the device 100 outputs for a patient with a murmur. As seen, this combined with the abnormal Cardiac Rhythm and Heart Sounds metrics lead to the specific recommendation for an echocardiogram and the like:Analysis Summary:This 11 -year-old spayed female Whippet presents with borderline cardiac parameters on auscultation. A discussion was had with the client regarding the heart rate, which is normal, and the presence of a minor sinus arrhythmia, which is common and normal for this breed. No murmurs or abnormal additional heart sounds were identified on auscultation.Cardiac Rhythm:Heart Rate: 89.3 BPM (Normal Range: 60-140 BPM) [OK]SDNN: 71.9 ms (Normal Range: 20-400 ms) [I]RMSSD: 105.3 ms (Normal Range: 20-300 ms) [I]Rhythm Regularity Score: 0.894 (Normal Range: >0.85) [I]Heart Sounds:S1 / S2 Amplitude Ratio: 1.6 (Expected Range at standard apical position: <2) [OK] Systolic Duration: 201.6 ms (Expected Range at this HR: 150-220 ms) [OK]Diastolic Duration: 470.5 ms (Expected Range at this HR: 400-500 ms) [OK]Patient Profile:This patient is an 11 -year-old spayed female Whippet.Breed-Specific Health Insights: Whippets typically enjoy good cardiac health, though they can be predisposed to conditions such as dilated cardiomyopathy (DCM). This was also discussed with the client during the consultation.Age-Related Considerations: At 11 years old, the patient is considered senior, and age-related cardiac changes are increasingly likely.Gender-Specific Trends: The potential impact of spaying on hormonal balance and cardiac health was discussed with the client.Impact of Spay / Neuter Status: While spaying reduces reproductive cancer risks, the possible increased risk of certain cardiac conditions was explained during the conversation.Actionable Insights: Given the breed, age, and current clinical findings, no immediate cardiac evaluation is necessary. Further evaluation may be considered based on clinical judgment or client preference.RecommendationsBased on the clinical discussion and findings, no immediate further diagnostics are necessary given that all findings are within acceptable limits for the breed and age, and the minor sinus arrhythmia is considered normal. However, the following tests could be considered if clinical concern arises or if the client prefers additional screening:1. Echocardiogram: To assess cardiac structure and function in more detail.2. Blood Pressure Measurement: To screen for hypertension.3. Bloodwork: CBC, chemistry panel, electrolytes for general health screening.4. Holter Monitor: For continuous heart rhythm monitoring if intermittent arrhythmias are suspected.Detailed Analysis:1. Heart Rate and Rhythm Assessment:A discussion was had with the client regarding the sinus arrhythmia, which is typical for Whippets. The reduced heart rate variability was noted and explained as a potential indicator of autonomic imbalance.2. Heart Sound Characteristics and Timing Relationships:S1 / S2 Amplitude Ratio: At standard left apical (mitral) positioning, S1 is expected to predominate with an S1 / S2 amplitude ratio typically <2. The recorded ratio (1.6) falls within this expected range. Systolic and diastolic durations are appropriate for the heart rate and breed.3. Overall Cardiac Function Assessment:Based on the findings and clinical discussion, no significant concerns were identified. The client was advised that further testing is optional and may be considered based on clinical judgment or preference.FIGS. 32A and 32B show an average heartbeat plot, segmented signal plot, and spectrogram outputted by the system 1000 as part of the above example medical report.

[0192] As seen in the examples of FIGS. 31 A, 31 B, 32A, and 32B, the device 100 is used to extract physiological metrics to later be used in conjunction with the language models for report generation. For these metrics, an initial motion analysis, using the IMU, can be used to select the most stable recording window based on minimal accelerometer variance to reduce motion artifacts. The microphone signals (i.e. , stethoscope signal) undergoes bandpass filtering, wavelet denoising, and normalization to enhance cardiac sound features. A Hilbert and homomorphic envelope is determined to isolate the fundamental heart sound energy. Spike artifacts are suppressed using autocorrelation-guided Schmidt spike removal. Heartbeat cycles are detected by identifying peaks in the processed envelope, from which inter-beat intervals are determined. Multiple heartbeat cycles are temporally segmented, resampled to a uniform length, and averaged point-by-point to generate a representative average heartbeat signal that preserves morphological features of the cardiac cycle.

[0193] To segment the average heartbeat signal into physiologically meaningful phases (S1, systole, S2, and diastole), time-domain and frequency-domain features can be extracted from overlapping windows. These features can undergo dimensionality reduction via, for example, Principal Component Analysis (PCA) and can be clustered using, for example, the Leiden algorithm, which constructs a graph of feature similarities and partitions it into distinct cardiac phases. Post-processing can be used to align clusters with detected S1 and S2 peaks and their temporal boundaries. In some cases, metrics can then be determined including, for example, heart rate (HR), heart rate variability (HRV) metrics, and spectral features including dominant and murmur frequencies. In some cases, diagnostic visualizations can be generated, including the average heartbeat waveform, segmentation plots, and spectrograms. In this way, the system1000 enables automated, non-invasive cardiac assessment suitable for veterinary and human healthcare applications.

[0194] The system 1000 can use an automated cardiology analysis byway of the large language models (LLMs) to generate precise and clinically insightful medical reports from the physiological data captured by the device 100. In a particular case, the system 1000 can obtain physiological patient data determined previously, including, for example, heart rate, heart rate variability metrics (such as SDNN and RMSSD), heart sound amplitude ratios, systolic and diastolic durations, and spectral analysis of heart sounds. In some cases, patient age can be added to the report determined from the patient’s birth date. In some cases, incomplete data can be handled by assigning default values where necessary. In some cases, body temperature can be evaluated against established clinical thresholds. In some cases, data segmentation and analysis can be performed through automated formatting routines to create structured inputs for the large language models. The large language models can employ specific safety settings and system instructions tailored to veterinary or human practice in order to synthesize the medical report. In some cases, the medical report can include a detailed analysis summary, a patient-friendly take-home report, and a comprehensive patient profile (in the case of a veterinary patient, including breed-specific, gender-specific, age-related, sterilization status considerations, and regionally relevant insights). The outputted medical report highlights clinically significant cardiac findings in an easy-to-understand format, which can include actionable recommendations and visual status indicators for immediate clinical interpretation. The present embodiments can integrate securely with cloud services, employing robust credential management and logging configurations optimized for error monitoring, ensuring reliable and efficient generation of veterinary cardiology reports in real-time clinical settings.

[0195] Advantageously, the present embodiments streamline the diagnostic process: sensing, analysis, and documentation become a unified workflow. This not only saves time but also creates a richer medical record (with objective data and even stored audio waveforms or motion traces). In the long term, widespread use of such devices could lead to a wealth of aggregated data that can further improve predictive analytics, enable epidemiological insights (for example, tracking health trends in populations of patients or animals), and support training of the next generation of clinicians by providing them with immediate feedback and high-quality documentation.

[0196] Also advantageously, the present embodiments provide a unique combination of multimodal sensors. The electronic stethoscope of the present embodiments having integration with adual-microphone for noise cancellation, an IMU for motion detection, and a temperature sensor, provides numerous advantages for all-in-one capabilities. Other electronic stethoscopes either focus solely on audio. The unified, non-invasive sensing unit of the present embodiments is particularly advantageous because it spans different sensing modalities - acoustics, kinematics, and thermal; which traditionally are handled by separate instruments (stethoscope, accelerometer, thermometer). The present inventors recognized that fusing these modalities in one device yields a better outcome; each sensor complements the others (for example, motion data helps interpret acoustic data), leading to more robust and informative measurements than any modality alone.

[0197] Also advantageously, the present embodiments provide automated generation of medical reports or similar documentation from sensor data incorporating more data than other approaches. Various digital stethoscopes might record sound or can be used to detect a specific condition, but they generally require the clinician to interpret the output and then manually document it. Separately, there are voice dictation tools that transcribe clinician-patient conversations, but those generally do not interpret raw physiological signals. The present embodiments advantageously use machine learning not only to analyze sensor data, and in particular at least stethoscope data, but to produce a textual narrative of the examination findings. No other approaches take live vital sign and auscultation data as input to a large language model to automatically generate a time-contextual medical note. The present embodiments leverage pattern recognition power of deep learning for signal analysis and the contextual language generation ability of large language models to perform a task (medical documentation) that was previously entirely manual.

[0198] Also advantageously, the present embodiments provide an end-to-end solution from data capture to EMR entry. This contrasts with other approaches that might address pieces of the workflow. For example, a traditional electronic stethoscope addresses capturing sounds, and a separate EMR template might help with documentation, but the clinician acts as the bridge between them. In contrast, in the present embodiments, the sensing device, analytics, and EMR integration work in concert. This holistic approach yields numerous benefits because it not only saves time but also reduces errors (since direct integration avoids transcription mistakes) and ensures that the analysis results are immediately recorded and actionable. By embedding the documentation step into the diagnostic process, the present embodiments provide greater efficiency. Traditionally, even if one had a great digital stethoscope, the doctor would still have to write the report; however, the present embodiments can integrate both seamlessly.

[0199] Also advantageously, the present embodiments can be applied to both human and animal patients. Multi-modal sensing and automated documentation can be implemented in a way that is adaptable to various species and use cases, and can be made configurable for different normal ranges and acoustic profiles. In this way, the present embodiments can be widely applied with minimal changes, which is not a trivial or obvious extension of single-purpose devices.

[0200] Also advantageously, a veterinarian or physician could use the device 100 concurrently with the automated note generation system 1000 as a synergistic combined solution. A person skilled in the art, would not readily conceive of integrating electronic stethoscopes, standalone vital sensors, or medical note software because of the traditionally stand-alone nature of such devices. The inventors have recognized and addressed a unique intersection of needs, accurate multi-modal health sensing and efficient documentation, and created a solution that enhances diagnostic processes and reduces administrative burden in healthcare settings.

[0201] Although the foregoing has been described with reference to certain specific embodiments, various modifications thereto will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the appended claims.

Claims

Claims1. A system for automated medical record generation, the system comprising one or more processors in communication with a data storage, the one or more processors configured to execute:a communications module to receive a bioacoustics signal from a physiological sensing device, the physiological sensing device receiving and subtracting contemporaneous signals from a primary microphone and a secondary microphone to generate the bioacoustics signal, the primary microphone measuring sound through a membrane that is in contact with a subject, the secondary microphone measuring sound at a separate location than the membrane;a machine learning module to use a trained machine learning classifier to determine health-related medical data from the bioacoustics signal, the machine learning classifier trained on a dataset of labeled bioacoustics signals;a document generation module to generate a medical report with a large language machine learning model taking the health-related medical data as input; andan output module to output the medical report.

2. The system of claim 1, wherein the health-related medical data comprises quantitative measures or categorical results.

3. The system of claim 1, wherein input to the large language model comprises a time of capture of the bioacoustics signal and patient context.

4. The system of claim 3, wherein input to the large language model further comprises previously captured bioacoustics signals.

5. The system of claim 1, wherein the bioacoustics signal comprises a pulse rate, a respiratory rate, or both.

6. The system of claim 1, wherein the communications module further receives orientation and movement signals from an inertial measurement unit, the bioacoustics signal weighted based on the received orientation and movement signals.

7. The system of claim 1, wherein the communications module further receives a temperature reading from a non-contact far-infrared temperature sensor associated withthe physiological sensing device, the temperature reading provided as further input to the trained machine learning classifier.

8. The system of claim 1, wherein the communications module further receives captured voice data, and the machine learning module determines a text-to-speech transcription using the captured voice data, the transcription forming a part of the health-related medical data.

9. The system of claim 1 , wherein the membrane conforms to a body surface to form an acoustic seal, an acoustic aperture behind the membrane defines a chamber leading to the primary microphone to attenuate external noise.

10. The system of claim 1, wherein the medical report is formatted as a structured medical report or a note in a subjective, objective, assessment, and plan format.

11. A computer-implemented method for automated medical record generation, the method comprising:receiving a bioacoustics signal from a physiological sensing device, the physiological sensing device receiving and subtracting contemporaneous signals from a primary microphone and a secondary microphone to generate the bioacoustics signal, the primary microphone measuring sound through a membrane that is in contact with a subject, the secondary microphone measuring sound at a separate location than the membrane;using a trained machine learning classifier to determine health-related medical data from the bioacoustics signal, the machine learning classifier trained on a dataset of labeled bioacoustics signals;generating a medical report with a large language machine learning model taking the health-related medical data as input; andoutputting the medical report.

12. The method of claim 11, wherein the health-related medical data comprises quantitative measures or categorical results.

13. The method of claim 11 , wherein input to the large language model comprises a time of capture of the bioacoustics signal and patient context.

14. The method of claim 13, wherein input to the large language model further comprises previously captured bioacoustics signals.

15. The method of claim 11, wherein the bioacoustics signal comprises a pulse rate, a respiratory rate, or both.

16. The method of claim 11, further comprising receiving orientation and movement signals from an inertial measurement unit, the bioacoustics signal weighted based on the received orientation and movement signals.

17. The method of claim 11, further comprising receiving a temperature reading from a noncontact far-infrared temperature sensor associated with the physiological sensing device, the temperature reading provided as further input to the trained machine learning classifier.

18. The method of claim 11, further comprising receiving captured voice data and determining a text-to-speech transcription using the captured voice data, the transcription forming a part of the health-related medical data.

19. The method of claim 11, wherein the membrane conforms to a body surface to form an acoustic seal, an acoustic aperture behind the membrane defines a chamber leading to the primary microphone to attenuate external noise.

20. The method of claim 11, wherein the medical report is formatted as a structured medical report or a note in a subjective, objective, assessment, and plan format.

Images