Electronic stethoscope and diagnostic algorithm

Abstract

Sound generated by internal organs of a body may be used to monitor health conditions associated with, for example, the heart and lungs. The present disclosure includes an electronic stethoscope positionable by a user on a location of the body to record sound from an organ. The disclosure also includes a visual algorithm for guiding the positioning of the electronic stethoscope and an audio diagnostic algorithm for providing analysis of the recorded sound.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 331,861, filed on April 17, 2022, entitled “Electronic Stethoscope with Analysis,” and U.S. Provisional Patent Application No. 63 / 358,086, filed on July 1, 2022, entitled “Electronic Stethoscope and Diagnostic Algorithm,” the contents of each of the above patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to electronic stethoscopes and computer-assisted auscultation, and particularly to electronic stethoscopes that incorporate visual and audio algorithms. Background Art

[0004] Cardiovascular disease (CVD) is the leading cause of death worldwide, causing approximately 17.9 million deaths each year, representing approximately one in every three deaths (World Health Organization, https: / / www.who.int / news-room / fact-sheets / detail / cardiovascular-diseases-(cvds)). CVD includes several diseases that affect the heart or blood vessels. There are significant differences in the risk and outcomes of CVD around the world, with low-income populations being disproportionately affected. Heart disease can be mitigated by changes in diet, environment, and physical activity, as well as through regular monitoring and screening, but there are many barriers to accessing regular cardiac monitoring due to cost, logistics, and lack of medical providers, particularly for populations in lower socioeconomic levels.

[0005] A heart murmur is a sound produced by blood flow in the heart that deviates from normal flow patterns and can be used to diagnose a variety of heart conditions. Several heart conditions involve improper flow of blood through the heart valves, causing turbulent blood flow, which produces audible sounds that can be heard using a stethoscope. Heart murmurs can also occur due to various physiological conditions external to the heart and are not related to heart disease. It is possible to distinguish benign murmurs from murmurs caused by heart valve defects and other heart diseases by auscultation, or listening to heart sounds with a stethoscope, but it usually requires a skilled practitioner; a general practitioner may not be as accurate as a specialist in diagnosing heart conditions by auscultation.

[0006] Respiratory diseases are another leading cause of death worldwide, accounting for more than 3 million deaths each year (World Health Organization, https: / / www.who.int / news-room / fact-sheets / detail / chronic-obstructive-pulmonary-disease-(copd)). These conditions affect the function of the airways and lungs and can be caused by smoking, air pollution, environmental contamination, and bacterial and viral infections. Respiratory diseases have similar disparities to cardiovascular disease, with low-income groups being more affected. Regular medical care, including auscultation with a stethoscope by a qualified clinician, is important for diagnosing respiratory diseases, providing treatment to improve breathing, and preventing deaths.

[0007] Auscultation is the act of listening to sounds from internal organs, usually using a stethoscope, in order to non-invasively examine and diagnose a patient. It is generally an inexpensive and widely used method, but the level of accuracy in diagnosing a particular condition or disease depends largely on the skill level, experience, and expertise of the individual performing the auscultation. This is particularly challenging in some communities where many people lack access to regular medical care or are unable to obtain medical care due to barriers related to cost, transportation, and time. Even for patients who can receive medical care from practitioners, doctor visits may be infrequent, and regular monitoring may not be possible.

[0008] There have been multiple efforts to develop electronic stethoscopes and electronic devices and software programs with the goal of enhancing the ability to hear sounds and using computer algorithms to improve diagnosis (Jani et al., Eur. Heart J. Digital Health, 2021, 2 (3), 456-466, https: / / doi.org / 10.1093 / ehjdh / ztab059). However, many of these products are expensive because the stethoscope contains complex processing hardware, or the accompanying software requires an expensive ongoing subscription. Other products target medical practitioners rather than a wider user group at home or in local non-medical institutions (such as pharmacies or stores), and therefore will not help increase access to auscultation-based screening. In addition, these products do not have a way to reliably help users or patients position the stethoscope in the correct position on their body to listen to heart and lung sounds without the involvement of a medical practitioner. Summary of the invention

[0009] The present disclosure provides an electronic stethoscope and related software algorithms for recording and analyzing heart sounds and lung sounds to provide diagnosis of heart and lung conditions. In some embodiments, the present disclosure provides an electronic stethoscope, which includes a light emitting device that can be observed and analyzed by a computer vision algorithm to guide a user to position the stethoscope in real time at a target position at the front and rear of the user's chest. In some embodiments, the electronic stethoscope also provides a microphone to record the sound at the target position at the front and rear of the user's chest. In some embodiments, the electronic stethoscope provides a membrane covering the concave surface of a cylindrical bell, which is pressed against the skin on the front or rear chest to transmit the sound from the inside of the chest to the microphone. In some embodiments, the electronic stethoscope provides a processor to convert the sound received by the microphone into a digital signal.

[0010] In some embodiments, the present disclosure provides technology for visually assessing the user's posture and the positioning of the electronic stethoscope on the user's posture in real time, and further guiding the user to move the stethoscope to multiple medically relevant target locations on the front and back of the user's chest. In some embodiments, the electronic stethoscope can record sounds at the multiple medically relevant target locations. In some embodiments, the present disclosure provides technology for processing and analyzing the recorded sounds. In some embodiments, the present disclosure provides technology for using computer-based machine learning algorithms and methods to classify heart sounds according to various known heart conditions and classify lung sounds according to various known lung conditions. In some embodiments, the present disclosure also provides technology for computer-based machine learning algorithms and methods to process and analyze records of unknown sounds recorded by the electronic stethoscope, and diagnose possible heart conditions and / or lung conditions.

[0011] In some embodiments, the present disclosure provides techniques for analyzing multiple anonymized sound recordings from a user's heart and / or lungs and diagnosing heart conditions and / or lung conditions. In some embodiments, the present disclosure provides techniques for storing, transmitting, and / or processing recordings of sounds, information related to a user's health status, and diagnosing heart conditions and / or lung conditions.

[0012] The present disclosure also provides embodiments of electronic stethoscopes and components associated with electronic stethoscopes, embodiments of methods of using electronic stethoscopes, embodiments of software algorithms that can process and classify sounds and provide diagnosis of cardiac conditions and / or lung conditions, embodiments of software algorithms that can process live video data to locate positions corresponding to user postures and corresponding to the position of the electronic stethoscope, embodiments of software algorithms that enable interaction with one or more users, embodiments of processing, storing and transmitting data obtained from diagnostic algorithms, and embodiments of using the system in various configurations.

[0013] Advantages of the methods, devices, and systems described in the present disclosure include, for example, improved access to monitoring of cardiac and pulmonary conditions, lower cost and ease of use compared to typical electronic stethoscopes, use of vision algorithms to precisely position the stethoscope to capture sounds from medically relevant target locations, automated diagnosis of common cardiac and pulmonary conditions, and connectivity to physicians, telemedicine providers, and electronic medical records.

[0014] In one aspect, the present embodiment relates to an electronic stethoscope, which includes: a chestpiece, which includes: a bell portion, the bell portion includes a hollow cylinder, a concave surface and a flat surface; and a circular membrane, which is attached to the concave surface of the cylindrical bell portion; a rod portion, the rod portion includes a hollow cylinder attached to the outer surface of the bell portion, wherein the longitudinal axis of the rod portion is oriented perpendicular to the central axis of the bell portion; a microphone compartment, which is arranged in the inner cavity of the rod portion; a microphone, which is arranged in the microphone compartment; a microphone seal, which is circumferentially arranged around the microphone and arranged in the microphone compartment to form a seal between the microphone and the microphone compartment; a light emitting device, which is arranged on the flat surface of the bell portion; an analog-to-digital converter, which is arranged in the inner cavity of the rod portion; a microcontroller, which is arranged in the inner cavity of the rod portion; an electrical circuit, which connects the microphone, the analog-to-digital converter and the microcontroller; and an electrical data connection, which is from the microcontroller.

[0015] In some embodiments, the film encloses a certain volume of air at the concave end of the bell portion. In some embodiments, the film includes a plastic sheet having a thickness of about 0.025 inches. In some embodiments, the film includes a polymer selected from the group consisting of acetate, polyester, polyethylene terephthalate (PET), polyethylene, polypropylene, polystyrene, epoxy fiberglass, resin, thermosetting plastic, polyvinyl chloride (PVC) and polytetrafluoroethylene (PTFE). In some embodiments, the film is attached to the bell portion by an adhesive selected from the group consisting of glue, latex, cyanoacrylate, epoxy resin, hot melt adhesive, rubber glue, polyvinyl acetate, polyvinyl alcohol (PVA), polyvinyl chloride (PVC) and polyvinyl chloride emulsion.

[0016] In some embodiments, the circumference of the membrane is surrounded by a flexible ring to form a rim. In some embodiments, the flexible ring comprises a polymer selected from the group consisting of silicone rubber and polyvinyl chloride (PVC). In some embodiments, the membrane is attached to the bell portion by mechanical clamping.

[0017] In some embodiments, the bell portion includes an inner hole extending between a concave surface of the bell portion and the microphone compartment, wherein the inner hole has a diameter of about 0.25 inches. In some embodiments, the stem portion includes a shoulder region having a diameter that varies along the length of the stem portion to accommodate a microphone housed in the microphone compartment. In some embodiments, the bell portion and the stem portion comprise a plastic material selected from the group consisting of polylactic acid (PLA), acrylonitrile butadiene styrene (ABS), or glycol-modified polyethylene terephthalate (PET-G).

[0018] In some embodiments, the bell portion and the stem portion are formed using at least one of additive manufacturing, molding, casting, and extrusion. In some embodiments, the bell portion has a diameter of approximately 1.625 inches and a height of approximately 1.75 inches. In some embodiments, the concave surface has an inverted tapered profile having a diameter of approximately 1.25 inches and a depth of approximately 0.0625 inches.

[0019] In some embodiments, the electrical connection includes at least one of a wired connection and a wireless connection. In some embodiments, the wired connection includes at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and a fiber optic connection. In some embodiments, the wireless connection includes at least one of a short-range wireless connection, a Bluetooth connection, a Wi-Fi connection, and a cellular connection.

[0020] In some embodiments, the light emitting device comprises a light emitting diode, an incandescent lamp, a fluorescent lamp, a halogen lamp, or a gas discharge lamp. In some embodiments, the light emitting device alternates between an on state and an off state at a controllable repetition frequency between about 2 Hz and 30 Hz. In some embodiments, the light emitting device alternates between an on state and an off state in a controllable pattern. In some embodiments, the light emitting device comprises at least two light emitting components, each light emitting component emitting light of a specific wavelength, and each light emitting component alternates between an on state and an off state in a controllable pattern. In some embodiments, the light emitting device emits light of a wavelength between 400 nm and 1000 nm.

[0021] In some embodiments, electrical wiring connects the microphone to the analog-to-digital converter and the analog-to-digital converter to the microcontroller.

[0022] In some embodiments, the chest piece is attached to at least one of a rigid rod and a flexible strap, wherein the rigid rod can be manually held by a user to position the stethoscope against the user's back, and wherein the flexible strap can be used to attach the stethoscope around the user's torso to position the stethoscope against the front or back of the user.

[0023] In some embodiments, the rigid rod comprises an elongated rod and a handle. In some embodiments, the rigid rod comprises at least one of metal, plastic, or wood. In some embodiments, the flexible band comprises one or more bands and one or more clasp mechanisms. In some embodiments, the one or more bands comprise at least one of woven fabric, knitted fabric, leather, metal chain mail, elastic fabric, nylon, cotton, webbing, polyurethane, polyvinyl chloride, polyethylene, polypropylene, wool, linen, cork, rubber, neoprene, spandex, elastic fiber, rayon, carbon fiber, composite material, silicon, rope, hemp, bamboo, cellulose, or paper.

[0024] In some embodiments, the microphone comprises at least one of a condenser microphone, a dynamic microphone, a ribbon microphone, a carbon microphone, a piezoelectric microphone, and a micro-electromechanical system (MEMS) microphone. In some embodiments, the microphone seal comprises a material disposed circumferentially around an inner wall of an inner cavity of the microphone compartment and disposed in contact with the inner wall of the inner cavity of the microphone compartment, and further disposed in contact with an outer surface of the microphone, such that air does not pass between the inner wall of the inner cavity of the microphone compartment and the outer surface of the microphone.

[0025] In some embodiments, the microphone seal includes at least one of silicone, rubber, adhesive, resin, epoxy, polymer, wax, acrylic or rubber glue, thermoplastic adhesive, hot melt adhesive, polytetrafluoroethylene, polyurethane, polycarbonate, polypropylene, nylon, nitrile, neoprene, polyvinyl chloride, fluorocarbon, and fluorosilicone.

[0026] On the other hand, the present embodiment relates to an electronic stethoscope, which includes: a chestpiece, which includes: a cylindrical bell-shaped portion; a hollow rod portion; a hole, which is arranged in the cylindrical bell-shaped portion and connected to the internal inner cavity of the hollow rod portion; and a circular membrane, which covers one end of the cylindrical bell-shaped portion; a microphone, which is arranged in the rod portion; a light-emitting device, which is arranged outside the bell-shaped portion; and an electronic data connection, which is between the microphone and the electronic device.

[0027] In some embodiments, the electronic stethoscope further comprises a plurality of electrical lines, a microcontroller, and an analog-to-digital converter. In some embodiments, the electrical data connection comprises at least one of a wired connection and a wired connection.

[0028] In another aspect, the present embodiment relates to a method for training a machine learning algorithm to recognize heart sounds, the method comprising: obtaining heart sound recordings corresponding to a known heart condition and corresponding to normal healthy heart function; obtaining non-heart sound recordings; providing the heart sound recordings and the non-heart sound recordings to an audio classifier model; and using the audio classifier model to train the machine learning algorithm to distinguish between heart sound recordings and non-heart sound recordings, wherein the heart sound recordings are obtained from recordings performed by an electronic stethoscope as described in claim 1, by an alternative electronic stethoscope, by an alternative microphone, and by a smart phone microphone, wherein the non-heart sound recordings may include sounds such as speech, vehicle traffic, dog barking, child crying, noise, and music, wherein the audio classifier model includes a pre-trained audio neural network (PANN), a data set of known audio recordings, and information about health conditions and demographics of patients with known heart conditions, and wherein providing the sound recordings to the audio classifier model is performed on a regularly repeated schedule and includes providing new sound recordings and new known health information.

[0029] In another aspect, the present embodiment is directed to a method of training a machine learning algorithm to recognize lung sounds, the method comprising: obtaining heart sound recordings corresponding to a known lung condition and corresponding to normal healthy lung function; obtaining non-lung sound recordings; providing the lung sound recordings and the non-lung sound recordings to an audio classifier model; and using the audio classifier model to train the machine learning algorithm to distinguish between the lung sound recordings and the non-lung sound recordings, wherein the heart sound recordings are obtained from recordings performed by an electronic stethoscope as described in claim 1, by an alternative electronic stethoscope, by an alternative microphone, and by a smartphone microphone, wherein the non-lung sound recordings may include sounds such as speech, vehicle traffic, dog barking, child crying, noise, and music, wherein the audio classifier model comprises a pre-trained audio neural network (PANN), a dataset of known audio recordings, and information about health conditions and demographics of patients with the known lung condition, and wherein providing the sound recordings to the audio classifier model is performed on a regularly repeated schedule and includes providing new sound recordings and new known health information.

[0030] In another aspect, the present embodiment relates to a system for diagnosing heart conditions and / or lung conditions using sound, the system comprising: an electronic stethoscope; an electronic device communicatively coupled to the electronic stethoscope via a data connection; a digital camera communicatively coupled to the electronic device via a data connection; a visual positioning algorithm for identifying the locations of key points on the user's body and identifying a target location for the electronic stethoscope; an audio classification algorithm for classifying and identifying sounds corresponding to heart conditions and / or lung conditions; and a user interface configured to guide the user to position the electronic stethoscope and record heart sounds and / or lung sounds; wherein the electronic stethoscope comprises a bell and a membrane, a light emitting device, a microphone, and an analog-to-digital converter, wherein the electronic device is selected from one of a mobile device or a personal computer and comprises a digital display, a physical user interface, a digital processor, a memory, and a storage device, wherein the digital camera captures a video image that includes at least the user's torso and light from the light emitting device on the electronic stethoscope, and transmits the video image to the electronic device, wherein the data connection comprises a wireless connection or a wired connection, wherein the electronic device is communicatively connected to a server via a network connection.

[0031] On the other hand, the present embodiment relates to a system for recording heart sounds and / or lung sounds, the system comprising: an electronic stethoscope; an electronic device, the electronic device being communicatively coupled to the electronic stethoscope via a data connection; a digital camera, the digital camera being communicatively coupled to the electronic device via a data connection; a visual positioning algorithm, the visual positioning algorithm being used to identify the locations of key points on the user's body and to identify the location of the electronic stethoscope; and a user interface, the user interface being configured to guide the user to position the electronic stethoscope and record sounds; wherein the electronic stethoscope comprises a bell and a membrane, a light emitting device, a microphone and an analog-to-digital converter, wherein the electronic device is selected from one of a mobile device or a personal computer, and comprises a digital display, a physical user interface, a digital processor, a memory and a storage device, wherein the digital camera captures a video image, the video image comprising at least the user's torso and light from the light emitting device on the electronic stethoscope, and transmits the video image to the electronic device, wherein the data connection comprises a wireless connection or a wired connection, wherein the electronic device is communicatively connected to a server via a network connection.

[0032] On the other hand, the present embodiment relates to a method for diagnosing a cardiac condition, the method comprising: providing an electronic stethoscope comprising: a chest piece having a bell-shaped shape covered by a membrane; a light-emitting device on a side opposite to the membrane; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connection; providing an electronic device comprising a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light-emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at a heart valve listening position on the user's chest; recording the duration of heart sounds at each of the heart valve listening positions; providing heart sound records to a trained audio classification algorithm; and outputting, by the trained audio classification algorithm, a classification of each of the heart sound records corresponding to normal heart function or at least one of one or more of a plurality of heart conditions.

[0033] On the other hand, the present embodiment relates to a method for recording heart sounds, the method comprising: providing an electronic stethoscope comprising: a chest piece having a bell-shaped portion covered by a membrane; a light-emitting device disposed on one side of the bell-shaped portion; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connection; providing an electronic device comprising a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light-emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at a heart valve listening position on the user's chest; and recording the duration of the heart sounds at each of the heart valve listening positions.

[0034] On the other hand, an embodiment of the present disclosure relates to a method for diagnosing a lung condition, the method comprising: providing an electronic stethoscope comprising: a chest piece having a bell-shaped portion covered by a membrane; a light emitting device disposed on one side of the bell-shaped portion; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connection; providing an electronic device comprising a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at lung listening positions on the user's chest and back; recording the duration of lung sounds at each of the lung listening positions; providing lung sound records to a trained audio classification algorithm; and outputting, by the trained audio classification algorithm, a classification of each of the lung sound records corresponding to normal lung function or at least one of one or more lung conditions among a plurality of lung conditions.

[0035] On the other hand, the present embodiment relates to a method for recording lung sounds, the method comprising: providing an electronic stethoscope comprising: a chest piece having a bell-shaped object covered by a membrane; a light emitting device on a side opposite to the membrane; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connection; providing an electronic device comprising a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at lung listening positions on the user's chest and back; and recording the duration of the lung sounds at each of the lung listening positions.

[0036] In another aspect, the present embodiment relates to a method for positioning an electronic stethoscope at a target heart valve location and recording heart sounds, the method comprising: (a) acquiring a live video feed from a camera, the live video feed showing the user's head and torso and showing variable switching lights on the electronic stethoscope following a controlled switching program; (b) providing the live video feed to a posture estimation algorithm, the posture estimation algorithm outputting key points of the user's posture, the key points including the position of at least the user's left shoulder and right shoulder; (c) displaying the live video feed on a display screen, and superimposing the key points on the display screen; (d) providing the live video feed and the key points to a heart valve position estimation algorithm, which outputs a plurality of heart valve listening positions; (e) displaying the live video feed on a display screen and superimposing the key points and the plurality of heart valve listening positions on the corresponding positions on the live video feed displayed on the screen; (f) providing the live video feed, the key points, the plurality of heart valve listening positions, and a controlled program of a variable switchable light-emitting device to a stethoscope position estimation algorithm, which outputs a plurality of heart valve listening positions relative to the stethoscope position estimation algorithm. (g) providing the live video feed, the key points, the plurality of heart valve listening positions, the controlled program of the variable switching light-emitting device, and the stethoscope position to the guided stethoscope positioning and sound recording algorithm; (h) determining whether the electronic stethoscope is correctly positioned at the first heart valve listening position among the plurality of heart valve listening positions by the guided stethoscope positioning and sound recording algorithm; (i) sending a first signal indicating incorrect stethoscope positioning by the guided stethoscope positioning and sound recording algorithm to assist the user in positioning the electronic stethoscope; and (j) The user repositions the electronic stethoscope; (k) a second signal indicating correct stethoscope positioning is sent through a guided stethoscope positioning and sound recording algorithm; (l) a third signal indicating a pause in movement within a first duration is sent through a guided stethoscope positioning and sound recording algorithm; (m) the sound received by a microphone provided in the electronic stethoscope within a second duration is recorded through a guided stethoscope positioning and sound recording algorithm; (n) steps (g) to (m) are repeated until a sound recording with a specified duration is recorded at each of the multiple heart valve listening positions.

[0037] In some embodiments, the plurality of heart valve listening locations include an aortic valve area, a pulmonary valve area, a tricuspid valve area, and a mitral valve area.

[0038] In some embodiments, the first signal comprises a first colored shape displayed on a video screen of the electronic device at the position of the stethoscope. In some embodiments, the second signal comprises a second colored shape displayed on a video screen of the electronic device at the position of the stethoscope, wherein the color of the second colored shape is different from the color of the first colored shape. In some embodiments, the first signal, the second signal, and the third signal each comprise a visual indication generated by the electronic device. In some embodiments, the first signal, the second signal, and the third signal each comprise an auditory indication generated by the electronic device.

[0039] In another aspect, the present embodiment relates to a method for positioning an electronic stethoscope at a target lung listening position and recording lung sounds, the method comprising: (a) acquiring a live video feed from a camera, the live video feed showing the user's head and torso and showing a variable switching light device on the electronic stethoscope following a controlled switching program; (b) providing the live video to a posture estimation algorithm, the posture estimation algorithm outputting key points of the user's posture, the key points including the position of at least the left and right shoulders of the user; (c) displaying the live video feed on a display screen and superimposing the key points on the (d) providing the live video feed and the key points to a lung listening position estimation algorithm, which outputs a plurality of lung listening positions; (e) displaying the live video feed on the display screen and superimposing the key points and the plurality of lung listening positions on the corresponding positions on the live video feed displayed on the screen; (f) providing the live video feed, the key points, the plurality of lung listening positions, and a controlled program of a variable switchable light emitting device to a stethoscope position estimation algorithm, which outputs a plurality of lung listening positions relative to the key points. (g) providing the live video feed, the key points, the plurality of lung listening positions, the controlled program of the variable switching light device, and the stethoscope position to the guided stethoscope positioning and sound recording algorithm; (h) determining, by the guided stethoscope positioning and sound recording algorithm, whether the electronic stethoscope is correctly positioned at the first lung listening position among the plurality of lung listening positions; (i) sending, by the guided stethoscope positioning and sound recording algorithm, a first signal indicating incorrect stethoscope positioning to assist the user in positioning the electronic stethoscope; and (j) repositioning the electronic stethoscope by the user. stethoscope; (k) sending a second signal indicating correct stethoscope positioning by guiding the stethoscope positioning and sound recording algorithm; (l) sending a third signal indicating pausing movement within a first duration by guiding the stethoscope positioning and sound recording algorithm; (m) recording the sound received by a microphone set in the electronic stethoscope within a second duration by guiding the stethoscope positioning and sound recording algorithm, wherein the second duration; (n) repeating steps (g) to (m) until a sound recording with a specified duration is recorded at each of the multiple lung listening positions.

[0040] In some embodiments, the plurality of lung listening positions include positions near the tip of the lungs, the upper lobe of the lungs, the middle lobe of the lungs, and the lower lobe of the lungs, on the left and right sides of the user's front torso and back torso. In some embodiments, the first signal includes a colored shape displayed on a video screen of the electronic device at the position of the stethoscope. In some embodiments, the second signal includes a colored shape displayed on a video screen of the electronic device at the position of the stethoscope, wherein the color of the colored shape is different from the color of the first colored shape. In some embodiments, the first signal, the second signal, and the third signal each include a visual indication generated by the electronic device. In some embodiments, the first signal, the second signal, and the third signal each include an auditory indication generated by the electronic device.

[0041] On the other hand, the present embodiment relates to a method for positioning a stethoscope at one or more cardiac listening positions, the method comprising: providing a live video feed of a user's torso; determining the one or more cardiac listening positions based on the live video feed by a heart valve position estimation algorithm; displaying the live video feed on a video screen, wherein the one or more cardiac listening positions are superimposed on the live video feed; providing an electronic stethoscope in the field of view of the live video feed, the electronic stethoscope comprising a variable switchable light emitting device; moving the electronic stethoscope within the field of view of the live video feed so that the variable switchable light emitting device is proximate to the one or more cardiac listening positions; rendering the one or more cardiac listening positions superimposed thereon in different colors on the live video feed to indicate that the electronic stethoscope is correctly positioned.

[0042] In some embodiments, determining the one or more cardiac listening locations based on a live video feed by a heart valve position estimation algorithm includes using a posture estimation algorithm that outputs key points of the user's posture, which include the position of at least a left shoulder and a right shoulder of the user.

[0043] In another aspect, the present embodiment relates to a method for diagnosing the health of at least one human organ via a computational framework including manually coded diagnostics and / or machine learning techniques, the method comprising: receiving audio data from at least one of the heart and the lungs, the audio data being received from an electronic stethoscope, processing the audio data via at least one of: a) segmenting the audio data via at least one of frequency, amplitude, and duration; and b) a U-net neural network that receives the audio data as input, preprocesses the audio input into a frequency spectrogram with equivariance in frequency, and outputs a corrected segmentation; evaluating the data via at least one of: c) a statistical diagnostic algorithm using clinical practice guidelines; and d) a machine learning technique comprising a neural network trained via a set of matched pairs, each matched pair comprising at least one data point from the electronic stethoscope and at least one data point from a diagnostic device; and displaying at least one result based on an evaluation of the data set.

[0044] In some embodiments, the displayed results include a diagnostic probability for at least one of the following diagnoses: normal heart, harmless murmur, aortic stenosis, and mitral valve prolapse.

[0045] In some embodiments, the diagnostic equipment includes ultrasound equipment and / or an echocardiogram.

[0046] In some embodiments, the electronic stethoscope includes a piezoelectric contact microphone. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] When with Figure 1 When read together, the present teachings described herein will be more fully understood from the following description of various illustrative embodiments. It should be understood that the drawings described below are for illustration purposes only and are not intended to limit the scope of the present teachings in any way.

[0048] Figure 1 is a graphic representation of the heart valve listening locations.

[0049] Figure 2A is a diagram of the position for auscultation of the anterior chest.

[0050] Figure 2B is a graphic representation of the posterior chest auscultation position.

[0051] Figure 3 is a schematic diagram of a general embodiment of a system of electronic devices, electronic stethoscopes, and associated software and algorithms according to aspects of the present embodiments.

[0052] Figure 4is a schematic diagram of a general embodiment of a system of electronic devices, electronic stethoscopes, and associated software and algorithms according to aspects of the present embodiments.

[0053] Figure 5A is a flow chart describing a method of using an electronic stethoscope and associated software to record chest sounds and obtain a diagnosis in accordance with aspects of the present embodiment.

[0054] Figure 5B is a flow chart describing a method of recording chest sounds using an electronic stethoscope and associated software in accordance with aspects of the present embodiment.

[0055] Fig. 6A is a diagram showing an exploded cross-sectional perspective view of an electronic stethoscope according to aspects of the present embodiment.

[0056] Figure 6B is an illustration of a perspective view showing an assembled electronic stethoscope having a membrane covering a concave surface of a bell portion of a chest piece of the electronic stethoscope according to aspects of the present embodiment.

[0057] Figure 6C is a diagram showing a side view of an electronic stethoscope according to aspects of the present embodiment, with dashed lines indicating internal features.

[0058] Fig.6D is a diagram showing an assembled electronic stethoscope having a light emitting device on a flat surface of a bell portion of a chest piece of the electronic stethoscope according to aspects of the present embodiment.

[0059] Fig. 6E is a diagram showing an electronic stethoscope with a rigid pole and handle attached according to aspects of the present embodiment.

[0060] Fig. 6F is a diagram showing an electronic stethoscope with a flexible band and a clasp attached according to aspects of the present embodiment.

[0061] Figure 6G is a diagram showing an electronic stethoscope with a flexible strap and clasp attached and a rigid pole and handle attached in accordance with aspects of the present embodiment.

[0062] Fig. 7A is a diagram showing a side view of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment.

[0063] Figure 7B is a diagram showing a top view of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment.

[0064] Fig. 8Ais a diagram showing a perspective view of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment.

[0065] Figure 8B is a diagram showing an internal cutaway perspective view of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment.

[0066] Fig. 9 is a photograph of a prototype electronic stethoscope assembly according to aspects of the present embodiment.

[0067] Fig.10 is an illustration of key points located on the user's posture that can be identified using vision algorithms.

[0068] Fig.11A is a diagram of four key points of a typical posture (i.e., right shoulder, left shoulder, right hip, and left hip) and four cardiac listening locations (i.e., atrial region, pulmonary artery region, tricuspid valve region, and mitral valve region). According to aspects of the present embodiment, the four cardiac listening locations can be specified by relative distances from the lines connecting the key points.

[0069] Fig. 11B is a table showing the positioning of the four cardiac listening positions. The vertical positioning can be a percentage of the shoulder to hip distance. The horizontal distance can be a percentage of the shoulder to shoulder distance.

[0070] Fig. 11C is an illustration of example cardiac listening locations associated with heart valves relative to key points on a posture in accordance with aspects of the present embodiment.

[0071] Fig.12 is a flow chart describing a visual localization method that provides a live camera feed to multiple algorithms to localize key points on a user's posture, heart valve locations, and stethoscope locations in accordance with aspects of the present embodiment.

[0072] Fig.13 is a flow chart describing a visual localization method that provides a live camera feed to multiple algorithms to localize key points on a user's posture, lung listening locations, and stethoscope locations in accordance with aspects of the present embodiment.

[0073] Fig.14A is an illustration of an example software interface that guides a user in positioning a stethoscope to measure heart sounds in accordance with aspects of the present embodiment.

[0074] Fig. 14B is an illustration of an example software interface that guides a user in positioning a stethoscope to measure heart sounds in accordance with aspects of the present embodiment.

[0075] Fig. 14Cis an illustration of an example software interface that guides a user in positioning a stethoscope to measure heart sounds in accordance with aspects of the present embodiment.

[0076] Fig.14D is an illustration of an example software interface that guides a user in positioning a stethoscope to measure heart sounds in accordance with aspects of the present embodiment.

[0077] 14E is an illustration of an example software interface that guides a user in positioning a stethoscope to measure heart sounds, in accordance with aspects of the present embodiment.

[0078] Fig.15A is a screenshot of an example software interface showing a digital image of a user having key points at the shoulders located by a visual positioning algorithm (i.e., black circles superimposed on the image) and heart valve locations located by another visual positioning algorithm (i.e., white circles superimposed on the image) in accordance with aspects of the present embodiment.

[0079] Fig. 15B is a screenshot of an example software interface showing a digital image of a user having key points at the shoulders located by a visual positioning algorithm (i.e., black circles superimposed on the image) and heart valve locations located by another visual positioning algorithm (i.e., white circles superimposed on the image) in accordance with aspects of the present embodiment.

[0080] Fig.16 is an example of a software interface output according to this embodiment, showing the user the results and possible heart condition diagnosis.

[0081] Fig.17 is a flow chart describing a method for training an audio classifier model using heart sounds according to aspects of the present embodiment.

[0082] Fig.18 is a flow chart describing a method of training an audio classifier model with lung sounds according to aspects of the present embodiment.

[0083] Fig.19 is a flow chart describing a method for classifying a user's heart sound recordings to diagnose a heart condition using a trained audio classifier model according to aspects of the present embodiment.

[0084] Fig. 20 is a flow chart describing a method for classifying chest sound recordings of a user to diagnose a lung condition using a trained audio classifier model according to aspects of the present embodiment.

[0085] Fig.21A is an illustration of a user using an electronic stethoscope connected to a laptop computer via a wired connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0086] Fig. 21B is an illustration of a user using an electronic stethoscope connected to a laptop computer via a wireless connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0087] Fig. 21C is an illustration of a user measuring sounds on the user's back using an electronic stethoscope and a laptop computer with the help of a second user in accordance with aspects of the present embodiment.

[0088] Fig.21D is an illustration of a user using an electronic stethoscope and a laptop computer and measuring sounds on the user's back by using a rigid pole connected to the stethoscope and a mirror behind the user in accordance with aspects of the present embodiment.

[0089] Fig.21E is an illustration of a user using an electronic stethoscope and a laptop computer and a webcam and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0090] Fig.21F is an illustration of a user using an electronic stethoscope and a laptop computer and a camera and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0091] Fig.22A is an illustration of a user using an electronic stethoscope connected to a mobile device through a wired connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0092] Fig. 22B is an illustration of a user using an electronic stethoscope connected to a mobile device through a wireless connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0093] Fig. 22C is an illustration of a user using an electronic stethoscope and a mobile device mounted on a device support and measuring sounds on the user's back with the assistance of a second user in accordance with aspects of the present embodiment.

[0094] Fig.22D is an illustration of a user using an electronic stethoscope and a mobile device mounted on a device support and measuring sounds on the user's back by using a rigid pole connected to the stethoscope and a mirror behind the user in accordance with aspects of the present embodiments.

[0095] Fig.22E is an illustration of a user using an electronic stethoscope connected to a mobile device mounted on a device support through a wired connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0096] Fig.22Fis an illustration of a user using an electronic stethoscope connected by a wired connection to a mobile device mounted on a device support and the user measuring sounds on the front of the user, the electronic stethoscope being connected by a wired connection to a mobile device mounted on a device support and the mobile device being connected by a wired connection to a processing device.

[0097] Fig.23A is an illustration of a user using an electronic stethoscope connected to an interactive kiosk via a wired connection and measuring sounds on the front of the user in accordance with aspects of the present embodiment.

[0098] Fig. 23B is an illustration of a user using an electronic stethoscope connected to an interactive kiosk via a wireless connection and measuring sounds on the user's front in accordance with aspects of the present embodiment.

[0099] Fig.23C is an illustration of a user using an electronic stethoscope and an interactive kiosk to measure sounds on the user's back with the help of a second user in accordance with aspects of the present embodiments.

[0100] Fig.23D is an illustration of a user using an electronic stethoscope and an interactive kiosk to measure sounds on the user's back using a rigid rod and a mirror in accordance with aspects of the present embodiment.

[0101] Fig.24 is an illustration of a user using an electronic stethoscope having three connected bells and a mobile device in accordance with aspects of the present embodiment.

[0102] Fig.25A is an illustration of an electronic stethoscope having two connected bells in accordance with aspects of the present embodiment.

[0103] Fig.25B is an illustration of an electronic stethoscope having three connected bells in accordance with aspects of the present embodiment.

[0104] Fig.26 is a flow chart describing the relationship between various aspects of hardware, software, inputs, outputs, and signals of an electronic stethoscope measurement and analysis system according to aspects of the present embodiment.

[0105] Fig.27A is a diagram showing various aspects of the present embodiment. Fig.26 A flowchart of the hardware details is shown in the flowchart.

[0106] Fig.27B is a diagram showing various aspects of the present embodiment. Fig.26 The flowchart has the details of the digital input of the flowchart.

[0107] Fig.27Cis a diagram showing various aspects of the present embodiment. Fig.26 A flowchart of the details of a set of inputs via a guided interface.

[0108] Fig.27D is a diagram showing various aspects of the present embodiment. Fig.26 A flowchart of the data analysis and diagnostics as well as a flowchart of the output details.

[0109] Fig.28 A graphical workflow is shown for assisting a user in navigating the process of properly positioning a stethoscope in accordance with aspects of the present embodiment.

[0110] Fig.29A A graphical workflow is shown for helping a user navigate the process of correctly positioning themselves relative to a camera in accordance with aspects of the present embodiment.

[0111] Fig.29B A graphical workflow is shown for helping a user navigate the process of correctly positioning themselves relative to a camera in accordance with aspects of the present embodiment.

[0112] Fig.29C A graphical workflow is shown for helping a user navigate the process of correctly positioning themselves relative to a camera in accordance with aspects of the present embodiment.

[0113] Fig.29D A graphical workflow is shown for helping a user navigate the process of correctly positioning themselves relative to a camera in accordance with aspects of the present embodiment.

[0114] Fig.29E A graphical workflow is shown for helping a user navigate the process of correctly positioning themselves relative to a camera in accordance with aspects of the present embodiment.

[0115] FIG30 illustrates a computational framework according to aspects of the present embodiment.

[0116] Fig.31 A diagnostic ecosystem using an electronic stethoscope according to aspects of the present embodiment is shown.

[0117] definition

[0118] About or approximately: The terms "about" and "approximately" as used with respect to a value refer to a value similar to the quoted value in the context. Generally speaking, those skilled in the art who are familiar with the context will understand the relevant degree of variation included in "about" and "approximately" in this context. For example, in some embodiments, the terms "about" and "approximately" may encompass a range of values ​​within 25%, 20%, 19%, 18%, 17%, 16%, 15%, 14%, 13%, 12%, 11%, 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1% or less of the reference value.

[0119] Auscultation: As used herein, the term "auscultation" refers to listening to internal sounds emitted by the body, typically using a stethoscope or other similar audio tool.

[0120] Electronic health records (EHR) and electronic medical records (EMR): As used herein, the terms "electronic health records (EHR)" and "electronic medical records (EMR)" refer to systems of health information that are stored in a digital format and can be transmitted and accessed over a network or device. EHRs and EMRs may contain medical history, demographics, lab test results, personal statistics, allergies, medication history, etc. Medical providers can use EHRs and EMRs to record and access medical information about patients.

[0121] Keypoints: As used herein, the term "keypoints" (also called "key locations") refers to specific landmarks on a user's face and body that can be identified and used to define the user's pose. Keypoints are used within the PoseNet pose recognition algorithm and include the nose, eyes, ears, shoulders, hips, elbows, hands, knees, and feet. See also poses below.

[0122] Kiosk: As used herein, the term "kiosk" (also referred to as "electronic kiosk" or "interactive kiosk") refers to an enclosure that houses a computer, one or more input devices, and one or more output devices. A kiosk is intended to allow a user to access the various functions implemented by the kiosk without being able to access or change internal features or system functions. An interactive kiosk or electronic kiosk is typically a stand-alone system intended for a specific purpose.

[0123] Posture: As used herein, the term "posture" (also referred to as "body position," "position," or "body configuration") refers to the physical configuration of the human body. Examples of postures include standing, lying, leaning, sitting, squatting, etc. More specifically, within the context of the present disclosure, a user's posture refers to the geometric arrangement of the user's shoulders, head, neck, and hips relative to each other, so that their relative positioning and angles can be used to accurately depict the overall body configuration, and can be used to locate other features within the user's body. Postures can be defined by identifying several specific landmarks or key points at important locations on the user's body. See also key points above.

[0124] Stethoscope: As used herein, the term "stethoscope" refers to a medical device used to listen to sounds from the body. It usually includes a resonator placed on the subject's skin and some mechanism for conducting the sound into the operator's ear or into a device for recording the sound.

[0125] User: As used herein, the term "user" refers to an individual who operates or uses one or more of the embodiments in the present disclosure. In the context of the present embodiment, a user is typically a person who is able to operate and position an electronic stethoscope in contact with the user's body. The user provides input to, interacts with, and receives information from an electronic device. In some embodiments, the user may be a secondary user who assists a primary user in operating one or more of the embodiments in the present disclosure, wherein the primary user is a user whose heart sounds and / or lung sounds are recorded by an electronic stethoscope. DETAILED DESCRIPTION

[0126] It is contemplated that the methods, compositions, and processes of the claimed embodiments encompass variations and modifications developed using information from the embodiments described herein.As contemplated by this specification, modifications and / or adaptations of the methods, compositions, and processes described herein may be performed.

[0127] Throughout the specification, where methods, compositions, and processes are described as having, comprising, or comprising specific components, or where processes and methods are described as having, comprising, or comprising specific steps, it is contemplated that there are additional compositions of the disclosure that consist essentially of, or consist of, the recited components, and that there are processes and methods according to the disclosure that consist essentially of, or consist of, the recited steps.

[0128] It should be understood that the order of steps or the order in which certain actions are performed is not important, as long as the embodiment remains operable. In addition, two or more steps or actions may be performed simultaneously.

[0129] Reference to any publication herein, such as in the Background section, is not an admission that the publication serves as prior art to any claim presented herein. The Background section is presented for clarity purposes and is not meant as a description of prior art with respect to any claim.

[0130] Said document is incorporated herein by reference. If there is any conflict in the meaning of a particular term, the meaning provided in the above definitions section shall control.

[0131] Headings are provided for the convenience of the reader; the presence and / or placement of headings is not intended to limit the scope of the subject matter described herein.

[0132] The present disclosure provides techniques for recording sounds from a user's heart and lungs using an electronic stethoscope at locations on the user's chest that are identified in real time by a computer vision algorithm. Figure 1 Four main cardiac listening locations 10 associated with the heart valves are shown: aortic valve area 12, pulmonary valve area 14, tricuspid valve area 16, and mitral valve area 18. A fifth area, the left sternal margin area just below the pulmonary artery area, may also be used. Figure 2A The ten lung listening locations 20 on the anterior (front) chest and the recommended listening order are shown, and Figure 2B Ten lung listening locations 22 on the posterior (posterior) chest and a recommended listening order are shown. The chest listening locations (20 and 22) may include locations on the left and right sides of the chest corresponding to the tip of the lung just above the clavicle, the upper lobe of the lung, the middle lobe of the lung, and the lower lobe of the lung. The present disclosure also provides techniques for diagnosing heart conditions and lung conditions by auscultation using an electronic stethoscope and analyzing the sound recordings by a machine learning algorithm.

[0133] Systems and methods for implementing electronic stethoscopes and algorithms

[0134] This section describes systems and methods for implementing an electronic stethoscope and at least one algorithm according to aspects of the present embodiment.

[0135] Figure 3Schematic diagram of a general embodiment of a system of an electronic device 50, an electronic stethoscope 38, and associated software and algorithms 64 according to various aspects of the present embodiment. In some embodiments, the system can be used to guide the user to position the electronic stethoscope at a specific target listening position on the user's chest and back. In some embodiments, guiding the positioning of the electronic stethoscope is achieved by a visual positioning algorithm 70 (i.e., a computer vision algorithm) that positions the position of the flashing light 42 on the electronic stethoscope 88 relative to the position of the joints (e.g., shoulders, hips, etc.) on the user's body. In some embodiments, a digital camera 102 is used to capture video for use with the visual positioning algorithm 70, and the electronic device 50 receives a video stream from the digital camera 52 and provides an interface 74 between the user 32 and the algorithm 64. In some embodiments, once the chest piece 40 of the electronic stethoscope 38 is positioned at the target listening position, the microphone 44 in the electronic stethoscope 38 records the sounds from the user's heart 34 and / or lungs 36. In some embodiments, the audio processing algorithm 66 may process the recorded sounds before the audio classification algorithm 68 may analyze the recorded sounds and classify them according to various cardiac conditions or lung conditions.

[0136] Figure 4 is a schematic diagram of another general embodiment of a system of electronic devices, electronic stethoscopes, and associated software and algorithms according to aspects of the present embodiment. In some embodiments, Figure 4 The system depicted in does not include an audio classification algorithm. In some embodiments, Figure 4 The system depicted in includes an electronic stethoscope, an electronic device and a camera, and a visual positioning algorithm. In some embodiments, Figure 4 The system described in will enable a user to accurately position the electronic stethoscope at a location on the user's chest corresponding to each of the four main heart valve listening locations. At each location, a microphone integrated into the electronic stethoscope can record the sounds produced by the user's heart. Figure 4 In some embodiments shown, the sound recording is not analyzed by any audio algorithm, but the sound recording can be saved for later analysis by another system, transmitted to an electronic medical record (EMR), transmitted to a physician's office, output to speakers or headphones, output to a display or sound waveform printed on paper, used during a telemedicine appointment, etc.

[0137] Figure 5A1 is a flow chart describing a method 130 of recording chest sounds and obtaining a diagnosis using an electronic stethoscope and associated software according to aspects of the present embodiment. In some embodiments, in step 132, the user may connect the electronic stethoscope to an electronic device, and then in step 134, the user may initialize a program on the electronic device and set parameters. In some embodiments, in step 136, the user may hold the stethoscope with the membrane side in contact with the bare skin of the user's chest under any clothing, with the light-emitting device on the stethoscope facing outward, and with the user facing a digital camera associated with the electronic device. In step 138, the user then begins the visual guidance and listening program on the electronic device.

[0138] Still reference Figure 5A , steps 140 to 150 may be repeated for all chest positions. In step 140, the program may indicate the chest position. In step 142, the user may move the stethoscope toward the indicated position. In step 144, the program may use the live video feed from the camera and the flashing of the light device to locate the point on the user's body and the stethoscope. In step 146, the user may move the stethoscope until the stethoscope is in the indicated position and the program confirms the correct position. In step 148, the program may instruct the user to remain still and hold his breath for an indicated time period. In step 150, the program may record the sound for another indicated time period using a microphone disposed in the stethoscope.

[0139] Still reference Figure 5A , in step 152, the program can then analyze all sound recordings and can apply algorithms to identify conditions. In step 154, the program displays the waveform of the sound and displays conclusions about the likelihood of various conditions. In step 156, the user instructs the program to save, discard and / or transmit the data and findings.

[0140] Figure 5B 1 is a flow chart describing a method 160 of recording chest sounds using an electronic stethoscope and associated software according to aspects of the present embodiment. In some embodiments, in step 162, the user may connect the electronic stethoscope to the electronic device, and then in step 164, the user may initialize the program on the electronic device and set parameters. In some embodiments, in step 166, the user may hold the stethoscope with the membrane side in contact with the bare skin of the user's chest under any clothing, with the light-emitting device on the stethoscope facing outward, and with the user facing the digital camera associated with the electronic device. In step 168, the user may then start the visual guidance and listening program on the electronic device.

[0141] Still reference Figure 5B, repeating steps 170 to 182 for all chest locations. In step 170, the program may indicate the location on the chest. In step 172, the user may move the stethoscope to the indicated location. In step 174, the program uses the live video feed from the camera and the flashing of the light device to locate the point on the user's body and the stethoscope. In step 176, the user may move the stethoscope until the stethoscope is in the indicated location, and the program may confirm the correct location. In step 178, the program may instruct the user to remain still and hold his breath for the indicated time period. In step 180, the program records sounds using a microphone provided in the stethoscope for the indicated time period. In step 182, the user instructs the program to save, discard, and / or transmit data and findings.

[0142] Hardware Implementation

[0143] The present disclosure provides certain technologies with hardware or physical implementations to perform the functions of the present disclosure. These hardware components mainly include electronic stethoscopes, electronic devices and cameras.

[0144] Electronic stethoscope

[0145] An electronic stethoscope is a stethoscope that has certain electronic components that enhance the functionality of the stethoscope in some way. FIG. 6A to FIG. 6G Various views of certain embodiments of an electronic stethoscope are shown. Fig. 6A 1 is a diagram showing an exploded cross-sectional perspective view of an electronic stethoscope 190 according to aspects of the present embodiment. The main portion of the electronic stethoscope 190 may include a chestpiece 191 , which may include two cylindrical portions: a bell portion 192 and a stem portion 194 .

[0146] The diameter of the bell portion 192 can be about 1.625 inches, or between about 1 inch and about 2 inches. The height of the bell portion 192 can be about 1.75 inches, or between about 1 inch and about 2 inches. The bell portion 192 can have a concave surface 196 at one end and a flat surface 220 at the other end. In some embodiments, the concave surface 196 can have a shallow profile 218 (see Figure 6C ), the shallow profile may have an inverted cone shape or may have an inverted spherical cap shape. In some embodiments, the depth of the concave surface 196 may be about 0.0625 inches, or between about 0.05 inches and 0.1 inches.

[0147] Still reference Fig. 6A, the stem 194 can be connected to the outer peripheral wall of the bell portion 192 so that the longitudinal axis of the bell portion 192 is perpendicular to the central axis of the stem portion 194. In some embodiments, the center of the concave surface 196 can have a circular hole 198 that leads to a cylindrical hole 200 that extends longitudinally through the center of the bell portion 196 and turns vertically to continue toward the circumference of the bell portion 192, where it can extend into the interior of the stem portion 194 to form an internal lumen 202 of the stem portion 194. In some embodiments, the diameter of the cylindrical hole 200 in the bell portion 192 can be approximately 0.25 inches, or between 0.1 inches and 0.5 inches. The size of the internal lumen 202 in the stem portion 194 can be varied to accommodate the size of the microphone 204 and the microphone seal 206 disposed within the internal lumen 202.

[0148] Still reference Fig. 6A , the electronic stethoscope may include a thin circular membrane 214 having a diameter equal to or slightly smaller than the diameter of the bell portion 192. In some embodiments, the membrane 214 may be attached to the circumference of the bell portion 192 around the circumference of the membrane at the concave surface 196 by an adhesive. The membrane 214 may include a plastic sheet (e.g., acetate, polyester, polyethylene terephthalate (PET), polyethylene, polypropylene, polystyrene, epoxy fiberglass, resin, thermosetting plastic, polyvinyl chloride (PVC), or polytetrafluoroethylene (PTFE)). The thickness of the membrane 214 may be about 0.025 inches, or between about 0.01 inches and about 0.05 inches. The adhesive may include a polymer-based glue (e.g., latex, cyanoacrylate, epoxy resin, hot melt glue, rubber glue, polyvinyl acetate, polyvinyl alcohol (PVA), polyvinyl chloride (PVC), or polyvinyl chloride emulsion). In some embodiments, the membrane 214 so attached to the concave surface 196 can enclose a volume of air.

[0149] Figure 6B is an illustration showing a perspective view of the assembled electronic stethoscope 190 , with the membrane 214 covering the concave surface 196 of the bell portion 192 . Figure 6C is an illustration showing a side view of the assembled electronic stethoscope 190 , with dashed lines indicating internal features, including the contour 218 of the concave surface 196 , the central hole 200 , and how the central hole connects to the inner cavity 202 of the stem portion 194 .

[0150] Reference again Fig. 6A, the microphone 204 can be disposed within the inner cavity 202 of the stem portion 194. In some embodiments, the microphone 204 can be at least one of a condenser microphone, a dynamic microphone, an aluminum ribbon microphone, a carbon microphone, a piezoelectric microphone, or a micro-electromechanical system (MEMS) microphone. The microphone 204 can be surrounded by a microphone seal 206 that is disposed circumferentially around the microphone within the inner cavity 202 of the stem portion 194. In some embodiments, the microphone seal 206 can include a flexible and / or compliant material so that it forms a close contact with both the exterior of the microphone 204 and the inner wall of the inner cavity 202, and so that air does not escape between the microphone 204 and the inner wall of the inner cavity 202.

[0151] Reference again Fig. 6A , the electronic stethoscope 190 also includes an analog-to-digital converter (ACD) 208 and a microcontroller 210, both of which are arranged in the inner cavity 202 of the rod portion 194 of the electronic stethoscope 190. The ADC 208 can convert the sound received by the microphone 204 into a digital signal, which can then be transmitted to the microcontroller 210. In some embodiments, the digital signal can be transmitted from the microcontroller 210 to the electronic device via an electronic connection that can be a wired connection or a wireless connection. In some embodiments, the wired connection can include at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and an optical fiber connection. In some embodiments, the wireless connection can include at least one of a short-range wireless connection, a Bluetooth connection, a Wi-Fi connection, and a cellular connection. The microcontroller can also receive instructions from the electronic device to control the light-emitting device by turning the light-emitting device on and off in a specified mode. In some embodiments, the mode of turning the light on and off can be flashing at a frequency between about 2 Hz and about 30 Hz. In some embodiments, the mode of turning the light on and off can be any mode.

[0152] Fig.6D The flat face 220 of the assembled electronic stethoscope 190 is shown. A light emitting device 222 can be disposed in the center of the flat face 220. In some embodiments, the light emitting device 222 can be turned on and off in a specific pattern (e.g., blinking or flashing, or another arbitrary pattern, etc.) that is intended to be observed by a camera for visually locating the location of the bell portion of the stethoscope on the user's body. The light emitting device 222 can include at least one of a light emitting diode, an incandescent lamp, a fluorescent lamp, a halogen lamp, or a gas discharge lamp. A preferred embodiment of the light emitting device can be a light emitting diode.

[0153] Still reference Fig.6DIn some embodiments, the membrane 214 attached to the bell portion 192 can be placed directly against the user's chest, under the user's clothing, to achieve better sound transmission, with the light emitting device 222 facing away from the user so that the light emitted from the light emitting device 222 passes through the user's clothing and can be visible to an observer or camera. The user's clothing may be designed to be thin enough to allow the light from the light emitting device to pass through. The wavelength of the emitted light can be between about 400nm and about 1000nm, which corresponds to visible light and near infrared (nIR) light. This wavelength range is selected to be visible to the user and the camera, and to be able to pass through relatively thin fabrics or textiles used to make clothing. The user can also use the electronic stethoscope without wearing clothing. In some embodiments, the brightness of the emitted light is bright enough to pass through a single layer of fabric typically used for clothing, and is dim enough to not cause any discomfort when viewed by a typical user without any intermediate fabric.

[0154] FIG. 6E to FIG. 6G An additional embodiment of an electronic stethoscope having a rigid rod and a flexible band is shown. The rigid rod and the flexible band can assist the user in moving and positioning the electronic stethoscope. Fig. 6E In the embodiment of the present invention, a rigid rod 224 having an optional handle portion 228 attached to an elongated rod portion 226 is attached to extend from the circumference of the bell portion 192. In some embodiments, the rigid rod 224 can include at least one of a metal, plastic, or wood material. In some embodiments, the rod 226 and handle 228 can be held in the hand of a user to help position the stethoscope on a user's body that may be difficult to reach (e.g., on the user's back, etc.).

[0155] exist Fig. 6F In some embodiments, a flexible band 230 having one or more straps is attached to the bell portion 196. In some embodiments, the flexible band may include one or more straps and one or more clasping mechanisms 232 to keep the band closed around the user's torso so that the stethoscope remains in proper position without shifting and does not require the user's hand to continuously hold it in proper position. In some embodiments, the band may be elastic to provide a stretch function, which can contribute to the comfortable and secure positioning of the stethoscope. In some embodiments, the flexible band may include at least one of woven fabric, knitted fabric, leather, metal chain mail, elastic fabric, nylon, cotton, webbing, polyurethane, polyvinyl chloride, polyethylene, polypropylene, wool, linen, cork, rubber, neoprene, spandex, elastic fiber, rayon, carbon fiber, composite material, silicon, rope, hemp, bamboo, cellulose, or paper.

[0156] exist Figure 6G, both the rigid rod 224 and the flexible band 230 are attached to the bell portion 196. In some embodiments, the flexible band 230 can hold the electronic stethoscope around the user's torso, while the rigid rod 224 can be used to reposition or move the electronic stethoscope 190. That is, the flexible band 230 and the rigid rod 224 can be used in conjunction to assist in positioning the electronic stethoscope 190.

[0157] Fig. 7A 1 is a diagram showing a side view of a chest piece 191 (ie, including a bell portion 192 and a stem portion 194 ) of an electronic stethoscope according to aspects of the present embodiment. Fig. 7A A side view of a chest piece 191 according to aspects of the present embodiment is shown with an optional shoulder portion 242 between the area of ​​the stem portion 194 proximate the bell portion 192 and the microphone compartment 240. The shoulder portion 242 can be adjusted to accommodate different sized microphones and microphone compartments 240 depending on the selection of microphones 204. In some embodiments, the chest piece 191 including the bell portion 192 and the stem portion 194 can be manufactured as a single continuous piece using at least one of additive manufacturing, molding, casting, or extrusion. Figure 7B 1 is a diagram showing a top view of a bell portion 192 and a stem portion 194 of an electronic stethoscope according to aspects of the present embodiment.

[0158] Fig. 8A is a diagram showing a perspective view of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment. Fig. 8A , the concave surface 196 of the bell portion 192 and the opening 198 of the cylindrical bore 200, the stem portion 194, the optional shoulder portion 242, the microphone compartment 240 and the interior cavity 202 of the stem portion are visible. Figure 8B 1 is a diagram showing a perspective view of the interior of a bell portion and a stem portion of an electronic stethoscope according to aspects of the present embodiment. Figure 8B 19, an upper portion of the chest piece 191 is cut away to reveal the point at which the hole 200 turns vertically and extends transversely along the longitudinal axis of the rod portion 202.

[0159] Fig. 9is a photograph of a prototype electronic stethoscope assembly 250. In this prototype, the bell portion 192, the stem portion 194, the shoulder portion 242, and the microphone compartment 240 can be one continuous piece that can be made of a polymer material (e.g., polylactic acid (PLA), acrylonitrile butadiene styrene (ABS), or ethylene glycol-modified polyethylene terephthalate (PET-G), etc.) using additive manufacturing. The microphone 204 can be disposed within the microphone compartment 240 and can be surrounded by a microphone seal 206. In this prototype example, the microphone is a small microphone having a cylindrical outer shape (e.g., a lavalier microphone, a wearable microphone, a portable microphone, etc.). The microphone seal 206 surrounding the microphone 204 can include a flexible polymer material (e.g., a flexible tube or the like). Both the ADC and the microcontroller are housed within the microphone compartment 240 and are not visible in the photograph. In this case, the electronic connection is a wired connection including a universal serial bus (USB) cable 242 and a USB connector 254. The USB connection provides the data connection between the microcontroller and the electronics and also provides power to the microcontroller, ADC and microphone.A membrane 214 made of a 0.025 inch thick circle of plastic sheet is adhered to the outer edge circumference of the concave side of the bell portion 192 using glue around the circumference of the membrane.

[0160] Electronic devices and cameras

[0161] The technology described in the present disclosure includes the use of an electronic device, which can be a personal computer (e.g., a laptop computer, a notebook computer, a desktop computer, etc.) or a handheld device (e.g., a smart phone, a tablet computer, a mobile device, etc.). In some embodiments, the electronic device is used to receive an audio recording from a microphone in an electronic stethoscope and receive a video input from a digital camera. In some embodiments, the audio recording can be processed and electronically transmitted to another computing device, such as another electronic device, a server, or and / or a network. In some embodiments, the machine learning algorithm for audio classification can be operated on an external server, or it can be operated on a local electronic device. In some embodiments, the machine learning algorithm for visual recognition of positioning in a live video feed can be operated on an external server, or it can be operated on a local electronic device.

[0162] In some embodiments, the electronic device may also include a visual display to show various signals, instructions, or information to the user, including instructions to guide the user to properly position the electronic stethoscope at a target location to record sounds associated with different parts of the heart valves and lungs. In some embodiments, the visual display displays instructions to the user to guide the user's movements.

[0163] In some embodiments, the visual information provided to the user may include listening for sounds from the user's body and providing the results of the diagnosis. In some embodiments, the visual information may include instructions related to electronic storage and / or transmission of the results and diagnosis.

[0164] In some embodiments, the electronic device may include a device for a user to provide input or instructions to the electronic device. Examples of devices providing input may include a keyboard, a touch screen, a mouse, a touch pad, a microphone, a stylus, a button, a knob, a switch, a dial, a slider, a joystick, and a trackball.

[0165] In some embodiments, the electronic device may include a network connection to communicate with external computing devices and systems. In some embodiments, the network connection may include an Ethernet connection, a Wi-Fi connection, and / or a cellular data connection. In some embodiments, the electronic device may use the network connection to transmit information about the sound recording and diagnosis to a physician, clinic, electronic record management system, or other entity.

[0166] In the present disclosure, a digital camera is used to provide a live video feed to an electronic device. In some embodiments, the digital camera can be integrated as part of the electronic device, such as a built-in camera on a smart phone, tablet computer, laptop, or kiosk, while in other embodiments, the digital camera can be a separate camera electrically connected to the electronic device.

[0167] FIG. 21A to FIG. 21F A diagram showing a user using an electronic stethoscope and a laptop computer in accordance with aspects of the present embodiment is shown. FIG. 22A to FIG. 22F A diagram showing a user using an electronic stethoscope and a mobile device according to aspects of the present embodiment is shown. FIG. 23A to FIG. 23D A diagram showing a user using an electronic stethoscope and a kiosk in accordance with aspects of the present embodiment is shown.

[0168] In some embodiments, the digital camera may include various optical components (e.g., lenses, apertures, light sensors, etc.) and may be connected to the electronic device via an electronic data connection (e.g., via a wired connection or a wireless connection). In some embodiments, the digital camera may acquire live video images at a frame rate sufficient to capture the continuous movement of the user.

[0169] Variations of usage

[0170] The present disclosure provides techniques that can be implemented in various forms, for example, by using different types of similar or equivalent hardware, changing certain geometries, or using different materials.

[0171] exist FIG. 21A to FIG. 21FIn each of the illustrated embodiments, the laptop computer 426 may include an integrated camera 430, a user input method 431 (e.g., a keyboard, a touch pad, touch points, buttons, etc.), and a display screen 428. Fig.21A In the embodiment, the connection between the electronic stethoscope and the electronic device is a wired connection 424. Fig. 21B In the embodiment, the connection between the electronic stethoscope and the electronic device is a wireless connection 432. Fig.21A and Fig. 21B In both, the user uses an electronic stethoscope to measure the sound on the user's front. FIG. 21C to FIG. 21F In the present invention, the connection between the electronic stethoscope and the electronic device can be a wired connection or a wireless connection.

[0172] exist Fig. 21C , according to aspects of the present embodiment, a second user 434 is assisting in measuring sound on the first user's back 420. In some embodiments, the first user 420 has his back to the camera and electronic device (which is shown as a laptop computer 426 in this illustration, but can be another electronic device in other embodiments), while the second user 434 moves the electronic stethoscope 422 and observes instructions and feedback provided on the display screen 428 of the electronic device 426.

[0173] exist Fig.21D In the embodiment, according to aspects of the present embodiment, a user 420 uses an electronic stethoscope 422 to take sound measurements of the user's back. In some embodiments, the electronic stethoscope 422 can be connected to a rigid pole 436 that the user can hold in order to move the electronic stethoscope 422. In some embodiments, there can be a mirror 438 behind the user to reflect the positioning of the user's back and the positioning of the electronic stethoscope 422 to be captured by a camera 430 associated with the electronic device 426.

[0174] exist Fig.21E In accordance with aspects of the present embodiment, a user 420 uses an electronic stethoscope 422 and a laptop computer 426 and a separate webcam 440 to measure sounds on the front of the user. In some embodiments, the webcam 440 is mounted in contact with the laptop computer 426, for example, at the top edge of the laptop computer's display screen. In some embodiments, the webcam 440 can be mounted or supported elsewhere, but in close proximity to the user 420 and the laptop computer 426 so that the webcam 440 can capture a clear view of the front of the user. In some embodiments, the webcam 440 can communicate with the laptop computer 426 via a wired connection or a wireless connection (not visible in this illustration). Although in Fig.21E A laptop computer 426 is shown in the illustrated embodiment, but in some embodiments another electronic device capable of performing similar functions may be used instead.

[0175] exist Fig.21F In accordance with aspects of the present embodiment, a user uses an electronic stethoscope 422 and a laptop computer 426 and a separate camera 442 to measure sounds on the front of the user. In some embodiments, the camera 442 can be mounted on a support, bracket, tripod, monopod, and / or clamp. In some embodiments, the camera 442 can be a digital camera, which can include a digital single-lens reflex (DSLR) camera, a digital mirrorless camera, a digital point-and-shoot camera (i.e., a digital compact camera), or a digital bridge camera. In some embodiments, the digital camera 442 can communicate with the laptop computer 426 via a connection 444, which can be a wired connection or a wireless connection. Although in Fig.21F A laptop computer 426 is shown in the illustrated embodiment, but in some embodiments another electronic device capable of performing similar functions may be used instead.

[0176] exist FIG. 22A to FIG. 22F In, according to various aspects of the present embodiment, it is shown that a user 420 uses an electronic stethoscope 422 connected to a mobile device by a wired connection or a wireless connection and measures the sound on the front of the user. In each of these figures, the mobile device can be one of a smart phone, a tablet computer or a handheld computing device. The components of the mobile device suitable for use in the present embodiment include one or more digital processors, digital memories, digital storage devices, one or more display screens, one or more built-in digital cameras, wirelessly (e.g., Wi-Fi, Bluetooth, etc.) or by a wired connection (e.g., USB, Ethernet, etc.) One or more devices for transmitting and receiving information and one or more devices (e.g., touch screen, button, switch, microphone, etc.) that interact with the user. If the digital camera is not built-in, a separate digital camera that communicates electronically with the mobile device can be used instead. In some embodiments, a software program can display visual indicators (e.g., symbols, outlines, lines, marks, etc.) on a display screen to guide the user to position the user's torso in the camera view in a manner that a visual positioning algorithm (e.g., a posture estimation algorithm, a stethoscope positioning algorithm, etc.) can operate correctly.

[0177] exist Fig.22A In accordance with aspects of the present embodiment, the electronic stethoscope 422 is connected to the mobile device 446 using a wired connection 452 while the user 420 performs sound measurements in front of the user facing the built-in camera 450 and display screen 448. Fig. 22B In accordance with aspects of the present embodiment, the electronic stethoscope 422 is connected to the mobile device 446 using the wireless connection 454 while the user is taking sound measurements in front of the user.

[0178] exist FIG. 22C to FIG. 22FIn accordance with aspects of this embodiment, an electronic stethoscope 422 is used with a mobile device 446 mounted on a device support 458 to measure the back or front sounds of a user 420. FIG. 22C to FIG. 22F In each of these images in , the device support 458 allows the mobile device 446 to be held in a stationary, stable position while capturing a live video feed of the user 420 and the electronic stethoscope 422 via a digital camera, and displaying the live video feed along with an overlay indicator generated by a computer vision algorithm. In some embodiments, the stable and stationary positioning of both the electronic device 446 and the digital camera improves the operation of the computer vision algorithm. In some embodiments, the device support 458 may refer to any structure that can support a device or electronic device in a specific and adjustable positioning or configuration and provide stability. Examples of device supports may include tripods, monopods, selfie sticks, stands, handles, holders, clamps, and protective cases with built-in supports. Specific implementations of the device support may be self-supporting, or may be partially or fully handheld. The device support may also be reversibly attached to another structure, for example, by a clamp, hook and loop attachment (Velcro), adhesive, magnets, handles, or straps. In FIG. 22C to FIG. 22D In the embodiment, the connection between the electronic stethoscope 422 and the mobile device 446 can be a wireless connection or a wired connection.

[0179] Fig. 22C is an illustration of a user 420 using an electronic stethoscope 422 and a mobile device 446 mounted on a device support 458 and measuring sounds on the user's back with the assistance of a second user 456 in accordance with aspects of the present embodiment. In some embodiments, the user 420 may have his back to the electronic device 446 while the second user 456 positions and moves the electronic stethoscope 422 and simultaneously watches the display screen 448 on the mobile device 446 for guidance and feedback on where to move the electronic stethoscope 422.

[0180] Fig.22D 4 is an illustration of a user 420 using an electronic stethoscope 422 and a mobile device 446 mounted on a device support 458 and measuring sounds on the user's back by using a rigid rod 436 connected to the stethoscope 420 and a mirror 438 behind the user in accordance with aspects of the present embodiment. In some embodiments, the user 420 holds the rigid rod 436 with his hand. In other embodiments, there may also be a flexible strap attached to the electronic stethoscope 422 to assist in its use by securing the electronic stethoscope around the user's torso, such as Figure 6G An exemplary electronic stethoscope is shown. In some embodiments, the mirror 438 can be placed behind the user so that the user's back and stethoscope 422 are reflected by the mirror 438 to be captured by the camera 450 on the mobile device 458.

[0181] Fig.22E is an illustration of a user 420 using an electronic stethoscope 422 connected to a mobile device 446 mounted on a device support 458 via a wired connection 452 and measuring sounds on the front of the user in accordance with aspects of the present embodiment. In some embodiments, the wired connection 452 can be connected to a port on the mobile device 446 that can transmit electronic data as well as provide power to a microphone, light emitting device, analog-to-digital converter, and microcontroller contained within the electronic stethoscope 422. In some embodiments, the wired connection 452 can include at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and a fiber optic connection.

[0182] Fig.22F It is a diagram of a user 420 using an electronic stethoscope 422 and the user measuring the sound on the front of the user, the electronic stethoscope is connected to a mobile device 446 mounted on a device support 458 via a wired connection 452, and the mobile device is connected to a processing device 460 via a wired connection 461. In some embodiments, the wired connection 452 can be connected to a port on the mobile device 446, which can transmit electronic data and provide power for the microphone, light emitting device, analog-to-digital converter, and microcontroller contained in the electronic stethoscope 422. In some embodiments, the wired connection 452 can include at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and an optical fiber connection. In some embodiments, an external processing device 460 can provide additional processing and analysis including a visual positioning algorithm and an audio classifier algorithm. In some embodiments, the external processing device 460 can be connected via a wired connection 461, which can include at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and an optical fiber connection.

[0183] exist FIG. 23A to FIG. 23D In accordance with aspects of this embodiment, an electronic stethoscope 422 is used with an interactive kiosk 470 to measure the sound of a user's back or front. FIG. 23A to FIG. 23DIn each of these images, the interactive kiosk 470 can include a computing or processing device, one or more input devices, one or more cameras, and one or more output devices in an independent system. In some embodiments, one or more input devices may include a keyboard, a touch screen, a mouse, a touch pad, a microphone, a stylus, a button, a knob, a switch, a dial, a slider, a joystick, and a trackball. In some embodiments, the output device may include a video screen, a liquid crystal display (LCD), a light emitting diode (LED) display, a speaker, a headset, and a braille reader. In some embodiments, the independent system may include a housing and a bracket, and may vary in size and shape. In some embodiments, the camera 474 is positioned near the display screen 472 and integrated into the interactive kiosk 470. In some embodiments, the interactive kiosk 470 may be deployed to non-medical public locations, such as grocery stores, drug stores, convenience stores, restaurants, hotels, community centers, post offices, libraries, and schools. In some embodiments, when the kiosk 470 and electronic stethoscope 422 are deployed in a non-medical public location, a disposable cover or cap can be used to cover the portion of the electronic stethoscope that can directly contact the user's skin on the front or back of the user; a new disposable cover can be used for each user to improve hygiene and cleanliness.

[0184] Fig.23A 4 is an illustration of a user 420 using an electronic stethoscope 422 connected to an interactive kiosk 470 via a wired connection 476 and measuring sounds on the front of the user in accordance with aspects of the present embodiment. In some embodiments, the user 420 may stand in front of the interactive kiosk 470 to face the camera 474 and the display screen 472. In some embodiments, the wired connection 476 may include at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and a fiber optic connection.

[0185] Fig. 23B is an illustration of a user 420 using an electronic stethoscope 422 connected to an interactive kiosk 470 via a wireless connection 478 and measuring sounds on the user's front, in accordance with aspects of the present embodiment. In some embodiments, the wireless connection 478 may include a Bluetooth connection.

[0186] Fig.23Cis an illustration of a user 420 using an electronic stethoscope 422 and an interactive kiosk 470 and measuring sounds on the user's back with the assistance of a second user 480 in accordance with aspects of the present embodiment. In some embodiments, the user 420 may be facing away from the interactive kiosk 470 while the second user 480 positions and moves the electronic stethoscope 422 and simultaneously watches the display screen 472 on the interactive kiosk 470 for guidance and feedback on where to move the electronic stethoscope 422.

[0187] Fig.23D 4 is an illustration of a user 420 using an electronic stethoscope 422 and an interactive kiosk and measuring sounds on the user's back by using a rigid pole 436 connected to the stethoscope and a mirror 438 behind the user in accordance with aspects of the present embodiment. In some embodiments, the user 420 holds the rigid pole 436 with his hand. In other embodiments, there may also be a flexible strap attached to the electronic stethoscope to assist in its use by securing the electronic stethoscope around the user's torso, such as Figure 6G In some embodiments, the mirror 438 can be placed behind the user so that the user's back and the stethoscope 422 are reflected by the mirror 438 to be captured by the camera 474 on the interactive kiosk 470.

[0188] Fig.24 4 is an illustration of a user 420 using an electronic stethoscope 482 having three connected bells and a mobile device 472 according to aspects of the present embodiment. In some embodiments, the use of an electronic stethoscope 482 having more than one bell (each bell containing a microphone) can enable further localization of the sound by recording the sound at more than one location. In some embodiments, multiple bells and multiple microphones can form an acoustic array. Fig.25A is an illustration of an electronic stethoscope having two connected bells in accordance with aspects of the present embodiment. Fig.25B is an illustration of an electronic stethoscope having three connected bells in accordance with aspects of the present embodiment.

[0189] Software and algorithms

[0190] Vision algorithms for position and location recognition

[0191] Fig.10 is an illustration of key points located on the user's pose that can be identified using vision algorithms (https: / / www.skypack.dev / view / @tensor flow-models / pose-detection). The indicated key points occur at important joints or landmarks on the user's pose and can include ears, eyes, nose, shoulders, elbows, hands, hips, knees, and feet.

[0192] Fig.11A The smaller image on the left shows Fig.10 , where rectangles indicate primary keypoints associated with the torso (i.e., shoulders and hips). Fig.11A The larger image on the right is an illustration of four key points of a typical posture relative to the torso (i.e., right shoulder, left shoulder, right hip, and left hip) and four cardiac listening locations (i.e., atrial region, pulmonary region, tricuspid region, and mitral region). According to aspects of the present embodiment, the four cardiac listening locations can be specified by relative distances (i.e., double arrows) from a line connecting the key points (i.e., a straight line between the circles representing the key points).

[0193] Fig. 11B is a table showing the positioning of the four cardiac listening positions. In some embodiments, the vertical positioning can be a percentage of the shoulder to hip distance, and the horizontal distance can be a percentage of the shoulder to hip distance. In some embodiments, the range of positioning is related to the variation of the human body and users with varying personal body sizes.

[0194] Fig. 11C is an illustration of example cardiac listening locations associated with heart valves relative to key points on a posture according to aspects of the present embodiment. These example locations have indicated vertical and horizontal distances, which are Fig. 11B In some embodiments, key points located relative to a user's posture by a visual positioning algorithm may provide flexibility of use and may not require prior knowledge of a specific user.

[0195] Fig.122 is a flow chart describing a visual localization method 270 of providing a live camera feed 272 to multiple algorithms to locate key points on a user's posture, heart valve positions, and stethoscope positions in accordance with aspects of the present embodiment. In some embodiments, the live video feed 272 can be obtained from a digital camera showing the user's head and torso and showing flashing lights on an electronic stethoscope. The live video feed can be provided to multiple algorithms, including a posture estimation algorithm 274, a heart valve position estimation algorithm 278, a stethoscope position estimation algorithm 282, and a guided stethoscope positioning and sound recording algorithm 286. In some embodiments, the posture estimation algorithm 274 can output key points 276 of the user's posture, which are then provided to the heart valve position estimation algorithm 278. In some embodiments, the heart valve positions 280 on the user can be identified by the heart valve position algorithm 278, and these heart valve positions are then provided to the stethoscope position estimation algorithm 282. In some embodiments, the position of the stethoscope 284 found by the stethoscope position estimation algorithm 282 is provided to the guided stethoscope positioning and sound recording algorithm 286. In some embodiments, the guided stethoscope positioning and sound recording algorithm 286 can then guide the user to position the stethoscope at a plurality of heart valve listening positions where heart sounds can be recorded.

[0196] Fig.13 2 is a flow chart describing a visual localization method 290 of providing a live camera feed 292 to multiple algorithms to locate key points on a user's posture, lung listening positions, and stethoscope positions according to aspects of the present embodiment. In some embodiments, the live video feed 292 can be obtained from a digital camera showing the user's head and torso and showing flashing lights on an electronic stethoscope. The live video feed 292 can be provided to multiple algorithms, including a posture estimation algorithm 294, a lung listening position estimation algorithm 298, a stethoscope position estimation algorithm 302, and a guided stethoscope positioning and sound recording algorithm 306. In some embodiments, the posture estimation algorithm 294 can output key points 296 of the user's posture, which are then provided to the lung listening position estimation algorithm 298. In some embodiments, lung listening positions 300 on the user can be identified by the lung listening position algorithm 298, and these lung listening positions are then provided to the stethoscope position estimation algorithm 302. In some embodiments, the position of the stethoscope 304 found by the stethoscope position estimation algorithm 302 may be provided to the guided stethoscope positioning and sound recording algorithm 306. In some embodiments, the guided stethoscope positioning and sound recording algorithm 306 may then guide the user to position the stethoscope at a plurality of lung listening positions where lung sounds may be recorded.

[0197] Audio algorithms for sound classification

[0198] A machine learning audio classification algorithm is trained using an audio dataset that includes various heart sounds corresponding to a healthy heart and a heart with one or more known health conditions, as well as various non-heart sounds. The heart sound dataset may include normal heartbeats at different rates from patients of different demographics (e.g., age, gender, activity level, general health level, etc.). The heart sound dataset may also include sounds corresponding to heart conditions, including aortic stenosis, mitral regurgitation, aortic regurgitation, mitral stenosis, patent ductus arteriosus, pulmonary stenosis, tricuspid regurgitation, and hypertrophic obstructive cardiomyopathy. The non-heart sound dataset may include speech, vehicle traffic, dogs barking, children crying, noise, and music.

[0199] Fig.17 3 is a flow chart of a method 320 for training an audio classifier model with heart sounds according to aspects of the present embodiment. In the method 320, several types of audio data sets 322 from different sources may be provided to an audio classifier model 334. In some embodiments, the audio data set source 322 may include an electronic stethoscope heart record 324, a commercial stethoscope heart record 326, a smart phone microphone heart record 328, and a non-heart sound and noise record 330. The heart sound records (324, 326, 328) may include both correct heart sound records and incorrect heart sound records, which correspond to records made by a correctly positioned stethoscope and an incorrectly positioned stethoscope, respectively. Incorrect heart sound records may include records made by stethoscopes placed on different parts of the body or on clothing. Correct heart sound records may also include sounds from patients whose heart conditions have been confirmed by prior art methods (e.g., electrocardiograms).

[0200] Still reference Fig.17 In some embodiments, a training procedure 332 for training an audio classifier model may include feeding new audio data and known data and known health data to an audio classifier model 334 in regular batches every few days. In some embodiments, the audio classifier model may include a pre-trained audio neural network (PANN) 336, a dataset of known sound recordings such as the AudioSet dataset 338, and health status, family history, and demographic information 340. In some embodiments, the training results of the audio classifier may include a training model 342 for classifying whether heart sounds are normal or abnormal, and a training model 344 for suggesting a diagnosis of a heart condition.

[0201] Fig.18350 is a flow chart describing a method 350 of training an audio classifier model 362 with lung sounds according to aspects of the present embodiment. In the method, several types of audio data sets 352 from different sources are provided to the audio classifier model 362. In some embodiments, the audio data set sources 352 may include electronic stethoscope lung recordings 354, commercial stethoscope lung recordings 356, smart phone microphone lung recordings 358, and non-lung sound and noise recordings 360. The lung sound recordings (354, 356, 358) may include both correct lung sound recordings and incorrect lung sound recordings, which correspond to recordings made by a correctly positioned stethoscope and an incorrectly positioned stethoscope, respectively. In some embodiments, a generative adversarial network (GAN) may also be used to train an audio classifier model using sound recordings from incorrect positioning so that the model can learn what sounds may be correct or incorrect.

[0202] Still reference Fig.18 In some embodiments, a training procedure 361 for training an audio classifier model may include feeding new audio data and known data and known health data to an audio classifier model 362 in regular batches every few days. In some embodiments, the audio classifier model 362 may include a pre-trained audio neural network (PANN) 364, a dataset of known sound recordings such as the AudioSet dataset 366, and health status and demographic information 368. In some embodiments, the training results of the audio classifier may be a model 370 for classifying whether lung sounds are normal or abnormal, and a model 372 for suggesting a diagnosis of a lung condition.

[0203] Fig.19 is a flow chart of a method 380 for classifying a user's heart sound recording to diagnose a heart condition using a trained audio classifier model according to the description of various aspects of the present embodiment. In the method 380, an audio input 382 including heart sounds recorded from multiple locations and an additional data input 384 can be provided to a trained audio classifier algorithm 386. In some embodiments, the heart sound recording 382 can include recordings obtained from the aortic valve area, the pulmonary valve area, the tricuspid valve area, the mitral valve area, and optionally the left sternal edge area. In some embodiments, the additional data input 384 can include information provided by the user about the user's own personal user information, including personal identification information, medical history, demographics, height, weight, previous medications, allergies, etc. In some embodiments, the additional data input 384 can assist in the classification of the heart condition, or they can be used for storage and / or transmission of the heart sound recording and diagnosis. In some embodiments, an analysis of the heart sounds is performed and the probability of the heart condition 388 is output, where the classified heart condition with the highest calculated probability is used as the primary prediction (e.g., see Fig.16The probability in the output of ).

[0204] Fig. 20 4 is a flow chart of a method 400 for classifying a chest sound recording of a user to diagnose a lung condition using a trained audio classifier model according to various aspects of the present embodiment. In the method 400, an audio input 402 including lung sounds recorded from multiple locations and an additional data input 404 may be provided to the trained audio classifier algorithm 386. In some embodiments, the lung sound recording 402 may include recordings taken from the user's anterior (front) chest and posterior (posterior) chest. In some embodiments, the additional data input 404 may include information provided by the user about the user's own personal user information, including personal identification information, medical history, demographics, height, weight, previous medications, allergies, etc. In some embodiments, the additional data input 404 may assist in lung condition classification, or they may be used for storage and / or transmission of lung sound recordings and diagnoses. In some embodiments, an analysis of the lung sounds is performed and a probability 408 of the lung condition is output, where the classified lung condition with the highest calculated probability is used as the primary prediction.

[0205] Software interface for user interaction

[0206] FIG. 14A to FIG. 1 4E is a diagram of an example software interface for guiding a user to position a stethoscope to measure heart sounds in accordance with aspects of the present embodiment. The images correspond to a specific implementation of the present disclosure in which a laptop computer with a built-in webcam is used with a browser running on the laptop computer, and in which the laptop computer is connected via an Internet connection to an external server on which a visual localization algorithm and an audio classifier algorithm are running.

[0207] exist Fig.14A In , the user authorizes access by clicking a designated button, thereby allowing the browser to access the built-in webcam. Fig. 14B , a schematic diagram shows a representation of a video image of a user. A red circle is superimposed on the user image to show where to place the stethoscope. In some embodiments, the software interface may also include a visual indication (such as an outline or multiple markers) superimposed on top of the live video image that may be displayed to the user to help guide the user in the position and orientation on the screen for optimal pose estimation by the vision algorithm.

[0208] exist Fig. 14C In , the image shows that the user moves the stethoscope from a starting position indicated by a white circle to a target position indicated by a red circle, and when the stethoscope is successfully placed at the indicated target position, the indicator changes to a green circle. Fig.14D14E , the program indicates that the procedure will be repeated for additional locations (i.e., for additional heart valve listening locations). FIG. 14B to FIG. 14D In some embodiments, the above-mentioned graphical indications (i.e., white circles, red circles, and green circles) can be other shapes and / or colors, but can be visually different from each other so that they can clearly indicate different positioning and / or conditions. In other words, the user moves the stethoscope so that the processor of the computer or electronic device on which the software interface is installed executes programming so that the graphical interface presents changes in the visual indicators to signal the user that the stethoscope is correctly positioned.

[0209] Fig.15A and Fig. 15B is a screenshot of an example software interface showing a digital image of a user with a keypoint at the shoulder located by a visual positioning algorithm (i.e., a dark red circle superimposed on the image) and a heart valve location located by another visual positioning algorithm (i.e., a white circle superimposed on the image) in accordance with aspects of the present embodiment. In some embodiments, the heart valve location can be indicated by markers of different shapes and colors. In some embodiments, the heart valve location can be indicated by a marker that includes letters or symbols representing the location of a specific heart valve (e.g., "A" for the atrial valve, "P" for the pulmonary valve, "T" for the tricuspid valve, and "M" for the mitral valve). Fig.15A and Fig. 15B Screenshots by Follow Fig.12 The general procedure for software generation is described in the detailed description.

[0210] Fig.16 In some embodiments, the output shows basic demographic information of the user, waveforms of sound recordings made at each of the cardiac listening positions. In some embodiments, the output shows possible diagnoses (e.g., different heart murmurs or other conditions), and the calculated probability of the likelihood of each diagnosis. In some embodiments, the output may display corresponding waveforms of healthy heart sounds and other information about the heart as educational information for the user.

[0211] The relationship between hardware, input, output, interface and algorithm

[0212] Fig.26496 is a flowchart describing the relationship between various aspects of the electronic stethoscope measurement and analysis system according to various aspects of the present embodiment, among hardware, software, inputs, outputs, and signals. In some embodiments, the hardware component 490 can provide various signals to the digital input 492 and the input set via the guide interface 494. In some embodiments, the digital input 492 can also provide signals to the input set via the guide interface 494. In some embodiments, the input set via the guide interface 494 can then output data for data analysis and diagnosis 496, which can ultimately generate output 498. FIG. 27A to FIG. 27D This is described in more detail in Fig.26 for further details of the section.

[0213] Fig.27A is a diagram showing various aspects of the present embodiment. Fig.26 490 details of the hardware of the flowchart. In some embodiments, the hardware may include an electronic stethoscope, which may include a chest piece 40 (which includes a bell and a membrane) and a microphone 44. In some embodiments, the electronic equipment may include a microphone 44, an analog-to-digital converter (ADC) 46, a microcontroller 48, and a light or light-emitting device 42 that can flash at a variable frequency or in a controllable pattern. In some embodiments, the microphone 44 can record the sound converted by the ADC 46 into a digital signal, which is then transmitted by the microcontroller 48 to the output signal 500. In some embodiments, the flashing of the light 42 can be controlled by the microcontroller 48, and the flashing program can be transmitted to the input set via the guidance interface 494. In some embodiments, the vision of the flashing light and its position relative to the user's body posture can also be recorded.

[0214] Fig.27B is a diagram showing various aspects of the present embodiment. Fig.26 492 of the flowchart of the details of the digital input 492 of the flowchart of the embodiment of the present invention. In some embodiments, the digital input can include user personal information 502, heart sound input 504 and a live video feed from a digital camera 506 with visual input of the user's body and the placement of the stethoscope. In some embodiments, these digital inputs can then be transmitted to the input collection via the guided interface.

[0215] Fig.27C is a diagram showing various aspects of the present embodiment. Fig.26A flowchart of the details of the input set via the guidance interface 494. In some embodiments, the frequency of the light flashing (or other flashing pattern of the light emitting device) and the live video feed from the camera can be provided to the computer vision guidance interface 508, which can use an algorithm to estimate the user's body landmarks (i.e., key points on the posture) to identify the heart listening position, and can use the flashing lights on the stethoscope to detect the position of the stethoscope relative to the user's body. In some embodiments, the sound recorded from the microphone can be provided to the user interface 510, which interacts with the computer vision guidance interface 508 to guide the user to record heart sounds at 4 or 5 target heart listening positions. In some embodiments, the user interface can use a combination of text, images, instructions, buttons, videos, etc. to provide information to the user and obtain input and instructions from the user. In some embodiments, the heart sounds recorded at the target heart listening position are provided to data analysis and diagnosis and / or output.

[0216] Fig.27D is a diagram showing various aspects of the present embodiment. Fig.26 496 and a flowchart of the details of the output 498. In some embodiments, the audio data can be provided to data processing 512, which can clean the data to remove extraneous noise and enhance the audio data with relevant information (e.g., user personal data, health status, etc.). In some embodiments, the processed audio data can be provided to an audio classification model 514, which can include a pre-trained audio neural network (PANN) or other algorithm. In some embodiments, the output 516 of the heart sound classification can include possible heart conditions with probabilities of occurrence, waveforms of the analyzed heart sound recordings, normal sounds and abnormal sounds, etc. In some embodiments, the output can be stored as anonymized data in a secure database 518, which can also include previous runs or account data, historical information, inputs from users and machine learning, and actual diagnoses from third-party medical practitioners (if selected by the user).

[0217] Still reference Fig.27D, output 498 may include the option of transmitting or saving data and diagnosis and transmitting the results to the user via the user interface. In some embodiments, saving or transmitting data may include transmitting the data to an electronic medical record (EMR) system, transmitting the data to a physician, saving the data to a user account on a local electronic device or on an external server, downloading or printing the data locally, and / or using the aggregated data for further analysis and comparison across multiple users. In some embodiments, the user interface may play audio of the recorded heart sounds, display waveforms of the heart sounds, and display diagnoses and results. In some embodiments, the user interface may also play audio of normal heart sounds for comparison, and may display waveforms of normal heart sounds.

[0218] Fig.28 A graphical workflow 542 is shown for helping a user navigate the process of correctly positioning a stethoscope according to various aspects of the present embodiment. At box / step 520, the workflow 542 includes a webcam that captures frames of the user in real time. In some embodiments, a separate image frame will be used as an input to a heart position identifier and a light tracking algorithm, as further discussed herein. The image can be processed (via Gaussian blur, downsampling, and / or torso masking). The poseNet model can be used to output key points such as right and left shoulders, hips, and eyes. The light tracking algorithm uses multiple frames captured over time to determine the number of frames in the light flash cycle and the position of the light. Enhanced user images (e.g., real-time camera images overlaid with drawings) and audio recordings can be stored for future access or further analysis. While the workflow 542 is occurring, the video feed is continuously fed as input to the screening algorithm (steps 522 to 532). Therefore, both the current heart position and light tracking are updated in real time as the user moves.

[0219] Still reference Fig.28, at step 522, the workflow 542 may include outputting 17 key points via PoseNet. The right shoulder and left shoulder associated with the hip point may be used to calculate the heart registration location. Other points may also be used. At step 524, the workflow 542 may include drawing a circle 534 at the heart location on the user's image frame. At step 526, the workflow 542 may include tracking (via an interface) a flashing light 536 on a stethoscope. The user is prompted to aim the flashing light 536 from the stethoscope at the circle 534. The guidance may include a message prompt, an arrow pointing in the direction of movement, and / or a color change when the user is in the wrong position relative to the correct position. At step 528, the workflow 542 may include moving the stethoscope and the flashing light 436 (i.e., manually by the user) to the heart location 534 along a path 538. The interface tracks the movement of the flashing light 536 on the stethoscope along the path 538. At step 530, the workflow 542 may include first determining that the stethoscope is in the correct position, and then having the user hold the stethoscope still for a certain period of time (e.g., 10 seconds, or 8 to 12 seconds, or 5 to 15 seconds, and / or other time periods) while the stethoscope records the heart sounds heard. In other words, when the user has positioned the stethoscope in the correct position (i.e., the tracked light point 536 is within the error range relative to the heart position point 534), the recording phase will begin. At step 532, the workflow 542 may include repeating steps 522 to 532 for the next (i.e., second heart position) of the APTM frame.

[0220] Still reference Fig.28, in some embodiments, the error range can be between about 1 inch and about 2.5 inches from the true heart location. The error range is visually represented to the user by the size of the drawn circle, which depends on the distance of the user from the camera (the circle is larger when the user is close to the camera; the circle is smaller when the user is far from the camera). Various quality checks can be performed before or during this stage to ensure that the correct screening procedures are followed. These checks can include conductivity testing and / or audio sound classification to verify that the stethoscope is on bare skin (i.e., not on cloth), use of a gyroscope to verify correct orientation, Euclidean distance measurement between the flashing light 536 and the heart point 534, and / or a message or visual representation describing the appropriate screening procedures. According to this embodiment, the first light tracking algorithm can include: 1) determining the number of frames (frequency) between light flashes; 2) testing multiple candidate frequencies; 3) identifying the best frequency based on the frequency that gives the maximum pixel intensity change over multiple frames; 3) determining the best frequency; and 4) calculating a score representing the likelihood that a given pixel is a light. The score can be calculated by subtracting the pixel intensity of the current frame from the previous flash cycle frame. The score incorporates a number of factors, including: a sliding window (e.g., this could be one score every 5 to 10 frames for a 100 to 120 frame batch), so that the score is the average of these individual subtractions in the batch; and an evaluation of the score at other frequencies. If there is also an intensity change in other frequency values, then the pixel is less likely to be a flickering light and the score is decreased. The score also incorporates the second derivative (rate of change). Higher derivative values ​​are more likely to be a light flickering and the score is increased.

[0221] Still reference Fig.28, the second light tracking algorithm may include: 1) using optical flow to track the direction and amplitude of movement between frames; 2) using neural networks to learn light spots in the image; 3) creating an ensemble to combine various algorithmic methods together; 4) using lights of different colors to enhance contrast with more shirt colors and / or improve the light tracking algorithm; 5) using multiple lights arranged in a circular or polygonal (3-4-5-6) shape to improve the light tracking algorithm; and / or 6) modifying the LED lights (flash speed, light intensity, size). According to this embodiment, the audio algorithm may include: 1) a machine learning-based model for classifying the audio recording as on the skin or on the cloth; 2) using ML as a clustering algorithm, a variant of a decision tree, a linear model, and / or a neural network; 3) a machine learning-based model for classifying the cardiac audio recording into various categories, such as but not limited to normal, abnormal, aortic stenosis, mitral regurgitation, aortic regurgitation, mitral stenosis, benign murmur, etc. The input to these machine learning algorithms may include the entire audio or a small segment of audio. The input may also be / or include processed audio sound input (ie, raw sound processed via Fourier transform, frequency filtering, noise reduction and / or compression).

[0222] FIG. 29A to FIG. 29E A graphical workflow 544 is shown for assisting a user in navigating the process of properly positioning themselves relative to the camera in accordance with aspects of the present embodiment. The figures show the camera image that the user will see displayed on the website. Background objects (table, sofa) are placed to represent the user's home environment. A color mask is generated by the system and superimposed on the live video. In one embodiment, as Fig.29A As shown, the workflow 544 may include a first area 546, a second area 548, and a third area 550, the first area corresponding to a first zone in which the user's head is expected to be located, the second area corresponding to a second zone in which the user's torso is expected to be located, and the third area corresponding to a third zone in which the user's lower body is expected to be located. The first area 546, the second area 548, and the third area 550 are oriented vertically, with the first area 546 at the top, the second area 548 in the middle, and the third area 550 at the bottom. Each of the first area 546, the second area 548, and the third area 550 is graphically drawn or superimposed on the live video feed so that the user can see themselves on the screen relative to the three areas 546, 548, 550 in order to correctly position themselves (that is, at the correct distance from the camera). The first area 546, the second area 548, and the third area 550 may include one or more color masks superimposed on top of the camera image to indicate where the user should position themselves in front of the camera.

[0223] Fig.29BA graphical workflow 544 is shown for assisting a user in navigating the process of properly positioning themselves relative to a camera in accordance with aspects of the present embodiment. Fig.29B In an embodiment, the workflow 544 graphically displays an outline 552 of a body including the boundaries of the three zones 546, 548, 550 (i.e., the color mask is shaped like a human figure showing the general head, torso, and hip regions) so that the user can position themselves relative to the three zones 546, 548, 550 and relative to the body outline 552 in order to correctly position themselves (i.e., at the correct distance from the camera).

[0224] Fig.29C A graphical workflow 544 is shown for assisting a user in navigating the process of properly positioning themselves relative to a camera in accordance with aspects of the present embodiment. Fig.29C In the example shown, the user's head is positioned incorrectly such that half of the head is in the first zone 546 and half is in the second zone 548 (i.e., not completely in the first zone 546). Therefore, the graphical workflow 544 displays an indication 554 (e.g., a red "X") indicating that the user is not positioned correctly. Fig.29C In the illustration of , the user is too close to the camera and is instructed to move away from the camera.

[0225] Fig.29D A graphical workflow 544 is shown for assisting a user in navigating the process of properly positioning themselves relative to a camera in accordance with aspects of the present embodiment. Fig.29D In the example shown, the user's head and half of the user's torso are incorrectly positioned in first zone 546, the user's lower body is completely in second zone 548, and no part of the user's body is positioned in third zone 550. Therefore, graphical workflow 544 also displays an indication 554 (e.g., a red "X") indicating that the user is incorrectly positioned. Fig.29D In the illustration of , the user is too far away from the camera and is instructed to move towards the camera.

[0226] Fig.29E A graphical workflow 544 is shown for assisting a user in navigating the process of properly positioning themselves relative to a camera in accordance with aspects of the present embodiment. Fig.29E In the example shown, the user's head is completely within the first zone 546, the user's torso is completely within the second zone 548, and the user's lower body is completely within the third zone 550. Accordingly, the graphical workflow 544 displays an indication 554 (e.g., a green check mark) indicating that the user is correctly positioned and that screening will begin.

[0227] Computational framework

[0228] FIG30 illustrates a computational framework in accordance with various aspects of the present embodiment. The computational framework 556 typically combines two different data sources (recorded data and self-reported information) and follows a two-step process for evaluating a cardiac recording. Step one includes converting the recording into a feature vector having entries indicating the presence or absence of various signs of heart disease that humans can hear in an audio recording, specifically aortic stenosis, mitral regurgitation, tricuspid regurgitation, mitral valve prolapse, ventricular septal defect, aortic regurgitation, mitral stenosis, and / or patent ductus arteriosus, among others. This is immediately interpretable because a medical provider can listen to the recording and confirm, for example, that there is a murmur between S1 and S2 that has characteristics indicative of aortic stenosis. Step two includes combining this feature vector with self-reported symptoms and patient demographics to calculate the likelihood of our target diagnosis.

[0229] Still referring to Figure 30, the present embodiment includes processing and analyzing heart sounds and lung sounds, etc. to make a diagnosis. The dotted lines indicate decision points at which we will evaluate the accuracy of each method and then select a method, the results of which are passed to the next point. The solid line borders on certain boxes indicate actionable outputs. The dotted line borders indicate the algorithm applied to the input to generate the output. Heart sounds and lung sounds are auscultated from standard auscultation points as described in the Bates textbook (Bates' Guide To Physical Examination and History Taking, 13th Edition, Lynn S. Bickley MD FACP, Peter G. Szilagyi MD MPH, Richard M. Hoffman MD MPH FACP, Rainier P. Soriano MD). For heart sounds, the patient holds his breath; if the algorithm detects breath sounds, the patient will be asked to re-record their heart sounds while holding their breath.

[0230] Still referring to Figure 30, the computing framework 556 can use manually coded algorithms and U-Net neural networks to process heart sounds and lung sounds. For example, at 558, the computing framework 556 may include a first manually coded algorithm (i.e., Algorithm 1) for processing heart sounds and a second manually coded algorithm (i.e., Algorithm 1B) at 562 for processing lung sounds. The computing framework 556 can also use a U-net neural network at 560 to process heart sounds, and use a third algorithm (i.e., Algorithm 2) and a fourth algorithm (i.e., Algorithm 3) to process lung sounds. The results from each of these first 4 algorithms (Algorithms 1A, 1B, 2, and 3) can be used as input in both the manually coded diagnostic algorithm at 564 (i.e., Algorithm 4) and the machine learning technique / algorithm at 566 (i.e., Algorithm 5). The various algorithms are further described here.

[0231] Still referring to Figure 30, initial sound processing and table generation can be performed as follows. The feature vector can be calculated in the "Processing" step of Figure 30 according to the following: Steps 558 and 562 describe Algorithms 1A and 1B of the hand-coded algorithm for segmenting audio features. At step 558, heart sound segmentation is feasible once the framework 556 has access to 20 to 30 recordings covering (i.e., to be detected) the disease of interest, which will be used for testing and validation. The manual code 558 uses hand-coded heuristics such as frequency, amplitude, and duration to detect S1, S2, murmurs, and clicks. Note: S1 refers to the "lub" of "lub-dub", which is the sound of the mitral and tricuspid valves closing. S2 refers to the "dub" of "lub-dub", which is the sound of the aortic and pulmonary valves closing.

[0232] Example 1: Pseudocode / Algorithm 1A for processing heart sounds:

[0233] 1) Identify S1 and S2 by sound peaks.

[0234] 2) Distinguish S1 and S2 by calculating the distance (the distance from S1 to S2 should be shorter than the distance from the next S2 to S1)

[0235] 3) Calculate the % probability of a systolic murmur by:

[0236] ax = average sound intensity of S1 and S2 (Hz)

[0237] bY = Average sound intensity at baseline (stethoscope on bare skin only, from a neutral area such as the palm of the hand)

[0238] cz = average sound intensity from S1 to S2

[0239] i. If there is a peak of intensity outlier, remove it from the calculation and count it as a "hit"

[0240] d. Probability of noise % = z / (xy)

[0241] 4) Repeat step 3 for all sounds except the diastolic murmur (step 3c is the average intensity of S2 to S1)

[0242] Still referring to FIG30, embodiments of the present disclosure may also include additional steps and refinements, such as also looking at the temporal quality of the murmurs (whether they are increasing, decreasing, or both) and whether the S1 and S2 sounds are together or separate. S1 refers to the closure of the mitral and tricuspid valves, and S2 refers to the closure of the aortic and pulmonary valves; usually these occur together, but if they are separated, they may indicate a heart valve problem (e.g., right bundle branch block, pulmonary stenosis, atrial septal defect, left bundle branch block, and / or advanced aortic stenosis, as commonly described in the literature).

[0243] Example 2: Lung Sound Segmentation - Algorithm 1B

[0244] For lung sounds, the present embodiment looks for the type of sound and whether these sounds are heard in an abnormal position. The present embodiment also characterizes the rate and rhythm of breathing. These are described in the table below and further characterized below. Lung sounds are classified using if-then statements based on the first three columns of Table 1, and then the location where the sound is heard is recorded. Using heart sounds, instead of recording binary (presence / absence) results, algorithm (1B) records the probability of the presence of the lung sound. In the case of hearing any abnormal sound, the patient is asked to perform some additional lung recordings while performing physical exercise, such as saying "ee" or speaking loudly, which is standard in clinical practice, and the results of these additional tests are also used to record the probability of the presence of goat sounds (egophony), bronchial sounds, and whispering chest sounds. Based on the audio spectrogram, the system also records whether there are occasional lung sounds such as crackles (defined as coarse or fine), wheezing, snoring, stridor, pleural friction sounds, or mediastinal squeeze sounds. Abnormalities in respiratory rate and rhythm can also be recorded.

[0245] Example 3: Using U-Net to segment audio features - Algorithm 2

[0246] This option is feasible once 500 to 2000 recordings are obtained, and requires, for example, a labor-intensive segmentation of each training, testing, and validation recording (i.e., using, for example, Praat (https: / / www.fon.hum.uva.nl / praat / )). In some embodiments, Algorithm 1A may be able to be used to segment most of the training data, and then manually segment any data accordingly (i.e., where it is wrong). In some embodiments, the collected cardiac recordings can be manually segmented to obtain a timeline of when the indication sounds of S1, S2, aortic stenosis, mitral regurgitation, tricuspid regurgitation, mitral valve prolapse, ventricular septal defect, aortic regurgitation, mitral stenosis, and patent ductus arteriosus occur. A U-Net can then be constructed so that it takes the recordings (preprocessed into spectrograms for frequency equivariance) as input and outputs the corrected segmentation (the segmentation is a (time x 11 class) one hot encoded tensor). The U-Net can then be trained via a categorical cross entropy loss. The resulting segmentation is interpretable and should follow a structure that can be inspected. Upon evaluation, if the algorithm indicates that there is aortic stenosis, for example, between S2 and S1, the algorithm (ie, Algorithm 2) may issue a warning that it is malfunctioning.

[0247] Options 1 and 2 have the same output: a timeline indicating when each type of feature appears in each recording. This can be converted to a standard feature vector via hand-coded logic such as "if there is noise between S1 and S2 in the recording obtained from location A (of APTM), then set this feature to true".

[0248] Example 4: Using U-Net to segment audio features - Algorithm 3

[0249] However, Algorithm 3 is substantially similar in approach (i.e., utilizing U-Net to segment audio features) when applied to lung recordings rather than cardiac recordings.

[0250] Example 5: Data collection table

[0251] The goal of the preceding algorithm is ultimately to generate tables for heart and lung sounds that can be used for further diagnostic analysis. These tables are shown below:

[0252]

[0253] Table 1 - Heart sound segmentation output

[0254] Referring to Table 1 above, each cell contains the probability of whether the feature was heard (on a scale from 0.00 to 1.00). In some embodiments, for each recording, the present embodiment includes a total of four copies (one each for the aortic valve region, pulmonary valve region, tricuspid valve region, and mitral valve region).

[0255]

[0256] Table 2 - Lung sound segmentation output

[0257] Referring to Table 2 above, the first and second columns ("Length" and "Time to Peak Flow") are given in seconds. Lung sounds are also given as probabilities of their presence or absence from 0.00 to 1.00. In some embodiments, a separate table may be used for each lung region being auscultated.

[0258] Example 6: Manually coded diagnostic method - Algorithm 4

[0259] Still referring to Figure 30, at step 564, the calculation framework 556 may include a series of if-then statements based on the use of clinical practice guidelines. For example, a systolic murmur in the aortic region usually means aortic stenosis. If the probability of hearing such a murmur is 0.6 (according to Table 1 above), the algorithm starts with this basic probability. If the murmur is heard elsewhere with a lower probability, the algorithm will assume that this is just the reflected sound of the aortic valve and remove these sounds from the data set, or significantly reduce their probability because the chance of having two valve problems at the same time is very small. According to this embodiment, the probability that the test result is true (i.e., the probability from Table 1 and Table 2) can be interpreted in the context of the demographic data and symptom data entered by the patient to generate a positive predictive value and provide a possible diagnosis. For example, if the patient is 70 and the probability of having aortic stenosis increases by 0.2 over 65, and they also have dyspnea, which increases the probability of having aortic stenosis by about 0.3, then the algorithm calculates the "one minus" probability (i.e., 1-the probability that they have this condition). In this example, the odds would become 1-(1-0.6)(1-0.2)(1-0.3)=1-0.224=0.776. According to aspects of the present embodiment, these criteria (ie, weighting factors) can be set based on clinical reasoning and can be updated as new literature and data become available.

[0260] Example 7: Modern Machine Learning Methods - Algorithm 5

[0261] Still referring to Figure 30, in step 566, a large data set of stethoscope readings obtained using the self-guided process of the present embodiment can be collected and paired with the echocardiogram. Using approximately 20,000 such pairs, a pair of networks are trained so that the echocardiogram and stethoscope readings can each be converted into a feature vector of approximately 100 dimensions, where the values ​​of these dimensions are learned from the data rather than manually specified. The training can be accomplished by feeding batches of matched pairs to the two networks, and requiring that the matching vectors have large dot products and the non-matching vectors have small dot products (e.g., according to OpenAI CLIP). The results can be fine-tuned to directly produce values ​​as shown in the final result representation ( Fig.16 ). This more modern semi-supervised approach is used to improve the robustness of the final results. The present embodiment provides an interpretable, accurate and attractive data presentation to patients and physicians.

[0262] Example 8: Longitudinal Monitoring of Recording Heart and Lung Sounds

[0263] According to aspects of the present embodiment, heart and lung sounds can be monitored longitudinally so that changing symptoms and other potentially concerning conditions can be flagged. In some aspects, the present embodiment incorporates an algorithm that automatically monitors changes in heart and lung sounds over time and flags anything associated with the sound. This can be calculated by hard code - each time Table 1 or Table 2 is calculated, any probability that increases by more than 0.1 (10%) is flagged, causing the algorithm to ask the user to record the sound again. If the second recording still reflects an increase of 0.1, the algorithm flags the user to see a doctor. This aspect of the present embodiment helps detect potentially dangerous changes in each patient's baseline; and not all murmurs are harmful, but keeping track of them over time helps identify those that may be harmful early. For example, aortic stenosis often changes over time; facilitating home checks for aortic stenosis can reduce the need for specialized / expensive tests such as echocardiograms for continuous monitoring.

[0264] Fig.31A diagnostic ecosystem 592 using an electronic stethoscope is shown in accordance with aspects of the present embodiment. In some embodiments, the diagnostic ecosystem 592 may include: 1) a stethoscope 568 including: a cable 570 including the ability to be powered at 5V and carry audio data 576 (e.g., via USB-A audio data 578); a housing housing a PCBA 572 for camera light tracking; and a diaphragm 574 that contacts the patient's skin; 2) a user PC, laptop, tablet, personal device 580; and 3) a web-based ecosystem / backend 582 including a service frontend 586, a Korion backend (e.g., including the computing framework 556 of FIG. 30 ), live user video / audio 588, and connected professional medical services.

[0265] The present embodiment describes both software and hardware used in conjunction with each other so that the hardware can be obtained at or about the production cost (approximately $30 in 2023 dollars), which disrupts the market for electronic stethoscopes (which currently sell for approximately $300). Various factors contribute to the low cost of the present stethoscope. The types of microphones used include piezoelectric contact microphones, which are much less expensive than most other electronic stethoscopes that use unidirectional microphones. The use of the piezoelectric contact microphone in the present embodiment is achieved by software-based acoustic processing to amplify the sound and convert it into a form ready for the human ear. By passing the amplification and processing to the software, the stethoscope of the present embodiment bypasses the cost that would be accumulated if an integrated physical amplifier were required to be included.

[0266] The electronic stethoscope and ecosystem of this embodiment include: (a) a guided interface that guides anyone to record their own heart and lung sounds at home; (b) data collected at scale by enabling anyone to collect sounds at home; (c) machine learning-based diagnoses that enable people to get on-demand feedback; and (d) additional educational tools that people can use to understand why they got their results and what to do next.

[0267] Equivalent solutions

[0268] Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation many equivalents to the specific embodiments of the disclosure described herein.The scope of the present disclosure is not intended to be limited to the above description, but rather is set forth in the appended claims.

Claims

1. An electronic stethoscope, comprising: A chest piece, the chest piece comprising: a bell portion, the bell portion comprising a hollow cylinder, a concave surface, and a flat surface; and a circular membrane attached to the concave surface of the cylindrical bell portion; a stem portion comprising a hollow cylinder attached at an outer surface of the bell portion, wherein a longitudinal axis of the stem portion is oriented perpendicular to a central axis of the bell portion; a microphone compartment disposed within the interior cavity of the stem portion; a microphone, the microphone being disposed in the microphone compartment; a microphone seal disposed circumferentially around the microphone and disposed within the microphone compartment to form a seal between the microphone and the microphone compartment; a light emitting device disposed on the flat surface of the bell portion; an analog-to-digital converter disposed within the inner cavity of the rod portion; a microcontroller disposed within the inner cavity of the rod portion; an electrical circuit connecting the microphone, the analog-to-digital converter, and the microcontroller; and An electrical data connection from the microcontroller.

2. The electronic stethoscope of claim 1, wherein the membrane encloses a volume of air at the concave end of the bell portion.

3. The electronic stethoscope of claim 1, wherein the membrane comprises a plastic sheet having a thickness of approximately 0.025 inches.

4. The electronic stethoscope of claim 1, wherein the membrane comprises a polymer selected from the group consisting of acetate, polyester, polyethylene terephthalate (PET), polyethylene, polypropylene, polystyrene, epoxy fiberglass, resin, thermoset plastic, polyvinyl chloride (PVC), and polytetrafluoroethylene (PTFE).

5. The electronic stethoscope of claim 1, wherein the membrane is attached to the bell portion by an adhesive selected from the group consisting of glue, latex, cyanoacrylate, epoxy, hot melt, rubber glue, polyvinyl acetate, polyvinyl alcohol (PVA), polyvinyl chloride (PVC), and polyvinyl chloride emulsion.

6. The electronic stethoscope of claim 1, wherein the circumference of the membrane is surrounded by a flexible ring to form a rim.

7. The electronic stethoscope of claim 6, wherein the flexible ring comprises a polymer selected from the group consisting of silicone rubber and polyvinyl chloride (PVC).

8. The electronic stethoscope of claim 1, wherein the membrane is attached to the bell portion by mechanical clamping.

9. The electronic stethoscope of claim 1, wherein the bell portion includes an inner hole extending between the concave surface of the bell portion and the microphone compartment, wherein the inner hole has a diameter of approximately 0.25 inches.

10. The electronic stethoscope of claim 1, wherein the stem portion includes a shoulder region having a diameter that varies along the length of the stem portion to accommodate a microphone received within the microphone compartment.

11. The electronic stethoscope of claim 1, wherein the bell portion and the stem portion comprise a plastic material selected from the group consisting of polylactic acid (PLA), acrylonitrile butadiene styrene (ABS), or glycol-modified polyethylene terephthalate (PET-G).

12. The electronic stethoscope of claim 1, wherein the bell portion and the stem portion are formed using at least one of additive manufacturing, molding, casting, and extrusion.

13. The electronic stethoscope of claim 1, wherein the bell portion has a diameter of approximately 1.625 inches and a height of approximately 1.75 inches.

14. The electronic stethoscope of claim 1, wherein the concave surface has an inverted tapered profile having a diameter of approximately 1.25 inches and a depth of approximately 0.0625 inches.

15. The electronic stethoscope of claim 1, wherein the electrical connection comprises at least one of a wired connection and a wireless connection.

16. The electronic stethoscope of claim 15, wherein the wired connection comprises at least one of a serial bus connection, a universal serial bus (USB) connection, a parallel bus connection, an audio connection, an Ethernet connection, and a fiber optic connection.

17. The electronic stethoscope of claim 15, wherein the wireless connection comprises at least one of a short-range wireless connection, a Bluetooth connection, a Wi-Fi connection, and a cellular connection.

18. The electronic stethoscope of claim 1, wherein the light emitting device comprises a light emitting diode, an incandescent lamp, a fluorescent lamp, a halogen lamp, or a gas discharge lamp.

19. The electronic stethoscope of claim 1, wherein the light emitting device alternates between an on state and an off state at a controllable repetition rate between about 2 Hz and 30 Hz.

20. The electronic stethoscope of claim 1, wherein the light emitting device alternates between an on state and an off state in a controllable pattern.

21. The electronic stethoscope of claim 1, wherein the light emitting device comprises at least two light emitting components, each light emitting component emitting light of a specific wavelength, and each light emitting component alternates between an on state and an off state in a controllable pattern.

22. The electronic stethoscope of claim 1, wherein the light emitting device emits light having a wavelength between 400 nm and 1000 nm.

23. The electronic stethoscope of claim 1, wherein the electrical circuit connects the microphone to the analog-to-digital converter and connects the analog-to-digital converter to the microcontroller.

24. The electronic stethoscope of claim 1, wherein the chest piece is attached to at least one of a rigid rod and a flexible band, wherein the rigid rod is manually holdable by a user to position the stethoscope against the user's back, and Wherein the flexible strap can be used to attach the stethoscope around the torso of a user to position the stethoscope against the front or back of the user.

25. The electronic stethoscope of claim 24, wherein the rigid rod comprises an elongated shaft and a handle.

26. The electronic stethoscope of claim 24, wherein the rigid rod comprises at least one of metal, plastic, or wood.

27. The electronic stethoscope of claim 24, wherein the flexible strap comprises one or more strap sections and one or more snap-fit ​​mechanisms.

28. The electronic stethoscope of claim 27, wherein the one or more lengths of strap comprise at least one of a woven fabric, a knitted fabric, leather, chain mail, elastic fabric, nylon, cotton, webbing, polyurethane, polyvinyl chloride, polyethylene, polypropylene, wool, linen, cork, rubber, neoprene, spandex, elastane, rayon, carbon fiber, a composite material, silicon, rope, hemp, bamboo, cellulose, or paper.

29. The electronic stethoscope of claim 1, wherein the microphone comprises at least one of a condenser microphone, a dynamic microphone, a ribbon microphone, a carbon microphone, a piezoelectric microphone, and a micro-electromechanical system (MEMS) microphone.

30. An electronic stethoscope as described in claim 1, wherein the microphone seal comprises a material that is circumferentially arranged around the inner wall of the inner cavity of the microphone compartment and is arranged to contact the inner wall of the inner cavity of the microphone compartment, and is further arranged to contact the outer surface of the microphone so that air does not pass between the inner wall of the inner cavity of the microphone compartment and the outer surface of the microphone.

31. The electronic stethoscope of claim 1, wherein the microphone seal comprises at least one of silicone, rubber, adhesive, resin, epoxy, polymer, wax, acrylic or rubber glue, thermoplastic adhesive, hot melt adhesive, polytetrafluoroethylene, polyurethane, polycarbonate, polypropylene, nylon, nitrile, neoprene, polyvinyl chloride, fluorocarbon, and fluorosilicone.

32. An electronic stethoscope comprising: A chestpiece, the chestpiece comprising: a cylindrical bell portion; a hollow rod portion; a hole disposed in the cylindrical bell portion and connected to an inner cavity of the hollow rod portion; and a circular membrane covering one end of the cylindrical bell portion; a microphone disposed within the rod portion; a light emitting device disposed outside the bell portion; and An electronic data connection is between the microphone and an electronic device.

33. The electronic stethoscope of claim 32, wherein the electronic stethoscope further comprises a plurality of electrical circuits, a microcontroller, and an analog-to-digital converter.

34. The electronic stethoscope of claim 32, wherein the electrical data connection comprises at least one of a wired connection and a wired connection.

35. A method of training a machine learning algorithm to recognize heart sounds, comprising: obtaining a recording of heart sounds corresponding to a known heart condition and corresponding to normal healthy heart function; Obtain non-heart sound recordings; providing the heart sound recording and the non-heart sound recording to an audio classifier model; as well as training the machine learning algorithm using the audio classifier model to distinguish between the heart sound recordings and the non-heart sound recordings, wherein the heart sound recording is obtained from recordings performed by the electronic stethoscope of claim 1, by an alternative electronic stethoscope, by an alternative microphone, and by a smartphone microphone, Non-cardiac sound recordings can include sounds such as speech, vehicle traffic, dogs barking, children crying, noise, and music. wherein the audio classifier model comprises a pre-trained audio neural network (PANN), a dataset of known audio recordings, and information about health conditions and demographics of patients with known cardiac conditions, and Wherein providing the sound recordings to the audio classifier model is performed on a regularly repeating schedule and includes providing new sound recordings and new known health information.

36. A method of training a machine learning algorithm to recognize lung sounds, comprising: obtaining a recording of heart sounds corresponding to known lung conditions and corresponding to normal healthy lung function; Obtain non-lung sound recordings; providing the lung sound recording and the non-lung sound recording to an audio classifier model; and training the machine learning algorithm using the audio classifier model to distinguish between the lung sound recordings and the non-lung sound recordings, wherein the heart sound recording is obtained from recordings performed by the electronic stethoscope of claim 1, by an alternative electronic stethoscope, by an alternative microphone, and by a smartphone microphone, Non-lung sound recordings can include sounds such as speech, vehicle traffic, dogs barking, children crying, noise, and music. wherein the audio classifier model comprises a pre-trained audio neural network (PANN), a dataset of known audio recordings, and information about health conditions and demographics of patients with known lung conditions, and Wherein providing the sound recordings to the audio classifier model is performed on a regularly repeating schedule and includes providing new sound recordings and new known health information.

37. A system for diagnosing a cardiac condition and / or a pulmonary condition using sound, comprising: Electronic stethoscope; an electronic device communicatively coupled to the electronic stethoscope via a data connection; a digital camera communicatively coupled to the electronic device via a data connection; A visual positioning algorithm for identifying the positions of key points on the user's body and identifying the position of the electronic stethoscope; an audio classification algorithm for classifying and identifying sounds corresponding to cardiac conditions and / or pulmonary conditions; as well as a user interface configured to guide the user in positioning the electronic stethoscope and recording heart sounds and / or lung sounds; wherein the electronic stethoscope comprises a bell and a membrane, a light emitting device, a microphone and an analog-to-digital converter, wherein the electronic device is selected from one of a mobile device or a personal computer and comprises a digital display, a physical user interface, a digital processor, a memory and a storage device, wherein the digital camera captures a video image containing at least a torso of the user and light from the light emitting device on the electronic stethoscope, and transmits the video image to the electronic device, The data connection includes a wireless connection or a wired connection, Wherein the electronic device is communicatively connected to a server via a network connection.

38. A system for recording heart sounds and / or lung sounds, comprising: Electronic stethoscope; an electronic device communicatively coupled to the electronic stethoscope via a data connection; a digital camera communicatively coupled to the electronic device via a data connection; A visual positioning algorithm for identifying the positions of key points on the user's body and identifying the position of the electronic stethoscope; as well as a user interface configured to guide the user in positioning the electronic stethoscope and recording sounds; wherein the electronic stethoscope comprises a bell and a membrane, a light emitting device, a microphone and an analog-to-digital converter, wherein the electronic device is selected from one of a mobile device or a personal computer and comprises a digital display, a physical user interface, a digital processor, a memory and a storage device, wherein the digital camera captures a video image containing at least a torso of the user and light from the light emitting device on the electronic stethoscope, and transmits the video image to the electronic device, The data connection includes a wireless connection or a wired connection, Wherein the electronic device is communicatively connected to a server via a network connection.

39. A method for diagnosing a cardiac condition, comprising: An electronic stethoscope is provided, comprising: a chest piece having a bell portion covered by a membrane; a light emitting device located on the opposite side of the membrane; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connector; Providing an electronic device including a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at a heart valve listening location on the user's chest; recording the duration of the heart sound at each of the heart valve listening positions; providing the heart sound recording to a trained audio classification algorithm; A classification of each of the heart sound recordings corresponding to at least one of normal heart function or one or more of a plurality of heart conditions is output by the trained audio classification algorithm.

40. A method for recording heart sounds, comprising: An electronic stethoscope is provided, comprising: a chest piece having a bell portion covered by a membrane; a light emitting device disposed on one side of the bell portion; a microphone; an analog-to-digital converter; a microcontroller; and electronic connections; Providing an electronic device including a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; turning the light emitting device on and off in a specified pattern; using a visual positioning algorithm to guide a user to position the electronic stethoscope at a heart valve listening location on the user's chest; and The duration of the heart sound at each of the heart valve listening positions is recorded.

41. A method for diagnosing a lung condition, comprising: An electronic stethoscope is provided, comprising: a chest piece having a bell portion covered by a membrane; a light emitting device disposed on one side of the bell portion; a microphone; an analog-to-digital converter; a microcontroller; and electronic connections; Providing an electronic device including a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; Turns light-emitting devices on and off in a specified pattern; Using a visual positioning algorithm to guide the user to position the electronic stethoscope at lung listening locations on the chest and the back of the user; recording the duration of the lung sounds at each of the lung listening positions; providing the lung sound recordings to a trained audio classification algorithm; A classification of each of the lung sound recordings corresponding to at least one of normal lung function or one or more lung conditions of a plurality of lung conditions is output by the trained audio classification algorithm.

42. A method for recording lung sounds, comprising: An electronic stethoscope is provided, comprising: a chest piece having a bell portion covered by a membrane; a light emitting device located on the opposite side of the membrane; a microphone; an analog-to-digital converter; a microcontroller; and an electronic connector; Providing an electronic device including a digital camera, at least one input mechanism, and a digital display; connecting the electronic stethoscope to the electronic device via the electronic connection; Turns light-emitting devices on and off in a specified pattern; Using a visual positioning algorithm to guide the user to position the electronic stethoscope at lung listening locations on the chest and the back of the user; and The duration of the lung sounds at each of the lung listening positions is recorded.

43. A method for positioning an electronic stethoscope at a target heart valve location and recording heart sounds, comprising: (a) acquiring a live video feed from a camera showing the user's head and torso and showing a variable switching light on the electronic stethoscope following a controlled switching program; (b) providing the live video feed to a pose estimation algorithm, the pose estimation algorithm outputting the key points of the user's pose, the key points comprising the positions of at least a left shoulder and a right shoulder of the user; (c) displaying the live video feed on a display screen, and superimposing the key points on corresponding positions on the live video feed displayed on the display screen; (d) providing the live video feed and the key points to a heart valve position estimation algorithm, the heart valve position estimation algorithm outputting a plurality of heart valve listening positions; (e) displaying the live video feed on the display screen and superimposing the key points and the plurality of heart valve listening positions on the corresponding positions of the live video feed displayed on the screen; (f) providing the live video feed, the key points, the plurality of heart valve listening positions, and a controlled program of a variably switchable light emitting device to a stethoscope position estimation algorithm, the stethoscope position estimation algorithm outputting a stethoscope position relative to the key points and the heart valve listening positions; (g) providing said live video feed, said key points, said plurality of heart valve listening locations, said controlled program of said variable switchable lighting device, and said stethoscope position to a guided stethoscope positioning and sound recording algorithm; (h) determining whether the electronic stethoscope is correctly positioned at a first heart valve listening position among the plurality of heart valve listening positions by the guided stethoscope positioning and sound recording algorithm; (i) sending a first signal indicating incorrect stethoscope positioning via the guided stethoscope positioning and sound recording algorithm to assist the user in positioning the electronic stethoscope; (j) repositioning of the electronic stethoscope by the user; (k) sending a second signal indicating correct stethoscope positioning via the guided stethoscope positioning and sound recording algorithm; (l) sending, by the guided stethoscope positioning and sound recording algorithm, a third signal indicating a pause in movement for a first duration; (m) recording, by the guided stethoscope positioning and sound recording algorithm, sounds received by a microphone disposed within the electronic stethoscope during a second duration; (n) Repeating steps (g) to (m) until a sound recording having a specified duration is recorded at each of the plurality of heart valve listening locations.

44. The method of claim 43, wherein the plurality of heart valve listening locations include an aortic valve area, a pulmonary valve area, a tricuspid valve area, and a mitral valve area.

45. The method of claim 43, wherein the first signal comprises a first colored shape displayed on a video screen of an electronic device at the location of the stethoscope.

46. ​​The method of claim 43, wherein the second signal comprises a second colored shape displayed on a video screen of the electronic device at the stethoscope location, wherein the second colored shape has a color that is different from the color of the first colored shape.

47. The method of claim 43, wherein the first signal, the second signal, and the third signal each comprise a visual indication generated by the electronic device.

48. The method of claim 43, wherein the first signal, the second signal, and the third signal each comprise an audible indication generated by the electronic device.

49. A method for positioning an electronic stethoscope at a target lung listening position and recording lung sounds, comprising: (a) acquiring a live video feed from a camera showing the head and torso of the user and showing a variable switching light device on an electronic stethoscope following a controlled switching program; (b) providing the live video feed to a pose estimation algorithm, the pose estimation algorithm outputting the key points of the user's pose, the key points comprising the positions of at least a left shoulder and a right shoulder of the user; (c) displaying the live video feed on a display screen and superimposing the key points on the corresponding positions on the live video feed displayed on the display screen; (d) providing the live video feed and the key points to a lung listening position estimation algorithm, the lung listening position estimation algorithm outputting a plurality of lung listening positions; (e) displaying the live video feed on the display screen and superimposing the key points and the plurality of lung listening positions on the corresponding positions on the live video feed displayed on the screen; (f) providing the live video feed, the key points, the plurality of lung listening positions, and the controlled program of the variably switchable light emitting device to a stethoscope position estimation algorithm, the stethoscope position estimation algorithm outputting a stethoscope position relative to the key points and the lung listening positions; (g) providing said live video feed, said key points, said plurality of lung listening locations, said controlled program of said variable switchable lighting device, and said stethoscope position to a guided stethoscope positioning and sound recording algorithm; (h) determining whether the electronic stethoscope is correctly positioned at a first lung listening position among the plurality of lung listening positions by the guided stethoscope positioning and sound recording algorithm; (i) sending a first signal indicating incorrect stethoscope positioning via the guided stethoscope positioning and sound recording algorithm to assist the user in positioning the electronic stethoscope; (j) repositioning of the electronic stethoscope by the user; (k) sending a second signal indicating correct stethoscope positioning via the guided stethoscope positioning and sound recording algorithm; (l) sending, by the guided stethoscope positioning and sound recording algorithm, a third signal indicating a pause in movement for a first duration; (m) recording, by the guided stethoscope positioning and sound recording algorithm, sounds received by a microphone disposed in the electronic stethoscope for a second duration, wherein the second duration; (n) Repeating steps (g) to (m) until a sound recording having the specified duration is recorded at each of the plurality of lung listening locations.

50. The method of claim 49, wherein the plurality of lung listening locations include locations near the tip of the lung, the upper lobe of the lung, the middle lobe of the lung, and the lower lobe of the lung, on the left and right sides of the front and back torsos of the user.

51. The method of claim 49, wherein the first signal comprises a first colored shape displayed on a video screen of an electronic device at the location of the stethoscope.

52. The method of claim 49, wherein the second signal comprises a second colored shape displayed on a video screen of an electronic device at the location of the stethoscope, wherein the color of the colored shape is different from the color of the first colored shape.

53. The method of claim 49, wherein the first signal, the second signal, and the third signal each comprise a visual indication generated by the electronic device.

54. The method of claim 49, wherein the first signal, the second signal, and the third signal each comprise an audible indication generated by the electronic device.

55. A method of positioning a stethoscope at one or more cardiac listening locations, the method comprising: providing a live video feed of the user's torso; determining the one or more cardiac listening locations based on the live video feed via a cardiac valve position estimation algorithm; displaying the live video feed on a video screen with the one or more cardiac listening locations superimposed on the live video feed; providing an electronic stethoscope in a field of view of the live video feed, the electronic stethoscope comprising a variably switchable light emitting device; moving the electronic stethoscope within the field of view of the live video feed so that the variably switchable light emitting device is proximate to the one or more cardiac listening locations; The one or more cardiac listening locations are rendered superimposed on the live video feed in a different color to indicate that the electronic stethoscope is properly positioned.

56. A method as claimed in claim 55, wherein determining the one or more cardiac listening positions based on the live video feed by a heart valve position estimation algorithm includes using a posture estimation algorithm, which outputs key points of the user's posture, and the key points include the position of at least the left shoulder and the right shoulder of the user.

57. A method of diagnosing the health of at least one human organ via a computational framework comprising manually coded diagnostics and / or machine learning techniques, the method comprising: receiving audio data from at least one of the heart and the lungs, the audio data being received from an electronic stethoscope, The audio data is processed via at least one of the following: a) segmenting the audio data via at least one of frequency, amplitude and duration; and b) a U-net neural network that receives the audio data as input, preprocesses the audio input into a frequency equivariance spectrogram, and outputs a corrected segmentation; The data is evaluated via at least one of the following: c) use of statistical diagnostic algorithms based on clinical practice guidelines; and d) a machine learning technique comprising a neural network trained via a set of matched pairs, each matched pair comprising at least one data point from the electronic stethoscope and at least one data point from a diagnostic device; and At least one result is displayed based on the evaluation of the data set.

58. The method of claim 57, wherein the at least one result comprises a diagnostic probability of at least one of the following diagnoses: normal heart, harmless murmur, aortic stenosis, and mitral valve prolapse.

59. The method of claim 57, wherein the diagnostic equipment comprises at least one of ultrasound equipment and echocardiography.

60. The method of claim 57, wherein the electronic stethoscope comprises a piezoelectric contact microphone.

Images