A system and method for acquiring and analyzing heart sounds from subjects to predict cardiovascular disease.

Abstract

A system (100) is provided for detecting, screening, diagnosing, and predicting cardiovascular disease by acquiring and analyzing the heart sounds of a subject (106). A laser device (102) irradiates the subject's cervical and / or chest region with a first laser signal, and a second laser signal is acquired by the reflection of the first laser signal. A camera unit (104) periodically acquires the second reflected laser signal and generates multiple image frame data containing spots. The multiple image frames are processed to extract time-series heart sound data. The time-series heart sound data is analyzed using a statistical model and / or a machine learning model to automatically determine normal / abnormal heart sounds by quantifying heart sounds and noises. The machine learning model is used on the heart sound data to predict the subject's (106) cardiovascular disease.

Description

Technical Field

[0001] The present disclosure relates generally to the field of precision cardiology, and more specifically, to systems and methods for detecting, screening, diagnosing, and predicting cardiovascular diseases in a subject by obtaining and analyzing heart sounds using statistical heuristic-based methods and / or machine learning-based methods and / or deep learning-based modeling.

Background Art

[0002] Detecting abnormalities in frequencies related to heart sounds is extremely important for several reasons, as it provides valuable information about the cardiovascular system. Currently known diagnostic methods for cardiovascular diseases or conditions are detected by recording frequencies related to mechanical vibrations caused by the opening and closing of heart valves. One of the initial diagnostic tests based on the detection of heart sounds and other acoustic signals is chest auscultation using a stethoscope. This technique listens to the sounds and noises emitted from the chest, and if they are different from normal sounds, it may suggest the presence of a pathological condition. In more advanced techniques for detecting such signals, more accurate devices such as digital stethoscopes are used. Furthermore, with the development of phonocardiography, it has become possible to visually represent the detected signals. In these techniques, the device needs to be placed close to the heart in order to detect the sound signal more accurately.

[0003] Since signals also propagate to the peripheral regions of the body through arteries and the like, techniques have been developed that can detect vibrations at positions away from the heart. This type of technique is based on the use of radar and visible light sources. It is advantageous in that it can be detected without the need for direct contact with the patient. However, remote diagnostic techniques can only detect a small part of the vibration frequencies from the heart. In most cases, low-frequency signals are detected, and only information related to the heart rate can be obtained from this information.

[0004] To overcome these limitations in processing obtained heart sound data, many acoustic-based diagnostic methods for health assessment have been disclosed in the prior art. However, the processing methods disclosed in the prior art are limited.

[0005] Therefore, in order to efficiently quantify and analyze heart sounds using biophotonics and machine learning for precision cardiology, it is necessary to address the aforementioned technical challenges in existing technologies. [Overview of the project]

[0006] This disclosure describes a method and system for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds. Embodiments of the present invention overcome the problems of the prior art by capturing digital biomarkers for the detection, screening, diagnosis, and prediction of cardiovascular disease in a non-contact manner. By decoding heart sounds at high frequency and with a high signal-to-noise ratio, and by automatically calibrating a laser device and camera unit to capture the optimal signal, the system contributes to providing medical care, diagnosis, and treatment tailored to the individual characteristics of each subject. The system and method of this disclosure enable the detection, screening, diagnosis, and prediction / prognosis modeling of detectable cardiovascular disease by evaluating heart sounds resulting from the mechanical contraction of the myocardium, the opening and closing of heart valves, and vibrations due to laminar and turbulent flow in the cardiovascular system.

[0007] According to the first aspect, this disclosure relates to a system for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds: A laser device configured to generate a first laser signal having a wavelength from 400 nm to 2500 nm and an output power of at least 0.1 mW to a maximum of 5 mW, wherein the first laser signal is directed toward a region including the neck and / or chest region of a subject, and a second laser signal is obtained by reflection of the first laser signal; (i) A camera unit configured to periodically acquire the second reflected laser signal at a signal acquisition frequency of at least 600 Hz, and (ii) generate a plurality of image frame data including a spot, wherein each acquired object is a camera unit that generates an image including a spot; A control unit configured to perform real-time calibration of the parameters of the laser device and camera unit; A server that is interconnected with a laser device, a camera unit, and a control unit, and comprises a processor and a memory that stores an operable machine-readable instruction set, and when executed by the processor: (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal that includes the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data. By doing so, multiple image frames are processed and time-series heart sound data is extracted. By analyzing time-series heart sound data using statistical and / or machine learning models and quantifying heart sounds and murmurs, normal / abnormal heart sounds can be automatically determined. A server that uses a machine learning model on heart sound data to predict the cardiovascular status of a subject; We provide a system equipped with these features.

[0008] Embodiments of this disclosure overcome the aforementioned shortcomings of existing known methods for detecting, screening, diagnosing, and predicting cardiovascular disease in subjects by acquiring and analyzing heart sounds. The advantage of embodiments of this disclosure lies in the ability of healthcare professionals to optimize the prevention, diagnosis, and treatment of cardiovascular disease by considering the unique characteristics of each subject. The system identifies and extracts heart sounds from regions with optimal signal quality by segmenting a second laser signal. These are then compared to each other to evaluate whether there are any changes in the quality of the decoded data and the signal-to-noise ratio. The system is useful for classifying subjects with cardiovascular disease from healthy subjects. Furthermore, it assesses the severity of the condition by providing a probability score, with higher probability scores indicating greater disease severity, allowing for further stratification into mild, moderate, and severe cardiovascular disease. The system provides detection and subclassification labels for cardiac diseases such as valvular heart disease, arrhythmias, and coronary artery disease. In addition, it quantifies the corresponding comorbidities of cardiac disease present in subjects as mild, moderate, or severe, enabling more appropriate triage and care.

[0009] One of the advantages of this system is that it allows testing to be performed without requiring direct contact with the subject, facilitating remote analysis. It also has the advantage of a simpler configuration compared to known solutions, making it usable in a variety of environments, not just outpatient settings. In particular, it offers the advantage of enabling analysis at home for subjects who have difficulty traveling.

[0010] In a second aspect, this disclosure relates to a method for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds: The process involves generating a first laser signal with a wavelength of 400 nm to 2500 nm and an output of at least 0.1 mW to a maximum of 5 mW using a laser device, irradiating the subject with the first laser signal towards a region including the neck and / or chest region, and obtaining a second laser signal by reflection of the first laser signal; The process involves periodically acquiring the second reflected laser signal by the camera unit at a signal acquisition frequency of at least 600 Hz, and in which the parameters of the laser device and the camera unit are calibrated in real time by the camera unit. The process involves generating multiple image frame data containing a spot, and generating an image containing the spot for each acquired object. The server receives the plurality of image frame data, including the spot, from the camera unit, and the server is interconnected with the laser device, the camera unit, and the control unit, and the server comprises a processor and a memory that stores a machine-readable instruction set that can be operated when executed by the processor; (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal that includes the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data. By doing so, multiple image frames are processed and time-series heart sound data is extracted. By analyzing time-series heart sound data using statistical and / or machine learning models and quantifying heart sounds and murmurs, normal / abnormal heart sounds can be automatically determined. A process for predicting a subject's cardiovascular disease using a machine learning model on heart sound data; Provides a method that includes this.

[0011] This method identifies and extracts heart sounds from regions with optimal signal quality by segmenting and comparing a second laser signal. This allows for evaluation of changes in the quality of the decoded data and the signal-to-noise ratio. This method is useful for classifying cardiovascular disease patients from healthy individuals. Furthermore, this method evaluates the severity of the condition by providing a probability score. A higher probability indicates a higher severity of the disease, and it is possible to subdivide cardiovascular diseases into mild, moderate, and severe. This method provides detection and subclassification labels for cardiac diseases such as aortic stenosis, arrhythmias, and coronary artery disease. Cardiovascular diseases predicted by the method disclosed herein, leading to cardiovascular pathology, can be altered by pharmacological and / or lifestyle changes and / or other therapies, bringing the (acoustic) characteristics of the condition closer to healthy baseline values ​​and / or ensuring that the disease does not worsen. This feedback loop is essential for realizing preventive medicine.

[0012] Additional aspects, advantages, features, and objectives of this disclosure will become apparent from the detailed description of the drawings and embodiments below, which will be interpreted in conjunction with the appended claims. It will be understood that the features of this disclosure can be combined in various combinations without departing from the scope of this disclosure as defined by the appended claims. [Brief explanation of the drawing]

[0013] The above summary and the detailed description of the following embodiments will be better understood when read in conjunction with the accompanying drawings. Exemplary configurations of the disclosure are shown in the drawings to illustrate this disclosure. However, this disclosure is not limited to the specific methods and means disclosed herein. Furthermore, those skilled in the art will understand that the drawings are not to scale. Wherever possible, identical elements are shown with the same number. Embodiments of this disclosure are described below with reference to the following drawings, for illustrative purposes only.

[0014] [Figure 1] This is a schematic diagram of a system that predicts a subject's cardiovascular disease by acquiring and analyzing heart sounds, according to embodiments of the present disclosure.

[0015] [Figure 2] According to an embodiment of the present disclosure, it is a schematic diagram of the server in FIG. 1 including various modules.

[0016] [Figure 3A-C] It is a flowchart showing a method for acquiring and analyzing heart sounds to predict a subject's cardiovascular disease according to an embodiment of the present disclosure.

[0017] [Figure 4] It is a schematic diagram of a computer architecture according to an embodiment of the present disclosure.

MODE FOR CARRYING OUT THE INVENTION

[0018] The following detailed description describes various embodiments of the present disclosure. Those skilled in the art will recognize that other embodiments for implementing or practicing the present disclosure are also possible.

[0019] Measured values, values, shapes, and geometric terms (such as right angles and parallelism) should be interpreted as measurement errors or inaccuracies based on manufacturing and / or production errors when combined with terms such as "approximately" or similar terms such as "substantially" or "essentially", and above all, should be understood as such except when there is a slight deviation from the relevant value, measured value, shape, or geometric term. For example, when these terms are associated with a numerical value, it preferably indicates a deviation within 10% of the numerical value itself.

[0020] [[ID=३०]]Furthermore, terms such as "first", "second", "upper", "lower", "main", "sub", etc. do not necessarily indicate order, priority, or relative position, and may simply be used to more clearly distinguish different components.

[0021] As will be apparent from the following description, unless otherwise specified, terms such as "processing", "calculation", "decision", etc. refer to operations and / or processes of a computer or similar electronic calculation that manipulate and / or transform data represented as physical quantities such as electronic quantities in computer system registers and / or memories, or other data similarly represented as physical quantities in computer systems, registers, other information storage devices, transmission devices, and display devices. The measured values and data reported in this text are to be considered as those executed within the ICAO International Standard Atmosphere (ISO 2533:1975) unless otherwise specified.

[0022] This diagnostic procedure is mainly (but not limited to) based on the detection of acoustic waves resulting from physiological processes that affect the cardiovascular system. This may include, in addition to signals related to the normal function of organs, rarer signals related to the presence of a pathological condition in the patient being examined.

[0023] According to a first aspect, the present disclosure is a system for detecting, screening, diagnosing, and predicting cardiovascular diseases in a subject by acquiring and analyzing heart sounds, comprising: a laser device configured to generate a first laser signal having a wavelength from 400 nm to 2500 nm and an output from at least 0.1 mW to a maximum of 5 mW, wherein the first laser signal is irradiated towards a region including the neck region and / or chest region of the subject, and a second laser signal is obtained by reflection of the first laser signal; (i) a camera unit configured to periodically acquire the second reflected laser signal at a signal acquisition frequency of at least 600 Hz, and (ii) generate a plurality of image frame data including spots, wherein each acquisition generates an image including a spot; a control unit configured to perform real-time calibration of the parameters of the laser device and the camera unit; A server that is interconnected with a laser device, a camera unit, and a control unit in a manner that enables communication between them, and comprising a processor and a memory that stores an operable machine-readable instruction set, wherein the server, when executed by the processor, The camera unit receives multiple image frame data, including a spot. (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal that includes the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data, thereby processing multiple image frames and extracting time-series heart sound data. By analyzing time-series heart sound data using statistical and / or machine learning models and quantifying heart sounds and murmurs, normal / abnormal heart sounds can be automatically determined. A server that uses a machine learning model on heart sound data to predict the cardiovascular status of a subject; We provide a system equipped with these features.

[0024] The advantage of the embodiments of this disclosure lies in the fact that healthcare professionals can optimize the detection, prevention, diagnosis, and treatment of cardiovascular disease by taking into account the unique characteristics of each subject. The system segments a second laser signal, identifies and extracts heart sounds from regions with optimal signal quality, and compares them to evaluate whether there are any changes in the quality of the decoded data and the signal-to-noise ratio. The system is useful for classifying patients with cardiovascular disease from healthy individuals. Furthermore, the system evaluates the severity of the condition by providing a probability score. A higher probability indicates a higher severity of the disease, and it is possible to subdivide cardiovascular disease into mild, moderate, and severe. The system provides detection and subclassification labels for cardiac conditions such as valvular heart disease, arrhythmias, and coronary artery disease. In addition, it quantifies the corresponding cardiac comorbidities present in subjects as mild, moderate, or severe, enabling more appropriate triage and care.

[0025] The laser device irradiates light onto the base of the subject's neck and / or chest region to extract heart sound data. The laser device may be a diode laser device. Diode laser devices are advantageous in that they can be small and compact in size, making system installation easier. The first laser signal is a laser beam with a diameter preferably in the range of 2 mm to 6 mm. The first laser signal may have an output of at least 2 mW. The first laser signal preferably has a wavelength between 500 nm and 560 nm, more preferably between 510 nm and 550 nm, and even more preferably between 520 nm and 540 nm. These wavelength values ​​have the advantage of improving signal acquisition by the camera unit.

[0026] The camera unit is positioned at a distance of 0.05 m to 15 m. More preferably, it is positioned between 0.25 m and 1.0 m, and even more preferably between 0.5 m and 0.75 m. In particular, it is preferable that the camera unit be positioned further away from the focal length, i.e., at a distance between 10 cm and 30 cm, more preferably between 15 cm and 25 cm. This distance has the advantage of facilitating the visualization of the second laser signal on the camera unit's screen. In some embodiments, the camera unit is configured to periodically acquire the second reflected laser signal at a signal acquisition frequency of at least 0.8 kHz, more preferably at least 1.0 kHz, and even more preferably at least 1.2 kHz.

[0027] The camera unit may be equipped with auxiliary components such as a filtered lens to capture reflected light. The secondary camera may or may not be equipped with an edge computing model for automatic detection of the base of the subject's neck and / or chest region. Image frame data generated from the camera unit is transmitted to the server via a wired data transfer module and / or a Bluetooth module and / or a Wi-Fi module. The image frame data may also be captured by the camera unit in MP4 format files or other relevant formats including NumPy arrays and transmitted to the server. The server may be equipped with additional modules for data acquisition feedback, data anonymization, and pre-processing and / or post-processing to convert the second laser signal into heart sound data. The image frame data may be recorded and stored in encrypted and anonymized form in local [device / clinic / hospital / laboratory] storage devices and / or cloud databases.

[0028] Time-series heart sound data may be split into iso-length or variable-length segments of typically 5-10 seconds for further analysis. The machine learning model's backend utilizes shell scripts, Python with corresponding signal processing and ML / AI libraries, Docker, a data protection and anonymization protocol system, cloud computing with real-time APIs to facilitate data transfer and analysis, and signal processing libraries. Finally, the inference results derived from the analysis are automatically generated as reports / outputs, contributing to improved patient management and the provision of individualized care for each patient by general practitioners and / or automated patient triage systems and / or cardiologists and / or other clinicians.

[0029] In some embodiments, a motion description algorithm is used to quantify image motion between consecutive image frames. Motion is estimated between consecutive frames, then quantified, and the image is converted into time-series heart sound data. Accordingly, the first algorithm reconstructs a spot displacement map at each pixel on the surface of the camera unit. In this way, the first algorithm calculates the vector displacement along the surface of each spot. From the displacement of each spot in the acquired identical image, the first algorithm calculates the average vector displacement of the entire image. The first algorithm can also display a graphic map reporting the spots, the vector displacements associated with the spots, and the entire image. The first algorithm generates a raw signal, which is a function obtained by interpolating the motion recorded at each point in time of image acquisition. Therefore, the raw signal contains the overall variation of images acquired continuously within a certain time interval. The raw signal contains both diagnostically valuable information and interfering elements.

[0030] In the filtering substep, a second algorithm is used. This analyzes the raw signal and converts it into a frequency spectrum. Thus, it consists of components, each with its own unique frequency. Since these components have different origins, it is necessary to remove the components of interest from the unwanted ones. Therefore, the second algorithm performs an operation to remove some of the components, obtaining the residual components. The residual components are, therefore, components with frequencies related to the phenomenon of interest, at least. In particular, it is desirable to remove components with frequency values ​​between 0 Hz and 15 Hz and 25 Hz, more preferably between 17 Hz and 23 Hz, and even more preferably between 18 Hz and 22 Hz, from the spectrum. In fact, no signals related to known phenomena of diagnostic importance are detected in these frequency ranges. Therefore, components falling within this frequency range are removed from the spectrum. The residual components constitute the signal. The latter is a signal of diagnostic importance. The frequency-separated signal correlates with the presence of a pathological condition. In the literature, the primary sound originating from the heart corresponds to the first signal, fluctuates in the range of 10 Hz to 140 Hz, and is known to be due to the closure of the mitral and tricuspid valves. This signal is distinguishable from the second heart sound originating from the heart. It is distinguished from the first signal by systolic delay. This signal fluctuates in the range of 10 Hz to 400 Hz and is caused by the closure of the aortic and pulmonary valves. This type of signal is clearly present in all patients. Other detectable signals reach higher frequencies. In fact, they can reach a frequency range of 20 Hz to 1000 Hz. These signals, in particular, are indicators of the presence of pathological conditions. In particular, it is desirable that the frequency range of the signal be analyzed at least at a value of 15 Hz and values ​​between 400 Hz and 1000 Hz, preferably between 650 Hz and 750 Hz, and more preferably between 675 Hz and 725 Hz.

[0031] Heart sound data includes prominent S1 and S2 sounds resulting from the opening and closing of heart valves, S3 and S4 sounds indicating pathological and non-pathological conditions, corresponding murmurs associated with blood flow, and / or turbulent blood flow phenomena due to interaction with plaque and cholesterol in the arteries. The server applies statistical and / or machine learning models to the heart sound data to automatically quantify heart sounds and murmurs and classify them as S1, S2, S3, S4, and murmurs.

[0032] The statistical model employed to analyze heart sound data extracts simple time-domain statistical features of heart sounds, such as mean, median, variance, standard deviation, skewness, and kurtosis. Furthermore, composite features such as peak-to-peak mean, mean square root, Hjorth parameter activity, Hjorth parameter mobility, Hjorth parameter complexity, maximum power spectral frequency, maximum power spectral density, and power sum are derived. Nonlinear entropy features such as Shannon entropy, singular entropy, Kolmogorov entropy, approximate entropy, C0 complexity, correlation dimension, Lyapunov exponent, permutation entropy, and spectral entropy are also derived. If necessary, ICA and other wavelet-based features are acquired to derive stronger features that are prominent in cardiac disease and comorbidity groups. All of these contribute to more interpretable analysis and the identification of pathological features. Features are selected based on information about disease groups. Low-variance filters, high-correlation filters, random forests, and forward feature selection are used for automated feature selection. Gender, age, and BMI are common covariates that influence the feature selection process. Based on the suspected disease pathology and the machine learning model used, different features are automatically weighted and selected for each subject. By integrating and combining multiple features, new features with increased information about the disease state are derived. This improves predictive accuracy in classification and prognosis prediction. The system can reduce the dimensionality of the feature set using dimensionality reduction algorithms such as wavelets, PCA, ICA, and factor analysis. This is used to improve the accuracy of the model with minimal data and to increase the speed of analysis using minimal features while retaining information about the data distribution.

[0033] In some embodiments, normative modeling is used to analyze sound heart data using a wide range of cardiac data collected from diverse and broad populations. Normative modeling establishes "normal" criteria for various cardiac parameters. By comparing individual cardiac health data to these criteria, deviations from the norm can be effectively identified. This comparison can function as an early warning system for potential cardiac disease. Normative modeling provides a comprehensive and contextually rich perspective on individual cardiac health assessments, enabling personalized cardiac care and long-term modeling.

[0034] In some embodiments, synthetic heart sound data is generated to account for class imbalances using methods such as synthetic small-group oversampling (SMOTE) or neural-based generative models. This allows for the generation of more data from the same distribution as the classes under investigation, enabling the development of more robust machine learning models for the detection and prognosis of cardiac diseases.

[0035] In some embodiments, biometric authentication of the subject is performed. Heart sound data is anonymized and / or complies with GDPR / HIPAA. The subject is linked to an existing ID or a new ID is created through a database search. The subject is clearly identifiable by a unique heart sound quantified using a second laser signal captured from the cervical and / or thoracic region. This is quantified as an anonymous subject ID to track individual subjects and matched with patient IDs in EHR records or other systems.

[0036] In some embodiments, summary data of a subject is generated using acquired data such as heart rate (beats per minute), rhythm, heart sound intensity, heart sound duration, presence or absence of additional sounds or noises, heart sound separation, and respiratory variability. Heart sounds and noises such as S1, S2, S3, and S4 are automatically labeled to improve the accuracy of data evaluation and patient management. An automatically generated report containing the subject's summary data is provided to the clinician. The clinician is presented with reference values, detection / disease scores, and / or prognosis scores, and / or the subject is provided with reference values ​​and feedback scores. The report also includes subjective data from the EHR, questionnaire scores, and symptom scores. This ensures a comprehensive approach to the patient and clinical reporting.

[0037] The server can be an edge PCB / microcontroller and / or a cloud server. In some embodiments, the processing of multiple image frames to extract time-series heart sound data is performed on the edge PCB / microcontroller, while the quantification of heart sounds and the prediction of cardiovascular disease using statistical models and / or machine learning models are performed on the cloud server. The time-series heart sound data can be transferred from the edge PCB / microcontroller to the cloud server or other external device using Wi-Fi and / or Bluetooth modules for the quantification of heart sounds and the prediction of cardiovascular disease using statistical models and / or machine learning models.

[0038] Optionally, a machine learning model predicts a subject's cardiovascular disease based on baseline percentile scores, time-series scores, and prognostic scores from heart sound data.

[0039] The system may reduce the dimensionality of the feature set using dimensionality reduction algorithms such as wavelets, principal component analysis, independent component analysis, and factor analysis. This is used to improve the accuracy of the model with minimal data and to speed up further analysis using minimal features while retaining information about the data distribution. Shallow learning algorithms such as logistic regression, support vector machines, random forests, decision trees, and naive Bayes, and / or deep learning-based algorithms such as convolutional neural networks (CNNs), transformer-based models, recurrent neural networks (RNNs), and long-term short-term memory (LSTM) can be used. Unsupervised clustering such as k-means and t-SNE can be applied to heart sound feature data to derive data-driven disease clusters and comorbidity groups. This improves the accuracy of prognosis prediction. Furthermore, Bayesian methods and Monte Carlo simulations can be used to estimate probabilistic disease stages and progression. As more data points are acquired, supervised and / or unsupervised learning approaches based on machine learning can be used to learn across different subjects and further improve the accuracy of cardiovascular disease diagnosis. Further leverage ensemble features and / or the model to improve the model's predictive accuracy.

[0040] By estimating how a subject's cardiac acoustic characteristics change over time from feature data within the same subject, we can understand the function and health status of the heart and circulatory system. This allows us to evaluate changes in disease pathology. One method involves evaluating the subject's baseline score at two different time points. By determining whether the change in the score is approaching or deviating from a healthy baseline, we can obtain useful insights for deriving a prognostic score.

[0041] Machine learning models derive acoustic (sound) markers associated with cardiovascular disease using interpretable models such as Grad-CAM and saliency maps. These markers can then be modulated using pharmacological and / or lifestyle changes and / or other therapies that alter the (sound) characteristics of the disease, leading to healthy baseline values ​​or preventing disease progression. This feedback loop is key to realizing preventive medicine. The effectiveness of the system and the accuracy of the output at each stage can be evaluated using metrics such as percentile deviation, positive predictive value (PPV), confusion matrix, precision, precision, recall (sensitivity, true positive rate), specificity, F1 score, precision-recall (PR) curve, receiver operating characteristic (ROC) curve, and PR vs. ROC curve. The model learns iteratively as more data is obtained, improving its performance.

[0042] Optionally, time-series heart sound data includes frequencies with values ​​between 20Hz and 1000Hz.

[0043] Optionally, the server is further configured to automatically detect the subject's neck and / or chest region and control the operation of the laser device, controlling the first laser signal and the camera unit to capture the second laser signal via the control unit.

[0044] Optional camera units include high-frequency CCD (charge-coupled digital image sensor), CMOS (complementary metal-oxide-semiconductor) image sensor, sCMOS (scientific complementary metal-oxide-semiconductor) image sensor, or Raspberry Pi camera.

[0045] Optionally, the camera unit can capture a second laser signal at a distance between 0.05m and 15m.

[0046] Optionally, the system includes a low-pass filter that filters out frequencies below 20 Hz.

[0047] Optionally, the second laser signal is captured by the camera unit with a field resolution of 400x400, divided into four 200x200 field frames, and / or 16 100x100 fields, and / or 64 50x50 fields, and / or 256 25x25 fields, and processed individually to obtain the corresponding heart sound signals.

[0048] Optionally, the server can be configured to process multiple image frames and extract time-series heart sound data using an image-motion description model or a motion tracking model.

[0049] Optionally, the server is configured to compare time-series audio data with each other to determine changes in the signal-to-noise ratio.

[0050] Optionally, the server is configured to perform real-time calibration of the laser device and camera unit via the control unit based on the determined signal-to-noise ratio change to obtain the optimal signal.

[0051] To record heart sound data with an optimal signal-to-noise ratio, precise alignment and / or parameterization of the laser device and camera unit are required to ensure that the camera unit reliably captures the reflected light from the laser device irradiated onto the subject. Furthermore, the camera unit's lens ensures that the second laser signal recorded by the camera unit is reliably focused, thereby capturing a robust signal ideal for further analysis. The server provides feedback to the laser device through configuration and parameter settings, enabling real-time rapid calibration of the laser device. The quality of the heart sound data is evaluated using a quality analysis / quality control module. Depending on the quality of the heart sound data, the server provides feedback to the laser device to readjust the configuration and / or parameters for optimal signal acquisition. Variables including patient information such as skin tone, height, weight, body mass index (BMI), and historical clinical data obtained from the electronic health record / medical record (EHR / EMR) system are used to further optimize the laser device parameters (i.e.) and laser irradiation position to collect heart sound data with the optimal signal-to-noise ratio and frequency for further analysis. The quality of the captured signal can be evaluated on a scale of 1 to 10, with 1 representing very low-quality data and 10 representing high-quality data, allowing for the quantification of heart sounds. This evaluation is provided as feedback to the laser device and camera unit, enabling automatic calibration.

[0052] Optionally, the machine learning model is trained using a multimodal dataset that includes (i) phonocardiogram (PCG) data from age- and sex-matched subjects who are assigned an ID and / or cardiovascular disease and / or heart disease and / or heart murmurs and / or sound labels such as S1, S2, etc., and (ii) subjects' clinical / symptom data and / or demographic data.

[0053] Clinical data includes medical history and relapse history, dietary changes / restrictions, blood parameters, past / present medication use and habits, and different treatment plans. Demographic data includes the patient's skin tone, age, sex, marital status, family structure, ethnicity, income range, and educational history. Clinical and demographic data may be anonymized and automated to maintain confidentiality. Patients and / or general practitioners, nurses, and electronic health record database systems can upload the necessary clinical and demographic data to the database manually or automatically.

[0054] To achieve robust predictive modeling using minimal data collected by the camera unit, a baseline model is trained using phonocardiograms (PCG), and the model weights are further optimized using data collected for the camera unit with a transfer learning approach. This results in a highly versatile diagnostic and prognostic model suitable for clinical application. The steps are as follows:

[0055] Phase 1: Construction of an initial model based on PCG data Initial baseline models are trained, tested, and validated using a phonocardiogram (PCG) dataset. The dataset includes relevant subjects with heart murmurs and / or valvular heart disease and / or coronary artery disease (labeled in the models). The baseline model training dataset is drawn from a representative population, taking into account age, sex, and other covariates.

[0056] The initial model is trained using PCG data from age- and sex-matched individuals with mild, moderate, and severe valvular heart disease, and further considers covariates such as BMI, skin tone, and other clinical comorbidities. Model validation and confirmation are also performed using PCG data from the same distribution. This model uses multimodal data for training, (i.e.) using age, sex, and other clinical findings as model inputs in addition to PCG data. At the end of this step, a robust baseline model that considers covariates is constructed. This model can be used for transfer learning.

[0057] Phase 2: Transfer Learning The baseline model weights are further trained and optimized using high-SNR, high-frequency data collected from the laser device and camera unit. While the subject data distribution in the PCG dataset matches the subject data collected by the laser device and camera unit, it does not necessarily need to match, as the final layer of the model is optimized using a multimodal dataset. The newly trained model is compared to an out-of-sample validation dataset collected from the laser device and camera unit to ensure the accuracy of the newly optimized model. The data distributions in both training and validation are similar and consist of covariates such as age, sex, and other clinical findings.

[0058] Optionally, a machine learning model is trained and / or modeled using a labeled dataset, with extracted features as input and corresponding labels as the target output.

[0059] Optionally, the server connects to an electronic health record / medical record (EHR / EMR) and / or a clinic / hospital / local backend server system to retrieve demographic data including subject height, weight, body mass index (BMI), and clinical / symptom data.

[0060] Optionally, the machine learning model is trained using the subject's heart rate data on a subject ID dataset labeled for biometric authentication.

[0061] Optionally, the total energy distribution in the recorded second laser signal is integrated across the entire image frame. This integration is performed individually for each of the multiple image frames and converted into time-series heart sound data.

[0062] Optionally, the baseline percentile score is determined based on the subject's (106) heart sound index, derived from the general population's heart sound index, taking into account factors such as sex, age, and other relevant statistical variables.

[0063] Optionally, the long-term score is determined based on changes in the subject's criteria for the derived heart sound index between one consultation and the next, and / or after a therapeutic intervention.

[0064] Changes in a subject's baseline heart sound data between consultations and / or after therapeutic interventions allow for comparison of the new activity state with a standard model, i.e., evaluating whether heart sound characteristics have improved or worsened, and whether the therapeutic intervention is effective for the individual. This facilitates the optimization of better treatment plans.

[0065] Optionally, the server can be configured to display heartbeats in real time on an interactive user interface or output them in real time via speakers / headphones.

[0066] Optionally, the server can be configured to use a machine learning model on heart sound data to classify heart murmurs into systolic, diastolic, and / or persistent murmurs.

[0067] According to a second aspect of this disclosure, a method for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds, wherein: The process involves generating a first laser signal with a wavelength of 400 nm to 2500 nm and an output of at least 0.1 mW to a maximum of 5 mW using a laser device, irradiating the subject with the first laser signal towards a region including the neck and / or chest region, and obtaining a second laser signal by reflection of the first laser signal; The process involves periodically acquiring the second reflected laser signal by the camera unit at a signal acquisition frequency of at least 600 Hz, and in which the parameters of the laser device and the camera unit are calibrated in real time by the camera unit. The process involves generating multiple image frame data containing a spot, and generating an image containing the spot for each acquired object. The server receives the plurality of image frame data, including the spot, from the camera unit, and the server is interconnected with the laser device, the camera unit, and the control unit, and the server comprises a processor and a memory that stores a machine-readable instruction set that can be operated when executed by the processor; (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal that includes the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data. By doing so, multiple image frames are processed and time-series heart sound data is extracted. By analyzing time-series heart sound data using statistical models and / or machine learning models, and quantifying heart sounds and murmurs, normal / abnormal heart sounds are automatically determined. The process of predicting a subject's cardiovascular disease using a machine learning model based on heart sound data; It is equipped with.

[0068] To acquire heart sound data, the laser device and camera unit are positioned at mutually determined locations, and the subject is held in a stable position until the next acquisition phase. The subject can assume any of the following positions: sitting, standing, or lying down, so that the first laser signal emitted from the laser device is directed onto a portion of the subject's skin. The portion of the subject's skin preferably corresponds to the base of the subject's neck. This body region is richly vascularized, making it easier to detect vibrations related to physiological processes and visceral functions. If abnormal heart sounds related to abnormalities are detected, this procedure allows for preliminary analysis and referral to further detailed examinations.

[0069] Optionally, a machine learning model predicts a subject's cardiovascular disease based on baseline percentile scores, time-series scores, and prognostic scores from heart sound data.

[0070] To help subjects recover to health management standards and alleviate symptoms, an individualized and optimal treatment strategy can be predicted from among drug therapy and lifestyle modifications. The subject's cardiac and circulatory system health is continuously monitored, changes in acoustic conditions due to optimal treatment interventions are evaluated, and interventions are further adjusted as needed based on the subject's response to treatment to derive an individualized treatment plan.

[0071] Embodiments of this disclosure substantially eliminate, or at least partially eliminate, the aforementioned technical shortcomings of existing technologies by providing systems and methods for detecting, screening, diagnosing, and predicting cardiovascular disease in subjects by acquiring and analyzing heart sounds using statistical heuristic-based methods and / or machine learning methods and / or deep learning-based modeling. Detailed description of the drawings

[0072] Figure 1 is a schematic diagram of a system 100 according to an embodiment of the present disclosure for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds. The system 100 includes a laser device 102 configured to generate a first laser signal having a wavelength from 400 nm to 2500 nm and an output of at least 0.1 mW to a maximum of 5 mW. The first laser signal is directed toward a region including the neck and / or chest region of a subject 106, and a second laser signal is acquired by reflection of the first laser signal. A camera unit 104 is configured to (i) periodically acquire the second reflected laser signal at a signal acquisition frequency of at least 600 Hz, and (ii) generate a plurality of image frame data including a spot. Each acquisition generates an image including the spot. A control unit 108 is configured to perform real-time calibration of the parameters of the laser device 102 and the camera unit 104. A server 110 is interconnected with the laser device 102, the camera unit 104, and the control unit 108. Server 110 is configured to receive multiple image frame data, including spots, from camera unit 104. Server 110 is configured to process multiple image frames and extract time-series heart sound data by (i) detecting positional changes of each spot between consecutive image frames, (ii) obtaining aggregated values ​​of positional changes in consecutive image frames to generate a raw signal containing the overall changes of image frames collected consecutively within a certain time interval, and (iii) converting the raw signal into amplitude to obtain time-series heart sound data. Server 110 is configured to automatically determine normal / abnormal heart sounds by analyzing the time-series heart sound data using a statistical model and / or a machine learning model and quantifying heart sounds and noises. Server 110 is configured to predict the cardiovascular disease of the subject (106) using a machine learning model on the heart sound data.

[0073] Figure 2 is a schematic diagram of the server 110 of Figure 1, which includes various modules according to embodiments of the present disclosure. The server 110 includes a database 202, a data receiving module 204, a time-series heart sound data extraction module 206, a heart sound quantification module 208, and a cardiovascular disease prediction module 210. The data receiving module 204 is configured to receive multiple image frame data, including spots, from the camera unit 104. The time-series heart sound data extraction module 206 is configured to process multiple image frames and extract time-series heart sound data by (i) detecting positional fluctuations of each spot between consecutive image frames, (ii) obtaining aggregated values ​​of positional fluctuations of consecutive image frames to generate a raw signal that includes the overall fluctuations of image frames collected consecutively within a certain time interval, and (iii) converting the raw signal into amplitude to obtain time-series heart sound data. The heart sound quantification module 208 is configured to automatically determine normal / abnormal heart sounds by analyzing time-series heart sound data using a statistical model and / or a machine learning model and quantifying heart sounds and noises. The cardiovascular disease prediction module 210 is configured to predict a subject's cardiovascular disease using a machine learning model on heart sound data. The server 110 includes a calibration module that calibrates the parameters of the laser device 102 and camera unit 104 in real time via a control unit. The server 110 also includes a quality analysis / quality control module configured to ensure the quality and consistency of time-series heart sound data.

[0074] Figures 3A-3C are flowcharts illustrating a method for predicting a subject's cardiovascular disease by acquiring and analyzing heart sounds according to embodiments of the present disclosure. In step 302, the method comprises generating a first laser signal with a laser device having a wavelength from 400 nm to 2500 nm and an output power of at least 0.1 mW to a maximum of 5 mW. The first laser signal is directed toward a region including the neck and / or chest region of the subject to obtain a second laser signal by reflection of the first laser signal. In step 304, the method comprises periodically acquiring the second reflected laser signal with a camera unit at a signal acquisition frequency of at least 600 Hz. The parameters of the laser device and camera unit are calibrated in real time through a control unit. In step 306, the method comprises generating a plurality of image frame data including a spot with the camera unit. Each acquisition generates an image including a spot. In step 308, the method comprises receiving the plurality of image frame data including the spot from the camera unit by a server. In step 310, the method includes the steps of (i) detecting the positional variation of each spot between consecutive image frames, (ii) obtaining an aggregate value of the positional variation of consecutive image frames to generate a raw signal, the raw signal including the overall variation of image frames collected consecutively within a certain time interval, and (iii) converting the raw signal into amplitude to obtain time-series heart sound data, thereby processing multiple image frames and extracting time-series heart sound data by a server. In step 312, the method includes the step of analyzing the time-series heart sound data using a statistical model and / or a machine learning model and quantifying heart sounds and noises to automatically determine normal / abnormal heart sounds by a server. In step 314, the method includes the step of predicting the subject's cardiovascular disease by a server using a machine learning model on the heart sound data.

[0075] Figure 4 is a schematic diagram of a computer architecture according to an embodiment of the present disclosure. Referring to Figures 1 to 3, a typical hardware environment for implementing this embodiment is shown in Figure 4. This schematic diagram shows the hardware configuration of a server 110 / computer system according to this embodiment. The server 110 / computer comprises at least one processing unit 10 and an encryption processing unit 11. The special purpose CPU 10 and the encryption processing unit (CP) 11 can be interconnected via a system bus 14 to various devices such as random access memory (RAM) 15, read-only memory (ROM) 16, and input / output (I / O) adapters 17. The I / O adapters 17 can be connected to peripheral devices such as disk units 12 and tape drives 13, or to other program storage devices that the system can read.

[0076] The server 110 / computer can read inventive instructions on program storage and execute the methods of this embodiment according to these instructions. The server 110 / computer system further includes a user interface adapter 20 that connects other user interface devices such as a keyboard 18, a mouse 19, a speaker 25, a microphone 23, and / or a touchscreen device (not shown) to the bus 14 for collecting user input. Furthermore, a communication adapter 21 connects the bus 14 to a data processing network 26, and a display adapter 22 connects the bus 14 to a display device 24. This display device becomes a graphical user interface (GUI) 30 of the output data according to this embodiment. Alternatively, it may be embodied as an output device such as a monitor, printer, or transmitter. Furthermore, a transceiver 27, a signal comparator 28, and a signal converter 29 may be connected to the bus 14 for processing, transmitting, receiving, comparing, and converting electrical or electronic signals.

[0077] Modifications to the embodiments described herein are possible without departing from the scope of this specification as defined by the appended claims. Expressions such as “includes,” “constitutes,” “incorporates,” “has,” and “is” used to describe this disclosure and in claiming a patent are intended to be interpreted in a non-exclusive manner (i.e., allowing for the inclusion of items, components, or elements not expressly described). Singular descriptions should also be interpreted as relating to plurals. [Explanation of symbols]

[0078] 100-System 102-Laser device 104-Camera Unit 106 subjects 108-Control Unit 110-Server 202-Database 204 - Data receiving module 206 - Time-series heart sound data extraction module 208 - Heart Sound Quantification Module 210 - Cardiovascular Status Prediction Module

Claims

1. A system for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject by acquiring and analyzing heart sounds (100): The system comprises a laser device (102), a camera unit (104), and a server interconnected with a control unit (108), wherein the laser device (102) is configured to generate a first laser signal having a wavelength from 400 nm to 2500 nm and an output of at least 0.1 mW to a maximum of 5 mW, the first laser signal being directed toward a region of the subject (106) including the neck and / or chest region, and a second laser signal being obtained by reflection of the first laser signal, and the camera unit (104) is configured to (i) at least (ii) The system is configured to periodically acquire the second reflected laser signal at a signal acquisition frequency of 600 Hz and generate a plurality of image frame data including a spot, each of which generates an image including a spot, and the control unit (108) is configured to perform real-time calibration of the parameters of the laser device (102) and the camera unit (104), and the server (110) comprises a processor and a memory that stores an operable machine-readable instruction set, and the server (110) is configured to perform when executed by the processor: The camera unit (104) receives multiple image frame data including a spot, (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal that includes the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data. By doing so, multiple image frames are processed and time-series heart sound data is extracted. By analyzing time-series heart sound data using statistical and / or machine learning models and quantifying heart sounds and murmurs, normal / abnormal heart sounds can be automatically determined. A system (100) that uses a machine learning model on heart sound data to predict cardiovascular disease in a subject (106).

2. The system (100) according to claim 1, characterized in that the machine learning model predicts cardiovascular disease in a subject (100) based on a reference percentile score, a time-series score, and a prognosis score of heart sound data.

3. The system (100) according to claim 1, characterized in that the time-series heart sound data includes frequencies having values ​​between 20 Hz and 1000 Hz.

4. The system (100) according to claim 1 is further configured to automatically detect the neck and / or chest region of the subject (106), control the operation of the laser device (102), and control the first laser signal and the camera unit (104) to capture the second laser signal via the control unit (108).

5. The system (100) according to claim 1 is characterized in that the camera unit (104) is a high-frequency CCD (charge-coupled device), CMOS (complementary metal-oxide-semiconductor), sCMOS (scientific complementary metal-oxide-semiconductor) image sensor, or a Raspberry Pi camera.

6. The system (100) according to claim 1 is characterized in that the camera unit (104) captures the second laser signal at a distance between 0.05 m and 15 m.

7. The system (100) according to claim 1 is characterized in that it includes a low-pass filter that filters frequencies below 20 Hz.

8. The system (100) according to claim 1, characterized in that the second laser signal is captured by the camera unit (100) with a field resolution of 400 × 400, divided into four 200 × 200 field frames, and / or 16 100 × 100 fields, and / or 64 50 × 50 fields, and / or 256 25 × 25 fields, and processed individually to obtain the corresponding heart sound signals.

9. The system (100) according to claim 1, characterized in that the server (110) is configured to process multiple image frames and extract time-series heart sound data using an image motion description model or a motion tracking model.

10. The system (100) according to claims 8 and 9 is characterized in that the server (110) is configured to compare time-series sound data with each other and determine the change in the signal-to-noise ratio.

11. The system (100) according to claims 8 and 9, characterized in that the server (110) is configured to perform real-time calibration of the laser device (102) and the camera unit (104) via the control unit (108) based on the determined change in the signal-to-noise ratio, and to acquire an optimal signal.

12. The machine learning model is trained using (i) phonocardiogram (PCG) data of subjects matched for age and sex who are assigned an ID and / or cardiovascular disease and / or heart disease and / or heart murmur and / or sound labels such as S1, S2, etc., and (ii) a multimodal dataset including clinical / symptom data and / or demographic data of subjects (106) according to claim 1 (100).

13. The system (100) according to 12, characterized in that the machine learning model is trained and / or modeled using a labeled dataset, with extracted features as input and corresponding labels as target outputs.

14. The system according to 12, characterized in that the server (110) is connected to an electronic health record / medical record (EHR / EMR) and / or a clinic / hospital / local backend server system to acquire demographic data including the height, weight, body mass index (BMI), and medical / symptom data of a subject (106).

15. The system (100) according to claim 12, wherein the machine learning model is trained using a labeled subject ID dataset for biometric authentication using heart sound data of the subject (106).

16. The total energy distribution in the recorded second laser signal is integrated over the entire image frame. This integration is performed individually for each of the multiple image frames and converted into time-series heart sound data, characterized in that the system (100) according to claim 1.

17. The system (100) according to claim 2, characterized in that the reference percentile score is determined based on the heart sound index of a subject (106) derived from the heart sound index of the general population, taking into account factors such as sex, age, and other relevant statistical variables.

18. The system (100) according to claims 2 and 17, characterized in that the long-term score is determined based on changes in the criteria for the derived heart sound index of the subject (106) between one examination and the next, and / or after a therapeutic intervention.

19. The system (100) according to claim 1, characterized in that the server (110) can be configured to display heart sounds in real time on an interactive user interface or to output them in real time via speakers / headphones.

20. The system (100) according to claim 1, characterized in that the server (100) is configured to use a machine learning model on heart sound data to classify heart murmurs into systolic murmurs, diastolic murmurs, and / or persistent murmurs.

21. A method for detecting, screening, diagnosing, and predicting cardiovascular disease in a subject (106) by acquiring and analyzing heart sounds, comprising: The process involves generating a first laser signal having a wavelength of 400 nm to 2500 nm and an output of at least 0.1 mW to a maximum of 5 mW using a laser device (102), irradiating the subject (106) with the first laser signal towards a region including the neck and / or chest region, and obtaining a second laser signal by reflection of the first laser signal; The process involves periodically acquiring the second reflected laser signal by a camera unit (104) at a signal acquisition frequency of at least 600 Hz; The process involves generating multiple image frame data containing a spot, generating an image containing a spot for each acquired object, and calibrating the parameters of the laser device (102) and camera unit (104) in real time by the control unit (106); The server (110) receives the plurality of image frame data, including the spot, from the camera unit (104), and the server (110) is connected to communicate with the laser device (102), the camera unit (104), and the control unit (108), and the server (110) comprises a processor and a memory that stores a set of machine-readable instructions that can be operated when executed by the processor: (i) Detect the positional variation of each spot between consecutive image frames, (ii) obtain an aggregate value of the positional variation of consecutive image frames to generate a raw signal, the raw signal including the overall variation of image frames collected consecutively within a certain time interval, and (iii) convert the raw signal into amplitude to obtain time-series heart sound data, thereby processing multiple image frames and extracting time-series heart sound data. By analyzing time-series heart sound data using statistical models and / or machine learning models, and quantifying heart sounds and murmurs, normal / abnormal heart sounds are automatically determined. A process of predicting cardiovascular disease in a subject (106) using a machine learning model on heart sound data; A method for providing this.

22. The method according to 21, characterized in that the machine learning model predicts cardiovascular disease in a subject (106) based on a reference percentile score, a time series score, and a prognosis score of heart sound data.

Images