Information processing method, program, and information processing device

Abstract

Provided are an information processing method and the like capable of supporting determination of a state of a heart on the basis of feature amounts calculated from an electrocardiogram, a heart sound waveform, or a pulse wave waveform. According to the present invention, a computer acquires an electrocardiogram, a heart sound waveform, and a pulse waveform of a patient. The computer acquires: a first feature amount by a first formula using a I sound position or a II sound position obtained from the heart sound waveform, a Q wave position obtained from the electrocardiogram, and a notch position and a pulse wave rising position obtained from the pulse wave waveform; a second feature amount by a second formula using a maximum amplitude of the I sound or an area obtained by time-integrating a waveform of the I sound, or the maximum amplitude of the II sound or the area obtained by time-integrating a waveform of the II sound; and a third feature amount by a third expression using an element feature amount obtained by using the pulse wave waveform. In a case in which the first feature amount to the third feature amount are input, the computer inputs the first feature amount to the third feature amount thus acquired to a trained model trained so as to output an estimated value of an intracardiac pressure of the patient, thereby acquiring the estimated value of the intracardiac pressure of the patient.

Description

Information Processing Method, Program, and Information Processing Apparatus

[0001] The present disclosure relates to an information processing method, a program, and an information processing apparatus.

[0002] Techniques for non-invasively estimating the state of the heart from an electrocardiogram (ECG), heart sound waveform, and pulse wave waveform of a patient have been proposed. For example, in Patent Document 1, as cardiac cycle time intervals, left ventricular systolic time (LVST), left ventricular diastolic time (LVDT), atrial pre-diastolic filling time (PADT), acceleration of atrial filling time (AAFT), electromechanical activation time (QS1), QS2, pre-ejection period (PEP), right ventricular systolic time (RVST), left atrial systolic time (LAST), right atrial systolic time (RAST), right ventricular ejection fraction (RVEF), right ventricular diastolic time (RVDT), left atrial diastolic time (LADT), right atrial diastolic time (RADT), systolic time interval (PEP / LVST), etc. are proposed to be calculated.

[0003] U.S. Patent Application Publication No. 2020 / 170527

[0004] Based on an electrocardiogram, heart sound waveform, and pulse wave waveform, it is possible to calculate various feature quantities representing the state of the heart, and based on the calculated feature quantities, the state of the heart is estimated. Although many types of feature quantities can be calculated from an electrocardiogram, heart sound waveform, and pulse wave waveform, it is not easy to identify effective feature quantities to be referred to in order to appropriately estimate the state of the heart that a doctor or the like wants to grasp.

[0005] In one aspect, an object is to provide an information processing method or the like that can assist in determining the state of the heart based on feature quantities calculated from an electrocardiogram, heart sound waveform, or pulse wave waveform.

[0006] (1) The present disclosure is an information processing method in which a computer acquires an electrocardiogram, sound waveform, and pulse wave waveform of a patient, and acquires a first feature calculated by a first formula using the position of the first sound or second sound obtained from the sound waveform, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rising position obtained from the pulse wave waveform, a second feature calculated by a second formula using the maximum amplitude of the first sound obtained from the sound waveform or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the sound waveform or the area obtained by integrating the waveform of the second sound over time, and a third feature calculated by a third formula using at least elemental features obtained using the pulse wave waveform, and inputs the acquired first feature, second feature, and third feature into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure relating to the input first feature, second feature, and third feature when the first feature, second feature, and third feature are input.

[0007] (2) In the information processing method of (1) above, it is preferable that the elemental features obtained using at least the pulse wave waveform include the time difference between the notch position and the I sound position, the time difference between the notch position and the II sound position, the angle of the pulse wave, an index obtained from the velocity pulse wave, an index obtained from the acceleration pulse wave, an index obtained from the approximate pulse wave waveform of the pulse wave waveform, an index obtained from the pulse waves at two positions on the limbs at different distances from the heart, the time difference between the R wave position obtained from the electrocardiogram and the pulse wave start position, the time difference between the pulse wave rise position and the I sound position, or an index obtained from the amplitude of the pulse wave waveform or the area obtained by integrating the pulse wave waveform over time.

[0008] (3) In the information processing method described in (2) above, it is preferable that the index obtained from the velocity pulse wave includes time information and amplitude information of the velocity pulse wave, the index obtained from the acceleration pulse wave includes time information and amplitude information of the acceleration pulse wave, the index obtained from the approximate pulse wave waveform includes angular frequency and area information of the approximate pulse wave waveform, and the index obtained from the pulse waves at two locations on the limbs includes time information and amplitude information of the upper arm, ankle, and toe pulse waves.

[0009] (4) In the information processing method described in any of (1) to (3) above, a fourth feature representing the heart rate variation between R waves obtained from the electrocardiogram is acquired, the learning model is trained to output an estimated value of the patient's intracardiac pressure related to the input first feature, second feature, third feature, and fourth feature when the input first feature, second feature, third feature, and fourth feature are input, and the computer further performs a process to acquire an estimated value of the patient's intracardiac pressure by inputting the acquired first feature, second feature, third feature, and fourth feature to the learning model.

[0010] (5) In the information processing method described in any of (1) to (4) above, a fifth feature is obtained by an equation that uses the time difference between the R wave position obtained from the electrocardiogram and the I sound position obtained from the heart sound waveform, the learning model is trained to output an estimated value of the patient's intracardiac pressure related to the input first feature, second feature, third feature, and fifth feature when the input first feature, second feature, third feature, and fifth feature are input, and the computer further performs a process to obtain an estimated value of the patient's intracardiac pressure by inputting the acquired first feature, second feature, third feature, and fifth feature into the learning model.

[0011] (6) In the information processing method described in any of (1) to (5) above, it is preferable that the first feature, the second feature, and the third feature are representative values ​​of the values ​​calculated for each beat in the electrocardiogram, heart sound waveform, and pulse wave waveform.

[0012] (7) The present disclosure also relates to a program that causes a computer to perform the following process: acquire an electrocardiogram, sound wave, and pulse wave waveform of a patient; acquire a first feature calculated by a first equation using the position of the first sound or second sound obtained from the sound wave, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rise position obtained from the pulse wave waveform; acquire a second feature calculated by a second equation using the maximum amplitude of the first sound obtained from the sound wave or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the sound wave or the area obtained by integrating the waveform of the second sound over time; and acquire a third feature calculated by a third equation using at least elemental features obtained using the pulse wave waveform; and input the acquired first feature, second feature, and third feature into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure relating to the input first feature, second feature, and third feature when the first feature, second feature, and third feature are input.

[0013] (8) The present disclosure also relates to an information processing apparatus having a control unit, wherein the control unit acquires a patient's electrocardiogram, heart sound waveform, and pulse wave waveform, and calculates a first feature quantity using a first formula that uses the position of the first sound or second sound obtained from the heart sound waveform, the Q wave position obtained from the electrocardiogram, and the notch position and pulse wave rise position obtained from the pulse wave waveform, the maximum amplitude of the first sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the second sound over time This information processing device acquires a second feature calculated by a second equation using the divided area, and a third feature calculated by a third equation using at least the elemental features obtained using the pulse wave waveform, and then acquires the first feature, second feature, and third feature, and then acquires the first feature, second feature, and third feature, and then acquires the first feature, second feature, and third feature, and then acquires the second feature, second feature, and third feature, and then acquires the third feature, second feature, and third feature, and then acquires the first third feature, second feature, and third feature, and when the first feature, second feature, and third feature are input, the first feature, second feature, and third feature are input to the information processing device acquires the second feature calculated by a second equation using the divided area, and a third feature calculated by a third equation using elemental features obtained using at least the pulse wave waveform, and then acquires the first feature, second feature, and third feature, and the third feature is acquired by a learning model that has been trained to output an estimated value of the patient's intracardiac pressure related to the input first feature, second feature, and third feature.

[0014] (9) The present disclosure also relates to an information processing method in which a computer performs the following steps: when different combinations of feature candidate values ​​derived from any or more of the patient's electrocardiogram, heart sound waveform, and pulse wave waveform are input, a plurality of learning models are identified from among a plurality of learning models that estimate the patient's intracardiac pressure based on the estimation accuracy of each learning model, and a plurality of feature candidate values ​​common to the identified plurality of learning models are extracted from among the feature candidate values ​​input to the identified plurality of learning models.

[0015] (10) In the information processing method of (9) above, each feature candidate is classified into a plurality of categories, and it is preferable that the computer further performs a process of identifying a plurality of categories that are common among the specified plurality of learning models from among the categories into which the feature candidates input to the specified plurality of learning models are classified, and extracting a plurality of feature candidates that are classified into the specified categories.

[0016] (11) In the information processing method of (9) or (10) above, it is preferable that the computer further performs a process to generate a learning model that outputs an estimated value of the patient's intracardiac pressure when patient features based on the extracted plurality of candidate features are input.

[0017] (12) In the information processing method described in any of (9) to (11) above, the first feature quantity candidate is a first feature quantity calculated by a first formula using the position of the first sound or the second sound obtained from the heart sound waveform, the Q wave position obtained from the electrocardiogram, and the notch position and pulse wave rising position obtained from the pulse wave waveform, and the second feature quantity candidate is the maximum amplitude of the first sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the second sound over time The second feature is calculated by the second equation using the divided area, and the third feature candidate is a third feature calculated by the third equation using at least the elemental features obtained using the pulse wave waveform. Preferably, the computer further performs a process to generate a learning model that outputs an estimated value of the patient's intracardiac pressure when the first feature calculated by the first equation, the second feature calculated by the second equation, and the third feature calculated by the third equation are input from the patient's electrocardiogram, heart sound waveform, and pulse wave waveform.

[0018] (13) In the information processing method described in any of (9) to (12) above, it is preferable that the computer further performs a process to obtain an estimated value of the patient's intracardiac pressure by inputting the patient's features based on the multiple feature candidates into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure when the patient's features based on the multiple feature candidates are input.

[0019] In one respect, it can help determine the state of the heart based on features calculated from electrocardiograms, heart sound waveforms, or pulse wave waveforms.

[0020] This is a block diagram showing an example of the configuration of an information processing device. This is an explanatory diagram showing an example of the configuration of a learning model. This is an explanatory diagram showing an example of the record layout of a calculation formula DB. This is a graph showing examples of electrocardiogram, heart sound waveform, and pulse wave waveform. This is an explanatory diagram of an index obtained from the pulse wave. This is a flowchart showing an example of the intracardiac pressure estimation process procedure. This is an explanatory diagram showing an example of the record layout of a calculation formula list DB. This is an explanatory diagram showing an example of the record layout of a model DB. This is an explanatory diagram showing an example of the record layout of a feature DB. This is a flowchart showing an example of the feature collection process procedure. This is a flowchart showing an example of the learning model generation process procedure. This is a flowchart showing an example of the feature identification process procedure used for input data of a learning model.

[0021] The information processing method, program, and information processing apparatus described herein will be described in detail below based on the drawings illustrating their embodiments.

[0022] (Embodiment 1) An information processing device for estimating the intracardiac pressure of a patient based on an electrocardiogram (ECG), heart sound waveform, and pulse wave waveform measured from the patient will be described. In this embodiment, the patient may be, for example, a patient diagnosed with heart failure, but may also be a subject without heart disease. The electrocardiogram, heart sound waveform, and pulse wave waveform can be measured, for example, using a blood pressure pulse wave analyzer, and by using a blood pressure pulse wave analyzer, each data can be measured synchronously.

[0023] Figure 1 is a block diagram showing an example configuration of an information processing device. The information processing device 10 in this embodiment is a computer capable of various information processing and information transmission and reception, and is composed of, for example, a personal computer, a server computer, a workstation, etc. The information processing device 10 has a control unit 11, memory 12, storage unit 13, communication unit 14, input unit 15, display unit 16, and reading unit 17, etc., and each of these units is connected via a bus. The control unit 11 has one or more processors (arithmetic processing units) such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), or AI chip (AI semiconductor). The control unit 11 executes the processing that the information processing device 10 should perform by appropriately reading the program P stored in the storage unit 13 into the memory 12 and executing it. If the control unit 11 has multiple processors, each process may be executed by the same processor, or each process may be executed by different processors.

[0024] Memory 12 can be SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc. Memory 12 temporarily stores data generated in conjunction with calculations performed by the control unit 11.

[0025] The storage unit 13 is a non-volatile storage device such as a hard disk or SSD (Solid State Drive). The storage unit 13 stores the program P (program product, computer program) executed by the control unit 11 and various data necessary for the execution of program P. The storage unit 13 also stores a learning model M that has been trained on training data by machine learning, for example. The learning model M is intended to be used as a program module that constitutes artificial intelligence software. The learning model M performs predetermined calculations on input data and outputs the calculation results, and the storage unit 13 stores data such as the coefficients and thresholds of the function that defines this calculation as the learning model M. Instead of storing the learning model M in the storage unit 13, the information processing device 10 may access and read the learning model M from a server that stores it. Furthermore, the storage unit 13 stores the calculation formula DB 13a. The storage unit 13 may be composed of multiple storage devices, and part of the storage unit 13 may be other storage devices connected to the information processing device 10, or other storage devices that the information processing device 10 can communicate with.

[0026] The communication unit 14 is a communication module for processing wired or wireless communication, and transmits and receives information with other devices via a network. The network may be the Internet or a public telephone network, or a LAN (Local Area Network) built in a facility where the information processing device 10 is installed. The input unit 15 receives user input and sends control signals corresponding to the operation to the control unit 11. The display unit 16 is a liquid crystal display or an organic EL (Electro Luminescence) display, etc., and displays various information according to instructions from the control unit 11. The input unit 15 and the display unit 16 may be a touch panel configured as an integrated unit. Note that the input unit 15 and the display unit 16 are not essential, and the information processing device 10 may be configured to receive operations through a connected terminal device, or to output the information to be displayed to an external display device.

[0027] The reading unit 17 reads information stored on a portable storage medium 10a, including CDs (Compact Discs), DVDs (Digital Versatile Discs), USB (Universal Serial Bus) memory, SD (Secure Digital) cards, etc. The program P and various data to be stored in the storage unit 13 may be read by the control unit 11 from the portable storage medium 10a via the reading unit 17 and stored in the storage unit 13. Alternatively, the program P and various data may be written to the storage unit 13 during the manufacturing stage of the information processing device 10, or the control unit 11 may download them from other devices via the communication unit 14 and store them in the storage unit 13.

[0028] The information processing device 10 is not limited to a single computer, but may be a multicomputer system comprising multiple computers. The information processing device 10 may also be a virtual machine virtually constructed within a single device by software. Furthermore, the information processing device 10 may be a local server installed within the facility where the information processing device 10 is located, or it may be a cloud server connected via a network. In the following description, the information processing device 10 will be described as a single computer. Also, the program P may be deployed and executed on a single computer or at a single site, or it can be distributed across multiple sites and deployed to run on multiple computers interconnected via a communication network.

[0029] In this embodiment, the information processing device 10 calculates multiple types of feature quantities that represent the state of the heart based on measurement data of an electrocardiogram, heart sound waveform, and pulse wave waveform measured from a patient. Then, the information processing device 10 estimates the intracardiac pressure of the patient based on the multiple types of feature quantities that have been calculated. The feature quantities used for estimating intracardiac pressure will be described later. In this embodiment, the patient's electrocardiogram, heart sound waveform, and pulse wave waveform are measured in advance, and the obtained measurement data is stored in an electronic medical record system (not shown), for example. Therefore, the information processing device 10 calculates feature quantities and estimates intracardiac pressure for the patient by acquiring the measurement data of the patient to be processed from the electronic medical record system. The electrocardiogram, heart sound waveform, and pulse wave waveform are measured using a measuring device such as a blood pressure and pulse wave analyzer installed in the patient's home, workplace, nursing home, medical institution, etc. The measuring device has a communication function and is configured to transmit the measurement data to the electronic medical record system or the information processing device 10. In addition to having a communication function, the measuring device may also be configured to be able to connect to a communication terminal and transmit measurement data to the electronic medical record system or information processing device 10 via the connected communication terminal.

[0030] The measuring device has the function of measuring an electrocardiogram (ECG), which is a graph of the electrical signals of the heart, by attaching electrodes for ECG measurement to both wrists, both ankles, and the chest, and detecting minute electrical signals generated from the heart through the electrodes. The measuring device also has the function of measuring heart sounds by attaching a microphone for measuring heart sounds to the chest, and detecting the sound of the heartbeat through the microphone and converting it into an electrical signal. Furthermore, the measuring device has the function of measuring pulse waves, which represent the speed at which the heartbeat is transmitted and the strength of the pulse, by attaching blood pressure cuffs to both arms and both ankles, and measuring the blood pressure of the limbs. With such a configuration, the measuring device can simultaneously measure the ECG, heart sounds, and pulse waves, and can acquire synchronized measurement data for the ECG, heart sounds, and pulse waves. Alternatively, the measuring device may have a sensor unit in which an ECG (Electrocardiogram) sensor for measuring the ECG, a heart sound sensor for measuring heart sounds, and a pulse wave sensor for measuring pulse waves are housed in a housing. The sensor unit can simultaneously measure the electrocardiogram, heart sound waveform, and pulse wave waveform by being positioned or fixed near the heart in the patient's chest. If the sensor unit does not have a housing, the ECG sensor, heart sound sensor, and pulse wave sensor may be positioned or fixed at designated locations for measurement. The measuring device may also have a configuration that measures the pulse wave using photoplethysmography (PPG) by transmitted pulse measurement using transmitted light or reflected pulse wave measurement using reflected light.

[0031] Figure 2 is an explanatory diagram showing an example of the configuration of the learning model M. The learning model M shown in Figure 2 is a model that takes multiple types of features representing the state of the heart as input, performs calculations to estimate the intracardiac pressure of the patient based on the input data, and outputs the calculation result (estimated value of intracardiac pressure). The multiple types of features are features calculated using predetermined calculation formulas (for example, calculation formulas registered in calculation formula DB13a) from measurement data of electrocardiogram, heart sound waveform, and pulse wave waveform measured from the patient. In the example in Figure 2, features from six categories (Category 1 to Category 6) are used as input data. In addition, in the example in Figure 2, three features are input as features of Category 1 (PEP component), two features as features of Category 2 (heart sound amplitude component), three features as features of Category 3 (vascular sclerosis index), one feature as features of Category 4 (heart rate variability), one feature as features of Category 5 (ECG heart sound time difference component), and two features as features of Category 6 (pulse wave amplitude component). The learning model M is not limited to a configuration in which features from the six categories described above are input; it may also be configured to input features from at least one of the six categories, or to input features from categories other than the six described above. Furthermore, the number and types of features in each category are not limited to the example in Figure 2.

[0032] The learning model M of this embodiment is configured to estimate pulmonary capillary wedge pressure (PCWP) as intracardiac pressure. Specifically, the learning model M outputs an estimated value of the diastolic PCWP value as an estimated value of PCWP in a single heartbeat. The learning model M may also be configured to output estimated values ​​of the systolic PCWP value and the average value of the PCWP value in addition to the diastolic PCWP value. In this case, a learning model that outputs an estimated value of the diastolic PCWP, a learning model that outputs an estimated value of the systolic PCWP, and a learning model that outputs the average value of the PCWP may be provided separately. Furthermore, the learning model M may be configured to estimate time-series data showing the time course of PCWP by inputting time-series data of each feature, and output it as waveform data.

[0033] In addition to PCWP, intracardiac pressure may also be measured using other pulmonary wedge pressure parameters (features) such as pulmonary artery wedge pressure (PAWP) and pulmonary artery occlusion pressure (PAOP). In addition to PWP, other parameters (features) such as pulmonary artery pressure (PAP), right atrium pressure (RAP), right ventricular pressure (RVP), left atrium pressure (LAP), left ventricular pressure (LVP), left ventricular end-diastolic pressure (LVEDP), pulmonary artery end-diastolic pressure (PAEDP), pulmonary artery diastolic pressure (PADP), and central venous pressure (CVP) may also be used to measure intracardiac pressure.

[0034] The learning model M can be constructed using algorithms such as XGBoost (eXtreme Gradient Boosting), decision trees, SVM (Support Vector Machine), random forests, CNN (Convolutional Neural Network), and Transformers, or it may be constructed by combining multiple algorithms.

[0035] The learning model M is generated by machine learning using training data that associates the features of each category for training with the correct intracardiac pressure (PCWP in this embodiment). The correct intracardiac pressure (PCWP) can be measured using data obtained from a catheterization procedure using a Swan-Ganz catheter, for example. In a catheterization procedure using a Swan-Ganz catheter, it is possible to measure items (features) other than PCWP as intracardiac pressure, and by using any of the measurement data as the correct intracardiac pressure, a learning model capable of estimating the value of that item (feature) can be realized. The features of each category for training can be calculated using measurement data of electrocardiogram, heart sound waveform, and pulse wave waveform measured simultaneously with the catheterization procedure using a blood pressure pulse wave analyzer.

[0036] The learning model M learns to output the correct intracardiac pressure when it receives feature quantities from each category in the training data. Specifically, the learning model M performs calculations based on the input feature quantities to estimate the intracardiac pressure of the patient for which the feature quantity was calculated, and obtains the estimation result (estimated intracardiac pressure). Then, the learning model M compares the estimated intracardiac pressure with the correct intracardiac pressure and optimizes parameters such as node weights (connection coefficients) so that the two approximate each other. Methods for optimizing parameters can include the steepest descent method and backpropagation. This results in a learning model M that outputs an estimated value of the patient's intracardiac pressure when it receives feature quantities from each category calculated from the patient's electrocardiogram, heart sound waveform, and pulse wave waveform measurement data.

[0037] The estimated intracardiac pressure output by the learning model M shown in Figure 2 is a continuous value, and the learning model M outputs one of the continuous values ​​as a so-called regression problem. However, it may also be configured to treat it as a classification problem, and to determine whether the estimated intracardiac pressure is a selectable numerical value or a numerical range (a numerical value or numerical range that can be taken as intracardiac pressure). In this case, the learning model M determines the optimal numerical value or numerical range from a set of multiple numerical values ​​or numerical ranges for the estimated intracardiac pressure and outputs the determination result.

[0038] The learning model M may be trained using another learning device. The trained learning model M generated by training using another learning device is downloaded from the learning device to the information processing device 10 via a network or portable storage medium 10a and stored in the storage unit 13. The learning model M is not limited to the configuration shown in Figure 2, and can be configured, for example, to estimate intracardiac pressure other than PCWP. Furthermore, the learning model M is not limited to a configuration in which six categories of feature quantities are input, but can be configured in which one to five or seven or more categories of feature quantities are input.

[0039] The information processing device 10 prepares the learning model M described above in advance, and, for example, when a doctor or other medical professional wants to know the patient's intracardiac pressure, it uses the learning model M to estimate the intracardiac pressure based on features calculated from the patient's electrocardiogram, heart sound waveform, and pulse wave waveform measurement data. Intracardiac pressure is an indicator of the state of cardiac function, and it is known that in heart failure, intracardiac pressure rises before subjective symptoms appear. Heart failure can be caused by myocardial infarction, angina pectoris, arteriosclerosis, hypertension, valvular heart disease, cardiomyopathy, or arrhythmias. Therefore, by monitoring intracardiac pressure, it is possible to detect the worsening of heart failure at an earlier stage than when subjective symptoms appear. By referring to the estimated intracardiac pressure along with the measurement data, doctors and other medical professionals can understand the patient's condition and enable the early detection of patients suspected of having worsening heart failure.

[0040] Next, the feature quantities used for estimating the intracardiac pressure will be described. FIG. 3 is an explanatory diagram showing an example of the record layout of the calculation formula DB13a. The calculation formula DB13a is a database in which calculation formulas used for calculating feature quantities used for estimating the intracardiac pressure are registered. In the calculation formula DB13a of FIG. 3, calculation formulas used for calculating each feature quantity input to the learning model M shown in FIG. 2 are registered. The calculation formula DB13a shown in FIG. 3 includes a category column, a feature quantity name column, and a calculation formula column, and stores the feature quantity name and the calculation formula for each feature quantity classified into each category. In the example of FIG. 3, information on feature quantities of the first category (PEP component), the second category (heart sound amplitude component), the third category (arteriosclerosis index), the fourth category (heartbeat fluctuation), the fifth category (electrocardiogram-heart sound time difference component), and the sixth category (pulse wave amplitude component) is stored, but the configuration is not limited to this, and information on feature quantities that are input data of the learning model M is stored.

[0041] The PEP component is a category of feature quantities related to the pre-ejection period from the start of heart contraction to the opening of the aortic valve. The feature quantity (the first feature quantity) of this category is calculated by a calculation formula (the first formula) using the I sound position or the II sound position obtained from the heart sound waveform, the Q wave position obtained from the electrocardiogram, and the notch position and the pulse wave rising position obtained from the pulse wave waveform. For example, the feature quantity of "Q_S2_PEP_ratio" shown in FIG. 3 is calculated by the calculation formula of (II sound position - Q wave position) / ((II sound position - Q wave position) - (notch position - rising position)). "II sound position - Q wave position" and "notch position - rising position" in this calculation formula are sometimes called element feature quantities. Feature points and indexes in the electrocardiogram, heart sound, and pulse wave used for calculating the feature quantity and the element feature quantity will be described later.

[0042] The heart sound amplitude component is a category of feature quantities related to the amplitude of the heart sound. The feature quantity (the second feature quantity) of this category is calculated by a calculation formula (the second formula) using the maximum amplitude of the I sound obtained from the heart sound waveform or the area obtained by time-integrating the waveform of the I sound, or the maximum amplitude of the II sound or the area obtained by time-integrating the waveform of the II sound. For example, the feature quantity of "shannon_max_diff" shown in FIG. 3 is calculated by the calculation formula of "I sound maximum amplitude - II sound maximum amplitude".

[0043] The vascular sclerosis index is a category of feature quantities related to the degree of arteriosclerosis. The feature quantities in this category (the third feature quantity) are calculated by a calculation formula (the third formula) using at least element feature quantities obtained using a pulse wave form. For example, the feature quantity of "tb_S1_tb_ratio" shown in FIG. 3 is calculated by a calculation formula of "(notch position - first heart sound position) / (notch position - second heart sound position)". The "notch position - first heart sound position" and "notch position - second heart sound position" in this calculation formula may also be referred to as element feature quantities. In addition, the element feature quantities obtained using a pulse wave form include the angle of the pulse wave, indices obtained from the velocity pulse wave, indices obtained from the acceleration pulse wave, indices obtained from the approximate shape (approximate pulse wave form) of the pulse wave, indices obtained from pulse waves at two positions in the limbs with different distances from the heart, the time difference between the R wave position obtained from an electrocardiogram and the start position of the pulse wave, the time difference between the rising position of the pulse wave and the first heart sound position, indices obtained from the amplitude of the pulse wave or the area obtained by time-integrating the pulse wave form, and the like.

[0044] Heart rate variability is a category of feature quantities related to the variation in the heart rate interval (the time between R waves). The feature quantities in this category (the fourth feature quantity) are calculated based on the variation between R waves obtained from an electrocardiogram. For example, the feature quantity of "RMSSD" shown in FIG. 3 is calculated by the square root of the average value of the squares of the differences in the intervals (RR intervals) between continuously appearing R waves.

[0045] The electrocardiogram-phonocardiogram time difference component is a category of feature quantities related to the time difference component using the R wave obtained from an electrocardiogram. The feature quantities in this category (the fifth feature quantity) are calculated by a formula using the time difference between the R wave position obtained from an electrocardiogram and the first heart sound position obtained from the phonocardiogram waveform. For example, the feature quantity of "ECG_PCG_diff" shown in FIG. 3 is calculated by a calculation formula of "first heart sound position - R wave position".

[0046] The pulse wave amplitude component is a category of features related to the amplitude of the pulse wave, and the features in this category are calculated using a formula that utilizes elemental features obtained from the pulse wave waveform. For example, the features of "RHDN_RRL" shown in Figure 3 are calculated using the formula (height of notch position - height of pulse wave end position) / (height of systolic peak - height of pulse wave rise position). In this formula, "height of notch position - height of pulse wave end position" and "height of systolic peak - height of pulse wave rise position" are sometimes also called elemental features.

[0047] Here, we will explain the feature points and indices of the electrocardiogram, heart sounds, and pulse wave used to calculate each feature and each element feature. Feature points are areas in the waveform data that should be focused on in order to understand the movement of the heart. Figure 4 is a graph showing examples of electrocardiogram, heart sound waveform, and pulse wave waveform. The measurement data for the electrocardiogram, heart sound waveform, and pulse wave waveform is the waveform data of the electrocardiogram, heart sounds, and pulse wave obtained through measurement, and it is desirable that the waveform data format be a general-purpose format such as CSV (Comma Separated Values) format or MFER (Medical waveform Format Encoding Rule) format. The data for each point that makes up the waveform data is associated with time information and can be synchronized between each waveform.

[0048] An electrocardiogram (ECG) shows waveforms called the P wave, QRS wave, T wave, and U wave with each heartbeat. The P wave represents the excitation (contraction) of the atria. The first half of the P wave shows the excitation of the right atrium, and the second half shows the excitation of the left atrium, and these combine to form the P wave. The QRS wave represents the excitation of the ventricles. The first downward waveform that appears is called the Q wave and indicates the beginning of ventricular excitation. The following upward waveform is called the R wave and indicates the point where the ventricle is most contracted. The last downward waveform that appears is called the S wave and indicates the end of ventricular excitation. The T wave represents the disappearance of ventricular excitation and the return to normal. The T wave is characterized by its bell-shaped appearance. The U wave is a small waveform that appears following the T wave. The U wave may not be seen in normal individuals. Characteristic features of an electrocardiogram (ECG) include the position of the Q wave and the position of the R wave. The Q wave position is, for example, the point indicating the beginning of the Q wave, and the R wave position is, for example, the point indicating the peak of the R wave. Other indicators in an ECG include, for example, the interval between R waves (RR interval), which is, for example, the interval between consecutively appearing R wave peaks.

[0049] The characteristic points of the heart sound waveform are the start point of the first heart sound, the peak point of the first heart sound, the start point of the second heart sound, and the second heart sound. A point, and II P These are the sounds. The first heart sound (I) is a sound that occurs during ventricular contraction and mainly consists of the mitral valve closing sound and the aortic valve opening sound. The second heart sound (II) is a sound that occurs at the beginning of ventricular diastole and mainly consists of the aortic valve closing sound (II). A ), pulmonary valve closure sound (II P The position of note I used to calculate the feature and element features may be either the start point of note I or the peak point of note I, and the position of note II may be either the start point of note II or the peak point of note II. A Even if it's a point, II P It may also be a point. Preferably, the points to be used as the positions of the first and second heart sounds are predetermined. Other indicators in the heart sound waveform include the maximum amplitude of the first heart sound, the maximum amplitude of the second heart sound, the area obtained by integrating the waveform of the first heart sound over time, the maximum amplitude of the second heart sound, or the area obtained by integrating the waveform of the second heart sound over time.

[0050] A pulse wave is composed of components such as the ejection wave (PW: Percussion Wave) from the heart, reflected waves (TW: Tidal Wave) from the blood vessel wall, dicrotic notch (DN: Dicrotic Notch), and dicrotic wave (DW: Dicrotic Wave). Characteristic features of a pulse wave include the start position, rising point (US (Up-Stroke) point), peak position, falling point, notch position (DN point), and end position. The start position of a pulse wave is where the slope of the pulse wave becomes positive, the rising point is where the rising angle of the pulse wave is maximum, the peak position is where the pulse wave is maximum, the falling point is where the slope of the pulse wave becomes negative from the peak position, the notch position is the position of the notch, and the end position is the start position of the next pulse wave. Other indicators of the pulse wave include the height at the start of the pulse wave, the height at the rise point, the height at the end of the pulse wave, the height at the peak of the systolic phase, the height at the notch point, the angle of the pulse wave (the rise angle of the systolic phase and the fall angle of the diastolic phase), the amplitude of the pulse wave waveform, and the area obtained by integrating the pulse wave waveform over time.

[0051] Figure 5 is an explanatory diagram of the indices obtained from pulse waves. The upper part of Figure 5 shows an example of a pulse wave (fingerpigmentation volume plethysmography) with a solid line, where the angle indicated by sys_angle is the rising angle during systole and the angle indicated by dia_angle is the falling angle during diastole. The middle part of Figure 5 shows an example of a velocity pulse wave obtained by first differentiation of the fingerpigmentation volume plethysmography, and the lower part of Figure 5 shows an example of an acceleration pulse wave obtained by second differentiation of the fingerpigmentation volume plethysmography. The indices obtained from velocity pulse waves include temporal and amplitude information of the velocity pulse wave, such as the maximum value (maximum velocity, height of the positive peak) and minimum value (minimum velocity, height of the negative peak) of the velocity pulse wave. The indices obtained from acceleration pulse waves include temporal and amplitude information of the acceleration pulse wave, such as the maximum value (amplitude of wave a (initial positive wave of systole), amplitude of wave e (initial positive wave of diastole)) and minimum value (amplitude of wave b (initial negative wave of systole)) of the acceleration pulse wave. As shown in the upper part of Figure 5, the approximate pulse wave waveform is a waveform obtained by replacing the systolic and diastolic pulse wave waveforms with approximate formulas, respectively. The approximate systolic pulse wave waveform is shown by a dashed line, and the approximate diastolic pulse wave waveform is shown by a dashed line. The approximate systolic pulse wave waveform can be expressed by, for example, the following formula (1), and the approximate diastolic pulse wave waveform can be expressed by, for example, the following formula (2). In the following formulas (1) and (2), A, B, c, P1(y), and P2(y) are coefficients, and t is time. The indices obtained from the approximate pulse wave waveform include ω1 and ω2 included in the following formulas (1) and (2). These ω1 and ω2 are called angular frequencies and are used as indices in the pulse wave to calculate feature quantities. In addition, other indices in the pulse wave include the area obtained by integrating the approximate pulse wave waveform over time. f1 = A sin(ω1t) + c - P1(y) …(1) f2 = B sin(ω2t) + c - P2(y) …(2)

[0052] An index obtained from pulse waves at two locations on the limbs at different distances from the heart includes the speed at which the pulse wave propagates between the two locations (pulse wave velocity, PWV), and is calculated, for example, by comparing the notch positions of the pulse wave waveforms at the upper arm and ankle from the heart. In addition to the propagation velocity between the upper arm and ankle, PWV may also be calculated using the propagation velocity between any two points on the limbs at different distances from the heart, such as the propagation velocity between the upper arm and toes, the propagation velocity between the upper arm and the back of the knee, or the propagation velocity between the upper arm and the wrist. Furthermore, an index obtained from pulse waves at two locations on the limbs may also include higher-order indices such as CAVI (Cardio-Ankle Vascular Index) and ABI (Ankle Brachial Index) calculated from PWV.

[0053] The values ​​indicating feature points in the electrocardiogram, heart sound waveform, and pulse wave waveform use relative time, which represents the elapsed time to the feature point, with the start of the waveform being time 0. If the time length between two feature points can be calculated, absolute time may be used for the value indicating each feature point. The feature points and indices of the electrocardiogram, heart sound, and pulse wave are not limited to the examples described above, and indices indicating the temporal relationship (time difference) between feature points in the electrocardiogram, heart sound waveform, and pulse wave waveform, and indices indicating the temporal relationship (time difference) between feature points in at least two measurement data of the electrocardiogram, heart sound waveform, and pulse wave waveform can be used. Furthermore, the feature quantities and elemental features calculated using the feature points and indices of the electrocardiogram, heart sound, and pulse wave as described above are not limited to the examples shown in Figure 3. Note that the feature points and indices of the electrocardiogram, heart sound, and pulse wave as described above are commonly used in the field of cardiovascular medicine, so an explanation of their medical meaning will be omitted. The contents of calculation formula DB13a are not limited to the example shown in Figure 3, and the number and types of categories, the number of features in each category, and the calculation formulas for each feature are not limited to the example in Figure 3.

[0054] The following describes the process by which the information processing device 10 estimates a patient's intracardiac pressure. Figure 6 is a flowchart showing an example of the intracardiac pressure estimation procedure. The control unit 11 acquires electrocardiogram, heart sound, and pulse wave measurement data for the patient whose intracardiac pressure is to be estimated (S11). For example, the control unit 11 accesses the electronic medical record system via a network to acquire the measurement data for the patient. The method of acquiring measurement data is not limited to this, and the control unit 11 may acquire measurement data via a network from a measuring device such as a blood pressure pulse wave analyzer, accept measurement data input via the input unit 15, or read measurement data stored in a portable storage medium 10a using the reading unit 17.

[0055] The control unit 11 divides the acquired measurement data into individual beats (S12) and extracts the measurement data for one beat (S13). The measurement data for the electrocardiogram, heart sound waveform, and pulse wave waveform is, for example, data measured for about 15 seconds, and the 15-second measurement data contains measurement data for 15 to 20 beats. Therefore, the control unit 11 divides the measurement data into individual beats and calculates the value of each feature for each beat by executing the processing from step S14 onward based on the measurement data for one beat. Note that the process of dividing the measurement data into individual beats can be performed, for example, by pattern matching using the electrocardiogram, heart sound waveform, and pulse wave waveform data for one beat.

[0056] The control unit 11 reads out one feature calculation formula from the calculation formulas registered in the calculation formula DB 13a (S14). Based on the extracted measurement data for one beat, the control unit 11 extracts feature points from the electrocardiogram, heart sound waveform, and pulse wave waveform to be used in the read calculation formula (S15). In addition to the feature points included in the read calculation formula, the control unit 11 also extracts indices in the electrocardiogram, heart sound waveform, and pulse wave waveform, as well as feature points and indices used to calculate the element features included in the calculation formula. Based on the feature points and indices in the electrocardiogram, heart sound waveform, and pulse wave waveform, the control unit 11 calculates each element feature included in the read calculation formula, and uses the calculated element features to calculate the feature (feature value) based on the read calculation formula (S16). Furthermore, for feature quantities calculated based on measurement data for multiple beats (feature points and indices for multiple beats), the control unit 11 may calculate the average value of the feature quantity for the measurement data for that single beat, for example, the average value of "Q_S2_PEP_ratio", using the feature points and indices for the previous one or more beats in the time series, or the feature points and indices for the subsequent one or more beats in the time series, in addition to the feature points and indices for that single beat. Alternatively, it may calculate the value of the feature quantity for the measurement data for multiple beats, for example, the variance of heart rate variability. The control unit 11 associates the calculated feature quantity with identification information (for example, the feature quantity name or the ID assigned to the feature quantity) and stores it in memory 12, for example. Based on all the calculation formulas registered in the calculation formula DB 13a, the control unit 11 determines whether or not the calculation of all feature quantities to be calculated has been completed (S17). If it determines that it has not been completed (S17: NO), it returns to step S14 and executes the processing in steps S14 to S16 for the uncalculated feature quantities. As a result, the control unit 11 performs the extraction of feature points to be used in the calculation formula, the calculation of elemental features, and the calculation of features for the uncalculated features.

[0057] If the control unit determines that the calculation of all features has been completed (S17: YES), it determines whether or not it has finished processing all of the acquired measurement data (S18). If it determines that it has not finished processing (S18: NO), it returns to step S13 and repeats the processing in steps S13 to S17 for the unprocessed measurement data for one beat. If the control unit determines that it has finished processing all of the measurement data (S18: YES), it calculates a representative value for each feature based on the values ​​of each feature calculated in step S16 and stored in memory 12 (S19). For example, for each feature, the control unit calculates the average or median value of the values ​​for each beat as the representative value.

[0058] The control unit 11 estimates the patient's intracardiac pressure based on the representative values ​​of the calculated features (S20). Here, the control unit 11 inputs the representative values ​​of the calculated features into the learning model M and obtains the estimated value of the patient's intracardiac pressure as the output value from the learning model M. The control unit 11 stores the estimated value of intracardiac pressure in the memory 12 or storage unit 13, associating it with the patient's information (e.g., patient ID, medical card number) (S21). Through the above-described process, the information processing device 10 of this embodiment can calculate the feature quantities of each category in the pulse for each beat of measurement data and estimate the intracardiac pressure using the representative values ​​calculated based on the feature quantities of multiple beats. In the above-described process, the values ​​of each feature quantity in the electrocardiogram, heart sound waveform, and pulse wave waveform are calculated for each beat, and the intracardiac pressure of the patient is estimated using the representative values ​​of each feature quantity calculated from the values ​​of each feature quantity, but the configuration is not limited to this. For example, the system may be configured to estimate the intracardiac pressure for each beat based on the values ​​of each feature in that beat for each measurement data point. In this case, the control unit 11 stores the estimated intracardiac pressure obtained for each beat in the memory 12 or storage unit 13, associating it with the time information of the measurement data. The time information may be, for example, the elapsed time from the start of the measurement data, or it may be information indicating which beat the measurement data for the beat being processed belongs to. In such a configuration, the estimated intracardiac pressure for each beat can be displayed as time-series data on, for example, the display unit 16 and presented to a doctor or other person. The control unit 11 may also calculate and present the average or median value of the intracardiac pressure for each beat based on the estimated intracardiac pressure obtained for each beat.

[0059] In this embodiment, the PEP component is designated as the first category, the heart sound amplitude component as the second category, the vascular sclerosis index as the third category, the heart rate variability as the fourth category, the electrocardiogram heart sound time difference component as the fifth category, and the pulse wave amplitude component as the sixth category. The feature quantities of each category are input to the learning model M, and the learning model M estimates the patient's intracardiac pressure based on the feature quantities of each category. However, the embodiment is not limited to this configuration. For example, the feature quantities of at least the PEP component (first category), the heart sound amplitude component (second category), and the vascular sclerosis index (third category) may be input, and the learning model M may be configured to estimate the patient's intracardiac pressure based on the feature quantities of the first to third categories. Alternatively, in addition to the first to third categories, the feature quantities of at least one of the fourth category (heart rate variability), fifth category (electrocardiogram heart sound time difference component), and sixth category (pulse wave amplitude component) may be input, and the learning model M may be configured to estimate the patient's intracardiac pressure based on the input feature quantities. In this case, by inputting each feature of the input data into the learning model M, it is possible to obtain an estimated value of the patient's intracardiac pressure output by the learning model M.

[0060] In this embodiment, the process of estimating the patient's intracardiac pressure based on multiple categories of feature quantities calculated from measurement data of electrocardiogram, heart sound waveform, and pulse wave waveform is not limited to being performed locally by the information processing device 10. For example, a server may be provided that performs the process of estimating the patient's intracardiac pressure using a learning model M. In this case, the information processing device 10 can be configured to send representative values ​​of the feature quantities of each category calculated in step S19 to the server in step S20 in Figure 6, and to receive estimated values ​​of the patient's intracardiac pressure estimated by the server based on the representative values ​​of the feature quantities of each category.

[0061] (Embodiment 2) An information processing device 10 for identifying feature quantities to be used in the input data of the learning model M will be described. The information processing device 10 of this embodiment has the same configuration as the information processing device 10 of Embodiment 1 shown in Figure 1, so the explanation of the configuration will be omitted. In addition to the configuration shown in Figure 1, the information processing device 10 of this embodiment stores a calculation formula list DB, a model DB, and a feature quantity DB in the storage unit 13.

[0062] The information processing device 10 of this embodiment determines multiple sets of different combinations of feature quantities to be used as input data for a learning model, selected from a plurality of categories of feature quantities derived based on one or more of the electrocardiogram, heart sound waveform, and pulse wave waveform. The number of feature quantities included in each set may be the same or different, and each set of feature quantities may include at least one of the categories shown in Figure 3. The information processing device 10 uses the feature quantities included in each set as input data to generate a learning model having the same configuration as the learning model M of Embodiment 1. The information processing device 10 then uses each generated learning model to perform intracardiac pressure estimation processing from the patient's electrocardiogram, heart sound, and pulse wave measurement data, and calculates the estimation accuracy for each learning model. Based on the estimation accuracy calculated for each learning model, the information processing device 10 identifies learning models whose estimation accuracy is above a predetermined value, and determines the feature quantities common to the identified learning models to be used as input data for the learning model M. By generating a learning model M using the feature quantities thus determined as input data, the processing of Embodiment 1 described above becomes possible, and the same effect is obtained.

[0063] Figure 7 is an explanatory diagram showing an example of the record layout of the calculation formula list DB13b. The calculation formula list DB13b is a database in which calculation formulas for various feature quantities that can be derived based on one or more of the electrocardiogram, heart sound waveform, and pulse wave waveform are registered. In addition to the configuration of the calculation formula list DB13a shown in Figure 3, the calculation formula list DB13b shown in Figure 7 has a feature quantity ID column, and stores the category, feature quantity name, and calculation formula for each feature quantity in association with the identification information (feature quantity ID) assigned to each feature quantity. The calculation formula list DB13b has a large number of feature quantities that can be used to estimate intracardiac pressure predetermined and stored.

[0064] Figure 8A is an explanatory diagram showing an example of the record layout of the model DB 13c, and Figure 8B is an explanatory diagram showing an example of the record layout of the feature DB 13d. The model DB 13c is a database in which sets of multiple features to be used as input data for each learning model to be generated are registered. The model DB 13c shown in Figure 8A has a model ID column and a 1st category column to a 6th category column, and stores the feature IDs assigned to the features of each category used as input data for each learning model, in association with the identification information (model ID) assigned to the learning model to be generated.

[0065] The Feature Database 13d is a database in which each feature calculated based on the patient's measurement data using the calculation formulas for each feature registered in the Calculation Formula List Database 13b is registered. The Feature Database 13d shown in Figure 8B has a Data ID column, a Feature ID column, and an Intracardiac Pressure column, and stores the values ​​of each feature calculated based on the measurement data and the intracardiac pressure measured together with the measurement data, associated with the Data ID assigned to the patient's measurement data. In this embodiment, for example, PCWP is used for intracardiac pressure. The Data ID is assigned, for example, to one beat's worth of measurement data. The Feature Database 13d stores each feature calculated when each feature is calculated based on the patient's measurement data. In this embodiment, each feature stored in the Feature Database 13d is used as information for the training data of the learning model.

[0066] The following describes the process of collecting features used in the training data of a learning model. Figure 9 is a flowchart showing an example of the feature collection procedure. The following process is performed by the control unit 11 of the information processing device 10 according to the program P stored in the storage unit 13, but it may also be performed by other learning devices.

[0067] The control unit 11 acquires electrocardiogram, heart sounds, and pulse wave measurement data, as well as intracardiac pressure, which can be used as training data for the learning model (S31). The control unit 11 accesses the electronic medical record system via the network and acquires electrocardiogram, heart sounds, and pulse wave measurement data, as well as intracardiac pressure, for a specified patient. Here too, the measurement data includes measurement data for multiple beats, so the control unit 11 divides the measurement data into individual beats (S32) and extracts the measurement data for one beat (S33).

[0068] The control unit 11 reads a calculation formula for one feature from the calculation formulas registered in the calculation formula list DB 13b (S34). Based on the extracted measurement data for one beat, the control unit 11 extracts feature points and indices from the feature points and indices in the electrocardiogram, heart sound waveform, and pulse wave waveform to be used in the read calculation formula (S35). Based on the feature points and indices in the electrocardiogram, heart sound waveform, and pulse wave waveform, the control unit 11 calculates a feature using the read calculation formula (S36), and stores the calculated feature in the feature database 13d, associating it with the data ID assigned to the measurement data and the feature ID of the feature (S37). For feature calculated based on measurement data for multiple beats (feature points and indices for multiple beats), the control unit 11 calculates the feature for the measurement data for one beat, or the feature for the measurement data for multiple beats, using, for example, the feature points and indices of the previous one or more beats in the time series, in addition to the feature points and indices for the one beat.

[0069] The control unit 11 determines whether or not the calculation of all features registered in the calculation formula list DB 13b has been completed (S38). If it determines that it has not been completed (S38: NO), it returns to step S34 and executes the processes of steps S34 to S37 for the uncalculated features. As a result, the control unit 11 extracts feature points to be used in the calculation formula, calculates the features, and stores them in the feature DB 13d for the uncalculated features. If it determines that the calculation of all features has been completed (S38: YES), the control unit 11 stores the intracardiac pressure obtained in step S31 in the feature DB 13d, associating it with the data ID assigned to the measurement data (S39).

[0070] The control unit 11 determines whether or not it has finished processing all of the acquired measurement data (S40). If it determines that it has not finished (S40: NO), it returns to step S33 and repeats the processing in steps S33 to S39 for the unprocessed measurement data for one beat. If it determines that it has finished processing all of the acquired measurement data (S40: YES), the control unit 11 determines, for example, whether or not there is any unprocessed measurement data among the electrocardiogram, heart sound, and pulse wave measurement data stored in the electronic medical record system that can be used as training data for the learning model (S41).

[0071] If the control unit 11 determines that there is unprocessed measurement data (S41: YES), it returns to step S31 and executes the processing in steps S31 to S40 for the unprocessed measurement data. If the control unit 11 determines that there is no unprocessed measurement data (S41: NO), it terminates the processing. Through the processing described above, the information processing device 10 of this embodiment calculates each feature using the calculation formulas registered in the calculation formula list DB 13b for each measurement data of electrocardiogram, heart sound, and pulse wave measurement data that can be used as training data, and stores them in the feature DB 13d. In addition, the intracardiac pressure acquired together with the measurement data is stored in the feature DB 13d along with each calculated feature. This makes it possible to use the features and intracardiac pressure stored in the feature DB 13d as training data used in the learning process when generating a learning model.

[0072] Next, we will explain the process of creating training data for the learning model to be generated using the feature DB 13d, in which each feature and intracardiac pressure are stored by the process in Figure 9, and then generating the learning model using the created training data. Figure 10 is a flowchart showing an example of the learning model generation process procedure. The following process is performed by the control unit 11 of the information processing device 10 according to the program P stored in the storage unit 13, but it may also be performed by another learning device. In the process shown in Figure 10, the control unit 11 of the information processing device 10 first generates training data, and then trains the learning model using the generated training data.

[0073] The control unit 11 identifies the features used in the input data of the learning model to be generated (S51). For example, the control unit 11 identifies one of the learning models (specifically, model IDs) registered in the model DB 13c and reads the feature ID of the feature set in the input data of the identified learning model. The control unit 11 reads the feature of the read feature ID and the intracardiac pressure corresponding to this feature from among the features of one measurement data (data ID) stored in the feature DB 13d (S52). Then, the control unit 11 creates training data by associating the read intracardiac pressure with the feature of each read feature ID as the correct intracardiac pressure (S53). The control unit 11 stores the created training data in, for example, the memory 12 or a training DB (not shown) provided in the storage unit 13.

[0074] The control unit 11 determines whether there are any unprocessed features among the features of each measurement data stored in the feature database 13d that have not been used to create the training data (S54). If it determines that there are unprocessed features (S54: YES), it returns to step S52. The control unit 11 then performs the processing in steps S52 to S53 on the unprocessed features. The control unit 11 repeats the processing in steps S52 to S54 until it determines that there are no unprocessed features. As a result, training data used to train the learning model is created and stored based on the features stored in the feature database 13d (features calculated based on each measurement data).

[0075] If it is determined that there are no unprocessed features (S54: NO), the control unit 11 uses the accumulated training data to train the learning model that was identified as the target for generation in step S51. Specifically, the control unit 11 reads one training data from the training DB (S55) and uses the read training data to perform the learning process on the learning model (S56). Here, the control unit 11 inputs each feature contained in the training data into the learning model and obtains the output value from the learning model. Then, the control unit 11 compares the output value from the learning model with the correct intracardiac pressure contained in the training data and optimizes the parameters such as the weights between nodes in the learning model, for example, using backpropagation, so that the two approximate each other.

[0076] The control unit 11 determines whether there is any unprocessed training data stored in the training DB that has not been subjected to learning processing (S57). If it determines that there is (S57: YES), it returns to step S55. The control unit 11 then executes the processes in steps S55 to S56 on the unprocessed training data. The control unit 11 repeats the processes in steps S55 to S57 until it determines that there is no unprocessed training data. This allows the learning model to be trained using the training data prepared by storing it in the training DB.

[0077] If the control unit 11 determines that there is no unprocessed training data (S57: NO), it associates the model ID of the learning model identified as the target for generation in step S51 with the learning model and stores the generated learning model in the memory 12 or storage unit 13 (S58). The control unit 11 determines whether it has finished generating all learning models registered in the model DB 13c (i.e., all learning models to be generated) (S59). If the control unit 11 determines that it has not finished generating all learning models (S59: NO), it returns to step S51 and performs the processing in steps S51 to S58 for the ungenerated learning models, and if it determines that it has finished generating all learning models (S59: YES), it terminates the process.

[0078] Through the above-described process, training data can be generated and training can be performed on the learning model registered in the model DB 13c, and a learning model can be generated that estimates and outputs the patient's intracardiac pressure when the features of the input data registered in the model DB 13c are input. In the above-described process, the training data creation process in steps S51 to S54 and the learning model generation process in steps S55 to S58 may be performed by separate devices.

[0079] The information processing device 10 of this embodiment generates each learning model registered in the model DB 13c through the processing described above. The information processing device 10 of this embodiment also performs a verification process to evaluate the estimation accuracy of each generated learning model. For example, the verification process can use methods such as the Leave-One-Out method, cross-validation, or holdout method. For example, the training data created by the processing in Figure 10 is divided into learning data and verification data, a learning process is performed using the learning data, and then a verification process is performed using the verification data. In the verification process, each feature included in the verification data is input to the generated learning model, and the estimation accuracy of the learning model is evaluated based on the difference between the estimated intracardiac pressure output from the learning model and the correct intracardiac pressure included in the verification data. The difference between the estimated intracardiac pressure output from the learning model and the correct intracardiac pressure included in the verification data can be, for example, the mean absolute error, mean squared error, coefficient of determination, etc.

[0080] Next, we will describe the process of identifying the features to be used in the input data of the learning model M, based on the estimation accuracy of each generated learning model. Figure 11 is a flowchart showing an example of the procedure for identifying the features to be used in the input data of the learning model M. The following process is performed by the control unit 11 of the information processing device 10 according to the program P stored in the storage unit 13, but it may also be performed by other learning devices. A verification process is performed for each learning model, and the obtained estimation accuracy is stored in the memory 12 or storage unit 13 in association with the model ID of the learning model.

[0081] The control unit 11 identifies learning models from among those generated by the processing in Figure 10 whose estimation accuracy is equal to or greater than a predetermined value (S61). The control unit 11 identifies the features used in the input data of each identified learning model (S62). The control unit 11 identifies categories for each identified feature and identifies categories common to each learning model (S63). For example, for each category, the control unit 11 counts the number of learning models that use the features of each category as input data and identifies the categories with the most counts. For example, it identifies a predetermined number of categories in descending order of counts. The control unit 11 also identifies features common to each learning model for each identified category (S64). For example, for each category, the control unit 11 counts the number of learning models that use each feature as input data and identifies the features with the most counts. For example, it may identify a predetermined number of features in descending order of counts, or it may identify a predetermined number of features in descending order of counts for each category. Then, the control unit 11 identifies the identified categories and the feature quantities of each category as features to be used as input data for the learning model M (S65). Subsequently, the control unit 11 generates a learning model M using the identified feature quantities as input data by performing the same processing as in Figure 10.

[0082] In this embodiment, multiple learning models are generated using different combinations of feature quantities as input data, and the estimation accuracy of each learning model is obtained. Then, the feature quantities that are commonly used as input data in the learning models whose estimation accuracy is above a predetermined value are determined to be used as the input data for learning model M. By generating learning model M using the determined feature quantities as input data, a learning model M capable of estimating intracardiac pressure with high accuracy can be realized.

[0083] In this embodiment, multiple learning models that take different combinations of features as input data may be configured to output the contribution of each feature in the input data to the obtained estimation result. That is, each learning model may be composed of a so-called explainable AI (XAI), and each feature may be taken as input and configured to output an estimated value of the patient's intracardiac pressure and explanatory data (the contribution of each feature to the estimated value) that shows the basis for the estimate. Such learning models are configured to calculate the contribution (SHAP value, Attention, etc.) for each input data that should be used as the basis for obtaining the estimation result (estimated value of intracardiac pressure) using a SHAP or Attention mechanism, etc. The contribution is a value that indicates the degree to which each input data (feature) of the learning model contributed to the estimation result. For example, Attention is a value that indicates how much attention was paid to each input data (feature) in order to estimate the estimation result. Therefore, input data (features) with a high contribution can be considered as the basis for obtaining the estimation result. Therefore, when a learning model is configured in this way, in learning models where the estimation accuracy is above a predetermined value, in addition to the features that are shared and used among the learning models, features that contribute highly to the estimation result in each learning model may be specified as input data for learning model M.

[0084] The matters described in each of the embodiments described above can be combined with one another. Furthermore, the independent claims and dependent claims described in the claims can be combined with one another in any combination, regardless of the form of reference. In addition, although the claims use a form in which claims referencing two or more other claims (multi-claim form), the claims are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used.

[0085] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications are intended to be in the sense and scope equivalent to the claims.

[0086] 10 Information processing device 11 Control unit 12 Memory 13 Storage unit 14 Communication unit 13a Calculation formula DB M Learning model

Claims

1. An information processing method in which a computer performs the following steps: acquires an electrocardiogram, heart sound waveform, and pulse wave waveform of a patient; acquires a first feature calculated by a first formula using the position of the first or second heart sound obtained from the heart sound waveform, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rise position obtained from the pulse wave waveform; acquires a second feature calculated by a second formula using the maximum amplitude of the first heart sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the first heart sound over time, or the maximum amplitude of the second heart sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the second heart sound over time; and acquires a third feature calculated by a third formula using at least elemental features obtained using the pulse wave waveform; and inputs the acquired first, second, and third features into a learning model trained to output an estimated value of the patient's intracardiac pressure related to the input first, second, and third features when the first, second, and third features are input.

2. The information processing method according to claim 1, wherein the elemental features obtained using at least the pulse wave waveform include the time difference between the notch position and the first heart tone position, the time difference between the notch position and the second heart tone position, the angle of the pulse wave, an index obtained from the velocity pulse wave, an index obtained from the acceleration pulse wave, an index obtained from the approximate pulse wave waveform of the pulse wave waveform, an index obtained from the pulse waves at two positions on the limbs at different distances from the heart, the time difference between the R wave position obtained from the electrocardiogram and the pulse wave start position, the time difference between the pulse wave rise position and the first heart tone position, or an index obtained from the amplitude of the pulse wave waveform or the area obtained by time integration of the pulse wave waveform.

3. The information processing method according to claim 2, wherein the index obtained from the velocity pulse wave includes time information and amplitude information of the velocity pulse wave, the index obtained from the acceleration pulse wave includes time information and amplitude information of the acceleration pulse wave, the index obtained from the approximate pulse wave waveform includes angular frequency and area information of the approximate pulse wave waveform, and the index obtained from the pulse waves at two locations in the limbs includes time information and amplitude information of the upper arm, ankle, and toe pulse waves.

4. An information processing method according to any one of claims 1 to 3, wherein a fourth feature is obtained from the electrocardiogram, which shows the heart rate variation between R waves, the learning model is trained to output an estimated value of the patient's intracardiac pressure related to the first, second, third, and fourth features that have been input, and the computer performs a process of obtaining an estimated value of the patient's intracardiac pressure by inputting the obtained first, second, third, and fourth features into the learning model.

5. An information processing method according to any one of claims 1 to 3, wherein a fifth feature is obtained by formula using the time difference between the R wave position obtained from the electrocardiogram and the I sound position obtained from the heart sound waveform, the learning model is trained to output an estimated value of the patient's intracardiac pressure related to the first feature, second feature, third feature, and fifth feature that have been input, and the computer performs a process of obtaining an estimated value of the patient's intracardiac pressure by inputting the acquired first feature, second feature, third feature, and fifth feature into the learning model.

6. The information processing method according to any one of claims 1 to 3, wherein each of the first feature, second feature, and third feature is a representative value of the respective values ​​calculated for each beat in the electrocardiogram, heart sound waveform, and pulse wave waveform.

7. A program that obtains an electrocardiogram, sound waves, and pulse wave waveform of a patient, and obtains a first feature calculated by a first equation using the position of the first or second sound obtained from the sound waves, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rise position obtained from the pulse wave waveform, a second feature calculated by a second equation using the maximum amplitude of the first sound obtained from the sound waves or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the sound waves or the area obtained by integrating the waveform of the second sound over time, and a third feature calculated by a third equation using at least elemental features obtained using the pulse wave waveform, and causes a computer to execute a process to obtain an estimated value of the patient's intracardiac pressure by inputting the obtained first, second, and third features into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure related to the input first, second, and third features when the first, second, and third features are input.

8. An information processing device having a control unit, wherein the control unit acquires an electrocardiogram, sound wave, and pulse wave waveform of a patient, and acquires a first feature quantity calculated by a first equation using the position of the first sound or second sound obtained from the sound wave, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rising position obtained from the pulse wave waveform, a second feature quantity calculated by a second equation using the maximum amplitude of the first sound obtained from the sound wave or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the sound wave or the area obtained by integrating the waveform of the second sound over time, and a third feature quantity calculated by a third equation using at least elemental feature quantities obtained using the pulse wave waveform, and acquires an estimated value of the patient's intracardiac pressure by inputting the acquired first feature quantity, second feature quantity, and third feature quantity into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure relating to the input first feature quantity, second feature quantity, and third feature quantity when the first feature quantity, second feature quantity, and third feature quantity are input.

9. An information processing method in which a computer performs a process to identify multiple learning models from among multiple learning models that output an estimated value of the patient's intracardiac pressure when different combinations of feature candidate values ​​derived based on one or more of the patient's electrocardiogram, heart sound waveform, and pulse wave waveform are input, based on the estimation accuracy of each learning model, and to extract multiple feature candidate values ​​that are common among the identified multiple learning models from among the feature candidate values ​​input to the identified multiple learning models.

10. The information processing method according to claim 9, wherein each feature candidate is classified into multiple categories, and the computer performs a process of identifying multiple categories common to the identified multiple learning models from among the categories to which the feature candidates input to the identified multiple learning models are classified, and extracting multiple feature candidates classified into the identified categories.

11. The information processing method according to claim 9 or 10, wherein the computer performs a process to generate a learning model that outputs an estimated value of the patient's intracardiac pressure when patient features based on the extracted plurality of candidate features are input.

12. The information processing method according to claim 9 or 10, wherein the computer performs a process to generate a learning model that outputs an estimated value of the patient's intracardiac pressure when the first feature quantity calculated by a first formula using the position of the first sound or second sound obtained from the heart sound waveform, the position of the Q wave obtained from the electrocardiogram, and the notch position and pulse wave rise position obtained from the pulse wave waveform; the second feature quantity calculated by a second formula using the maximum amplitude of the first sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the first sound over time, or the maximum amplitude of the second sound obtained from the heart sound waveform or the area obtained by integrating the waveform of the second sound over time; and the third feature quantity calculated by a third formula using at least elemental features obtained using the pulse wave waveform.

13. The information processing method according to claim 9 or 10, wherein the computer performs a process to obtain an estimated value of the patient's intracardiac pressure by inputting the patient's features based on the multiple feature candidates into a learning model that has been trained to output an estimated value of the patient's intracardiac pressure when the patient's features based on the multiple feature candidates are input.

Images