Heart failure determination data acquisition device, heart failure determination data acquisition method, and computer program

Abstract

A support device (3) acquires collected audio which has been collected with a wearable device (2); displays a pre-scan waveform screen for making a subject (10) confirm whether or not heart sounds of the subject are appropriately included in the collected audio; and extracts audio data of a prescribed portion of the collected audio as determination data for determining the presence or absence of heart failure.

Description

Data acquisition device for assessing heart failure, data acquisition method for assessing heart failure, and computer program

[0001] The present invention relates to a technology for assisting in the collection of a subject's heart sounds using a wearable device.

[0002] In recent years, the incidence of chronic heart failure in Japan has been steadily increasing due to the aging population. As of 2016, approximately 1 million people in Japan suffered from chronic heart failure annually, with approximately 100,000 of them requiring hospitalization. Even after completing treatment and being discharged from the hospital, approximately 40% of patients experience a recurrence of severe symptoms within one year, resulting in readmission. The medical costs incurred by these readmissions across Japan are said to be 48 billion yen.

[0003] Avoiding readmission depends on how reliably the patient's cardiac condition can be monitored after discharge, and how early recurrence can be detected and intervention can be performed.

[0004] Research into useful indices to monitor is being conducted worldwide, and one existing monitoring method that has been proposed as an effective one is Abbott's CardioMEMS, which directly monitors pulmonary artery pressure (Non-Patent Document 1).

[0005] Other proposed technologies include the following: The cardiac monitoring device described in Patent Document 1 includes an enclosure equipped with various sensors and components. The top surface of the enclosure has first grooves sized to fit the phalanges of the subject's right hand. When the cardiac monitoring device is held with the corresponding phalanges pressed against the first grooves, the cardiac monitoring device is oriented appropriately for recording the subject's cardiac activity. The cardiac monitoring device includes a plurality of electrodes configured to create one or more electrical circuits passing through a human heart. The plurality of electrodes includes a right thumb electrode located on the side of the enclosure and connected to the thumb of the subject's right hand, and upper and lower chest electrodes located on the bottom surface of the enclosure and connected to the subject's chest. The cardiac monitoring device also includes a plurality of pulse oximeters located in the first grooves and configured to measure the blood oxygen levels of the corresponding phalanges.

[0006] The heart rate monitor described in Patent Document 2 is hung around the neck of a subject like a pendant and measures the heart rate near the chest.

[0007] The heart rate monitor described in Patent Document 3 is a wristwatch-type heart rate monitor with a microphone attached to the wristband.

[0008] CardioMEMS HF System, https: / / www.cardiovascular.abbott / us / en / hcp / products / heart-failure / pulmonary-pressure-monitors / cardiomems / about.html, retrieved February 29, 2024

[0009] JP 2021-501029 A JP 2-74232 A JP 5-29686 A

[0010] The technology described in Non-Patent Document 1 can obtain heart sounds more effectively than the technologies described in Patent Documents 1 to 3. However, since a device is implanted in the patient's pulmonary artery, it is invasive. This places a burden on the patient's body and requires a lot of effort from the doctor. Furthermore, there are problems with the technology, such as its high cost and the fact that measurements can only be taken on a special bed sensor.

[0011] The techniques described in Patent Documents 1 to 3 can be carried by the patient, and therefore heart sounds can be acquired more easily than with the technique described in Non-Patent Document 1. However, because the patient operates the device themselves without the support of medical personnel, it may not be possible to acquire heart sounds properly.

[0012] In view of the above problems, an object of the present invention is to make it possible to record a patient's heart sounds more efficiently than ever before when using a portable heart sound measuring device.

[0013] A data acquisition system for assessing heart failure according to one embodiment of the present invention includes an acquisition means for acquiring collected audio, which is audio collected by a portable device; a display means for displaying an image representing the heart sounds on a display so that the subject can confirm whether the heart sounds of the subject are well contained in the collected audio; and an extraction means for extracting audio data of a predetermined portion of the collected audio as assessment data for assessing the presence or absence of heart failure.

[0014] Preferably, the acquisition means acquires the collected voice from the portable device in real time, and the display means displays the image in real time.

[0015] According to the present invention, when a portable heart sound measuring device is used, the heart sounds of a patient can be recorded more effectively than in the past.

[0016] 1 is a diagram illustrating an example of the overall configuration of a heart failure detection system. FIG. 1 is a diagram illustrating an example of the external appearance of a wearable device. FIG. 2 is a diagram illustrating an example of the hardware configuration of a device main body. FIG. 3 is a diagram illustrating an example of the hardware configuration of a support device. FIG. 4 is a diagram illustrating an example of the functional configuration of a wearable device and a support device. FIG. 5 is a flowchart illustrating an example of the processing flow of a pre-scan. FIG. 6 is a diagram illustrating examples of a measurement screen, a pre-scan guidance screen, and a pre-scan waveform screen. FIG. 7 is a diagram illustrating an example of wearing the wearable device when collecting heart sounds. FIG. 8 is a diagram illustrating an example of a recording status screen. FIG. 9 is a diagram illustrating an example of a method for generating a trained model. FIG. 10 is a diagram illustrating an example of a waveform of a heart sound. FIG. 11 is a flowchart illustrating an example of the overall processing flow by an observation program. FIG. 12 is a flowchart illustrating an example of the overall processing flow by a support program. FIG. 13 is a diagram illustrating an example of a transmission completion screen. FIG. 14 is a diagram illustrating examples of a symptom input screen, a physical condition input screen, and an activity amount screen. FIG. 15 is a diagram illustrating examples of an individual symptom input screen and a transmission screen. FIG. 16 is a diagram illustrating an example of a pre-scan waveform screen. FIG. 17 is a diagram illustrating examples of advice screens 69M and 69N.

[0017] 1. Overall Configuration of Heart Failure Detection System 1 Fig. 1 is a diagram showing an example of the overall configuration of a heart failure detection system 1. Fig. 2 is a diagram showing an example of the appearance of a wearable device 2.

[0018] The heart failure detection system 1 is a system for detecting heart failure in a patient subject 10, and as shown in Figure 1, is composed of a wearable device 2, a support device 3, a heart failure detection server 40, a medical personnel terminal 41, a machine learning device 42, and a communication line 5.

[0019] The wearable device 2 and the support device 3 are paired in advance and communicate with each other via short-range wireless communication such as Bluetooth (registered trademark). The support device 3, the heart failure detection server 40, the medical staff terminal 41, and the machine learning device 42 communicate with each other via a communication line 5. The communication line 5 may be the Internet, a public line, a local area network (LAN), or the like.

[0020] The wearable device 2 is a wristwatch-type device for recording the heart sounds of the subject 10. Hereinafter, a wristwatch-type wearable device from A-wave Corporation will be used as the wearable device 2. As described in "https: / / awave.co.jp / news / 2023-08-22" or "https: / / www.ccb.osaka-u.ac.jp / news / Osaka University venture a-wave Inc. receives 100-200 million yen in funding / ", this wristwatch-type wearable device has functions such as collecting the heart sounds of the subject 10 and transmitting them to a cloud server via a smartphone.

[0021] As shown in FIG. 2, the wearable device 2 has a device main body 20 and a wristband 28 .

[0022] In principle, the subject 10 wears the wearable device 2 on their wrist all day long. When not recording heart sounds, that is, when the subject normally wears the wearable device 2 so that the device main body 20 is located on the back of the hand. The device may be worn on either the left or right wrist, but it is preferable to wear the device so that the button 20k is located on the thumb side when the device main body 20 is located on the palm side.

[0023] The support device 3 is a device for supporting the subject 10 in inputting information such as the subject's own heart sounds and uploading the information to the heart failure detection server 40. The support device 3 also outputs information from the heart failure detection server 40 for the subject 10.

[0024] A portable device equipped with short-range wireless communication and IP (Internet Protocol) communication functions, such as a smartphone or tablet computer, is used as the support device 3. The communication line 5 is an IP communication network such as an Internet line or a LAN line. The following description will be given taking the case where a smartphone is used as the support device 3 as an example.

[0025] The heart failure detection server 40 detects heart failure of the subject 10 by analyzing the heart sounds transferred from the support device 3. As the heart failure detection server 40, a so-called server machine or a cloud server is used.

[0026] The medical staff terminal 41 is a terminal device used by a medical staff member in charge of the subject 10. As the medical staff terminal 41, a personal computer, a smartphone, a tablet computer, or the like is used.

[0027] The machine learning device 42 generates a trained model through machine learning and provides it to the support device 3.

[0028] 2. Hardware Configuration of the Wearable Device 2 and the Support Device 3 Fig. 3 is a diagram showing an example of the hardware configuration of the device main body 20. Fig. 4 is a diagram showing an example of the hardware configuration of the support device 3.

[0029] As shown in FIG. 2 or 3, the wearable device 2 is composed of a CPU (Central Processing Unit) 20a, a main memory 20b, a non-volatile memory 20c, a three-axis acceleration sensor 20d, a wireless communication circuit 20e, a microphone 20f, a speaker 20g, a skin temperature measurement unit 20h, an ambient temperature measurement unit 20i, a blue light-emitting element 20j1, a green light-emitting element 20j2, an orange light-emitting element 20j3, a button 20k, and a liquid crystal display 20m.

[0030] The CPU 20a is the main central processing unit of the device main body 20 and executes programs loaded into the main memory 20b. An observation program 20p (see FIG. 5) is installed in the non-volatile memory 20c. The observation program 20p is loaded into the main memory 20b and executed by the CPU 20a.

[0031] The three-axis acceleration sensor 20d is used to measure the amount of activity of the subject 10. The wireless communication circuit 20e is a communication device for communicating with the support device 3 via Bluetooth.

[0032] The microphone 20f is provided on the front side of the device main body 20 and collects the heart sounds of the subject 10. The speaker 20g outputs sounds such as beeps.

[0033] The skin temperature measuring unit 20h is provided on the back side of the device main body 20 and measures the temperature of the skin on the wrist of the subject 10. The ambient temperature measuring unit 20i measures the temperature around the wearable device 2.

[0034] The blue light emitter 20j1, the green light emitter 20j2, and the orange light emitter 20j3 emit blue, green, and orange light, respectively. The liquid crystal display 20m displays a message, the time, or the like.

[0035] As shown in FIG. 4, the support device 3 is composed of a CPU 30a, a RAM (Random Access Memory) 30b, a ROM (Read Only Memory) 30c, a flash memory 30d, a touch panel display 30e, an audio unit 30f, a mobile phone communication device 30g, a wireless LAN communication device 30h, a short-range wireless communication device 30i, and an operation button 30j.

[0036] The CPU 30a is the main central processing unit of the support device 3, and executes programs loaded into the RAM 30b. The RAM 30b is the main memory of the support device 3.

[0037] Various programs are installed in the ROM 30c or the flash memory 30d. In particular, a support program 30p (see FIG. 5) is installed in the flash memory 30d to support the subject in operating the wearable device 2. The support program 30p is loaded into the RAM 30b and executed by the CPU 30a.

[0038] The touch panel display 30e displays a screen showing a message to the subject 10, a screen for the subject 10 to input commands or information, a screen showing the results of processing executed by the CPU 30a, and the like.

[0039] The audio unit 30f is composed of a sound board, a microphone, a speaker, etc., and digitizes input audio and outputs audio guidance.

[0040] The mobile phone communication device 30g communicates with other devices via a mobile phone network such as LTE (Long Term Evolution).

[0041] The wireless LAN communication device 30h communicates with other devices using a protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol) via a wireless base station of the communication line 5. A wireless communication device certified by the Wi-Fi Alliance is used as the wireless LAN communication device 30h.

[0042] The short-range wireless communication device 30i communicates with other devices via short-range wireless communication such as Bluetooth.

[0043] The operation buttons 30j include a button for returning to the home screen, a button for adjusting the volume, and a button for switching the power on / off.

[0044] 3. Functions of Each Device FIG. 5 is a diagram showing an example of the functional configuration of each of the wearable device 2 and the support device 3.

[0045] 5, the observation program 20p is composed of modules such as a heart sound data providing unit 201 and a subject data providing unit 202. The support program 30p is composed of modules such as an input support processing unit 301, a heart sound recording unit 302, a subject data collecting unit 303, an activity amount calculating unit 304, a quality evaluating unit 305, a trained model storing unit 306, an environmental noise determining unit 307, a re-input requesting unit 308, a subject data storing unit 309, a subject data transmitting unit 310, and a message output processing unit 311.

[0046] Below, the functions of the wearable device 2, the support device 3, the heart failure detection server 40, the medical professional terminal 41, and the machine learning device 42 will be explained in order.

[0047] 3.1 Recording Heart Sounds of Subject 10 Fig. 6 is a flowchart illustrating an example of the flow of pre-scan processing. Fig. 7 is a diagram illustrating an example of a measurement screen 61, a pre-scan guide screen 62, and a pre-scan waveform screen 63. Fig. 8 is a diagram illustrating an example of how the wearable device 2 is worn when collecting heart sounds. Fig. 9 is a diagram illustrating an example of a recording status screen 64.

[0048] To check whether or not the subject 10 is suffering from heart failure, the subject 10 must record his or her own heart sounds using the wearable device 2 at predetermined times (for example, 8:00, 13:00, and 18:00 every day).

[0049] The input support processing unit 301 (see FIG. 5) of the support device 3 performs processing to support the subject 10 so that heart sounds can be input properly, for example, according to the procedure shown in FIG.

[0050] When the input support processing unit 301 detects that the predetermined time has arrived, it prompts the subject 10 to start inputting heart sounds (#801 in FIG. 6 ). For example, a start prompting message 6A prompting the subject 10 to start inputting heart sounds may be displayed on the touch panel display 30e or output from the speaker of the audio unit 30f. Alternatively, the speaker 20g of the wearable device 2 may be controlled to output a beep. Alternatively, the wearable device 2 may detect that the predetermined time has arrived and output a beep. Alternatively, a message may be displayed or output by voice shortly before the predetermined time (e.g., three minutes before) informing the subject 10 that the time to input heart sounds is approaching.

[0051] When the subject 10 performs a predetermined operation, the input support processing unit 301 displays the measurement screen 61 shown in FIG. 7A on the touch panel display 30e (#802).

[0052] When the subject 10 touches the record button 61a, the input support processing unit 301 displays a pre-scan guide screen 62 on the touch panel display 30e, as shown in Fig. 7B, which guides the subject 10 on how to place the wearable device 2 on their chest. Referring to the pre-scan guide screen 62, the subject 10, while still wearing the wearable device 2, turns the device main body 20 toward the palm of their hand and places the wearable device 2 on their chest, as shown in Fig. 8. Then, the subject 10 touches the scan start button 62a.

[0053] Then, the input support processing unit 301 transmits a sound collection start command 71 to the wearable device 2 (#803). Then, pre-scanning starts, and the processes from step #804 onwards are performed.

[0054] If the wearable device 2 and the support device 3 are not yet connected, the subject 10 performs a predetermined operation for connection before touching the recording button 61a on the measurement screen 61. For example, the subject 10 presses button 20k on the wearable device 2. Then, the identifier of the wearable device 2 is displayed on the touch panel display 30e of the support device 3, and the subject touches and selects the identifier. Then, the wearable device 2 and the support device 3 execute processing for establishing a connection with each other. A sound collection start command 71 is sent to the wearable device 2 after the connection is established.

[0055] In the wearable device 2, when the heart sound data providing unit 201 receives the sound collection start command 71, it turns on the microphone 20f. Then, sound collection by the microphone 20f begins. The heart sound data providing unit 201 generates sound data 7A by sequentially sampling and quantizing (A / D conversion) the collected sound signal, and transmits it to the support device 3 in real time.

[0056] When the input support processing unit 301 starts receiving the voice data 7A (#804), it displays a prescan waveform screen 63 showing the voice waveform of the voice data 7A on the touch panel display 30e (#805), as shown in Figure 7(C) or (D). During the prescan, the prescan waveform screen 63 is updated in real time based on the most recently received voice data 7A.

[0057] If this voice contains heart sounds and almost no environmental noise, a waveform like that shown in Fig. 7(C) will appear on the prescan waveform screen 63. On the other hand, if there is a lot of environmental noise, a waveform like that shown in Fig. 7(D) will appear. Note that "environmental noise" refers to the speaking or breathing of the subject 10 himself, sounds of daily life around the subject 10 (speaking voices other than the subject 10, operating sounds of equipment, etc.), or the friction noise between the subject 10's clothing and the microphone 20f, and is not heart murmur.

[0058] During the pre-scan, the input support processing unit 301 further monitors the amplitude of the waveform, i.e., the volume level of the sound collected by the microphone 20f (hereinafter referred to as "volume level Vi") (#806), and when the volume level Vi falls below the threshold Wt (Yes in #807), displays an adjustment prompting message 6B in a pop-up form to prompt the subject 10 to adjust the position of the microphone 20f on the chest, i.e., the sound collection position (#808). The adjustment prompting message 6B continues to be displayed until the volume level Vi becomes equal to or greater than the threshold Wt.

[0059] The volume level Vi may be the most recent amplitude (volume), but is preferably the peak value of the sound collected over a recent predetermined period of time (e.g., 5 seconds). Alternatively, it may be the average or median value of the sound collected over a recent predetermined period of time. An image (numerical value or indicator) representing the most recent volume level Vi may be displayed together with the adjustment prompt message 6B.

[0060] The subject 10 judges whether his or her heart sounds are being collected well by looking at the prescan waveform screen 63. If a waveform like that shown in Figure 7(D) appears, the subject should take measures such as moving to a quiet place, stopping conversation, breathing quietly, or keeping his or her arm still so that a waveform like that shown in Figure 7(C) appears. Also, if the adjustment prompt message 6B is displayed, the subject should adjust the position of the microphone 20f until the volume level Vi reaches or exceeds the threshold Wt and the adjustment prompt message 6B disappears. If an image representing the volume level Vi is displayed together with the adjustment prompt message 6B, the subject can adjust the position while referring to this image.

[0061] When the volume level Vi exceeds the threshold Wt and a waveform like that shown in Figure 7(C) appears, preparation for recording is complete. At this time, the subject 10 touches the recording button 63a while still holding the wearable device 2 against their chest.

[0062] If the sound recording button 63a is touched when the volume level Vi is above the threshold value Wt (No in #807, Yes in #809), the input support processor 301 ends support for pre-scanning, etc. Note that even after the sound recording button 63a is touched, the wearable device 2 continues to collect sound and transmit the voice data 7A.

[0063] When the record button 63a is touched while the volume level Vi is above the threshold Wt, the heart sound recording unit 302 begins recording the portion of the audio data 7A from the time the record button 63a is touched to the subject data storage unit 309. In other words, the heart sound recording unit 302 begins recording the audio collected by the microphone 20f from the time the button 63a is touched to the subject data storage unit 309. Furthermore, the touch panel display 30e displays a recording status screen 64, as shown in FIG. 9A, which shows the waveform of the audio shown in the audio data 7A during recording. The recording status screen 64 is sequentially updated to show the most recent waveform.

[0064] Then, when a predetermined recording time Tr has elapsed since the record button 63a was touched, the heart sound recording unit 302 ends recording. As a result, audio data of the heart sounds for the recording time Tr is recorded in the subject data storage unit 309. Hereinafter, the audio data recorded in the subject data storage unit 309 will be referred to as "audio data 7B."

[0065] The recording time Tr is approximately 2 to 10 heartbeats long, for example, 15 seconds. The recording time Tr can be set arbitrarily to suit the subject 10.

[0066] Furthermore, when the heart sound recording unit 302 finishes recording, it sends a sound collection end command 72 to the wearable device 2 and updates the recording status screen 64 so that a completion message 64a indicating that recording has been completed is displayed, as shown in Figure 9 (B).

[0067] Upon receiving the sound collection end command 72, the heart sound data providing unit 201 turns off the microphone 20f, thereby ending sound collection by the microphone 20f.

[0068] 3.2 Recording Skin Temperature, Ambient Temperature, and Activity of Subject 10 In the wearable device 2, the skin temperature measurement unit 20h and the ambient temperature measurement unit 20i measure the skin temperature of the wrist of the subject 10 (hereinafter referred to as "skin temperature Hs") and the ambient temperature (hereinafter referred to as "ambient temperature Hc") at predetermined times (for example, at the top of every hour), respectively, and store these values ​​together with the measurement time Tm in the main memory 20b or the non-volatile memory 20c. The following description will be given taking the case where these values ​​are stored in the main memory 20b as an example. The same applies to accelerations ax, ay, and az, which will be described later.

[0069] As will be described later, the skin temperature Hs and the ambient temperature Hc are transmitted to the support device 3 when the heart sounds are recorded. After transmission, they are deleted from the main memory 20b. Therefore, the main memory 20b stores the skin temperature Hs and the ambient temperature Hc for each predetermined time during the period from the most recent heart sound recording to the next heart sound recording. For example, if heart sounds are recorded at 8:00, 1:00, and 6:00 every day and the skin temperature and the ambient temperature are measured every hour, 5 to 14 values ​​of the skin temperature Hs and the ambient temperature Hc will be stored in the main memory 20b.

[0070] During the time period when the wearable device 2 is worn by the subject 10 and when heart sounds are not being recorded, the three-axis acceleration sensor 20d measures acceleration in each of the X, Y, and Z directions at a predetermined sampling rate (e.g., 20 Hz), and stores the respective accelerations ax, ay, and az in the main memory 20b.

[0071] As will be described later, the accelerations ax, ay, and az are transmitted to the support device 3 when recording heart sounds in order to calculate the amount of activity. After transmission, they are deleted from the main memory 20b. Therefore, the main memory 20b stores the accelerations ax, ay, and az for the period from the most recent recording of heart sounds to the next recording. Furthermore, the main memory 20b stores the start time Ts of that period.

[0072] In the support device 3, when the specified time for recording heart sounds arrives (for example, 8:00, 13:00, or 18:00 every day as mentioned above), the subject data collection unit 303 sends an information transmission command 73 to the wearable device 2 to obtain information other than heart sounds, before, after, or in parallel with the processing by the input support processing unit 301 and the heart sound recording unit 302.

[0073] In the wearable device 2, upon receiving the information transmission command 73, the subject data providing unit 202 transmits skin temperature data 7C, ambient temperature data 7D, and acceleration data 7E to the support device 3. The skin temperature data 7C indicates multiple skin temperatures Hs and their respective measurement times Tm stored in the main memory 20b. The ambient temperature data 7D indicates multiple ambient temperatures Hc and their respective measurement times Tm stored in the main memory 20b. The acceleration data 7E indicates multiple accelerations ax, ay, az, and start times Ts stored in the main memory 20b. After transmission, the information stored in the main memory 20b is deleted.

[0074] Then, in the support device 3, upon receiving the skin temperature data 7C and the ambient temperature data 7D, the subject data collection section 303 stores these data in the subject data storage section 309.

[0075] Furthermore, upon receiving the acceleration data 7E, the activity amount calculation unit 304 calculates the activity amount Vw of the subject 10 based on the accelerations ax, ay, and az indicated in the acceleration data 7E. The activity amount Vw may be calculated using a known method. For example, the number of steps may be calculated as the activity amount Vw. The number of steps can be found by counting the number of times a pattern appears in the change in acceleration in the X direction while walking.

[0076] The activity amount calculation unit 304 may further calculate the walking speed based on the frequencies of the accelerations ax, ay, and az. Alternatively, the activity amount calculation unit 304 may classify the walking speed into one of the activity levels of "resting," "slow walking," "normal walking," "slightly brisk walking," "brisk walking," and "jogging or faster." Alternatively, the activity amount Vw may be calculated as calories burned based on the number of steps, walking speed, and attributes of the subject 10 (such as age, sex, height, weight, or muscle mass).

[0077] Then, the activity amount calculation section 304 generates activity amount data 7F indicating the calculated activity amount Vw, and stores the data in the subject data storage section 309 .

[0078] As described above, the acceleration data 7E indicates the accelerations ax, ay, and az for the period from the previous recording to the current recording. Therefore, the activity amount Vw for this period is calculated by the activity amount calculation unit 304. Note that the activity amount Vw for each hour of this period may be calculated. Alternatively, the subject data providing unit 202 may calculate the activity amount Vw instead of the activity amount calculation unit 304.

[0079] 3.3 Evaluation of Recorded Audio Unless heart sounds are recorded under ideal conditions, the audio data 7B may contain some environmental noise. If the level of environmental noise is low, the presence or absence of heart failure can be determined with a certain degree of accuracy based on the audio data 7B. However, if this is not the case, the accuracy of the determination may be poor.

[0080] Therefore, the quality evaluation unit 305 evaluates the quality of the voice data 7B using a trained machine learning model (trained model).

[0081] [3.3.1 Generation of Trained Model] FIG. 10 is a diagram showing an example of a method for generating a trained model 7M.

[0082] The trained model is generated in advance by the machine learning device 42 (see FIG. 1) or the like, for example, according to the procedure shown in FIG. 10 .

[0083] (1) Preparing training data

[0084] (1-1) Calculation of Feature Amounts The developer collects a large amount of audio data 7G by recording the heart sounds of various subjects. The audio data 7G is digitized, i.e., discretized, and its sampling frequency is preferably the same as the sampling frequency of the audio data (audio data 7A) collected by the microphone 20f of the wearable device 2, e.g., 1 kHz. As described below, the audio data 7G is processed and used as a problem (explanatory variable) when generating a machine learning model for estimating the presence or absence of environmental noise. Therefore, audio data with various levels of environmental noise is prepared as the audio data 7G.

[0085] The machine learning device 42 performs preprocessing on each piece of speech data 7 G. The preprocessing consists of three steps: a frequency band extraction step (#811), a noise removal step (#812), and a standardization step (#813).

[0086] The frequency band extraction step is a step of extracting a sound in a specific frequency band from the sound of the sound data 7G, and is executed, for example, as follows: In this embodiment, sound in a frequency band of 20 to 400 Hz is extracted.

[0087] The machine learning device 42 extracts the voice in the 20 to 400 Hz band using a second-order Butterworth bandpass filter with low- and high-frequency cutoff frequencies of 20 Hz and 400 Hz, respectively, to obtain voice data 7H.

[0088] The noise removal step is a step of removing spike noise from the sound of the sound data 7H (the sound obtained by the frequency band extraction step), and is performed, for example, as follows.

[0089] The machine learning device 42 divides the speech data 7H into windows (frames) of a predetermined time length (#821). In this embodiment, the windows are divided into 1000 ms long windows. The machine learning device 42 then calculates the maximum absolute amplitude (MAA) and its position for each window (#822) and calculates the average value of these MAAs (average MAA) (#823).

[0090] Furthermore, the machine learning device 42 identifies positions (noise peak points) in the speech data 7H where the amplitude is more than twice the average MAA, and also identifies positions (zero-crossing points) before and after each of these MAA positions where the waveform passes through zero (#824). The machine learning device 42 then determines that the area between the zero-crossing points immediately before and after the noise peak point is spike noise, and attenuates the amplitude between these points by a predetermined percentage (#825). For example, it attenuates the amplitude by a tenth. Steps #824 and #825 are then repeated until the MAA of all windows is less than twice the average MAA. This results in the speech data 7J.

[0091] The standardization step is a step for standardizing the waveform of the voice of the voice data 7J, and is executed, for example, as follows.

[0092] The machine learning device 42 calculates the average value and standard deviation of the amplitude of the audio in the audio data 7J, subtracts the average value from the value of each position of the audio in the audio data 7J, and divides this value (difference) by the standard deviation. This converts the waveform of the audio in the audio data 7J so that the average and standard deviation become "0" and "1," respectively. Hereinafter, the converted waveform data will be referred to as "audio data 7K."

[0093] The machine learning device 42 obtains the voice data 7K by performing preprocessing on each of the voice data 7G in this way, and then calculates the feature quantity Hf of the voice data 7G that is the source of the voice data 7K by performing frequency analysis. An example of the frequency analysis will be described below.

[0094] The machine learning device 42 divides the speech data 7K into windows of a predetermined time length (#831), and calculates an amplitude spectrum for each window by applying a window function to each window (#832). The calculated amplitude spectrum is converted into a mel spectrum by compressing it through a mel filter bank based on human hearing (#833). Mel-Frequency Cepstrum Coefficients (MFCC) are calculated by applying a discrete cosine transform to the mel spectrum (#834). The average value of the MFCC for each window is then calculated as the feature Hf of the original speech data 7G of the speech data 7K (#835). Note that the feature Hf is multidimensional, and the number of dimensions, along with the window length, is adjusted in hyperparameter tuning during machine learning.

[0095] (1-2) Tagging An expert (e.g., a doctor) listens to the audio (heart sounds) of each audio data 7G, determines whether or not it contains environmental noise that may affect the diagnosis of heart failure, and inputs the determination result into the machine learning device 42.

[0096] The machine learning device 42 then generates data indicating the feature amount Hf calculated by preprocessing from each piece of speech data 7G and a correct label corresponding to each discrimination result, and stores the data in the database as a data set 7L. Specifically, if such environmental noise is present, "0" is used as the correct label, and if not, "1" is used as the correct label.

[0097] Therefore, for example, if an expert determines that certain speech data 7G1 contains environmental noise, data indicating the feature Hf of the speech data 7G1 and the correct label "0" is generated as data set 7L and stored in the database. Alternatively, if an expert determines that certain speech data 7G2 does not contain environmental noise, data indicating the feature Hf of the speech data 7G2 and the correct label "1" is generated as data set 7L and stored in the database.

[0098] (2) Machine Learning The machine learning device 42 generates a trained model by performing machine learning using a dataset 7L stored in a database as training data. The feature values ​​Hf shown in the dataset 7L are explanatory variables, and the correct labels are objective variables.

[0099] As a machine learning algorithm, for example, a decision tree algorithm using gradient boosting regression is used. In this case, the value to be estimated (estimated value) is set to a continuous value between 0 and 1. The estimated value is then used as a quality score. Note that the smaller the environmental noise, the larger the quality score.

[0100] The machine learning device 42 may use a portion of the dataset 7L stored in the database as training data and the remainder for validation, but it is preferable to divide these datasets 7L into n equal parts (e.g., into four equal parts) and perform machine learning and validation by cross-validation. In this case, machine learning is performed using the average accuracy rate (Accuracy) as the evaluation index. This results in a trained model. Furthermore, hyperparameter tuning is performed, and the parameters and trained model with the highest evaluation index are stored in the trained model storage unit 306 of the support device 3 (see FIG. 5 ). Hereinafter, the trained model stored (memorized) in the trained model storage unit 306 will be referred to as the "trained model 7M."

[0101] [3.3.2 Estimation Based on Trained Model] When the recording of the heart sounds of the subject 10 is completed, the quality evaluation unit 305 reads the voice data 7B from the subject data storage unit 309 and performs preprocessing and frequency analysis on the voice data 7B in the same manner as in machine learning, thereby calculating the feature quantity Hj of the voice data 7B.

[0102] Then, the quality evaluation unit 305 calculates the quality score Sq by inputting the calculated feature amount Hj into the trained model 7M.

[0103] [3.3.3 Determining the level of environmental noise] If the quality score Sq is equal to or greater than the threshold value St, the environmental noise determination unit 307 determines that the environmental noise in the audio data 7B, i.e., the audio recorded this time, is low, and if the quality score Sq is less than the threshold value St, it determines that the environmental noise is high.

[0104] 3.4 Promoting Re-input When the environmental noise determination unit 307 determines that the environmental noise of the audio data 7B is large, the re-input request unit 308 displays a re-input prompt message 6C as a pop-up on the touch panel display 30e, prompting the user to re-input (record) the heart sounds in a quiet environment without moving the arm on which the wearable device 2 is worn.

[0105] When the re-input prompt message 6C is displayed, the subject 10 performs a predetermined operation to display the measurement screen 61. Then, the subject 10 re-inputs (records) his / her own heart sounds in the same manner as described above.

[0106] The re-input request unit 308 deletes the voice data 7B from the subject data storage unit 309 before starting the re-input.

[0107] 3.5 Uploading Various Information When the environmental noise determination unit 307 determines that the environmental noise of the voice data 7B is low, the subject data transmission unit 310 transmits the voice data 7B, skin temperature data 7C, ambient temperature data 7D, and activity amount data 7F stored in the subject data storage unit 309 to the heart failure detection server 40. Then, these data and the acceleration data 7E are deleted from the subject data storage unit 309. A message may be output to indicate that the heart sounds were recorded well.

[0108] Upon receiving this data, the heart failure detection server 40 records it in the database in association with the identifier of the subject 10. This completes the uploading of information about the subject 10.

[0109] 3.6 Determining Heart Failure FIG. 11 is a diagram showing an example of a waveform of a heart sound.

[0110] Upon receiving the audio file 7B, the skin temperature data 7C, the ambient temperature data 7D, and the activity amount data 7F, the heart failure detection server 40 performs a process to detect heart failure. For example, this can be performed as follows using the detection method described in Japanese Patent Application Laid-Open No. 2020-39472.

[0111] The heart sounds in the audio file 7B include the first and second sounds as shown in Figure 11. In addition, the third sound may also be included. The first sound is the sound produced when the ventricle contracts, the second sound is the sound produced at the beginning of ventricular diastole, and the third sound is the sound produced when the ventricular wall vibrates due to the impact of blood flow.

[0112] The heart failure detection server 40 extracts the above-mentioned sounds from the heart sounds in the audio file 7B. If the third heart sound is extracted as a result, it determines that heart failure has occurred, generates abnormal heart sound notification data 7N indicating an abnormal heart sound, and transmits it to the support device 3 or the medical staff terminal 41. The audio file 7B, skin temperature data 7C, ambient temperature data 7D, and activity amount data 7F may also be transmitted to the medical staff terminal 41.

[0113] Furthermore, the heart failure detection server 40 may determine that peripheral cold sensation is occurring if the skin temperature Hs indicated in the skin temperature data 7C is equal to or lower than a predetermined temperature.

[0114] The heart failure detection server 40 may also detect heart failure based on peripheral coldness and activity level. For example, it may determine that heart failure has occurred when the skin temperature Hs is equal to or lower than a predetermined temperature, the number of steps indicated as the activity level Vw in the activity level data 7F is reduced by 20% or more from a reference value, and the maximum walking speed is reduced by 20% or more from a reference value. If the wearable device 2 has a function for measuring skin impedance, it may also determine that heart failure has occurred when the skin impedance is further reduced by 10% or more.

[0115] In the support device 3, when the message output processing unit 311 receives the abnormal heart sound notification data 7N from the heart failure detection server 40, it displays the abnormal heart sound message 6D indicating that an abnormality has been found in the heart sounds on the touch panel display 30e.

[0116] 4. Overall Processing Flow and Effects of This Embodiment Fig. 12 is a flowchart illustrating an example of the overall processing flow by the observation program 20p. Fig. 13 is a flowchart illustrating an example of the overall processing flow by the support program 30p.

[0117] Next, the overall processing flow of each of the wearable device 2 and the support device 3 will be described with reference to flowcharts.

[0118] The wearable device 2 executes processing according to the procedure shown in FIG. 12 based on the observation program 20p.

[0119] The wearable device 2 measures the skin temperature Hs, the ambient temperature Hc, and the accelerations ax, ay, and az at predetermined times (Yes in #840 of FIG. 12 ), and stores the skin temperature data 7C, the ambient temperature data 7D, and the acceleration data 7E in the main memory 20b (#841). For example, the skin temperature Hs and the ambient temperature Hc are measured at the top of every hour. The device also continues to measure the accelerations ax, ay, and az during the time period when the microphone 20f is off.

[0120] When the wearable device 2 receives a sound collection start command 71 from the support device 3 (Yes in #842 in Figure 12), it switches on the microphone 20f and begins collecting sound (#843), and sequentially transmits the collected sound data to the sound data 7A (#844).

[0121] Alternatively, when the sound collection end command 72 is received (Yes in #845), the wearable device 2 ends sound collection by switching on the microphone 20f (#846), and also ends transmission of the voice data 7A (#847).

[0122] Alternatively, when the information transmission command 73 is received (Yes in #848), the wearable device 2 transmits the skin temperature data 7C, ambient temperature data 7D, and acceleration data 7E stored in the main memory 20b to the support device 3 (#849), and then deletes these data from the main memory 20b (#850).

[0123] The wearable device 2 executes the processes of steps #840 to #850 as appropriate until the power is turned off (No in #851).

[0124] On the other hand, the support device 3 executes processing in accordance with the procedure shown in FIG. 13 based on the support program 30p.

[0125] At a predetermined time (e.g., 8:00, 13:00, or 18:00 every day) (Yes in #861 in Figure 13), the support device 3 transmits an information transmission command 73 to the wearable device 2, thereby acquiring data other than heart sounds, i.e., skin temperature data 7C, ambient temperature data 7D, and acceleration data 7E from the wearable device 2 (#862), and calculates the activity amount Vw based on the acceleration data 7E to generate activity amount data 7F (#863).

[0126] Before, after, or in parallel with the processing of steps #862 to #863, the support device 3 performs a pre-scanning process to assist the subject 10 in inputting heart sounds (#864). The procedure for the pre-scanning process is as previously described with reference to Figure 6. When the subject 10 is able to input heart sounds of a suitable volume into the wearable device 2, he or she performs an operation to start recording.

[0127] The support device 3 then records the heart sounds to generate voice data 7B (#865), calculates feature quantities Hj of the voice data 7B by performing preprocessing and frequency analysis (see FIG. 10 ) (#866), and estimates (calculates) the quality score Sq by inputting the feature quantities Hj into the trained model 7M (#867).

[0128] The support device 3 determines whether the environmental noise is loud or quiet based on the quality score Sq (#868). If the environmental noise is loud (Yes in #869), the support device 3 prompts the subject 10 to re-record the heart sounds by displaying a re-input prompt message 6C (#870). Then, the process returns to step #864 and starts over from the pre-scan process.

[0129] On the other hand, if the environmental noise is low (No in #869), the audio data 7B is uploaded to the heart failure detection server 40 along with the skin temperature data 7C and ambient temperature data 7D acquired in step #862 and the activity data 7F calculated in step #863 (#871).

[0130] Alternatively, when the support device 3 receives the abnormal heart sound notification data 7N from the heart failure detection server 40 (Yes in #872), it displays the abnormal heart sound message 6D (#873).

[0131] The support device 3 executes the processes of steps #861 to #873 as appropriate until the support program 30p is terminated (No in #874).

[0132] According to this embodiment, a portable device such as the wearable device 2 can be used to more reliably capture the heart sounds of a subject than in the past. In particular, the wearable device 2 presents the waveform of the sound collected to the subject 10 as a pre-scan waveform screen 63 (FIG. 7C or 7D), or presents an adjustment prompting message 6B when the volume level Vi of the sound falls below a threshold Wt, thereby prompting the subject 10 to make adjustments and achieving better sound collection. Furthermore, the quality score Sq of the sound is estimated, and if the quality score Sq is less than the threshold St, the recording is retried, thereby enabling better heart sounds to be recorded for the diagnosis of heart failure.

[0133] Furthermore, the support device 3 acquires not only the heart sounds but also the skin temperature Hs, the ambient temperature Hc, and the activity amount Vw as information about the subject 10, and uploads this information to the heart failure detection server 40. This can improve the accuracy of the determination of heart failure.

[0134] 5. Modifications Fig. 14 is a diagram showing an example of a transmission completion screen 60. Fig. 15 is a diagram showing examples of a symptom input screen 65, a physical condition input screen 66, and activity amount screens 67A and 67B. Fig. 16 is a diagram showing examples of individual symptom input screens 65A to 65D and a transmission screen 68. Fig. 17 is a diagram showing an example of a pre-scan waveform screen 69. Fig. 18 is a diagram showing examples of advice screens 69M and 69N.

[0135] In this embodiment, the support device 3 acquires the heart sounds, skin temperature, and activity level as biometric information of the subject 10 and uploads them to the heart failure detection server 40, but other biometric information of the subject 10 may also be acquired and uploaded.

[0136] For example, after transmitting the voice data 7B, the activity amount data 7F, and the like to the heart failure detection server 40, the support device 3 displays a transmission completion screen 60 as shown in Fig. 14. The transmission completion screen 60 displays, for example, a waveform 60a of heart sounds indicated in the voice data 7B and an indicator 60b of the activity amount indicated in the activity amount data 7F.

[0137] Here, when the subject 10 touches the next button 60c, the support device 3 may display a symptom input screen 65 as shown in Fig. 15(A), prompt the subject 10 to input the presence or absence or level of these symptoms, and upload the input information as symptom data to the heart failure detection server 40. Alternatively, the support device 3 may display a physical condition input screen 66 as shown in Fig. 15(B), prompt the subject 10 to input weight, blood pressure, and physical condition, and upload the input information as physical condition data to the heart failure detection server 40.

[0138] The heart failure detection server 40 may determine the presence or absence of heart failure based on not only the voice data 7B, the skin temperature data 7C, or the activity amount data 7F but also the symptom data or the physical condition data.

[0139] The support device 3 may display transitions in the biological information of the subject 10. For example, an activity amount screen 67A showing the activity amount Vw for each hour of a certain day, as shown in FIG. 15(C), may be displayed. Alternatively, an activity amount screen 67B showing the activity amount Vw for each three hours of a certain week, as shown in FIG. 15(D), may be displayed. Alternatively, transitions in skin temperature, heart failure assessment results, blood pressure, etc. may be displayed.

[0140] Instead of the symptom input screen 65, the support device 3 may sequentially display individual symptom input screens 65A to 65D as shown in Fig. 16 (A) to (D), allowing the user to input information about shortness of breath, palpitations, physical fatigue, and whether lying down makes it difficult to sleep by touching a button on each screen. For example, the support device 3 may display the individual symptom input screen 65A as a screen for inputting information about shortness of breath. The touched button is then highlighted by a thicker or more colored frame as shown in Fig. 16 (E), and the next screen (individual symptom input screen 65B in this example) is displayed after a predetermined time has elapsed (for example, one second).

[0141] When input on the individual symptom input screens 65A to 65D is complete, the support device 3 displays a send screen 68 as shown in FIG. 16(F). When the send button 68a is touched, the information entered on the individual symptom input screens 65A to 65D is uploaded to the heart failure detection server 40 as physical condition data. If the subject 10 wishes to correct any symptom, he or she can simply touch the corresponding edit button 68b. The support device 3 then re-displays the individual symptom input screen corresponding to the touched symptom.

[0142] The support device 3 and the medical personnel terminal 41 may be provided with a video call function so that the subject 10 and the medical personnel can converse while watching each other's videos. Alternatively, the support device 3 and the medical personnel terminal 41 may be provided with a messenger call function so that the subject 10 and the medical personnel can communicate via text messages. However, the video call function may be restricted so that calls can only be made from the medical personnel terminal 41. Alternatively, the messenger call function may be restricted so that text messages can only be sent from the medical personnel terminal 41.

[0143] In this embodiment, the heart failure detection server 40 determines whether or not a patient has heart failure, but the determination may be made by the support device 3. In this case, the support device 3 may execute the above-described processing of the heart failure detection server 40.

[0144] In this embodiment, the quality score Sq is estimated by the trained model 7M by the support device 3, but this may be performed by another device. For example, this may be performed by the heart failure detection server 40, the machine learning device 42, or another device.

[0145] In the present embodiment, the support device 3 acquires the skin temperature data 7C, the ambient temperature data 7D, and the acceleration data 7E from the wearable device 2 when recording heart sounds, but the data may be acquired at other times. For example, the data may be acquired every hour. Then, each time the data is acquired, activity amount data 7F may be generated based on the acceleration data 7E, and the skin temperature data 7C, the ambient temperature data 7D, and the activity amount data 7F may be transmitted to the heart failure detection server 40.

[0146] In the present embodiment, the activity amount Vw is calculated by the support device 3, but the wearable device 2 may calculate it.

[0147] In this embodiment, the support device 3 first monitors the volume level Vi of the sound (heart sounds) collected by the wearable device 2 and then calculates the quality score Sq of the sound, but the order of calculation may be changed, or the sound and heart sounds may be calculated simultaneously. If the calculations are performed simultaneously, the calculations may be performed during a pre-scan. In other words, the input support processing unit 301 and the quality evaluation unit 305 may determine whether the heart sounds are being collected satisfactorily.

[0148] Then, when the record button 63a (see FIG. 7) is touched when the volume level Vi exceeds the threshold Wt and the quality score Sq is equal to or greater than the threshold St, the support device 3 may record the portion of the voice data 7A from the time the record button 63a is touched as voice data 7B. Alternatively, even if the record button 63a is not touched, voice data for a predetermined continuous period (e.g., 5 seconds) during which the volume level Vi exceeds the threshold Wt and the quality score Sq is equal to or greater than the threshold St may be extracted from the voice data 7A and uploaded to the heart failure detection server 40 as voice data 7B. Alternatively, when the state in which the volume level Vi exceeds the threshold Wt and the quality score Sq is equal to or greater than the threshold St continues for a predetermined period (e.g., 3 seconds), recording to the subject data storage unit 309 may be started immediately and automatically to acquire the voice data 7B without waiting for the record button 63a to be touched.

[0149] Furthermore, the support device 3 may display not only an image (numerical value or indicator) representing the latest volume level Vi but also an image (numerical value or indicator) representing the latest quality score Sq.

[0150] Alternatively, instead of the prescan waveform screen 63, a prescan waveform screen 69 as shown in FIG. 17 may be displayed.

[0151] The pre-scan waveform screen 69 is provided with an actual waveform window 69A, an ideal waveform window 69B, an indicator 69C, a message window 69D, and the like.

[0152] The measured waveform window 69A, like the pre-scan waveform screen 63, displays the waveform of the audio of the audio data 7A (i.e., the audio collected by the microphone 20f of the wearable device 2) in real time. Furthermore, two reference lines 69A1 and 69A2 are provided to inform the subject 10 of a target value for the amplitude of the waveform (e.g., a threshold value Wt). The subject 10 attempts to bring the amplitude outside the area between the reference lines 69A1 and 69A2 by, for example, changing the position of the microphone 20f. The reference lines 69A1 and 69A2 are drawn in a conspicuous color, for example, red.

[0153] An example of a correct (ideal) waveform is shown in the ideal waveform window 69B. Two reference lines 69B1 and 69B2 have the same functions as the reference lines 69A1 and 69A2 in the measured waveform window 69A.

[0154] The indicator 69C represents the quality score Sq. In the pre-scanning, the quality score Sq may also be calculated by the quality evaluation unit 305 using the method described above.

[0155] The message window 69D displays a message informing the subject 10 that if the volume level Vi exceeds the threshold value Wt and the quality score Sq remains above the threshold value St for a predetermined period of time (e.g., 1.5 seconds), recording will automatically begin shortly.

[0156] Furthermore, the support device 3 may recognize the first sound and the second sound by analyzing in real time the voice input to the support device 3 during the pre-scan. Furthermore, if it is determined that the second sound is recorded louder during the pre-scan, the support device 3 may display a message instructing the user to move the microphone 20f of the wearable device 2 downward and to the left by approximately 3 to 5 cm and move closer to the left nipple.

[0157] In this embodiment, a common threshold value Wt is used as the threshold value for determining the timing for displaying the adjustment prompt message 6B (see step #807 in Figure 6) and the threshold value for ending support for pre-scanning, etc. and starting recording to obtain audio data 7B (see #807, #809), but different threshold values ​​may also be used.

[0158] In this embodiment, the support device 3 uploads to the heart failure detection server 40 as audio data 7B audio data when the volume level Vi exceeds the threshold value Wt and the quality score Sq is equal to or greater than the threshold value St, but audio data that satisfies either of the conditions may also be uploaded.

[0159] In this embodiment, in order to communicate various matters to the subject 10, a screen or message is mainly displayed on the touch panel display 30e of the support device 3, but the blue light-emitting element 20j1, the green light-emitting element 20j2, and the orange light-emitting element 20j3 of the wearable device 2 may be lit or flashed depending on the matter to be communicated.

[0160] In this embodiment, the machine learning device 42 generates the trained model 7M using a decision tree algorithm based on gradient boosting regression, but the trained model 7M may be generated using other machine learning algorithms, such as random forest or deep learning.

[0161] In this embodiment, the support device 3 extracts voice data 7B (good voice data) from voice data 7A (voice data of voice collected by the wearable device 2), stores the voice data 7B in the subject data storage unit 309, and then transmits the voice data 7B to the heart failure detection server 4. However, the voice data 7A may be continuously transmitted from the support device 3 to the heart failure detection server 4 in real time, and the heart failure detection server 4 may extract the voice data 7B.

[0162] If the analysis or detection of cardiac failure fails, it is possible that the heart sounds were not recorded correctly. Therefore, in such a case, an advice screen 69M as shown in Fig. 18(A) may be displayed on the touch panel display 30e. During recording, instead of the recording status screen 64 shown in Fig. 9(A), an advice screen 69N as shown in Fig. 18(B) may be displayed to provide tips on recording heart sounds. The recording status screen 64 may also display indicators showing the required recording time and the elapsed time from the start of recording to the present.

[0163] In addition, the overall or individual configurations of the heart failure detection system 1, the wearable device 2, and the support device 3, the processing content, the processing order, the data configuration, the screen configuration, etc. can be modified as appropriate in accordance with the spirit of the present invention.

[0164] The following inventions are also included in this embodiment.

[0165] (Supplementary Note 1) A data acquisition device for determining heart failure comprising: acquisition means for acquiring collected voice, which is voice collected by a portable device; display means for displaying an image representing the heart sounds to allow the subject to confirm whether the heart sounds of the subject are well included in the collected voice; and extraction means for extracting, after the image is displayed, voice data representing a portion of the collected voice after a predetermined operation has been performed, as determination data for determining the presence or absence of heart failure. (Supplementary Note 2) The data acquisition device for determining heart failure according to Supplementary Note 1, further comprising: determination means for determining whether the volume of the collected voice exceeds a threshold, wherein the extraction means extracts the determination data if the volume exceeds the threshold. (Supplementary Note 3) The data acquisition device for determining heart failure according to Supplementary Note 2, further comprising: output means for outputting a message urging the subject to adjust the position of the portable device on the chest of the subject if the volume is below the threshold. (Supplementary Note 4) The data acquisition device for cardiac failure assessment according to any of Supplementary Note 1 to Supplementary Note 3, further comprising: a second assessment means for assessing the quality of the collected audio based on the extent to which the collected audio contains environmental audio, which is audio other than the heart sounds, and wherein the extraction means extracts the assessment data when the quality exceeds a predetermined standard. (Supplementary Note 5) The data acquisition device for cardiac failure assessment according to Supplementary Note 4, further comprising: a second output means for outputting a second message urging the user to collect audio in a quiet environment when the quality falls below the predetermined standard. (Supplementary Note 6) The data acquisition device for cardiac failure assessment according to any of Supplementary Note 1 to Supplementary Note 3, further comprising: a third output means for outputting a third message urging the user to record the heart sounds when the time to collect the heart sounds has arrived or is approaching. (Supplementary Note 7) The data acquisition device for cardiac failure assessment according to any of Supplementary Note 1 to Supplementary Note 3, wherein the portable device is a wristwatch-type wearable device worn on a human arm.(Supplementary Note 8) A data acquisition device for determining heart failure comprising: acquisition means for acquiring collected sound, which is sound collected by a portable device; first determination means for determining whether the volume of the collected sound exceeds a threshold; second determination means for determining the quality of the collected sound based on the extent to which the collected sound contains environmental sound, which is sound other than the heart sounds of the subject; and extraction means for extracting, from the collected sound, sound data representing a portion of a time period of a predetermined length in which the volume exceeds the threshold and the quality exceeds a predetermined quality, as determination data for determining the presence or absence of heart failure. (Supplementary Note 9) The data acquisition device for determining heart failure according to Supplementary Note 8, comprising: display means for displaying a first image representing the volume and a second image representing the quality.

[0166] DESCRIPTION OF SYMBOLS 2 Wearable device (portable device) 3 Support device (cardiac failure assessment data acquisition device) 301 Input support processing unit (acquisition means, display means, determination means, output means, third output means) 302 Heart sound recording unit (extraction means) 305 Quality evaluation unit (second determination means) 311 Message output processing unit (second output means) 30e Touch panel display (display means, output means, second output means, third output means) 30i Short-range wireless communication device (acquisition means) 6A Start prompt message (third message) 6B Adjustment prompt message (message) 6C Re-input prompt message (second message) 63 Pre-scan waveform screen (image) 7A Voice data (collected voice) 7B Voice data (assessment data)

Claims

1. A data acquisition system for assessing heart failure, comprising: acquisition means for acquiring collected voice, which is voice collected by a portable device; display means for displaying an image representing the heart sounds on a display so that the subject can confirm whether the heart sounds of the subject are well included in the collected voice; and extraction means for extracting a predetermined portion of voice data from the collected voice as assessment data for assessing the presence or absence of heart failure.

2. The data acquisition system for assessing heart failure according to claim 1, wherein the acquisition means acquires the collected voice from the portable device in real time, and the display means displays the image in real time.

3. The data acquisition system for assessing heart failure according to claim 2, further comprising: an output means for outputting a message prompting the subject to adjust the position of the portable device on the subject's chest when the volume level of the collected voice is below a threshold.

4. A data acquisition system for assessing heart failure according to claim 2 or 3, wherein the extraction means extracts, as the voice data, data from the collected voice after a predetermined operation has been performed.

5. A data acquisition system for assessing heart failure as described in claim 2 or claim 3, further comprising a discrimination means for discriminating in real time whether the collected voice is good or bad, and wherein the extraction means extracts, as the voice data, data from the collected voice that is determined by the discrimination means to be good.

6. The data acquisition system for assessing heart failure according to claim 5, wherein the determining means determines that the collected voice is good when the volume level of the collected voice exceeds a second threshold value.

7. The data acquisition system for assessing heart failure according to claim 5, wherein the determining means determines that the collected voice is good when the score of the quality of the collected voice exceeds a third threshold value.

8. The data acquisition system for assessing heart failure according to claim 5, wherein the discrimination means determines that the collected voice is good when the volume level of the collected voice exceeds a second threshold and the quality score of the collected voice exceeds a third threshold.

9. A data acquisition system for assessing heart failure as described in claim 2 or claim 3, wherein the extraction means extracts, as the audio data, data of a portion of the collected audio where the volume level of the collected audio exceeds a second threshold or the quality score of the collected audio exceeds a third threshold.

10. A data acquisition system for assessing heart failure as described in claim 2 or claim 3, wherein the extraction means extracts, as the audio data, data from the collected audio where the volume level of the collected audio exceeds a second threshold and the quality score of the collected audio exceeds a third threshold.

11. A data acquisition system for assessing heart failure as described in any one of claims 7 to 10, further comprising: a second output means for outputting a second message urging the subject to collect sound in a quiet environment when the score is below the third threshold.

12. A data acquisition system for assessing heart failure according to any one of claims 5 to 11, wherein the display means displays the score together with the image.

13. A data acquisition system for assessing heart failure according to any one of claims 1 to 12, further comprising: a third output means for outputting a third message prompting the user to record the heart sounds when the time for collecting the heart sounds arrives or approaches.

14. A data acquisition system for determining heart failure according to any one of claims 1 to 13, wherein the portable device is a wristwatch-type wearable device worn on a human arm.

15. A data acquisition device for determining heart failure, comprising: an acquisition means for acquiring collected audio, which is audio collected by a portable device; a first discrimination means for determining whether the volume of the collected audio exceeds a threshold; a second discrimination means for discriminating the quality of the collected audio based on the extent to which the collected audio contains environmental audio, which is audio other than the subject's heart sounds; and an extraction means for extracting audio data from the collected audio representing a portion of a time period of a predetermined length in which the volume exceeds the threshold and the quality exceeds a predetermined quality, as determination data for determining the presence or absence of heart failure.

16. A method for acquiring data for assessing heart failure, comprising: causing a computer to execute a process for acquiring collected voice, which is voice collected by a portable device; causing the computer to execute a process for displaying an image representing the heart sounds on a display so that the subject can confirm whether the heart sounds of the subject are well included in the collected voice; and causing the computer to execute a process for extracting voice data of a predetermined portion from the collected voice as assessment data for assessing the presence or absence of heart failure.

17. A method for acquiring data for assessing heart failure, comprising: causing a computer to execute a process for acquiring collected voice, which is voice collected by a portable device; causing the computer to execute a process for determining whether the volume of the collected voice exceeds a threshold; causing the computer to execute a process for determining the quality of the collected voice based on the extent to which the collected voice contains environmental voice, which is voice other than the subject's heart sounds; and causing the computer to execute a process for extracting voice data representing a portion of the collected voice for a predetermined time period in which the volume exceeds the threshold and the quality exceeds a predetermined quality, as assessment data for assessing the presence or absence of heart failure.

18. A computer program comprising: causing a computer to execute a process for acquiring collected voice, which is voice collected by a portable device; causing the computer to execute a process for displaying an image representing the heart sounds on a display so that the subject can confirm whether the heart sounds of the subject are well included in the collected voice; and causing the computer to execute a process for extracting voice data of a predetermined portion from the collected voice as determination data for determining the presence or absence of heart failure.

19. A computer program comprising: causing a computer to execute a process for acquiring collected audio, which is audio collected by a portable device; causing the computer to execute a process for determining whether the volume of the collected audio exceeds a threshold; causing the computer to execute a process for determining the quality of the collected audio based on the extent to which the collected audio contains environmental audio, which is audio other than the subject's heart sounds; and causing the computer to execute a process for extracting audio data representing a portion of the collected audio over a predetermined time period in which the volume exceeds the threshold and the quality exceeds a predetermined quality, as determination data for determining the presence or absence of heart failure.

Images