Learning device, analysis device, learning method, analysis method, and program

Abstract

One aspect of the present invention is a learning device comprising: a time series acquisition unit that acquires an observed time series, which is a time series expressed as an oscillator linear sum, which is a linear sum of oscillator time series, where the oscillator time series is a time series of the amplitude of a fluctuating oscillator, which is an oscillator whose amplitude changes periodically; and a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model that estimates the oscillator linear sum of the observed time series based on the observed time series, using an equation that expresses the generation mechanism of the observed time series and a mathematical model that expresses the relationship between probabilistic state transitions of the state of a generation source of the observed time series and symbol output, which is information that is probabilistically output in each of the states, and the learning process execution unit updates the linear sum estimation learning model based on a result of execution of the linear sum estimation learning model.

Description

Technical Field

[0001] The present invention relates to a learning device, an analysis device, a learning method, an analysis method, and a program thereof.

Background Art

[0002] Heart sound is one of the important clues for knowing the state of a patient's circulatory system.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

Non-Patent Document 4

Non-Patent Document 13

Non-Patent Document 14

Non-Patent Document 15

Non-Patent Document 16

Non-Patent Document 17

Summary of the Invention

Problems to be Solved by the Invention

[0004] Although a plurality of prior studies have proposed methods for automatically estimating the state of the heart at each time from the heart sound time series, the accuracy of estimating the state of the heart at each time is not very good. Therefore, even now, estimating the state of the heart from heart sounds requires the labor of creating supervised data using electrocardiograms in advance.

[0005] To explain in a little more detail the point that the accuracy of estimating the state of the heart at each time is not very good. The waveform shown by the time series of heart sounds is decomposed into the waveforms of the amplitudes of a plurality of oscillators whose amplitudes change periodically. That is, the waveform of the time series of heart sounds is the linear sum of the waveforms of the amplitudes of a plurality of fluctuating oscillators. A fluctuating oscillator is an oscillator whose amplitude changes periodically. The fact that the accuracy of estimating the state of the heart at each time is not very good specifically means that the accuracy of decomposing the time series of heart sounds into the linear sum of the time series of the amplitudes of the fluctuating oscillators is not good.

[0006] Note that such a problem is not necessarily limited to heart sounds, but is a common problem for the analysis of time series expressed as the linear sum of the waveforms of the amplitudes of fluctuating oscillators.

[0007] In view of the above circumstances, an object of the present invention is to provide a technique for improving the accuracy of analysis of a time series expressed as the linear sum of the waveforms of the amplitudes of fluctuating oscillators, which are oscillators whose amplitudes change periodically.

Means for Solving the Problems

[0008] One aspect of the present invention is a learning device including: a time series acquisition unit that acquires an observed time series which is a time series represented by a linear sum of oscillators (oscillator linear sum) that is a time series of the amplitudes of a variable oscillator, which is an oscillator whose amplitude changes periodically, as an oscillator time series; a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the oscillator linear sum of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is information probabilistically output in each of the states; and the learning process execution unit updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model.

[0009] One aspect of the present invention is an analysis device including: an analysis target acquisition unit that acquires a time series to be analyzed; a time series acquisition unit that acquires an observed time series which is a time series represented by a linear sum of oscillators (oscillator linear sum) that is a time series of the amplitudes of a variable oscillator, which is an oscillator whose amplitude changes periodically, as an oscillator time series; a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the oscillator linear sum of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is information probabilistically output in each of the states; and an analysis unit that analyzes the time series to be analyzed using the learned linear sum estimation learning model obtained by the learning device, in which the learning process execution unit updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model.

[0010] One aspect of the present invention includes a time series acquisition step of acquiring an observed time series, which is a time series represented by a linear sum of oscillators (oscillator linear sum) that is a linear sum of the time series of the amplitudes of variable oscillators, which are oscillators whose amplitudes change periodically, as the oscillator time series; and a learning process execution step of executing a linear sum estimation learning model, which is a mathematical model for estimating the oscillator linear sum of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is information probabilistically output in each of the states. In the learning process execution step, the linear sum estimation learning model is updated based on the result of the execution of the linear sum estimation learning model. This is a learning method.

[0011] One aspect of the present invention includes an analysis target acquisition step of acquiring a time series to be analyzed; a time series acquisition unit that acquires an observed time series, which is a time series represented by a linear sum of oscillators (oscillator linear sum) that is a linear sum of the time series of the amplitudes of variable oscillators, which are oscillators whose amplitudes change periodically, as the oscillator time series; and a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the oscillator linear sum of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is information probabilistically output in each of the states. The learning process execution unit analyzes the time series to be analyzed using the learned linear sum estimation learning model obtained by a learning device that updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model. This is an analysis method.

[0012] One aspect of the present invention is a computer program for causing a computer to function as the above learning device.

[0013] One aspect of the present invention is a computer program for causing a computer to function as the above analysis device.

Advantages of the Invention

[0014] According to the present invention, it is possible to improve the accuracy of the analysis of a time series represented by the linear sum of the waveforms of the amplitudes of a variable oscillator, which is an oscillator whose amplitude changes periodically.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0016] (Embodiment) FIG. 1 is a diagram showing an example of the configuration of the analysis system 100 of the embodiment. For the sake of simplicity of the following description, an example in which the analysis target of the analysis system 100 is the time series of heart sounds (hereinafter referred to as "heart sound time series"), and the analysis system 100 estimates the state of the heart which is the generation source of the heart sound time series will be used to describe the analysis system 100.

[0017] However, the time series of the analysis target of the analysis system 100 does not necessarily have to be the time series of heart sounds. The time series of the analysis target of the analysis system 100 may be any time series as long as it is a time series represented by the linear sum of the amplitude waveforms of an oscillator (hereinafter referred to as "fluctuating oscillator") whose amplitude changes periodically. Since the waveform is the time series, the linear sum of the amplitude waveforms of the fluctuating oscillator is the linear sum (hereinafter referred to as "oscillator linear sum") of the time series of the amplitude of the fluctuating oscillator (hereinafter referred to as "oscillator time series").

[0018] Also, the result of the analysis output by the analysis system 100 does not necessarily have to be the result of estimating the state of the heart. The result of the analysis output by the analysis system 100 may be any result as long as it is a result obtained based on the oscillator linear sum. The result of the analysis output by the analysis system 100 may be the time series of the amplitude of each fluctuating oscillator of the oscillator linear sum.

[0019] The analysis system 100 includes a learning device 1 and an analysis device 2. The learning device 1 updates, by learning, a machine learning model (hereinafter referred to as "linear sum estimation learning model") that estimates an oscillator linear sum based on the input heart sound time series.

[0020] Note that a machine learning model is a mathematical model including one or more processes whose execution conditions and order (hereinafter referred to as "execution rules") are predetermined. Learning means updating a machine learning model by a machine learning method. Further, updating a machine learning model means suitably adjusting the values of predetermined parameters included in the machine learning model. Further, executing a machine learning model means executing each process included in the machine learning model according to the execution rules.

[0021] Note that a machine learning model is represented by, for example, a neural network. Note that a neural network is a circuit such as an electronic circuit, an electric circuit, an optical circuit, or an integrated circuit that represents a machine learning model. Updating a machine learning model also means that the neural network representing the machine learning model is updated by learning. That the neural network is updated by learning means that the values of the parameters of the neural network are updated. Further, the parameters of the neural network are the parameters of the circuit constituting the neural network and are also the parameters of the learning model represented by the circuit constituting the neural network.

[0022] The neural network representing the linear sum estimation learning model may be any neural network as long as it can represent the linear sum estimation learning model. The neural network representing the linear sum estimation learning model is, for example, a deep neural network.

[0023] The learning device 1 performs learning of the linear sum estimation learning model until a predetermined end condition (hereinafter referred to as "learning end condition") is satisfied. The learning end condition is, for example, a condition that learning has been performed a predetermined number of times. The learning end condition may be, for example, a condition that the change in the linear sum estimation learning model due to the update is smaller than a predetermined change.

[0024] The analysis device 2 estimates the oscillator linear combination indicating the input heart sound time series by using the learned linear combination estimation learning model obtained by the learning device 1. The learned linear combination estimation learning model is the linear combination estimation learning model at the timing when the learning end condition is satisfied.

[0025] <Regarding the linear combination estimation learning model and the learning process of the linear combination estimation learning model> The relationship between the linear combination estimation learning model and the process by which the learning device 1 learns the linear combination estimation learning model (hereinafter referred to as the "learning process") will be described. The linear combination estimation learning model includes a machine learning model (hereinafter referred to as the "heart cycle state posterior distribution learning model") that estimates the posterior distribution of the state of the heart cycle based on the input heart sound time series. The linear combination estimation learning model includes a machine learning model (hereinafter referred to as the "oscillator time series posterior distribution learning model") that estimates the posterior distribution of the oscillator time series based on the posterior distribution of the state of the heart cycle. The linear combination estimation learning model includes a machine learning model (hereinafter referred to as the "heart sound time series marginal distribution learning model") that estimates the marginal distribution of the heart sound time series using the posterior distribution of the oscillator time series.

[0026] The learning process includes a time series input process, a heart cycle state posterior distribution estimation process, an oscillator time series posterior distribution estimation process, a heart sound time series marginal distribution estimation process, and an update process.

[0027] The time series input process is a process in which the heart sound time series is input to the linear combination estimation learning model.

[0028] The heart cycle state posterior distribution estimation process is a process of estimating the posterior distribution of the state of the heart cycle based on the input heart sound time series by executing the heart cycle state posterior distribution learning model.

[0029] The oscillator time series posterior distribution estimation process is a process of estimating the posterior distribution of the oscillator time series based on the posterior distribution of the state of the heart cycle by executing the oscillator time series posterior distribution learning model.

[0030] The processing for estimating the marginal distribution of the heart sound time series is a process of estimating the marginal distribution of the heart sound time series using the posterior distribution of the oscillator time series by executing the learning model for the marginal distribution of the heart sound time series.

[0031] The update processing is a process of updating the linear sum estimation learning model so that the marginal likelihood under the marginal distribution estimated by the heart sound time series marginal distribution estimation processing is increased for the heart sound time series input by the time series input processing.

[0032] More specifically, updating the linear sum estimation learning model means updating the cardiac cycle state posterior distribution learning model, the oscillator time series posterior distribution learning model, and the heart sound time series marginal distribution learning model.

[0033] <Regarding the cardiac cycle state posterior distribution learning model> The cardiac cycle state posterior distribution learning model can be any mathematical model as long as it can calculate the posterior distribution of the cardiac cycle state based on the heart sound time series. The states of the cardiac cycle include four states: S1 sound, systolic phase, S2 sound, and diastolic phase. The states of the cardiac cycle repeat the four states of S1 sound, systolic phase, S2 sound, and diastolic phase periodically. More specifically, the state of the cardiac cycle transitions from the state of the S1 sound to the state of the systolic phase. The state of the cardiac cycle transitions from the state of the systolic phase to the state of the S2 sound. The state of the cardiac cycle transitions from the state of the S2 sound to the state of the diastolic phase. The state of the cardiac cycle transitions from the state of the diastolic phase to the state of the S1 sound.

[0034] The sound of the S1 sound state and the sound of the S2 sound state are extremely loud sounds in the heart sound, and are sounds caused by the vibration of the valves generated at the closure of the mitral valve and the aortic valve, respectively. Therefore, the heart sound time series is mainly a time series in which non-linear oscillators corresponding to the vibration of the valves in the heart are added together while periodically changing their amplitudes.

[0035] The cardiac cycle state posterior distribution learning model is, for example, a mathematical model that obtains the posterior probability distribution represented by the following formula (1) based on the heart sound time series. The heart sound time series is represented by the following formula (2).

[0036] [Number]

[0037] [Number]

[0038] [Number]

[0039] [Number]

[0040] [Number]

[0041] [Number]

[0042] J is the number of microphones for acquiring heart sounds. R is the length of the heart sound time series. z r is the unobserved state of the heart sound time series at the r-th time. The unobserved state is specifically the state of the cardiac cycle. Since there are four states of the cardiac cycle as described above, z r can take values from 1 to 4 as shown in Equation (4).

[0043] The state of the cardiac cycle does not immediately transition to the next state immediately after a state transition occurs. After a state transition occurs, there is a finite time during which it stays in the transitioned state. That is, there is a finite duration of existence in each state of the cardiac cycle until it transitions to the next state. Therefore, the transition probability defined by the following Equation (7) is defined.

[0044] [Number]

[0045]

Number

[0046] A in formula (8) is a transition matrix. p j (δ) is the distribution of the survival period. p j The distribution of the survival period represented by (δ) is represented by, for example, the negative binomial distribution of the following formula (9).

[0047]

Number

[0048] θ j is the success probability of the trial. m is the shape parameter. e in formula (6) r is the pseudo state. The pseudo state is a quantity indicating how much time has been spent in each state. This pseudo state e r and z r Using e and z, z defined by formula (5) r ^- follows a Markov process with a certain probability transition matrix determined by (7), (8), and (9). Hereinafter, a symbol with a bar on it will be expressed by the symbol ^-. For example, z r ^- means the symbol with a bar on the symbol z r . Therefore, z r ^- means the symbol on the left side of formula (5). Also, as in formula (10), the probability of z r is consistent with the probability obtained by marginalizing z r ^- with respect to the pseudo state.

[0049]

Number

[0050] For the calculation of Equation (1), after calculating the numerator of Equation (1) for all combinations of latent variables z⁻, it is necessary to normalize so that the sum of the posterior probabilities on the left - hand side becomes 1. However, the number of possible state combinations is (4m) R pieces, and the computational complexity is of the order of (4m) R This increase in computational complexity can be suppressed, for example, by replacing the right - hand side of Equation (13), which is an approximate equation using the function of Equation (12) (hereinafter referred to as "potential") instead of the term of Equation (11) in the numerator of Equation (1).

[0051] [Number]

[0052] [Number]

[0053] [Number]

[0054] By the approximation of Equation (13), the computational complexity for calculating the value of Equation (1) can be reduced to the order of 4mR. Note that the approximate equation of (13) above is just an example, and the potential can be any function as long as it is a part of the function representing the posterior probability distribution of the states of the cardiac cycle and the content to be expressed is defined in advance.

[0055] The mathematical formula representing the potential is obtained based on the heart sound time series using a machine learning model such as a neural network, for example. Therefore, when the posterior probability distribution of the states of the cardiac cycle is expressed using the potential, the mathematical formula representing the potential is also updated in the learning of the linear - sum estimation learning model.

[0056] [Regarding the oscillator time - series posterior distribution learning model] The oscillator time series posterior distribution learning model will be described in more detail. The oscillator time series posterior distribution learning model is a mathematical model that represents the relationship between probabilistic state transitions such as hidden Markov models and hidden semi-Markov models, and symbol outputs that are probabilistically output in each state. The state in the mathematical model representing the relationship between probabilistic state transitions and symbol outputs is the state of the cardiac cycle.

[0057] The symbol output in the mathematical model representing the relationship between probabilistic state transitions and symbol outputs is the oscillator time series. When the mathematical model representing the relationship between probabilistic state transitions and symbol outputs is a hidden semi-Markov model, the mathematical model also represents the duration of each state.

[0058] <Regarding the Peripheral Distribution Learning Model of Heart Sound Time Series> The peripheral distribution learning model of the heart sound time series will be described in more detail. The heart sound time series is observed as a linear sum of variable oscillators according to Equation (14). Equation (19) is the distribution followed by the variable oscillator and is an equation derived from Equation (17) that represents the generation mechanism of the heart sound time series. Equation (17) is a differential equation that represents the generation mechanism of the heart sound time series.

[0059]

Number

[0060]

Number

[0061]

Number

[0062]

Number

[0063] τ is the variance of the distribution of the observation noise. The quantity y on the left side of Equation (16) rlindicates the amplitude of the l-th variable oscillator at time r.

[0064] M is the mass of the heart valve. u is the displacement of the heart valve. ΔP is the pressure applied to the heart valve. D is the damping coefficient indicating the magnitude of the damping of the movement of the heart valve. K is the stiffness coefficient of the heart valve.

[0065] The formal solution of Equation (17) is the following Equation (18).

[0066]

Equation

[0067] C and α are constants. ω is the angular frequency. ψ is the initial phase shift. t is the time.

[0068] Since the amplitude of the waveform indicated by the heart sound time series is proportional to the first derivative of the displacement u of the heart valve (i.e., the speed of the heart valve membrane), the variable oscillator is represented by a second-order autoregressive model. Therefore, the following Equation (19) holds.

[0069]

Equation

[0070] a l is the damping coefficient of the l-th variable oscillator. f l is the average frequency of the l-th variable oscillator. f s is the sampling frequency. σ li 2 is the variance of the distribution of the system noise. The variance σ li 2 strongly depends on the state of the cardiac cycle and changes the dominant variable oscillator included in the heart sound time series based on the state of the cardiac cycle. In particular, the variance σ li 2 having a large value means that the l-th variable oscillator is dominant when the state of the cardiac cycle is i.

[0071] <Regarding the update process> The update process will be described in more detail. In the update process, for example, the linear combination estimation learning model is updated so as to maximize the log marginal likelihood represented by the following equation (20).

[0072]

Number

[0073] In the update process, for example, the linear combination estimation learning model may be updated so as to maximize the lower limit value of the likelihood represented by the following equation (21). Note that the update of the linear combination estimation learning model specifically means, for example, updating {θ_j}, {a l}, {f l}, {σ li}, {g jl} and τ.

[0074]

Number

[0075] The log marginal likelihood shown in Equation (20) and the lower limit value of the likelihood shown in Equation (21) are in the relationship of the following Equation (22).

[0076]

Number

[0077] The term of the following Equation (23) in Equation (22) represents Equation (14). The term of the following Equation (24) in Equation (22) represents Equation (19). The term of the following Equation (25) in Equation (22) is the transition probability of z^- derived from Equations (7)(8)(9)(10).

[0078]

Number

[0079]

Number

[0080]

Number

[0081] When maximizing the lower limit value of the likelihood shown in Equation (21), for reducing the amount of calculation, the relationships of the following Equations (26) and (27) are used.

[0082]

Number

[0083]

Number

[0084] Equation (27) includes a potential. As described above, the mathematical formula indicating the potential is obtained by a machine learning model such as a neural network. The potential is, for example, the following Equation (28).

[0085]

Number

[0086] φ(·) appearing on the right side of Equation (28) is a mapping from the following Equation (29) to the following Equation (30) upward.

[0087]

Number

[0088]

Number

[0089] s is a window length of a predetermined fixed length. The symbols of the following formula (31) appearing on the right side of formula (28) are operators that return the (i, δ)-th element of the input matrix. Specifically, the input matrix is formula (30).

[0090]

Number

[0091] As the mathematical formula form for expressing the potential, those used in supervised heart sound segmentation such as CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network) may be used.

[0092] Note that the following formula (32) and the following formula (33) are the z^- initial probability and transition probability derived from formulas (7), (8), (9), and (10).

[0093]

Number

[0094]

Number

[0095] From formula (23) to formula (33), the relationship of formula (22) is converted into the relationship of the following formula (34).

[0096]

Number

[0097]

Number

[0098] Maximization of the lower limit value of the likelihood shown in Equation (21) is performed with a smaller amount of computation than the amount of computation required for maximization of the log marginal likelihood shown in Equation (20) by using the message passing method.

[0099] Note that Equation (34) is specifically derived by transforming the following Equations (36) and (37).

[0100]

Equation

[0101]

Equation

[0102] FIG. 2 is a diagram showing an example of the hardware configuration of the learning device 1 in the embodiment. The learning device 1 includes a control unit 11 including a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. The learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15 by executing a program.

[0103] More specifically, the processor 91 reads out a program stored in the storage unit 14 and stores the read program in the memory 92. By executing the program stored in the memory 92, the learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15.

[0104] The control unit 11 controls the operations of various functional units included in the learning device 1. The control unit 11 executes a learning process. The control unit 11 controls, for example, the operation of the output unit 15 and causes the output unit 15 to output the execution result of the learning process. The control unit 11 records, for example, various information generated by the execution of the learning process in the storage unit 14. The various information stored in the storage unit 14 includes, for example, the learning result of the linear sum estimation learning model.

[0105] The input unit 12 is configured to include an input device such as a mouse, keyboard, touch panel, etc. The input unit 12 may be configured as an interface for connecting these input devices to the learning device 1. The input unit 12 receives the input of various information to the learning device 1. For example, a heart sound time series is input to the input unit 12.

[0106] The communication unit 13 is configured to include a communication interface for connecting the learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless means. The external device is, for example, a device that is the source of the heart sound time series. The external device is, for example, the analysis device 2. The communication unit 13 transmits the learned linear sum estimation learning model to the analysis device 2 through communication with the analysis device 2.

[0107] The storage unit 14 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 14 stores various information related to the learning device 1. The storage unit 14 stores, for example, information input via the input unit 12 or the communication unit 13. The storage unit 14 stores, for example, a linear sum estimation learning model. The storage unit 14 stores, for example, various information generated by the execution of the learning process.

[0108] The output unit 15 outputs various information. The output unit 15 is configured to include a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. The output unit 15 may be configured as an interface for connecting these display devices to the learning device 1. The output unit 15 outputs, for example, information input to the input unit 12 or the communication unit 13. The output unit 15 may display, for example, the execution result of the learning process.

[0109] FIG. 3 is a diagram showing an example of the configuration of the control unit 11 included in the learning device 1 in the embodiment. The control unit 11 includes a heart sound time series acquisition unit 111, a learning process execution unit 112, an end determination unit 113, a recording unit 114, and an output control unit 115.

[0110] The heart sound time series acquisition unit 111 acquires the heart sound time series input to the input unit 12 or the communication unit 13. If the heart sound time series has been recorded in the storage unit 14 in advance, the heart sound time series acquisition unit 111 may read the heart sound time series from the storage unit 14.

[0111] The learning process execution unit 112 executes the learning process. The end determination unit 113 determines whether the learning end condition is satisfied. The linear sum estimation learning model obtained by the learning process executed by the learning process execution unit 112 and at the time when the learning end condition is satisfied by the end determination unit 113 is the learned linear sum estimation learning model.

[0112] The learning process execution unit 112 includes a time series input unit 121, a cardiac cycle state posterior distribution estimation unit 122, an oscillator time series posterior distribution estimation unit 123, a heart sound time series marginal distribution estimation unit 124, and an update unit 125.

[0113] The time series input unit 121 executes time series input processing on the heart sound time series acquired by the heart sound time series acquisition unit 111. That is, the time series input unit 121 inputs the heart sound time series acquired by the heart sound time series acquisition unit 111 into the linear sum estimation learning model.

[0114] The cardiac cycle state posterior distribution estimation unit 122 executes cardiac cycle state posterior distribution estimation processing. The oscillator time series posterior distribution estimation unit 123 executes oscillator time series posterior distribution estimation processing. The heart sound time series marginal distribution estimation unit 124 executes heart sound time series marginal distribution estimation processing. The update unit 125 executes update processing.

[0115] The recording unit 114 records various information in the storage unit 14. The output control unit 115 controls the operation of the output unit 15.

[0116] FIG. 4 is a flowchart showing an example of the flow of processing executed by the learning device 1 in the embodiment. The time-series input unit 121 inputs the heart sound time series into the linear sum estimation learning model (step S101). Next, the cardiac cycle state posterior distribution estimation unit 122 executes the cardiac cycle state posterior distribution estimation process (step S102). That is, the cardiac cycle state posterior distribution estimation unit 122 estimates the posterior distribution of the state of the cardiac cycle based on the input heart sound time series.

[0117] Next, the oscillator time series posterior distribution estimation unit 123 executes the oscillator time series posterior distribution estimation process (step S103). That is, the oscillator time series posterior distribution estimation unit 123 estimates the posterior distribution of the oscillator time series based on the posterior distribution of the state of the cardiac cycle. Next, the heart sound time series marginal distribution estimation unit 124 executes the heart sound time series marginal distribution estimation process (step S104). That is, the heart sound time series marginal distribution estimation unit 124 estimates the marginal distribution of the heart sound time series using the posterior distribution of the oscillator time series. Next, the update unit 125 executes the update process (step S105). By executing the update process, the linear sum estimation learning model is updated.

[0118] Next, the end determination unit 113 determines whether the learning end condition is satisfied (step S106). If the learning end condition is not satisfied (step S106: NO), the process returns to the process of step S101. On the other hand, if the learning end condition is satisfied (step S106: YES), the process ends.

[0119] FIG. 5 is a diagram showing an example of the hardware configuration of the analysis device 2 in the embodiment. The analysis device 2 includes a control unit 21 including a processor 93 such as a CPU and a memory 94 connected by a bus, and executes a program. The analysis device 2 functions as a device including the control unit 21, the input unit 22, the communication unit 23, the storage unit 24, and the output unit 25 by executing the program.

[0120] More specifically, the processor 93 reads out the program stored in the storage unit 24 and stores the read program in the memory 94. By executing the program stored in the memory 94, the analysis device 2 functions as a device including the control unit 21, the input unit 22, the communication unit 23, the storage unit 24, and the output unit 25.

[0121] The control unit 21 controls the operations of various functional units included in the analysis device 2. The control unit 21 executes a learned linear sum estimation learning model. The control unit 21 controls, for example, the operation of the output unit 25 and causes the output unit 25 to output the execution result of the learned linear sum estimation learning model. The control unit 21 records, for example, various information generated by the execution of the learned linear sum estimation learning model in the storage unit 24.

[0122] The input unit 22 is configured to include input devices such as a mouse, a keyboard, and a touch panel. The input unit 22 may be configured as an interface for connecting these input devices to the analysis device 2. The input unit 22 receives the input of various information to the analysis device 2. For example, a heart sound time series to be analyzed is input to the input unit 22.

[0123] The communication unit 23 is configured to include a communication interface for connecting the analysis device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless means. The external device is, for example, a device that is the transmission source of the heart sound time series to be analyzed. The external device is, for example, the learning device 1. The communication unit 23 acquires a learned linear sum estimation learning model through communication with the learning device 1.

[0124] The storage unit 24 is configured using a computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 24 stores various information related to the analysis device 2. The storage unit 24 stores, for example, information input via the input unit 22 or the communication unit 23. The storage unit 24 stores, for example, a learned linear sum estimation learning model. The storage unit 24 stores, for example, various information generated by the execution of the learned linear sum estimation learning model.

[0125] The output unit 25 outputs various types of information. The output unit 25 includes, for example, a display device such as a CRT display, a liquid crystal display, or an organic EL display. The output unit 25 may be configured as an interface for connecting these display devices to the analysis device 2. The output unit 25 outputs, for example, the information input to the input unit 22 or the communication unit 23. The output unit 25 may display, for example, the execution result of the learned linear sum estimation learning model.

[0126] FIG. 6 is a diagram showing an example of the configuration of the control unit 21 in the embodiment. The control unit 21 includes an analysis target acquisition unit 211, an analysis unit 212, a recording unit 213, and an output control unit 214.

[0127] The analysis target acquisition unit 211 acquires the heart sound time series of the analysis target input to the input unit 22 or the communication unit 23. When the heart sound time series of the analysis target has been recorded in the storage unit 24 in advance, the analysis target acquisition unit 211 may read out the heart sound time series of the analysis target from the storage unit 24.

[0128] The analysis unit 212 analyzes the heart sound time series of the analysis target. More specifically, the analysis unit 212 executes the learned linear sum estimation learning model on the heart sound time series of the analysis target, and acquires the output of the learned linear sum estimation learning model as the result of the analysis. Specifically, executing the learned linear sum estimation learning model on the heart sound time series of the analysis target means inputting the heart sound time series of the analysis target into the learned linear sum estimation learning model and executing the learned linear sum estimation learning model into which the heart sound time series of the analysis target has been input.

[0129] In the execution of the learned linear sum estimation learning model for the heart sound time series of the analysis target, first, the posterior distribution of the state of the cardiac cycle is estimated based on the heart sound time series of the analysis target. In the execution of the learned linear sum estimation learning model for the heart sound time series of the analysis target, next, the posterior distribution of the oscillator time series is estimated based on the obtained posterior distribution of the state of the cardiac cycle.

[0130] The recording unit 213 records various information in the storage unit 24. The output control unit 214 controls the operation of the output unit 25.

[0131] FIG. 7 is a flowchart showing an example of the flow of processing executed by the analysis device 2 in the embodiment. The analysis target acquisition unit 211 acquires the heart sound time series of the analysis target (step S201). Next, the analysis unit 212 executes the learned linear sum estimation learning model on the heart sound time series of the analysis target (step S202). By executing the learned linear sum estimation learning model, the posterior distribution of the oscillator time series is estimated. Next, the output control unit 214 causes the output unit 25 to output the obtained posterior distribution of the oscillator time series (step S203).

[0132] <Experimental Results> The results of the experiment using the analysis system 100 will be described. In the experiment, four types of datasets were used. The first dataset and the second dataset were normal heart sound time series and abnormal heart sound time series in various types of symptoms that accompany a auscultation textbook. These datasets included a total of 119 time series.

[0133] FIG. 8 is a first diagram showing an example of the results of an experiment using the analysis system 100 of the embodiment. FIG. 8 is an example of the heart sound time series included in the first dataset. The vertical axis in FIG. 8 indicates the normalized amplitude.

[0134] FIG. 9 is a second diagram showing an example of the results of an experiment using the analysis system 100 of the embodiment. FIG. 9 is an example of the heart sound time series included in the second dataset. The vertical axis in FIG. 9 indicates the normalized amplitude.

[0135] The third dataset was a heart sound time series obtained with one microphone. The heart sound time series included in the third dataset was a heart sound time series with the annotation of S1 sound and S2 sound manually added. The fourth dataset was a heart sound time series obtained simultaneously with a large number of microphones.

[0136] In the experiment, a comparison was made with the ensemble empirical mode decomposition method with a kurtosis feature (EEMD) described in Non-Patent Document 9.

[0137] In the method using EEMD, first, empirical mode decomposition (EMD) was applied to extract intrinsic mode functions (IMFs). During the periods of S1 sound and S2 sound, the amplitudes of each IMF increase simultaneously. In EEMD, for detecting this increase, the kurtosis of the IMF was calculated using a sliding window. When the window contains the start timings of S1 sound and S2 sound, the peripheral distribution of the window has a heavier tail and a higher kurtosis than when the window does not contain the start timings of S1 sound and S2 sound. Therefore, by detecting the peak of the product of the kurtoses of the peripheral distributions of the IMFs with windows of different scales, the start timings of S1 sound and S2 sound can be estimated.

[0138] In the experiment, all heart sound time series were downsampled to 2000 Hz and passed through a band-pass filter with a frequency band from 10 Hz to 150 Hz. The time series thus obtained was applied to the analysis system 100 and EEMD. In the experiment, as a potential, the potential of Equation (28) was used. s was 64. And φ was learned by a two-layer CNN. Although the noise level differed for each data, for verifying robustness, the same values were used for the hyperparameters of both methods in the experiment.

[0139] It was determined that the estimation of the start timings of S1 sound and S2 sound was appropriate if the interval between the estimated start timing and the true start timing was 100 ms.

[0140] The F1 score was used to verify the appropriateness of the classification process. The F1 score is a quantity defined by the following Equation (38).

[0141]

Equation

[0142] P + is accuracy, and S e is the detection rate.

[0143] FIG. 10 is a third diagram showing an example of the result of an experiment using the analysis system 100 of the embodiment. FIG. 10 is an example of the oscillator decomposition obtained from a normal heart sound time series using the analysis system 100. The oscillator decomposition means acquiring the waveforms of the respective fluctuating oscillators of the oscillator linear sum. PCG represents the heart sound time series to be analyzed, and IMF1 to IMF3 respectively represent the time series of the fluctuating oscillators obtained from the heart sound time series to be analyzed. The vertical axis of each graph of PCG, IMF1, IMF2, and IMF3 represents the amplitude. That PCG represents the heart sound time series to be analyzed, IMF1 to IMF3 respectively represent the time series of the fluctuating oscillators obtained from the heart sound time series to be analyzed, and the vertical axis of each graph of PCG, IMF1, IMF2, and IMF3 represents the amplitude is the same for FIGS. 11 to 13.

[0144] FIG. 11 is a fourth diagram showing an example of the result of an experiment using the analysis system 100 of the embodiment. FIG. 11 is an example of the oscillator decomposition obtained from a normal heart sound time series using EEMD.

[0145] FIG. 12 is a fifth diagram showing an example of the result of an experiment using the analysis system 100 of the embodiment. FIG. 12 is an example of the oscillator decomposition obtained from an abnormal heart sound time series using the analysis system 100.

[0146] FIG. 13 is a sixth diagram showing an example of the result of an experiment using the analysis system 100 of the embodiment. FIG. 13 is an example of the oscillator decomposition obtained from an abnormal heart sound time series using EEMD.

[0147] FIG. 14 is a seventh diagram showing an example of the results of an experiment using the analysis system 100 of the embodiment. "Our model" in FIG. 14 means the analysis system 100. "N" in FIG. 14 means the number of S1 sounds and S2 sounds. "TP" in FIG. 14 means true positive. "FP" in FIG. 14 means false positive. "FN" in FIG. 14 means false negative. "F1" in FIG. 14 means the F1 score. (a) in FIG. 14 means the first data set. (b) in FIG. 14 means the second data set. (c) in FIG. 14 means the third data set.

[0148] FIG. 14 shows that, for all three data sets from the first data set to the third data set, the estimation of the start timing of the S1 sound and the S2 sound by the analysis system 100 is more accurate than EEMD.

[0149] FIG. 15 is an eighth diagram showing an example of the results of an experiment using the analysis system 100 of the embodiment. Ch1 and Ch2 in FIG. 15 each show the time series of the oscillator to be analyzed, which is the time series of each dimension of the two-dimensional oscillator time series. IMF1, IMF2, IMF3, and IMF4 in FIG. 15 show the waveforms of the four variable oscillators obtained from the oscillator time series of Ch1 and Ch2. The vertical axis of each graph of Ch1, Ch2, IMF1, IMF2, IMF3, and IMF4 indicates the amplitude. The graph of Label in FIG. 15 shows the estimated state of the cardiac cycle. FIG. 15 shows that the analysis system 100 can extract variable oscillators from the heart sound time series acquired by a plurality of microphones.

[0150] The analysis system 100 of the embodiment configured as described above obtains a learned linear sum estimation learning model based on an expression representing the generation mechanism of the heart sound time series and a mathematical model representing the relationship between probabilistic state transition and symbol output. Therefore, the linear sum of the variable oscillators can be obtained from the heart sound time series to be analyzed with higher accuracy than the mathematical model obtained without using the expression representing the generation mechanism of the heart sound time series. Therefore, the analysis system 100 can improve the accuracy of the analysis of the time series represented by the linear sum of the variable oscillators.

[0151] Moreover, the analysis system 100 of the embodiment does not use the technique of empirical mode decomposition. When using empirical mode decomposition, it is known that a problem called mode mixing occurs. Since the analysis system 100 does not use the technique of empirical mode decomposition, the occurrence of mode mixing can be suppressed.

[0152] Also, in the technique of empirical mode decomposition, due to the heuristic nature of the calculation and the loose constraints imposed on the time series after decomposition, it is difficult to incorporate the generation mechanism of the time series into the mathematical model that obtains the linear sum of the fluctuating oscillators. Since the analysis system 100 of the embodiment does not use the technique of empirical mode decomposition, as shown in Equation (18), it is possible for the analysis system 100 to incorporate the generation mechanism of the time series into the mathematical model that obtains the linear sum of the fluctuating oscillators.

[0153] Furthermore, it is known that in the technique of empirical mode decomposition, it is impossible to obtain the linear sum of the fluctuating oscillators from the multi-channel time series recorded simultaneously by multiple microphones. Since the analysis system 100 assumes a situation where multi-dimensional time series are observed, it can obtain the linear sum of the fluctuating oscillators from the multi-channel time series.

[0154] (Modification example) As described above, the heart sound time series is an example, and the analysis system 100 does not necessarily need to analyze the heart sound time series. The time series to be analyzed by the analysis system 100 may be any time series as long as it is a time series expressed as a linear sum of the waveforms of the amplitudes of the fluctuating oscillators. The time series expressed as a linear sum of the waveforms of the amplitudes of the fluctuating oscillators may be, for example, the time series of breath sounds. When the time series expressed as a linear sum of the waveforms of the amplitudes of the fluctuating oscillators is the time series of breath sounds, the analysis system 100 uses an equation that expresses the generation mechanism of the breath sound time series instead of the equation that expresses the generation mechanism of the heart sound time series. Thus, the equation that expresses the generation mechanism of the heart sound time series is an example of the equation that expresses the generation mechanism of the time series to be analyzed.

[0155] In addition, when the state of the mathematical model representing the relationship between the probabilistic state transition and the symbol output is a time series represented by the linear sum of the waveforms of the amplitudes of the variable oscillators and is the breath sound, it is a two-phase state of the expiration phase and the inspiration phase. Thus, the state of the mathematical model representing the relationship between the probabilistic state transition and the symbol output is the state of the source of the time series represented by the linear sum of the waveforms of the amplitudes of the variable oscillators.

[0156] The learning device 1 and the analysis device 2 may each be implemented using a plurality of information processing devices communicably connected via a network. In this case, each functional unit included in each of the learning device 1 and the analysis device 2 may be implemented in a distributed manner across the plurality of information processing devices.

[0157] Note that the learning device 1 and the analysis device 2 do not necessarily have to be implemented as different devices. The learning device 1 and the analysis device 2 may be implemented, for example, as a single device having both functions.

[0158] Note that all or part of each function of the analysis system 100, the learning device 1, and the analysis device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk incorporated in a computer system. The program may be transmitted via a telecommunication line.

[0159] Note that the heart sound time series is an example of an observed time series.

[0160] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Description of Symbols

[0161] 100…Analysis system, 1…Learning device, 2…Analysis device, 11…Control unit, 12…Input unit, 13…Communication unit, 14…Memory unit, 15…Output unit, 111…Heart sound time series acquisition unit, 112…Learning process execution unit, 113…End determination unit, 114…Recording unit, 115…Output control unit, 121…Time series input unit, 122…Heart cycle state posterior distribution estimation unit, 123…Oscillator time series posterior distribution estimation unit, 124…Heart sound time series peripheral distribution estimation unit, 125…Update unit, 21…Control unit, 22…Input unit, 23…Communication unit, 24…Memory unit, 25…Output unit, 211…Analysis target acquisition unit, 212…Analysis unit, 213…Recording unit, 214…Output control unit, 91…Processor, 92…Memory, 93…Processor, 94…Memory

Claims

1. A time series acquisition unit that obtains an observation time series, which is a time series represented by a linear sum of oscillators, where the time series of the amplitudes of variable oscillators that periodically change in amplitude is used as the oscillator time series; A learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the linear sum of oscillators of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is the information probabilistically output in each state; Comprising: The learning process execution unit updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model. A learning device.

2. The mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is the information probabilistically output in each state, is a hidden semi-Markov model. The learning device according to Claim 1.

3. The observed time series is a time series of heart sounds. The learning device according to Claim 1 or 2.

4. An analysis target acquisition unit that acquires a time series to be analyzed; A time series acquisition unit that obtains an observation time series, which is a time series represented by a linear sum of oscillators, where the time series of the amplitudes of variable oscillators that periodically change in amplitude is used as the oscillator time series; a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the linear sum of oscillators of the observed time series based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is the information probabilistically output in each state; the learning process execution unit updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model. An analysis unit that analyzes the time series to be analyzed using the learned linear sum estimation learning model obtained by the learning device; Comprising: In the execution of the learned linear sum estimation learning model, the analysis unit estimates the linear sum of oscillators of the time series based on the time series, using an expression representing the generation mechanism of the time series to be analyzed obtained by the analysis target acquisition unit and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the time series and the symbol output, which is the information probabilistically output in each state. Analysis device.

5. A time series acquisition step of acquiring an observed time series that is a time series represented by a linear sum of oscillators, where the time series of the amplitudes of a variable oscillator, which is an oscillator whose amplitude changes periodically, is used as the oscillator time series; A learning process execution step of executing a linear sum estimation learning model, which is a mathematical model for estimating the linear sum of oscillators of the observed time series, based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is the information probabilistically output in each state; In the learning process execution step, the linear sum estimation learning model is updated based on the result of the execution of the linear sum estimation learning model. Learning method.

6. An analysis target acquisition step of acquiring a time series to be analyzed; A time series acquisition unit that acquires an observed time series that is a time series represented by a linear sum of oscillators, where the time series of the amplitudes of a variable oscillator, which is an oscillator whose amplitude changes periodically, is used as the oscillator time series; a learning process execution unit that executes a linear sum estimation learning model, which is a mathematical model for estimating the linear sum of oscillators of the observed time series, based on the observed time series, using an expression representing the generation mechanism of the observed time series and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the observed time series and the symbol output, which is the information probabilistically output in each state; and an analysis step of analyzing the time series to be analyzed using the learned linear sum estimation learning model obtained by a learning device that updates the linear sum estimation learning model based on the result of the execution of the linear sum estimation learning model; It has In the analysis step, in the execution of the learned linear sum estimation learning model, based on the time series, the linear sum of oscillators of the time series is estimated using an expression representing the generation mechanism of the time series obtained in the analysis target acquisition step and a mathematical model representing the relationship between the probabilistic state transition of the state of the source of the time series and the symbol output, which is the information probabilistically output in each state. Analysis method.

7. A program for causing a computer to function as the learning device according to any one of Claims 1 to 3.

8. A program for causing a computer to function as the analysis device according to Claim 4.

Images