Time-series information generation device, time-series information generation method, and program

Abstract

One aspect of the present invention is a time-series information generation device comprising: a recognition unit that recognizes observed time-series information from among a plurality of channels; and a generator that generates time-series information of a missing channel among the plurality of channels. The generator: receives, as inputs, a variable indicating a channel of the observed time-series information, time-series information of the acquired plurality of channels, and additional noise; outputs latent noise corresponding to the time-series information of the observed channel; converts the latent noise corresponding to the time-series information of the observed channel into time-series information of the plurality of channels; combines the acquired time-series information of the plurality of channels with the output of the converted time-series information of the plurality of channels; and outputs the generated time-series information.

Description

Time series information generating device, time series information generating method, and program

[0001] The present invention relates to a time-series information generating device, a time-series information generating method, and a program technology.

[0002] Heart sounds provide important clues for understanding the state of a patient's circulatory system. Heart sounds are characterized by cyclical cycles of four states: S1, systole, S2, and diastole, as the heart beats to pump blood throughout the body. The S1 and S2 sounds are extremely loud and result from the vibrations of the mitral and aortic valves, respectively, which occur when the valves close. Abnormalities in the cardiac valves cause changes in the S1 and S2 sounds. For this reason, doctors listen to heart sounds by placing a stethoscope on various positions on the patient's chest and identifying the spatial location of the abnormal sound to determine which valve is causing the abnormal sound. This type of diagnosis is based on a face-to-face meeting between the patient and doctor. Therefore, to perform a comparable examination remotely, it is essential to, for example, measure heart sounds over a wide spatial range in advance using multiple microphones.

[0003] In recent years, devices for measuring heart sound time series using multi-channel microphones have been developed. However, there is no standard for the placement of microphones in these devices. Therefore, when analyzing multiple multi-channel heart sound datasets, it is possible that a region measured by one device may not be measured by another. Furthermore, due to the patient's body shape, chest condition, or measurement errors, some microphones may not be able to measure heart sounds properly.

[0004] A method has been proposed for creating a generator that generates a one-channel heart sound time series from a dataset consisting of one-channel heart sound time series (see, for example, Non-Patent Document 1). Also, when there are gaps in the 3D point cloud data, it has been proposed to interpolate the missing data using a trained generator (see, for example, Non-Patent Document 2).

[0005] P Narv´aez and WS Percybrooks, “Synthesis of normal heart sounds using generative adversarial networks and empirical wavelet transform,” Appl Sci, vol. 10, no. 19, pp. 7003, 2020J Zhang, X Chen, Z Cai, L Pan, H Zhao, S Yi, C Yeo, B Dai, and CC Loy, “Unsupervised 3D shape completion through GAN inversion,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 1768-1777, 2021

[0006] However, in the conventional technology, there is a problem that a generator cannot be created when there are multiple missing channels. In view of the above circumstances, an object of the present invention is to provide a technology that can generate time-series information even when there are missing data.

[0007] One aspect of the present invention is a time series information generating device including: a recognition unit that recognizes time series information observed among a plurality of channels; and a generator that generates time series information of a missing channel among the plurality of channels, wherein the generator receives as input a variable indicating a channel of the observed time series information, the acquired time series information of the plurality of channels, and additional noise; outputs latent noise corresponding to the time series information of the observed channel; converts the latent noise corresponding to the time series information of the observed channel into time series information of the plurality of channels; combines the acquired time series information of the plurality of channels with the converted output of the time series information of the plurality of channels; and outputs the generated time series information.

[0008] One aspect of the present invention is a time series information generating device including: a recognition unit that recognizes time series information observed among a plurality of channels; and a generator that generates time series information of a missing channel among the plurality of channels, wherein the generator converts latent noise into time series information of the plurality of channels using a generator, receives the converted time series information of the plurality of channels, a variable indicating the channel of the observed time series information, and additional noise as input, outputs latent noise corresponding to the time series information of the observed channel, converts the latent noise corresponding to the time series information of the observed channel into time series information of the plurality of channels, combines the acquired time series information of the plurality of channels with the output of the converted time series information of the plurality of channels, and outputs the generated time series information.

[0009] One aspect of the present invention is a time series information generating method for a time series information generating device, the time series information generating method comprising: a recognition unit recognizing observed time series information from among a plurality of channels; a generator generating time series information for a missing channel from among the plurality of channels; the generator receiving a variable indicating a channel of the observed time series information, the acquired time series information of the plurality of channels, and additional noise as input; outputting latent noise corresponding to the time series information of the observed channel; converting the latent noise corresponding to the time series information of the observed channel into time series information of the plurality of channels; combining the acquired time series information of the plurality of channels and the converted output of the time series information of the plurality of channels; and outputting the generated time series information.

[0010] One aspect of the present invention is a program that causes a computer to function as the time-series information generating device.

[0011] According to the present invention, it is possible to generate time-series information even when multiple channels are missing.

[0012] FIG. 1 is a diagram illustrating an example of the configuration of a time series information generating device of an embodiment. FIG. 1 is a diagram illustrating a schematic configuration and a schematic processing example during learning. FIG. 2 is a diagram illustrating a schematic configuration and a schematic processing example during use. FIG. 2 is a diagram illustrating an example of the configuration of a learning model when all channels are generated. FIG. 3 is a flowchart illustrating an example of the processing procedure of a generator when all channels are generated. FIG. 3 is a diagram illustrating an example of the configuration of a learning model when missing channels are generated. FIG. 4 is a flowchart illustrating an example of the processing procedure of a generator when missing channels are generated. FIG. 4 is a flowchart illustrating an example of the learning procedure of a generator. FIG. 5 is a flowchart illustrating an example of the learning procedure of a classifier. FIG. 6 is a diagram illustrating an example of the arrangement of a point sound source and microphones in evaluation. FIG. 7 is time series information used in evaluation. FIG. 7 is a diagram illustrating an example of an interpolation result.

[0013] An embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is a diagram showing an example of the configuration of a time-series information generating device according to the present embodiment. As shown in Fig. 1, the time-series information generating device 1 includes, for example, an acquisition unit 11, a generator 12, an output unit 13, a classifier 14, a parameter update unit 15, a storage unit 16, and a recognition unit 17.

[0014] The acquisition unit 11 acquires multi-channel time-series information from, for example, an external device. Note that in the following example, heart sound data will be used as an example of time-series information, but time-series information is not limited to heart sound data. Furthermore, multi-channel refers to two or more channels. Note that the acquired multi-channel time-series information includes information indicating which channel the information is from. For example, the nth channel (n is an integer equal to or greater than 2) includes information indicating that the sound was collected by the nth microphone.

[0015] The recognition unit 17 recognizes the observed channels based on information included in the multi-channel time-series information acquired by the acquisition unit 11. For example, the recognition unit 17 recognizes that, in four-channel time-series information, the first channel is missing and the second to fourth channels are observed. The recognition unit 17 outputs the recognition result as a variable M indicating which channel is observed. Note that, in the case of a two-channel time series, the variable M is M={1} when the first channel is output. Also, when all channels are observed, the variable M is M={1, 2}. Also, in the case of a four-channel time series, the variable M is, for example, M={1, 3} when only the first and third channels are observed, and M={1, 2, 3, 4} when all channels are observed.

[0016] The generator 12 is configured, for example, by a neural network. The generator 12 has a function of generating time series information for all channels and a function of interpolating (generating) time series information for missing channels. During learning, the generator 12 updates parameters using a parameter update unit 15 to perform learning. The learning method will be described later. The generator 12 receives input of multi-channel time series information including missing channels, generates interpolated multi-channel time series information, and outputs the generated interpolated multi-channel time series information to the output unit 13.

[0017] The output unit 13 outputs the interpolated multi-channel time series information output by the generator 12 to, for example, an external device.

[0018] The classifier 14 is configured by, for example, a neural network and is used when training the generator 12. The classifier 14 receives, for example, multi-channel time-series information and outputs a non-negative scalar value.

[0019] The parameter update unit 15 is used during learning of the generator 12. The parameter update unit 15 calculates a loss function based on the information output by the classifier 14, and updates the parameters of the generator 12 and the classifier 14 based on the calculated loss function.

[0020] The storage unit 16 stores mathematical formulas, parameters, programs, algorithms, etc. used by the generator 12 and the classifier 14. The storage unit 16 stores, for example, the total number of microphones used when acquiring time-series information.

[0021] Note that part or all of the time-series information generating device 1 may be configured using a processor such as a CPU (Central Processing Unit) and a memory. Part or all of the time-series information generating device 1 may function as the acquisition unit 11, generator 12, output unit 13, discriminator 14, and parameter update unit 15 by the processor executing a program. Note that part or all of the functions of part or all of the time-series information generating device 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., a solid-state drive (SSD)), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0022] <Configuration during learning and use> Fig. 2 is a diagram showing a schematic configuration and a schematic processing example during learning. As shown in Fig. 2, during learning, the generator 12 receives multi-channel time series information including missing parts, and inputs the generated multi-channel time series information and interpolated multi-channel time series information to the classifier 14. The parameter updater 15 calculates a loss function based on the generated multi-channel time series information and interpolated multi-channel time series information input from the generator 12, and updates the parameters of the generator 12 and the classifier 14 based on the calculated loss function.

[0023] 3 is a diagram showing a schematic configuration and a schematic processing example during use. As shown in FIG. 3, during use, the generator 12 receives multi-channel time series information including missing parts and outputs interpolated multi-channel time series information.

[0024] Consider a PCG (phonocardiogram) signal of length L observed simultaneously by D microphones (D is an integer equal to or greater than 2). Let x(d)i∈R be the ith signal observed by the dth microphone. Ｌ (R is a double-underlined character, a real number). If we assume that some microphones fail to pick up the signal, we define the set (variable) of microphones that successfully pick up the signal as M ｉ ⊂{1, ..., D}. For example, if D = 4 and M ｉ = {1, 3}, then the first and third microphones among the four microphones have successfully recorded the i-th signal.

[0025] In this embodiment, when an observed signal (the following equation (1)) is given, a missing signal (the following equation (2)) is interpolated.

[0026]

[0027]

[0028] However, since the microphone that successfully picked up the signal is different for each i, the signal {x ｉ （ｄ）} ｄ＝１ Ｄ It is difficult to learn the joint distribution of the entire set of x, y ... （１） and x （２） In this situation, (i) if both signals are observed (M={1,2}), (ii) x （１）If only x is observed (M={1}), (iii) （２） There are three cases where only M is observed (M={2}).

[0029] <Generation of All Channels> First, the case of generating all channels will be described. Fig. 4 is a diagram showing an example of the configuration of a learning model when generating all channels. Fig. 5 is a flowchart showing an example of the processing procedure of the generator when generating all channels. In this case, the generator 12 has, for example, four layers, and uses the four layers when generating time-series data for all channels.

[0030] First, define a generator. In this embodiment, for two-channel time-series information, a generator q φ (x （１） , x （２） ), conditional generator q φ (x （２） |x （１） ), q φ (x （１） |x （２） To meet these requirements, in this embodiment, we find a generator network G φ and the inversion network F ω is used.

[0031] (Step S1) The first layer G of the generator 12 φ (FIG. 4, reference numeral 121) converts latent noise ε into two-channel time-series information as shown in the following equation (3). That is, the first layer G φ maps the latent noise ε to a multidimensional signal. （１） , x （２） is the provisional PCG signal, G φ is a generator.

[0032]

[0033] (Step S2) The second layer F of the generator 12 ω (FIG. 4, reference numeral 122) receives the variable M and the output of the first layer as input, and outputs the latent noise ε^ corresponding to the time-series information of the observed channel as shown in the following equation (4). ωis an inverse transformation network that maps the additive noise ζ to the corresponding latent noise ε̂. This second layer corresponds to the inversion network in GAN inversion. Also, ζ is the additive noise.

[0034]

[0035] (Step S3) The third layer G of the generator 12 φ (FIG. 4, reference numeral 123) converts the noise output by the second layer into two-channel time-series information as shown in the following equation (5). Note that the third layer uses the same network as the first layer.

[0036]

[0037] (Step S4) The fourth layer H (FIG. 4, reference numeral 123) of the generator 12 combines the outputs of the first and third layers as shown in the following equation (6) to output two-channel time-series information. Note that the fourth layer uses the outputs of the first layer for channels included in M ​​(=observed channels) and the outputs of the third layer for channels not included in M ​​(=missing channels).

[0038]

[0039] Furthermore, it is defined as in the following equation (7).

[0040]

[0041] Under this definition, (x （１） , x （２） The joint distribution of q depends on M. More specifically, M determines which outputs from the first and third layers are selected for the joint sample. φ，ω (x （１） , x （２ ) | M) given M (x （１） , x （２） ) and the joint distribution q φ，ω (x （１） , x （２） ) is defined as a mixture of them.

[0042]

[0043] where q(M) is the uniform distribution over the possible M. The inverse network F ω But (x （１） , x （２） If the distribution of x is well trained so that it does not change for all possible M, then the learned model can generate conditional samples given observed samples. For example, x （１） Given x （２） can be generated as shown in the following equation (9).

[0044]

[0045] It should be noted that a pool layer may be used in the second layer as shown in the following equation (10).

[0046]

[0047] In formula (10), F ω （ｄ） is the signal of microphone d converted into latent vector u （ｄ） In this case, the second equation in equation (10) returns a pooled vector for only those latent vectors contained in M. The second output layer, F ω ｏｕｔ maps the pooled vector v and the additive noise ζ to the latent noise ε̂. Note that when the corresponding signal is missing, a zero vector is added to the second layer F ω （ｄ） Note that this zero padding does not affect the output ε^ due to the definition of the second pooling layer.

[0048] <Interpolation of time series information of missing channels> Next, a configuration example and a processing example for interpolating time series information of missing channels will be described. Fig. 6 is a diagram showing a configuration example of a learning model for generating missing channels. Fig. 7 is a flowchart showing an example of the processing procedure of a generator for generating missing channels. In this case, when interpolating time series information of missing channels, the second layer (Fig. 6, reference numeral 122) to the fourth layer (Fig. 6, reference numeral 124) are used.

[0049] (Step S11) The second layer F of the generator 12 ω (Figure 6, reference numeral 122) takes the variable M, the acquired time series information, and the additional noise ζ as input, and outputs the latent noise ε^ corresponding to the time series information of the observed channel, as shown in equation (4).

[0050] (Step S12) The third layer G of the generator 12 φ (FIG. 6, reference numeral 123) converts the noise output from the second layer into two-channel time-series information as in equation (5).

[0051] (Step S13) The fourth layer H (FIG. 6, reference numeral 123) of the generator 12 combines the input to the second layer and the output of the third layer as shown in equation (6), and outputs two-channel time-series information as shown in equation (7).

[0052] <Parameter Update Processing> The parameter update unit 15 estimates the parameters of the generator 12 and the classifier 14 based on the multi-channel heart sound time series. The parameters are estimated by optimization. Specifically, the parameter update unit 15 alternately repeats an operation of updating the parameters of the generator 12 and the classifier 14 using a gradient method so as to reduce the respective loss functions. Note that the parameter update unit 15 appropriately interpolates time series information of missing channels using the generator 12 when training the classifier 14. The following describes the case of a two-channel time series.

[0053] The data distribution is q(x （１） , x （２） ), the distribution determined by the generator 12 is q φ，ω (x （１） , x （２） ) and x （１） Given x （２） The conditional distribution of q φ，ω (x （２） |x （１） ) (x （１） ) is missing, q φ，ω (x （１） |x （２） ) where the classifier 14 is θ (x （１） , x （２） )

[0054] The learning is performed in accordance with GAN using Kullback-Leibler divergence. During learning of the generator 12, the parameter update unit 15 updates the parameters φ and ω so as to minimize the loss function of the following equation (11). Note that the learning of the generator 12 is performed by updating the parameters φ and ω using the discriminator λ θ only requires q and does not use data samples. φ，ω (x （１） , x （２） ) is approximated by, for example, Monte Carlo integration using the pseudo multi-channel time series generated by the generator 12.

[0055]

[0056] The parameter update unit 15 updates (x （１） , x （２） ) are observed, the following equation (12) is minimized: （１） , x （２） The integral for ∑ is approximated by, for example, Monte Carlo integration using the observed multi-channel time series. The second term is approximated by, for example, Monte Carlo integration using the pseudo multi-channel time series generated by generator 12.

[0057]

[0058] If either of the two channels is missing, the integral of the first term cannot be replaced with the observed data, and therefore the loss function described above cannot be calculated. For this reason, in this embodiment, the generator 12 interpolates the missing channel before calculating the loss function. That is, for example, x （１） If only x is observed, then x is calculated using the generator 12. （２） After interpolating, the loss function of the following equation (13) is calculated.

[0059]

[0060] <Objective function> The objective function is log(λ θ (・) and λ θSince both (·) and (·) are included, there is a possibility that learning may become unstable due to overflow or underflow. In this embodiment, in order to stabilize learning, the objective function is rewritten and used for parameter estimation. First, the classifier 14 is defined as shown in the following equation (14).

[0061]

[0062] In equation (14), λ ０ >0 is a scalar parameter, and σ θ (・)>0 is a neural network with bounded output. From this definition, the parameter λ that maximizes the objective function is ０ can be solved in closed form as follows:

[0063]

[0064] λ in Equation (15) ０ Substituting the above, a new objective function is obtained as shown in the following equation (16).

[0065]

[0066] During learning, the integral is replaced with a Monte Carlo approximation as shown in the following equation (17).

[0067]

[0068] In formula (17), x ｉｊ （２） Haq φ，ω (x （２） |x ｉ （１） ) and jth sample from (x ｋ （１） , x ｋ （２） ) is q φ，ω (x （１） , x （２） ) is the kth sample from

[0069] <Learning of Generator> In this embodiment, in order to train the generator 12, the discriminator 14 (λ θ (x （１） , x （２） )) and train the generator 12 and the classifier 14 simultaneously in an adversarial manner.

[0070] 8 is a flowchart showing an example of a generator learning procedure. (Step S21) The parameter update unit 15 randomly generates noise ε, ζ, and a set M (variables). (Step S22) The parameter update unit 15 inputs the noise ε, ζ, and set M generated in step S21 to the generator 12, which outputs pseudo multi-channel time-series information. (Step S23) The parameter update unit 15 inputs the pseudo multi-channel time-series information output in step S22 to the discriminator 14, and calculates a loss function of the generator 12 based on the output of the discriminator 14. (Step S24) The parameter update unit 15 updates the parameters φ and ω of the generator 12 so as to reduce the loss, thereby causing the generator 12 to learn.

[0071] <Classifier Training> FIG. 9 is a flowchart showing an example of the classifier training procedure. (Step S31) The parameter updater 15 randomly generates noise ε, ζ, and set M. (Step S32) The parameter updater 15 inputs the noise ε, ζ, and set M generated in step S21 to the generator 12, which outputs pseudo multi-channel time series information. (Step S33) (When the observed multi-channel time series information is missing), the parameter updater 15 inputs M and noise ζ corresponding to the time series information of the observed channel to the generator 12, which outputs an interpolated multi-channel time series. (Step S34) The parameter updater 15 inputs the observed multi-channel time series information (or the time series information interpolated in step S33) and the pseudo multi-channel time series information output in step S32 to the classifier 14, and calculates the loss function of the classifier 14 based on the output. (Step S35) The parameter update unit 15 updates the parameter θ of the classifier 14 so as to reduce the loss, thereby causing the classifier 14 to learn.

[0072] <Evaluation> An example of evaluation results obtained by applying the above-described configuration and method is described below. In the evaluation, four microphones (Mic 1 to Mic 4) are arranged around a point sound source (Source), as shown in FIG. 10 , and sound propagates from this point sound source to the microphones at a velocity c = 10 m / s. Under this assumption, the evaluation generated a non-stationary sine wave or heart sound time series information from the point sound source, generating a multi-channel time series like the one shown in FIG. 11 . FIG. 10 is a diagram showing an example of the arrangement of the point sound source and the microphones used in the evaluation. FIG. 11 shows the time series information used in the evaluation. Image g100 is the field of view time series information generated from the non-stationary sine wave. Image g110 is the field of view time series information generated from heart sounds. Waveforms g101 and g111 are waveforms of the first channel. Waveforms g102 and g112 are waveforms of the second channel. Waveforms g103 and g113 are waveforms of the third channel, and waveforms g104 and g114 are waveforms of the fourth channel.

[0073] In the evaluation, it was assumed that 90% of the generated data was missing at least one microphone. FIG. 12 shows an example of the interpolation results. Image g120 is field of view time-series information generated from an interpolated unsteady sine wave. Image g130 is field of view time-series information generated from interpolated heart sounds. Waveform g122 is time-series information interpolated using waveforms g101, g103, and g104. Waveform g131 is time-series information interpolated using waveforms g112, g113, and g114. As shown in FIG. 12 , according to this embodiment, even when there is missing data, the missing data can be appropriately interpolated and generated.

[0074] As described above, the time series information generating device 1 configured as described above estimates the parameters of the generator 12 and the classifier 14 based on a multi-channel heart sound time series. The parameters are estimated by alternately repeating an operation of updating the parameters of the generator 12 and the classifier 14 using a gradient method to reduce their respective loss functions, thereby optimizing the parameters. When estimating the parameters of the classifier 14, the generator 12 appropriately interpolates the time series of missing channels. This makes it possible to create a generator 12 that can generate data that interpolates the time series information of missing channels for multi-channel, e.g., heart sound time series information observed by a multi-channel microphone. This makes it possible to analyze, for example, heart sounds even when data is missing.

[0075] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0076] The present invention is applicable to systems that handle time-series information, devices and systems that handle heart sounds, and the like.

[0077] 1... Time-series information generating device, 11... Acquisition unit, 12... Generator, 13... Output unit, 14... Classifier, 15... Parameter update unit, 16... Storage unit

Claims

1. A time series information generating device comprising: a recognition unit that recognizes time series information observed from among a plurality of channels; and a generator that generates time series information for a missing channel from among the plurality of channels, wherein the generator receives as input a variable indicating the channel of the observed time series information, the acquired time series information of the plurality of channels, and additional noise, and outputs latent noise corresponding to the time series information of the observed channel; converts the latent noise corresponding to the time series information of the observed channel into time series information of the plurality of channels; combines the acquired time series information of the plurality of channels and the converted output of the time series information of the plurality of channels, and outputs the generated time series information.

2. A time series information generating device comprising: a recognition unit that recognizes time series information observed from among a plurality of channels; and a generator that generates time series information for missing channels from among the plurality of channels, wherein the generator converts latent noise into time series information for the plurality of channels using a generator, receives the converted time series information for the plurality of channels, a variable indicating the channel of the observed time series information, and additional noise as input, and outputs latent noise corresponding to the time series information of the observed channel, converts the latent noise corresponding to the time series information of the observed channel into time series information for the plurality of channels, combines the acquired time series information for the plurality of channels with the output of the converted time series information for the plurality of channels, and outputs the generated time series information.

3. The time series information generating device according to claim 1 or claim 2, comprising: a classifier that outputs a non-negative scalar value using the acquired time series information of the multiple channels; and a parameter update unit that calculates a loss function based on the information output by the classifier and updates parameters of the generator and the classifier based on the calculated loss function.

4. The time series information generating device according to claim 3, wherein the parameter updating unit alternately repeats an operation of updating the parameters of the generator and the classifier by a gradient method so as to reduce the respective loss functions, and calculates the loss function by interpolating time series information of missing channels using the generator during training of the classifier.

5. The time series information generating device according to claim 3, wherein the parameter updating unit randomly generates latent noise, the additional noise, and the variables; inputs the generated latent noise, the additional noise, and the variables to the generator, causing it to output a pseudo multi-channel time series; inputs the pseudo multi-channel time series to the classifier, calculates a loss function of the generator based on the output of the classifier; and updates the parameters of the generator so as to reduce the loss, thereby causing the generator to learn.

6. The time series information generating device according to claim 3, wherein the parameter updating unit: randomly generates latent noise, the additional noise, and the variables; inputs the generated latent noise, the additional noise, and the variables to the generator and outputs a pseudo multi-channel time series; inputs the variables and the latent noise to the generator and outputs interpolated multi-channel time series information; inputs the interpolated time series information and the pseudo multi-channel time series to the classifier, calculates a loss function of the classifier based on the output of the classifier; and updates parameters of the classifier so as to reduce the loss, thereby training the classifier.

7. A time series information generating method for a time series information generating device, comprising: a recognition unit recognizing time series information observed from among a plurality of channels; a generator generating time series information for a missing channel from among the plurality of channels; the generator receiving a variable indicating the channel of the observed time series information, the acquired time series information of the plurality of channels, and additional noise as input, and outputting latent noise corresponding to the time series information of the observed channel; converting the latent noise corresponding to the time series information of the observed channel into time series information of the plurality of channels; combining the acquired time series information of the plurality of channels and the converted output of the time series information of the plurality of channels; and outputting the generated time series information.

8. A program that causes a computer to function as the time-series information generating device according to claim 1 or claim 2.

Images