Waveform generation identification method and program

JP2024167004AInactive Publication Date: 2024-11-29RICOH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023083507
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Benefits of technology

【0007】 本発明によれば、IEDの特徴的な波形情報が現れる時刻およびセンサについて、より正確な解析結果を抽出することができる。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167004000001_ABST
    Figure 2024167004000001_ABST
Patent Text Reader

Abstract

To provide waveform generation identification method and a program that allow a more accurate analysis result to be extracted on the time that characteristic waveform information on IED appears and a sensor involved in the waveform information.SOLUTION: A waveform generation identification method includes: an acquisition step for acquiring waveform data on a biological signal measured by a plurality of sensors; a probability calculation step for calculating probability information from the waveform data using a model of deep learning that has learned the waveform data with a label on whether characteristic waveform information on IED appears as teacher data; and a first extraction step for extracting the time when characteristic waveform information appears and a sensor involved in the waveform information on the basis of the probability information. The probability calculation step includes a second extraction step for extracting a characteristic map from the waveform data, a generation step for generating an attention map from the characteristic map, and an identification step for inputting information based on what is obtained by multiplying the characteristic map by the attention map, and identifying the probability information.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a waveform generation specifying method and a program. [Background technology]

[0002] In clinical epilepsy diagnosis using magnetoencephalography and electroencephalography, a technique called the equivalent current dipole method is used to evaluate the localization of epileptic lesions in the brain. In the equivalent current dipole method, the current source (dipole) that generates the magnetic field measured on the scalp is estimated. In order to estimate the dipole, it is necessary to narrow down the time when characteristic waveform information called interictal epileptic discharge (IED) occurred from the time series data detected by multiple sensors, and the sensor where the waveform information appears. In conventional analysis of electroencephalograms and magnetoencephalograms for epilepsy, doctors or technicians manually search for IEDs, but since IEDs are detected by a large number of sensors, with a sampling frequency of about 2000 [Hz] and about 30 minutes per person, it is difficult and time-consuming to manually analyze each individual IED. Therefore, a method has been proposed to identify the time of occurrence of IEDs and the sensor using deep learning.

[0003] As a technology for extracting (identifying) the time and sensor at which such characteristic waveform information of an IED (hereinafter sometimes simply referred to as characteristic waveform information) appears, a technology has been disclosed that uses deep learning algorithms such as Object Detection or Segmentation to calculate the probability that characteristic waveform information appears for each time in the waveform data detected by each sensor, and identifies the time and sensor at which the waveform information appears (for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technologies that use deep learning to extract the time and sensor at which characteristic waveform information from IEDs appears can have the problem that the extracted time and sensor may not match the analysis results of experts such as doctors, resulting in false positives and oversights.

[0005] The present invention has been made in consideration of the above, and aims to provide a waveform occurrence identification method and program that can extract more accurate analysis results for the time and sensor at which characteristic waveform information of an IED appears. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the present invention includes an acquisition step of acquiring waveform data of biological signals measured by a plurality of sensors, a probability calculation step of calculating probability information that the characteristic waveform information appears from the waveform data using a deep learning model trained using as training data a label indicating whether or not the acquired waveform data contains characteristic waveform information of an IED (Interictal Epileptiform Discharge), and a first extraction step of extracting the time and the sensor at which the characteristic waveform information appears in the waveform data based on the probability information, wherein the probability calculation step includes a second extraction step of extracting a feature map indicating the characteristics of the waveform data from the waveform data, a generation step of generating an attention map indicating a gaze area from the feature map, and an identification step of inputting information based on a product of the feature map and the attention map to identify the probability information. Effect of the Invention

[0007] According to the present invention, more accurate analysis results can be extracted regarding the time and sensor at which characteristic waveform information of an IED appears. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a biological signal measuring system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional block configuration of the server according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the information processing device according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a functional block configuration of the information processing device according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a model configuration of the probability calculation unit of the information processing device according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of an IED probability map. [Figure 7] FIG. 7 illustrates an example of a thresholded IED probability map. [Figure 8] FIG. 8 is a diagram showing an example of a probability map of an IED after extraction processing using the attention map. [Figure 9] FIG. 9 is a conceptual diagram showing an example of a method for expanding the number of extracted sensors. [Figure 10] FIG. 10 is a flowchart showing an example of the flow of the learning process of the information processing device according to the first embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of the flow of a dipole estimation process in the information processing device according to the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a model configuration of the probability calculation unit of the information processing device according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a model configuration of a probability map calculation unit of an information processing device according to the third embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a model configuration of a probability map calculation unit of an information processing device according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, with reference to the drawings, an embodiment of a waveform generation specification method and a program according to the present invention will be described in detail. Furthermore, the present invention is not limited to the following embodiment, and the components in the following embodiment include those that a person skilled in the art can easily conceive, those that are substantially the same, and those that are within the scope of what is called equivalent. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the following embodiment.

[0010] [First embodiment] (Overall configuration of biosignal measurement system) Fig. 1 is a diagram showing an example of the overall configuration of a biosignal measurement system according to the first embodiment. Fig. 2 is a diagram showing an example of the functional block configuration of a server according to the first embodiment. The overall configuration of a biosignal measurement system 1 according to this embodiment will be described with reference to Figs. 1 and 2.

[0011] The biosignal measurement system 1 is a system that measures and displays multiple types of biosignals (e.g., magneto-encephalography (MEG) signals and electro-encephalography (EEG) signals) of a subject from a specific source (body part). Note that the biosignals to be measured are not limited to magneto-encephalography (MEG) signals and electro-encephalography (EEG) signals, but may be, for example, electrical signals generated in response to cardiac activity (electrical signals that can be expressed as an electrocardiogram).

[0012] As shown in Fig. 1, the biosignal measurement system 1 includes a measurement device 3 that measures one or more biosignals of a subject, a server 40 that records one or more types of biosignals measured by the measurement device 3, and an information processing device 50 that is a biosignal display device that analyzes one or more types of biosignals recorded in the server 40. The measurement device 3 is a magnetoencephalograph that measures, for example, a brain magnetic field or a magnetoencephalogram (an example of a biosignal) generated at the timing of application of a stimulus. Note that, although the server 40 and the information processing device 50 are depicted separately in Fig. 1, for example, at least a part of the functions of the server 40 may be incorporated into the information processing device 50.

[0013] In the example of FIG. 1, the subject (subject) lies on his / her back on the measurement table 4 with electrodes (or sensors) for measuring electroencephalograms attached to his / her head, and places his / her head in a recess 32 of a Dewar 31 of the measurement device 3. The Dewar 31 is a holding container in an extremely low temperature environment using liquid helium. A number of magnetic sensors (e.g., SQUID (Superconducting Quantum Interference Device) sensors) for measuring magnetoencephalograms are arranged inside the recess 32 of the Dewar 31. The measurement device 3 collects electroencephalogram signals from the electrodes and magnetoencephalogram signals from the magnetic sensors, and outputs time-series waveform data (hereinafter, sometimes simply referred to as waveform data) including the collected electroencephalogram signals and magnetoencephalogram signals to a server 40. The waveform data output to the server 40 is read by an information processing device 50 for display and analysis. In general, the Dewar 31 and the measurement table 4 incorporating the magnetic sensors are arranged in a magnetically shielded room, but for convenience, the magnetically shielded room is not shown in FIG. 1.

[0014] The information processing device 50 is a device that analyzes waveform data of MEG signals from multiple magnetic sensors and waveform data of EEG signals from multiple electrodes. For example, the information processing device 50 displays waveform data of MEG signals and EEG signals in synchronization on the same time axis. Here, the EEG signal is a signal that represents the electrical activity of nerve cells (the flow of ionic charges that occurs in the dendrites of neurons during synaptic transmission) as a voltage value between electrodes. Also, the MEG signal is a signal that represents minute electric field fluctuations caused by electrical activity of the brain.

[0015] As shown in FIG. 2, the server 40 includes a data acquisition unit 401 and a data storage unit 402.

[0016] The data acquisition unit 401 is a functional unit that periodically acquires waveform data such as MEG signals and EEG signals measured by the measurement device 3. The waveform data includes time series data of each MEG signal measured by a plurality of magnetic sensors in the Dewar 31 of the measurement device 3, and time series data of each signal measured by a plurality of electrodes for measuring EEG attached to the head of the subject (person to be measured).

[0017] The data storage unit 402 is a functional unit that stores the waveform data acquired from the measurement device 3 .

[0018] 1 shows a configuration in which the measuring device 3 and the server 40 are directly connected, and the server 40 and the information processing device 50 are directly connected, but they may be configured to communicate data with each other via a network. The network connection method may be wired or wireless. The server 40 may also be configured to be a server on a network and connected to the cloud, for example.

[0019] (Hardware configuration of information processing device) 3 is a diagram showing an example of a hardware configuration of an information processing device according to the first embodiment. The hardware configuration of an information processing device 50 according to the present embodiment will be described with reference to FIG.

[0020] As shown in FIG. 3, the information processing device 50 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, an auxiliary storage device 104, a network I / F 105, an input device 106, and a display device 107.

[0021] The CPU 101 is a calculation device that controls the overall operation of the information processing device 50. The RAM 102 is a volatile storage device that is used as a work area for the CPU 101. The ROM 103 is a non-volatile storage device that stores programs for the information processing device 50.

[0022] The auxiliary storage device 104 is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD) that stores various data, programs, and the like.

[0023] The network I / F 105 is an interface for communicating data with an external device such as the server 40 via a network. The network I / F 105 is, for example, a network interface card (NIC) that supports Ethernet (registered trademark) and is capable of wired or wireless communication in accordance with TCP (Transmission Control Protocol) / IP (Internet Protocol) or the like.

[0024] The input device 106 is a touch panel input function, a mouse, a keyboard, etc. for selecting letters, numbers, various instructions, and moving a cursor.

[0025] The display device 107 is a display configured with liquid crystal or organic EL (Electro-Luminescence) or the like, which displays various information such as a cursor, a menu, a window, characters, or an image.

[0026] The above-mentioned CPU 101, RAM 102, ROM 103, auxiliary storage device 104, network I / F 105, input device 106 and display device 107 are communicably connected to one another via a bus 108 such as an address bus and a data bus.

[0027] It should be noted that the hardware configuration of the information processing device 50 shown in FIG. 3 is merely an example, and it is not necessary for the information processing device 50 to include all of the components shown in FIG. 3, or it may include other components.

[0028] (Configuration and operation of functional blocks of information processing device) FIG. 4 is a diagram showing an example of the configuration of the functional blocks of the information processing device according to the first embodiment. FIG. 5 is a diagram showing an example of a model configuration of the probability calculation unit of the information processing device according to the first embodiment. FIG. 6 is a diagram showing an example of a probability map of an IED. FIG. 7 is a diagram showing an example of a probability map of an IED after threshold processing. FIG. 8 is a diagram showing an example of a probability map of an IED after extraction processing using an attention map. FIG. 9 is a conceptual diagram showing an example of a method for expanding the number of extracted sensors. The configuration and operation of the functional blocks of the information processing device 50 according to this embodiment will be described with reference to FIGS. 4 to 9. In particular, FIG. 5 details the model configuration of the probability calculation unit 503 based on ABN (Attention Branch Network) (see, for example, Non-Patent Document 1), which is an example of a deep learning algorithm.

[0029] As shown in FIG. 4, the information processing device 50 includes an acquisition unit 501, a preprocessing unit 502, a probability calculation unit 503, a threshold processing unit 504, a time / sensor extraction unit 505, and a dipole estimation unit 506.

[0030] The acquiring unit 501 is a functional unit that acquires waveform data of magnetoencephalographic signals and electroencephalographic signals from the server 40 via the network I / F 105 .

[0031] The pre-processing unit 502 is a functional unit that performs pre-processing such as sensor extraction / extension, down-sampling, application of frequency filters, artifact removal, bad channel processing, extraction of time windows (predetermined sections), and standardization of magnetic field data on the waveform data acquired by the acquisition unit 501.

[0032] For sensor extraction and expansion, when calculating the IED probability value described later, all sensors are used, or only a certain number of sensors that have been grouped in advance are used. Sensor grouping can be based on anatomical criteria such as the temporal lobe and the frontal lobe, or simply a given number of nearby sensors are grouped together. In addition, if the number of sensors used in the calculation is less than the number of sensors used in the learning process described later, it is also possible to expand the number of fictitious sensors.

[0033] Downsampling is applied to match the sampling frequency used during training. The same frequency filter is also applied as the filtering applied during training. Commonly used filters include a 35[Hz] low-pass filter and a 3[Hz]~35[Hz] band-pass filter.

[0034] For artifact removal, ICA (see, for example, Non-Patent Document 2) or DSSP (see, for example, Non-Patent Document 3) is used for the purpose of removing cardiac artifacts and artifacts due to blinking and body movement.

[0035] The bad channel processing refers to a process of excluding a magnetic sensor that observes a magnetic field change exceeding a preset threshold, or of interpolating using surrounding sensor values.

[0036] There are several methods for extracting time windows, including shifting them by the length of the time window without overlapping, overlapping them by half the length of the time window, or overlapping them by 1 / 4 the length. When overlapping, the overlapping portion is averaged when calculating the IED probability value described below.

[0037] Standardization of the magnetic field data is performed so that the average is 0 and the variance is 1 within the extracted time window. In addition to standardization, a method of normalizing the magnetic field within a preset range to be between -1 and 1 may be used.

[0038] The probability calculation unit 503 is a functional unit that calculates a probability value (i.e., a value between 0 and 1) (an example of probability information) that characteristic waveform information of an IED appears in the waveform data preprocessed by the preprocessing unit 502. Specifically, the probability calculation unit 503 calculates the probability value by inference using a trained model based on ABN, which is an example of a deep learning algorithm. The ABN divides a CNN (Convolutional Neural Network) network into two, with the first stage being a feature map extraction mechanism and the second stage being a perception branch that outputs a probability value based on a feature map weighted by an attention map output from the attention branch. The characteristic waveform information of an IED indicates waveform information such as spike wave, spike and wave, poly spike and wave, and sharp wave that are characteristic of an epileptic lesion. The trained model described above may output a probability map (an example of probability information) as shown in FIG. 6 (to be described later) that indicates a probability distribution where characteristic waveform information of an IED appears, rather than inferring a probability value from input waveform data. In addition, the probability calculation unit 503 can input waveform data of a specific sensor extracted at a specified time (time window) by the pre-processing unit 502 into the learned model and obtain a probability value as output, thereby constructing a probability map from the correspondence between the time, sensor, and probability value.

[0039] As shown in FIG. 5, the probability calculation unit 503 includes a feature extraction unit 5031, an attention generation unit 5032, and a classification unit 5033.

[0040] The feature extraction unit 5031 is a functional unit that inputs the waveform data preprocessed by the preprocessing unit 502 and generates a feature map indicating the distribution of characteristic information extracted from the waveform data by a predetermined kernel. The feature extraction unit 5031 corresponds to a feature map extraction mechanism in the front stage of ABN.

[0041] The attention generation unit 5032 is a functional unit that generates an attention map indicating a gaze area from the feature map generated by the feature extraction unit 5031. The attention generation unit 5032 corresponds to an Attention branch of an ABN. Here, the feature map and the attention map have the same size. In addition, the attention map generated by the attention generation unit 5032 is output to the time / sensor extraction unit 505.

[0042] The classification unit 5033 is a functional unit that inputs a weighted feature map obtained by multiplying the feature map output from the feature extraction unit 5031 and the attention map output from the attention generation unit 5032, and further adds the feature map itself to the weighted feature map, and classifies (calculates) a probability value of the appearance of characteristic waveform information of an IED based on the input information. The classification unit 5033 corresponds to the Percption branch of ABN.

[0043] As described above, the model trained by the ABN of the probability calculation unit 503 is trained using a feature map weighted by multiplying a feature map for waveform data by an attention map indicating a gaze area. This allows the weighted feature map to be input to the recognition unit 5033, which is the perception branch, and training can be focused on the gaze area indicated by the attention map, making it possible to output the probability value of an IED with high accuracy.

[0044] Note that, as information input to the classification unit 5033, the feature map weighted by multiplying it with the attention map is further added with the feature map itself, but this addition is not necessarily required. However, by adding the feature map itself, it is possible to emphasize the feature map in the area where the attention map reacts strongly, and it also has the effect of not losing the feature map in the area where the value is 0 in the attention map.

[0045] FIG. 6 shows an example of a probability map calculated by the probability calculation unit 503. In FIG. 6, the horizontal axis represents time, and the vertical axis represents the index of the magnetic sensor (or electrode) (hereinafter, sometimes simply referred to as the sensor). In the example shown in FIG. 6, it is estimated that an IED appeared when the time index is 100 and the sensor index is around 40 to 80. Note that, although FIG. 6 shows the IED probability in shades of black and white, a color display in which the color changes according to the IED probability may also be used.

[0046] Returning to FIG. 4, the explanation will be continued.

[0047] The threshold processing unit 504 is a functional unit that judges whether or not the probability value calculated by the probability calculation unit 503 is equal to or greater than a predetermined threshold. In the case where a probability map is calculated by the probability calculation unit 503, the threshold processing unit 504 can also narrow down the probability map to determine whether or not it is equal to or greater than a predetermined threshold. An example of a probability map after the narrowing down is shown in FIG. 7. In the example shown in FIG. 7, the threshold processing unit 504 uses "0.5" as the threshold. That is, the probability map shown in FIG. 7 corresponds to an extraction of an area where characteristic waveform information of an IED exists with a probability of 50% or more.

[0048] The time / sensor extraction unit 505 is a functional unit that extracts a region showing a strong reaction in the attention map generated by the attention generation unit 5032 of the probability calculation unit 503, i.e., the time and sensor of the gazed region indicated by the attention map, from the probability value determined to be equal to or greater than the threshold by the threshold processing unit 504. Here, assuming that the probability map is calculated by the probability calculation unit 503, an example of the probability map of the region showing a strong reaction by the time / sensor extraction unit 505 using the attention map is shown in FIG. 8. In the example shown in FIG. 8, the time / sensor extraction unit 505 extracts a time 601 and a sensor group 602 from the probability map of the region showing a strong reaction using the attention map.

[0049] As another method of extracting sensors by the time / sensor extraction unit 505, a group of sensors may be defined in advance, and all sensors belonging to the group to which the extracted sensor belongs may be expanded as the extracted sensor, thereby improving the stability of dipole estimation. Here, the group is basically set according to an anatomical standard such as the vicinity of the temporal lobe or the vicinity of the frontal lobe, but a neighboring area including any number of sensors may simply be set as one group.

[0050] In addition, the time / sensor extraction unit 505 is not limited to extracting the time and sensor using an attention map for the probability value determined by the threshold processing unit 504 to be equal to or greater than the threshold, but may also extract the time and sensor using an attention map for the probability value calculated by the probability calculation unit 503.

[0051] Here, FIG. 9 shows an example of a sensor expansion method. FIG. 9(a) shows a head 700 of a subject, a sensor 710 arranged to cover the head 700, and a group 720 of the sensors 710 set in advance. In FIG. 9(b), the sensor 710a shown by a black circle represents a sensor extracted at the stage where the threshold processing unit 504 narrows down the number of sensors. It is possible to perform dipole estimation using only the sensor 710a extracted at the stage where the threshold processing unit 504 narrows down the number of sensors, but as described above, it is also possible to expand the number of sensors in order to improve the stability of the dipole estimation solution. FIG. 9(c) shows the result of expanding the number of sensors. That is, as shown in FIG. 9(c), the sensor 710a extracted at the stage where the threshold processing unit 504 makes a threshold judgment is expanded to all the sensors 710 in the group 720 to which the sensor 710a belongs. This improves the stability of the dipole estimation.

[0052] The dipole estimation unit 506 is a functional unit that performs dipole estimation (for example, see Non-Patent Document 4) using the waveform data corresponding to the time and sensor extracted by the time / sensor extraction unit 505.

[0053] The above-mentioned acquiring unit 501, pre-processing unit 502, probability calculation unit 503, threshold processing unit 504, time / sensor extraction unit 505 and dipole estimation unit 506 are realized by executing a program by CPU 101 shown in Fig. 3. Note that some or all of the functional units of the acquiring unit 501, pre-processing unit 502, probability calculation unit 503, threshold processing unit 504, time / sensor extraction unit 505 and dipole estimation unit 506 may be realized by a hardware circuit (integrated circuit) such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC) instead of a software program.

[0054] Note that the functions of each functional unit of the information processing device 50 shown in Fig. 4 are conceptually shown, and the configuration is not limited to this. For example, the multiple functional units illustrated as independent functional units in the information processing device 50 shown in Fig. 4 may be configured as one functional unit. On the other hand, the function of one functional unit in the information processing device 50 shown in Fig. 4 may be divided into multiple functions and configured as multiple functional units.

[0055] (Learning process flow of information processing device) 10 is a flowchart showing an example of the flow of the learning process of the information processing device according to the first embodiment. The flow of the learning process of the information processing device 50 according to the present embodiment will be described with reference to FIG.

[0056] <Step S11> The acquisition unit 501 acquires waveform data to be used as training data for learning in the probability calculation unit 503. Next, the preprocessing unit 502 cuts out the waveform data acquired by the acquisition unit 501 for every certain interval (for example, 1 second) as preprocessing. Then, for the waveform data for every certain interval that has been preprocessed, a label of a positive IED case is added if characteristic waveform information of an IED appears, and a label of a negative IED case is added if characteristic waveform information of an IED does not appear, and the data is used as training data (training data) to be used for learning in the probability calculation unit 503. For example, the waveform data to which the label of a positive IED case is added may be waveform data that has already been used in dipole estimation. Then, the probability calculation unit 503 inputs the training data to which the label is added. Then, the process proceeds to step S12.

[0057] <Step S12> Using the input teacher data, the probability calculation unit 503 performs a learning process based on the ABN algorithm for outputting an IED probability value from the identification unit 5033. Then, the process proceeds to step S13.

[0058] <Step S13> Then, the probability calculation unit 503 generates a trained model of the ABN as a result of the learning process in step S12. The generated trained model is used for inference to detect an IED from unknown waveform data.

[0059] (Flow of dipole estimation process of information processing device) 11 is a flowchart showing an example of the flow of the dipole estimation process of the information processing device according to the first embodiment. With reference to FIG. 11, the flow of the dipole estimation process of the information processing device 50 according to the present embodiment will be described.

[0060] <Step S21> The acquiring unit 501 acquires waveform data of the magnetoencephalographic signal and the electroencephalographic signal from the server 40 via the network I / F 105. Then, the process proceeds to step S22.

[0061] <Step S22> The preprocessing unit 502 performs preprocessing such as extraction and expansion of sensors, downsampling, application of frequency filters, removal of artifacts, processing of bad channels, extraction of time windows, and standardization of magnetic field data on the waveform data acquired by the acquisition unit 501. Then, the process proceeds to step S23.

[0062] <Step S23> The probability calculation unit 503 inputs the waveform data preprocessed by the preprocessing unit 502, and calculates (infers) the probability value of an IED using the trained model of the ABN generated by the above-mentioned learning process. The specific calculation process of the probability value of an IED in the probability calculation unit 503 is as described above in FIG. 5. Then, the process proceeds to step S24.

[0063] <Step S24> Threshold processing unit 504 determines whether or not the probability value calculated by probability calculation unit 503 is equal to or greater than a predetermined threshold. If the probability value is equal to or greater than the predetermined threshold (step S24: Yes), the process proceeds to step S25, and if the probability value is less than the predetermined threshold (step S24: No), it is determined that the waveform data input to probability calculation unit 503 does not contain waveform information characteristic of an IED, and the dipole estimation process ends.

[0064] <Step S25> The time / sensor extraction unit 505 extracts the time and the sensor of the area showing a strong reaction in the attention map generated by the attention generation unit 5032 of the probability calculation unit 503 from the probability value determined to be equal to or greater than the threshold by the threshold processing unit 504. Then, the process proceeds to step S26.

[0065] <Step S26> The dipole estimation unit 506 performs dipole estimation using the waveform data corresponding to the time and sensor extracted by the time / sensor extraction unit 505. This ends the dipole estimation process.

[0066] As described above, in the information processing device 50 according to the present embodiment, the acquisition unit 501 acquires waveform data of biological signals measured by a plurality of sensors, and the probability calculation unit 503 calculates a probability value of characteristic waveform information appearing from the waveform data using a learned model of deep learning (e.g., ABN) that has been learned using as teacher data a data that has been labeled with a label indicating whether characteristic waveform information of an IED appears in the waveform data, and the time / sensor extraction unit 505 extracts the time and sensor at which characteristic waveform information appears in the waveform data based on the probability value. In the probability calculation unit 503, the feature extraction unit 5031 extracts a feature map indicating the features of the waveform data from the waveform data, the attention generation unit 5032 generates an attention map indicating a gaze area from the feature map, and the identification unit 5033 inputs information based on a product of the feature map and the attention map, and identifies the probability value. This makes it possible to extract more accurate analysis results for the time and sensor at which characteristic waveform information of an IED appears.

[0067] [Second embodiment] The biosignal measurement system 1 according to the second embodiment will be described, focusing on the differences from the biosignal measurement system 1 according to the first embodiment. In this embodiment, a method will be described in which, in addition to labeled waveform data, as teacher data, information about the time (sampling time) and the sensor (master information) recorded when dipole estimation is performed is used for waveform data in which characteristic waveform information of an IED appears. The overall configuration of the biosignal measurement system 1 according to this embodiment and the hardware configuration of the information processing device 50 are the same as those described in the first embodiment.

[0068] (Configuration and operation of functional blocks of information processing device) Fig. 12 is a diagram showing an example of a model configuration of a probability calculation unit of an information processing device according to the second embodiment. The configuration and operation of functional blocks of an information processing device 50 according to the present embodiment will be described with reference to Fig. 12.

[0069] The information processing device 50 according to this embodiment has a configuration in which the probability calculation unit 503 in the functional block configuration shown in Fig. 4 described above is replaced with a probability calculation unit 503a shown in Fig. 12. As shown in Fig. 12, the probability calculation unit 503a has a feature extraction unit 5031, an attention generation unit 5032, and a classification unit 5033. Note that the functions of the feature extraction unit 5031 and the classification unit 5033 are similar to those of the information processing device 50 of the first embodiment described above.

[0070] The attention generation unit 5032 is a functional unit that generates an attention map indicating a gaze area from the feature map generated by the feature extraction unit 5031. Here, the feature map and the attention map have the same size. In addition, the attention map generated by the attention generation unit 5032 is output to the time / sensor extraction unit 505.

[0071] Here, the model trained by the ABN of the probability calculation unit 503a uses, as teacher data, information on the time (sampling time) and the sensor recorded when performing dipole estimation, etc., in addition to labeled waveform data, for waveform data in which characteristic waveform information of an IED appears. In this case, the information on the time (sampling time) and the sensor is referred to as master information. Then, the probability calculation unit 503a compares the attention map generated by the attention generation unit 5032 with the master information, and generates a trained model by performing training so as to reduce the difference between the two.

[0072] Usually, when detecting IEDs using data from epilepsy patients, the master information used as training data is the result of analysis by experts such as doctors or medical technicians. In this way, because training is performed using master information that reflects the expert's knowledge as training data, the expert's knowledge can be incorporated into the trained model. This makes it possible to extract more accurate analysis results regarding the time and sensor at which characteristic waveform information of an IED appears.

[0073] In addition, by fine-tuning the weights of the attention generation unit 5032 (attention branch) and the classification unit 5033 (perception branch) in Fig. 12 as the initial weights of the trained model (see, for example, Non-Patent Document 5), it becomes possible to train with even higher accuracy. In addition, instead of using pre-prepared master information, a method may be used in which the attention map output from the attention generation unit 5032 is checked, and then the area to be noted (fixated area) is manually corrected, and fine-tuning is performed using the corrected attention map.

[0074] [Third embodiment] The biosignal measurement system 1 according to the third embodiment will be described, focusing on the differences from the biosignal measurement system 1 according to the first embodiment. In the first and second embodiments described above, a method using a trained model based on ABN was explained. In this embodiment, a method using a model based on Deep U-Net, which is one of segmentation networks, will be explained. Note that the overall configuration of the biosignal measurement system 1 according to this embodiment and the hardware configuration of the information processing device 50 are similar to the configurations explained in the first embodiment.

[0075] (Configuration and operation of functional blocks of information processing device) Fig. 13 is a diagram showing an example of a model configuration of a probability map calculation unit of an information processing device according to the third embodiment. The configuration and operation of functional blocks of an information processing device 50 according to this embodiment will be described with reference to Fig. 13.

[0076] The information processing device 50 according to this embodiment has a configuration in which the probability calculation unit 503 in the functional block configuration shown in FIG. 4 is replaced with a probability map calculation unit 503b shown in FIG. 13. The probability map calculation unit 503b is a functional unit that calculates information on the probability that characteristic waveform information of an IED appears in the waveform data preprocessed by the preprocessing unit 502, similar to the above-mentioned probability calculation units 503 and 503a. Specifically, the probability map calculation unit 503b calculates a probability map by inference using a trained model based on a configuration based on Deep U-Net (see, for example, Non-Patent Document 6), which is one of segmentation networks that is an example of a deep learning algorithm. Deep U-Net is configured as an encoder that extracts a feature map in the front stage and a decoder that outputs a probability map based on the feature map output from the encoder in the rear stage. Note that the deep learning algorithm used by the probability map calculation unit 503b is not limited to Deep U-Net, and may be a normal U-Net.

[0077] As shown in Fig. 13, the probability map calculation unit 503b has a feature extraction unit 5031b, an attention generation unit 5032, and a classification unit 5033b. In Fig. 13, the feature extraction unit 5031b corresponds to the encoder of Deep U-Net, and the classification unit 5033b corresponds to the decoder of Deep U-Net. That is, the configuration of the probability map calculation unit 503b shown in Fig. 13 is a configuration in which an attention generation unit 5032 (Attention branch) is added to the model configuration of Deep U-Net.

[0078] The feature extraction unit 5031b is a functional unit that inputs the waveform data that has been preprocessed by the preprocessing unit 502, and generates a feature map by performing convolution processing on the waveform data over several stages ("Res Block" shown in FIG. 13).

[0079] The attention generation unit 5032 is a functional unit that generates an attention map indicating a gaze area from the feature map generated by the feature extraction unit 5031b. Here, the feature map and the attention map have the same size. In addition, the attention map generated by the attention generation unit 5032 is output to the time / sensor extraction unit 505.

[0080] The identification unit 5033b is a functional unit that inputs a feature map weighted by multiplying the feature map output from the feature extraction unit 5031b and the attention map output from the attention generation unit 5032, and further adds the feature map itself to the weighted feature map, and identifies (calculates) a probability map indicating a probability distribution in which characteristic waveform information of an IED appears based on the input information. Specifically, the identification unit 5033b identifies (calculates) a probability map indicating a probability distribution in which characteristic waveform information of an IED appears by performing a process reverse to the convolution process on the input weighted feature map.

[0081] As described above, the model learned by the Deep U-Net-based configuration of the probability map calculation unit 503b is learned using a feature map weighted by multiplying the feature map for the waveform data by the attention map indicating the gaze area. The teacher data used for this learning is the same as the teacher data described in the first embodiment. As a result, the weighted feature map is input to the identification unit 5033b, which is a decoder, and learning can be focused on the gaze area indicated by the attention map, and the probability map of the IED can be output with high accuracy.

[0082] Note that, as information input to the classification unit 5033b, the feature map itself is added to the feature map weighted by multiplying it by the attention map, but this addition is not necessarily required. However, by adding the feature map itself, the feature map of the area where the attention map reacts strongly can be emphasized, and there is also an effect of not losing the feature map in the area where the value is 0 in the attention map.

[0083] [Fourth embodiment] The biosignal measurement system 1 according to the fourth embodiment will be described, focusing on the differences from the biosignal measurement system 1 according to the third embodiment. In this embodiment, in addition to the labeled waveform data used as teacher data in the third embodiment, a method will be described in which, for waveform data in which characteristic waveform information of an IED appears, the time (sampling time) recorded when dipole estimation is performed and information about the sensor (master information) are used. The overall configuration of the biosignal measurement system 1 according to this embodiment and the hardware configuration of the information processing device 50 are the same as those described in the first embodiment.

[0084] (Configuration and operation of functional blocks of information processing device) Fig. 14 is a diagram showing an example of a model configuration of a probability map calculation unit of an information processing device according to the fourth embodiment. The configuration and operation of functional blocks of an information processing device 50 according to the present embodiment will be described with reference to Fig. 14.

[0085] The information processing device 50 according to this embodiment has a configuration in which the probability calculation unit 503 in the functional block configuration shown in Fig. 4 described above is replaced with a probability map calculation unit 503c shown in Fig. 14. As shown in Fig. 14, the probability map calculation unit 503c has a feature extraction unit 5031b, an attention generation unit 5032, and a classification unit 5033b. Note that the functions of the feature extraction unit 5031b and the classification unit 5033b are similar to those of the information processing device 50 of the third embodiment described above.

[0086] The attention generation unit 5032 is a functional unit that generates an attention map indicating a gaze area from the feature map generated by the feature extraction unit 5031b. Here, the feature map and the attention map have the same size. In addition, the attention map generated by the attention generation unit 5032 is output to the time / sensor extraction unit 505.

[0087] Here, the model learned by the Deep U-Net-based configuration of the probability map calculation unit 503c uses, as teacher data, not only labeled waveform data, but also master information about the time (sampling time) recorded when dipole estimation was performed and the sensor for waveform data in which characteristic waveform information of an IED appears.The probability map calculation unit 503c compares the attention map generated by the attention generation unit 5032 with the master information, and generates a learned model by learning so as to reduce the difference between the two.

[0088] In this way, because learning is performed using master information that reflects expert knowledge as training data, the knowledge of experts can be incorporated into the trained model, making it possible to extract more accurate analysis results for the time and sensor at which characteristic waveform information of an IED appears.

[0089] In addition, instead of using pre-prepared master information, a method may be used in which the attention map output from the attention generation unit 5032 is checked, the area to be noted (gaze area) is manually corrected, and the corrected attention map is compared.

[0090] In each of the above-mentioned embodiments, when at least one of the functional units of the information processing device 50 is realized by executing a program, the program is provided by being incorporated in advance in a ROM or the like. In each of the above-mentioned embodiments, the program executed by the information processing device 50 may be provided by being recorded in a computer-readable recording medium such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk-Recordable), or a DVD (Digital Versatile Disc) in the form of an installable or executable file. In each of the above-mentioned embodiments, the program executed by the information processing device 50 may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. In each of the above-mentioned embodiments, the program executed by the information processing device 50 may be provided or distributed via a network such as the Internet. Furthermore, in each of the above-mentioned embodiments, the program executed by the information processing device 50 has a modular configuration including at least any one of the above-mentioned functional units, and in terms of actual hardware, the CPU 101 reads out and executes the program from the above-mentioned storage device (e.g., ROM 103 or auxiliary storage device 104, etc.), so that each of the above-mentioned functional units is loaded and generated on the main storage device (RAM 102).

[0091] The aspects of the present invention are as follows. <1> An acquisition step of acquiring waveform data of biological signals measured by a plurality of sensors; A probability calculation step of calculating probability information of the occurrence of characteristic waveform information from the waveform data using a deep learning model trained with teacher data in which a label indicating whether or not characteristic waveform information of an IED (Interictal Epileptiform Discharge) appears in the acquired waveform data; a first extraction step of extracting a time and the sensor at which the characteristic waveform information appears in the waveform data based on the probability information; having The probability calculation step includes: a second extraction step of extracting a feature map indicating features of the waveform data from the waveform data; A generating step of generating an attention map indicating a gaze area from the feature map; an identification step of inputting information based on a product of the feature map and the attention map to identify the probability information; The method for identifying waveform occurrence includes the steps of: <2> In the first extraction step, the time and the sensor of the gazed area indicated by the attention map generated in the generation step are extracted based on the probability information. <1> 2 is a waveform generation identification method according to the present invention. <3> In the identification step, information obtained by multiplying the feature map and the attention map and adding the feature map to the product is input to identify the probability information. <1> or <2> 2 is a waveform generation identification method according to the present invention. <4> The method further includes a threshold processing step of determining whether the probability information calculated in the probability calculation step is equal to or greater than a predetermined threshold, In the first extraction step, the time and the sensor at which the characteristic waveform information appears in the waveform data are extracted based on the probability information determined to be equal to or greater than the threshold in the threshold processing step. <1> ~ <3> 13. A waveform occurrence identification method according to claim 12. <5> In the probability calculation step, master information, which is the time when the characteristic waveform information appears and information of the sensor, which is further prepared in advance, is used as teacher data, and the model trained so that a difference between the master information and the attention map generated in the generation step is reduced. <1> ~ <4> 13. A waveform occurrence identification method according to claim 12. <6> In the probability calculation step, the model trained using the attention map generated in the generation step that is manually modified is used. <1> ~ <4> 13. A waveform occurrence identification method according to claim 12. <7> The method further includes a pre-processing step of performing a pre-processing on the waveform data acquired in the acquisition step, the pre-processing step including a process of extracting at least a predetermined section of the waveform data, The probability calculation step calculates the probability information from the pre-processed waveform data using the model. <1> ~ <6> 13. A waveform occurrence identification method according to claim 12. <8> In the probability calculation step, the model based on an Attention Branch Network (ABN) is used, The generation step is processed by a function of the attention branch of the ABN, The identification step is performed by a function of the Perception branch of the ABN. <1> ~ <7> 13. A waveform occurrence identification method according to claim 12. <9> In the probability calculation step, the model based on Deep U-Net is used, The second extraction step is processed by an encoder function of the Deep U-Net, The identification step is processed by the decoder function of the Deep U-Net. <1> ~ <7> 13. A waveform occurrence identification method according to claim 12. <10> On the computer, An acquisition step of acquiring waveform data of biological signals measured by a plurality of sensors; A probability calculation step of calculating probability information of the occurrence of characteristic waveform information from the waveform data using a deep learning model trained with teacher data in which a label indicating whether or not characteristic waveform information of an IED (Interictal Epileptiform Discharge) appears in the acquired waveform data; a first extraction step of extracting a time and the sensor at which the characteristic waveform information appears in the waveform data based on the probability information; A program for executing The probability calculation step includes: a second extraction step of extracting a feature map indicating features of the waveform data from the waveform data; A generating step of generating an attention map indicating a gaze area from the feature map; an identification step of inputting information based on a product of the feature map and the attention map to identify the probability information; It is a program that has the following features. [Explanation of symbols]

[0092] 1. Biosignal measurement system 3. Measuring Equipment 4 Measurement Table 31 Duwa 32 Depression 40 Servers 50 Information processing device 101 CPU 102 RAM 103 ROM 104 Auxiliary storage 105 Network I / F 106 Input Devices 107 Display device 108 Bus 401 Data Acquisition Department 402 Data storage unit 501 Acquisition Department 502 Pre-processing section 503, 503a Probability calculation section 503b, 503c Probability map calculation unit 504 Threshold processing unit 505 Time and Sensor Extraction Unit 506 Dipole Estimation Unit 601 Time 602 Sensor Group 700 heads 710, 710a Sensor 720 Group 5031, 5031b Feature Extraction Unit 5032 Attention Generation Unit 5033, 5033b Identification Unit

Prior Art Documents

Patent Documents

[0093]

Patent Document 1

Non-Patent Documents

[0094]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

Outdoor Tools 4

Direct Environment 5

Outdoor Configuration 6

Claims

1. An acquisition step of acquiring waveform data of biological signals measured by a plurality of sensors; a probability calculation step of calculating probability information of the occurrence of characteristic waveform information from the waveform data by using a deep learning model trained using teacher data obtained by adding a label indicating whether characteristic waveform information of IED (Internal Epileptiform Discharge) appears in the acquired waveform data; a first extraction step of extracting a time and the sensor at which the characteristic waveform information appears in the waveform data based on the probability information; having The probability calculation step includes: a second extraction step of extracting a feature map indicating features of the waveform data from the waveform data; A generating step of generating an attention map indicating a gaze area from the feature map; an identification step of inputting information based on a product of the feature map and the attention map to identify the probability information; The method for determining waveform occurrence comprising:

2. The waveform occurrence identification method according to claim 1 , wherein the first extraction step extracts the time and the sensor of the gazed upon area indicated by the attention map generated in the generation step based on the probability information.

3. 3. The method for identifying waveform occurrence according to claim 1, wherein in the identification step, information obtained by multiplying the feature map and the attention map and adding the feature map to the product is input to identify the probability information.

4. The method further includes a threshold processing step of determining whether the probability information calculated in the probability calculation step is equal to or greater than a predetermined threshold, 3. The waveform occurrence identification method according to claim 1 or 2, wherein in the first extraction step, the time and the sensor at which the characteristic waveform information appears in the waveform data are extracted based on the probability information determined to be equal to or greater than the threshold in the threshold processing step.

5. The waveform occurrence identification method described in claim 1 or 2, wherein in the probability calculation step, master information which is prepared in advance and is the time at which the characteristic waveform information appears and information from the sensor is used as training data, and the model is trained so that the difference between the master information and the attention map generated in the generation step is small.

6. 3. The method of claim 1, wherein the probability calculation step uses the model trained using a manually modified version of the attention map generated in the generation step.

7. The method further includes a pre-processing step of performing a pre-processing on the waveform data acquired in the acquisition step, the pre-processing step including a process of extracting at least a predetermined section of the waveform data, 3. The method according to claim 1, wherein in the probability calculation step, the probability information is calculated from the pre-processed waveform data by using the model.

8. In the probability calculation step, the model based on an Attention Branch Network (ABN) is used, The generating step is processed by a function of the Attention branch of the ABN, 3. The method for identifying waveform occurrence according to claim 1, wherein said identifying step is performed by a function of a Perception branch of said ABN.

9. In the probability calculation step, the model based on Deep U-Net is used, The second extraction step is processed by the function of the Deep U-Net encoder; The method for identifying waveform occurrence according to claim 1 or 2, wherein the identification step is processed by a decoder function of the Deep U-Net.

10. On the computer, An acquisition step of acquiring waveform data of biological signals measured by a plurality of sensors; a probability calculation step of calculating probability information of the occurrence of characteristic waveform information from the waveform data by using a deep learning model trained using teacher data obtained by adding a label indicating whether characteristic waveform information of IED (Internal Epileptiform Discharge) appears in the acquired waveform data; a first extraction step of extracting a time and the sensor at which the characteristic waveform information appears in the waveform data based on the probability information; A program for executing The probability calculation step includes: a second extraction step of extracting a feature map indicating features of the waveform data from the waveform data; A generating step of generating an attention map indicating a gaze area from the feature map; an identification step of inputting information based on a product of the feature map and the attention map to identify the probability information; A program having:

Citation Information

Patent Citations

  • Waveform generation specification method and program

    JP2021069929A