Waveform display device, waveform display system, waveform display method, and program

Abstract

To visually display necessary information and present information that makes it easy for a user to notice.SOLUTION: A waveform display device includes: an incidental information drawing unit that draws incidental information of a waveform acquired by multiple sensors to a waveform displayed on a screen according to a predetermined time; a waveform extraction unit that extracts a characteristic waveform from waveform data; a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit; and a waveform classification unit that classifies the characteristic waveform extracted by the waveform extraction unit. The incidental information drawing unit displays the incidental information corresponding to the representative time of the waveform determined by the representative time determination unit according to a classification result by the waveform classification unit.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to a waveform display device, a waveform display system, a waveform display method, and a program. [Background technology]

[0002] Conventionally, magnetoencephalography (MEG) tests have been used to identify epileptogenesis by dipole analysis of the source of epilepsy in the brain from changes in the magnetic field near the brain and depicting the location of the source on an MRI.

[0003] In recent years, with the improvement of computer performance and the development of artificial intelligence, attempts have been made to reduce the burden on analysts by extracting sections with characteristic waveforms from long-term MEG waveform data or from multiple recorded MEG waveform data, performing cross-sectional dipole analysis, and detecting candidate epileptic spikes.

[0004] Patent Document 1 discloses a technology that determines the probability of occurrence of characteristic waveform information (IED: Interictal Epileptiform Discharge) and displays the probability of occurrence in color on the waveform, for the purpose of determining the time when the IED appears and extracting the sensor. Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the conventional technology, when examining the waveforms of epilepsy spike candidates and the results of dipole extraction, the reasons for positive results were not detailed, making it difficult to decide whether to accept or reject the results. In addition, with the conventional technology, the information presented to discover and correct false negatives that should have been extracted from the many negatives was insufficient, causing problems such as users overlooking them.

[0006] The present invention has been made in consideration of the above, and has an object to visually display necessary information and to present information that makes it easy for a user to notice. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the present invention comprises an attendant information drawing unit that draws attendant information of a waveform acquired by a plurality of sensors and displayed on a screen according to a predetermined time, a waveform extraction unit that extracts a characteristic waveform from the waveform data, a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit, and a waveform classification unit that classifies the characteristic waveform extracted by the waveform extraction unit, and is characterized in that the attendant information drawing unit displays attendant information corresponding to the representative time of the waveform determined by the representative time determination unit according to the classification result by the waveform classification unit. Effect of the Invention

[0008] According to the present invention, it is possible to visually display necessary information and to present information that is easily noticeable by the user. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of a biological signal measuring system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating a schematic functional configuration of the server. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 4] FIG. 4 is a diagram illustrating an example of a functional block configuration of the information processing device. [Diagram 5] FIG. 5 is a flowchart showing the flow of the dipole estimation process. [Figure 6] FIG. 6 is a diagram showing an example of a calculated probability map of an IED. [Figure 7] FIG. 7 is a diagram showing an example of a waveform display. [Figure 8] FIG. 8 is a diagram showing an example of the waveform display cursor. [Figure 9]FIG. 9 is a diagram showing an example of a superimposed display of waveform additional information. [Figure 10] FIG. 10 is a diagram showing an example of layers of waveform superimposition display. [Figure 11] FIG. 11 is a diagram showing another example of layers for waveform superimposition display. [Figure 12] FIG. 12 is a diagram showing an example of selection of a display layer. [Figure 13] FIG. 13 is a diagram showing another example of the superimposed display of the waveform additional information. [Figure 14] FIG. 14 is a diagram showing an example of a dipole extraction condition setting screen. [Figure 15] FIG. 15 is a diagram showing an example of a display for correcting the result. [Figure 16] FIG. 16 is a diagram showing an example of annotation display. [Figure 17] FIG. 17 is a diagram showing an example of a dipole list display. [Figure 18] FIG. 18 is a diagram showing a modified example of the annotation and dipole list. [Figure 19] FIG. 19 is a flow chart showing the flow of a series of operations. [Figure 20] FIG. 20 is a flowchart showing the flow of a learning process when generating an IED probability map using machine learning according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a waveform display device, a waveform display system, a waveform display method, and a program will be described in detail with reference to the accompanying drawings.

[0011] In addition, the present invention is not limited to the following embodiments, and the components in the following embodiments include those that a person skilled in the art can easily come up with, those that are substantially the same, and those that are within the scope of what is called equivalent. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the scope of the gist of the following embodiments.

[0012] This embodiment has the following features when calculating the probability map of an IED (Interictal Epileptiform Discharge). In short, the feature is that by using the probability map of an IED, it becomes possible to detect the onset time of an IED and select a sensor, which are necessary for the equivalent current dipole method.

[0013] When a doctor performs an analysis, it is ideal to analyze the origin of the IED using the equivalent current dipole method, but since the origin is known to have a low S / N ratio, the time of the IED is actually determined between the origin and the peak of the IED. In this embodiment, the analysis is not limited to the origin.

[0014] (First embodiment) (Outline of biosignal measurement system) 1 is a schematic diagram showing the configuration of a biosignal measurement system according to a first embodiment of the present invention, with reference to which an overview of the biosignal measurement system 1 according to the present embodiment will be described.

[0015] The biosignal measurement system 1 is a waveform display system that measures and displays multiple types of biosignals (e.g., magneto-encephalography (MEG) signals and electro-encephalography (EEG) signals) of a subject from a specific source (biological part). Note that in the present invention, the biosignals to be measured are not limited to magneto-encephalography (MEG) signals and electro-encephalography (EEG) signals, but may be, for example, electrical signals generated in response to cardiac activity (electrical signals that can be expressed as an electrocardiogram).

[0016] As shown in Fig. 1, the biosignal measurement system 1 includes a measurement device 3 that measures one or more biosignals of a subject, a server 40 that records one or more types of biosignals measured by the measurement device 3, and an information processing device 50 that is a waveform display device that analyzes and displays one or more types of biosignals recorded in the server 40. The measurement device 3 is, for example, a magnetoencephalograph that collects measurement values ​​such as brain magnetic fields and timing of application of stimuli. Note that, although the server 40 and the information processing device 50 are shown separately in Fig. 1, for example, at least a part of the functions of the server 40 may be incorporated into the information processing device 50.

[0017] In the example of FIG. 1, the subject (subject) lies on his / her back on the measurement table 4 with electrodes attached to his / her head as sensors for measuring brain waves, and places his / her head in a recess 32 of a Dewar 31 of the measurement device 3. The Dewar 31 is a holding container in an extremely low temperature environment using liquid helium, and a number of magnetic sensors for measuring magnetoencephalography are arranged inside the recess 32 of the Dewar 31. The measurement device 3 collects brain wave signals from the electrodes and magnetoencephalography signals from the magnetic sensors, and outputs data including the collected brain wave signals and magnetoencephalography signals (hereinafter, sometimes referred to as "measurement data") to a server 40. The measurement data output to the server 40 is read out and displayed by an information processing device 50, and is analyzed. In general, the Dewar 31 and the measurement table 4 incorporating the magnetic sensors are arranged in a magnetically shielded room, but for convenience, the magnetically shielded room is not shown in FIG. 1.

[0018] The information processing device 50 is a device that displays waveform data of MEG signals from multiple magnetic sensors and waveform data of EEG signals from multiple electrodes in synchronization on the same time axis. An EEG signal is a signal that represents the electrical activity of nerve cells (the flow of ionic charges that occurs in the dendrites of neurons during synaptic transmission) as a voltage value between electrodes. A MEG signal is a signal that represents minute electric field fluctuations caused by electrical activity in the brain. The brain magnetic field is detected by a highly sensitive superconducting quantum interference device (SQUID) sensor. These EEG signals and MEG signals are examples of "biological signals".

[0019] 2 is a block diagram showing a schematic functional configuration of the server 40. As shown in FIG. 2, the server 40 includes a data acquisition unit 41 and a data storage unit .

[0020] The data acquiring unit 41 periodically acquires measurement data from the measuring device 3. Here, the measurement data is individual waveform data measured by a plurality of magnetic sensors in the Dewar 31 of the measuring device 3.

[0021] The data storage unit 42 stores the measurement data acquired from the measurement device 3 .

[0022] (Hardware configuration of information processing device) 3 is a diagram showing an example of a hardware configuration of the information processing device 50. The hardware configuration of the information processing device 50 will be described with reference to FIG.

[0023] As shown in FIG. 3, the information processing device 50 has a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, an auxiliary storage device 104, a network I / F 105, an input device 106, and a display device 107, which are interconnected by a bus 108.

[0024] The CPU 101 is a computing device that controls the overall operation of the information processing device 50 and performs various information processing. The CPU 101 executes an information display program stored in the ROM 103 or the auxiliary storage device 104, and controls the display operation of the measurement collection screen and the analysis screen.

[0025] The RAM 102 is a volatile storage device used as a work area for the CPU 101 and stores main control parameters and information. The ROM 103 is a non-volatile storage device that stores basic input / output programs and the like. For example, the above-mentioned information display program may be stored in the ROM 103.

[0026] The auxiliary storage device 104 is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The auxiliary storage device 104 stores, for example, a control program that controls the operation of the information processing device 50, and various data and files necessary for the operation of the information processing device 50.

[0027] The network I / F 105 is a communication interface for communicating with devices on a network, such as the server 40. The network I / F 105 is realized by, for example, a network interface card (NIC) conforming to the Transmission Control Protocol (TCP) / Internet Protocol (IP).

[0028] The input device 106 is a user interface such as a touch panel input function, a keyboard, a mouse, and operation buttons. The display device 107 is a display device that displays various information. The display device 107 is realized by, for example, a touch panel display function, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The display device 107 displays a measurement collection screen and an analysis screen, and the screens are updated in response to input / output operations via the input device 106.

[0029] The hardware configuration of the information processing device 50 shown in Fig. 3 is an example, and other devices may be included. The information processing device 50 shown in Fig. 3 has a hardware configuration assuming, for example, a PC (Personal Computer), but is not limited thereto, and may be a mobile terminal such as a tablet. In this case, the network I / F 105 may be a communication interface having a wireless communication function.

[0030] (Functional block configuration of information processing device) 4 is a diagram showing an example of a functional block configuration of the information processing device 50. The functional block configuration of the information processing device 50 will be described with reference to FIG.

[0031] 4, the information processing device 50 includes a pre-processing unit 501, an IED probability map calculation unit 502, a threshold processing unit 503, a post-processing unit 504, and a dipole estimation unit 505. The information processing device 50 also includes a display control unit 506 and an auxiliary information drawing unit 507.

[0032] The preprocessing unit 501 performs preprocessing such as extraction and expansion of sensors, downsampling and application of frequency filters, removal of artifacts, poor channel processing, extraction of time windows, and standardization of magnetic field data.

[0033] In addition to using all sensors to calculate the IED probability map described below, sensor extraction and expansion can also be performed using only a certain number of sensors that have been grouped in advance. Sensor grouping can be based on anatomical criteria such as the temporal lobe or frontal lobe, or simply by grouping an arbitrary number of nearby sensors. In addition, if the number of sensors used in the calculation is less than the number of sensors used in the learning process described below, it is also possible to expand the number of fictitious sensors.

[0034] Downsampling is applied to match the sampling frequency used during training. The same frequency filter is also applied, and the same filtering applied during training is applied. Commonly used filters include a 35Hz low-pass filter and a 3Hz-35Hz band-pass filter.

[0035] For the purpose of removing artifacts due to heartbeats and artifacts due to blinking and body movements, ICA (see Non-Patent Document 1) and DSSP (see Non-Patent Document 2) are used.

[0036] Bad channel processing refers to the process of excluding sensors that observe magnetic field changes exceeding a preset threshold, or of interpolating using surrounding sensor values.

[0037] There are several ways to extract the time window, including no overlap where the time window is shifted by the length of the time window, overlapping half the length of the time window, overlapping 1 / 4 the length, etc. When overlapping, the overlapping amount is averaged when calculating the IED probability map described below.

[0038] Standardization of the magnetic field data is performed so that the average is 0 and the variance is 1 within the extracted time window. In addition to standardization, a method of normalizing the magnetic field within a preset range to be between -1 and 1 may be used.

[0039] The IED probability map calculation unit 502 functions as a waveform extraction unit that extracts a characteristic waveform (IED) from the waveform data. The IED probability map calculation unit 502 calculates a probability map of the characteristic waveform. In addition, the IED probability map calculation unit 502 compares each waveform data acquired by a plurality of sensors with at least one or more characteristic waveform information.

[0040] IED refers to waveforms such as Spike Wave, Spike and Wave, Poly Spike and Wave, and Sharp Wave, which are characteristic of epileptic lesions.

[0041] The threshold processing unit 503 uses a threshold to narrow down the time and sensor area with high IED probability from the IED probability map obtained by the IED probability map calculation unit 502. The threshold processing unit 503 functions as a waveform classification unit that classifies the characteristic waveforms extracted by the IED probability map calculation unit 502. The threshold processing unit 503 determines the classification of the characteristic waveform according to the time specified by the post-processing unit 504 and the appearance probability information corresponding to the relevant sensor.

[0042] The post-processing unit 504 executes post-processing to extract sampling time points and sensors to be used for dipole estimation from the IED probability map thresholded by the threshold processing unit 503. In addition, when the peak detection method or map has multiple peaks (such as when there are two lesions), the post-processing unit 504 performs post-processing to separate the map. The post-processing unit 504 functions as a representative time determination unit that determines the representative time of the waveform extracted by the IED probability map calculation unit 502. The post-processing unit 504 determines the appearance probability of characteristic waveform information in at least a certain section of the waveform data based on the correlation ratio between the peak section of the waveform data and the characteristic waveform information. Then, the post-processing unit 504 identifies the time when the section matching the characteristic waveform information appeared and the corresponding sensor based on the appearance probability.

[0043] The dipole estimation unit 505 performs dipole estimation (see Non-Patent Document 3) using the sensor obtained in the processing before the dipole estimation unit 505 and the sampling time point of the IED's origin.

[0044] Next, the flow of the dipole estimation process in dipole estimator 505 will be described.

[0045] 5 is a flowchart showing the flow of the dipole estimation process. Prior to the process, the measurement device 3 performs magnetoencephalography measurement and outputs each waveform data measured by the multiple magnetic sensors of the Dewar 31 to the server 40. The server 40 stores each waveform data measured by the multiple magnetic sensors of the Dewar 31 of the measurement device 3 in the data storage unit 42.

[0046] Then, as shown in FIG. 5, first, the preprocessing unit 501 acquires each piece of waveform data measured by the multiple magnetic sensors of the Dewar 31 of the measuring device 3 from the data storage unit 42 of the server 40 (step S1).

[0047] Next, the preprocessing unit 501 performs preprocessing on the acquired data, such as extracting and expanding sensors, downsampling and applying frequency filters, removing artifacts, processing bad channels, extracting time windows, and standardizing magnetic field data (step S2).

[0048] Next, the IED probability map calculation unit 502 calculates a probability map of the characteristic waveform (IED) (step S3).

[0049] Next, the threshold processing unit 503 performs threshold processing on the IED probability map obtained by the IED probability map calculation unit 502 (step S4). Specifically, the threshold processing unit 503 narrows down the time and sensor area with a high IED probability from the IED probability map obtained by the IED probability map calculation unit 502 using a predetermined threshold.

[0050] In the threshold processing unit 503, the higher the threshold is set, the more likely the waveforms are to be selected as those that are IED-like, but the number of IEDs for which dipole estimation is performed decreases. On the other hand, in the threshold processing unit 503, when the threshold is set low, the number of false positives increases, but more IEDs can be detected. Furthermore, since magnetoencephalography data is strongly influenced by individual differences, there is a possibility that IEDs will hardly be detected unless the threshold is lowered.

[0051] Therefore, the threshold processing unit 503 may be configured to interactively control the number of IEDs to be detected in cooperation with a UI (User Interface).

[0052] The threshold processing unit 503 basically uses a preset value such as "0.8" as the threshold.

[0053] Next, the post-processing unit 504 executes post-processing to extract sampling time points and sensors to be used for dipole estimation from the IED probability map thresholded by the threshold processing unit 503 (step S5).

[0054] Specifically, when the threshold processing unit 503 is applied, the time and the sensor area having a value equal to or greater than a predetermined probability are extracted. In order to clarify the peak detection or the detection time of the IED, a filter such as a Gaussian filter may be applied to the extracted sensor area before the peak detection. The time of the IED is determined by applying a filter such as a Gaussian filter to the extracted sensor area before the peak detection and then performing the peak detection. When performing the peak detection, the detection may be performed on a one-dimensional probability map obtained by averaging the probability map including the extracted sensor area in the sensor direction. If the averaging is not performed, the peak detection may be performed for each sensor, and the time of the IED may be determined by averaging the time points.

[0055] The threshold processing unit 503 extracts sensors having a value equal to or greater than a predetermined probability at the time of the detected peak as sensors to be used for dipole estimation. At this time, it is known that the stability of the dipole estimation solution decreases when the number of sensors used for dipole estimation is reduced. Therefore, it is possible to prevent dipole estimation from being performed at the IED when the number of selected sensors is less than a specified number.

[0056] As another method of extracting sensors, the stability of the dipole estimation solution can be improved by defining a group of sensors in advance and expanding the extracted sensor to all sensors in the group to which the sensor belongs. Here, the group is basically set based on anatomical criteria such as the vicinity of the temporal lobe or the vicinity of the frontal lobe, but a neighboring area containing any number of sensors may simply be set as one group.

[0057] Finally, the dipole estimation unit 505 performs dipole estimation (see Non-Patent Document 3) using the sensor obtained by the processing before the dipole estimation unit 505 and the sampling time point of the IED's origin (step S6). In this case, in addition to dipole estimation, it is possible to use a current source analysis method using a spatial filter method such as the minimum norm method (see Non-Patent Document 6) or the LCMV Beamformer method (see Non-Patent Document 7). The spatial filter method is a method in which tens of thousands of dipoles are placed in the brain in advance and the change in current over time in each dipole can be obtained.

[0058] Fig. 6 is a diagram showing an example of calculating the probability map of an IED. In the example shown in Fig. 6, the IED probability map calculation unit 502 uses template matching. The IED probability map calculation unit 502 detects an IED by using a template that generalizes the shape of an IED, which is created from a document (see Non-Patent Document 8) that investigates the morphological characteristics of IEDs in magnetoencephalograms.

[0059] As shown in Fig. 6(a), the IED probability map calculation unit 502 scans templates such as templates 601 to 603 for the time series of MEG data of one sensor in MEG data 604, and creates a two-dimensional IED probability map 605 based on the similarity as shown in Fig. 6(b). Although the map created here is not strictly a probability map, it can be used as a map for the purpose, and so will be referred to as a probability map for convenience.

[0060] Similarity can be calculated using one-dimensional methods of calculating similarity in template matching in images, such as normalized cross-correlation (NCC) or zero-mean normalized cross-correlation (ZNCC), or a simple normalized convolution with a filter. In addition to template matching, several methods can be applied to detect IEDs, such as a method that derives the IED by detecting a steep transition from the baseline.

[0061] In addition, if machine learning is not used in the IED probability map calculation unit 502, dipole estimation at the peak position may produce undesirable results, so post-processing is required, such as shifting the sampling time slightly earlier on the time axis than the peak position.

[0062] By using an automatic IED detection method that uses an algorithm in this way, it is possible to automate the detection of IEDs and the determination of their origin.

[0063] The display control unit 506 is configured to display the waveforms acquired by the multiple sensors side by side on the display screen.

[0064] The additional information drawing unit 507 draws the additional information of the waveform according to a predetermined time. The additional information drawing unit 507 displays the additional information corresponding to the representative time of the waveform determined by the post-processing unit 504 according to the classification result of the waveform by the threshold processing unit 503.

[0065] For example, the incidental information drawing unit 507 draws the number of sensors corresponding to the time determined as incidental information. Also, the incidental information drawing unit 507 highlights the number of sensors corresponding to the time determined as incidental information.

[0066] Next, an example of a waveform display method in the display control unit 506 will be described.

[0067] Fig. 7 is a diagram showing an example of a waveform display. In the waveform display example shown in Fig. 7, the display control unit 506 displays a label name 7001 of the measured sensor, a measured waveform 7002 corresponding to the label name 7001, and a number of sensors 7003 that have exceeded a predetermined probability threshold in the IED probability map. Note that the number of sensors 7003 that have exceeded the predetermined probability threshold refers to the number of sensors at the time of estimation if there is a dipole estimation result.

[0068] The number of sensors 7003a is an example of annotation display when the detection result is positive. The dashed line display shows an example where this annotation highlights the possibility of a false positive. Note that the display is not limited to the dashed line display, and other means may be used as long as they can distinguish and highlight false positives and positives. For example, it may be possible to change the display color of the numerical value, or to set an exclamation mark "!" above the numerical value. Also, in the case of a display example such as that shown in FIG. 15 described later, if Annotation 1505 (see FIG. 15) is set, a specific name may be displayed as well. These may also be combined. Note that examples of criteria for determining the possibility of a false positive will be described later.

[0069] The number of sensors 7003b is an example of annotation display when the detection result is negative. For example, when the IED probability at the indicated time includes at least one sensor that is greater than 0, it indicates that the number of sensors that exceed the threshold after threshold processing of the IED probability is 0. Note that in the case of a negative result such as the number of sensors 7003b, which does not correspond to the number of sensors 7003c described later, the number display may be omitted to display in a simple format. In such a case, since there is little need for the user to check in detail, it is better to hide it.

[0070] The sensor number 7003c is an example of an annotation display for a case where the detection result is negative and the case is detected as a false negative candidate. As with the false positive, this is an example of a display method for highlighting an annotation that is highly likely to be a false negative. In the example shown in FIG. 7, an example is shown where an annotation is highlighted by displaying a dashed dotted line to highlight an annotation that is highly likely to be a false negative. The highlighting method may be the same as the sensor number 7003a, but a different display format may be used to distinguish it from the sensor number 7003a. The criteria for determining the possibility of a false negative will be described later.

[0071] The number of sensors 7003d is a display example when the detection result is negative. Here, an example is shown in which the number of sensors exceeding the threshold value of the probability value of the IED probability map is less than the predetermined number, resulting in a negative determination. In this example, too, if the determination corresponds to a false negative, the display is highlighted in the same way as the number of sensors 7003c. An example of this is, for example, a case in which the IED probability is high, and the number of selected sensors does not reach the predetermined number, resulting in a negative determination.

[0072] 7, the waveform display example shows positive or negative time 7004. Negative only includes cases where the final result was determined to be negative for cases where the probability value on the IED probability map is greater than 0, and cases where the probability value on the IED probability map is all 0 are not displayed.

[0073] Time 7004a is a display showing a representative positive time. In FIG. 7, a solid arrow shows a representative positive time, which can be distinguished from a dotted negative arrow, which will be described later. Note that, in addition to solid and dotted lines, a display method that can distinguish between negative and positive, such as changing the color or shape of the arrow, may be used. This case shows that a positive judgment was made after dipole estimation was performed. Dipole estimation time 7007a is shown by a dashed line.

[0074] Time 7004b is a display showing a negative representative time. In Fig. 7, the negative representative time is indicated by a dotted arrow.

[0075] Time 7004c is a display showing a representative negative time. In Fig. 7, the representative negative time is shown with a dotted arrow. This case shows that after dipole estimation was performed, a negative determination was made. Dipole estimation time 7007b is shown with a dashed line.

[0076] Time 7004d is a display showing a negative representative time. In Fig. 7, the negative representative time is indicated by a dotted arrow. Note that since no dipole is estimated, the dashed line of the dipole estimation time 7007 is not displayed.

[0077] In the waveform display example shown in FIG. 7, the display control unit 506 displays an IED probability map 7005 in the background of the waveform 7002. The IED probability map 7005 is an IED probability map corresponding to each area of ​​the display sensors arranged at equal intervals across the entire waveform drawing area. In the waveform display example shown in FIG. 7, the inside of a rectangle including the lines on which the vertical and horizontal axes are drawn is the drawing area, and the vertical direction is an example in which the area is divided into 10 areas, the number of sensor selections. Here, only the IED probability map is displayed, but it is possible to simultaneously draw search information for the dipole estimation time after the IED probability map, as in the example of FIG. 9 described later.

[0078] In the waveform display example shown in FIG. 7, the display control unit 506 displays a grayscale map 7006 that indicates the meaning of the grayscale of the IED probability map. In the example of the grayscale map 7006, the IED probability threshold (Threshold) is expressed as a decimal value from 0 to 1, which corresponds to a probability of 0 to 100%. The threshold value in the IED probability after threshold processing corresponds to the threshold value (0.5 (50%)). The dotted line in the center of the grayscale map 7006 indicates the corresponding position on the grayscale map of the threshold.

[0079] In the waveform display example shown in Fig. 7, the display control unit 506 displays dipole estimation times 7007 (7007a, 7007b) that estimate the dipole of the detected characteristic waveform. In the waveform display example shown in Fig. 7, the dipole estimation times 7007 (7007a, 7007b) are displayed as dashed lines. Note that the dipole estimation times 7007 (7007a, 7007b) may be displayed in different line formats depending on the classification type (positive, negative, false negative, false positive) to prevent erroneous recognition, or the format of the line, such as the color, may be changed depending on the group defined by the user, for example, the type of annotation of Annotation 1505 (see Fig. 15).

[0080] In the waveform display example shown in Figure 7, the display control unit 506 displays a display unit and scale 7008 of the vertical axis indicating the measurement value, a representative scale 7009 of the horizontal axis indicating the measurement time, and a waveform information display section 7010 indicating the sampling number, time, number of displayed waveforms, etc. at the focus point on the waveform.

[0081] Next, a display example of the waveform display cursor in the display control unit 506 will be described.

[0082] Fig. 8 is a diagram showing an example of a waveform display cursor. The waveform display cursor 8001 shown in Fig. 8 is a cursor that indicates the focus time of the waveform. The waveform display cursor 8001 can be displayed transparently as a solid line so that it can be simultaneously displayed even if it overlaps with the dipole estimated time 7007. The waveform display cursor 8001 can be operated at any time by input from the keyboard or mouse, etc.

[0083] 8 is also a cursor indicating the focus time of the waveform. The waveform display cursor 8002 is placed to maintain visibility when the information overlaps with the dipole estimated time 7007. The waveform display cursor 8002 may be omitted depending on the display format.

[0084] As shown in FIG. 8, the display control unit 506 displays the times corresponding to the positions of the waveform display cursors 8001 and 8002 in the waveform information display unit 7010.

[0085] Next, an example of waveform additional information, which is additional information of a waveform that the additional information drawing unit 507 draws at a predetermined time, will be described.

[0086] Fig. 9 is a diagram showing an example of a superimposed display of waveform additional information. Waveform additional information 9001 shown in Fig. 9 is an example of a display range of an IED probability map. The waveform additional information 9001 indicates a time range showing a characteristic waveform.

[0087] Moreover, waveform additional information 9002 is an example of an area display of a time width searched for as a candidate for the dipole estimated time within the display range of the IED probability map.

[0088] For the time of dipole estimation, the time and the sensor are selected from the IED probability map. In dipole estimation, a dipole that meets a certain criterion using a statistical index (e.g., Good of Fitness, Confidence Volume, Intensity, etc.) is adopted as a dipole, and if it does not meet the criteria, it is not adopted. In other words, there is a possibility of a negative judgment. On the other hand, a method is also implemented in which a dipole is estimated cross-sectionally for a certain time before and after the relevant time, and a dipole with a better statistical evaluation result is adopted (positive judgment), and if no dipole is found as a result of the cross-sectional search, it is not adopted (negative judgment). The time group in which the dipole is estimated cross-sectionally at this time is set as the search range of the dipole estimation time. The search range of the dipole estimation time is defined as the range detected from the IED probability map range, and has the characteristic of being defined inside.

[0089] Basically, the waveform additional information 9001 and the waveform additional information 9002 can be displayed simultaneously in an overlapping manner.

[0090] Next, an example of layers for waveform superimposition display in the display control unit 506 will be described.

[0091] Fig. 10 is a diagram showing an example of layers of waveform superimposition display. The example shown in Fig. 10 is a configuration example of the superimposition display of waveform additional information shown in Fig. 9. Since the search range of the dipole estimation time is defined inside the IED probability map range, it can be expressed by a display layer structure as shown in Fig. 10.

[0092] In the example shown in FIG. 10, the display control unit 506 displays the IED probability map drawing layer and the time search range layer in this order. Furthermore, in the example shown in FIG. 11 described later, the display control unit 506 displays the waveform, time line, and cursor shown in FIG. 9 in this order. However, the waveform, time line, and cursor shown in FIG. 9 are configured in a two-dimensional arrangement as the actual drawing, so they are easy to distinguish. For this reason, they do not necessarily have to be displayed in this order. Also, if the cursor layer is not necessary as a display form, it may be omitted.

[0093] FIG. 11 is a diagram showing another example of a layer of a waveform superimposition display. The example shown in FIG. 11 is another form of the configuration example of the superimposition display of the waveform additional information shown in FIG. 9. In the example shown in FIG. 11, the area images of the IED probability map drawing layer and the time search range layer are replaced with the color of the waveform. In the example shown in FIG. 11, for example, the basic display color of the waveform is set to black, and the color of the time range of the IED probability value>0 is set as a waveform color defined as a blue to red gradation according to the actual IED probability value. Furthermore, in the example shown in FIG. 11, the search range of the dipole estimation time may be redefined and displayed in a color that does not belong to any of the above (see reference numeral 1303 in FIG. 13 described later).

[0094] In addition, by making the color of the section of the waveform additional information 9002 shown in FIG. 9 the complementary color of the IED probability value, the time search range can be expressed without losing the probability value information.

[0095] Next, an example of selection of a display layer in the display control unit 506 will be described.

[0096] Fig. 12 is a diagram showing an example of selection of a display layer. The example shown in Fig. 12 is an example in which the user can select the layer he / she wants to display as necessary. When the check box CB shown in Fig. 12 is enabled, the data targeted by the enabled check box CB is displayed, and unnecessary data can be hidden.

[0097] In the example shown in Fig. 10, the visibility of each layer corresponds to the checkbox CB. On the other hand, in the example shown in Fig. 11, the search range area of ​​the IED probability map and dipole estimation time is expressed as the color of the waveform, but in this case, it is possible to express it as if it were not displayed by returning the color of the waveform to the basic color (black in this example).

[0098] The check boxes CB for positive, negative, and false negative candidates shown in Fig. 12 are an example of displaying each determination result. Note that, since false positive candidates are included in the positive display, an independent check box CB is not provided in this embodiment, but a check box CB for false positive candidates may be provided.

[0099] Next, another example of the waveform additional information, which is the additional information of the waveform that the additional information drawing unit 507 draws according to a predetermined time, will be described.

[0100] Fig. 13 is a diagram showing another example of the superimposed display of waveform additional information. As with that described in Fig. 9, waveform additional information 9001 shown in Fig. 13 is an example of the display range of an IED probability map. The waveform additional information 9001 indicates a time range showing a characteristic waveform. Waveform additional information 9002 is an example of an area display of a time width searched for as a candidate for the dipole estimated time within the display range of the IED probability map.

[0101] Reference numeral 1301 in Fig. 13 indicates an example of a normal waveform display. A dipole estimated time region is defined by adding vertical lines to the boundaries of the upper and lower time limits of the waveform additional information 9002, and the range of the time estimation region can be known by defining the color of the waveform according to the probability value of the IED probability map of the waveform additional information 9001.

[0102] Note that the example shown in Figure 13 has the advantage of good visibility because only the color of the IED probability map can be drawn, but if there are many selected sensors, it may be difficult to distinguish between the vertical line boundaries and the vertical lines indicating the estimated dipole times and the waveforms, making it difficult to interpret the information.

[0103] Reference numeral 1302 in Fig. 13 indicates a display example in which the section of the IED probability map where a dipole estimation search has been performed is displayed as a solid line, and the time range in the search section where the IED probability value is present is displayed as a dotted line. Note that the dotted line may be combined with a method of displaying the area where no dots are drawn at a predetermined transparency so that the waveform line is not completely interrupted, or with the interval between dots, a dotted line, a dashed line, a one-dot chain line, a multiple-dot chain line, etc. In this way, changing the method of displaying the area where there are no dots is also included in this representation method.

[0104] In the example shown in Figure 13, in either case of Figures 10 and 11, there is no need to distinguish the search range of the dipole estimation time and the IED probability map by color, so the color of the waveform in the relevant area can be defined according to the IED probability value.

[0105] 13 indicates a display example in which the IED probability map layer and the search range of the dipole estimated time are displayed in different colors or shading. According to this display example, although there is a disadvantage that the IED probability map value is lost in the range of the dipole estimated time, it is possible to improve visibility.

[0106] In addition, to hide it strictly, it would be more work to switch the layer display, but in practice, the value of the IED probability map displayed at the adjacent time is an approximate value, and after finding the waveform, the focus shifts to the problem of selecting the time at which to estimate the dipole. In this respect, if the search range for the estimated time is clearly indicated, the user can use it practically without any inconvenience in making the judgment. In addition, the method of waveform additional information 9002 also has a method that can achieve both visualization and loss of the numerical difference in the probability map value.

[0107] Next, a method for determining whether or not a false negative is suspected in the threshold processing unit 503 will be described.

[0108] The threshold processing unit 503 extracts sensors that have a value equal to or greater than a predetermined probability at the time of the detected peak as sensors to be used for dipole estimation. At this time, it is known that the stability of the dipole estimation solution decreases when the number of sensors used for dipole estimation is reduced. Therefore, it is also possible to prevent dipole estimation from being performed at the IED when the number of selected sensors is less than a specified number. Here, it is assumed that dipole estimation is not performed when the number of sensors is less than the specified number, as underlined.

[0109] If there are more than a certain number of sensors with a value equal to or greater than the predetermined probability obtained by the threshold processing unit 503, they are extracted, but if there are fewer than the specified number, dipole estimation is not performed and a negative judgment is made. Even though the IED probability is high near the boundary value of this specified number, there is a risk of a false negative when the specified number is not met and a negative judgment is made. For this reason, for cases where the specified number is not met, a specified criterion can be set and a false negative (candidate) annotation label can be added to that time to make it clear that it is to be displayed.

[0110] For example, if the number of sensors is equal to or greater than X% (e.g., 90%) of a given number and the IED probability map is equal to or greater than Z% (e.g., 90%) for Y% (e.g., 75%) or more of the sensors, a false negative (candidate) annotation is added. In this way, it becomes possible to display on the waveform screen those that have a risk of false negatives, and the user can determine whether or not to accept them after understanding the risk.

[0111] As another embodiment, a part of the conditions may be set, such as simply adopting only X% or more of a predetermined number.

[0112] As another method of determining whether a false negative is suspected in the threshold processing unit 503, when the number of sensors selected based on the threshold value of the IED probability map is expanded, a false negative (candidate) annotation may be added if a predetermined number of sensors is exceeded.

[0113] For example, when the sensors selected in the IED probability map are viewed in a three-dimensional space, spatially isolated points may occur in actual operation. If such isolated points are considered to be continuous areas, dipole estimation may be possible if the number of sensors exceeds a specified number, and they may become positive, so they can be extracted as false negatives (candidates). As mentioned above, the method of setting thresholds for Y and Z may be combined.

[0114] Next, a method for determining whether or not a false positive is suspected in the threshold processing unit 503 will be described.

[0115] The threshold processing unit 503 performs a judgment opposite to the judgment example of a false negative as an example of a judgment of a suspected false positive. When the distribution of the IED probability value of the selected sensor is concentrated near the threshold, the threshold processing unit 503 may extract it as a suspected false positive and annotate it. For example, a method of statistically evaluating the IED probability distribution may be applied, such as calculating an interquartile range using the threshold value and upper limit value of the IED probability value adopted by the threshold processing unit 503, and determining that a case in which the distribution of the IED probability value is entirely or at least a predetermined percentage below the first quartile is a suspected false positive.

[0116] As another example of a method for determining whether a result is a false positive, there is a method for determining whether a result is a false positive by statistical evaluation of the dipole estimation result.

[0117] Here, an example of a setting screen for dipole extraction conditions in the threshold processing unit 503 will be described.

[0118] FIG. 14 is a diagram showing an example of a dipole extraction condition setting screen. For example, as shown in FIG. 14, the criteria of the goodness of fit (GoF), moment (Intensity), and confidence volume (CV) are set as the false positive judgment threshold for the dipole estimation results registered as positive judgments, and if applicable, a false positive annotation is added. It is possible to set a similar value in the statistical evaluation of the dipole estimation results in the waveform additional information 9002, but here it can be set separately from the statistical evaluation. If there is no difference, no false positive occurs, and if it does occur, it can be classified as a false positive. It can also be applied to cases where all or part of the statistical evaluation of the dipole estimation is not performed.

[0119] The method of determining whether a false positive is suspected as described above may combine conditions, or may have a mechanism that allows the user to select a threshold value or applicable conditions as shown in FIG.

[0120] Furthermore, as another example of determining whether a false positive is suspected, if sensors with a high IED probability are distributed separately around the left and right temporal lobes (for example, a total of 60 sensors are selected, a=30 on the left side and b=30 on the right side, and condition parameters such as D=0.6>a÷b>e=0.4 are set), a typical pattern that shows a measurement waveform originating from the heart can be registered as a false positive pattern even if the number of sensors exceeds a specified value, and if that condition is met, it can be extracted as a false positive candidate.

[0121] Next, a practical example of display by the display control unit 506 will be shown.

[0122] Here, Fig. 15 is a diagram showing an example of a display for correcting the result. The display shown in Fig. 15 shows a practical example of the display including Figs.

[0123] Contour Map 1501 shown in Fig. 15 is a two-dimensional bird's-eye view projection of sensors that exist in a three-dimensional space, and displays the magnetic field values ​​as contour shading based on the magnetic field values ​​of each sensor. The (shading) steps at the bottom of Contour Map 1501 define colors (shading) that correspond to the source and sink of the magnetic field.

[0124] The white circles in Contour Map 1501 indicate sensors, and the black circles indicate displayed waveforms. The labels given to the ● in Contour Map 1501 are the corresponding sensor labels.

[0125] A sensor can be selected on the Contour Map 1501 by dragging the mouse, which is the input device 106, on the Contour Map 1501. Also, a sensor can be added to the Contour Map 1501 by dragging the mouse while pressing a specific key (shift key) on the keyboard, which is the input device 106, and a sensor can be deselected by dragging the mouse while pressing the control key.

[0126] In the above description, the spatial distribution of the sensor and the distribution based on the magnetic field value are displayed as an example, but the present invention is not limited to this. For example, when applied to an electroencephalograph or other waveform display system, the same function can be realized by displaying a different display means for visualizing a small waveform in a predetermined range before and after the cursor time for each sensor in a format corresponding to the corresponding sensor, instead of displaying the contour line in shading. For example, a method of drawing the small waveform itself in a predetermined rectangular range of the corresponding sensor. As another embodiment, a method of expressing a discrete representative value as a rectangular color of the sensor value instead of a contour line may be used.

[0127] Sensor Select 1502 shown in Fig. 15 is a button that allows you to select or cancel a specific set of sensors at once. Pressing the "AI Channel Select" button in Sensor Select 1502 selects the sensor corresponding to the annotation that is in focus (the arrow line above the waveform) or the annotation that is closest to the currently pointed cursor line. In other words, the "AI Channel Select" button has the function of selecting the sensor that was selected during dipole estimation at the target time.

[0128] "Select all" in Sensor Select 1502 selects all sensors, and "Deselect" in Sensor Select 1502 changes all sensors to an unselected state.

[0129] "LT (Left Temporal), LF (Left Frontal), LP (Left Parietal), LO (Left Occipital)" in Sensor Select 1502 refers to selecting all sensors adjacent to each brain area on the left. Also, "RT, RF, RP, RO" in Sensor Select 1502 selects the sensor groups corresponding to each area on the right. "ZT, ZC, ZP, ZO" in Sensor Select 1502 selects the sensor group located in the center that separates them. "L ALL" in Sensor Select 1502 selects all (LT, LF, LP, LO) simultaneously, and "R ALL" selects all (RT, RF, RP, RO) simultaneously.

[0130] A Display Mode 1503 shown in FIG. 15 is implemented with the check box S for selecting the display layer shown in FIG.

[0131] Move Result 1504 shown in FIG. 15 moves the cursor to the time position where the arrow annotation is added. At this time, the waveform related to the corresponding annotation is selected as the sensor selection. However, it is not necessary to implement the selection in a manner that forces the selection, and an embodiment in which only the cursor is moved while maintaining the previous selection state is also acceptable. In that case, for example, an operation in which the "AI Channel Selection" button is pressed after moving the cursor can be considered, in which the sensor of interest of the corresponding annotation is selected. The sensor of interest is the sensor used in the calculation when dipole estimation is performed.

[0132] The "Next" button in Move Result 1504 moves the cursor to the next annotation. The "Prev" button in Move Result 1504 has the function of moving the cursor to the previous annotation. If there is no next annotation, the move function may be disabled, or a message dialog may be displayed to notify the user of this.

[0133] As another embodiment, separate navigation buttons may be provided to focus on specific annotations such as positive, false negative, and false positive. By implementing in this manner, the user can make transitions.

[0134] Annotation 1505 in Fig. 15 can attach a label to the time position pointed to by the cursor. By attaching the corresponding annotation to the annotation time of a false positive, the system can reflect the user's opinion on the positive result presented by the system and provide correction feedback information.

[0135] Define1 (Negative) in Annotation 1505 is an example of setting something defined by the user, and the user can arbitrarily set a name other than the specific annotation labels listed here.

[0136] Dipole Estimation 1506 shown in Fig. 15 has a function for performing dipole estimation. When the "Analysis" button in Dipole Estimation 1506 is pressed, a dipole is estimated using the time indicated by the cursor and the waveform data of the currently selected sensor. The estimation results are drawn as objects consisting of spheres and lines on the Axial section 1507, Coronal 1508, and Sagittal 1509 planes on the MRI image with their coordinates and orientation, and numerical statistical information is displayed in Numerical Information 1510 of the Dipole Estimation Results.

[0137] When the "Modify" button in Dipole Estimation 1506 is pressed, if a dipole analysis result for the corresponding annotation is registered, it is replaced with the estimation result in user analysis in Numerical Information of Dipole Estimation Results 1510. The effective range of the Modify function can be implemented in such a way that the Modify button processing is enabled only when the cursor is included in a continuous region corresponding to the annotation of interest (a continuous time region in which the probability map of each selected sensor used in dipole estimation contains values ​​greater than 0 is calculated, and the widest time range among these is regarded as the continuous region) and dipole estimation analysis is performed within the continuous region.

[0138] "Add" in Dipole Estimation 1506 allows you to add as an annotation the dipole estimation result obtained by executing "Analysis" at the time indicated by the specified cursor position. If an annotation already exists, it will be registered as the dipole estimation result in the corresponding annotation, and if there is no annotation, it will be registered as a new dipole estimation result. In either case, the dipole that the user "Adds" can be used as a positive annotation thereafter.

[0139] "Delete" in Dipole Estimation 1506 deletes the dipole estimation result. If a positive annotation has been added, it is corrected to a negative annotation. Alternatively, the user may re-annotate the annotation.

[0140] Reference numeral 1507 denotes an axial section on an MRI image. The result of the numerical information 1510 on the dipole estimation result and the dipole estimation corresponding to the user analysis can be simultaneously displayed on the MRI in a distinguishable manner. For example, the color of the dipole drawing can be changed.

[0141] Reference numeral 1508 denotes a coronal section on the MRI image, and reference numeral 1509 denotes a sagittal section on the MRI image.

[0142] Reference numeral 1510 denotes numerical information on the dipole estimation result. Time denotes the time when the dipole is estimated, GoF denotes the degree of fit, CV denotes the confidence volume, X, Y, and Z denote the estimated dipole coordinates on the MRI, Zdir and Xdir denote the estimated dipole direction on the MRI, Q denotes the moment (intensity), and result denotes the corresponding annotation result. Note that Zdir denotes the angle θ between the estimated dipole direction on the MRI and the Z axis, and Xdir denotes the angle φ between the estimated dipole direction on the MRI and the X axis.

[0143] Moreover, the result in the reference numeral 1510 indicates the result of the corresponding annotation. In the user analysis in the reference numeral 1510, the dipole estimation result when the “Analysis” button in the Dipole Estimation 1506 is pressed is displayed.

[0144] Next, a specific example of correction in the auxiliary information drawing unit 507 will be described.

[0145] The waveform in Figure 15 has two annotations: a false negative candidate annotation with a dotted arrow and an exclamation mark, and a positive annotation with an arrow.

[0146] The condition for determining a false negative is that the number of selected sensors is 90% or more of a specified number (50). In other words, the condition is that 45 or more but less than 50 sensors have an IED probability value that is equal to or greater than the threshold. In this example, the number of sensors with IED probability values ​​that are equal to or greater than the threshold is 47, so they are extracted as false negative candidates.

[0147] The cursor is set to the time of the positive annotation. The display mode is as shown, with IED probability, waveform, cursor, spike time (estimated dipole time), and time search range enabled, and annotations enabled for positive and false negative candidates.

[0148] (Confirmation and correction of positive annotations) The following describes how to confirm and correct positive annotations.

[0149] The user judges whether the annotations, such as the measurement values, selected sensors, estimated dipole coordinates, estimated time, etc. visualized in the Contour Map 1501, are valid. If they are not valid, they are corrected.

[0150] The user moves the cursor line within the range where the IED probability value is greater than 0 and selects an appropriate time while checking the Contour Map 1501.

[0151] The estimated time range indicates the range searched by the system, so the user can decide whether to correct it within that range or to deviate from that range. If the user wants to select a more preferable sensor, the user reselects the corresponding sensor on the Contour Map 1501. Then, the user executes the Dipole Estimation 1506 analysis with the combination of the specified cursor time and the reselected sensor.

[0152] The results of executing the Dipole Estimation 1506 analysis are displayed as dipole estimation results in user analysis 1510, the axial section 1507 on the MRI image, the coronal section 1508 on the MRI image, and the sagittal section 1509 on the MRI image.

[0153] If a more favorable result than Result needs to be modified, the user can execute "Modify" in Dipole Estimation 1506, which overwrites the dipole estimation time and selected sensor as a modified annotation. In other words, although the result is positive, the annotation time and selected sensor are deemed inappropriate and the content is modified.

[0154] Moreover, if the user judges that this annotation itself is negative, the user can specify an appropriate negative annotation from the annotation list of Annotation 1505, which will be overwritten as a corrected annotation. In this way, the user's judgment can be reflected in the final result, and it can also be used as feedback information for improving the accuracy of the probability map.

[0155] (Check and correct false negative annotations) The following describes how to check and correct false negative annotations.

[0156] When the user selects Prev in Move Result 1504 from the state of Fig. 15, the cursor moves to the time indicated by the arrow of the false negative annotation. Note that the time indicated by the arrow in the annotation in this embodiment, which does not have the dipole estimated time registered, is the time determined during threshold processing.

[0157] The selected time 47 is automatically selected, or the user may execute "AI channel selection" in Sensor Select 1502 to reselect it.

[0158] For false negative candidate annotations, if the user judges them to be negative as set, no action needs to be taken, but more appropriate feedback can be obtained by selecting a more appropriate Negative annotation in Annotation 1505. If the user judges them to be positive, as with confirmation of a "Positive annotation," the user selects the sensor and time, performs dipole estimation in Dipole Estimation 1506, and once an appropriate dipole estimation result is obtained, executes Add in Dipole Estimation 1506 and registers the new dipole estimation result. At the time of registration, the false negative annotation is corrected to a positive annotation. The user may also register with a Positive annotation in Annotation 1505.

[0159] Thereafter, the user executes Next in Move Result 1504 and sequentially corrects the annotations, thereby completing the result confirmation by the user.

[0160] As another embodiment, a display system that displays summarized annotations and dipoles showing positive results in a list as shown in Figures 16 and 17 below can be laid out as part of the screen configuration, or implemented as a UI that operates in conjunction with another window. These list displays can also be aggregated and linked to form a single table. In that case, the dipole information can be left blank in areas where there are no dipole estimation results.

[0161] Also, by displaying only a specific classification list in conjunction with the selected state of Display Mode 1503, a system may be created that allows users to more efficiently reach the desired annotation and check and correct it in a concentrated manner.

[0162] Next, a display example of the annotation in the display control unit 506 will be described.

[0163] Here, Fig. 16 is a diagram showing an example of annotation display. The example shown in Fig. 16(a) is a display example after User A who measured magnetoencephalography data added the annotation "Artifact" to the analysis data specified by the user at 180 seconds of recording, and then performed the analysis flow of Fig. 5 on the recorded data.

[0164] The example shown in FIG. 16(b) is a display example in which user B subsequently corrects the result of FIG. 16(a) after viewing it.

[0165] - Explanation of each cell item The annotation can be set independently to indicate whether it is positive, a false negative candidate, or a negative, and the annotation name defined in Annotation 1505. It is assumed that positives are epileptic abnormal waves, false negative candidates are candidates extracted by the system, and negatives are noise from metals, heartbeats, and other biological noise that are not directly epileptic abnormal waves.

[0166] Annotations can also be changed by users. The annotator is set to the user who actually added the annotation, but the user of the annotation added by the analysis flow in Figure 5 is displayed as automatic analysis. It is possible to set it to suit the system, such as setting the system user name (for example, the user name logged into the system). The time refers to the time the annotation was made.

[0167] Explanation of changes in Figure 16(b) 121.384sec: Example of user B correcting a false negative candidate to positive, Spike 148.016sec: Example of User B correcting a false negative candidate to negative, Define1 243.785sec: Example of user B further classifying the Spike annotation into Poly Spike based on the automatic analysis results.

[0168] In this embodiment, the automatic analysis only assigns the Spike annotation to all positive results, and the doctor can then further subdivide this, so while this is a correct broad classification, the doctor can make more detailed classifications as necessary.

[0169] In addition, if a doctor performs additional dipole analysis for a time that was not analyzed in the analysis flow of Figure 5, it can be added to Figure 16(a). The time and annotator should be specified by the user, and the annotation should be set to a predetermined initial value (for example, annotation should be set to positive, spike, annotation should be set to unspecified, etc.).

[0170] By recording the edit results in this way, not only can another user verify the results, but the analysis program can also learn from them.

[0171] Additionally, annotation details and the corresponding display time are displayed. For example, when the user clicks to select a specified cell, the time transitions to the target annotation time in Figure 15 included in the row of the selected cell. It is also possible for the user to directly select and edit an annotation by selecting the annotation cell. When an annotation is modified, this display and the waveform display in Figure 17 are synchronized and the results are modified.

[0172] In annotation display, such as the annotation "(Confirmation Required)," a specific identification result can be highlighted to present it to the user in an easy-to-understand manner.

[0173] FIG. 16(a) shows the display that is displayed after the analysis flow of FIG. 5 is performed on the analysis data designated by the user.

[0174] Next, a dipole list display example in the display control unit 506 will be described.

[0175] Fig. 17 is a diagram showing an example of a dipole list display. The example shown in Fig. 17 is an example of displaying dipole estimation results of positive annotations.

[0176] For example, by selecting a cell and then selecting a dipole corresponding to the row of the cell, it is possible to transition to the target annotation time in FIG.

[0177] If the dipole is corrected, the display will be updated accordingly. The "!" is an example of a false positive candidate being highlighted among the positives. By configuring the dipole list display to highlight certain identification results like this, it is possible to present the results in an easy-to-understand way for the user.

[0178] This system allows users to instantly jump to the point of interest, check the results, and add or modify dipole estimation results or annotation labels as necessary. In addition, the modified result data can be saved and fed back as information to improve the performance of the system that outputs the probability map as systematic data.

[0179] The examples shown in Fig. 17 correspond to Fig. 16(a) and Fig. 16(b), respectively. Fig. 17(a) corresponds to Fig. 16(a), and Fig. 17(b) corresponds to Fig. 16(b). 121.384sec: An example in which User B corrects a false negative candidate to positive, Spike. The "!" is changed to "P" (Positive). 148.016sec: User B corrects a false negative candidate to negative, Define1. The "!" is changed to "N" (Positive). 243.785sec: An example in which User B further classifies the Spike annotation into Poly Spike in the automated analysis results. In this example, there was no change in positive or negative, only a refinement, so nothing was displayed.

[0180] In the above example, false negative candidates are highlighted with "!", while P and N are displayed to indicate that the user has edited them. However, other formats are also possible.

[0181] Here, a modified example of the annotation and dipole list in the display control unit 506 will be described.

[0182] Fig. 18 is a diagram showing a modified example of the annotation and dipole list. The modified example shown in Fig. 18 is an example in which the annotation and the dipole list are integrated into one.

[0183] As shown in Fig. 18(a), a separate column for automatic analysis is prepared to display positive (P), false negative candidate (FN?), and no automatic analysis performed (-). Annotations are prepared using "judgment (correction)" and "label" to reflect the user's judgment results.

[0184] By using this format, users can see whether an analysis was performed automatically and whether an annotator made corrections or additions. Furthermore, as shown in Figure 18(b), a checkbox "Reflect annotator's results" is provided, and when checked, it is possible to switch between positive and negative display with the annotator's corrections reflected in the results of the automatic analysis. This allows users to consider the results by successively switching between the results provided by the AI ​​and the results provided by the annotator.

[0185] Next, a series of work flows will be described.

[0186] Fig. 19 is a flowchart showing a series of operations. As shown in Fig. 19, first, the display device of the information processing device 50 is provided with a mechanism for performing login authentication according to the user of the device, and authentication is required before starting and using the biosignal measurement system 1, etc. (step S101). As a result, when the user logs in when starting up the biosignal measurement system 1, the user can access the biosignal measurement system 1.

[0187] Next, the biosignal is measured by the biosignal measurement system 1 (step S102). In this embodiment, magnetoencephalography data is recorded as the biosignal.

[0188] Next, the information processing device 50 displays the waveform data of the MEG signals from the multiple magnetic sensors and the waveform data of the EEG signals from the multiple electrodes in synchronization with each other on the same time axis. This allows the user to check the recorded data. In addition to this, the information processing device 50 can also add annotations to designated times on the time axis, such as Contour Map 1501 (step S103). For example, at 243.4545 sec in FIG. 16, user A can add "Artifact" to the disturbance noise waveform.

[0189] In step S103, it is assumed that the user visually checks the waveform and adds annotations in a situation where the waveform display is updated in real time during waveform recording (online display), but this mechanism may be omitted.

[0190] Next, the information processing device 50 performs pre-analysis processing (step S104). As shown in JP 2006-167350 A (Patent Document 2), the pre-analysis processing is performed by performing image superimposition processing to link MRI and magnetoencephalography measurement data, and then, as shown in FIG. 4 of JP 2023-020297 A (Patent Document 3), a conductor model (sphere model) is set while viewing the MRI, so that subsequent analysis processing can be performed.

[0191] In addition, if an MRI is not available, the image overlay process can be omitted, and instead of displaying the MRI shown in Figure 4 of JP 2023-020297 A (Patent Document 3), a spherical model can be set by displaying the sensor coordinates and marker coil coordinates in the spatial coordinate system of the magnetoencephalography measurement, or automatic control can be used to calculate specified coordinates and radius from the sensor coordinates and marker coil coordinates.

[0192] Next, the information processing device 50 automatically analyzes the recorded data (step S105). The user during automatic analysis is set to "automatic analysis" in the example of FIG.

[0193] Next, the information processing device 50 stores the automatic analysis data (step S106).

[0194] Next, the information processing device 50 performs user authentication (step S107). Here, it is assumed that the work has been completed once in step S106, but the user who has been authenticated by login in step S101 can continue the analysis in step S108 described later. In that case, the user authentication in step S107 may be omitted as appropriate.

[0195] Next, the user who has been authenticated in step S107 confirms and corrects the magnetoencephalogram data of the annotation on the displays of FIGS. 15 to 18 (step S108).

[0196] Finally, the information processing device 50 stores the analysis data confirmed and corrected in step S108 in the same manner as in step S106 (step S109).

[0197] In this manner, according to the present embodiment, by displaying the auxiliary information corresponding to the representative time of the determined waveform according to the waveform classification result, it is possible to visually display necessary information and present information that makes it easier for the user to notice it. For example, it is possible to visually display information necessary for verifying a positive result of an epilepsy spike candidate, and to present information that makes it easier for the user to notice a false negative.

[0198] In addition, the user can easily verify the positive / negative judgment results of the extracted characteristic waveform.

[0199] (Second embodiment) Next, a second embodiment will be described.

[0200] The second embodiment differs from the first embodiment in that machine learning is used to generate the IED probability map. In the following description of the second embodiment, the description of the same parts as in the first embodiment will be omitted, and only the parts different from the first embodiment will be described.

[0201] The IED probability map may be calculated using a conventional spike position detection algorithm (see Non-Patent Document 4) in addition to a method of applying a model calculated using machine learning. When machine learning is used, the time of IED manually detected by a doctor is learned as the correct answer, so compared to conventional spike position detection methods, it has the effect of obtaining results similar to those obtained when a doctor performs analysis. Conventional methods are developed with the aim of detecting the origin or peak of an IED, but in actual analysis, doctors perform analysis while adjusting the time at which dipoles are estimated from the waveform, findings, etc., between the origin and the peak.

[0202] Here, a case where the IED probability map calculation unit 502 uses machine learning to generate the IED probability map will be described.

[0203] Fig. 20 is a flowchart showing the flow of the learning process when generating an IED probability map using machine learning according to the second embodiment. As shown in Fig. 20, when the IED probability map calculation unit 502 acquires preprocessed magnetoencephalography data preprocessed by the preprocessing unit 501 (step S11), the IED probability map calculation unit 502 performs learning for outputting an IED probability map for the preprocessed magnetoencephalography data (step S12). Note that, as the ground truth data used in the learning, the sampling time recorded when the dipole estimation was performed and the information of the sensor used at that time are used.

[0204] Specifically, this is realized by applying a network that is often used in a task called Semantic Segmentation, such as U-Net described in Non-Patent Document 5. Semantic segmentation is a task that labels each element of all input sequences, and is used in medical images to automatically estimate brain tumors and cancer regions. Here, the mask data created from the sensor selected when the doctor performed dipole estimation and the time of estimation is used as the correct answer, and learning is performed so that the input magnetoencephalography data becomes mask data. At this time, in order to improve generalization performance, a data augmentation method such as randomly rearranging the order of sensors or setting a certain range of values ​​of a specific sensor to 0 may be applied.

[0205] This can also be achieved with networks used in tasks other than Semantic Segmentation, such as Object Detection and Instance Segmentation. Object Detection is a method for estimating where a target object is located in an input sequence using rectangles, etc. When using this method, it is not possible to calculate an IED probability map, but it is possible to directly obtain the time (e.g., the midpoint of the rectangle) and sensor for dipole estimation from the detected rectangle. Instance Segmentation is a task that combines Semantic Segmentation and Object Detection, and is capable of calculating the classification and number of detected objects as well as the area of ​​the object. Instance Segmentation can calculate an IED probability map, so it can be applied as is.

[0206] Next, when the learning in step S2 is completed, the IED probability map calculation unit 502 creates a trained model (step S13). The created trained model is used for machine learning inference, that is, when detecting IEDs in unknown data.

[0207] Thus, according to this embodiment, it is possible to additionally learn the automatic analysis results confirmed by a doctor, and to additionally learn only the dipole estimation results of the time data reviewed by a specific user, making it easier to improve the accuracy of the learning model.

[0208] Furthermore, in each of the above-mentioned embodiments, when at least any of the functional units of the biosignal measurement system 1 is realized by execution of a program, the program is provided by being pre-installed in a ROM, etc. Furthermore, the program executed by the biosignal measurement system 1 according to the above-mentioned embodiment may be provided by being recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R (Compact Disc Recordable), or a DVD (Digital Versatile Disc) in the form of an installable or executable file.

[0209] Furthermore, the programs executed by the biosignal measuring system 1 of each of the above-described embodiments may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network.

[0210] The programs executed in the biosignal measurement system 1 of each of the above-mentioned embodiments may be configured to be provided or distributed via a network such as the Internet. The programs executed in the biosignal measurement system 1 of each of the above-mentioned embodiments have a modular configuration including at least any one of the above-mentioned functional units, and as actual hardware, the CPU reads out the programs from a ROM or the like and executes them, so that the above-mentioned functional units are loaded and generated on a main storage device.

[0211] For example, aspects of the present invention are as follows. <1> an additional information drawing unit that draws additional information of a waveform acquired by a plurality of sensors and displayed on a screen according to a predetermined time; a waveform extraction unit that extracts a characteristic waveform from the waveform data; a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit; a waveform classification unit that classifies the characteristic waveforms extracted by the waveform extraction unit; Equipped with the auxiliary information drawing unit displays auxiliary information corresponding to the representative time of the waveform determined by the representative time determination unit according to a classification result by the waveform classification unit. 1. A waveform display device comprising: <2> The waveform extraction unit comparing each of the waveform data acquired by the plurality of sensors with at least one characteristic waveform information; The representative time determination unit determining an appearance probability of characteristic waveform information in at least a certain section of the waveform data based on a correlation ratio between a peak section of the waveform data and the characteristic waveform information; Identifying a time when a section matching the characteristic waveform information appeared and a corresponding sensor based on the appearance probability; The waveform classification unit determines a classification of the characteristic waveform according to the time determined by the representative time determination unit and occurrence probability information corresponding to the corresponding sensor. Characterized by <1> Waveform display device. <3> The auxiliary information drawing unit draws the number of corresponding sensors corresponding to the determined time as auxiliary information. Characterized by <2> The waveform display device according to claim 1 . <4> the auxiliary information drawing unit highlights the number of sensors corresponding to the time determined as the auxiliary information. Characterized by <3> Waveform display device. <5> The auxiliary information drawing unit can selectively display or highlight the auxiliary information according to the classification result of the waveform by the waveform classification unit. Characterized by <1> Or <3> Any one of the above waveform display devices. <6> The classification of the waveform classification unit is positive, negative, false negative candidate, and false positive candidate. Characterized by <5> Waveform display device. <7> The representative time determination unit determines the time of the appearance and the corresponding sensor from a plurality of candidate times; the auxiliary information drawing unit displays the auxiliary information to be displayed including the appearance probability in the predetermined range determined by the representative time determination unit and a plurality of candidate times; Characterized by <2> Waveform display device. <8> the additional information drawing unit displays the additional information in colors corresponding to the probability value, the plurality of candidate times, and the determined time. Characterized by <7> Waveform display device. <9> the additional information drawing unit displays the plurality of candidate times in a complementary color to a color of a probability value. Characterized by <8> Waveform display device. <10> the additional information drawing unit displays the additional information by changing the line type corresponding to the probability value, the plurality of candidate times, and the determined time. Characterized by <7> Waveform display device. <11> <1> Or <10> A waveform display device according to any one of the above items, A plurality of sensors; A waveform display system comprising: <12> A waveform display method for a waveform display device that displays waveforms acquired by a plurality of sensors on a screen, comprising: an additional information drawing step of drawing additional information of the waveform in accordance with a predetermined time; A waveform extraction step of extracting a characteristic waveform from the waveform data; a representative time determination step of determining a representative time of the characteristic waveform extracted in the waveform extraction step; a waveform classification step of classifying the characteristic waveforms extracted by the waveform extraction step; Including, the additional information drawing step displays additional information corresponding to the representative time of the waveform determined in the representative time determining step in accordance with a classification result in the waveform classification step. 23. A waveform display method comprising: <13> Computer, an additional information drawing unit that draws additional information of a waveform acquired by a plurality of sensors and displayed on a screen according to a predetermined time; a waveform extraction unit that extracts a characteristic waveform from the waveform data; a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit; a waveform classification unit that classifies the characteristic waveforms extracted by the waveform extraction unit; Function as a the auxiliary information drawing unit displays auxiliary information corresponding to the representative time of the waveform determined by the representative time determination unit according to a classification result by the waveform classification unit. A program characterized by: [Explanation of symbols]

[0212] 1. Waveform display system 50 Waveform display device 502 Waveform extraction section 503 Waveform classification section 504 Representative time determination section 506 Display control unit 507 Additional Information Drawing Department [Prior art documents] [Patent documents]

[0213] [Patent Document 1] Patent Publication No. 2021-069929 [Patent Document 2] JP 2006-167350 A [Patent Document 3] JP 2023-020297 A [Non-patent literature]

[0214] [Non-Patent Document 1] “Artifact removal in magnetoencephalogram background activity with independent component analysis,” IEEE Trans Biomed Eng, vol. 54, no. 11, pp. 1965-1973,

Outdoor Tool2

Outdoor Tools3

Outdoor Tools 4

Claims

1. an additional information drawing unit that draws additional information of a waveform acquired by a plurality of sensors and displayed on a screen according to a predetermined time; A waveform extraction unit that extracts a characteristic waveform from the waveform data; a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit; a waveform classification unit that classifies the characteristic waveforms extracted by the waveform extraction unit; Equipped with the auxiliary information drawing unit displays auxiliary information corresponding to the representative time of the waveform determined by the representative time determination unit according to a classification result by the waveform classification unit.

1. A waveform display device comprising:

2. The waveform extraction unit comparing each of the waveform data acquired by the plurality of sensors with at least one characteristic waveform information; The representative time determination unit determining an appearance probability of characteristic waveform information in at least a certain section of the waveform data based on a correlation ratio between a peak section of the waveform data and the characteristic waveform information; Identifying a time when a section matching the characteristic waveform information appeared and a corresponding sensor based on the appearance probability; The waveform classification unit determines a classification of the characteristic waveform according to the time determined by the representative time determination unit and occurrence probability information corresponding to the corresponding sensor.

2. The waveform display device according to claim 1 .

3. The auxiliary information drawing unit draws the number of corresponding sensors corresponding to the determined time as auxiliary information.

3. The waveform display device according to claim 2.

4. the auxiliary information drawing unit highlights the number of sensors corresponding to the time determined as the auxiliary information.

4. The waveform display device according to claim 3.

5. The auxiliary information drawing unit can selectively display or highlight the auxiliary information according to the classification result of the waveform by the waveform classification unit.

2. The waveform display device according to claim 1 .

6. The classification of the waveform classification unit is positive, negative, false negative candidate, and false positive candidate.

6. The waveform display device according to claim 5.

7. The representative time determination unit determines the time of the appearance and the corresponding sensor from a plurality of candidate times; the auxiliary information drawing unit displays the auxiliary information to be displayed including the appearance probability in the predetermined range determined by the representative time determination unit and a plurality of candidate times; 3. The waveform display device according to claim 2.

8. the additional information drawing unit displays the additional information in colors corresponding to the probability value, the plurality of candidate times, and the determined time.

8. The waveform display device according to claim 7,

9. the additional information drawing unit displays the plurality of candidate times in a complementary color to a color of a probability value.

9. The waveform display device according to claim 8.

10. the additional information drawing unit displays the additional information by changing the line type corresponding to the probability value, the plurality of candidate times, and the determined time.

8. The waveform display device according to claim 7,

11. A waveform display device according to any one of claims 1 to 10, A plurality of sensors; A waveform display system comprising:

12. A waveform display method for a waveform display device that displays waveforms acquired by a plurality of sensors on a screen, comprising: an additional information drawing step of drawing additional information of the waveform in accordance with a predetermined time; A waveform extraction step of extracting a characteristic waveform from the waveform data; a representative time determination step of determining a representative time of the characteristic waveform extracted in the waveform extraction step; a waveform classification step of classifying the characteristic waveforms extracted by the waveform extraction step; Including, the additional information drawing step displays additional information corresponding to the representative time of the waveform determined in the representative time determining step in accordance with a classification result in the waveform classification step.

23. A waveform display method comprising:

13. Computer, an additional information drawing unit that draws additional information of a waveform acquired by a plurality of sensors and displayed on a screen according to a predetermined time; A waveform extraction unit that extracts a characteristic waveform from the waveform data; a representative time determination unit that determines a representative time of the characteristic waveform extracted by the waveform extraction unit; a waveform classification unit that classifies the characteristic waveforms extracted by the waveform extraction unit; Function as a the auxiliary information drawing unit displays auxiliary information corresponding to the representative time of the waveform determined by the representative time determination unit according to a classification result by the waveform classification unit. A program characterized by:

Images