IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS AND PROGRAM

Abstract

To quickly perform integrated display of a plurality of pieces of modality data on a desired modality without mismatch even under such a situation that a plurality of pieces of inspection data are collectively input.SOLUTION: An image processing method comprises: an image extraction step of extracting a candidate of a modality image for visualization from a single or a plurality of modality image data groups; a first association step of associating the extracted modality data with another modality data that refers to the extracted modality image; an image decision step of uniquely deciding an image for visualization on the basis of the modality image extracted in the image extraction step and an association result of the first association step; a second association step of associating another modality data that has not been associated in the first association step with the modality image extracted in the image extraction step; and an image display step of displaying the other modality data on the modality image extracted in the image extraction step.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing method, an image processing device, and a program. [Background technology]

[0002] In recent years, with the improvement of computer performance and the development of artificial intelligence, it is known that sections with characteristic waveforms can be extracted from multiple recorded modality data (magnetoencephalography measurement data), and then cross-sectional dipole analysis can be performed to detect them as candidates for epileptic spikes.

[0003] Patent Document 1 discloses a technique for associating the positional relationship during brain magnetic field measurement with the positional relationship during MR image measurement. Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when dipole position information from multiple modality data (magnetoencephalography measurement data) of the same subject was integrated and displayed on a single modality (MR image) and dipole clusters were statistically analyzed, a problem arose in which inconsistencies occurred between the position information required to calculate dipole estimation and the MR image position information.

[0005] The present invention has been made in consideration of the above, and aims to quickly and consistently display multiple modality data on a desired modality, even in a situation where multiple modality data are input together. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the present invention is characterized by including an image extraction step of extracting candidates for modality images to be visualized from a data group of single or multiple modality images; a first matching step of matching the extracted modality image with another modality data that references the extracted modality image; an image determination step of uniquely determining an image to be visualized based on the modality image extracted in the image extraction step and the matching result of the first matching step; a second matching step of matching other modality data that could not be matched in the first matching step with the modality image extracted in the image extraction step; and an image display step of displaying the other modality data on the modality image extracted in the image extraction step. Effect of the Invention

[0007] According to the present invention, even in a situation where a plurality of pieces of examination data are input together, it is possible to quickly and consistently display a plurality of pieces of modality data on a desired modality. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the information processing device. [Diagram 3] FIG. 3 is a block diagram of a functional configuration of the information processing apparatus according to the first embodiment. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing a sphere model definition. [Diagram 5] FIG. 5 is a diagram showing a display example based on the spatial filter method. [Figure 6] FIG. 6 is a diagram showing a display example based on the dipole estimation method. [Figure 7] FIG. 7 is a diagram showing an example of a display by the signal source display unit. [Figure 8]FIG. 8 is a flowchart showing the flow of the selection process in the display modality selection unit. [Figure 9] FIG. 9 is a diagram showing an example of a statistical map. [Figure 10] FIG. 10 is a flowchart illustrating a flow of a selection process in the display modality selecting unit according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a statistical map. [Figure 12] FIG. 12 is a flowchart illustrating a flow of a selection process in the display modality selecting unit according to the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a statistical map according to the fourth embodiment. [Figure 14] FIG. 14 is a block diagram of a functional configuration of the information processing device according to the fifth embodiment. As shown in FIG. [Figure 15] FIG. 15 is a diagram illustrating an example of grouping by the analysis grouping unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image processing method, an image processing device, and a program will be described in detail with reference to the accompanying drawings.

[0010] (First embodiment) <System configuration> FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to a first embodiment. In FIG. 1, the information processing system 1 measures and displays a plurality of types of biosignals, for example, magnetoencephalography (MEG) signals and electroencephalography (EEG) signals. The information processing system 1 includes a measurement device 3, a data recording server 40, and an information processing device 20 which is an image processing device. The information processing device 20 has a monitor display 26 which displays signal information (measurement information) obtained by measurement and analysis results. Here, the data recording server 40 and the information processing device 20 are depicted separately, but at least a part of the data recording server 40 may be incorporated into the information processing device 20.

[0011] The subject lies on his / her back on the measurement table 4 with electrodes (or sensors) for measuring EEG attached to his / her head, and places his / her head in the recess 31 of the Dewar 30 of the measurement device 3. The Dewar 30 is a holding container in an extremely low temperature environment using liquid helium, and a number of magnetic sensors for measuring magnetoencephalography are arranged inside the recess 31 of the Dewar 30. The measurement device 3 collects EEG signals from the electrodes and magnetoencephalography signals from the magnetic sensors, and outputs the collected biosignals (measurement information) to the data recording server 40. The measurement information recorded in the data recording server 40 is read out by the information processing device 20, where it is displayed and analyzed. In general, the Dewar 30 containing the magnetic sensors and the measurement table 4 are arranged in a magnetically shielded room, but for convenience of illustration, the magnetically shielded room is omitted.

[0012] As described above, the measurement device 3 is a biomagnetic measurement device such as a magnetoencephalogram (MEG) measurement device, and functions as a multi-channel measurement device. The measurement device 3, which is a multi-channel measurement device, has known positions and orientations of multiple magnetic sensors for measuring the brain magnetism, and can calculate the response values ​​of the magnetic sensors when a signal source close to the magnetic sensors is assumed.

[0013] The information processing device 20 displays the waveforms of magnetoencephalographic signals from multiple magnetic sensors and the waveforms of electroencephalographic signals from multiple electrodes, synchronized on the same time axis. The electroencephalographic signals represent the electrical activity of nerve cells (the flow of ionic charges that occurs in the dendrites of neurons during synaptic transmission) as a voltage value between the electrodes. The magnetoencephalographic signals represent minute magnetic field fluctuations caused by electrical activity in the brain. The brain magnetic field is detected by a highly sensitive superconducting quantum interference device (SQUID) sensor.

[0014] <Hardware configuration> 2 is a block diagram showing an example of a hardware configuration of the information processing device 20. The information processing device 20 has a CPU (Central Processing Unit: processor) 21, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 23, an auxiliary storage device 24, an input / output interface 25, and a monitor display (display device) 26. The CPU 21, the RAM 22, the ROM 23, the auxiliary storage device 24, the input / output interface 25, and the monitor display (display device) 26 are connected to each other via a bus 27.

[0015] The CPU 21 controls the overall operation of the information processing device 20 and performs various information processing operations. The CPU 21 also executes an information processing program stored in the ROM 23 or the auxiliary storage device 24 to control the display operations of a measurement recording screen and an analysis screen.

[0016] The information processing program executed by the information processing device 20 of this embodiment is provided in the form of a file in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0017] The information processing program executed by the information processing device 20 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The information processing program executed by the information processing device 20 of the present embodiment may be provided or distributed via a network such as the Internet.

[0018] Furthermore, the information processing program executed by the information processing device 20 of this embodiment may be provided by being pre-installed in a ROM or the like.

[0019] The RAM 22 is used as a work area for the CPU 21, and may include a non-volatile RAM for storing main control parameters and information. The ROM 23 stores a basic input / output program, etc. The information processing program of the present invention may also be stored in the ROM 23.

[0020] The auxiliary storage device 24 is a storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and stores, for example, an information processing program that controls the operation of the information processing device 20, as well as various data, files, etc. necessary for the operation of the information processing device 20.

[0021] The input / output interface 25 includes both a user interface such as a touch panel, keyboard, display screen, and operation buttons, and a communication interface that takes in information from various sensors or the data recording server 40 and outputs analysis information to other electronic devices.

[0022] The monitor display 26 displays a measurement recording screen and an analysis screen, and the screen is updated in response to input / output operations via the input / output interface 25.

[0023] <Functional configuration> Next, functions of the information processing device 20 according to the present embodiment will be described.

[0024] The test data input to the information processing device 20 is composed of MR image measurement data (modality image) and magnetoencephalography measurement data (modality data for dipole analysis). The magnetoencephalography measurement data always includes something unique to the test data. However, the test data may be only magnetoencephalography measurement data (modality for dipole analysis). Also, the MR image measurement data may reuse MR images that have already been taken. For the sake of simplicity, the modality image is described as an MR image in the following description, but the system is not limited to an MR image, and can be similarly applied to other modality images such as CT and X-ray.

[0025] Fig. 3 is a block diagram showing a functional configuration of the information processing device 20. As shown in Fig. 3, the information processing device 20 includes a display modality selection unit 101, a modality association unit 102, a conductor model parameter selection unit 103, a signal source estimation unit 104, and a signal source display unit 105.

[0026] The display modality selection unit 101, the modality association unit 102, the conductor model parameter selection unit 103, the signal source estimation unit 104, and the signal source display unit 105 may be realized by the CPU 21 reading and executing an information processing program stored in the ROM 23 or the auxiliary storage device 2, or may be realized by hardware such as an IC (Integrated Circuit), or may be realized by a combination of software and hardware.

[0027] The display modality selection unit 101 functions as an image extraction unit that extracts candidates for modality images (MR images) to be visualized from a data group of single or multiple modality images (MR images). When multiple pieces of test data (magnetoencephalography measurement data) that refer to modality images are input to the information processing device 20, the display modality selection unit 101 first determines a modality image (MR image) to be displayed. This will be described in detail later.

[0028] The modality matching unit 102 functions as a first matching unit that matches another modality data (magnetoencephalography measurement data) that refers to the extracted modality image with the extracted modality image. As shown in the following formula (1), the modality matching unit 102 matches coordinate values ​​in the three-dimensional coordinate space of the MR image with the three-dimensional coordinate space of the magnetoencephalography measurement data. Similarly, the modality matching unit 102 can perform mutual conversion between the coordinate spaces by defining a simultaneous conversion matrix that matches coordinate values ​​of the analysis result of the signal source analyzed in the three-dimensional coordinate space of the magnetoencephalography measurement data with the three-dimensional coordinate space of the MR image.

[0029]

number

[0030] The transformation coefficients of the simultaneous transformation matrix can be calculated by, for example, defining the coordinates of a marker coil attached to the subject when measuring magnetoencephalography data and the coordinates of a marker that can be similarly imaged in MR images and attached to the subject as corresponding points, applying the solutions of each corresponding point to the simultaneous transformation matrix, and solving the equations for each coefficient, as disclosed in JP 2006-167350 A.

[0031] The conductor model parameter selection unit 103 uniquely determines an image to be visualized based on the modality image extracted by the display modality selection unit 101 and the association result by the modality association unit 102. More specifically, the conductor model parameter selection unit 103 selects parameters of a conductor model. As the conductor model, a spherical conductor model, which assumes the brain to be a uniform spherical conductor, is often used.

[0032] Here, Fig. 4 is a diagram showing the definition of the spherical model. As shown in Fig. 4, the conductor model parameter selection unit 103 displays a user interface, allowing the user to confirm and set the parameters of the spherical model (the central coordinates of the sphere in the three-dimensional coordinate space of the MR image, and the radius of the sphere).

[0033] If parameters for the MR image selected by the display modality selection unit 101 have already been determined, the conductor model parameter selection unit 103 displays the parameter values.

[0034] The signal source estimation unit 104 estimates the position, strength, distribution, etc. of the brain activity source (signal source) from the magnetoencephalography measurement data. More specifically, the signal source estimation unit 104 estimates the signal source by a method such as a dipole estimation method or a spatial filter method. The dipole estimation method generally assumes the number of current sources generated in the brain, and estimates the position (point) and strength of the current source from a certain time point of the measured magnetic field data. The spatial filter method estimates the current density distribution and strength, rather than points, from magnetic field data at multiple time points.

[0035] The signal source estimation unit 104 performs pre-processing of the magnetoencephalogram waveform data, such as band-pass filtering, baseline correction for aligning waveform offsets, and noise removal processing such as DSSP, as appropriate, before signal source estimation.

[0036] When unanalyzed test data (magnetoencephalography measurement data) is input, the signal source estimation unit 104 performs signal source estimation using a predetermined method. For example, the signal source estimation unit 104 determines the time when characteristic waveform information (IED) appears and extracts the sensor, and performs dipole estimation analysis based on the time and selected sensor information. In addition, when measurement is performed using a measurement protocol determined on the premise that a specified spatial filter is applied and reference time point information of the measurement data is explicitly specified, the signal source estimation unit 104 may perform a predetermined analysis such as performing calculations using a spatial filter method corresponding to the test protocol.

[0037] The signal source display unit 105 displays the distribution and intensity of the signal sources on the MR image. An example of the display method will now be described.

[0038] FIG. 5 is a diagram showing a display example by the spatial filter method. FIG. 5 shows an example of the display result of signal intensity and distribution by the spatial filter method. As shown in FIG. 5, the signal source display unit 105 displays each cross section of the 3D MR image (coronal plane 1051, sagittal plane 1052, axial plane 1053) and a display unit 1054 capable of 3D free viewpoint display of the cerebral cortex surface. The signal source display unit 105 displays the distribution and intensity of the signal source by shading 1056 on the brain image. The signal source display unit 105 also displays the signal intensity corresponding to the shading in the symbol 1055. Furthermore, the signal source display unit 105 displays the position and value of the peak point having a maximum and minimum in the signal intensity in the distribution 1057.

[0039] FIG. 6 is a diagram showing a display example using the dipole estimation method. FIG. 6 shows an example 1058 in which an estimated point and its direction using the dipole estimation method are displayed in three dimensions. As shown in FIG. 6, the signal source display unit 105 determines that the distance between the dipoles is within a predetermined range and that the positions of the dipoles are close to each other, and displays the dipoles collectively as a dipole cluster. Note that, as shown in FIG. 6, the signal source display unit 105 displays isolated dipoles in white. That is, the signal source display unit 105 functions as a second association unit that associates other modality data, which is modality data (magnetoencephalography measurement data) that has not been associated by the modality association unit 102, with the modality image extracted by the display modality selection unit 101. Also, the signal source display unit 105 functions as an image display unit that displays other modality data on the modality image extracted by the display modality selection unit 101.

[0040] In addition, as described above, the signal source display unit 105 can display statistical analyses such as spatial filter analysis and dipole cluster analysis accurately without error at the position on the MR image by superimposing the data (magnetoencephalography measurement data) of a plurality of tests and analysis methods. Here, Fig. 7 is a diagram showing a display example by the signal source display unit 105. The example shown in Fig. 7 displays the estimation result of the signal source by dipole estimation and spatial filter in a superimposed display format.

[0041] In this way, the signal source display unit 105 displays, on the extracted modality image (MR image), other modality data that is modality data (magnetoencephalography measurement data) that could not be associated by the modality association unit 102. The signal source display unit 105 displays all associated data together, thereby enabling the analysis results of multiple tests to be integrated and analyzed visually and statistically.

[0042] Next, the display modality selection unit 101 will be described in detail.

[0043] Here, Fig. 8 is a flowchart showing the flow of the selection process in the display modality selection unit 101. As shown in Fig. 8, when each test data is input, the display modality selection unit 101 extracts an MR image (step S1). Specifically, if there is an MR image linked to the input test data (magnetoencephalography measurement data), the display modality selection unit 101 extracts the MR image. That is, if there is test data (magnetoencephalography measurement data) including an MR image among a plurality of test data groups, the display modality selection unit 101 selects the MR image and proceeds with the subsequent process.

[0044] Note that, if there is no MR image linked to the input examination data (magnetoencephalography measurement data) because only magnetoencephalography measurement has been performed, the display modality selection unit 101 proceeds directly to step S2.

[0045] Next, the display modality selection unit 101 creates a statistical map (step S2). Here, FIG. 9 is a diagram showing an example of the statistical map. As shown in FIG. 9, the display modality selection unit 101 registers the extracted MR image in the statistical map. The display modality selection unit 101 registers an identifier that can uniquely identify the data of the specified MR image, such as the file name, path, and DICOM identification ID of the MR image. Hereinafter, the identifier is referred to as the display modality ID shown in FIG. 9.

[0046] The display modality selection unit 101 registers information supplementing the display modality ID in the statistical map. As shown in FIG. 9, the display modality selection unit 101 sets an item called the number of reference examinations of MR images (information supplementing the display modality ID) in the statistical map as basic information for selecting a display modality ID. If a certain display modality ID is not registered, the display modality selection unit 101 adds it to the statistical map and sets the number of reference examinations to '1'. In addition, the display modality selection unit 101 performs similar processing with another examination data, and if the same display modality ID is selected, adds '1' to the number of reference examinations of the corresponding display modality ID. Note that, even when multiple MR images are referred to in one examination, it is sufficient to register or add '1' to the number of reference examinations of each display modality ID in the same manner, and there is no particular limit to the number of MR images per examination.

[0047] Next, the display modality selection unit 101 determines an MR image from the information of the statistical map generated in step S2 (step S3). As shown in Fig. 9, the statistical map has a display modality ID and the number of reference examinations. The display modality selection unit 101 selects the display modality ID with the largest number of reference examinations. If there are display modality IDs with the same number of reference examinations, the display modality selection unit 101 selects the display modality ID registered first (or last).

[0048] In this way, the display modality selection unit 101 selects a display modality ID that is referenced frequently, thereby making it possible to reduce the number of test data items that require recalculation of signal source estimation, thereby reducing the analysis time when a large amount of test data is input.

[0049] In addition, when no data is registered in the statistical map, it means that there is no MR image. In this case, the display modality selection unit 101 notifies the user to urge him / her to review the input data, or prepares data defining MR image marker coordinates as shown in Fig. 4 on the MR image of a standard brain (ICBM152, Colin27, etc.) published by The McConnell Brain Imaging Centre, and determines the MR image as the display modality, so that the display modality can be applied even if it does not exist in all examinations.

[0050] The reason why this method is so effective is the use of automatic analysis by AI. Magnetoencephalogram analysis is difficult and time-consuming. It is generally unrealistic for a doctor to thoroughly check all the magnetoencephalogram waveforms recorded over a long period of time. Therefore, it is assumed that an initial analysis is first performed using the specified test data to create tested data. Then, it is assumed that the subsequent test data is left to automatic analysis while reusing the doctor's analysis parameters and settings. After that, it is assumed that a workflow will be implemented in which reanalysis and statistical analysis are efficiently performed based on the results of the automatic analysis and the findings are utilized.

[0051] In this way, according to the present embodiment, it is possible to eliminate the need for signal source estimation of the magnetoencephalography measurement data (modality data for dipole analysis) corresponding to the most frequent modality (MR image). Therefore, it is possible to reduce time, and even in a situation where multiple test data (magnetoencephalography measurement data) are input together, it is possible to quickly and consistently integrate and display the dipole analysis results, which are multiple test data (magnetoencephalography measurement data), on the desired modality (MR image).

[0052] (Second embodiment) Next, a second embodiment will be described.

[0053] The second embodiment differs from the first embodiment in that it assumes a case where only dipole estimation and simple spatial filtering are performed. In the following description of the second embodiment, the description of the same parts as in the first embodiment will be omitted, and only the parts different from the first embodiment will be described.

[0054] The display modality selection unit 101 according to the second embodiment will be described in detail.

[0055] 10 is a flowchart showing the flow of the selection process in the display modality selection unit 101 according to the second embodiment. Step S1 shown in FIG. 10 has been described in the first embodiment, so description thereof will be omitted here.

[0056] Next, the display modality selection unit 101 creates a statistical map (step S4). Here, Fig. 11 is a diagram showing an example of the statistical map. As shown in Fig. 11, the display modality selection unit 101 has a display modality ID, a total analysis time point, an analysis time point, and a recorded data ID in the statistical map.

[0057] In the actual situation of magnetoencephalography test data, test data (magnetoencephalography measurement data) is recorded over multiple sessions per test, and magnetic field waveform data is stored for the number of sessions. Furthermore, when performing analysis using a spatial filter, multiple sub-sessions are configured per session, and analysis may be performed using data obtained by averaging the magnetoencephalography measurement data of the sub-sessions as input. In any case, multiple data files to be analyzed exist in one test, and recorded data IDs for identifying these files are assigned and managed, and signal source estimation analysis is performed for some or all of the recorded data IDs. In this embodiment, the recorded data ID is synonymous with the unit of one analysis.

[0058] Specifically, in step S4, the display modality selection unit 101 defines and calculates the number of reference data files indicating the number of times each recording data ID refers to the display modality ID. Next, the display modality selection unit 101 calculates the analysis time point indicating the number of references of the waveform referred to in the analysis of the magnetoencephalogram included in the analysis data of each recording data ID. For example, in the case of dipole analysis, the display modality selection unit 101 calculates one dipole per time point of the recorded waveform. As in the example of the statistical map shown in FIG. 11, if the recording data ID: A001 analyzes 50 dipoles, the analysis time point is 50. Also, as in the example of the statistical map shown in FIG. 11, if the recording data ID: C001 is analyzed by the minimum norm method of the spatial filter method, the analysis time point is 1000 time points when the magnetoencephalogram waveform is recorded at a sampling frequency of 2000 Hz and analyzed in an analysis section of 0.5 seconds. If no analysis is performed, the display modality selection unit 101 sets the analysis time score to 0.

[0059] Moreover, the display modality selection unit 101 defines the analysis time points determined for each recorded data ID as described above as a total analysis time point by grouping and aggregating the analysis time points by display modality ID.

[0060] As described above, the display modality selection unit 101 determines the modality image based on the analysis time. By selecting the modality image with the longest analysis time from among the already analyzed modality images in this manner, the analysis time required for association can be shortened.

[0061] Next, the display modality selection unit 101 determines an MR image from the information of the statistical map generated in step S4 (step S5).

[0062] In step S5, the display modality selection unit 101 uses the statistical map to select in three priorities. Specifically, the display modality selection unit 101 extracts the modality with the largest total analysis time point (step S5-1), extracts the modality with the largest number of reference files when the total analysis time points are the same (step S5-2), and selects the newest one (step S5-3). In this way, it is possible to determine the display modality ID that references the analysis file with the largest number of analysis time points. This makes it possible to select a modality ID that can further reduce the recalculation time.

[0063] As described above, according to this embodiment, by selecting the modality image that has already been analyzed and that requires the longest analysis time, the analysis time required for association can be shortened.

[0064] In the second embodiment, the statistical map uses an analysis time point indicating the number of waveforms referenced in the analysis of the magnetoencephalogram, but this is not limited to this and may also be the analysis time required to analyze the magnetoencephalogram.

[0065] For example, in the analysis by the spatial filter method described in the second embodiment, the sampling frequency and the number of analysis time points contribute to the calculation time. For example, in "Sekihara K., Kumihashi I. (2010) Array-Gain Constraint Minimum-Norm Spatial Filter with Recursively Updated Gram Matrix for Biomagnetic Source Reconstruction. In: Supek S., Susac A. (eds) 17th International Conference on Biomagnetism Advances in Biomagnetism - Biomag2010. IFMBE Proceedings, vol 28. Springer, Berlin, Heidelberg.", the minimum norm method of the spatial filter method is applied recursively to update the calculation parameters. In this way, the number of recursions n is analyzed and multiplied by the calculation time of the spatial filter method to make an evaluation index that takes into account the actual analysis time, thereby improving the estimation accuracy.

[0066] In addition, in the analysis using spatial filters using time-frequency analysis such as "Dalal, Sarang S., et al. "Five-dimensional neuroimaging: localization of the time-frequency dynamics of cortical activity." Neuroimage 40.4 (2008): 1686-1700.", the sampling frequency and the number of analysis time points contribute to the calculation time. This results in the resolution of the analysis interval of the time-frequency analysis, Analysis time division × frequency division number ÷ division window Since the spatial filter calculation is repeated for the overlap rate of , the actual analysis time can be estimated by multiplying this by the spatial filter calculation time.

[0067] In this way, by estimating the analysis time taking into account the analysis time points of the magnetoencephalography measurement data and the properties of the selected MR image, it is possible to select a display modality ID that takes into account the calculation time. In other words, by correcting the estimation according to the environment, it is possible to determine the image without making an error in the estimation.

[0068] Note that if the analysis environment is the same, the estimate is the same and valid, so the modality association unit 102 may record the analysis time in the modality data (magnetoencephalography measurement data) in advance and refer to it as the analysis time.

[0069] Furthermore, if the modality association unit 102 is compatible with a GPU or a multi-core CPU, it can estimate the analysis time more accurately by calculating the analysis time while taking into consideration hardware information that can be acquired from the OS.

[0070] In addition, if the analysis environment of the multiple test data (magnetic brain measurement data) is diverse and different from the analysis environment of the embodiment, the time required for reanalysis may vary or be reversed. To avoid such a situation, it is necessary to re-estimate the execution time according to the execution environment.

[0071] (Third embodiment) Next, a third embodiment will be described.

[0072] The third embodiment differs from the first and second embodiments in that it is assumed to use a conductor model that utilizes brain region extraction from MR images. In the following description of the third embodiment, the description of the same parts as the first and second embodiments will be omitted, and only the parts that differ from the first and second embodiments will be described.

[0073] The display modality selection unit 101 according to the third embodiment will be described in detail.

[0074] 12 is a flowchart showing the flow of the selection process in the display modality selection unit 101 according to the third embodiment. Steps S1 and S4 shown in FIG. 12 have been described in the second embodiment, and therefore will not be described here.

[0075] The display modality selection unit 101 determines an MR image from the information of the statistical map generated in step S4 (step S6).

[0076] In step S6, the display modality selection unit 101 uses a statistical map to select in four priority orders. Specifically, the display modality selection unit 101 automatically extracts, with priority, those having the surface shape of the cerebral cortex as depicted in advance on the display unit 1054 shown in FIG. 5 from the MR image (step S6-1). Note that methods for automatically extracting those having the surface shape of the cerebral cortex include, for example, statistical method-based "FreeSurfer (Fischl et al., 2002), BrainSuite (Shattuck and Leahy, 2002), SPM (Friston et al., 2007), ANTs (Avants et al., 2009), FSL (Jenkinson et al. 2012), and CNN-based FastSurfer (Henschel,, et al. 2020)".

[0077] Next, the display modality selection unit 101 obtains an inscribed sphere inscribed in the cerebral cortex surface shape extracted in step S6-1, and sets the central coordinates of the inscribed sphere as the central coordinates of the sphere. Next, the longest distance from the center of the inscribed circle to the surface of the cerebral cortex shape is set as the radius. Finally, from the center and radius of the sphere, a sensor lead field is calculated using the Sarvas formula, which is a spherical model, to formulate the relationship between the measurement data of the magnetoencephalography sensor and the current data in the brain, thereby enabling subsequent signal source estimation.

[0078] As an example other than the spherical model, there is an overlapping sphere model that takes into account a more realistic shape than a sphere, as described in "Huang, MX, John C. Mosher, and RM Leahy. "A sensor-weighted overlapping-sphere head model and exhaustive head model comparison for MEG." Physics in Medicine & Biology 44.2 (1999): 423." This extracts the brain cortical area contained in a predetermined range of each of multiple magnetoencephalography sensors, and calculates the center coordinates and radius of a sphere inscribed in the extracted brain cortical area. The centers and radii of the spheres corresponding to all sensors are calculated, and the sensor lead field is calculated using Sarvas' formula.

[0079] If the brain region is not automatically extracted as described in the first embodiment, it is necessary to set the center and radius of the sphere for each MEG sensor, which is not a realistic option. However, if the brain cortex is extracted, automatic processing and calculations can be performed.

[0080] Alternatively, a real shape model using numerical approximation such as the boundary element method or the finite element method may be used.

[0081] The subsequent steps S5-1 to S5-3 have been explained in the second embodiment, so the explanation thereof will be omitted here.

[0082] With this configuration, it is possible to estimate the signal source and display the signal source without bothering the user. Of course, it is also possible to display the automatically extracted spherical model result and the surface shape of the cerebral cortex on the display unit 1054 shown in Fig. 5 to prompt the user to confirm them, and to specify whether or not to reset the parameters and recalculate.

[0083] (Fourth embodiment) Next, a fourth embodiment will be described.

[0084] The fourth embodiment differs from the first to third embodiments in that the analysis is performed by reusing the surface shape of the cerebral cortex contained in the MR image. In the following description of the fourth embodiment, the description of the same parts as the first to third embodiments will be omitted, and only the parts different from the first to third embodiments will be described.

[0085] In the configuration for performing the analysis shown in the third embodiment, it is assumed that the examination data includes the surface shape of the cerebral cortex in addition to the MR images, and it is also possible to perform analysis by reusing the surface shape of the cerebral cortex.

[0086] Here, Fig. 13 is a diagram showing an example of the statistical map according to the fourth embodiment. As shown in Fig. 13, the display modality selection unit 101 has, in the statistical map, a display modality ID, the number of reference data files, a total analysis time (including all processing), a signal source estimation analysis time, the presence or absence of a cerebral cortex surface shape, and a recorded data ID.

[0087] The display modality selection unit 101 defines and calculates the number of reference data files indicating the number of times each recorded data ID has referred to a display modality ID.

[0088] Furthermore, the display modality selection unit 101 calculates the analysis time for signal source estimation (signal source estimation analysis time) required for analyzing the magnetoencephalogram included in the analysis data of each recorded data ID, as described in the third embodiment.

[0089] The display modality selection unit 101 also checks whether the analysis data includes shape data of the cerebral cortex. Note that the display modality selection unit 101 may set and manage the extraction time of the surface shape of the cerebral cortex instead of the information on whether the analysis data includes shape data of the cerebral cortex. In FIG. 13, the extraction time of the surface shape of the cerebral cortex is also shown.

[0090] Moreover, the display modality selection unit 101 calculates the total analysis time (signal source estimation analysis time+(time to extract the surface shape of the cerebral cortex)) for each display modality ID.

[0091] Here, the flow of processing in the display modality selection unit 101 according to the fourth embodiment will be described in detail.

[0092] In step S6-1 in Fig. 12 shown in the third embodiment, the display modality selection unit 101 uses the statistical map shown in Fig. 13 to extract only the display modality IDs for which the surface shape of the cerebral cortex is "present." On the other hand, if all the surface shapes of the cerebral cortex are "absent," the display modality selection unit 101 proceeds to the next flow without performing the extraction process of the statistical map.

[0093] 12, the display modality selection unit 101 extracts the display modality ID having the longest total analysis time. If a unique display modality ID is not determined at this point, the display modality selection unit 101 selects a unique display modality ID as shown in S5-2 to S5-3 in the second embodiment.

[0094] In this way, according to the present embodiment, by selecting data that avoids recalculation of the surface shape of the cerebral cortex, which takes a long time to process, it is possible to shorten the reanalysis time. It is also possible to shorten the reanalysis time when a magnetoencephalography test is added at a later date. By preferentially selecting MR images from which brain regions have been extracted in this way, it is possible to automatically perform the analysis without bothering the user.

[0095] In step S6-1 of the fourth embodiment, if one cortical algorithm is defined and only the parameter settings that control the quality of the cortical surface shape extraction can be changed, the cortical surface shape extraction algorithm that requires the longest processing time can be preferentially extracted. In this way, only the cortical model with the highest quality is reused, thereby improving the quality of the conductor model and saving the time cost of recalculation.

[0096] (Fifth embodiment) Next, a fifth embodiment will be described.

[0097] The fifth embodiment differs from the first to fourth embodiments in that the combination of the conductor model and the signal source estimation is unified for each group. In the following explanation of the fifth embodiment, the explanation of the same parts as the first to fourth embodiments will be omitted, and only the parts different from the first to fourth embodiments will be explained.

[0098] Fig. 14 is a block diagram showing a functional configuration of an information processing device 20 according to a fifth embodiment. As shown in Fig. 14, the information processing device 20 includes an analysis grouping unit 106 in addition to the display modality selecting unit 101, the modality associating unit 102, the conductor model parameter selecting unit 103, the signal source estimating unit 104, and the signal source display unit 105 described above.

[0099] The analysis grouping unit 106 performs grouping of test data (magnetoencephalography measurement data) and analysis methods (conductor model, signal source estimation) More specifically, the analysis grouping unit 106 classifies the test data according to the modality recording format.

[0100] Here, Fig. 15 is a diagram showing an example of grouping by the analysis grouping unit 106. Note that the user may be allowed to arbitrarily select grouping in a table format as shown in Fig. 15, or grouping may be automatically performed based on the examination protocol.

[0101] 15, an example of grouping by the analysis grouping unit 106 is automatically grouping test data (magnetoencephalogram measurement data) including only continuously recorded magnetoencephalogram data into group A. Also, test data (magnetoencephalogram measurement data) in which a predetermined time, timing and frequency of electrical stimulation performed to confirm somatosensory evoked magnetic fields (SEF) for the purpose of identifying the central sulcus before surgery are defined as a test protocol is automatically grouped into group B.

[0102] Thereafter, an MR image is selected by the display modality selection unit 101, and based on the selected MR image, modality association is performed by the modality association unit 102 for each group, conductor model parameter selection unit 103 selects conductor model parameters, and signal source estimation unit 104 performs signal source estimation.

[0103] Then, the signal source display unit 105 displays the signal source estimation results for each group in a superimposed display format of the spatial filter and dipole estimation as shown in FIG.

[0104] In the first and third embodiments, the determination of the conductor model and the signal source estimation for unanalyzed data (magnetoencephalography measurement data) are limited to a predetermined method, but are not limited to one in particular. By unifying the combination of the conductor model and the signal source estimation for each group as in the fifth embodiment, even when unprocessed test data is added, it becomes possible to automatically process and execute the grouped processing according to the test protocol without bothering the user.

[0105] Thus, according to this embodiment, by automatically sorting the input test data (magnetoencephalography measurement data) according to its type, such as setting up automatic dipole analysis for epilepsy analysis and automatic analysis of somatosensory cortex evoked magnetic fields (SEF) using a spatial filter method, even if the user adds unanalyzed data to this processing step, the appropriate analysis method can be automatically selected, and the data can be analyzed and simultaneously displayed.

[0106] In the above-described embodiment, when at least any of the functional units of the information processing system 1 is realized by executing a program, the program is provided by being pre-installed in a ROM, etc. Also, the program executed in the biosignal measurement system 1 according to the above-described embodiment and modified example may be provided by being recorded in a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R (Compact Disc Recordable), or a DVD (Digital Versatile Disc) in the form of an installable or executable file.

[0107] The programs executed by the information processing system 1 of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The programs executed by the information processing system 1 of the above embodiment may be provided or distributed via a network such as the Internet.

[0108] In addition, the program executed in the information processing system 1 of the above-mentioned embodiment has a modular structure including at least any one of the above-mentioned functional units, and in terms of actual hardware, the CPU reads out the program from a ROM or the like and executes it, thereby loading and generating each of the above-mentioned functional units onto the main memory device. [Explanation of symbols]

[0109] 20 Image processing device 101 Display modality selection unit 102 Modality Matching Unit 103 Conductor model parameter selection section 105 Signal source display section [Prior art documents] [Patent documents]

[0110] [Patent Document 1] JP 2006-167350 A

Claims

1. an image extraction step of extracting candidates for modality images to be visualized from a data set of single or multiple modality images; a first association step of associating another modality data, which refers to the extracted modality image, with the extracted modality image; an image determination step of uniquely determining an image to be visualized based on the modality image extracted in the image extraction step and the matching result in the first matching step; a second association step of associating other modality data that was not associated in the first association step with the modality image extracted in the image extraction step; an image display step of displaying the other modality data on the modality image extracted in the image extraction step; 13. An image processing method comprising:

2. The first association step includes associating an analysis time previously analyzed using modality data that refers to the extracted modality image with the extracted modality image; The image determination step determines an image based on the analysis time.

2. The image processing method according to claim 1.

3. a hardware information acquiring step of acquiring hardware information of a computing device before the first association step; the first association step further includes a step of calculating an analysis time from a processing procedure previously analyzed using modality data that refers to the extracted modality image and the hardware information acquired in the hardware information acquisition step, and associating the processing procedure with the hardware information; The image determination step determines an image based on the analysis time.

2. The image processing method according to claim 1.

4. The first association step includes associating a reference time point of modality data that refers to the extracted modality image; The image determination step determines an image based on the reference time points.

2. The image processing method according to claim 1.

5. The image extraction step preferentially extracts modality images from which the surface shape of the cerebral cortex is extracted.

2. The image processing method according to claim 1.

6. before the second matching step, classifying the modality data by modality acquisition format; the second matching step matches other modality data that was not matched in the first matching step with the modality image extracted in the image extraction step, using a method that differs for each classification in the classification step; 6. The image processing method according to claim 1, wherein the image processing method further comprises:

7. an image extraction unit that extracts candidates for modality images to be visualized from a data group of single or multiple modality images; a first association unit that associates another modality data, which refers to the extracted modality image, with the extracted modality image; an image determination unit that uniquely determines an image to be visualized based on the modality image extracted by the image extraction unit and the association result of the first association unit; a second association unit that associates other modality data that has not been associated by the first association unit with the modality image extracted by the image extraction unit; an image display unit that displays the other modality data on the modality image extracted by the image extraction unit; An image processing device comprising:

8. an image extraction step of extracting candidates for modality images to be visualized from a data set of single or multiple modality images; a first association step of associating another modality data, which refers to the extracted modality image, with the extracted modality image; an image determination step of uniquely determining an image to be visualized based on the modality image extracted in the image extraction step and the matching result in the first matching step; a second association step of associating other modality data that was not associated in the first association step with the modality image extracted in the image extraction step; an image display step of displaying the other modality data on the modality image extracted in the image extraction step; A program for causing a computer to execute the following.

Images