Diagnostic support program

Abstract

Provided is a diagnosis assisting program that can display organ motion. A diagnosis assisting program that analyzes images of a human organ and displays the result of the analysis, said diagnosis assisting program being characterized by causing a computer to implement processing to acquire a plurality of frame images, processing to calculate a characterizing frequency of a condition of the organ on the basis of each frame image, processing to calculate the phase difference between a waveform of an organ model that was acquired in advance and a waveform corresponding to the calculated frequency, and processing to output a signal indicating the phase difference.

Description

TECHNICAL FIELD

[0001] The present invention relates to a diagnostic support program that analyzes images of human organs and displays the analysis results. BACKGROUND ART

[0002] In recent years, the mortality rate from heart diseases has been on the rise, and the need for useful and convenient diagnostic techniques has been increasing. MRI diagnostic techniques have made rapid progress, and it has become possible to perform a wide variety of cardiac examinations in a short time, so the importance of MRI in image diagnosis of the cardiac region is increasing. In "cardiac MRI", examinations such as "cine MRI", "perfusion", "delayed contrast enhancement", and "BB (Black Blood)" method are performed. In particular, cine MRI is characterized by having no restriction on the observation range, allowing observation of any arbitrary cross-section, and having excellent reproducibility, as compared with ultrasonography, SPECT examinations and the like. For this reason, it is commonly imaged in many medical institutions. In cine MRI, for example, using an electrocardiogram gating method, data collection of "about 10 slices / 20 phases" is performed for the entire left ventricle. Recently, the "Steady State method" has been used to achieve high contrast between blood and myocardium. Also, in "cardiac function analysis", evaluation of cardiac function by MRI can provide accurate values even when compared with CT, LVG (left ventriculography), and SPECT, and its need is increasing.

[0003] Thus, cardiac MRI is clinically useful, and cine MRI in particular is performed in many medical facilities. However, the images have rarely been analyzed using software. This is said to be because the software used for conventional cardiac function analysis required complicated operations to extract and correct the contours of the myocardium and adventitia. Furthermore, the reproducibility of analysis results has been a challenge, as the contour tracing of the myocardium and adventitia is susceptible to the influence of the individual operator. Moreover, with the spread of cardiac MRI and the increasing number of medical facilities using MRI devices from multiple manufacturers, the fact that sequence names differ among companies has created a challenge in handling data easily.

[0004] To address these challenges, software is available that improves the accuracy of tracing myocardial contours to reduce the effort involved in the complex process. Furthermore, even if unintended tracing occurs, it automatically interpolates the image, minimizing the need for correction. This software allows for stress-free image viewing in the MRI cardiac function analysis viewer by displaying images side-by-side and using flick gestures. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] https: / / www.zio.co.jp / ziostation2 / [Overview of the project] [Problems that the invention aims to solve]

[0006] However, simply displaying MRI images side-by-side, as in the technology described in Non-Patent Document 1, does not make it easy for physicians to understand the patient's condition. Therefore, it is desirable to display images that are relevant to the state of the heart. In other words, it is desirable to understand the heart of the human body being studied and display images that show its actual movement based on the heart's waveform or frequency, or the trend of changes in the images.

[0007] Furthermore, it has become clear that the movement of abnormal organs differs in phase from that of normal organs, but technology to display images of organs based on this fact has not yet been put into practical use.

[0008] This invention has been made in view of these circumstances and aims to provide a diagnostic support program capable of displaying organ movement. More specifically, it aims to calculate numerical values ​​that assist in diagnosis by quantifying the agreement rate, phase shift, and other discrepancies between the data of a new target to be measured and the already acquired wave shape and Hz, and further generate images that assist in diagnosis by imaging these numerical values. [Means for solving the problem]

[0009] (1) In order to achieve the above objective, the present invention employs the following means. Specifically, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring a plurality of frame images; calculating a frequency that characterizes the state of the organ based on each of the frame images; calculating the phase difference between a waveform of a previously acquired organ model and a waveform corresponding to the calculated frequency; and outputting a signal indicating the phase difference.

[0010] (2) In addition, in a diagnostic support program according to one aspect of the present invention, the organ model is characterized in that the organ image is divided and the average value of the pixel values ​​within each divided region is used to determine the organ model.

[0011] (3) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by dividing the image of the organ using the Voronoi tessellation method.

[0012] (4) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized in that, if the organ is the lung, it divides the lung field region according to the rate of change in lung volume.

[0013] (5) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring a plurality of frame images; extracting pixel values ​​of specific points included in the images of the organs; calculating the temporal change of the extracted pixel values; extracting at least one frequency signal from the temporal change of the pixel values; and outputting the extracted frequency signal as the signal of the specific point.

[0014] (6) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring images of the organ taken from multiple directions under multiple shooting conditions; extracting pixel values ​​of specific points included in the images of the organ; calculating the temporal change of the extracted pixel values; extracting at least one frequency signal from the temporal change of the pixel values; outputting the extracted frequency signal as a signal of the specific point; and creating a cross-sectional view of all or part of the organ using the output signal of the specific point.

[0015] (7) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by dividing the image of the organ into a certain region including the specific point using the Voronoi tessellation method.

[0016] (8) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring signals output from a diagnostic support program described in any of (1) to (7) above; causing an AI (Artificial Intelligence) to learn signals indicating normal organs or abnormal organs from the acquired signals; and recording the learning results by the AI.

[0017] (9) A diagnostic support program according to one aspect of the present invention further includes the process of acquiring a plurality of frame images, identifying an organ from the acquired frame images, comparing the identified organ with the recorded learning results, and outputting the percentage of abnormal values.

[0018] (10) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by further including a process that outputs the phase difference between a waveform showing the periodic operation of a normal organ and a waveform showing the periodic operation of an abnormal organ.

[0019] (11) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized in that, in each of the plurality of frame images, the program further includes a process of adding or subtracting the signals of parts with different transparency from the signals of other parts.

[0020] (12) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by further including a process of converting the waveform of a periodic signal into a trigonometric function, and a process of outputting a signal showing the waveform converted into the trigonometric function.

[0021] (13) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring a plurality of frame images taken by applying a periodic signal to the human body; performing a Fourier transform on the changes in pixel values ​​of a specific region of each frame image; extracting a spectrum within a certain band from the spectrum obtained after the Fourier transform, which includes a spectrum corresponding to the frequency of the signal applied to the human body; performing an inverse Fourier transform on the spectrum extracted from the certain band; and displaying each image after the inverse Fourier transform on a display.

[0022] (14) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: a process of acquiring a plurality of frame images taken by applying a periodic signal to the human body; a process of calculating the rate of change of pixel values ​​in a specific region of each frame image; a process of extracting only regions in which the synchronization rate is within a predetermined range, using a synchronization rate which is the ratio of the rate of change of pixel values ​​in the specific region to the periodic rate of change of the signal applied to the human body; and a process of displaying each image including the extracted region on a display.

[0023] (15) Furthermore, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring a plurality of frame images; calculating a function that shows the frequency and waveform of pulmonary blood flow based on each of the frame images; calculating a function that shows the waveform of pulmonary blood pressure based on the calculated function that shows the frequency and waveform of pulmonary blood flow and information that shows the diameter of pulmonary blood vessels; and estimating pulmonary blood pressure from the calculated pulmonary blood pressure waveform.

[0024] (16) A diagnosis support program according to one aspect of the present invention is characterized by further comprising: a process of acquiring a plurality of frame images; and a process of identifying an organ and blood flow flowing through the organ from the acquired frame images, comparing the identified organ and blood flow with the recorded learning result, and outputting a feature quantity of the blood flow in the organ.

[0025] (17) A diagnosis support program according to one aspect of the present invention is characterized by further comprising a process of outputting a feature quantity of blood flow in major blood vessels of the lung, or blood flow in pulmonary capillaries and peripheral pulmonary blood vessels.

[0026] (18) A diagnosis support program according to one aspect of the present invention is characterized by further comprising a process of comparing the movement of the lung during respiration with the movement of pulmonary blood flow, and outputting a feature quantity indicating the linkage between the two.

[0027] (19) A diagnosis support program according to one aspect of the present invention is characterized by further comprising: a process of acquiring a plurality of frame images; and a process of identifying a lung field region from the acquired frame images, comparing a wave indicating the movement of the identified lung field region with a reference wave, and outputting a feature quantity indicating the linkage of the waves.

[0028] (20) In the diagnosis support program according to one aspect of the present invention, the reference wave is a wave indicating a respiratory cycle.

[0029] (21) A diagnosis support program according to one aspect of the present invention is characterized by further comprising: a process of calculating a maximum value of a pixel value; and a process of acquiring the waveform based on a signal after the calculated maximum value.

[0030] (22) A diagnosis support program according to one aspect of the present invention is characterized by inputting data having periodicity, performing Fourier transform processing, performing filtering processing for extracting a specific frequency, and performing inverse Fourier transform processing.

[0031] (23) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by superimposing multiple waveforms and detecting the phase peak in one period of any of the waveforms to calculate the phase difference of any of the other waveforms.

[0032] (24) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized by dividing an image of the lung into multiple regions, calculating the mean and distribution of the intensity values ​​of each region, and correlating them with the count values ​​in lung scintigraphy.

[0033] (25) Furthermore, a diagnostic support program according to one aspect of the present invention is characterized in that it generates a basic waveform by superimposing a plurality of original waveforms obtained from each image, generates a master waveform based on the basic waveform, and generates a waveform corresponding to the organ in each image by setting or weighting the bandwidth of the original waveforms. [Effects of the Invention]

[0034] According to one aspect of the present invention, it is possible to identify the organs of the human body that are the subject of the invention and display an image showing the actual movement based on the waveform or frequency of the organs, or the trend of changes in the image. [Brief explanation of the drawing]

[0035] [Figure 1] This figure shows the schematic configuration of the diagnostic support system according to this embodiment. [Figure 2A] This is a diagram showing a cross-section of the heart. [Figure 2B] This is a diagram showing a cross-section of the heart. [Figure 2C] This is a diagram showing a cross-section of the heart. [Figure 2D] This figure shows an example of a Voronoi tessellation. [Figure 3A] This figure shows the change in "intensity" of a specific block and the results of Fourier analysis. [Figure 3B]This figure shows the Fourier transform result of extracting frequency components close to the heartbeat, and the inverse Fourier transform of this result, which shows the change in "intensity" of the frequency components close to the heartbeat. [Figure 3C] This figure shows an example of extracting a specific frequency band from the spectrum obtained after the Fourier transform. [Figure 4] This flowchart outlines the image processing according to this embodiment. [Figure 5] This flowchart outlines the image processing according to this embodiment. [Figure 6] This flowchart outlines the image processing according to this embodiment. [Figure 7A] This is a schematic diagram showing the left lung of the human body from a frontal view. [Figure 7B] This is a schematic diagram showing the left lung of the human body from the left side. [Figure 8A] This is a schematic diagram showing the left lung of the human body from a frontal view. [Figure 8B] This is a schematic diagram showing the left lung of the human body from the left side. [Figure 9A] This is a schematic diagram showing the left lung of the human body from a frontal view. [Figure 9B] This is a schematic diagram showing the left lung of the human body from the left side. [Figure 10A] This is a schematic diagram showing the left lung of the human body from a frontal view. [Figure 10B] This is a schematic diagram showing the left lung of the human body from the left side. [Figure 11] This figure shows an example of a lung field detection method according to the present invention. [Figure 12] This diagram shows a state where blood flow in the lungs is not uniform. [Figure 13] This diagram shows a state where blood flow in the lungs is not uniform. [Figure 14] This figure shows the lung analysis area divided into four sections. [Figure 15] This figure shows the result of approximating the change in the pixel values ​​of the lungs as a sine wave. [Figure 16] This figure shows the result of approximating the change in the pixel values ​​of the lungs as a sine wave. [Figure 17A] This figure shows the correlation between FEV (Forced Expiratory Volume in one second) and phase shift. [Figure 17B] This is an example illustrating the concept of phase difference. [Figure 18] This is a diagram showing the process of breathing. [Figure 19] This figure shows the changes in pulmonary blood flow within one heartbeat in a normal lung. [Figure 20] This figure shows the changes in pulmonary blood flow within one heartbeat in an abnormal lung. [Figure 21] This diagram shows a normal lung and a lung with PAH (pulmonary arterial hypertension, including pulmonary arteriosclerosis). [Figure 22] This is a diagram showing an image of a normal lung. [Figure 23] This diagram shows a normal lung and a lung with COPD. [Figure 24] This diagram shows the three-dimensional structure of the lung apex towards the diaphragm. [Figure 25] This is a diagram showing the peripheral areas of the lungs. [Figure 26] This diagram illustrates shallow and deep breathing in a normal lung. [Figure 27] This is a diagram showing the lungs of a heart failure patient. [Figure 28] This is a diagram showing the lungs of a heart failure patient. [Figure 29A] This figure shows a low-resolution difference image obtained by increasing the block size and aggregating the changes in brightness values. [Figure 29B] This figure shows a high-resolution difference image obtained by reducing the block size and aggregating the changes in brightness values. [Figure 29C] This figure shows example images of MBDP and ABDP. [Figure 29D] This diagram shows how the lung image is divided into six areas. [Figure 30A] This diagram shows the pulmonary blood flow during a single heartbeat, frame by frame. [Figure 30B]This figure shows examples of measurement results for MPA (Main Pulmonary Artery), DPA (Distal Pulmonary Artery), and DPV (Distal Pulmonary Vein). [Figure 30C] This figure shows the signal intensity in an ultrasound image of the atrium. [Figure 30D] This figure shows the signal intensity in an ultrasound image of the atrium. [Figure 30E] This diagram shows the wave-like movement of breath. [Figure 30F] This figure shows the three-dimensional structure and cross-section based on chest X-ray images. [Figure 30G] This diagram shows a normal lung and a patient's lung. [Figure 30H] This is a diagram showing the patient's lungs. [Figure 30I] This is a diagram showing the patient's lungs. [Figure 30J] This diagram shows the contrast between deep breathing and shallow breathing. [Figure 30K] This diagram shows how the waves of the diaphragm and other structures are misaligned with the original waves. [Figure 30L] This is a diagram showing images of pneumonia patients. [Figure 30M] This figure shows an example of an electrocardiogram. [Figure 31A] This figure shows an example of comparing the waveforms of aortic blood flow and ventricular volume. [Figure 31B] This figure shows two examples of lung images. [Figure 31C] This figure shows an example where the difference between two lung images is calculated, and the difference is visualized by setting a certain threshold. [Modes for carrying out the invention]

[0036] The inventors of this invention focused on the fact that, conventionally, there has been no practical application of technology to visualize the movement of organs (for example, the myocardium of the heart). They discovered that by representing not only the movement of organs but also deviations in organ movement and areas that are not moving, it is possible to support doctors' diagnoses, leading to the present invention.

[0037] In other words, a diagnostic support program according to one aspect of the present invention is a diagnostic support program that analyzes images of human organs and displays the analysis results, characterized in that it causes a computer to perform the following processes: acquiring a plurality of frame images; calculating frequencies that characterize the state of the organs based on each of the frame images; calculating the phase difference between a waveform of a previously acquired organ model and a waveform corresponding to the calculated frequencies; and outputting a signal indicating the phase difference.

[0038] This simplifies the diagnostic process, which previously required expertise, and also allows for the qualitative analysis of moving images, thereby increasing objectivity in the displayed content. In this specification, the heart is used as an example organ, but it goes without saying that the present invention is not limited to the heart and can be applied to various organs such as the lungs and blood vessels.

[0039] In the present invention, organ movement is adjusted (normalized) so that it can be considered constant, and judgment can be made by comparing it with the subject's previous images, by comparing it with an average of the area being analyzed and the entire organ, by comparing it with normal images, or by comparing it with age-appropriate samples.

[0040] [Basic Concepts] The basic concept of this invention will now be explained. In this invention, periodic changes that characterize the state of organs in the human body, such as the cross-sectional area, surface area, and volume of the heart, are captured as movements that repeat at a constant period, and the constant repetition or movement (routine) over time, either in whole or in a partial range, is captured and measured as a wave. The measurement results of the wave are either (a) the wave shape itself, or (b) the interval between waves (frequency: Hz).

[0041] For example, in an image of the heart, there may be waves that are linked in the same way at the same time. For example, in the case of a heartbeat, the following approximation can be conceived. (Average density change over a rough range) ≈ (Heart rate) ≈ (Changes in the heart) ≈ (Electrocardiogram) ≈ (Changes in the surface area and volume of the heart)

[0042] In this invention, for example, with respect to the heart, it is possible to analyze by focusing on the deformation of the local ventricular wall (wall thickness expansion, wall thickness contraction), and to capture and periodically represent the systolic thickness, diastolic thickness, and wall motion (adventiary and intima). Furthermore, in the heart, it is possible to capture and periodically represent the "relative wall thickness (RWT)" obtained from "(2 × posterior wall thickness) / (left ventricular end-diastolic diameter)" and the "ejection fraction" obtained from "(end-diastolic volume - end-systolic volume) / (end-diastolic volume)".

[0043] This invention enables more accurate image extraction using any of these data, or a combination thereof. This may involve multiple reciprocal calculations. During this process, artifacts are again excluded from the results, and features are extracted from the new data extraction waveform, the initial base data waveform, waveforms from other modalities, the surrounding data, and multiple waveforms. This process can be performed once or multiple times.

[0044] Here, when creating the base data, multiple modalities (for example, changes composed of a certain density, volume, heart movement, etc.) or multiple waveform measurements of heart rate, etc., complement each other's component extraction to improve accuracy. This makes it possible to reduce artifacts and improve accuracy based on certain predictions such as lines.

[0045] Here, "density" is translated as "density," but in imaging, it refers to the "absorption value" of a pixel in a specific area. For example, in CT scans, air is represented as "-1000," bone as "1000," and water as "0."

[0046] In this specification, "density" and "intensity" are used distinctly. "Density," as described above, refers to the absorption value. In the source image of XP or XP video, air has high permeability, and areas with high permeability are represented as white, with air being "-1000," water "0," and bone "1000." On the other hand, "intensity" is a relative change from "density," for example, a normalized value that "converts" it into a range of density or signal intensity. In other words, "intensity" is a relative value in an image, such as brightness or emphasis. While directly dealing with the absorption value of an XP image, it is expressed as "density" or "change in density (Δdensity)." Then, for the sake of image representation, this is converted as described above and expressed as "intensity." For example, when displaying color in 256 gradations from 0 to 255, it becomes "intensity." This distinction in terminology applies to XP and CT.

[0047] On the other hand, in the case of MRI, even if one tries to define air as "-1000," water as "0," and bone as "1000," the values ​​change significantly depending on the MRI pixel values, the type of measuring machine, the person's physical condition and build at the time of measurement, and the measurement time. Furthermore, the way MRI signals are acquired, such as T1-weighted images, varies depending on the facility and the type of measuring machine, and is not consistent. For this reason, in the case of MRI, a "density" definition like that used for XP or CT is not possible. Therefore, in MRI, relative values ​​are handled from the initial imaging stage, and are expressed as "intensity" from the beginning. The signal processed is also "intensity."

[0048] As a result, master data can be obtained. For the new target to be measured, the waveform and Hz of the master data are extracted within a certain width and range. For example, only heart rate data or blood vessel data can be extracted within a specific width and range. The width of this waveform and Hz is determined relatively and statistically, taking into account waveform elements from other functions, artifacts such as noise, waveforms of other modalities that appear to have synchronicity, and reproducibility obtained through multiple trials. Adjustment and experience are necessary (machine learning can also be applied). This is because widening the width and range introduces elements from other functions, while narrowing it too much removes elements from the function itself; therefore, adjustment of the range is necessary. For example, having multiple data sets makes it easier to define the range, Hz, and measurement agreement width. It is also possible to estimate the axis, width, range, and Hz fluctuations and widths by extracting components from each other. That is, by superimposing multiple trials, the optimal range of the axis, width, range, and Hz is calculated through averaging and variance of the Hz axis setting. In this process, noise (Hz) from other activities may be extracted, and if such waves are present, the relative extent to which they are suppressed may also be measured.

[0049] Next, numerical values ​​that aid in diagnosis are calculated by quantifying the waveform shape, the degree of agreement with Hz, and other discrepancies with the data of the new target to be measured. For example, by measuring the waveform agreement rate of the master disease along with noise rejection for pulse rate measurement and auscultation, and calculating the agreement rate of the disease waveform, it becomes possible to apply this to a diagnostic aid. This specification describes cases in which the synchronization agreement rate of pixel values ​​can be used.

[0050] [Regarding the rate of agreement / consensus] In this specification, the trend of image changes is described as the synchronization rate. For example, a region of myocardium is detected and divided into multiple block regions, and the average density (pixel value x) of the block region in each frame image is calculated. Then, the ratio (x') of the average pixel value of the block region in each frame image to the range of change from the minimum to the maximum value of the average density (pixel value x) (0% to 100%) is calculated. On the other hand, using the ratio (x' / y') of the ratio (y') of the change in myocardium (y) in each frame image to the range of change from the minimum to the maximum position of the myocardium (0% to 100%), only block regions whose ratio (x' / y') falls within a predetermined range are extracted.

[0051] Here, if y'=x' or y=ax (where a is a coefficient for the amplitude or density of the myocardium), it is a perfect match. However, not only perfect matches are meaningful values; values ​​within a certain range should be extracted. Therefore, in one aspect of the present invention, a certain range is defined using logarithms (log) as follows. That is, when calculated as the percentage of y=x, a perfect match in synchronization is "log y' / x'=0". Furthermore, when extracting a range of synchronization agreement rates that is narrow (mathematically narrow), for example, a range close to 0 is defined as "log y' / x'=-0.05 to +0.05", and if the range of synchronization agreement rates is wide (mathematically wide), for example, a range close to 0 is defined as "log y' / x'=-0.5 to +0.5". The narrower this range is, and the higher the numerical values ​​that match within that range, the higher the agreement rate can be said to be. If we calculate the value of this ratio for each pixel and count the number of occurrences, a normal distribution with a peak at the case of perfect match is obtained for healthy individuals. In contrast, for individuals with disease, the distribution of this ratio value becomes distorted. It should be noted that the method of determining the width using logarithms as described above is merely one example, and the present invention is not limited to this.

[0052] In other words, the present invention performs "image extraction" by assuming that (average of density changes over a certain rough range) ≈ (heart rate) ≈ (changes in the heart) ≈ (electrocardiogram) ≈ (changes in the surface area and volume of the heart), and methods other than those using logarithms can also be applied. Through such methods, it becomes possible to display frequency-synchronized images.

[0053] In the case of blood vessels, a series of density changes (x (a single waveform in the myocardium)) that occur in response to a series of cardiac contractions (y) have a slight time delay (phase change) in their original form, so they can be expressed as y = a'(xt). In the case of perfect match, t = 0, so y = x, or y = a'x. Here, when extracting a range of synchronization rate that is narrow (mathematically narrow), for example, a range close to 0 is defined as "log y' / x' = -0.05 to +0.05", and when extracting a range of synchronization rate that is wide (mathematically wide), for example, a range close to 0 is defined as "log y' / x' = -0.5 to +0.5". The narrower this range is, and the higher the numerical values ​​that match within that range, the higher the synchronization rate can be said to be.

[0054] For other blood vessels, the "part corresponding to the heart" mentioned above can be excluded, and the central density plotted from the pulmonary hilum may be used. Peripheral blood vessels can be treated similarly.

[0055] Furthermore, the present invention can also be applied to the circulatory system. For example, changes in cardiac density are directly related to changes in blood flow density from the hilum to the peripheral lung fields, and a series of cardiac density changes and hilar density changes are propagated directly after undergoing a kind of transformation. This is thought to occur with a slight phase difference from the relationship between cardiac density changes and hilar density changes. Also, since changes in density in the hilum and other areas are directly related to changes in blood flow density in the lung fields, it is possible to express the synchronization as a direct reflection of the rate (a relationship of agreement between y and x). Similarly, in the cervical vascular system, changes in density plotted in nearby central cardiovascular vessels are thought to be directly related or related with a slight phase difference. And, assuming that this density fluctuates according to the background and propagates, the manner of the density change is transmitted, and this can be considered as a synchronization agreement rate.

[0056] Here, if we try to display the change amount and the rate of change in a single image as relative values ​​(Standard Differential Signal Density / Intensity) with the change amount set to 1 based on the heart's density, we can visualize the change amount and rate of change for the following: (1) the difference image for each image, with each image set to 1 (normal assumption); (2) the ratio when the sum of the densities (change amount and rate of change) of each difference image for each heartbeat is set to 1; and (3) the ratio when the total density for each heartbeat in multiple scans is set to 1.

[0057] Furthermore, in the case of 3D imaging such as MRI, the sum of heart rate intensity (in the case of MRI) and density (in the case of CT) (with this value set to 1), and the difference between these intensity and density values, can be converted into "peak flow volume data" for heart rate (both at rest and under load). By calculating the ratio of this value to that intensity and density, it is possible to convert the actual measured cardiac activity and activity rate, at least when calculating "3D × time" in MRI and CT. Similarly, by inputting stroke output, it is possible to present an estimated value that converts the distribution of the "capillary phase" in the "flow" of the lung field into the distribution and volume of peripheral pulmonary blood flow.

[0058] In other words, (average of density changes over a rough range) ≈ (heart rate) ≈ (changes in the heart) ≈ (electrocardiogram) ≈ (changes in the surface area and volume of the heart), and if we take only the change amount of 10% or 20% from a single image, we can estimate the value by calculating (total number of images) × (change amount over that time).

[0059] Then, by imaging the original wave shape, the agreement rate with Hz, and other discrepancies with the data of the new target to be measured, images that can assist in diagnosis are calculated. For example, the difference between normal swallowing and a patient's swallowing can be visualized, showing the difference between actions performed in the past and actions performed now. For example, this could include changes and differences in foot movement when walking or swinging.

[0060] The extracted changes are visualized and depicted as images. This is the cardiac function analysis and vascular (blood flow) analysis described below. The rate of change in the myocardium is then visualized. At this time, artifacts to the results are removed again, and function may be extracted from the new data extraction waveform, the initial base data waveform, waveforms from other modalities, surrounding data, and multiple waveforms. The method for removing artifacts will be described later.

[0061] Furthermore, it is sometimes possible to grasp the characteristics even after excluding the altered components extracted from sources other than those mentioned above. For example, when understanding the movement of the abdominal intestines, the effects of respiration and blood vessels are excluded from the abdominal area to extract the movement of the abdominal intestines.

[0062] Furthermore, based on the rate of change obtained through extraction, corrections are applied to images that require a certain amount of acquisition time (such as CT, MRI, special radiographs, and PET / scintigraphy) to provide clearer and more accurate images. For example, it is effective for ascending aortic cardiac correction, cardiac morphology correction, bronchial blur correction, evaluation of the thoracic region, and acquisition in situations where breath-holding is not possible (patient acquisition may take several minutes).

[0063] Furthermore, a basic waveform can be obtained by dividing multiple waveforms into one period and superimposing the data. The bandwidth (frequency range) of this basic waveform, the weighting applied to the basic waveform, the mean and variance of the basic waveform, etc., are used to obtain a master waveform component. This master waveform component is then applied to the waveforms obtained from the original images by matching the weighting and bandwidth components of the waveforms obtained from each original image to the waveforms obtained from the original images. By applying the weighting and bandwidth components of logarithmic transformation to the waveforms of each transformed image, it can be used in calculations during image transformations such as logarithmic transformation. In other words, a basic waveform is generated by superimposing multiple original waveforms obtained from each image, and a master waveform is generated based on the basic waveform. At the same time, by setting the bandwidth or applying weighting to the original waveforms, waveforms corresponding to the organs in each image are generated. This makes it possible to contribute to diagnostic support. In addition, the width of the synchronization rate, the difference for synchronization, and the ratio become clearer, and a constant bandwidth can be measured. By forming a band in this way and simulating by changing the phase, abnormalities can be extracted. By measuring the density at the peak of a phase cycle in the target image, the pixel value at the peak can be calculated. Using the peak as a base, it becomes easier to identify areas where the phase difference is clearly evident. If the phase shifts within a cycle, areas with large phase shifts can be extracted. Areas with large shifts can also be extracted for other components with shifted periods, and these can be displayed as anomalies. When an anomaly is present, both the waveform and amplitude will be shifted, making it easy to clearly identify. Furthermore, it can be used to improve the correlation with "Waveform" and to correct (reconstruct) it. For example, in the case of pulmonary blood flow, it takes 0.016 seconds from the onset of the heartbeat to reach the pulmonary blood vessels, meaning the peak shifts from the onset of the heartbeat. In some individuals, it may take 0.024 seconds. Therefore, it is possible to obtain the waveform of pulmonary blood flow from the heartbeat. The peaks of pulmonary blood flow and peripheral blood vessels can be obtained from changes in density. Furthermore, when displaying the phase shift of pulmonary blood flow, the phase shift relative to the central nervous system is sometimes fixed at 2 frames, but in this method, blood flow that appears with a delay is not visible.The present invention makes it possible to display these blood flows. However, the present invention is not limited to blood flow. That is, the above method can also be applied to other organs, respiration, and other processes.

[0064] Regarding the method for finding the synchronization match rate, simulations are performed for all phases, including phases that deviate from a certain threshold, not just transformations that observe a certain phase shift. By searching for match rate data and images, abnormal lesions are extracted. Similarly, for amplitude, simulations are performed for different states to produce images that closely match the "waveform." This notifies the detection of abnormalities and allows for identification based on different match rates. When capturing moving phases, it is desirable to use a wide frequency range. For example, capturing in the range of 60-80 Hz. In the so-called "Doppler effect," the frequency increases or decreases, which can result in a change in phase. To capture these, it is necessary to widen the range.

[0065] Furthermore, it is possible to perform "tune imaging" of the relative lung field space based on the structure of the organ and the proportion of its space. This is because, with current CT scanners, even large ones such as 320-slice scanners cannot image the entire lung field at once. In partial imaging, during movement, spaces that were previously outside the imaging range at the end of inspiration may appear in the image at the end of expiration, and vice versa. This means that what could only be measured from the overall lung field space can now be similarly tuned using relative position, relative "volume," and air fluctuations. In addition, while "reconstruction" of CT and MRI images was previously performed by eliminating gaps in the difference of "CT data," by performing "reconstruction" on organs that move in accordance with changes in the original "basic data (based on waveform)," it becomes possible to collect clearer difference data and difference images for more periodic movements.

[0066] [Regarding phase shift] It is known that abnormal organs (organs that are not functioning normally) respond more slowly to the histogram of normal organs. For example, in the heart or lungs, the movement of abnormal parts lags behind the movement of other parts. Therefore, by displaying how much the frequency deviates from the normal frequency, it is possible to visualize the parts of abnormal organs. In the image, the abnormal parts will appear to move slowly and have higher signal values. Also, the initial movement is always delayed, and when graphed, a shift in the graph will be noticeable. In this way, since abnormal organs lag behind normal organs in both phase and amplitude, the difference (gap) between them is calculated. That is, it displays how much the phase of the pixel value change changes relative to the change in normal pixel values. In addition, the frequency may be the same, but it may appear as a different waveform. In this specification, these relationships are referred to as "phase relative imaging" or "phase differential imaging". Furthermore, as shown in Figure 17B, the concepts of phase difference and phase shift refer not only to the phase difference PH1 in the same wave, but also to the phase difference PH2 in multiple different waves. In this specification, however, PH2 refers to the phase difference PH2 in multiple different waves.

[0067] For example, by dividing the shape of the lung into eight sections and plotting the average value (intensity) of the pixel values ​​in each area, it was found that in a normal lung, the phase of motion perfectly matched, but in an abnormal lung, the phase of motion was shifted. This phase shift becomes a numerical indicator for determining abnormality and normality. Specifically, phenomena such as "shifting of the bright lines in the image" and "a shift in phase from the starting point" are observed. Furthermore, when the average value of the pixel values ​​in a certain area was plotted, the phase shift was confirmed.

[0068] In the present invention, differences in density are displayed as phase differences. In the present invention, the phase difference represents the degree of movement. Areas with small movement show small changes, while areas with large movement show large changes, which appear in the image. In other words, a phase difference appears as a waveform, but it appears in the image with a time delay. In particular, the start is delayed, and the delay gradually increases. For example, in the case of an abnormal lung, the upper lung field follows well, but the delay becomes more pronounced as you move to the middle and lower lung fields. There are also areas that do not move at all.

[0069] Waveforms obtained from organs are composed of multiple waves. Phase differences are extracted from these waveforms and visualized. In other words, instead of focusing only on the locations where there is a shift, attention is paid to the temporal delay or shift. For example, in the lungs of a patient with pulmonary thromboembolism, blood flow is not uniform, and waveforms with phase shifts appear. There is a temporal shift, and the blood flow is slower in the shifted areas. Also, the shift becomes larger as you move from the center to the periphery of the lung. In this invention, it is possible to display images showing phase differences using only Dycom data. With conventional technology, it is difficult to show phase differences using only X-ray images. In this invention, information hidden in X-ray images is visualized, enabling diagnostic support.

[0070] Furthermore, the following methods can be used to adjust the phase to a standard value, such as the average value. For example, the effect of holding one's breath during breathing is not significant, but breathing involves a mixture of voluntary and involuntary movements. Therefore, as shown in Figure 18, some phases unrelated to normal breathing (normal wave continuity) may be mixed in (in the example above, this is the exhalation state, and in Figure 18, the central plateau region is mixed in). In such cases, certain waveform components that deviate from the average phase can be removed, for example, by cutting out parts that deviate from the normal trend within a certain range of waveform element components (shin values), or by removing phases that do not pass through a certain value (for example, in the upper and lower waves, breathing that does not reach the upper or lower 50% range of related amplitude values, or waves where the phase of the inspiratory value deviates from the average range of 90% to 110% from the calculation, or by partially fitting and correcting to a standard value). This allows for the extraction of waveform images that guarantee a certain level of clinically required quality of dynamic findings of breathing and blood flow. Furthermore, the AI ​​can be trained on changes in the values ​​of adjacent pixels, as well as their variance.

[0071] [Regarding differences in density in organ imaging] The present invention tracks the phase shift in the waveform of an organ and displays the difference in density in the image. For example, in the case of the lung, the phase shift of a part of the lung field, rather than the entire lung field, is tracked and the difference in density is calculated. That is, the relative density difference of a part of the lung is calculated. In this case, the Voronoi tessellation method can be used to enlarge to a certain area. The present invention maintains the relative positional relationship and compares the density before and after movement, outputting the difference. For example, it is displayed at the location of each branch of a blood vessel (such as a blood vessel branching from an artery). Then, the change in density at a certain location is filtered and the difference is displayed. Rather than the relative position of the entire lung field, organs (parts, components) or organs that tend to appear radially (such as the front and middle lobes of both lobes of the lung) are tracked.

[0072] In other words, since the movement of each blood vessel can be tracked, the density of each blood vessel is filtered and the difference is displayed as is. Since the movement of structures such as organs can be tracked, the shape of the lung can be formed by fleshing out the tracking results. Since the change in density is displayed at the location of each organ, the relative positional relationship is irrelevant. Focusing on "how to track the lung field," the process involves tracking a part of the lung field and then expanding the region using Voronoi tessellation. While it is possible to determine the overall shape of the lung field and track its movement, if a core (point) of a certain block region is determined, the overall shape can be estimated from there. Instead of defining a frame and tracking it, it is possible to move the lung field by defining points and tracking the movement of those points. Note that tracking all points of an organ in an image may not be practical due to the amount of computation required, so it is preferable to define points within the image of an organ such as the lung and track those points. Then, the region that each point is responsible for is defined.

[0073] Alternatively, the density can be calculated by removing unwanted frequencies with a filter. At locations where the density changes, the position may or may not shift. In other words, it is not a relative positional relationship. In the present invention, the waveform of organs can be separated and understood using software. This can be applied not only to camera images but also to biological information obtained from chest bands, electrocardiograms, etc. Furthermore, biological information may contain noise. When noise enters from the outside, the amplitude suddenly increases, and a spike-like waveform appears. This can be removed. It is also possible to remove noise that enters periodically. Regarding the method of outputting the image, it is also possible to keep the shape of the lung constant and display the changes in pixel values ​​within that shape as a color image. This method makes it possible to display blood flow as a color image.

[0074] [Regarding the removal of specific frequencies] It is possible to remove known frequencies using filters. For example, frequencies appearing in an electromyogram (EMG) can be filtered out. Frequencies of 50Hz-60Hz appear in the upper arm, 100-200Hz in the thigh and abdomen, and 0.2Hz in respiration. These frequencies can then be removed using filters. In this way, it is possible to remove noise frequencies contained in an ECG, allowing only the cardiac components to be extracted. Thus, there are methods for extracting the cardiac and heartbeat components, and conversely, methods for removing them. In other words, it is possible to extract necessary frequencies or remove unnecessary frequencies using the natural vibration frequencies inherent in organs. Such techniques can be applied to pulse monitors, chest bands, cameras, ECGs, and other devices.

[0075] For noise reduction and separation, FFT processing and bandpass filtering are performed. Furthermore, since pulse rate can be represented by frequency and frequencies can be extracted from chest bands, they can also be removed or extracted from the spectrum. For example, it is possible to either extract respiratory elements contained in lung and heart images as respiration or remove them as noise.

[0076] [Regarding integration with AI] By training AI (Artificial Intelligence) with normal and abnormal images, it can be used to support diagnosis. For example, the thickness of the lungs is thinner on the outside and thicker on the inside. By taking the difference in breathing from a spirograph, it is possible to determine high and low levels. Normal lungs have the characteristics of being "thin at the periphery and thicker on the inside," and "the slope is small at the beginning of movement and gradually increases." When this is visualized, the colors gradually change as the movement progresses. Here, the slope of lung expansion is represented by differentiating the lung volume with respect to time. For example, areas shown in red indicate areas where density is rapidly increasing, which in the case of the lungs indicates rapid contraction. Conversely, areas shown in blue indicate areas where density is rapidly decreasing, which in the case of the lungs indicates rapid expansion. In cases of mild abnormalities, the lungs do not move properly, resulting in areas that are neither red nor blue. Also, the red and blue may be reversed. In all or part of the lungs, there is a movement where the person exhales when they should inhale and inhales when they should exhale. While changes are constant under normal circumstances, the way changes occur can be reversed or uneven. We can train the AI ​​to recognize these different patterns of change. Then, we can have the AI ​​output features corresponding to these patterns. By associating these features with diseases and cases, we can support diagnosis.

[0077] In the process of creating a model through AI learning and making judgments using that model, the step of collecting information and creating a model involves dividing the waveform into phases and capturing the parts where the waveform rises and falls. For example, the model learns the phases of normal exhalation and inhalation. That is, it first creates a normal model, and then learns abnormal lungs that do not match normal lungs. Next, in the step of applying the model to phenomena, the model is used to make judgments, and an output indicating what percentage is abnormal can be obtained. For example, it can display phenomena such as the trunk or branches of blood vessels suddenly narrowing or disappearing, blood vessels that are narrow suddenly becoming denser and visible later, phase shifts, defects, and sparse images. If these phenomena are recognized, pulmonary thromboembolism (PTE) can be diagnosed. Here, PTE is used as an example, but this is only a part of the pathology, and similar phenomena can occur in various other diseases. For example, it is possible to understand the pathology by observing decreases or increases in signal intensity, variability in responsiveness (sensitivity to signal changes), and signal defects.

[0078] A characteristic of PTE patients is that, as shown in Figures 12 and 13, the blood flow in the lungs is not uniform but staggered; in other words, a phase shift occurs. In the present invention, the basis is that, morphologically, the high signal areas are usually radial, and in the next phase, the whole becomes visible. When a phase shift occurs, it can be determined that it is an abnormality caused by a lesion. AI can determine these phenomena. A key feature of such combinations with AI is the tracking of periodically moving objects.

[0079] "Regarding artifact removal" Artifacts caused by the ribs need to be captured and removed. In the lungs, the structures of vascular branches and leaves are visible from the hilum to the periphery. These are almost uniform, symmetrical, and spread throughout. When visualized, the vascular branches and leaves appear, and the spaces between the branches and leaves are displayed in red. This is because changes in density are extracted as frequency signals and displayed. Artifacts tend to appear around the diaphragm and heart. Artifacts also tend to appear on the upper inner side of the lungs because the ascending aorta is visible. The AI ​​is made to model the movement of the lungs one cycle at a time, learns the abnormalities, and then removes the artifacts.

[0080] The inventors have devised a method for removing artifacts caused by ribs. It is necessary to distinguish between artifacts caused by ribs and respiratory changes, and the following techniques are used individually or in combination to achieve artifact removal. These techniques can be applied to fluoroscopic images, difference images of fluoroscopic images, or images after filtering. As a secondary effect, it is possible to identify the position of the ribs, thus not only removing the ribs but also visualizing how the ribs move.

[0081] Block intensity, which indicates the pixel value of a minute region, is calculated for each frame. However, in blocks where ribs are present in a minute region only in specific frames, the absolute value of the change in this block intensity becomes relatively larger compared to the respiratory change. By considering this relatively large change as a rib-related artifact and applying a correction, it becomes possible to identify and remove artifacts caused by ribs. Examples of correction methods include: "performing clamping by defining maximum and minimum values ​​(constraining the numerical value x to a predetermined range [a,b])", "determining whether an artifact is present based on whether a predetermined threshold is exceeded", "if a predetermined threshold is exceeded, shrinking the image so that the maximum change falls within the threshold", and "treating an artifact if it deviates by a certain percentage from the reference wave form."

[0082] As described above, the source of artifacts is that in any given block, there are frames in which ribs are included in a small area and frames in which they are not. Therefore, artifacts can be reduced by changing the shape of any given block so that ribs are included in all frames or not. Examples of logic for changing the block shape include "increasing the block size," "making the block a rectangle about twice the thickness of the ribs," and "excluding areas with outlier pixel values ​​in the block."

[0083] The aforementioned method has the disadvantage that increasing the block size reduces resolution. By taking the difference between the low-resolution difference image shown in Figure 29A (an image in which changes in brightness values ​​are aggregated with a larger block size) and the high-resolution difference image shown in Figure 29B (an image in which changes in brightness values ​​are aggregated with a smaller block size), it is possible to extract blocks that show larger changes compared to surrounding blocks in the high-resolution difference image. This method makes it possible to reduce artifacts while maintaining resolution. Furthermore, since the difference between the low-resolution difference image and the high-resolution difference image can be repurposed as a parameter to characterize the movement of lung structures, it was suggested that this could potentially be used as a parameter with some clinical significance.

[0084] Another method for removing artifacts involves extracting high-density areas from outside the lung contour and identifying them as bone. This method involves measuring density from outside the lung contour, performing positional calculations, and identifying ribs as artifacts. In other words, while the depiction of ribs can be problematic, the original image is an X-ray image, and it can be estimated that areas with high density are ribs. Therefore, it is thought that the rib area can be removed by performing a masking process and canceling out the pixel values ​​by overlaying them. After removing the ribs, it is desirable to perform interpolation or difference processing.

[0085] [AI-based learning and assessment of blood flow in the lungs] In X-ray images, the transparency of the lungs is high during inhalation because air enters and the density of the lung structure decreases, making the lungs appear white in the image. Conversely, during exhalation, the amount of air decreases and the density of the lung structure increases, resulting in low transparency and making the lungs appear black (dark) in the image. Similarly, in blood vessels, X-ray transparency is low during diastole because there is more blood in the lumen and the vessel diameter is larger. During systole, X-ray transparency is high because there is less blood in the lumen and the vessel diameter is smaller. Thus, the density of lungs and blood vessels fluctuates periodically. In this invention, by tuning this fluctuation period, or frequency, specific regions (such as organs) within the image are visualized.

[0086] For example, to visualize the movement of blood vessels in the lungs, follow these steps: (1) Recognize the frequency of blood flow during imaging. (2) Create a frequency spectral component table for the difference in density before and after each region of the XP video. (3) Extract the frequencies obtained in (1) from the spectral components. (4) The extracted frequency (respiratory frequency) obtained is imaged. (5) Output and display the frequency-tuned image.

[0087] The videos obtained using this procedure exhibit the following characteristics: In healthy lungs, the video radiates outwards from the center. In contrast, in patients (e.g., PTE: pulmonary thromboembolism), visible blood vessels disappear, resulting in a video with unevenness.

[0088] In this invention, for example, a video with 12.5 frames per second is used. That is, 12 images (photographs) can be taken per second. Then, a normal lung is examined from two main perspectives. "The Vascular Structure of the Pulmonary Region" In a healthy lung, the peripheral blood vessels appear branched from the main vessels. They flow uniformly in a radial pattern. The central vessels appear thicker and denser. "Capillary and Peripheral Pulmonary Vascular Flow (Normal Pulmonary Flow)" It's a continuous arrangement of tiny dots, resembling clouds. The reason why the vascular structure of the lungs, including capillaries and peripheral pulmonary vessels, is visible is that blood flow velocity is faster in the pulmonary artery trunk and centrally, while it is slower in the peripheral vessels.

[0089] Figure 19 shows the changes in pulmonary blood flow within one heartbeat in a normal lung. Here, one heartbeat is represented by 12 frames. In a normal lung, blood flow is observed throughout the lung during about 45% of the respiration. In the latter half of a heartbeat, the heart rests, so blood flow is not as observed as in the first half. In contrast, the appearance is different in patients with pulmonary thromboembolism. Figure 20 shows the changes in pulmonary blood flow within one heartbeat in an abnormal lung. Here, one heartbeat is represented by 6 frames. As shown in Figure 20, the vascular structure of the lung does not appear to radiate outwards from the center, but is heterogeneous. The missing areas in the image are abnormal. The vascular structure of the left lung is normal, but the right lung is sparse, and blood flow is considerably less. The blood vessels themselves are not visible. In a normal lung, the vascular structure of the lung should appear neatly radial, but this is not the case in Figure 20. In this way, it is possible to diagnose the vascular structure of the lung. Furthermore, in the case of abnormal lungs, "MIP images" are characterized by gaps in the image, making it difficult to see fine details. There are many voids and low density. Thus, according to the present invention, it is possible to check the vascular structure and, based on the display pattern of the image, to grasp the possibility of pulmonary thromboembolism.

[0090] [Regarding capillary and peripheral pulmonary vascular flow (normal pulmonary flow)] In abnormal lungs, capillary and peripheral pulmonary vascular flow does not appear in the image. This indicates an abnormality. For example, in the peripheral part of the lung, the lung thickness is reduced, but in abnormal lungs, the areas where density increases (areas that appear red in the image) are almost completely absent. In normal lungs, the areas where density increases are distributed in a gradient, but this is absent in abnormal lungs. This is how the difference between normal and abnormal lungs is shown.

[0091] In cases of suspected acute pulmonary embolism (APE), the following characteristics appear in the images: blood flow appears in the peripheral lungs after the respiratory motion; that is, delayed blood flow appears in the image. This is because blood begins to flow into the occluded blood vessel, causing the blood flow to appear in the image with a delay. In "stenosis" (partial occlusion / narrowing), as shown in Figure 20, the blood vessel appears in the latter half of a heartbeat; that is, it becomes visible towards the end of the heartbeat cycle. In "occlusion" (complete occlusion), as shown in Figure 20, no blood flow appears in the image at all. It is not visible even in MIP images. In this way, images of normal lungs and abnormal lungs are used as training subjects for the AI.

[0092] Furthermore, if the flow in the capillaries and peripheral pulmonary vessels is not visualized immediately after the pulmonary artery (in the phase thereafter), a visualization abnormality due to thromboembolism should be suspected. Here, (a) if the flow in the capillaries and peripheral pulmonary vessels is visualized with a delay, it is due to "stenosis," etc., where the vessels are narrowed and visualized in a later phase, and (b) if the flow in the capillaries and peripheral pulmonary vessels is not visualized in any phase, it can be assumed that it is "occlusion," and blood flow is interrupted due to complete occlusion.

[0093] In other words, it is possible to identify defects in the vascular structure and check the flow in the capillaries and peripheral pulmonary vessels of terminal vessels in conjunction with the vascular structure. Furthermore, by checking areas where there is no flow in the early phase (the phase immediately after visualization of the vessels) and areas where there is flow from the delayed phase onward, it is possible to determine whether there is a suspicion of "partial occlusion (stenosis)". In addition, by checking areas where there is no flow in all phases from the early phase to the delayed phase, it is possible to determine whether there is a suspicion of "complete occlusion (occlusion)".

[0094] For example, in APE, blood flow in the right lung may suddenly appear. In other words, capillary and peripheral pulmonary blood flow may appear in the area that should be occluded. This phenomenon occurs because the main blood flow is interrupted and blocked, resulting in a sudden influx of blood into the peripheral blood vessels.

[0095] Figure 21 shows a normal lung and a lung with PAH (pulmonary arterial hypertension, including pulmonary arteriosclerosis). In PAH, there are no major gaps when looking at the lung as a whole, but unevenness appears. The normal vascular structure is configured to extend radially from the center of the lung toward the periphery. Branching is not very noticeable. However, when there are symptoms (abnormalities) of PAH in the lung, the branching of blood vessels becomes "irregular in shape". In other words, when there are abnormalities in the lung due to PAH, the number of radial lines in the vascular structure is small, and the vessels become abruptly narrower in the periphery. That is, the vascular structure is sparse and irregular, and the periphery becomes extremely narrow. The distribution appears uniform, but when viewed closely, the distribution is random and different from normal. Gaps also appear in the vascular structure of the central part of the lung. This is evidence that there is no movement in that area. Furthermore, the rate of blood flow is slow, and the appearance of blood vessels is delayed. For example, the blood vessels appear 1 to 2 frames late, and the vascular structure begins to be seen from around the center of the lung. This is abnormal. Furthermore, the peripheral blood vessels in the lungs are narrow, but blood flow appears with a delayed response. This is evidence of delayed blood flow and indicates a lung abnormality. Additionally, the vascular structures themselves may exhibit flickering in the image or appear with a delayed response. Also, the absence of flow in capillaries and peripheral pulmonary vessels, or their inability to be seen or only partially visible, is abnormal. These phenomena are visualized as differences from a normal image.

[0096] Here, we will explain the mechanism by which signals from pulmonary blood vessels are visualized. In normal pulmonary blood vessels, first, the heart's movement creates a pressure difference in the vessels. Next, the elasticity of the vessels causes expansion and contraction. In other words, the elastic force of the vessels causes them to expand and contract. As a result, there are fluctuations in the diameter of the vessels. That is, in normal cases, there are variations in the size of the lumen of the vessels. Then, the signal is recognized as a periodic change in the density of the X-rays and reflected in the image. Next, in cases of arteriosclerosis such as PAH, even when the heart is moving, it becomes difficult to create a pressure difference in the pulmonary blood vessels. Also, expansion and contraction due to the elasticity of the vessels becomes less likely. This is because arteriosclerosis (fibrosis) reduces the elastic force due to thickening of the lumen. As a result, the force with which the pressure difference from the heart is transmitted to the peripheral blood vessels decreases. Also, it becomes less likely to be reflected in the diameter of the vessels. In other words, even with the heart moving, the changes in the size of the vessels become weaker. As a result, it is displayed in the image as various abnormal factors as a periodic change in the density of the X-rays. For example, an overall decrease in signal value (pixel value) can be seen. Furthermore, the reduced pressure difference and decreased elasticity of blood vessels lead to delayed pulmonary artery ejection and an overall weaker ejection pattern. The reduced elasticity of blood vessels also decreases capillary and peripheral pulmonary vessel flow from the central to peripheral parts of the lung, resulting in a more fragmented vascular phase pattern. The weakened variation in vessel diameter makes visualization of peripheral vessels and even smaller vessels difficult. Additionally, central vessels exhibit a club-like or branched shape, and the "diffuse" pattern, particularly in the periphery, is diminished.

[0097] Based on the above, it can be said that when the elasticity of blood vessels is lost due to changes in vascular hardening (elasticity), it becomes more difficult to visualize blood flow. Also, if the internal density does not change easily, the signal (pixel) will not be visible. In addition to PAH as described above, similar image patterns can be predicted to appear in chronic heart failure, hyperparathyroidism, chronic renal failure, collagen diseases such as amyloidosis, and other pulmonary artery sclerotic changes.

[0098] [Regarding ventilation-blood flow mismatch] Traditionally, ventilation-blood flow mismatches in the lungs have been diagnosed using pulmonary perfusion scintigraphy. The relationship between these two is physiologically understood. In this invention, ventilation and blood flow are displayed simultaneously on a screen, and the diagnosis is made based on the combination of respiration and blood flow. For example, if lung movement is generally normal but pulmonary blood flow does not correspond, it can be judged as abnormal. For example, if ventilation is normal but blood flow is abnormal, it could be pulmonary hypertension or pulmonary thromboembolism; if both are poor and blood flow is also poor, it could be emphysema. On the other hand, there are cases where blood vessel movement is normal but lung movement is abnormal. For example, this applies to cases of heart failure where breathing is abnormal. Even in healthy people, if the heart rate is normal but they experience shortness of breath, for example, if food gets stuck in the throat, this could also apply. AI-based machine learning is used to generate a model of such "ventilation-blood flow mismatch".

[0099] For example, in cases of hypoxic lung disease (HLD), capillary and peripheral pulmonary vascular flow is impaired. In other words, there is a deficit in lung respiration, and blood flow in the corresponding area is also poor. The same phenomenon is observed in COPD. This invention enables diagnosis by having AI learn by correlating and comparing both ventilation and blood flow.

[0100] [Regarding correlation with lung scintigraphy] Figure 29D shows example images of MBDP (Maximum Blood-flow-related Differential Projection) and ABDP (Accumulated Blood-flow-related Differential Projection). MBDP projects the largest value between frames for each block onto a single image, while ABDP projects the sum of positive changes for each block onto a single image. Furthermore, as shown in Figure 29E, MBDP and ABDP are intended to be used as clinical indicators by dividing lung images into six areas and calculating the mean and distribution of intensity values ​​for each area.

[0101] For example, it is possible to correlate with lung scintigraphy (an existing method for evaluating pulmonary blood flow). Lung scintigraphy measures the degree of blood accumulation by injecting a radioactive reagent and taking the "count of radioactive material," and is characterized by a higher signal in areas where blood is clogged. Similarly, in lung scintigraphy, the lung image is divided into six areas, and the count of radioactive material is taken in each area. In lung scintigraphy, since the evaluation is relative, for example, if the counts are "100, 200, 300, 400, 500, 600," the percentages relative to the total count of 2100 are calculated as "4.7%, 9.5%, 14.3%, 19%, 23.9%, 28.6%." In MBDP and ABDP, it is also possible to replace the sum of the intensity of each area with the counts from lung scintigraphy and evaluate it as a percentage of the total intensity. This makes it possible to obtain images similar to those obtained with scintigraphy. For example, when counting respiration and blood flow, it's possible to count signals indicating whether respiration or blood flow is slow or fast, or whether there is actual flow at all. Since the total flow can be understood, it contributes to diagnostic support.

[0102] [Regarding reproducible events] In this invention, the AI ​​is trained using machine learning on events that occur repeatedly, i.e., reproducible events. This makes it possible to employ methods such as learning normal states to determine abnormalities, or learning abnormal states to output candidate cases. The output method for AI determination involves outputting feature quantities for each case. These feature quantities can be determined based on image patterns (such as the average pixel value for each region or the rate of change in pixel values) and can include multiple diseases.

[0103] [Regarding pulmonary blood flow and AI] Figure 30A shows the pulmonary blood flow during one heartbeat, frame by frame. Figure 30A shows the pulmonary blood flow during one heartbeat in 10 frames. In the flow of pulmonary blood during one heartbeat, blood begins to flow from the central blood vessels of the lungs and then spreads out to the peripheral terminal vessels. Terminal vessels are narrow in diameter (small lumen), flow at a slower speed, and are numerous, so in the image they appear to spread out like a cloud after the central area. In other words, the blood flow of the main pulmonary artery and distal pulmonary artery is visible first, and then the terminal vessels become visible.

[0104] Figure 30B shows an example of measurement results for MPA (Main Pulmonary Artery), DPA (Distal Pulmonary Artery), and DPV (Distal Pulmonary Vein). DPA is observed slightly after MPA, but then drops sharply from MPA. Also, as mentioned above, TV (Terminal Vessels) appears before DPV. In Figure 30A, as shown in the measurement results in Figure 30B, blood vessels are visible from the central to the peripheral regions in the first two and three frames, and then small vessels appear from the fourth, fifth, and sixth frames.

[0105] To train the AI ​​on this, we use a "bullseye" model that represents the lungs and heart in concentric circles. Blood vessels generally spread out in concentric circles from the hilum of the lung, but there is a phase where the pixel values ​​of blood vessels are at their peak (the third and fourth frames in Figure 30A). Here, both vascular structures and peripheral blood vessels are included. Therefore, in order to grasp the signal (pixel value) of pulmonary blood flow, we first detect the peak of the pixel value. Then, since there is a phase just before that peak where vascular structures are visible, we extract this. In this detection method, there are two methods: (1) "extracting vascular structures" and (2) "extracting at or just before the hilum of the lung, since pulmonary blood flow flows from the hilum to the periphery."

[0106] For example, in the lungs of patients with acute pulmonary embolism (APE), the vascular structure becomes sparse, so the lung field is divided into approximately 17 equal parts using the concentric circles of a bullseye model. By distributing the shape of the lungs in a concentric pattern, it is possible to see how it changes during the phases when blood flow is present and the phase of blood flow in a terminal vessel. Displaying the variability in the distribution makes it easier to apply AI. This is because the variability in the distribution makes it easier to distinguish between normal and abnormal areas. In other words, the area is divided into airspace-like areas, and the pixel values ​​of areas with and without blood flow are quantified. This can be calculated automatically. Because it can be quantified, it is possible to apply AI. Using this method, calculations can be performed according to the segmentation, and areas with defects can be detected.

[0107] For terminal vessels, subtracting the pixel values ​​of the vascular structure area from the overall pixel values ​​yields the pixel values ​​of the terminal vessels. In other words, excluding the vascular structure, the remaining area becomes the terminal vessels, and it is possible to show what kind of blood flow occurs in the terminal vessels.

[0108] Furthermore, it is possible to detect which parts of the three-dimensional structure are missing (whether or not there is blood flow). The same display is possible when the model is in 3D or 4D. In other words, the vascular structure is divided into areas, and the area before the part with the highest pixel value (signal) of the vascular structure is detected, and the phase at that point is determined. Using this method, the vascular structure can be detected, terminal vessels can be detected, and the blood flow can be displayed. When these signals are collected, a bar graph that rises and falls is shown, representing the movement of the blood vessels within the area. It can also represent the movement of the terminal vessels within the area. The overall flow can also be represented. When these are calculated within a single heartbeat, the waveform shown in Figure 30B is obtained.

[0109] Figures 30C and 30D show signal intensity in ultrasound images of the atrium. In Figure 30C, a peak signal E and a signal A, which indicates the effect of atrial movement, are observed. In Figure 30D, a residual signal L appears between signals E and A. When L appears between the peak signal E and the signal A, which indicates the effect of atrial movement, it is considered a possible sign of heart failure. In this way, by graphing and quantifying the data, it becomes possible to predict lesions by understanding the difference compared to normal organs.

[0110] Furthermore, a method of comparing past and present electrocardiogram waveforms is known. For example, by overlaying waves shown in red, blue, and green, automated diagnosis becomes possible by comparing pre-treatment and post-treatment data, comparing patients with normal individuals, and comparing the same patient's data in the past. For instance, by identifying discrepancies such as a shifted T wave or an unstable rise after the peak, automated diagnosis is possible, and the physician can even write down their findings.

[0111] [AI-based learning and judgment regarding lung respiration] Regarding lung respiration, the synchronization of changes within the lung field is tracked. That is, the degree to which lung movement expands from the central to the peripheral regions is judged to be correct. Specifically, the coordinated movement of the diaphragm, ribs, and lung tissue is observed. In addition to these, the movement of the pectoralis major muscle may also be taken into consideration. Furthermore, the degree of gaps can also be evaluated in the image. Moreover, even in normal lungs, the appearance may differ depending on whether it is spontaneous breathing or labored breathing, but these are judged as normal lungs. Furthermore, "coordination" can be judged based on "the area over which changes in lung movement propagate," "the continuity and flow of blood," "the continuity of movement from the central to the peripheral regions of the lungs," and "the movement of the diaphragm itself."

[0112] More specifically, a single wave extracted from the image (for example, a respiratory wave) is used as a reference, and the correlation with it is evaluated. Lung movement is condensed into a wave, and the amplitude, phase shift, and vector direction are checked, and the AI ​​makes a judgment in conjunction with anatomical insights. In this case, the object of judgment changes depending on which wave is extracted from the image. That is, since there are disease (case) specific frequencies and phase shifts, the disease (case) is identified based on the detected frequency and phase shifts. Where the lung is fixed due to tuberculosis, the frequency is zero. For example, in cases where neurotransmitters are difficult to process, in myocardial damage, the phase is shifted, and the waveform is different due to a delay in the web form. Since disease-specific frequencies exist, disease-specific frequency-synchronized images can be output. This makes it possible to obtain new features or output new images.

[0113] In this invention, imaging is performed in accordance with the respiratory frequency. In other words, by extracting images that match the respiratory frequency, an image of lung movement due to respiration becomes visible. Here, the differential value of lung movement is imaged. Such images extract only the changes in the X-ray image that are in line with the respiratory frequency. Since changes in density are tracked, white areas in XP (decreased transmittance) are due to increased density (green, yellow, red), and dark areas in XP (increased transmittance) are due to decreased density (green, light blue, blue). These changes are visualized in the image. Note that the images in this invention are the reverse of the inspiratory / expiratory graphs used in precise pulmonary function tests.

[0114] The signal changes during lung movement: at the beginning of respiration, signal values ​​increase from the trachea / bronchi (central), bronchioles, and alveoli (peripheral), while towards the end of respiration, signal values ​​decrease from the alveoli (peripheral), bronchioles, and trachea / bronchi (central). This is true for both inspiration and expiration. The same applies to X-rays (X-rays); in inspiration, increased permeability corresponds to decreased density, and the difference is a negative vector (blue in color). Conversely, in expiration, decreased permeability corresponds to increased density, and the difference is a positive vector (red in color).

[0115] Figure 22 shows an image of a normal lung. As the inspiratory movement (higher permeability) becomes stronger, the color changes from light blue to blue (stronger negative density vector). Conversely, as the expiratory movement (lower permeability) becomes stronger, the color changes from yellow to red (stronger positive density vector). The thinner the outer part of the lung, the weaker the overall signal, while the thicker the inner part of the lung or towards the diaphragm, the stronger the overall signal. The strength of the signal increases or decreases according to the wave-like movement of inhalation and exhalation during respiration.

[0116] [Regarding the correspondence between images and diseases] Figure 23 shows a normal lung and a lung with COPD. In a normal lung, the lung expands from the bronchi to the collateral bronchi, bronchioles, and alveoli, but in a COPD lung, it is observed that the lung only expands from the bronchi to the collateral bronchi, and does not expand to the bronchioles and alveolar terminals. Thus, in a COPD lung, the central part of the lung appears as an uneven and patchy image. Asymmetry between the left and right sides also contributes to the abnormality detection. Coordination is also a factor. The movements of the diaphragm, ribs, and lung tissue are disjointed. In this invention, the coordination between a reference wave and each structure is evaluated in relation to the anatomical structure.

[0117] Figure 24 shows the three-dimensional structure of the lung apex-diaphragm side. As shown in Figure 24, "Lung apex In the "upper diaphragm" region, the lung structure is thin in three dimensions. Therefore, in the images according to this invention, the changes themselves are small. Also, there is less poor visualization due to overlapping lesions. In the "upper diaphragm" region, the lung structure is thick in three dimensions. Therefore, in the images according to this invention, changes appear strongly. Also, there is more poor visualization due to overlapping lesions. In the "slightly caudal to the lung apex (below the lung apex)" region, the changes in the images according to this invention are moderately strong. Overlap of lesions is also moderate. For example, due to the lung structure, there is a tendency for left-right differences in thickness to occur between individuals. In the "lower diaphragm" region, the soft tissue shadows are strong, making it difficult to quantify the differences between individual XP images, and evaluation is not easy.

[0118] Therefore, even in healthy lungs, the following events may occur. • Decrease in the rate of change in the peripheral lung fields • Decreased lung structure around the interlobar cleft • Non-uniformity of changes due to the movement of structures near the lung hilum. • Mild signal reduction in the medial right lower lung field • Changes in the thickness of the apical part of the lung due to the thickness of the thoracic cage in the anterior and posterior directions, and the pectoralis major muscle, etc. • Artifact of rib bones • The shadows of only the branches and leaves are like those of a sculpture.

[0119] Figure 25 shows the peripheral lung field. In the peripheral lung field, the lobular structure is three-dimensional and complex, and this is reflected in the image, resulting in a symmetrical peripheral change. The extent of peripheral lung field may be relatively reduced depending on the shape of the lung, the depiction of the ribs, and the depiction of the pectoralis major muscle. Also, there may be differences in the depiction of the lung margins, from the peripheral bronchioles to the alveoli. This also changes depending on the degree and condition of respiration. That is, signal changes corresponding to the respiratory cycle appear uniformly. In addition, changes in thickness, such as thinning from the lung apex towards the diaphragm and from the outside to the inside of the lung, appear according to the respiratory cycle. However, it is generally symmetrical. In this invention, we evaluate whether the movement of the wave as a respiratory cycle is linked to the movement of the lungs. We evaluate whether the thoracic cage, diaphragm, ribs, etc. move in conjunction with the wave as a respiratory cycle.

[0120] A decrease in signal value (pixel value) is observed in the lung structures around the interlobar fissures. This is because, particularly at the periphery of the left lower lung field, alveolar density changes are low, resulting in almost no signal changes. A slight decrease in signal is also observed in the medial right lower lung field. This may be due to negative signals caused by the structure beneath the diaphragm, but corrections are made to bring these within the normal range. In this invention, when determining a normal video, the three-dimensional chest structure and the rate of change in respiratory physiological changes are combined and considered comprehensively during interpretation. In addition to these, by considering the structure and image transformation in various XPs during interpretation, it becomes possible to determine various things such as partial lung function.

[0121] When using AI for machine learning and diagnosis, a reference wave, such as the respiratory cycle, is defined, and the key point of the diagnosis is whether the lung tissue is correlated with this reference wave. In other words, the diagnosis is based on correlation with the wave, specifically whether the fluctuations are large, small, or nonexistent relative to the wave. If fluctuations that should be large are small, or if fluctuations that should be small are large, it can be judged as abnormal. Furthermore, if the phase is shifted relative to the reference wave, it can also be judged as abnormal. In short, lung tissue that exhibits a certain correlation with the wave consisting of the respiratory cycle is considered normal. When this is trained on an AI, it becomes a model of normal lung tissue. AI diagnosis determines whether this correlation is maintained or lost. Abnormal amplitude or phase can also be considered.

[0122] Figure 26 shows shallow and deep breathing in a normal lung. In shallow breathing in a normal lung, the slope of the wave of change is weak and the change is small, so the periphery (from peripheral bronchioles to alveoli) cannot be visualized. In this case, only the bronchi and bronchioles are visualized. On the other hand, in deep breathing in a normal lung, the slope of the wave of change is strong and the change is sufficiently large, so the periphery (from peripheral bronchioles to alveoli) can be visualized. In addition, in deep breathing in an abnormal lung, the movement lags behind the movement of the diaphragm. Thus, in this invention, breathing is evaluated by "coordination," making diagnosis possible based on this coordination.

[0123] Figure 30E shows the movement of respiratory waves. A steeper waveform slope makes the image clearer and easier to see. This is because the shape and slope of the wave are all interconnected. In a normal lung, the diaphragm, thoracic cage, and trachea all exhibit similar signal patterns. Diagnosis becomes possible by graphing how well the waveform matches the surrounding tissue.

[0124] Furthermore, the three-dimensional structure allows us to capture how thick or thin the front and back of the lung are. For example, in chest X-ray images, CT images are represented as cross-sections, but the three-dimensional structure is represented by superimposed images. Figure 30F shows the three-dimensional structure and cross-section based on chest X-ray images. The outer part of the lung is three-dimensionally thin, resulting in less image variation. If there is a lesion, the degree of image rendering problems due to overlap is small. The inner part of the lung is three-dimensionally thick, resulting in greater image variation. If there is a lesion, the degree of image rendering problems due to overlap is large. From the outer to slightly inner (mid-outer layer), the image variation is somewhat large, and if there is a lesion, the degree of image rendering problems due to overlap is also somewhat large.

[0125] Figure 24 shows a side view of the lung. Similarly, when viewed from the side, the thickness from front to back is relevant. It is possible to understand whether the pixel value is increasing or not within the three-dimensional structure. The three-dimensional structure of the lung and whether each structure appears to be expanding can be displayed. In the case of shallow breathing, the lung expansion is small, so the signal is not strong. However, if it is based on a wave, it can be displayed by understanding the slope, which can be useful for diagnosis. Regarding the structure of the trachea itself, it should be easier to see at the beginning of expansion (because the slope of change is large), but if it is not visible, it is considered abnormal, and the possibility of a lesion can be predicted. Also, if it is not visible when it should be visible on the central side of the trachea, it can be judged as abnormal. The amplitude moves along the three-dimensional structure and the shape of the wave. If it does not follow, it can be judged as abnormal. Whether the amplitude is the same or not, if it is not the same, it can be judged as abnormal.

[0126] Figure 30G shows a normal lung and a patient's lung. Figure 30H shows a patient's lung. In these lung images, the "red" areas indicate increased pixel values, representing the exhaling phase of breathing. Both the lung area and volume are decreasing, resulting in positive pixel values. The "blue" areas indicate decreased pixel values, representing the inhaling phase of breathing. Both the lung area and volume are increasing, resulting in negative pixel values. These abnormalities are displayed during movement. In the patient's lung, blue is displayed when red should be displayed, and red is displayed when blue should be displayed. In other words, it is moving in the opposite direction to normal, and an abnormality can be determined. Also, if the diaphragm is moving but there is no change in color, an abnormality can also be determined. For example, if an AI learns from images of normal lung movement and builds a learning model, it can then determine the presence of lesions in images of a patient's lung. It is also possible to quantify the characteristics of images of normal lungs and compare them with the quantified characteristics of images of a patient's lung to determine abnormalities.

[0127] Figure 30I shows a patient's lung. In this case, the central vascular structure of the lung is visible, but the peripheral vascular structure is not. The peripheral alveoli are not moving. Normal lungs show wave correlation, but abnormal lungs do not. AI can learn and make judgments based on this. It can also quantify the differences within a certain area and understand how different they are. Figure 30J shows a comparison between deep and shallow breathing. Shallow breathing results in a faint image, while deep breathing results in a darker image. Segmentation is calculated and graphed according to the wave shape.

[0128] Figure 30K shows how waves from the diaphragm and other structures are shifted relative to the original wave. These shifts are based on phase shifts. For example, in the lungs of patients with interstitial pneumonia, the wave is out of phase with the original wave, as shown in Figure 30K. This phase shift is quantified. The heart is particularly useful for this. If neurotransmitters are delayed, the heart muscle will move with a delay. This delay in heart muscle movement is due to a phase delay. These phase shifts are quantified and represented as pixels. If a delay in the transmission of neurotransmitters (or neurotransmitter dysfunction) is observed, there is a high probability of developing a myocardial infarction one or two years later. In other words, whether the amplitude of the wave is shifted, or whether the phase of the wave is shifted, is quantified or represented as pixels.

[0129] In COPD, the phase of changes in lung pixels is shifted. The movement is misaligned relative to the lung apex, resulting in poor response. Patients with interstitial pneumonia also exhibit a phase shift in changes in lung pixel values. In contrast, healthy individuals do not have this phase shift. It becomes possible to learn and analyze these phase shift patterns using AI. This is particularly useful because these patterns are difficult to perceive with the naked eye; therefore, quantifying them, training the AI, and then using it for analysis is beneficial.

[0130] Figure 30L shows an image of a pneumonia patient. The lung structure appears to be missing. The vascular structure, which normally extends radially from the hilum, is displayed completely randomly. This can be judged as abnormal. Patients with partial pleural effusion also show a random display of vascular structure, as in Figure 30L. In such cases, white spots appear on the X-ray image, allowing the doctor to make a diagnosis in conjunction with this. In other words, it is possible to make a comprehensive judgment using images of respiration alone, images of blood flow alone, images of combinations of these, and X-ray images.

[0131] Figure 30M shows an example of an electrocardiogram (ECG). As shown in Figure 30M, in addition to the pulses that appear at a constant period, signals with shifted periods may also be displayed. This is mainly because muscle movements, lung movements, and rib creaking have entered the ECG data as noise. In other words, the ECG data is shifted due to the influence of other organs and body parts. Conventionally, these shifts have been corrected manually, but the present invention automates this correction process. That is, these data are treated as "frequency-tunable data" and processed as frequency-tunable images. More specifically, by performing a Fourier transform on these data, filtering out only the frequencies of the periodic pulse signals appearing in the ECG, and finally performing an inverse Fourier transform, it is possible to obtain ECG data with noise removed. Respiratory noise can also be removed in the same way.

[0132] [Regarding the lungs in patients with heart failure] Figures 27 and 28 show the lungs of a heart failure patient. Figure 27 shows one aspect of the beginning of a heartbeat, and Figure 28 shows one aspect of the latter part of the first half of a heartbeat. A normal vascular structure has a configuration where blood vessels radiate from the center of the lung toward the periphery, with little branching. However, in patients with heart failure, the branching of pulmonary blood vessels becomes "irregular in shape." That is, when there is an abnormality in the lungs due to heart failure, the number of radial lines in the vascular structure is small, and the vessels become abruptly narrower at the periphery. In other words, the vascular structure is sparse and irregular, and the periphery becomes extremely narrow. The distribution appears uniform, but upon closer inspection, the vascular structure becomes "sparse," and can be described as "diffusely sparse." This is different from normal. While capillary and peripheral pulmonary blood vessel flow can be observed, a delay in blood flow is observed overall. The movement is generally out of sync, and there is a phase lag in the blood flow wave. Furthermore, as shown in Figure 28, blood flow is observed even though it is a time when blood flow should have finished. This indicates that blood flow appears with a delay from the pulmonary hilum. Blood flow is observed during a time when it would not be observed in a normal lung. These phenomena are visualized as differences from normal.

[0133] [About 3D tomography] In three-dimensional image display technology, projected images based on three-dimensional information are displayed on image display devices such as LCDs, which are two-dimensional planes. This three-dimensional image display technology is classified into projection methods (parallel projection, perspective projection) and image display methods (surface rendering, volume rendering, maximum value display, minimum value display, sum value projection, etc.) onto the two-dimensional plane that serves as the image display surface. Furthermore, multi-stage surface reconstruction (MPR), which displays arbitrary cross-sections without using projection methods and evokes a sense of three dimensions in the observer, is also understood to be included in three-dimensional display methods.

[0134] Traditionally, during interventional radiology (IVR), patients have been transported to a CT room to acquire tomographic images, or IVR-CT devices have been used. However, C-arm CT imaging using an angiography system equipped with a C-arm avoids the risks of transport by obtaining tomographic images on-site during IVR, and also offers superior cost performance in terms of cost and operation compared to IVR-CT devices. While 3D angiography is a technique for reconstructing tomographic images from rotational images taken with a C-arm, 3D angiography is primarily used to visualize contrast-enhanced vessels, whereas C-arm CT imaging has excellent low-contrast resolution, making it possible to visualize faint tumor enhancements and other features.

[0135] In C-arm CT imaging, the acquired rotational image undergoes three stages of image processing: conversion to water transmission length, pre-reconstruction processing, and reconstruction processing, before a tomographic image is obtained. In the first stage, conversion to water transmission length, the pixel values ​​of the rotational image are converted to water transmission length. Water transmission length is the amount obtained by converting a given pixel value to how many millimeters of water it would pass through under the same X-ray conditions. Specifically, after detector gain correction, the scattered radiation component is corrected by scattered radiation correction processing, and then converted to water transmission length during beam hardening correction processing. Dark areas in the rotational image indicate high X-ray absorption, which results in a longer converted water transmission length, making the converted image appear as if it has been negative-positive inverted.

[0136] The next pre-reconstruction process involves four types of corrections: C-arm trajectory correction, ring artifact correction, truncation correction, and π correction. C-arm trajectory correction corrects the discrepancy between the actual C-arm trajectory and the theoretically ideal trajectory. Ring artifact correction reduces ring-shaped artifacts in tomographic images that interfere with image observation. Truncation correction improves the flatness of the tomographic image by interpolating the sinogram of the portion of the subject that extends beyond the field of view during rotation. π correction corrects the difference in the amount of data collected between the central and peripheral areas during half-scan reconstruction due to the cone-beam effect. After these corrections, the tomographic image is calculated by the reconstruction process. The algorithm for the reconstruction process is called the FBP (Filtered Back Projection) method.

[0137] In C-arm CT scans, images are taken from various angles over a period of about 30 minutes. For example, since the respiratory cycle is not constant, reconstruction is used to correct it. Here, areas near the CT tube are clearly visible, but areas far away are not. Increasing the voltage makes them visible, but processing is required to remove noise by taking the difference while maintaining the difference in transmittance. For example, by looking only at the area in the foreground, looking at that area from behind, and subtracting the difference, both the front and back become visible. If the voltage is increased, everything becomes visible, but it becomes difficult to understand, so the foreground is pulled back to make the background easier to see, and conversely, the background is pulled back to make the foreground easier to see.

[0138] Conventionally, no method has been realized to correct and display images while the patient is breathing. In this invention, the breathing motion is kept constant. Phase and frequency are matched, and reconstruction is performed. That is, multiple images taken from multiple angles under multiple conditions are processed together, and tomographic images are created by changing multiple conditions. These are then used in X-ray imaging. In other words, tomographic images are created in frequency-analyzed X-ray images. Finally, fine adjustments are made by reconstruction. Tomographic images are created for specific organs at multiple locations and under multiple conditions. For example, for moving lungs and blood flow, the waveform is matched based on phase. Phase matching is also performed for breathing. Then, during visualization, positional alignment and three-dimensionality are created. Blood flow also needs to be matched in time (synchronization). In this case, for example, images are taken every 10 heartbeats, and then the phase is matched.

[0139] In the present invention, since images are taken while the device is moving, the frequency of the organs must be taken into consideration. Therefore, temporal and phase alignment, phase alignment, and angle adjustment are performed. In C-arm CT imaging, the imaging location relative to the organs is not exactly the same, so positional and angle adjustments are necessary. For example, when attempting to image the lungs from the right front at a 45-degree angle, the actual angle may be 43 degrees or 49 degrees. Such deviations need to be finely adjusted based on the positional relationship with the ribs and rib cage.

[0140] In other words, CT images, C-arm CT cross-sectional images, and XP images acquired from various angles are aligned to match the positions and angles of previously acquired "vertebrae, heart / mediastinal structure, ribs, etc." and 3D images of other structures such as bronchi. The order of highest agreement is "vertebrae, heart / mediastinal structure, ribs," but in particular, the coefficient for matching the angle and position of the vertebrae is strengthened, while the coefficient for matching due to lung movement, such as the ribs, is slightly reduced. The angle and positional relationship that best matches each acquired XP image is grasped as an image pattern using AI, etc., and when creating the 3D structure, cross-sections, 3D structures, and 4D structures are created based on this information. In particular, since CT images are "inspiratory images" taken when the subject is inhaling, it is thought that matching based on the origin of the XP image during maximum effort inspiration will yield the best results. Furthermore, when taking X-ray images from multiple angles, if there are no pre-existing images specific to the subject, such as CT or C-arm CT tomographic images, the appropriate angles may be calculated from a standard image model based on information such as age and height. Through fine-tuning, cross-sectional and three-dimensional structures, as well as three-dimensional cross-sectional and three-dimensional structures corresponding to the movement of organs such as respiration and the heart, may be created.

[0141] Furthermore, albumin is injected into the patient to intentionally create a state of microscopic pulmonary thrombosis. In a normal lung, albumin is distributed throughout the lung. In the case of pulmonary thromboembolism, the periphery is blocked, so albumin cannot reach the periphery. This results in uneven imaging. When observed as a single 3D structure, it can be seen that albumin is present in the anterior (front side of the lung) but not in the posterior (back side of the lung). Then, a tomographic image is created from the 3D image. If there is a large phase shift in the image, it indicates that the albumin is not flowing properly, which is an abnormality.

[0142] Furthermore, in AP (anterior-posterior) radiography, (1) due to the proximity to the radiograph tube and the distance from the detector, abnormalities are observed in the lungs and anterior (A) side. (2) Because multiple images are taken, the radiation dose is insufficient (low dose), and the dose does not easily reach the back (P side). For these reasons, in the default imaging for fluoroscopic confirmation, the effect closer to the radiograph tube is stronger (A side in AP imaging, P side in PA imaging), while areas closer to the detector (far from the radiograph tube (P side in AP imaging, A side in PA imaging)) show relative changes, and the signal becomes less identifiable than on the radiograph tube side. Therefore, (1) by taking similar images on areas with a higher dose (resulting in more uniform image quality and images that emphasize the P side) and on conventional low-dose areas, it is possible to differentiate between anterior, overall, and posterior imaging. (2) AP imaging and PA imaging allow for imaging of each respective surface. (3) In areas such as the dorsal side of the lower lobe, the signal may be delayed, making it easier for differences in phase signals to occur. (4) During breathing, images may be more likely to be drawn from the posterior direction compared to breath-holding. (5) Changes are more likely to occur with deep breathing compared to shallow breathing or normal breathing.

[0143] By utilizing the above trends, and by superimposing both (superimposing AP and PA, objectively depicting the overall signal ratio), and by subtracting them, including the above, imaging is performed to emphasize the front (A side) or back (P side) depending on whether "the anterior is dominant or the whole is positive," "the posterior is slightly dominant," or "what is more visible with higher voltage (high voltage > low voltage)." In addition, imaging is performed from multiple directions, including RAO, LAO, and RL directions. Furthermore, by performing imaging while rotating and spiraling the tube, various multi-angle images are obtained. Since respiration is not always constant, by using standardized expansion, reduction, and temporal reconstruction based on a master waveform, etc., it is possible to match signals in terms of position and relative phase by superimposing them on a standard lung or previously acquired 3D tomographic images of the patient, and to visualize approximate respiration and blood flow elements in that cross-section. In addition, through multi-angle transmission imaging, it becomes possible to obtain multi-angle 4D images of 3D x time and signals in each cross-section.

[0144] [Regarding methods for applying pulses to the human body] Arterial spin labeling (ASL) is an imaging method that obtains perfusion images and relative cerebral blood flow images without the use of contrast agents by labeling arterial blood flowing into the imaging plane using RF pulses. RF pulses are impact signals, and when applied periodically, they become markings, indicating how far around the neck the pulse has reached. For example, when an RF pulse is applied to the neck, the pulse diffuses throughout the entire head. While pulse diffusion is normal, there are cases where it does not. Here, it is predicted that frequency synchronization can be recognized and predicted within the brain, allowing for an understanding of whether blood flow is easy or difficult. Since the frequency applied to the human body is clear, deviations or delays from this frequency make it possible to determine whether the situation is normal or abnormal.

[0145] [Regarding representation using sine waves] As shown in Figure 14, the lung image was divided into four sections for analysis, and the following method was used to check for differences in signal changes in each region. Specifically, the "signal change: f(x)" was approximated as "g(x) = A * sin(Cx + B)", and the correlation between each parameter and the FEV% predicted value was investigated.

[0146] As a result, as shown in Figures 15 to 17, in healthy lungs, the amplitude tended to increase in the order "R4→R3→R2→R1" and the phases tended to coincide. In contrast, in COPD cases, variability in both amplitude and phase was visually observed. From the examination of the approximation results to a sine wave, it was found that the phase shift showed a negative correlation (R = -0.775) with the FEV% predicted value. When calculating the correlation, one point was excluded as an outlier, which is thought to be due to the way the lung field (analysis range) was constructed. Furthermore, artifacts from the diaphragm and ribs mainly affected the amplitude, and no tendency to contribute to the phase shift was observed. At least, we believe that it is not necessary to actively exclude R4 / L4 when evaluating the phase shift. On the other hand, it is necessary to be aware that artifacts will have an effect when performing any evaluation using amplitude.

[0147] [Regarding the estimation of pulmonary blood pressure] Traditionally, measuring pulmonary blood pressure has involved inserting a catheter, which is highly invasive. In the present invention, waveform data can be obtained from a portion of an organ, and averaging this portion reveals a sine wave. This sine wave is thought to be related to pulmonary blood pressure. Specifically, image analysis using the present invention makes it possible to calculate the pulmonary blood pressure waveform based on the frequency, phase, and signal value of the waveform. Here, periodic movements synchronized with the heartbeat are extracted and imaged. During this process, changes in the image are tracked in accordance with blood flow. For example, the sine wave does not propagate to the hilum or periphery of the lung. Furthermore, a delay occurs from the center of the lung outwards. Therefore, it is possible to calculate the blood flow velocity from this delay. Pulmonary blood pressure can be estimated based on the diameter and flow velocity of the blood vessels. Moreover, it becomes possible to understand the pathological conditions and disease severity in heart failure, the overall disease, and in parts of the disease, as described herein, considering factors such as blood vessel thickness, phase shift, differences in diagrams, and respiratory status. For example, by training an AI with values ​​such as cardiac index and pulmonary artery pressure inhalation pressure, it can contribute to the treatment strategy for heart failure. By calculating the degree and proportion of respiratory movement and overlaying the respiratory coefficients from the lung apex to the diaphragm, lateral, and medial regions, estimated respiratory function can be calculated. Furthermore, by measuring the mismatch coefficient in matching respiratory function constants with signals such as blood flow, it becomes possible to predict ventilation-perfusion mismatch, ventilation-perfusion imbalance, and conditions where both pulmonary ventilation and perfusion are impaired (corresponding to pulmonary pulmonary blood flow impairment). In other words, it becomes possible to classify patients into four categories: good pulmonary ventilation and good perfusion, good pulmonary ventilation and abnormal perfusion, and bad pulmonary ventilation and abnormal perfusion, allowing for the assessment of the condition and the determination of treatment strategies based on the degree of these classifications.

[0148] The pulmonary blood flow waveform calculated using the present invention is a numerical representation of the degree of change in any given region, and it approximates the pulmonary artery waveform recorded during actual right heart catheterization. The portion of the pulmonary artery pressure waveform after the pulmonary valve closure notch corresponds to the right ventricular diastole phase, and the angle until it returns to the baseline reflects the pulmonary vascular resistance. By utilizing this feature, it is possible to detect differences in the relationship between the pulmonary blood flow waveform calculated using the present invention and the diastolic angle of the actual pulmonary artery pressure waveform between high-PVR and low-PVR groups, i.e., CpcPH patients and IpcPH patients.

[0149] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a diagram showing the schematic configuration of a diagnostic support system according to this embodiment. This diagnostic support system performs specific functions by having a computer execute a diagnostic support program. The basic module 1 consists of a cardiac function analysis unit 3, a blood flow analysis unit 5, other blood flow analysis units 7, a Fourier analysis unit 9, a waveform analysis unit 10, and a visualization / digitization unit 11. The basic module 1 acquires image data from a database 15 via an input interface 13. The database 15 stores images, for example, in DICOM (Digital Imaging and Communication in Medicine). The image signal output from the basic module 1 is displayed on a display 19 via an output interface 17. Next, the functions of the basic module according to this embodiment will be described. Note that the input image is not limited to the database 15; it is also possible to input images from other external devices via the input interface 13 or to input them manually.

[0150] [Refining dynamic site detection] The contrast of dynamic areas such as the lungs, rib cage, and heart may not be uniform along the lines. In such cases, the shape of dynamic areas can be detected more accurately by changing the threshold used for noise reduction and performing the detection process multiple times. For example, in the left lung, the contrast of the diaphragm line tends to weaken as it moves towards the interior of the body. Also, the apex and base of the heart move less, while the central part of the heart moves more. In such cases, by changing the threshold setting used for noise reduction or multiplying the pixel value by a different coefficient, it is possible to detect the remaining left half of the diaphragm, or the parts of the heart that move more or less. By repeating this process multiple times, it becomes possible to detect the shape of the entire diaphragm or the entire heart. Thus, this method makes it possible to quantify not only the position of the diaphragm, but also the rate and amount of change of lines and surfaces of the rib cage, heart, and dynamic areas, which can be useful for new diagnoses.

[0151] The detected position or shape of the diaphragm and heart can be used for diagnosis. Specifically, the coordinates of the diaphragm and heart can be graphed, and the coordinates of the rib cage, diaphragm, and heart can be calculated using the curves (surfaces) or straight lines calculated as described above. Furthermore, heart rate, vascular rate, and lung field "density" can be graphed as positions and coordinates corresponding to their cycles. Such methods can also be applied to dynamic areas that are linked to respiration and heartbeat.

[0152] This method allows for measurement not only of Hz during inspiration, expiration, systole, and diastole, but also of the diaphragm or other dynamic parts linked to respiration, as well as the frequency of dynamic parts of the heart, when these frequencies change, by measuring in a frequency band corresponding to the changes. Furthermore, during spectral extraction of the band-pass filter (BPF), it becomes possible to set the BPF within a certain range according to the state of respiration and heart, and to create a variable BPF where the axis of the BPF position changes during the "reconstruction phase" of respiration and heart, potentially creating an optimal state. This makes it possible to provide images that correspond to fluctuations in respiratory rhythm, such as when breathing slows down or stops (Hz=0), or when there is temporary cardiac fibrillation (extremely high frequency) or when the heart stops (Hz=0).

[0153] In this specification, the concept of "waveform tunability" is used for "(a) the wave form itself," and images are displayed based on this (Wave form tunable imaging). Furthermore, the concept of "frequency tunability" is used for "(b) the interval between waves (frequency: Hz)," and images are displayed based on this (Frequency tunable imaging).

[0154] For example, in the case of the heart, as shown in Figure 31A, "An example comparing the waveforms of aortic blood flow and ventricular volume," the peak of aortic blood flow and the peak or waveform of ventricular volume do not coincide. However, if we define one cycle as a time interval of equal intervals, such as from time t1 to t2, from time t2 to t3, from time t3 to t4, etc., then one cycle of aortic blood flow and one cycle of ventricular volume will be repeated many times, and it can be said that the frequencies of each waveform are synchronized. Focusing on these waveforms, by identifying one cycle from measured values ​​as shown in Figure 31A and using a model waveform, it is possible to predict the shape of the wave (wave form). In other words, as a way to create the "waveform as base data," it can be done by measurement, generated from frequency (cycle), using a model waveform, or by averaging waveforms between individuals. If the cycle (period) of organs with frequencies, such as the heart, is known, the wave shape can be predicted. This allows for the understanding of waveforms such as aortic blood flow and ventricular volume, and based on these waveforms, it becomes possible to display dynamic images of the organs.

[0155] Furthermore, it is possible to detect the movement of the diaphragm or other dynamic parts linked to respiration in a series of images. When images are selected at arbitrary intervals from a series of images and the difference between the images is calculated, the difference is particularly large in areas with high contrast. By appropriately visualizing this difference, areas where movement has occurred can be detected. During visualization, the aforementioned waveform can be created by emphasizing the continuity of areas with large absolute differences using noise reduction with thresholds or curve fitting using the least squares method. In the lung field, the contrast of the lines where the diaphragm and heart meet is prominent, and as shown in Figure 31B, when the difference is taken between two lung images and a certain threshold is set to visualize the difference, the lines where the diaphragm and heart meet can be visualized, as shown in Figure 31C.

[0156] Furthermore, the overall frequency of exhalation or inhalation may be calculated based on the proportion of the respiratory element (including all or part of exhalation or inhalation) to the total exhalation or inhalation. Similarly, the frequency of systole or diastole, frequency elements, and other overall frequencies may be calculated based on the proportion of the cardiac dynamic element (including all or part of systole or diastole) to the cardiac systole or diastole, a single heartbeat, and the total number of beats measured. In addition, when detecting the diaphragm or heart, multiple measurements may be performed and those with stable signals or waveforms may be selected. As a result, it becomes possible to calculate the frequency of at least one respiratory element or cardiac beat element, or the frequency representing the heartbeat, from the detected position or shape of the diaphragm or heart, or the position or shape of dynamic parts linked to respiration. Once the position or shape of the diaphragm, heart, or dynamic parts is known, it becomes possible to determine the frequencies of the respiratory element or cardiac beat element and the heartbeat. With this method, even if a part of the waveform is segmented, the subsequent waveform can be tracked. Therefore, even if the frequencies of respiratory or cardiac elements change midway through, it is possible to track the original respiratory and cardiac elements. Furthermore, even if the heartbeat and other rhythms change suddenly, this method can be similarly applied to the cardiovascular system and organs related to cardiovascular waveforms.

[0157] [Lung field detection] In this invention, as one aspect of the "refinement of dynamic area detection" described above, it is possible to refine lung field detection. In this process, after setting the largest and smallest lung fields, the remaining lung fields are calculated using these values. Figure 11 shows an example of the lung field detection method in this invention. In this method, a "B-spline curve" is used to represent the "coefficient of each image". In Figure 11, the waveform X represents the "coefficient of each image L that shows a lung field", and from left to right, it represents the first image The coefficients are as follows: the coefficient for the first image, the coefficient for the second image, and so on. Moving the control point Y in Figure 11 smoothly changes the "coefficient for each image". In this invention, it is possible to directly edit the coefficient graph in this way. In Figure 11, the "gray line Z" represents the "average pixel value for each image". When images are taken under optimal conditions, the change in lung field size and the change in the average pixel value coincide. Here, this average pixel value can also be smoothed by curve fitting and used directly as a "coefficient". The same method can be applied to the heart and other organs involved in cardiovascular frequency.

[0158] [Cardiac Function Analysis] Figure 2 is a cross-sectional view showing the schematic structure of the heart. "Cardiac function" is generally defined as "the pumping function of the left ventricle that circulates blood throughout the body." In "ischemic heart disease," especially "myocardial infarction," cardiac function analysis is important for estimating the patient's prognosis. For example, a decrease in the left ventricular ejection fraction (EF) reduces the heart's pumping output, making it unable to pump enough blood to the whole body. Other cardiac function metrics include left ventricular end-diastolic volume (EDV), left ventricular end-systolic volume (ESV), stroke volume (SV), cardiac output (CO), and cardiac index (CI). For local evaluation of the myocardium, a "Bull's eye map" showing wall thickness, wall motion, and rate of change of wall thickness, as shown in Figures 2B and 2C, which are cross-sections of the plane perpendicular to axis A in Figure 2A, is used. This "Bull's eye map" is an image that places the cross-section of the cardiac apex in the center of a circle, with short-axis tomographic images arranged concentrically around it, and the cross-section of the cardiac base on the outermost edge.

[0159] In this embodiment, the "Bull's eye map" is used, and the period of cardiac movement is analyzed based on the following indicators. Specifically, the period of cardiac movement is analyzed using density / intensity in a certain region within the cardiac area. Alternatively, data obtained from other measurement methods such as spirometry, or external input information, may be used, as well as a range consisting of a certain volume density / intensity measured in areas with high X-ray (and other modalities such as CT and MRI) transparency. It is desirable to compare the analysis results for each heartbeat and analyze trends from multiple data to improve the accuracy of the data. It is also possible to identify the edge of the heart and obtain the frequency based on the changes in this edge. Furthermore, it is possible to identify the edge of the lung field and obtain the frequency from the movement of this edge.

[0160] [Vascular pulse analysis] In this embodiment, vascular beats are analyzed based on the following indicators. Specifically, the heart, pulmonary hilum, and major blood vessels are identified from measurement results of other modalities such as electrocardiograms and pulse meters, or from the lung contour, and vascular beats are analyzed using changes in density / intensity of each area. Alternatively, the changes in density / intensity of the target area may be analyzed by manually plotting on the image. It is also possible to use heart rate elements obtained from heart rate or vascular beats. It is desirable to compare the analysis results for each beat and analyze trends from multiple data to improve the accuracy of the data. Furthermore, the accuracy of extracting density / intensity for each area can be improved by performing the analysis multiple times or over a certain range. Alternatively, cardiovascular beat frequency or frequency band can be input.

[0161] [Identification of the cardiac region] Images are extracted from the database (DICOM), and the cardiac region (especially the myocardium) is automatically detected using the results of the cardiac function analysis described above. Next, the myocardium is divided into multiple block regions, and the changes in each block region are calculated. Here, the size of the block region may be determined according to the scanning speed. If the scanning speed is slow, it becomes difficult to identify the corresponding area in the next frame image of a given frame image, so the block region is made larger. On the other hand, if the scanning speed is fast, the number of frame images per unit time is large, so it is possible to track even if the block region is small. The size of the block region may also be calculated according to which timing in the cardiac movement cycle is selected. Here, it may be necessary to correct for the displacement of the myocardial region. In that case, the movement of the heart is identified, and the relative position of the heart's contour is grasped, and it is evaluated relatively based on that movement. Note that if the block region is too small, image flickering may occur. To prevent this, the block region needs to have a certain size.

[0162] [Create a block region] Next, a method for dividing the myocardium into multiple block regions will be described. Figures 2B and 2C illustrate a method for dividing the myocardium radially from the center of the heart. The regions of the heart should be evaluated relatively based on their movement, the positional relationship of the heart and blood vessels, and the relative position of the heart's contour. For this reason, in the present invention, after automatically detecting the contour of the heart, the myocardium region is divided into multiple block regions, and the values ​​of image changes (pixel values) included in each block region are averaged. As a result, even if the morphology of the heart changes over time, it becomes possible to track the temporal changes of the region of interest.

[0163] On the other hand, if the image is divided into block regions without specifying the heart region, the region of interest may move outside the heart region due to changes in the heart over time, resulting in a meaningless image. Alternatively, the heart rate or frequency band can be input. These methods are also applicable to three-dimensional stereoscopic images. By keeping the number of pixels constant in a 3D stereoscopic image, region division calculations can also be performed for 3D images. This relative evaluation based on relative positional movement can be performed between adjacent frame images, or at integer multiples such as every two or three images. Alternatively, several images can be grouped together and processed for each group.

[0164] Figure 7A is a schematic diagram showing the left lung of the human body from the front, and Figure 7B is a schematic diagram showing the left lung of the human body from the left side. Both Figures 7A and 7B represent the lung in the state of inhalation, i.e., when breathing in. Figure 8A is a schematic diagram showing the left lung of the human body from the front, and Figure 8B is a schematic diagram showing the left lung of the human body from the left side. Both Figures 8A and 8B represent the lung in the state of exhalation, i.e., when breathing out. As shown in these figures, the morphology of the lung field changes significantly during respiration, but the rate of change in the lung field on the diaphragm side is large, while the rate of change in the lung field on the opposite side of the diaphragm is small. In this invention, the position of each region within the lung field is changed according to this rate of change. This makes it possible to perform a relative evaluation based on the relative positional relationship of each region within the lung field region. Furthermore, it is possible to represent relative positional relationships within the lung field region with a constant rate of change based on the rate of change of the lung field (for example, the average rate of change), or to adaptively change the rate of change within the lung field region according to the distance from the diaphragm. In this way, by using the rate of change in the lung field region, it becomes possible to display images synchronized with the respiratory cycle.

[0165] Figure 9A is a schematic diagram showing the left lung of the human body from the front, and Figure 9B is a schematic diagram showing the left lung of the human body from the left side. Both Figures 9A and 9B represent the lung in the state of inhalation, that is, when breathing in. Figure 10A is a schematic diagram showing the left lung of the human body from the front, and Figure 10B is a schematic diagram showing the left lung of the human body from the left side. Both Figures 10A and 10B represent the lung in the state of exhalation, that is, when breathing out. For example, suppose a marker P1 is plotted at a certain location within the lung field region in the inhalation state, as shown in Figures 9A and 9B. If marker P1 is a fixed point defined by two-dimensional coordinates, the coordinates do not change even in the exhalation state, so as shown in Figures 10A and 10B, marker P1 remains in the same position. On the other hand, in the present invention, as described above, the evaluation is based on the relative positional relationship with respect to the entire lung field region, so in the exhalation state, the marker moves to the position of marker P2, not the position of marker P1. Using the vectors generated at this time, it is also possible to evaluate the points plotted during inhalation and the points to which the data moved during exhalation.

[0166] When dividing a region, it is possible to apply Voronoi tessellation (Thiessen tessellation). As shown in Figure 2D, Voronoi tessellation is a method of dividing the region into its nearest neighbors by drawing perpendicular bisectors on the lines connecting adjacent generator points. Applying such Voronoi tessellation can reduce computation time. Furthermore, when drawing the lines connecting adjacent generator points, weighting may be applied depending on the object being analyzed. For example, when dividing the region of the pulmonary arteries, thicker areas may be given higher weighting, and thinner areas may be given lower weighting. This reduces the processing burden and allows for division according to the object being analyzed. The multiple block regions resulting from the division may be classified based on indicators such as changes in pixel values ​​(periodic changes).

[0167] After dividing the region into multiple block regions in this way, the image changes of each block region are calculated based on the relative position of each block region and dynamic parts such as the heart. Here, the signal difference is not only taken using the range of the mass itself as a single unit, but the signal difference may also be taken over a range smaller than the mass, or a larger range that surrounds the mass. Furthermore, the range may be enlarged only in the vertical direction near the diaphragm, or only in the horizontal direction for other dynamic parts, or the shape of the range may be deformed, or the pixel regions may be connected. In addition, after calculating one or more differences, it is desirable to redefine the shape of the mass to match the overall morphology of the lung field or heart. For example, after processing from the first to the second image, the shape of the mass may be recreated again to match the shape of the second image, and then compared from the second to the third image.

[0168] The above explanation described "relative positional relationships" that take into account the movement of organs, but the present invention is not limited to this, and it is also possible to perform image processing while maintaining the "absolute positional relationships of each pixel" within multiple frame images. The "absolute positional relationships of each pixel" refers to the relationship between pixels that have coordinates identified based on two-dimensional coordinate axes defined on the frame image. In other words, it is a method of pixel processing that treats the pixel of interest as invariant. Processing while maintaining the "absolute positional relationships of each pixel" presupposes multiple frame images, but the number of frame images is not specified. Multiple frame images may be classified into multiple groups, and each group may contain an equal number of frame images, or each group may contain a different number of frame images.

[0169] In other words, multiple frame images are classified into multiple groups, and while maintaining the absolute positional relationship of each pixel within the multiple frame images belonging to each group, changes in the pixel values ​​of organs, changes in the distance from the center to the outer edge of organs, or changes in the volume of organs are calculated. This allows even if the pixel values ​​change slightly, they can be treated as uniform pixel values, making it possible to reduce the amount of data and the number of processing steps.

[0170] Furthermore, a positional relationship that is neither relative nor absolute can be conceived. That is, a point P with specific coordinates can be defined within a specific frame image, and another point Q with different coordinates from the above specific point can be defined in subsequent frame images, but in this case, the magnitude of the vector PQ is assumed to be small relative to the movement of the organ. Furthermore, for point Q, another point R with slightly different coordinates from point Q can be defined in subsequent frames. By repeating this operation, other points with a slight shift from the specific point P can be extracted for each frame image, and this embodiment can be applied. Specifically, multiple frame images are acquired, and for each pixel in a specific frame image, pixels with different coordinates from the above pixel in subsequent frame images are extracted, and a periodic change that characterizes the state of the organ is calculated.

[0171] Then, the periodic changes characterizing the state of the organs are subjected to a Fourier transform, and from the resulting spectrum, a spectrum within a certain band containing the spectrum corresponding to the frequency of the organ's movement is extracted. An inverse Fourier transform is then performed on the spectrum extracted from the certain band, and each image after the inverse Fourier transform is output. Alternatively, pixels that change in accordance with the frequency of the periodic changes characterizing the state of the organs in the image may be extracted using a digital filter, and an image containing the pixels extracted by the digital filter may be output. Furthermore, a color corresponding to the rate of change characterizing the state of the organs may be selected, and the selected color may be applied to the rate of change of the pixel values ​​to display the image on a display. This makes it possible to represent the movement of the organs.

[0172] Next, the image data is interpolated to remove artifacts. In other words, since bones and other objects within the analysis range will appear as noise, it is desirable to remove the noise using a noise-cutting filter. In X-ray images, air is typically set to -1000 and bone to 1000, so areas with high transmittance have low pixel values ​​and are displayed as black, while areas with low transmittance have high pixel values ​​and are displayed as white. For example, if the pixel value is represented in 256 levels, black is 0 and white is 255.

[0173] Within the heart region, areas where blood vessels and bones are present are difficult for X-rays to penetrate, resulting in higher pixel values ​​in the X-ray image and a whiter appearance. The same applies to CT and MRI. Based on the results of the cardiac function analysis described above, it is possible to interpolate the data using values ​​with the same phase based on the waveform per heartbeat and eliminate artifacts. Furthermore, artifacts such as "different coordinates," "extreme fluctuations in pixel values," and "abnormally high frequency or density" can be present. When "this" is detected, a cutoff is performed on these, and the remaining obtained images can be used to identify a continuous and smooth wave shape using, for example, the least squares method, so that they can be used for calculating the Hz of cardiac motion and adjusting the myocardial region. Furthermore, when superimposing images, there are two methods: (1) acquiring one image from before and superimposing the acquired comparison image at its original coordinates, and (2) acquiring one image from before or after as a base, then relatively expanding the image and superimposing its relative position information onto the base. By using the above methods, it becomes possible to correct the morphology of the cardiac region or correct changes in the block region image.

[0174] Here, we will explain "reconstruction" in terms of the time axis. For example, if the intake time is 2 seconds at 15 f / s, 30+1 images will be obtained. In this case, 10% "reconstruction" can be performed simply by stacking 3 images together. However, if, for example, 0.1 seconds is 10%, and only images taken at 0.07 seconds and 0.12 seconds are available, then "reconstruction" of 0.1 seconds is necessary. In this case, the "reconstruction" is performed by using an intermediate value (the average of both) of the images around 10%. Alternatively, the coefficient can be changed according to the proportion of time. For example, if there is a difference in the time axis and there is no image taken at 0.1 seconds, but there are images taken at 0.07 seconds and 0.12 seconds, the "reconstruction" can be performed by recalculating as "(the value for 0.07 seconds) × 2 / 5 + (the value for 0.12 seconds) × 3 / 5". Furthermore, it is desirable to calculate with thickness, including 0-100% of "Maximum Differential Intensity Projection," such as "reconstruction" of 10% to 20% or "reconstruction" of 10% to 40%. In this way, it is possible to perform "reconstruction" at a rate of one heartbeat even for parts that have not been photographed. Moreover, the present invention can similarly perform "reconstruction" for the heart, blood flow, and a series of movements linked to them.

[0175] [Fourier analysis] Based on the cardiac movement period and vascular rhythm period analyzed as described above, Fourier analysis is performed on the density / intensity values ​​and their changes in each block region. Figure 3A shows the intensity change of a specific block and the result of its Fourier analysis. Figure 3B shows the Fourier transform result with frequency components close to the heartbeat extracted, and the inverse Fourier transform of this result to show the intensity change of frequency components close to the heartbeat. For example, performing a Fourier transform (Fourier analysis) on the intensity change of a specific block yields the result shown in Figure 3A. Then, extracting the frequency components close to the heartbeat from the frequency components shown in Figure 3A yields the result shown on the right side of Figure 3B. By performing an inverse Fourier transform on this, an intensity change synchronized with the heartbeat can be obtained, as shown on the left side of Figure 3B.

[0176] Here, when performing an inverse Fourier transform on a spectrum consisting of frequency components, it is possible to take into account both the frequency elements identified from the density of heart rate and blood flow (heart rate, cardiovascular frequency) and the bandwidth of the spectrum (BPF may be used), or to perform the inverse Fourier transform based on either one of these elements.

[0177] Furthermore, when performing a Fourier transform, the Autoregressive Moving Average (AR) method can be used to enable faster calculations. The AR method employs the Yule-Walker equation and Kalman filter in an autoregressive moving average model, and the Yule-Walker estimates derived from these, along with the PARCOR method and the least squares method, can be used to supplement the calculations. This makes it possible to acquire images more quickly and in near real-time, and to perform computational assistance and artifact correction. Through such Fourier analysis, it becomes possible to extract and display the image properties of each block region.

[0178] Here, the image changes of each block region in each frame image are Fourier transformed, and from the spectrum obtained after the Fourier transform, a spectrum within a certain band, including the spectrum corresponding to the period of heart movement, can be extracted. Figure 3C shows an example of extracting a certain band from the spectrum obtained after the Fourier transform. The frequency f of the composite wave spectrum has the relationship "1 / f = 1 / f1 + 1 / f2" between it and the original frequencies f1 (heartbeat component) and f2 (pathological blood flow component), and the following method can be used when extracting the spectrum.

[0179] (1) Extract the portion of the heart rate spectrum that has a high spectral ratio. (2) The spectrum is extracted by dividing it at the midpoint between the peak of the spectrum corresponding to the heart rate / pathological blood flow and the peaks of multiple composite waves in its vicinity. (3) The spectrum is extracted by dividing it at the peak of the spectrum corresponding to the heart rate / pathological blood flow and the trough of the spectrum of multiple composite waves in its vicinity. (4) The spectrum contained within a certain bandwidth may be extracted from the heart rate component (blood flow component). In this case, a spectrum in which multiple spectra overlap will be obtained, but each spectrum can be reconstructed by separating each component.

[0180] As described above, the present invention does not use a fixed BPF, but rather extracts spectra within a certain bandwidth that include spectra corresponding to the period of cardiac movement. Furthermore, the present invention also makes it possible to extract spectra within a certain bandwidth from the spectra obtained after the Fourier transform that include frequencies other than cardiac movement obtained from the frame image (for example, density / intensity of each part, cardiac beats or vascular beats obtained from heartbeats or vascular beats), or spectra corresponding to frequencies input externally by the operator (for example, spectral models).

[0181] Here, if the composite wave spectrum consists of only two components (heartbeat and pathological blood flow), the distribution will be 50% + 50%, and if there are three components, the distribution will be 1 / 3 each. Therefore, it is possible to calculate the composite wave spectrum to some extent from the components and their heights, such as what percentage of the spectrum is from the heartbeat component and what percentage is from the pathological blood flow component. It is possible to extract the spectrum where that ratio (%) is high. In other words, the ratio of the pathological blood flow component / heartbeat component to the composite wave component is calculated, and the high spectral value of the pathological blood flow component / heartbeat component is calculated and extracted. In addition, in cases such as diaphragm identification, it is sometimes possible to extract only the spectrum or superposition of spectra corresponding to areas where the Hz (frequency) is relatively constant, i.e., regions with little change in Hz, from data obtained from heartbeat and cardiovascular frequencies. Furthermore, when defining the spectral bandwidth, it is sometimes possible to define the spectral bandwidth in the range where the change in Hz occurs and the surrounding region. As a result, it is possible to extract not only spectra that perfectly match the period of cardiac movement and vascular cycle, but also spectra that should be considered, thereby contributing to diagnostic imaging.

[0182] It is known that "heart rate" and "respiration" fall within specific frequency bands. Therefore, for example, in the case of respiration, a filter of "0-0.5 Hz (respiratory rate 0-30 breaths / min)" can be used, and for the circulatory system, a filter of "0.6-2.5 Hz (heart rate / pulse rate 36-150 beats / min)" can be used to pre-determine the respiratory frequency and the circulatory frequency. This makes it possible to display frequency-synchronized images. This is because when acquiring changes in cardiac density, changes in respiratory (lung) density may be picked up, and vice versa.

[0183] [Waveform analysis] Waveform analysis is performed on waveforms recognized as consistent in cardiac, vascular, electroencephalogram (EEG), and other examinations. This includes repetitive movements such as leg movements. Furthermore, the Hz of repetitive movements is superimposed and analyzed to determine if there is a similar trend. The waveform data is compared, and the agreement rate between the two sets of data is calculated. Finally, the data after Fourier analysis is compared.

[0184] [Digital Filter] Alternatively, digital filters can be used instead of the Fourier analysis described above. Digital filters perform a transformation between the time domain and the frequency domain based on the "Fast Fourier Transform and Inverse Fast Fourier Transform," which are mathematical algorithms for extracting the frequency components of a signal, in order to adjust the frequency components of the signal. This makes it possible to achieve the same effect as the Fourier analysis described above.

[0185] [Visualization and Quantification] The results of the analysis described above are visualized and quantified. As a standard uptake, the average value of the density / intensity of the entire measured cardiac region is set to 1 and the values ​​are displayed relatively / logarithmically. In addition, since only the direction of blood flow is adopted, changes in a specific direction may be isolated. This makes it possible to extract only meaningful method data. Using the identification results of the cardiac region, pseudo-colorization is performed to track changes in the analysis range. That is, the analysis results of each individual (subject) are fitted to the relative region according to a specific shape (minimum, maximum, average, median) that matches the phase. Furthermore, the results of multiple analyses are transformed into a specific shape / phase that allows for comparison.

[0186] Furthermore, when creating a "standard heart," the relative positional relationships within the heart (myocardium) are calculated using the results of the analysis of heart movement described above. The "standard heart" is created using a line that comprehensively averages the outline and density of multiple patients' hearts. This concept can be applied not only to the heart but also to the lungs (standard lungs) and other organs (standard organs). For example, "organ models" can be created based on age, sex, country, and disease severity.

[0187] In addition to the changes in the pixel values ​​of the heart as described above, it is also possible to calculate the change in the distance from the center of the heart to the myocardium (distance L shown in Figures 2B and 2C) and perform Fourier analysis. Furthermore, it is also possible to calculate the change in the volume of the heart and perform Fourier analysis.

[0188] Once a "standard heart" is created, as described above, it becomes possible to quantify and present the synchronicity, agreement rate, and mismatch rate (display of frequency synchronicity image). Furthermore, deviations from a normal state can be displayed. According to this embodiment, by performing Fourier analysis, it becomes possible to discover the possibility of new diseases, compare with a normal self, compare hands and feet, and compare with the opposite hand and foot. In addition, it becomes possible to understand what is abnormal in foot movement, swallowing, etc., by quantifying the synchronicity. Moreover, it becomes possible to determine whether a person with a disease has changed after a certain period of time, and if so, to compare the state before and after the change.

[0189] [Drawing of the heart] This specification employs a method for provisionally drawing the contour of the heart using a combination of Bézier curves and straight lines, and then adjusting the heart to improve its accuracy. For example, if the contour of the heart is represented by four Bézier curves and one straight line, it becomes possible to draw the contour of the heart by finding five points on the contour and four control points. By shifting the position of the points and drawing multiple contours of the heart, the accuracy of the heart contour can be determined by evaluating the accuracy using conditions such as "maximizing the total density within the contour" or "maximizing the difference in the total density of a few pixels inside and outside the contour line." It is also possible to extract points close to the outer edge using classical binarization and adjust the control point positions of the Bézier curves using methods such as the least squares method. The above method is not limited to the heart and can be applied to other organs as "organ detection." Furthermore, it can be applied not only to planar images but also to three-dimensional images (3D images). By defining the equation of a curved surface and setting its control points, it becomes possible to estimate that an object enclosed by multiple curved surfaces is an organ.

[0190] [Cardiac function analysis using Fourier analysis] Next, we will describe the cardiac function analysis using Fourier analysis according to this embodiment. Figure 4 is a flowchart showing an overview of the cardiac function analysis according to this embodiment. Basic module 1 extracts DICOM images from database 15 (step S1). Here, at least multiple frame images contained within one heartbeat are acquired. Next, in each acquired frame image, the period of cardiac movement is identified using the change in pixel value in at least a certain region of the myocardium, for example, the change in density (density / intensity) (step S2). Next, the region of the heart (myocardium) is detected (step S3), and the detected myocardium is divided into multiple block regions (step S4). Here, as described above, the myocardium is divided radially from the center of the heart using Voronoi tessellation (Thiessen tessellation). Then, the change in pixel value of each block region in each frame image is calculated (step S5). Here, the values ​​of the changes within each block region are averaged and expressed as a single data point.

[0191] It is also possible to display pixels somewhat ambiguously, blurring the overall image. In particular, in the case of blood vessels, low-level signals are mixed in with high-level signals, but if we can roughly grasp only the high-level signals, the overall ambiguity is acceptable. For example, in the case of blood flow, we can extract only the signals above a threshold. Specifically, if we take the numbers in the following table as one pixel and obtain the middle value, we can obtain the proportion occupied by the middle value and average it within one pixel to achieve a smooth representation between adjacent pixels.

[0192] [Table 1]

[0193] Furthermore, noise reduction may be performed on the values ​​of change within each block region using a cutoff. Next, Fourier analysis is performed on the density / intensity values ​​of each block region, as well as the amount of change thereof, based on the period of heart movement described above (step S6). This makes it possible to extract and display the image properties of each block region.

[0194] Here, from the spectrum obtained after the Fourier transform, it is possible to extract a spectrum within a certain band that includes the spectrum corresponding to the heart cycle. Here, the frequency f of the composite wave spectrum is related to the original frequencies f1 and f2 by the relationship "1 / f = 1 / f1 + 1 / f2", and the following method can be used when extracting the spectrum.

[0195] (1) Extract the portion of the heart's movement with a high spectral ratio. (2) The spectrum is extracted by dividing it at the midpoint between the peak of the spectrum corresponding to heart rate / blood flow and the peaks of multiple composite waves in its vicinity. (3) The spectrum is extracted by dividing it at the peak of the spectrum corresponding to heart rate / blood flow and the trough of the spectrum of multiple composite waves in its vicinity.

[0196] Here, if the composite wave spectrum consists of only two components (heartbeat and blood flow), the distribution will be 50% + 50%, and if there are three components, the distribution will be 1 / 3 each. Therefore, it is possible to calculate the composite wave spectrum to some extent from the components and their heights, such as what percentage of the spectrum is for the heartbeat component and what percentage is for the blood flow component. It is possible to extract the spectrum where that ratio (%) is high. In other words, the ratio of the blood flow component / heartbeat component to the composite wave component is calculated, and the high spectral value of the blood flow component / heartbeat component is calculated and extracted.

[0197] Next, the results obtained from the Fourier analysis are denoised (step S7). Here, cutoffs and artifacts can be removed as described above. The operations from steps S5 to S7 are performed one or more times, and it is determined whether the process is complete (step S8). If it is not complete, the process returns to step S5; if it is complete, the results obtained from the Fourier analysis are displayed on the screen as a pseudo-color image (step S9). Alternatively, a black and white image may be displayed. In this way, the accuracy of the data can be increased by repeating multiple cycles. This makes it possible to display the desired video. Alternatively, the desired video can be obtained by modifying the image displayed on the screen.

[0198] In addition to the myocardial division process in steps S4 and S5 described above, it is also possible to perform Fourier analysis by calculating the change in distance from the center of the heart to the myocardium instead of steps S4 and S5. Furthermore, it is also possible to perform Fourier analysis by calculating the change in the volume of the heart instead of steps S4 and S5.

[0199] [Cardiac function analysis using digital filters] Next, we will describe the functional analysis using the digital filter according to this embodiment. Figure 5 is a flowchart outlining the cardiac function analysis according to this embodiment. Steps S1 to S5 and steps S7 to S9 are the same as in the "cardiac function analysis using Fourier analysis" described above, so we will omit the explanation. In step T1 of Figure 5, digital filtering is performed (step T1). The digital filter performs a transformation between the time domain and the frequency domain based on the "fast Fourier transform and inverse fast Fourier transform," which are mathematical algorithms for extracting the frequency components of a signal, in order to adjust the frequency components of the signal. This makes it possible to obtain the same effect as the Fourier analysis described above.

[0200] [Cardiac function analysis using synchronization rate] Next, the cardiac function analysis using the synchronization rate according to this embodiment will be described. Figure 6 is a flowchart showing an overview of the cardiac function analysis according to this embodiment. Steps S1 to S5 and steps S7 to S9 are the same as in the "cardiac function analysis using Fourier analysis" described above, so the explanation will be omitted. In step R1 of Figure 6, the synchronization rate is analyzed (step R1). That is, the region of the heart (myocardium) is detected (step S3), the myocardium is divided into multiple block regions (step S4), the average density (pixel value x) of the block region in each frame image is calculated, and the ratio (x') of the average pixel value of the block region in each frame image to the range of change from the minimum to the maximum value of the average density (pixel value x) (0% to 100%) is calculated (step S5). On the other hand, the ratio (x' / y') of the change in the heart (y) in each frame image to the range of change from the minimum to the maximum value of the surface area (or volume) of the heart (y) (y') is calculated (step S5). Using these methods, we can extract only the block regions where the ratio value (x' / y') falls within a predetermined range (Step R1).

[0201] Here, if y'=x' or y=ax (where a is a coefficient for the amplitude of the surface area or volume of the heart, or for the density), then it is a perfect match. However, not only cases of perfect match are meaningful values; values ​​within a certain range should be extracted. Therefore, in one aspect of the present invention, a certain range is defined using logarithms (log) as follows. That is, when calculated as the percentage of y=x, a perfect match in synchronization is "log y' / x'=0". Furthermore, when extracting a range of synchronization agreement rates that is narrow (mathematically narrow), for example, a range close to 0 is defined as "log y' / x'=-0.05 to +0.05", and if the range of synchronization agreement rates is wide (mathematically wide), for example, a range close to 0 is defined as "log y' / x'=-0.5 to +0.5". The narrower this range is, and the higher the numerical values ​​that match within that range, the higher the agreement rate can be said to be. If we calculate the value of this ratio for each pixel and count the number of occurrences, a normal distribution with a peak at the case of a perfect match will be obtained for the heart of a healthy person. In contrast, for a person with a disease, the distribution of this ratio value will be distorted. It should be noted that the method of determining the range using logarithms as described above is merely one example, and the present invention is not limited to this. In other words, the present invention performs "image extraction" by assuming that (average of density changes over a certain rough range) ≈ (heart rate) ≈ (changes in the heart) ≈ (electrocardiogram) ≈ (changes in the surface area and volume of the heart), and methods other than those using logarithms can also be applied.

[0202] Furthermore, when considering this in 3D, by measuring heart rate, cardiac surface area or volume, cardiac output, and central blood flow with a separate device, it becomes possible to measure the "partial cardiac surface area," "partial cardiac volume," and "blood flow rate" in each region from their proportions. If quantitative measurements of cardiac surface area, cardiac volume, cardiac output, and central blood flow become possible using other modalities, it becomes possible to estimate functional values ​​from the volume and proportion of a single frame, and the percentage change in the region. In other words, in cardiac function analysis, it becomes possible to estimate cardiac volume from cardiac movement, and in blood flow analysis, it becomes possible to estimate pulmonary blood flow from cardiac output, and estimate estimated blood flow (proportion) in branched vessels depicted from central blood flow (proportion).

[0203] Furthermore, as mentioned above, while calculating all the data in the acquired database would allow for more accurate judgments, performing computer analysis can be time-consuming. Therefore, it is also possible to extract only a certain number of images (phases) and perform calculations. For example, instead of starting from the beginning of the acquired images, the latter half, i.e., images from the middle to the end, can be automatically acquired (or manually acquired). This makes it easier to extract more stable images by cutting out images taken during the initial tension state that occurred when imaging the patient. In addition, instead of calculating the acquired images (e.g., 300 images) as they are, it is also possible to select points where the "heart movement" changes, such as the initially measured position of the heart, and then perform calculations. This makes it possible to create image markers that appear to show a continuous heartbeat, which is useful when it is necessary to repeatedly view video markers. This can also be calculated using image markers. When identifying the heart (myocardium) region, even if only a part of the shape is manually changed or only part of the graph markers are manually altered, it is desirable to correct the graph using noise reduction and the least squares method.

[0204] As described above, this embodiment makes it possible to evaluate images of the human body using an X-ray video scanner. If digital data can be acquired, calculations can generally be performed well with existing facility equipment, resulting in low implementation costs. For example, in an X-ray video scanner using a flat panel detector, it becomes possible to easily examine the subject. In cardiac function analysis, it becomes possible to screen for myocardial infarction. For example, in an X-ray video scanner using a flat panel detector, unnecessary examinations can be eliminated by running the diagnostic support program according to this embodiment before performing a CT scan. Furthermore, because the examination is simple, it becomes possible to detect highly urgent diseases early and respond to them preferentially. Currently, there are several challenges with other modalities such as CT and MRI in imaging methods, but if these can be resolved, detailed diagnosis of each area will be possible.

[0205] Furthermore, it can be applied to screening various blood vessels, such as cervical blood flow narrowing, and also to evaluating and screening blood flow in large vessels. In addition, it can be applied to understanding pre- and post-operative characteristics. Moreover, by performing Fourier analysis on the cardiac movement cycle and blood flow cycle, and removing the cardiac movement waveform and blood flow waveform from abdominal X-ray images, it becomes possible to observe remaining abnormalities in biological movement, such as intestinal ileus.

[0206] Note that if the initial image acquired is of a relatively high resolution, the calculation time may be longer due to the large number of pixels. In that case, you can reduce the image to a certain number of pixels before calculating. For example, you can reduce the calculation time by reducing a "4000 x 4000" image to "1028 x 1028" pixels before calculating.

[0207] Furthermore, conventional methods involved manually adjusting the contrast range of the target image to be judged. Alternatively, a method was employed to depict the target image relatively based on certain criteria. However, this was not done with strict recognition of the frame of the object to be analyzed (e.g., lung field). In this invention, by detecting the cardiac region and filtering the bone density (cutting out areas with low transparency within a certain range), the range of transparency within the remaining region is strictly limited to the range required to recognize the cardiac region. As a result, when detecting the cardiac region, the transparency adjustment becomes more precise for those making the judgment, such as doctors or technicians. In addition, the transparency adjustment becomes more precise for evaluating coloring.

[0208] Furthermore, in X-ray and CT scans, the X-ray transmission is adjusted according to the body's condition to reduce radiation exposure, and in some cases, the transmission may change depending on the movement of the lungs during imaging. Similarly, in MRI scans, changes in the magnetic field in a specific direction result in a certain degree of non-uniform signal. In contrast to these, the transmission, which has changed from the transmission of the surrounding "background," is being standardized. By adjusting the recalculation to match the transmittance of a specific organ, or by uniformly correcting the non-uniformity of the overall magnetic field change to apply a certain correction to the signal value, it becomes possible to measure the change in "density / intensity" of a specific organ with greater precision. Furthermore, by changing the transmittance based on the characteristics of each organ rather than the numerical changes in imaging conditions, and correcting the change in "density / intensity" of a specific organ with greater precision, it becomes possible to calculate the amount of change with greater accuracy.

[0209] In this invention, the average value and changes in intensity are sometimes calculated, but data (intensity) obtained from X-rays, CT scans, etc., may have "gamma correction" applied, and the density may not be accurately reflected. Gamma correction is a process to correct the saturation and brightness of images displayed on a display. Normally, a computer displays an image on a display according to the input signal, but the displayed brightness and saturation may differ depending on the characteristics of the display. Therefore, gamma correction is used to correct these errors. Gamma correction adjusts the relative relationship of the signals and color data when they are input and output to the display to produce a display that is close to natural. However, since the image after gamma correction is no longer the original image, problems may arise when applying the image processing according to the present invention. Therefore, by applying "inverse gamma correction," that is, an inverse filter corresponding to the gamma correction value, to the image after gamma correction, the image before gamma correction is obtained. This makes it possible to obtain the image before gamma correction, and to perform the image processing according to the present invention appropriately.

[0210] Furthermore, in this invention, images that have undergone other image processing, not just gamma correction, may be restored to their original state before being processed. For example, by inferring the image processing performed on an image from the pixel value changes in areas where the density remains constant during shooting, such as space outside the human body, a function that makes the pixel value changes constant is applied to all pixels. This makes it possible to obtain an image close to the original image, and to perform the image processing according to the present invention appropriately.

[0211] Furthermore, this invention makes it possible to reconstruct individual images from superimposed images. Conventionally, "X-ray difference imaging technology" has been known. In this technology, "past and present X-ray images" of the same patient taken during a medical examination are superimposed, and parts that have changed from the past to the present, i.e., parts that are suspected to be abnormal, are highlighted. This makes it possible to detect cancer at an early stage, for example. As for the superimposition method, when multiple images are taken as "1st, 2nd, 3rd, 4th...", for example, "1st, 2nd, 3rd" is made into "1st superimposed image", "2nd, 3rd, 4th" is made into "2nd superimposed image", and "3rd, 4th, 5th" is made into "3rd superimposed image". In such superimposed images, the pixel value is high in the parts where multiple images overlap, and the pixel value is low in the parts that do not overlap. Since the outlines of organs may be blurred in such superimposed images, it is desirable to reconstruct them into the original individual images. According to the present invention, since the contours of organs can be identified, it becomes possible to reconstruct the original image from the superimposed image.

[0212] Furthermore, this invention makes it possible to visualize the differences in frequencies across different regions of an organ. Specifically, in an organ that undergoes periodic movement, the change values ​​for each region can be Fourier transformed, and weighting can be applied, such as color-coding, based on the composition ratio of frequency components, thereby indicating the characteristics of each region. For example, it is possible to identify the frequency components that peak in each region and color-code each region. It is also possible to identify the proportion of each frequency component within a specific frequency band in each region and color-code each region according to that proportion. Moreover, in a specific region, if the reference frequency composition ratio is, for example, "50% at 10Hz and 50% at 20Hz," it is also possible to visualize the deviation rate from that reference frequency composition ratio. As a result, for example, in the case of the heart, by displaying the spectral distribution of frequencies in different colors, it becomes possible to grasp at a glance whether it is moving correctly or incorrectly. [Explanation of Symbols]

[0213] 1. Basic Module 3. Cardiac Function Analysis Department 5 Blood flow analysis section 7. Other blood flow analysis units 9. Fourier Analysis Section 10 Waveform analysis section 11 Visualization and Numericalization Section 13 Input Interfaces 15 Databases 17 Output Interface 19 displays

Claims

1. A process to calculate a diagnostic target organ waveform that characterizes the state of the diagnostic target organ, showing the relationship between time and pixel value, based on the temporal changes in pixel values ​​of multiple frame images corresponding to multiple different time timings, in which the diagnostic target organ, which is expected to exhibit periodic movements in the human body being diagnosed, is captured in the image. A process for calculating the phase difference between a comparison organ waveform, which characterizes the state of a comparison organ corresponding to the diagnostic target organ that performs normal periodic operation and shows the relationship between time and pixel values, and the diagnostic target organ waveform, To have the diagnostic support system computer execute this, A diagnostic support program characterized by the following features.

2. The diagnostic support system's computer is instructed to perform a process to calculate a waveform of the comparison organ that characterizes the state of the comparison organ, based on the temporal changes in pixel values ​​of multiple frame images corresponding to multiple different time timings, which show the relationship between time and pixel values, and which captures the comparison organ performing normal periodic movements. The diagnostic support program according to feature 1.

3. In order to calculate the temporal change in pixel values ​​of multiple frame images showing the organ to be diagnosed, the computer of the diagnostic support system is instructed to divide each frame image into multiple divided regions and calculate the temporal change in pixel values ​​of each divided region. The diagnostic support program according to feature 1.

4. When calculating the temporal change in pixel values ​​for each divided region, the diagnostic support system's computer is instructed to perform a process that uses the average value of multiple pixel values ​​contained within each divided region. The diagnostic support program according to feature 3.

Images