Program, information processing method, and information processing device.

Abstract

Provided are a program and the like whereby an image containing information related to the size of an imaging target can be generated.　The program causes a computer to execute a process for: acquiring a plurality of tomographic images (41) generated by means of three-dimensional scanning using an image acquisition catheter that acquires images while moving a scanning plane in an axial direction; determining a representative line (42) for each of the tomographic images (41); calculating an index value for each scanning line composing each of the tomographic images (41), the index value being calculated in a calculation range that has been determined on the basis of the intersection point of each scanning line and the representative line (42); and displaying a color that has been determined on the basis of the index value at coordinates established by the position of the tomographic image (41) and the position of the intersection point.

Description

Technical Field

[0001] The present invention relates to a program, an information processing method, and an information processing apparatus.

Background Art

[0002] There has been proposed a diagnostic support apparatus that generates a two-dimensional image arranged on two axes of the angle of a scanning line and the axial direction of a catheter, based on a set of tomographic images obtained using a catheter for three-dimensional scanning type IVUS (Intravascular Ultrasound), and the similarity between the luminance pattern of each scanning line and a predefined classification pattern (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By observing the image generated by the diagnostic support apparatus of Patent Document 1, the user can confirm, for example, how the wires of a stent placed in a blood vessel are arranged, in an image in a manner as if the blood vessel were dissected.

[0005] However, the image generated by the apparatus of Patent Document 1 has a problem that it cannot evaluate the absolute amount of the size of the imaging object included in the scanning line. For example, the area of a plaque, the expansion amount of a stent, the change in blood vessel diameter, etc. cannot be observed in the image generated by the apparatus of Patent Document 1.

[0006] On one aspect, an object is to provide a program or the like that can generate an image including information on the size of an imaging object.

Means for Solving the Problems

[0007] The program acquires multiple tomographic images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially, determines a representative line for each of the tomographic images, calculates an index value for the scan line in a calculation range determined based on the intersection point of each scan line constituting each tomographic image with the representative line, and causes the computer to perform a process of displaying a color determined based on the index value at coordinates determined by the position of the tomographic image and the position of the intersection point. [Effects of the Invention]

[0008] In one respect, it is possible to provide programs that can generate images containing information about the size of the object being imaged. [Brief explanation of the drawing]

[0009] [Figure 1] This is an explanatory diagram illustrating the overview of the image generation method. [Figure 2] This is an explanatory diagram illustrating the configuration of an information processing device. [Figure 3] This is an explanatory diagram illustrating how to generate index value stripes. [Figure 4A] This is an explanatory diagram illustrating the calculation method for Example-1 of the indicator value. [Figure 4B] This is an explanatory diagram illustrating the calculation method for Example-1 of the indicator value. [Figure 5A] This is an explanatory diagram illustrating the calculation method for Example-1 of the indicator value. [Figure 5B] This is an explanatory diagram illustrating a modified example of the calculation method for the indicator value example-1. [Figure 6A] This is an explanatory diagram illustrating the calculation method for example-2 of the indicator value. [Figure 6B] This is an explanatory diagram illustrating the calculation method for example-2 of the indicator value. [Figure 7A] This is an explanatory diagram illustrating the calculation method for example-3 of the indicator value. [Figure 7B] This is an explanatory diagram illustrating the calculation method for example-3 of the indicator value. [Figure 8]It is an explanatory diagram for explaining the calculation method of the example - 3 of the index value. [Figure 9A] It is an explanatory diagram for explaining the conversion method between an angle and a length. [Figure 9B] It is an explanatory diagram for explaining the conversion method between an angle and a length. [Figure 9C] It is an explanatory diagram for explaining the conversion method between an angle and a length. [Figure 10] It is an explanatory diagram for explaining the conversion method between an angle and a length. [Figure 11] It is an explanatory diagram for explaining the method of determining a representative line. [Figure 12] It is an explanatory diagram for explaining the method of determining a representative line. [Figure 13A] It is an explanatory diagram for explaining the record layout of a tomographic image DB. [Figure 13B] It is an explanatory diagram for explaining the record layout of a maximum gradient point DB. [Figure 13C] It is an explanatory diagram for explaining the record layout of a representative line DB. [Figure 14] It is an explanatory diagram for explaining the record layout of an index value DB. [Figure 15] It is a flowchart for explaining the processing flow of a program. [Figure 16] It is a flowchart for explaining the processing flow of a subroutine for calculating a provisional representative line. [Figure 17] It is a flowchart for explaining the processing flow of a subroutine for calculating a representative line. [Figure 18] It is a flowchart for explaining the processing flow of a subroutine for calculating an index value. [Figure 19] It is a flowchart for explaining the processing flow of a subroutine for calculating a first index value. [Figure 20] It is a flowchart for explaining the processing flow of a subroutine for calculating a second index value. [Figure 21] It is a flowchart for explaining the processing flow of a subroutine for calculating a third index value. [Figure 22]This is a flowchart explaining the processing flow of the display subroutine. [Figure 23A] This is an explanatory diagram illustrating the relationship between angle and area in a modified example. [Figure 23B] This is an explanatory diagram illustrating the relationship between angle and area in a modified example. [Figure 23C] This is an explanatory diagram illustrating the relationship between angle and area in a modified example. [Figure 24] This is an example screen of Embodiment 2. [Figure 25] This is an explanatory diagram illustrating the configuration of the catheter system according to Embodiment 3. [Figure 26] This is an explanatory diagram illustrating the configuration of the information processing device in the fourth embodiment. [Modes for carrying out the invention]

[0010] [Embodiment 1] Figure 1 is an explanatory diagram illustrating the overview of the image generation method. The user, such as a physician, inserts an image acquisition catheter 28 (see Figure 25) near the affected area. The image acquisition catheter 28 is for three-dimensional scanning and can continuously acquire multiple tomographic images 41 with the scanning plane slightly changed in the axial direction.

[0011] The interval between the tomographic images 41 is indicated by interval T. In the following explanation, a set of tomographic images 41 created in a single three-dimensional scan may be referred to as a single pair of tomographic images 41.

[0012] For each tomographic image 41, a representative line 42 is determined. The representative line 42 is a closed curve determined based on, for example, the depth at which the stent is placed in the tomographic image 41, the external elastic lamina of the blood vessel into which the image acquisition catheter 28 is inserted, or the depth at which the brightness changes significantly. The representative line 42 may also be a closed curve specified by the user.

[0013] The following explanation uses the example where the representative line 42 is a circle surrounding the image acquisition catheter 28. For each tomographic image 41, the center position and radius of the representative line 42 are determined. A specific example of how the representative line 42 is determined will be described later.

[0014] An index value is calculated for each scan line 45 (see Figure 3) that forms the tomographic image 41. The index value is a constant calculated based on the characteristics of the scan line 45 and the position of the intersection point 48 (see Figure 3) on the scan line 45. A specific example of how to calculate the index value will be described later.

[0015] An index value stripe 43 can be created by assigning the index value calculated for each scan line 45 to the intersection 48 of the scan line 45 and the representative line 42. The index value stripe 43 is a virtual band formed by cutting the representative line 42, which is colored based on the index value, at one point on the lower side of Figure 1, i.e., at a scanning angle of 180 degrees, and then stretching it out. In areas where the density of intersection 48 is low, interpolation processing is performed using known interpolation methods such as linear interpolation.

[0016] The index value stripe 43 has the same width as the spacing T between the tomographic images 41 and the same length L as the representative line 42. In Figure 1, the short arrows tangent to the representative line 42 and the index value stripe 43 indicate the position directly above the image acquisition catheter 28 in the tomographic image 41, i.e., the position where the scanning angle is 0 degrees. Details of the method for generating the index value stripe 43 will be described later.

[0017] An index value image 44 is generated by arranging index value stripes 43 corresponding to each tomographic image 41, for example, so that the positions with a scanning angle of 0 degrees are aligned in a straight line. The index value image 44 is an image constructed by mapping index values ​​onto the surface of a three-dimensional object formed by representative lines 42 corresponding to each tomographic image 41, and then cutting it open at a scanning angle of 180 degrees to form a plane.

[0018] The user can intuitively understand the size of the representative line 42 and the index values ​​around the depth where the representative line 42 exists, based on the index value image 44, and perform diagnosis and treatment.

[0019] Furthermore, the position where the representative line 42 is cut open to form a single index value stripe 43 is not limited to a scanning angle of 180 degrees. For example, if the area of ​​interest, such as a diseased area, is depicted on the lower side of the tomographic image 41, cutting open the representative line 42 at the 180-degree position would cause the diseased area to be divided into upper and lower parts of the index value image 44. However, by arranging the index value stripe 43, which is formed by cutting open the representative line 42 at a scanning angle of 0 degrees, so that the positions at the 180-degree scanning angle are aligned in a straight line, an index value image 44 is generated that makes it easier to observe the area of ​​interest.

[0020] Similarly, the reference scanning angle for placing the index value stripe 43 is not limited to 0 degrees or 180 degrees. By arranging the representative line 42 so that the scanning angles corresponding to the area of ​​interest are aligned in a straight line, an index value image 44 is generated in which the area of ​​interest can be easily observed.

[0021] The scanning angle used to cut open the representative line 42, and the scanning angle used as a reference when positioning the index value stripe 43, may be selected by the user as appropriate.

[0022] Figure 2 is an explanatory diagram illustrating the configuration of the information processing device 200. The information processing device 200 comprises a control unit 201, a main memory 202, an auxiliary memory 203, a communication unit 204, a display unit 205, an input unit 206, and a bus. The control unit 201 is an arithmetic control device that executes the program of this embodiment. One or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), or multi-core CPUs are used in the control unit 201. The control unit 201 is connected to each hardware component of the information processing device 200 via the bus.

[0023] The main memory 202 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory. The main memory 202 temporarily stores information necessary during processing performed by the control unit 201 and the program currently being executed by the control unit 201.

[0024] The auxiliary storage device 203 is a storage device such as SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 203 stores a tomographic image database 36, a maximum gradient point database 37, a representative line database 38, an index value database 39, a program to be executed by the control unit 201, and various data necessary for program execution. The tomographic image database 36, the maximum gradient point database 37, the representative line database 38, and the index value database 39 may also be stored in an external mass storage device connected to the information processing device 200. The communication unit 204 is an interface for communication between the information processing device 200 and the network.

[0025] The display unit 205 is, for example, a liquid crystal display panel or an organic EL (electro-luminescence) panel. The input unit 206 is, for example, a keyboard or a mouse. The display unit 205 and the input unit 206 may be stacked to form a touch panel.

[0026] The information processing device 200 is a general-purpose personal computer, tablet, mainframe computer, virtual machine running on a mainframe computer, or quantum computer. The information processing device 200 may also be composed of hardware such as multiple personal computers or mainframe computers that perform distributed processing. The information processing device 200 may also be composed of a cloud computing system. The information processing device 200 may also be composed of hardware such as multiple personal computers or mainframe computers that operate in cooperation with each other.

[0027] Figure 3 is an explanatory diagram illustrating the method for generating the index value stripe 43. Using Figure 3, we will explain how to generate one index value stripe 43 based on a single tomographic image 41.

[0028] The tomographic image 41 can be represented in both XY-format tomographic image 411 and RT-format tomographic image 412. The XY-format tomographic image 411 is an image constructed to match the actual shape. The center of the XY-format tomographic image 411 corresponds to the center of the image acquisition catheter 28. The RT-format tomographic image 412 is an image constructed by arranging the scan lines 45 in parallel in order of scanning angle. The left edge of the RT-format tomographic image 412 corresponds to the center of the image acquisition catheter 28.

[0029] Each scan line 45 is represented by brightness data that associates the distance from the center of the image acquisition catheter 28 with the brightness of the tomographic image 41 at that distance. The method for calculating brightness data for each scan line 45 using the image acquisition catheter 28, the method for creating the tomographic image 41 using the brightness data, and the method for converting between RT format and XY format are all well known, so their explanation will be omitted.

[0030] In the following explanation, we will use as an example a case where a single tomographic image 41 is formed by scan lines 45 with scanning angles ranging from -180 degrees to 180 degrees. In the XY-type tomographic image 411, we will define the upward direction as a scanning angle of 0 degrees, clockwise as a positive scanning angle, and counterclockwise as a negative scanning angle, and continue the explanation.

[0031] The control unit 201 acquires a single tomographic image 41. Based on the tomographic image 41, the control unit 201 determines a representative line 42. In the XY-type tomographic image 411, the representative line 42 is a circle surrounding the image acquisition catheter 28, and in the RT-type tomographic image 412, it is a curve extending from the top to the bottom of the image.

[0032] In the XY-type tomographic image 411, the scan line 45 is a straight line extending radially from the center of the image, and in the RT-type tomographic image 412, it is a horizontal line. Figure 3 shows an example of a scan line 45 with a scanning angle of 90 degrees. In both the XY-type tomographic image 411 and the RT-type tomographic image 412, one scan line 45 intersects with the representative line 42 at one point. Therefore, there is one intersection point 48 for each scan line 45.

[0033] The control unit 201 calculates an index value for each scan line 45 based on the procedure described later. An example of an index value is shown in the angle-index value graph 54. The horizontal axis of the angle-index value graph 54 is the scan angle θ of each scan line 45, with a minimum value of -180 degrees and a maximum value of 180 degrees. The vertical axis of the scan line 45 is the index value. As will be described later, there are multiple definitions of the index value. The unit of the index value differs depending on the definition.

[0034] The control unit 201 converts the horizontal axis of the angle-index value graph 54 to the arc length of the representative line 42. A position-index value graph 55, which schematically illustrates the converted state, is shown. The horizontal axis of the position-index value graph 55 is the arc length of the representative line 42 from the scanning angle 0 degrees position to the scanning angle θ position. The vertical axis of the position-index value graph 55 is the index value. Details of the conversion method will be described later.

[0035] The control unit 201 generates index value stripes 43 by assigning colors to the magnitude of the index value at each position on the horizontal axis of the position-index value graph 55. The colors can be monochrome (from white to black) or in color. Specific examples of color assignment will be described later.

[0036] Following the above procedure, the control unit 201 generates an index value stripe 43 based on each of the tomographic images 41 that constitute a set of tomographic images 41. Based on the multiple index value stripes 43, the control unit 201 generates an index value image 44 as described with reference to Figure 1.

[0037] [Example of indicator value - 1] This section explains an example of an index value calculated using a Gaussian function. Figures 4A and 4B are explanatory diagrams illustrating the calculation method for example-1 of the index value. Figure 4A shows a luminance graph 59 that displays the luminance data contained in a single scan line 45. The horizontal axis of the luminance graph 59 represents the distance from the center of the image acquisition catheter 28. In the following explanation, the distance from the center of the image acquisition catheter 28 is indicated by the number of data points on the luminance graph 59. That is, the unit of the horizontal axis of the luminance graph 59 is the number of data points.

[0038] The vertical axis of the luminance graph 59 represents luminance. The unit of the vertical axis is the luminance value normalized to an integer from 0 to 255. In the following explanation, the x-th luminance data point from the center of the image acquisition catheter 28 will be denoted as B(x).

[0039] The horizontal axis R1 indicates the position where the scan line 45, displayed on the luminance graph 59, intersects with the representative line 42. b is a constant. The constant b is defined, for example, by equation (1).

[0040]

number

[0041] The control unit 201 may accept user instructions to change Mag. Note that equation (1) is an example of the definition of the constant b. The constant b may be a value specified by the user as appropriate. Note that in Figure 4A, data points are omitted from the illustration for regions where the horizontal axis is less than (R1-b / 2).

[0042] Figure 4B shows a Gaussian function graph 51 with a mean of R1 and a standard deviation of b. The horizontal axis of the Gaussian function graph 51 represents the distance from the center of the image acquisition catheter 28, similar to the brightness graph 59 shown in Figure 4A. The vertical axis of the Gaussian function graph 51 is the Gaussian function. The value of the Gaussian function N(x) corresponding to the x-th data from the center of the image acquisition catheter 28 is calculated by equation (2).

[0043]

number

[0044] The peak position of the Gaussian function graph 51 corresponds to the position of the intersection point 48 between the scan line 45 and the representative line 42. The mean R1 and standard deviation b are examples of the first statistic.

[0045] Figure 5A is an explanatory diagram illustrating the calculation method for example-1 of the index value. Figure 5A is a graph showing the luminance shown in Figure 4A weighted by the Gaussian function shown in Figure 4B. The horizontal axis of Figure 5A represents the distance from the center of the image acquisition catheter 28, similar to the luminance graph 59 shown in Figure 4A. The vertical axis of Figure 5A represents the weighted luminance. The weighted luminance Bw(x) corresponding to the x-th data from the center of the image acquisition catheter 28 is calculated by equation (3). Bw(x)=N(x)·B(x) ‥‥‥ (3)

[0046] The average value of the weighted brightness Bw(x) in the calculation range 47, where the distance from the center of the image acquisition catheter 28 is greater than or equal to (R1-b / 2) and less than or equal to the maximum value of the image acquisition range, is used as the index value. The average value is, for example, the arithmetic mean. The average value may also be the geometric mean or the harmonic mean, etc. The control unit 201 calculates the index value for each scan line 45. In the following description, the average value of the weighted brightness Bw(x) in the calculation range 47 may be referred to as the first index value.

[0047] Any statistical measure, such as the standard deviation or variance of the weighted luminance Bw(x) in the calculation range 47, may be used as the index value. A multiple or power of the statistical measure may also be used as the first index value.

[0048] [Differentiation Example 1] Figure 5B is an explanatory diagram illustrating a modified example of the calculation method for the index value example-1. In this modified example, the calculation range 47 is the range where the distance from the center of the image acquisition catheter 28 is greater than or equal to (R1-b / 2) and less than or equal to (R1+b). The control unit 201 calculates the average value of the weighted brightness Bw(x) in the calculation range 47 and uses it as the first index value.

[0049] According to this modified method, a first index value that is less affected by areas far from the image acquisition catheter 28 can be calculated and used. However, if the image acquisition catheter 28 is an IVUS catheter, artifacts such as multiple reflections may occur outside the scanning area if there are reflectors with strong acoustic impedance, such as calcified lesions or stents, in the scanning area. By making the influence of areas far from the image acquisition catheter 28 less significant, the effects of artifacts such as multiple reflections can be avoided. Note that the calculation range 47 is not limited to the ranges shown in Figures 5A and 5B.

[0050] For example, the calculation range 47 may be in the range of (R1±b / 2), (R1±b / 4), or (R1±b) for the distance from the center of the image acquisition catheter 28. The calculation range 47 may also be in the range of R1 or greater and (R1+b / 2) or less for the distance from the center of the image acquisition catheter 28, or (R1-b / 2) or greater and R1 or less for the distance from the center of the image acquisition catheter 28.

[0051] As illustrated above, by defining the calculation range 47 to include the vicinity of the intersection point 48 between the scan line 45 and the representative line 42, an index value that strongly reflects the information near the representative line 42 in the tomographic image 41 can be calculated.

[0052] [Example of indicator value - 2] This section explains an example of an index value calculated using the Rayleigh distribution. Parts common to Example 1 of the index value will be omitted from the explanation.

[0053] Figures 6A and 6B are explanatory diagrams illustrating the calculation method for example-2 of the index value. Figure 6A shows a brightness graph 59 that shows the brightness of a single scan line 45. The horizontal axis of the brightness graph 59 represents the distance from the center of the image acquisition catheter 28. The vertical axis of the brightness graph 59 represents the brightness. The calculation range 47 is the range where the distance from the center of the image acquisition catheter 28 is greater than or equal to (R1-b / 2) and less than or equal to (R1+b).

[0054] Figure 6B shows the Rayleigh distribution graph 52 estimated from luminance data within the calculation range 47. The horizontal axis of the Rayleigh distribution graph 52 represents the distance from the center of the image acquisition catheter 28, similar to the luminance graph 59. The vertical axis of the Rayleigh distribution graph 52 represents the probability density function of the Rayleigh distribution. The probability density function PL(x) of the Rayleigh distribution corresponding to the x-th data point from the center of the image acquisition catheter 28 is calculated by equation (4).

[0055]

number

[0056] The expected value of the Rayleigh distribution shown in equation (4) is given by equation (5), and its variance is given by equation (6).

[0057]

number

[0058] The maximum likelihood estimate of the parameter σ, σg, is calculated using equation (7) with luminance data within the calculation range 47.

[0059]

number

[0060] The control unit 201 calculates an index value for each scan line 45 using equation (8). In the following explanation, the index value calculated using equation (8) may be referred to as the second index value.

[0061]

number

[0062] As is clear from equations (6) and (8), the second index value shown in equation (8) is the variance of the Rayleigh distribution calculated based on the maximum likelihood estimate of the parameter σ. However, the second index value is not limited to the variance of the Rayleigh distribution.

[0063] For example, the standard deviation, which is the square root of the variance calculated by equation (8), may be used as the second indicator. The expected value of the Rayleigh distribution shown in equation (5), calculated based on the maximum likelihood estimate of the parameter σ, may be used as the second indicator value. The maximum likelihood estimate of the parameter σ, σg, shown in equation (7), may be used as the second indicator value. Values ​​calculated based on these values ​​may be used as the second indicator value.

[0064] [Example of indicator value - 3] This section explains an example of an index value calculated using the luminance histogram 53 (see Figure 7B). Parts common to Example 1 of the index value will be omitted from the explanation.

[0065] Figures 7A and 7B are explanatory diagrams illustrating the calculation method for example-3 of the index value. Figure 7A shows a brightness graph 59 that shows the brightness of a single scan line 45. The horizontal axis of the brightness graph 59 represents the distance from the center of the image acquisition catheter 28. The vertical axis of the brightness graph 59 represents the brightness. The calculation range 47 is the range where the distance from the center of the image acquisition catheter 28 is (R1-b / 2) or more and less than or equal to the maximum value of the image acquisition range.

[0066] Figure 7B shows a luminance histogram 53 calculated based on luminance data within the calculation range 47. The horizontal axis of the luminance histogram 53 represents luminance. The vertical axis of the luminance histogram 53 represents frequency. In the following explanation, the portion of the luminance histogram 53 where the luminance is below a predetermined first threshold is referred to as the low-luminance region 531. The first threshold is, for example, 64.

[0067] The control unit 201 extracts data from the luminance data within the calculation range 47 in which the luminance is less than the first threshold. The control unit 201 calculates the mean value μL and standard deviation σL of the extracted data. The control unit 201 calculates the second threshold, which is the high-luminance threshold, based on equation (9). Second threshold = μL + 3σL (9)

[0068] Figure 8 is an explanatory diagram illustrating the calculation method for example-3 of the index value. Similar to Figure 7B, Figure 8 shows a luminance histogram 53 calculated based on luminance data within the calculation range 47. In Figure 8, the vertical axis is enlarged to show the range from 0 to 12. The horizontal axis of Figure 8 shows the second threshold calculated by equation (9). In the following explanation, the portion of the luminance histogram 53 where the luminance is above a predetermined second threshold will be referred to as the high-luminance region 532.

[0069] The control unit 201 extracts data included in the high-luminance region 532 from the luminance data within the calculation range 47. The control unit 201 calculates the standard deviation σH of the extracted data. The standard deviation σH is an example of a second statistic. In the following explanation, the standard deviation σH may be referred to as the third index value. Note that the third index value is not limited to σH.

[0070] For example, the variance of the data included in the high-luminance region 532 may be used as the third indicator value. Any statistical measure of the data included in the high-luminance region 532, such as the arithmetic mean, geometric mean, harmonic mean, median, or mode, may be used as the third indicator value.

[0071] The formula for calculating the second threshold is not limited to formula (9). The second threshold may be a predetermined constant. The calculation range 47 is not limited to a range where the distance from the center of the image acquisition catheter 28 is greater than or equal to (R1-b / 2) and less than or equal to the maximum value of the image acquisition range. For example, the calculation range 47 may be a range where the distance from the center of the image acquisition catheter 28 is greater than or equal to (R1-b / 2), (R1+b), (R1±b / 2), (R1±b / 4), or (R1±b). The calculation range 47 may be a range where the distance from the center of the image acquisition catheter 28 is greater than or equal to R1 and less than or equal to (R1+b / 2), or greater than or equal to (R1-b / 2) and less than or equal to R1.

[0072] The calculation ranges 47 used when calculating the first, second, and third indicator values ​​may be a common range or may be different ranges from one another.

[0073] The index value may be calculated based on intermediate data obtained when converting the signal acquired by sensor 282 (see Figure 25) into luminance data. Alternatively, the index value may be calculated based on the signal itself acquired by sensor 282.

[0074] For each scan line 45, one of the following index values ​​can be selected and used: for example, a first index value, a second used value, or a third index value. In this case, the index value stripe 43 can be defined as a band of black and white shades, with the index values ​​assigned to a grayscale from black to white, and the index value image 44 can be defined as a black and white image.

[0075] Three index values—a first index value, a second usage value, or a third index value—may be used simultaneously. For example, the control unit 201 normalizes the first index value, the second usage value, and the third index value to integer values ​​from 0 to 255. The control unit 201 then assigns each of the normalized index values ​​to the R (red), G (green), and B (blue) luminances in any order that does not overlap. The combination of index value numbers and colors is arbitrary.

[0076] When assigning brightness levels to each of the RGB values ​​in this manner, the index value stripe 43 can be defined as a band colored with 24-bit so-called full color, and the index value image 44 can be defined as a full-color image.

[0077] Note that RGB is an example of the color channels that make up the color image displayed on the display unit 205. The control unit 201 may assign index values ​​to channels corresponding to any color space, such as the HLS color space or the HSV (Hue, Saturation, Value) color space, instead of RGB.

[0078] The control unit 201 may calculate four index values ​​and assign them to RGBA, which adds an alpha channel to RGB. The control unit 201 may also assign index values ​​to the four channels of CYMK (Cyan Magenta Yellow Black).

[0079] Figures 9A and 9C are explanatory diagrams illustrating the method of converting between angle and length. Figure 9A schematically shows an image in which a representative line 42 is superimposed on an XY-type tomographic image 411. Of the scan lines 45 that constitute the XY-type tomographic image 411, two scan lines 45, the first scan line 451 and the second scan line 452, are shown in Figure 9A. In the XY-type tomographic image 411, the intersection point 48 between the reference line extending upward from the center of the image acquisition catheter 28 and the representative line 42 is indicated as the reference intersection point 489. The reference intersection point 489 is an example of a reference point defined on the representative line 42.

[0080] The first scan line 451 is located at position θ1 clockwise from the reference line. The intersection point 48 of the first scan line 451 and the representative line 42 is denoted as the first intersection point 481. The length from the reference intersection point 489 to the first intersection point 481 along the representative line 42 is denoted as L1.

[0081] Similarly, the second scan line 452 is located at a position θ2 counterclockwise from the reference line. The intersection point 48 of the second scan line 452 and the representative line 42 is denoted as the second intersection point 482. The length from the reference intersection point 489 to the second intersection point 482 along the representative line 42 is denoted as L2.

[0082] Figure 9B shows the angle-index value graph 54. The index value corresponding to the first scan line 451 is shown as A1, and the index value corresponding to the second scan line 452 is shown as A2. Figure 9C shows the position-index value graph 55. In Figure 9A, the center of the representative line 42 is located to the upper right of the center of the image acquisition catheter 28. Therefore, in the position-index value graph 55, the horizontal axis position corresponding to the reference intersection 489 is located to the left of the center of the horizontal axis. In the position-index value graph 55, the distance between the minimum and maximum values ​​on the horizontal axis is the length L of the index value stripe 43.

[0083] Figure 10 is an explanatory diagram illustrating the method for converting between angles and lengths. Using Figure 10, we will explain how to convert the angle-index value graph 54 shown in Figure 9B to the position-index value graph 55 shown in Figure 9C.

[0084] Figure 10 shows a magnified view of the upper right portion of Figure 9A. The distance between the center of the image acquisition catheter 28 and the first intersection 481 is indicated by R11. The dashed line 453 is the scan line 45 adjacent to the first scan line 451. The intersection 48 of the third scan line 453 and the representative line 42 is indicated by the third intersection 483.

[0085] The angle between the first scan line 451 and the third scan line 453 is denoted by Δθ. Δθ is constant for all scan lines 45 that constitute the XY-type tomographic image 411. For example, if a single tomographic image 41 is composed of 512 scan lines 45, Δθ is approximately 0.7 degrees. In Figure 10, Δθ is schematically shown as a large angle. The length from the first intersection 481 to the third intersection 483 along the representative line 42 is denoted by ΔL.

[0086] Since Δθ is a small angle, ΔL can be approximated by equation (10). ΔL=R11×sinΔθ ‥‥‥ (10) Similarly, the arc length ΔLn between the nth scan line 45 and the (n+1)th scan line 45 can be approximated by equation (11). ΔLn=R1n×sinΔθ ‥‥‥ (11) R1n is the distance between the intersection point 48 of the nth scan line 45 and the representative line 42 and the center of the image acquisition catheter 28.

[0087] By sequentially moving the scan line 45 from the reference line to the first scan line 451 and adding ΔL as shown in equation (10), the length L1 along the representative line 42 can be calculated. Through the above process, the control unit 201 can generate a position-index value graph 55 based on the angle-index value graph 54. By converting the index values ​​shown on the vertical axis of the position-index value graph 55 into colors as described above, the control unit 201 can generate a single index value stripe 43 corresponding to a single tomographic image 41.

[0088] [Decision on Representative Line - 1] The control unit 201 generates a circle passing through three points specified by the user on the XY-format tomographic image 411, for example, and uses it as the representative line 42. The control unit 201 may also superimpose a template showing the circle onto the XY-format tomographic image 411, accept changes to the position and radius by the user, and use the circle confirmed by the user as the representative line 42. In addition, the control unit 201 may accept the user's specification of the representative line 42 through any user interface.

[0089] [Decision on the representative line - 2] Figures 11 and 12 are explanatory diagrams illustrating the method for determining the representative line 42. Using Figures 11 and 12, an example of how the control unit 201 automatically determines the representative line 42 is illustrated.

[0090] The control unit 201 divides the RT-type tomographic image 412 into, for example, 25 blocks 46 using lines parallel to the R-axis. If a single RT-type tomographic image 412 consists of 512 scan lines 45, then each block 46 contains 20 or 21 scan lines 45.

[0091] Moving to Figure 12, the relationship between multiple tomographic images 41 acquired by three-dimensional scanning is explained. Hatching indicates the tomographic image 41 being processed. The control unit 201 processes a group of tomographic images 41, which is the tomographic image 41 being processed combined with K tomographic images 41 before and after it. In the following explanation, K is 3, and the control unit 201 processes a total of 7 tomographic images 41, from the (n-3)th to the (n+3)th tomographic image 41.

[0092] In Figure 11, the case where there are eight blocks 46 is schematically shown. The blocks are labeled as 1st block 461 to 8th block 468, starting from the side with a scanning angle of -180 degrees. The control unit 201 calculates the maximum gradient point G for each block 46. The maximum gradient point G is the position where the change in brightness in the R direction is greatest, i.e., the gradient is greatest.

[0093] The gradient of brightness at each point on the RT-type tomographic image 412 corresponding to coordinate z can be calculated by equation (12). Equation (12) represents a convolution operation in which the Sobel operator F is applied to a plane obtained by cutting multiple RT-type tomographic images 412 acquired by three-dimensional scanning in the direction of the scanning angle θ.

[0094]

number

[0095] The Sobel operator F used in equation (12) is an operator used for horizontal contour detection. As mentioned above, an example of the Sobel operator used when processing seven tomographic images 41 in combination is shown in equation (13). The element in row j and column k of equation (13) is F(k,j) in equation (12).

[0096]

number

[0097] When using the 7x7 Sobel operator F shown in equation (13), w in equation (12) is 3, which is the kernel half-width of the Sobel operator F. Note that the Sobel operator F is not limited to equation (13). Any Sobel operator F capable of horizontal contour detection can be used. When processing by combining K tomographic images 41, the Sobel operator F is a square matrix with a height of (2K+1) and a width of (2K+1).

[0098] The control unit 201 extracts the coordinate (r,θ,z) for each block 46 of the RT-type tomographic image 412 corresponding to each coordinate z, where g(r,θ,z) calculated by equation (12) is maximized. The extracted coordinate (r,θ,z) is the maximum gradient point G of block 46. Figure 11 schematically shows the maximum gradient points G1 to G8 extracted for each block 46 from the first block 461 to the eighth block 468 on the RT-type tomographic image 412.

[0099] The control unit 201 can convert the RT-type tomographic image 412 to an XY-type tomographic image 411 using known coordinate transformations. In Figure 11, the points from the point of maximum gradient G1 to the point of maximum gradient G8 are arranged in a roughly circular shape on the XY-type tomographic image 411.

[0100] The control unit 201 calculates the coordinates of the centroid C from the point of maximum gradient G1 to the point of maximum gradient G8 on the XY-type tomographic image 411. Since the method for calculating the coordinates of the centroid C from the coordinates of multiple points on a plane is well known, a detailed explanation is omitted. The control unit 201 calculates the distance D between the calculated centroid C and each of the points of maximum gradient G. The control unit 201 calculates the median value Dc, which is a representative value of the calculated distance D.

[0101] Through the above process, a provisional representative line 422 is determined, whose center is the centroid C of the maximum gradient point G, and whose radius is the median Dc of the distance D between the centroid C and each maximum gradient point G. The mean value may be used as the representative value of the distance D.

[0102] Furthermore, noise and artifacts within the tomographic image 41 may result in an inappropriate position for the extracted maximum gradient point G. To avoid adverse effects caused by the inclusion of inappropriate maximum gradient points G, it is desirable for the control unit 201 to perform a process to remove outliers of the maximum gradient point G.

[0103] For example, in the second and subsequent tomographic images 41, the control unit 201 calculates the distance D between the centroid C and each maximum gradient point G, and then removes the distance D whose difference from the representative value Dc of the previous tomographic image 41 exceeds a predetermined threshold. After that, the control unit 201 calculates the representative value Dc of the remaining distance D. Through the above process, the control unit 201 can determine a provisional representative line 422 that is less affected by inappropriate maximum gradient points G.

[0104] Moving to Figure 12, we will explain the process after determining the provisional representative line 422 for each tomographic image 41. In the following explanation, the center of the provisional representative line 422 in the nth tomographic image 41 will be denoted as Cn, and the radius of the provisional representative line 422 will be denoted as Dn.

[0105] The control unit 201 calculates the moving average values ​​of the coordinates of the center C and the radius D for the K preceding and succeeding tomographic images 41 for the nth tomographic image 41. For example, if K is 3, the control unit 201 calculates the average value of the coordinates of the center C, Cnavg, and the average value of the radius D, Dnavg, for the provisional representative lines 422 of a total of 7 tomographic images 41 from the (n-3)th to the (n+3)th tomographic image 41. The control unit 201 uses the calculated Cnavg and Dnavg for the center and radius of the representative line 42 in the nth tomographic image 41.

[0106] In the above explanation, we have used the same number of tomographic images 41 when calculating the gradient using equations (12) and (13), and when calculating Cnavg and Dnavg, as an example. However, different numbers of tomographic images 41 may be used for the two calculations. For example, the control unit 201 may calculate the gradient at K=3 and Cnavg and Dnavg at K=7.

[0107] For fault images 41 located near both ends of a pair of fault images 41, for example, the center and radius of the provisional representative line 422 are used as the center and radius of the representative line 42. For fault images 41 located near both ends of a pair of fault images 41, the number of fault images 41 up to the outermost fault image 41 may be used as the value of K.

[0108] Through the above processing, representative lines 42 are created that avoid the influence of noise and artifacts in the tomographic image 41 and that can construct a smoothly connected three-dimensional shape in the three-dimensional space formed by a set of tomographic images 41.

[0109] The methods for determining the representative line 42 described in "Determination of Representative Line-1" and "Determination of Representative Line-2" are illustrative and not limiting. For example, the representative line 42 may be determined on the XY-type tomographic image 411 using a method such as pattern matching.

[0110] Figure 13A is an explanatory diagram illustrating the record layout of the tomographic image DB36. The tomographic image DB36 is a database in which tomographic images 41 created by three-dimensional scanning are recorded. The tomographic image DB36 has a 3D scanning ID field, a fault number field, and a tomographic image field. The tomographic image field has an XY format field and an RT format field.

[0111] The 3D scan ID field records the 3D scan ID assigned to each three-dimensional scan. The tomography number field records a number indicating the sequence of the tomographic images 41 created in a single three-dimensional scan. The XY format field records the XY format tomographic image 411. The RT format field records the RT format tomographic image 412. The tomographic image DB 36 has one record for each tomographic image 41.

[0112] Furthermore, only RT-format tomographic images 412 are recorded in the tomographic image DB36, and the control unit 201 may create XY-format tomographic images 411 by coordinate transformation as needed. Instead of the tomographic image DB36 in which the tomographic images 41 are recorded, the control unit 201 may create the necessary tomographic images 41 as needed based on a database containing data such as sound line data, which is recorded prior to the creation of the tomographic images 41.

[0113] The tomographic image DB36 may have a field that records the distance from the starting point of the three-dimensional scan to the position where the tomographic image 41 was acquired, instead of using a tomographic number field. The control unit 211 can obtain the distance from the starting point of the three-dimensional scan to the position where each tomographic image 41 was acquired, for example, from the MDU289 (see Figure 25).

[0114] Figure 13B is an explanatory diagram illustrating the record layout of the Maximum Gradient Point DB37. The Maximum Gradient Point DB37 is a database that records information about the Maximum Gradient Point G described using Figure 11. The Maximum Gradient Point DB37 has a 3D scan ID field, a fault number field, a distance representative value field, a block number field, a maximum gradient location field, a distance field, and a flag field. The Maximum Gradient Location field has an R field and a T field.

[0115] The 3D scan ID field records the 3D scan ID assigned to each three-dimensional scan. The tomographic number field records the number indicating the order of the tomographic images 41 created in a single three-dimensional scan. The distance representative value field records the representative value of the distance D between the centroid C and each maximum gradient point G, as explained using Figure 11.

[0116] The block number field records the block number 46. The R field records the R coordinate of the point of maximum gradient G within block 46, i.e., the distance from the center of the image acquisition catheter 28. The T field records the T coordinate of the point of maximum gradient G within block 46, i.e., the scanning angle.

[0117] The distance field records the distance D between the centroid C and the maximum gradient point G. The flag field records a flag indicating whether the maximum gradient point G is an outlier. "1" means that the maximum gradient point G is not an outlier and will be used to calculate the representative value recorded in the distance representative value field. "0" means that the maximum gradient point G is an outlier and will not be used to calculate the representative value recorded in the distance representative value field. The maximum gradient point DB37 has one record for each block 46.

[0118] Figure 13C is an explanatory diagram illustrating the record layout of the representative line DB38. The representative line DB38 is a database that records information about the representative line 42. The representative line DB38 has a 3D scan ID field, a fault number field, a provisional representative line field, and a representative line field. The provisional representative line field and the representative line field each have a centroid field and a distance representative value field. The centroid fields of the provisional representative line field and the representative line field each have an X field and a Y field.

[0119] The 3D scan ID field records the 3D scan ID assigned to each three-dimensional scan. The tomographic number field records the number indicating the order of the tomographic images 41 created in a single three-dimensional scan. The centroid field of the provisional representative line field records the X and Y coordinates of the centroid C of the maximum gradient point G, as explained using Figure 11. The distance representative value field of the provisional representative line field records the representative value Dc of the distance D.

[0120] The centroid field of the representative line field records the moving average values ​​of the X and Y coordinates of centroid Cnavg, which is the moving average of centroid C as explained using Figure 12. The distance representative value field of the representative line field records the distance representative value Dnavg, which is the moving average of distance representative value D as explained using Figure 12. The representative line DB38 has one record for each tomographic image 41.

[0121] Figure 14 is an explanatory diagram illustrating the record layout of the index value DB39. The index value DB39 is a database that records index values ​​corresponding to each scan line 45. The index value DB39 has a 3D scan ID field, a tomography number field, a scan line number field, an intersection field, and an index value field.

[0122] The intersection field has an R1 field and an L field. The index value field has a first index value field, a second index value field, and a third index value field. Note that the index value field does not have to have subfields. The index value field may have four or more subfields.

[0123] The 3D scan ID field records the 3D scan ID assigned to each three-dimensional scan. The tomographic number field records the number indicating the order of the tomographic images 41 created in a single three-dimensional scan. The scan line number field records the scan line number. The R1 field records the distance R1 between the center of the image acquisition catheter 28 and the intersection 48. The L field records the length L from the reference intersection 489 to the intersection 48 along the representative line 42. The first index value field, second index value field, and third index value field record the first index value, second index value, and third index value, respectively.

[0124] Figure 15 is a flowchart illustrating the program's processing flow. The control unit 201 acquires a single tomographic image 41 from the tomographic image DB 36 (step S501). The control unit 201 then starts a subroutine for calculating the provisional representative line (step S502). The subroutine for calculating the provisional representative line calculates the center and radius of the provisional representative line 422, as explained using Figure 11. The processing flow of the subroutine for calculating the provisional representative line will be described later.

[0125] The control unit 201 determines whether it has finished processing the tomographic images 41 included in a set of tomographic images 41 (step S504). If it determines that it has not finished (NO in step S504), the control unit 201 returns to step S501. If it determines that it has finished (YES in step S504), the control unit 201 starts the representative line calculation subroutine (step S505). The representative line calculation subroutine calculates the center and radius of the representative line 42 based on the moving average value of the provisional representative line 422, as explained using Figure 12. The processing flow of the representative line calculation subroutine will be described later.

[0126] The control unit 201 selects a tomographic image 41 for which to calculate an index value (step S506). The tomographic image 41 selected in step S506 is one of the tomographic images 41 acquired in the loop from step S501 to step S504.

[0127] The control unit 201 selects one of the scan lines 45 that make up the tomographic image 41 selected in step S506 (step S507). Each time step S507 is executed, the control unit 201 selects one scan line 45 at a time clockwise from the reference intersection 489 as described using Figures 9A and 10.

[0128] The control unit 201 starts the subroutine for calculating the index value (step S508). The subroutine for calculating the index value calculates the index value based on a single scan line 45. The processing flow of the subroutine for calculating the index value will be described later.

[0129] The control unit 201 extracts the record from the index value DB39 that corresponds to the scan line 45 selected in step S507. The control unit 201 records the distance R1, length L, and index value calculated by the index value calculation subroutine in the R1 field, L field, and index value field, respectively (step S510).

[0130] The control unit 201 determines whether or not it has finished processing the scan lines 45 that constitute the tomographic image 41 acquired in step S506 (step S511). If it determines that it has not finished (NO in step S511), the control unit 201 returns to step S507. If it determines that it has finished (YES in step S511), the control unit 201 determines whether or not it has finished processing a set of tomographic images 41 (step S512).

[0131] If it is determined that the process is not finished (NO in step S512), the control unit 201 returns to step S506. If it is determined that the process is finished (YES in step S512), the control unit 201 starts the display subroutine (step S513). The display subroutine displays the index value image 44 based on the index value DB39. The processing flow of the display subroutine will be described later. After that, the control unit 201 terminates the process.

[0132] Figure 16 is a flowchart illustrating the processing flow of the provisional representative line calculation subroutine. The provisional representative line calculation subroutine calculates the center and radius of the provisional representative line 422, as explained using Figure 11, based on a single tomographic image 41.

[0133] The control unit 201 divides the tomographic image 41 being processed into a predetermined number of blocks 46, as explained using Figure 11 (step S521). The control unit 201 selects one block 46 (step S522). The control unit 201 calculates the coordinates of the point G with the maximum gradient within the block 46 (step S523). Specifically, the control unit 201 calculates the luminance gradient at each position in the block 46 based on equation (12). The control unit 201 extracts the coordinates of the position where the calculated luminance gradient is maximum.

[0134] The control unit 201 determines whether or not the processing of the block 46 divided in step S521 has been completed (step S524). If it determines that the processing has not been completed (NO in step S524), the control unit 201 returns to step S522.

[0135] If it is determined that the process has ended (YES in step S524), the control unit 201 calculates the coordinates of the centroids C of the multiple maximum gradient points G (step S525). The control unit 201 calculates the distance D between each maximum gradient point and its centroid C (step S526).

[0136] The control unit 201 determines whether the tomographic image 41 being processed is the first of a set of tomographic images 41 (step S527). If it is determined to be the first (YES in step S527), the control unit 201 calculates a representative value Dc of the distance D calculated in step S526 (step S528). The representative value Dc is, for example, the median.

[0137] If it is determined that it is not the first image (NO in step S527), the control unit 201 removes outliers from the distance D calculated in step S526 (step S531). Outliers are, for example, distances D whose difference from the representative value Dc calculated for the previous tomographic image 41 exceeds a predetermined threshold. The control unit 201 calculates a representative value Dc for the distance D (step S532).

[0138] After step S528 or step S532 is completed, the control unit 201 records the centroid C calculated in step S525 and the representative value Dc calculated in step S528 or step S532 on the representative line DB38 (step S533).

[0139] Specifically, the control unit 201 extracts the record from the representative line DB38 that corresponds to the tomographic image 41 being processed. The control unit 201 records the X and Y coordinates of the centroid C calculated in step S525 in the centroid field of the provisional representative line field of the extracted record. The control unit 201 records the representative value Dc calculated in step S528 or step S532 in the representative value field of the provisional representative line field of the extracted record. After that, the control unit 201 terminates the process.

[0140] Figure 17 is a flowchart illustrating the processing flow of the representative line calculation subroutine. The control unit 201 obtains information recorded in the provisional representative line field for a set of 41 fault images from the representative line DB 38 (step S541). The control unit 201 selects the fault number of the 41 fault image to be processed (step S542). For example, each time step S542 is executed, the control unit 201 increments the fault number by 1 sequentially from 1.

[0141] The control unit 201 determines whether the fault number selected in step S542 represents the vicinity of both ends of a set of tomographic images 41 (step S543). Specifically, as explained using Figure 12, when using the moving average value of the preceding and succeeding K frames of the tomographic images 41, the control unit 201 determines that the selected fault number represents the vicinity of both ends if the fault image number selected in step S542 is less than or equal to K, and is greater than or equal to (total number of tomographic images 41 - K).

[0142] If it is determined that the area is near both ends (YES in step S543), the control unit 201 records the same information as the information recorded in the provisional representative line field in the representative line field (step S544). If it is determined that the area is not near both ends (NO in step S543), the control unit 201 uses data from a predetermined number of fault numbers before and after the fault number being processed to calculate the centroid C and the moving average of the representative value Dc, respectively (step S545).

[0143] The control unit 201 records the calculated moving average in the representative line field of the representative line DB38 (step S546). Specifically, the control unit 201 extracts the record from the representative line DB38 that corresponds to the tomographic image 41 being processed. The control unit 201 records the moving average of the centroid C calculated in step S545 in the centroid field of the representative line field of the extracted record. The control unit 201 records the moving average of the representative value Dc calculated in step S545 in the representative value field of the representative line field of the extracted record.

[0144] The control unit 201 determines whether or not it has finished processing the temporary representative line 422 acquired in step S541 (step S547). If it determines that it has not finished (NO in step S547), the control unit 201 returns to step S542. If it determines that it has finished (YES in step S547), the control unit 201 terminates the processing.

[0145] Figure 18 is a flowchart illustrating the processing flow of the subroutine for calculating the index value. The subroutine for calculating the index value calculates the index value based on a single scan line 45.

[0146] The control unit 201 calculates the coordinates of the intersection point 48 between the scan line 45 being processed and the representative line 42 (step S551). Here, the control unit 201 calculates the coordinates of the intersection point 48 in both the XY coordinate system and the RT coordinate system. The control unit 201 calculates ΔL, which is the arc length from the intersection point 48 on the previous scan line 45, using equation (10) (step S552).

[0147] The control unit 201 calculates the arc length L from the reference intersection point 489, which is the sum of the ΔL calculated in the past (step S553). For scan lines 45 with a scanning angle of 180 degrees or more, the control unit 201 calculates the arc length L by subtracting the sum of ΔL from the circumference of the representative line 42. Step S553 realizes the process of converting the horizontal axis of the angle-index value graph 54, as explained using Figure 3, to calculate the position-index value graph 55.

[0148] The control unit 201 acquires scan line data for the scan line 45 being processed (step S554). The control unit 201 starts a subroutine for calculating the first index value (step S555). The subroutine for calculating the first index value is the subroutine that calculates the first index value as explained using Figures 4A, 4B, and 5A. The processing flow of the subroutine for calculating the first index value will be described later.

[0149] The control unit 201 starts the subroutine for calculating the second index value (step S556). The subroutine for calculating the second index value is the subroutine for calculating the second index value as explained using Figures 6A and 6B. The processing flow of the subroutine for calculating the second index value will be described later.

[0150] The control unit 201 starts a subroutine for calculating the third index value (step S557). The subroutine for calculating the third index value is the subroutine for calculating the third index value as explained using Figures 7A, 7B, and 8. The processing flow of the subroutine for calculating the third index value will be described later. After that, the control unit 201 terminates its processing.

[0151] Figure 19 is a flowchart illustrating the processing flow of the subroutine for calculating the first indicator value. The subroutine for calculating the first indicator value is the subroutine that calculates the first indicator value as explained using Figures 4A, 4B, and 5A.

[0152] The control unit 201 generates a Gaussian function as described using Figure 4B and equation (2) (step S561). The control unit 201 calculates the weighted luminance Bw, which is weighted by the Gaussian function, as described using equation (3) (step S562). The control unit 201 extracts data from the weighted luminance Bw that falls within the calculation range 47 as described using Figure 5A (step S563).

[0153] The control unit 201 calculates a first index value based on the data included in the calculation range 47 (step S564). As mentioned above, the first index value is, for example, the average value of the weighted luminance Bw. After that, the control unit 201 terminates the process.

[0154] Figure 20 is a flowchart illustrating the processing flow of the subroutine for calculating the second indicator value. The subroutine for calculating the second indicator value is the subroutine that calculates the second indicator value as explained using Figures 6A and 6B.

[0155] The control unit 201 extracts data from the luminance data that falls within the calculation range 47 described using Figure 6A (step S571). The control unit 201 calculates the second index value based on equation (8) (step S572). After that, the control unit 201 terminates the process.

[0156] Figure 21 is a flowchart illustrating the processing flow of the subroutine for calculating the third indicator value. The subroutine for calculating the third indicator value is the subroutine for calculating the third indicator value as explained using Figures 7A, 7B, and 8.

[0157] The control unit 201 extracts data from the luminance data that falls within the calculation range 47 described using Figure 7A (step S581). The control unit 201 calculates the frequency distribution of the extracted data (step S582). Since the process of calculating the frequency distribution based on a large amount of data is well known, a detailed explanation is omitted.

[0158] The control unit 201 extracts data from the low-luminance region 531 as described using Figure 7B. The control unit 201 calculates two statistics from the extracted data: the mean value μL and the standard deviation σL (step S583). The control unit 201 calculates a second threshold, which is the threshold on the high-luminance side, based on equation (9) (step S584).

[0159] The control unit 201 extracts data for the high-luminance region 532, as explained using Figure 8, from the luminance data. The control unit 201 calculates a third index value, which is the standard deviation σH of the extracted data (step S585). After that, the control unit 201 terminates the process.

[0160] Figure 22 is a flowchart illustrating the processing flow of the display subroutine. The display subroutine displays the index value image 44 based on the index value DB39.

[0161] The control unit 201 obtains a first index value for a set of scan lines 45 that constitute a tomographic image 41 from the first index value field of the index value DB39. The control unit 201 normalizes the obtained first index value to an integer between 0 and 255 (step S591).

[0162] The control unit 201 obtains the second index value for the scan lines 45 that constitute a set of tomographic images 41 from the second index value field of the index value DB39. The control unit 201 normalizes the obtained second index value to an integer between 0 and 255 (step S592).

[0163] The control unit 201 obtains the third index value for the scan lines 45 that constitute a set of tomographic images 41 from the third index value field of the index value DB39. The control unit 201 normalizes the obtained third index value to an integer between 0 and 255 (step S593).

[0164] The control unit 201 converts the normalized first to third index values ​​for each scan line 45 into a color code (step S594). Specifically, if the index value assigned to R (red) is AR, the index value assigned to G (green) is AG, and the index value assigned to B (blue) is AB, then the color code can be expressed as RGB(AR,AG,AB).

[0165] The control unit 201 sets a color represented by a color code at the coordinate position determined by the length L recorded in the L field of the index value DB39 and the fault position Z obtained by integrating the fault number recorded in the fault number field with the interval T, for each scan line 45. The control unit 201 determines the color code of the coordinate position corresponding to the gap between the scan lines 45 by interpolation (step S595).

[0166] The control unit 201 displays the image generated by the color code of each coordinate position on the display unit 205 (step S596). The control unit 201 rotates, enlarges, reduces, etc., the image according to the user's instructions. The control unit 201 may also display indicators and handles on the display unit 205 that the user can use to manipulate the image. After that, the control unit 201 terminates processing.

[0167] According to this embodiment, an image containing information about the size of the scan lines 45 can be generated. By displaying an index value image 44 in a manner resembling an unfolded diagram, with emphasis on the information of the portion of the scan lines 45 near the representative line 42, users such as doctors can intuitively grasp the parts they wish to focus on.

[0168] For example, if the representative line 42 is set to a position corresponding to the lumen of a blood vessel, the length of each representative line 42 is represented by the vertical length of the index value image 44 shown in Figure 1, so the user can easily understand whether the area of ​​interest is narrowed or dilated.

[0169] By assigning three different indicators, each calculated using a different method, to R, G, and B and displaying them on a single indicator value image 44, users can grasp a large amount of information in a short time. The types of indicators used, and the combinations of indicators and colors, may be changed by the user as needed.

[0170] The control unit 201 may display the index value image 44 and the tomographic image 41 side by side on the display unit 205. If the user clicks on a point on the index value image 44, the control unit 201 may display the tomographic image 41 including the clicked position on the display unit 205.

[0171] [Modification - 2] The control unit 201 may determine the lengths L and ΔL along the tomographic image 41, as explained using Figure 10, based on the center point of the representative line 42. Specifically, the control unit 201 calculates index values ​​for each intersection point between the scan line 45 and the representative line 42, such as the first intersection point 481 and the second intersection point 482, as explained using Figure 10.

[0172] The control unit 201 calculates the center point of the representative line 42. The control unit 201 determines ΔL at regular angle intervals from the center point and generates an index value stripe 43. That is, ΔL is a constant value over the entire representative line 42.

[0173] According to this modified version, a catheter system 10 can be provided that clearly displays changes in blood vessel size due to stenosis, etc.

[0174] The angle between the first scan line 451 and the third scan line 453 is denoted by Δθ. Δθ is constant for all scan lines 45 that constitute the XY-type tomographic image 411. For example, if a single tomographic image 41 is composed of 512 scan lines 45, Δθ is approximately 0.7 degrees. In Figure 10, Δθ is schematically shown as a large angle. The length from the first intersection 481 to the third intersection 483 along the representative line 42 is denoted by ΔL.

[0175] Since Δθ is a small angle, ΔL can be approximated by equation (10). ΔL=R11×sinΔθ ‥‥‥ (10) Similarly, the arc length ΔLn between the nth scan line 45 and the (n+1)th scan line 45 can be approximated by equation (11). ΔLn=R1n×sinΔθ ‥‥‥ (11) R1n is the distance between the intersection point 48 of the nth scan line 45 and the representative line 42 and the center of the image acquisition catheter 28.

[0176] By sequentially moving the scan line 45 from the reference line to the first scan line 451 and adding ΔL as shown in equation (10), the length L1 along the representative line 42 can be calculated. Through the above process, the control unit 201 can generate a position-index value graph 55 based on the angle-index value graph 54. By converting the index values ​​shown on the vertical axis of the position-index value graph 55 into colors as described above, the control unit 201 can generate a single index value stripe 43 corresponding to a single tomographic image 41.

[0177] [Modification 3] In this modified example, area is used instead of length L along the arc of representative line 42. Figures 23A to 23C are explanatory diagrams illustrating the relationship between angle and area in this modified example.

[0178] Figure 23A schematically shows an image in which a representative line 42 is superimposed on an XY-type tomographic image 411. In this modified example, instead of the length L1 from the reference intersection 489 to the first intersection 481, the area S1 of a roughly sector enclosed by the representative line 42, the first scan line 451, and the Y-axis is used. Similarly, instead of the length L2 from the reference intersection 489 to the second intersection 482, the area S2 of a roughly sector enclosed by the representative line 42, the second scan line 452, and the Y-axis is used.

[0179] Figure 23B shows an angle-index value graph 54 similar to Figure 9B. Figure 23C shows an area-index value graph 56. The horizontal axis of Figure 23 represents the area, as explained using Figure 23A. The vertical axis of Figure 23 represents the index value.

[0180] As is clear from Figure 23A, there is a one-to-one correspondence between the scanning angle and the area. The area S1 can be calculated, for example, by multiplying the number of pixels included in the area with downward-sloping hatching in Figure 23A by the area of ​​one pixel. Therefore, the control unit 201 can convert the horizontal axis of the angle-index value graph 54 to area and create the area-index value graph 56.

[0181] By generating index value stripes 43 based on the area-index value graph 56, an index value image 44 can be generated in which index values ​​corresponding to each scan line 45 are set at coordinate positions determined by the area S and fault position Z.

[0182] [Modification - 4] In this modified example, after generating index value stripes 43 of the same length for each tomographic image 41, the stripes are stretched or compressed overall based on the values ​​relating to the representative line 42.

[0183] Specifically, in the generation method described using Figure 3, the control unit 201 does not create a position-index value graph 55, but generates index value stripes 43 based on the angle-index value graph 54. The length of the index value stripes 43 generated here corresponds to a 360-degree angle, and index value stripes 43 of the same length are generated for all tomographic images 41.

[0184] The control unit 201 calculates values ​​related to the representative line 42 for each tomographic image 41. These values ​​include, for example, the length of the representative line 42, the area inside the representative line 42, and the average diameter of the representative line 42. The control unit 201 expands or contracts the index value stripe 43 based on the values ​​related to the representative line 42. Through this procedure, the control unit 201 generates index value stripes 43 of different lengths for each tomographic image 41, as explained using Figure 1.

[0185] [Difference 5] The control unit 201 may display an index on the index value image 44 based on a value relating to the representative line 42. For example, the control unit 201 displays an index on the edge of the index value image 44 corresponding to the position of the tomographic image 41 where the average diameter of the representative line 42 is smallest. The control unit 201 may also color the index value stripe 43 corresponding to the tomographic image 41 where the average diameter of the representative line 42 is smallest with a different color than the other index value stripes 43.

[0186] [Differentiation Example 6] The control unit 201 may create multiple representative lines 42 and calculate index values. For example, the control unit 201 may create two representative lines 42: one corresponding to the vascular lumen and another corresponding to the external elastic lamina. The control unit 201 subtracts the area of ​​the representative line 42 corresponding to the vascular lumen from the area of ​​the representative line 42 corresponding to the external elastic lamina. In this way, the control unit 201 can calculate index values ​​corresponding to the plaque area for each tomographic image 41.

[0187] [Embodiment 2] Figure 24 shows an example screen of Embodiment 2. The control unit 201 sets a color corresponding to the index value at a coordinate position determined by the length L and the three-dimensional space obtained by adding a third axis U to the fault position Z. The index value image 44 is displayed in the form of a curved surface in three-dimensional space.

[0188] The third axis U is, for example, the distance from the center of the XY-format tomographic image 411 in Figure 9A, i.e., the center of the image acquisition catheter 28, to the intersection point 48. Any other parameter may also be set as the third axis U.

[0189] According to this modified example, an information processing device 200 can be provided that displays an index value image 44 that simultaneously represents a larger amount of information.

[0190] [Embodiment 3] This embodiment relates to a catheter system 10 that acquires tomographic images 41 in real time and displays index value images 44. Parts common to Embodiment 1 will not be described.

[0191] Figure 25 is an explanatory diagram illustrating the configuration of the catheter system 10 of Embodiment 3. The catheter system 10 comprises an image processing device 210, a catheter control device 27, an MDU (Motor Driving Unit) 289, and an image acquisition catheter 28. The image acquisition catheter 28 is connected to the image processing device 210 via the MDU 289 and the catheter control device 27.

[0192] The image processing device 210 comprises a control unit 211, a main memory 212, an auxiliary memory 213, a communication unit 214, a display unit 215, an input unit 216, and a bus. The control unit 211 is an arithmetic control device that executes the program of this embodiment. One or more CPUs, GPUs, or multi-core CPUs are used in the control unit 211. The control unit 211 is connected to each hardware component of the image processing device 210 via the bus.

[0193] The main memory 212 is a storage device such as SRAM, DRAM, or flash memory. The main memory 212 temporarily stores information necessary during processing performed by the control unit 211, as well as the program currently being executed by the control unit 211.

[0194] The auxiliary storage device 213 is a storage device such as SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 213 stores the tomographic image DB36, the maximum gradient point DB37, the representative line DB38, the index value DB39, the program to be executed by the control unit 211, and various data necessary for the execution of the program. The tomographic image DB36, the maximum gradient point DB37, the representative line DB38, and the index value DB39 may also be stored in an external mass storage device connected to the image processing device 210. The external mass storage device may be so-called cloud storage.

[0195] The communication unit 214 is an interface for communication between the image processing device 210 and the network. The display unit 215 is, for example, a liquid crystal display panel or an organic EL panel. The input unit 216 is, for example, a keyboard and mouse. The input unit 216 may be stacked on the display unit 215 to form a touch panel. The display unit 215 may also be a display device connected to the image processing device 210.

[0196] The image processing device 210 is a general-purpose personal computer, tablet, mainframe computer, or virtual machine running on a mainframe computer. The image processing device 210 may also be composed of multiple personal computers or mainframe computers or other hardware that perform distributed processing. The image processing device 210 may also be composed of a cloud computing system. The image processing device 210 and the catheter control device 27 may constitute a single piece of hardware.

[0197] The image acquisition catheter 28 comprises a sheath 281, a shaft 283 inserted inside the sheath 281, and a sensor 282 positioned at the tip of the shaft 283. The MDU 289 rotates and moves the shaft 283 and sensor 282 back and forth inside the sheath 281.

[0198] Sensor 282 is, for example, an ultrasonic transducer that transmits and receives ultrasonic waves, or a transmitting and receiving unit for OCT (Optical Coherence Tomography) that irradiates near-infrared light and receives reflected light. In the following description, the image acquisition catheter 28 will be described as an IVUS (Intravascular Ultrasound) catheter used to take ultrasonic tomographic images from inside the cardiovascular system.

[0199] The catheter control device 27 creates one tomographic image 41 for each rotation of the sensor 282. By rotating the sensor 282 while pulling or pushing it in the axial direction with the MDU 289, the catheter control device 27 continuously creates multiple tomographic images 41 that are approximately perpendicular to the sheath 281.

[0200] The control unit 211 sequentially acquires tomographic images 41 from the catheter control device 27 and records them in the tomographic image database 36. In this way, so-called three-dimensional scanning is performed and a set of tomographic images 41 is recorded in the tomographic image database 36.

[0201] The advancement and retraction of the sensor 282 includes both advancing and retracting the entire image acquisition catheter 28 and advancing and retracting the sensor 282 inside the sheath 281. The advancement and retraction may be performed automatically at a predetermined speed by the MDU 289 or manually by the user.

[0202] Furthermore, the image acquisition catheter 28 is not limited to a mechanical scanning method that mechanically rotates and moves forward and backward. For example, it may be an electronic radial scanning type image acquisition catheter 28 that uses a sensor 282 in which multiple ultrasonic transducers are arranged in a ring.

[0203] The control unit 211 uses the tomographic image DB36 recorded by the above operations to perform the processing described in Embodiment 1 and displays the index value image 44 on the display unit 215.

[0204] [Embodiment 4] Figure 26 is an explanatory diagram illustrating the configuration of the information processing device 200 of Embodiment 4. This embodiment relates to a configuration in which the information processing device 200 of this embodiment is realized by operating a general-purpose computer 90 in combination with a program 97. Parts common to Embodiment 1 will not be explained.

[0205] The computer 90 includes the aforementioned control unit 201, main memory 202, auxiliary memory 203, communication unit 204, display unit 205, input unit 206, and bus, in addition to a read unit 209.

[0206] Program 97 is recorded on a portable recording medium 96. The control unit 201 reads Program 97 via the reading unit 209 and saves it to the auxiliary storage device 203. The control unit 201 may also read Program 97 stored in a semiconductor memory 98, such as flash memory, implemented in the computer 90. Furthermore, the control unit 201 may download Program 97 from another server computer (not shown) connected via the communication unit 204 and a network (not shown) and save it to the auxiliary storage device 203.

[0207] Program 97 is installed as a control program for the computer 90, loaded into the main memory 202, and executed. This completes the realization of the information processing device 200 described in Embodiment 1. Program 97 in this embodiment is an example of a program product.

[0208] The technical features (constituent elements) described in each embodiment and modification are combinable with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0209] 10 Catheter Systems 200 Information Processing Devices 201 Control Unit 202 Main storage 203 Auxiliary storage device 204 Communications Department 205 Display section 206 Input section 209 Reading Unit 210 Image Processing Device 211 Control Unit 212 Main storage 213 Auxiliary storage 214 Communications Department 215 Display section 216 Input section 27 Catheter control device 28. Catheter for image acquisition 281 Sheath 282 sensors 283 Shaft 289 MDU 36 Fault Image Database 37 Maximum gradient point DB 38 Representative line DB 39 Metric Value DB 41 Fault images 411 XY format tomographic image 412 RT-type tomographic images 42 Representative line 422 Provisional Representative Line 43 Index Value Stripes 44 Metric Value Images 45 scan lines 451 First scan line 452 Second scan line 453 Third scan line 46 blocks 461 Block 1 468 Block 8 47 Calculation range 48 intersection 481 1st intersection 482 Second intersection 489 Reference intersection 51 Gaussian function graph 52 Rayleigh distribution graph 53 Luminance Histogram 531 Low-luminance area 532 High-brightness area 54 Angle-Index Value Graph 55. Position-Index Value Graph 56 Area-Index Value Graph 59 Brightness Graph 90 Computer 96 Portable recording media 97 Programs 98 Semiconductor memory

Claims

1. Multiple tomographic images are obtained by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially. A representative line was determined for each of the aforementioned fault images. The index value of the scan line is calculated within the calculation range determined based on the intersection point of each scan line constituting each of the aforementioned tomographic images and the aforementioned representative line. The coordinates determined by the position of the tomographic image and the position of the intersection point are displayed with a color determined based on the index value. A program that instructs a computer to perform a process.

2. The aforementioned representative line is a closed curve surrounding the image acquisition catheter. The program according to claim 1.

3. The aforementioned representative line is a circle surrounding the image acquisition catheter. The program according to claim 1.

4. The aforementioned coordinates are determined based on the position of the tomographic image and the values ​​relating to the representative line. The program according to claim 1.

5. The value relating to the aforementioned representative line is the length along the representative line from a reference point defined on the representative line to the intersection point. The program according to claim 1.

6. The aforementioned index value is determined based on one or more of the first index value, second index value, and third index value, which are calculated using different calculation methods. The program according to claim 1.

7. The aforementioned first index value is, The brightness values ​​in the scan line are weighted by a Gaussian function with the intersection point as the peak position. This value is calculated by averaging the weighted luminance values ​​within the aforementioned calculation range. The program according to claim 6.

8. The second index value is a function of the maximum likelihood estimate of the parameters of the Rayleigh distribution estimated based on the luminance values ​​in the calculation range of the scan lines. The program according to claim 6.

9. The aforementioned third index value is, The frequency distribution of brightness values ​​in the aforementioned scan line is calculated, For the range of the frequency distribution in which the luminance value is lower than a predetermined first threshold, a first statistic of the luminance value is calculated. Based on the first statistic calculated above, a second threshold is calculated. This is a second statistic calculated for the range of the frequency distribution in which the luminance value is higher than the second threshold. The program according to claim 6.

10. The first statistic is the mean and the standard deviation, The second statistic is the standard deviation. The program according to claim 9.

11. The colors determined based on the aforementioned index values ​​are: The first index value, the second index value, and the third index value are each normalized to an integer from 0 to 255. The normalized first index value, second index value, and third index value are assigned to the luminance of each of the multiple color channels constituting the color image in any order that does not overlap, thereby generating the image. The program according to claim 6.

12. The calculation range is the range that includes the intersection point. The program according to claim 1.

13. When R1 is the distance between the center of the image acquisition catheter and the intersection point, and b is a constant specified by the user, the calculation range is such that the distance from the center of the image acquisition catheter is (R1 - b / 2) or greater and less than or equal to the maximum depth in the distance direction. The program according to any one of claims 1 to 12.

14. When R1 is the distance between the center of the image acquisition catheter and the intersection point, and b is a constant specified by the user, the calculation range is such that the distance from the center of the image acquisition catheter is (R1 - b / 2) or greater and (R1 + b) or less. The program according to any one of claims 1 to 12.

15. When R1 is the distance between the center of the image acquisition catheter and the intersection point, and b is a constant represented by equation (1), the calculation range is such that the distance from the center of the image acquisition catheter is (R1 - b / 2) or greater and less than or equal to the maximum image acquisition depth. The program according to any one of claims 1 to 12. [Math 1] Mag is a fixed magnification. Peff is the number of effective pixels in the tomographic image. Dep is the image acquisition depth.

16. When R1 is the distance between the center of the image acquisition catheter and the intersection point, and b is a constant represented by equation (1), the calculation range is such that the distance from the center of the image acquisition catheter is (R1 - b / 2) or greater and (R1 + b) or less. The program according to any one of claims 1 to 12.

17. In addition to the position of the tomographic image and the position of the intersection, the coordinates in the three-dimensional space defined by the third axis are displayed with colors determined based on the index values. The program according to claim 1.

18. The third axis is the distance between the image acquisition catheter and the intersection point. The program according to claim 17.

19. Multiple tomographic images are obtained by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially. A representative line was determined for each of the aforementioned fault images. The index value of the scan line is calculated within the calculation range determined based on the intersection point of each scan line constituting each of the aforementioned tomographic images and the aforementioned representative line. The coordinates determined by the position of the tomographic image and the position of the intersection point are displayed with a color determined based on the index value. An information processing method in which a computer performs the processing.

20. An information processing device comprising a control unit, The control unit, Multiple tomographic images are obtained by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially. A representative line was determined for each of the aforementioned fault images. The index value of the scan line is calculated within the calculation range determined based on the intersection point of each scan line constituting each of the aforementioned tomographic images and the aforementioned representative line. The coordinates determined by the position of the tomographic image and the position of the intersection point are displayed with a color determined based on the index value. Information processing device.

Images