Program, information processing method, and information processing device

Abstract

Provided are a program and the like whereby an image containing information related to the size of an imaging target can be generated.　The program causes a computer to execute a process for: acquiring a plurality of tomographic images (41) generated by means of three-dimensional scanning using an image acquisition catheter that acquires images while moving a scanning plane in an axial direction; determining a representative line (42) for each of the tomographic images (41); calculating an index value for each scanning line composing each of the tomographic images (41), the index value being calculated in a calculation range that has been determined on the basis of the intersection point of each scanning line and the representative line (42); and displaying a color that has been determined on the basis of the index value at coordinates established by the position of the tomographic image (41) and the position of the intersection point.

Description

Program, information processing method and information processing device

[0001] The present invention relates to a program, an information processing method, and an information processing device.

[0002] A diagnostic support device has been proposed that generates a two-dimensional image in which the similarity between the brightness pattern of each scanning line and a predefined classification pattern is arranged along two axes: the angle of the scanning line and the axial direction of the catheter, based on a set of tomographic images acquired using a three-dimensional scanning IVUS (Intravascular Ultrasound) catheter (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-41029

[0004] By observing the images generated by the diagnostic support device of Patent Document 1, a user can confirm, for example, how the wires of a stent placed in a blood vessel are positioned, as if the blood vessel were cut open.

[0005] However, the image generated by the device of Patent Document 1 has a problem in that it is not possible to evaluate the absolute size of the imaged object included in the scanning line. For example, the area of ​​plaque, the expansion amount of a stent, changes in blood vessel diameter, etc. cannot be observed in the image generated by the device of Patent Document 1.

[0006] In one aspect, an object is to provide a program or the like that can generate an image that includes information about the size of an imaging target.

[0007] The program causes a computer to perform the following process: acquire multiple tomographic images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane in the axial direction; determine a representative line for each of the tomographic images; calculate an index value for each scanning line within a calculation range determined based on the intersection of each scanning line constituting each of the tomographic images with the representative line; and display a color determined based on the index value at coordinates determined by the position of the tomographic image and the position of the intersection.

[0008] In one aspect, a program or the like can be provided that can generate an image that includes information about the size of an imaging target.

[0009] 1 is an explanatory diagram illustrating an overview of an image generation method; an explanatory diagram illustrating the configuration of an information processing device; an explanatory diagram illustrating a method for generating index value stripes; an explanatory diagram illustrating a method for calculating index value example 1; an explanatory diagram illustrating a method for calculating index value example 1; an explanatory diagram illustrating a modification of the method for calculating index value example 1; an explanatory diagram illustrating a method for calculating index value example 2; an explanatory diagram illustrating a method for calculating index value example 2; an explanatory diagram illustrating a method for calculating index value example 3; an explanatory diagram illustrating a method for calculating index value example 3; an explanatory diagram illustrating a method for converting between angles and lengths; an explanatory diagram illustrating a method for converting between angles and lengths; an explanatory diagram illustrating a method for converting between angles and lengths; an explanatory diagram illustrating a method for determining a representative line; an explanatory diagram illustrating a record layout of a tomographic image DB; an explanatory diagram illustrating a record layout of a maximum gradient point DB; and an explanatory diagram illustrating a record layout of a representative line DB. 1 is an explanatory diagram illustrating the record layout of an index value DB. FIG. 1 is a flowchart illustrating the processing flow of a program. FIG. 1 is a flowchart illustrating the processing flow of a subroutine for calculating a provisional representative line. FIG. 2 is a flowchart illustrating the processing flow of a subroutine for calculating a representative line. FIG. 2 is a flowchart illustrating the processing flow of a subroutine for calculating an index value. FIG. 3 is a flowchart illustrating the processing flow of a subroutine for calculating a first index value. FIG. 4 is a flowchart illustrating the processing flow of a subroutine for calculating a second index value. FIG. 5 is a flowchart illustrating the processing flow of a subroutine for calculating a third index value. FIG. 6 is a flowchart illustrating the processing flow of a subroutine for display. FIG. 7 is an explanatory diagram illustrating the relationship between angle and area in a modified example. FIG. 8 is an explanatory diagram illustrating the relationship between angle and area in a modified example. FIG. 9 is an example screen of embodiment 2. FIG. 10 is an explanatory diagram illustrating the configuration of a catheter system of embodiment 3. FIG. 11 is an explanatory diagram illustrating the configuration of an information processing device of embodiment 4.

[0010] [Embodiment 1] Figure 1 is an explanatory diagram illustrating an overview of an image generation method. A user such as a doctor inserts an image acquisition catheter 28 (see Figure 25) near an affected area. The image acquisition catheter 28 is for three-dimensional scanning, and can successively capture multiple tomographic images 41 by gradually changing the scanning plane in the axial direction.

[0011] The interval between the tomographic images 41 is indicated by an interval T. In the following description, the tomographic images 41 created by one three-dimensional scan may be referred to as a set of tomographic images 41 .

[0012] A representative line 42 is determined for each tomographic image 41. The representative line 42 is a closed curve determined based on, for example, the depth at which the stent is placed in the tomographic image 41, the external elastic lamina of the blood vessel into which the image acquisition catheter 28 is inserted, or the depth at which brightness changes significantly. The representative line 42 may also be a closed curve specified by the user.

[0013] In the following description, an example will be described in which the representative line 42 is a circle surrounding the image acquisition catheter 28. The center position and radius of the representative line 42 are determined for each tomographic image 41. A specific example of a method for determining the representative line 42 will be described later.

[0014] An index value is calculated for each scanning line 45 (see FIG. 3) that forms the tomographic image 41. The index value is a constant calculated based on the characteristics of the scanning line 45 and the position of an intersection 48 (see FIG. 3) on the scanning line 45. A specific example of a method for calculating the index value will be described later.

[0015] The index value calculated for each scanning line 45 is assigned to the intersections 48 between the scanning lines 45 and the representative lines 42, thereby creating an index value stripe 43. The index value stripe 43 is a virtual band formed by cutting the representative line 42, which is colored based on the index value, at a single location on the lower side in Figure 1, i.e., at a scanning angle of 180 degrees, and then extending it. Note that for locations where the density of intersections 48 is low, interpolation processing is performed using a known interpolation method such as linear interpolation.

[0016] The index value stripes 43 have the same width as the interval T between the tomographic images 41 and the same length as the length L of the representative line 42. In Fig. 1, the short arrows contacting the representative line 42 and the index value stripes 43 indicate the position directly above the imaging catheter 28 in the tomographic image 41, i.e., the position where the scan angle is 0 degrees. Details of the method for generating the index value stripes 43 will be described later.

[0017] Index value stripes 43 corresponding to each tomographic image 41 are arranged so that, for example, the positions where the scan angle is 0 degrees are aligned in a straight line, thereby generating an index value image 44. The index value image 44 is an image in which index values ​​are mapped onto the surface of a solid formed by representative lines 42 corresponding to each tomographic image 41, and the image is cut open at a position where the scan angle is 180 degrees to form a plane.

[0018] The user can intuitively grasp the size of the representative line 42 and the index values ​​around the depth where the representative line 42 exists from the index value image 44, and can make a diagnosis and a treatment.

[0019] The position where one index value stripe 43 is formed by cutting through the representative line 42 is not limited to the position where the scanning angle is 180 degrees. For example, if a region of interest such as an affected area is depicted on the lower side of the tomographic image 41, cutting through the representative line 42 at the 180-degree position will separate the affected area into upper and lower parts in the index value image 44. However, by arranging the index value stripes 43, which are obtained by cutting through the representative line 42 at a scanning angle of 0 degrees, in a straight line at the position where the scanning angle is 180 degrees, an index value image 44 is generated in which the region of interest is easy to observe.

[0020] Similarly, the reference scanning angle for arranging the index value stripes 43 is not limited to the position of 0 degrees or 180 degrees. By arranging the representative line 42 so that the scanning angles corresponding to the area where the region of interest exists are aligned in a straight line, an index value image 44 is generated that makes it easy to observe the area at the angle of interest.

[0021] The scanning angle at which the representative line 42 is cut and the scanning angle used as a reference when arranging the index value stripes 43 may be selectable by the user as appropriate.

[0022] 2 is an explanatory diagram illustrating the configuration of the information processing device 200. The information processing device 200 includes a control unit 201, a main memory device 202, an auxiliary memory device 203, a communication unit 204, a display unit 205, an input unit 206, and a bus. The control unit 201 is an arithmetic and control device that executes the program of this embodiment. The control unit 201 uses one or more central processing units (CPUs), graphics processing units (GPUs), multi-core CPUs, or the like. The control unit 201 is connected to each hardware unit that constitutes the information processing device 200 via the bus.

[0023] The main memory device 202 is a storage device such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory device 202 temporarily stores information required during processing performed by the control unit 201 and programs currently being executed by the control unit 201.

[0024] The auxiliary storage device 203 is a storage device such as an SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 203 stores a tomographic image database 36, a maximum gradient point database 37, a representative line database 38, an index value database 39, programs executed by the control unit 201, and various data required for executing the programs. The tomographic image database 36, the maximum gradient point database 37, the representative line database 38, and the index value database 39 may be stored in an external large-capacity storage device connected to the information processing device 200. The communication unit 204 is an interface for communication between the information processing device 200 and a network.

[0025] The display unit 205 is, for example, a liquid crystal display panel or an organic EL (electro-luminescence) panel, etc. The input unit 206 is, for example, a keyboard or a mouse, etc. The display unit 205 and the input unit 206 may be stacked to form a touch panel.

[0026] The information processing device 200 is a general-purpose personal computer, a tablet, a mainframe computer, a virtual machine running on a mainframe computer, or a quantum computer. The information processing device 200 may be configured with hardware such as multiple personal computers or mainframe computers that perform distributed processing. The information processing device 200 may be configured with a cloud computing system. The information processing device 200 may be configured with hardware such as multiple personal computers or mainframe computers that operate in cooperation with each other.

[0027] 3 is an explanatory diagram for explaining a method for generating an index value stripe 43. A method for generating one index value stripe 43 based on one tomographic image 41 will be explained using FIG.

[0028] The tomographic image 41 can be expressed in both the form of an XY-type tomographic image 411 and an RT-type tomographic image 412. The XY-type tomographic image 411 is an image constructed to match the actual shape. The center of the XY-type tomographic image 411 corresponds to the center of the image acquisition catheter 28. The RT-type tomographic image 412 is an image constructed by arranging the scanning lines 45 in parallel in the order of the scanning angles. The left end of the RT-type tomographic image 412 corresponds to the center of the image acquisition catheter 28.

[0029] Each scanning line 45 is represented by brightness data that associates the distance from the center of the imaging catheter 28 with the brightness of the tomographic image 41 at that distance. The method of calculating the brightness data for each scanning line 45 using the imaging catheter 28, the method of creating the tomographic image 41 using the brightness data, and the method of converting between the RT format and the XY format are all well known, so their explanation will be omitted.

[0030] In the following explanation, an example will be described in which one tomographic image 41 is formed by scanning lines 45 with a scanning angle ranging from -180 degrees to 180 degrees. In the XY-type tomographic image 411, the upward direction is defined as a scanning angle of 0 degrees, a clockwise direction as a positive scanning angle, and a counterclockwise direction as a negative scanning angle.

[0031] The control unit 201 acquires one tomographic image 41. The control unit 201 determines a representative line 42 based on the tomographic image 41. The representative line 42 is a circle surrounding the image acquisition catheter 28 in the XY-type tomographic image 411, and is a curve extending from the top to the bottom of the image in the RT-type tomographic image 412.

[0032] The scanning lines 45 are straight lines extending radially from the center of the image in the XY-type tomographic image 411, and are horizontal lines in the RT-type tomographic image 412. Fig. 3 shows an example of a scanning line 45 with a scanning angle of 90 degrees. In both the XY-type tomographic image 411 and the RT-type tomographic image 412, one scanning line 45 intersects with the representative line 42 at one point. Therefore, one intersection 48 exists for one scanning line 45.

[0033] The control unit 201 calculates an index value for each scanning line 45 based on a procedure described below. An example of the index value is shown in the angle-index value graph 54. The horizontal axis of the angle-index value graph 54 is the scanning angle θ of each scanning line 45, with a minimum value of -180 degrees and a maximum value of 180 degrees. The vertical axis of the scanning line 45 is the index value. As described below, there are multiple definitions for the index value. The unit of the index value differs depending on the definition.

[0034] The control unit 201 converts the horizontal axis of the angle-index value graph 54 into the arc length of the representative line 42. A position-index value graph 55 is shown to schematically explain the converted state. The horizontal axis of the position-index value graph 55 is the arc length of the representative line 42 from the position at a scan angle of 0 degrees to the position at a scan angle of θ. The vertical axis of the position-index value graph 55 is the index value. Details of the conversion method will be described later.

[0035] The control unit 201 assigns a color to the magnitude of the index value at each position on the horizontal axis of the position-index value graph 55, and generates the index value stripes 43. The color may be a monotone ranging from white to black, or may be a color. Specific examples of color assignment will be described later.

[0036] Through the above procedure, the control unit 201 generates one index value stripe 43 based on each of the tomographic images 41 that make up the set of tomographic images 41. Based on the multiple index value stripes 43, the control unit 201 generates an index value image 44 as described using FIG.

[0037] [Index Value Example - 1] An example of an index value calculated using a Gaussian function will be described. FIGS. 4A and 4B are explanatory diagrams illustrating a calculation method for index value example - 1. FIG. 4A shows a luminance graph 59 showing luminance data contained in one scanning line 45. The horizontal axis of the luminance graph 59 indicates the distance from the center of the image acquisition catheter 28. In the following description, the distance from the center of the image acquisition catheter 28 is represented by the number of data points on the luminance graph 59. In other words, the unit of the horizontal axis of the luminance graph 59 is the number of data points.

[0038] The vertical axis of the brightness graph 59 represents brightness. The units of the vertical axis are brightness values ​​normalized to integers between 0 and 255. In the following description, the x-th brightness data from the center of the imaging catheter 28 will be referred to as B(x).

[0039] R1 on the horizontal axis indicates the position where the scanning line 45 displayed on the luminance graph 59 intersects with the representative line 42. b is a constant. The constant b is defined by, for example, equation (1).

[0040] Mag is a fixed magnification, Peff is the number of effective pixels, and Dep is the image acquisition depth.

[0041] The control unit 201 may accept an instruction from the user to change Mag. Note that equation (1) is an example of the definition of constant b. Constant b may be a value appropriately specified by the user. Note that in FIG. 4A, data points are omitted for the region on the horizontal axis where the value is less than (R1-b / 2).

[0042] 4B shows a Gaussian function graph 51 with a mean value of R1 and a standard deviation of b. The horizontal axis of the Gaussian function graph 51 represents the distance from the center of the imaging catheter 28, similar to the brightness graph 59 shown in FIG. 4A. The vertical axis of the Gaussian function graph 51 represents the Gaussian function. The value N(x) of the Gaussian function corresponding to the x-th data point from the center of the imaging catheter 28 is calculated by equation (2).

[0043]

[0044] The peak position of the Gaussian function graph 51 corresponds to the position of the intersection 48 between the scanning line 45 and the representative line 42. The mean value R1 and the standard deviation b are examples of the first statistical quantity.

[0045] FIG. 5A is an explanatory diagram illustrating a method for calculating index value example 1. FIG. 5A is a graph showing the brightness shown in FIG. 4A weighted by the Gaussian function shown in FIG. 4B. The horizontal axis of FIG. 5A indicates the distance from the center of the imaging catheter 28, similar to the brightness graph 59 shown in FIG. 4A. The vertical axis of FIG. 5A is the weighted brightness. The weighted brightness Bw(x) corresponding to the x-th data from the center of the imaging catheter 28 is calculated by equation (3). Bw(x)=N(x)·B(x) (3)

[0046] The average value of the weighted brightness Bw(x) in the calculation range 47, where the distance from the center of the image acquisition catheter 28 is equal to or greater than (R1-b / 2) and equal to or less than the maximum value of the image acquisition range, is used as the index value. The average value is, for example, the arithmetic mean value. The average value may also be the geometric mean value, the harmonic mean value, or the like. The control unit 201 calculates the index value for each scanning line 45. In the following description, the average value of the weighted brightness Bw(x) in the calculation range 47 may be referred to as the first index value.

[0047] Any statistical quantity, such as the standard deviation or variance of the weighted brightness Bw(x) in the calculation range 47, may be used as the index value. A value such as a multiple or power of the statistical quantity may be used as the first index value.

[0048] 5B is an explanatory diagram illustrating a modification of the calculation method of index value example 1. Calculation range 47 in this modification is the range where the distance from the center of imaging catheter 28 is equal to or greater than (R1-b / 2) and equal to or less than (R1+b). Controller 201 calculates the average value of weighted brightness Bw(x) within calculation range 47 and uses it as the first index value.

[0049] According to this modification, a first index value that is less susceptible to the influence of regions distant from the imaging catheter 28 can be calculated and used. If the imaging catheter 28 is an IVUS catheter, artifacts such as multiple reflections may occur on the outside if a reflector with a high acoustic impedance, such as a calcified lesion or a stent, is present in the scanning target region. By reducing the influence of regions distant from the imaging catheter 28, the influence of artifacts such as multiple reflections can be avoided. The range of the calculation range 47 is not limited to that shown in FIGS. 5A and 5B .

[0050] For example, the calculation range 47 may be a range of distances from the center of the image acquisition catheter 28 that is (R1±b / 2), (R1±b / 4), or (R1±b). The calculation range 47 may be a range of distances from the center of the image acquisition catheter 28 that is equal to or greater than R1 and equal to or less than (R1+b / 2), or equal to or greater than (R1-b / 2) and equal to or less than R1.

[0051] As illustrated above, by defining the calculation range 47 to include the vicinity of the intersection 48 between the scanning line 45 and the representative line 42, an index value that strongly reflects the information in the vicinity of the representative line 42 in the tomographic image 41 can be calculated.

[0052] [Example of Index Value-2] An example of an index value calculated using the Rayleigh distribution will be described. Explanation of parts common to Example of Index Value-1 will be omitted.

[0053] 6A and 6B are explanatory diagrams illustrating a method for calculating index value example 2. FIG. 6A shows a luminance graph 59 indicating the luminance of one scanning line 45. The horizontal axis of the luminance graph 59 indicates the distance from the center of the image acquisition catheter 28. The vertical axis of the luminance graph 59 indicates the luminance. The calculation range 47 is the range where the distance from the center of the image acquisition catheter 28 is equal to or greater than (R1-b / 2) and equal to or less than (R1+b).

[0054] 6B shows a Rayleigh distribution graph 52 estimated from the luminance data within the calculation range 47. The horizontal axis of the Rayleigh distribution graph 52 indicates the distance from the center of the imaging catheter 28, as with the luminance graph 59. The vertical axis of the Rayleigh distribution graph 52 indicates the probability density function of the Rayleigh distribution. The probability density function PL(x) of the Rayleigh distribution corresponding to the x-th data point from the center of the imaging catheter 28 is calculated using equation (4).

[0055] σ is a parameter of the Rayleigh distribution.

[0056] The expected value of the Rayleigh distribution shown in equation (4) is given by equation (5), and the variance is given by equation (6).

[0057]

[0058] The maximum likelihood estimate σg of the parameter σ is calculated by equation (7) using the luminance data within the calculation range 47.

[0059] n is the number of luminance data within the calculation range, n1 is the minimum value of x within the calculation range, and n2 is the maximum value of x within the calculation range.

[0060] The control unit 201 calculates an index value for each scanning line 45 using equation (8). In the following description, the index value calculated using equation (8) may be referred to as a second index value.

[0061]

[0062] As is clear from equations (6) and (8), the second index value shown in equation (8) is the variance of the Rayleigh distribution calculated based on the maximum likelihood estimate of the parameter σ. Note that the second index value is not limited to the variance of the Rayleigh distribution.

[0063] For example, the standard deviation, which is the square root of the variance calculated by equation (8), may be used as the second index. The expected value of the Rayleigh distribution shown in equation (5), calculated based on the maximum likelihood estimate of the parameter σ, may be used as the second index value. The maximum likelihood estimate σg of the parameter σ shown in equation (7) may be used as the second index value. A value calculated based on these values ​​may be used as the second index value.

[0064] [Example of Index Value-3] An example of an index value calculated using the brightness histogram 53 (see FIG. 7B) will be described. Explanation of parts common to Example of Index Value-1 will be omitted.

[0065] 7A and 7B are explanatory diagrams illustrating a calculation method for Example 3 of the index value. FIG. 7A shows a brightness graph 59 indicating the brightness of one scanning line 45. The horizontal axis of the brightness graph 59 indicates the distance from the center of the image acquisition catheter 28. The vertical axis of the brightness graph 59 indicates the brightness. The calculation range 47 is the range where the distance from the center of the image acquisition catheter 28 is equal to or greater than (R1-b / 2) and equal to or less than the maximum value of the image acquisition range.

[0066] 7B shows a luminance histogram 53 calculated based on the luminance data within the calculation range 47. The horizontal axis of the luminance histogram 53 represents luminance. The vertical axis of the luminance histogram 53 represents frequency. In the following description, a portion of the luminance histogram 53 where the luminance is less than a predetermined first threshold is referred to as a low luminance region 531. The first threshold is, for example, 64.

[0067] The control unit 201 extracts data whose luminance is less than the first threshold value from the luminance data in the calculation range 47. The control unit 201 calculates the average value μL and standard deviation σL of the extracted data. The control unit 201 calculates the second threshold value, which is a threshold value on the high luminance side, based on equation (9). Second threshold value = μL + 3σL (9)

[0068] FIG. 8 is an explanatory diagram illustrating a method for calculating index value example 3. Similar to FIG. 7B, FIG. 8 shows a luminance histogram 53 calculated based on luminance data within calculation range 47. In FIG. 8, the vertical axis is expanded to cover a range from 0 to 12. The horizontal axis in FIG. 8 indicates the second threshold calculated using equation (9). In the following description, a portion of luminance histogram 53 where the luminance is equal to or greater than the predetermined second threshold will be referred to as a high luminance region 532.

[0069] The control unit 201 extracts data included in the high brightness region 532 from the brightness data in the calculation range 47. The control unit 201 calculates the standard deviation σH of the extracted data. The standard deviation σH is an example of a second statistical quantity. In the following description, the standard deviation σH may be referred to as a third index value. Note that the third index value is not limited to σH.

[0070] For example, the variance of the data included in the high-brightness region 532 may be used as the third index value. Any statistical quantity of the data included in the high-brightness region 532, such as the arithmetic mean, geometric mean, harmonic mean, median, or mode, may be used as the third index value.

[0071] The formula for calculating the second threshold is not limited to formula (9). The second threshold may be a predetermined constant. The calculation range 47 is not limited to a range in which the distance from the center of the image acquisition catheter 28 is equal to or greater than (R1-b / 2) and equal to or less than the maximum value of the image acquisition range. For example, the calculation range 47 may be a range in which the distance from the center of the image acquisition catheter 28 is equal to or greater than (R1-b / 2), (R1+b), (R1±b / 2), (R1±b / 4), or (R1±b). The calculation range 47 may be a range in which the distance from the center of the image acquisition catheter 28 is equal to or greater than R1 and equal to or less than (R1+b / 2), or equal to or greater than (R1-b / 2) and equal to or less than R1.

[0072] The calculation range 47 used to calculate the first index value, the second index value, and the third index value may be a common range or may be different ranges.

[0073] The index value may be calculated based on data obtained during the conversion of the signal acquired by the sensor 282 (see FIG. 25) into brightness data. Alternatively, the index value may be calculated based on the signal acquired by the sensor 282 itself.

[0074] For example, one of the first index value, the second use value, and the third index value can be selected and used as the index value of each scanning line 45. In this case, the index value stripe 43 can be defined as a band of black and white shading in which the index values ​​are assigned to a gray scale from black to white, and the index value image 44 can be defined as a black and white image.

[0075] Three index values, namely, the first index value, the second usage value, and the third index value, may be used simultaneously. For example, the control unit 201 normalizes the first index value, the second usage value, and the third index value to integer values ​​ranging from 0 to 255. For example, the control unit 201 assigns each of the normalized index values ​​to the brightness of R (red), G (green), and B (blue) in an arbitrary order that does not overlap. The combination of index value numbers and colors is arbitrary.

[0076] When the brightness of each of RGB is assigned in this manner, the index value stripe 43 can be defined as a band in which the index value is colored in 24-bit full color, and the index value image 44 can be defined as a full-color image.

[0077] Note that RGB is an example of color channels that form a color image displayed on the display unit 205. The control unit 201 may assign each index value to a channel corresponding to an arbitrary color space, such as an HLS color space or an HSV (Hue, Saturation, Value) color space, instead of RGB.

[0078] The control unit 201 may calculate four index values ​​and assign them to RGBA, which is an alpha channel in addition to RGB. The control unit 201 may also assign index values ​​to four channels of CYMK (Cyan, Magenta, Yellow, Black).

[0079] 9A to 9C are explanatory diagrams illustrating a method for converting angles and lengths. FIG. 9A schematically shows an image in which a representative line 42 is superimposed on an XY-type tomographic image 411. Of the scanning lines 45 that make up the XY-type tomographic image 411, two scanning lines 45, a first scanning line 451 and a second scanning line 452, are shown in FIG. 9A. In the XY-type tomographic image 411, the intersection 48 between a reference line extending upward from the center of the image acquisition catheter 28 and the representative line 42 is indicated as a reference intersection 489. The reference intersection 489 is an example of a reference point defined on the representative line 42.

[0080] The first scanning line 451 is located at a position θ1 clockwise from the reference line. The intersection 48 between the first scanning line 451 and the representative line 42 is indicated as a first intersection 481. The length along the representative line 42 from the reference intersection 489 to the first intersection 481 is indicated as L1.

[0081] Similarly, the second scanning line 452 is located at a position θ2 counterclockwise from the reference line. The intersection 48 between the second scanning line 452 and the representative line 42 is indicated as a second intersection 482. The length along the representative line 42 from the reference intersection 489 to the second intersection 482 is indicated as L2.

[0082] FIG. 9B shows an angle-index value graph 54. The index value corresponding to the first scan line 451 is indicated by A1, and the index value corresponding to the second scan line 452 is indicated by A2. FIG. 9C shows a position-index value graph 55. In FIG. 9A, the center of the representative line 42 is located to the upper right of the center of the imaging catheter 28, and therefore the position on the horizontal axis corresponding to the reference intersection point 489 in the position-index value graph 55 is located to the left of the center of the horizontal axis. In the position-index value graph 55, the distance between the minimum and maximum values ​​on the horizontal axis is the length L of the index value stripe 43.

[0083] Fig. 10 is an explanatory diagram for explaining a method of converting between angles and lengths. Using Fig. 10, a method of converting the angle-index value graph 54 shown in Fig. 9B into the position-index value graph 55 shown in Fig. 9C will be explained.

[0084] 10 shows an enlarged view of the upper right portion of FIG. 9A. R11 denotes the distance between the center of the image acquisition catheter 28 and the first intersection 481. The third scanning line 453, indicated by a dashed line, is the scanning line 45 adjacent to the first scanning line 451. The intersection 48 between the third scanning line 453 and the representative line 42 is indicated by a third intersection 483.

[0085] The angle formed between the first scanning line 451 and the third scanning line 453 is indicated by Δθ. Δθ is constant for all scanning lines 45 constituting the XY-type tomographic image 411. For example, when one tomographic image 41 is composed of 512 scanning lines 45, Δθ is approximately 0.7 degrees. In FIG. 10 , Δθ is schematically indicated by a large angle. The length from the first intersection 481 to the third intersection 483 along the representative line 42 is indicated by ΔL.

[0086] Because Δθ is a small angle, ΔL can be approximated by equation (10): ΔL=R11×sin Δθ (10) Similarly, the arc length ΔLn between the nth scanning line 45 and the (n+1)th scanning line 45 can be approximated by equation (11): ΔLn=R1n×sin Δθ (11) R1n is the distance between the intersection 48 of the nth scanning line 45 and the representative line 42 and the center of the image acquisition catheter 28.

[0087] The length L1 along the representative line 42 can be calculated by adding ΔL shown in equation (10) while sequentially moving the scanning line 45 from the reference line to the first scanning line 451. Through the above processing, the control unit 201 can generate a position-index value graph 55 based on the angle-index value graph 54. By converting the index value shown on the vertical axis of the position-index value graph 55 into a color as described above, the control unit 201 can generate a single index value stripe 43 corresponding to one tomographic image 41.

[0088] [Determination of Representative Line-1] The control unit 201 may, for example, generate a circle passing through three points specified by the user on the XY-type tomographic image 411, and use the circle as the representative line 42. The control unit 201 may superimpose a template representing the circle on the XY-type tomographic image 411, accept changes to the position and radius made by the user, and use the circle confirmed by the user as the representative line 42. Alternatively, the control unit 201 may accept designation of the representative line 42 by the user via an arbitrary user interface.

[0089] 11 and 12 are explanatory diagrams illustrating a method for determining the representative line 42. An example of a method in which the control unit 201 automatically determines the representative line 42 will be described using FIGS.

[0090] The control unit 201 divides the RT tomographic image 412 by lines parallel to the R axis into, for example, 25 blocks 46. When one RT tomographic image 412 is composed of 512 scanning lines 45, one block 46 includes 20 or 21 scanning lines 45.

[0091] 12, the relationship between the multiple tomographic images 41 acquired by three-dimensional scanning will be described. Hatching indicates a tomographic image 41 currently being processed. The control unit 201 processes a group of tomographic images 41 obtained by combining the tomographic image 41 currently being processed with K tomographic images 41 before and after it. In the following description, K is 3, and the control unit 201 combines and processes a total of seven tomographic images 41, from the (n-3)th tomographic image 41 to the (n+3)th tomographic image 41.

[0092] 11 shows a schematic example of a case where there are eight blocks 46. Blocks 461 to 468 are numbered in order from the side where the scanning angle is −180 degrees and will be described. The control unit 201 calculates a maximum gradient point G for each block 46. The maximum gradient point G is the position where the change in the direction in which the brightness increases in the R direction occurs, i.e., the gradient is at its maximum.

[0093] The brightness gradient at each point on the RT-type tomographic image 412 corresponding to the coordinate z can be calculated by equation (12). Note that equation (12) is an equation representing a convolution operation in which the Sobel operator F is applied to a plane obtained by cutting the multiple RT-type tomographic images 412 acquired by three-dimensional scanning in the direction of the scanning angle θ.

[0094] (r, θ, z) is the coordinate in the RT tomographic image corresponding to the coordinate z. r is the position in the distance direction from the center of the image acquisition catheter 28. θ is the position in the scanning angle direction. z is the coordinate in the axial direction of the image acquisition catheter 28 during three-dimensional scanning. g(r, θ, z) is the gradient of brightness at the coordinate (r, θ, z). I(r, θ, z) is the brightness of the RT tomographic image corresponding to the (r, θ, z) coordinate z. F(k, j) is the (k, j) element of the Sobel operator F. j, k, and w are parameters for the convolution operation.

[0095] The Sobel operator F used in equation (12) is an operator used for detecting contours in the horizontal direction. As described above, an example of the Sobel operator used when combining and processing seven tomographic images 41 is shown in equation (13). The element in row j and column k of equation (13) is F(k, j) of equation (12).

[0096]

[0097] When a 7 × 7 Sobel operator F shown in equation (13) is used, w in equation (12) is 3, which is the kernel half-width of the Sobel operator F. Note that the Sobel operator F is not limited to equation (13). Any Sobel operator F that can detect a contour in the horizontal direction can be used. The Sobel operator F when processing a combination of K previous and next tomographic images 41 is a square matrix with vertical (2K+1) and horizontal (2K+1).

[0098] The control unit 201 extracts the coordinate (r, θ, z) at which g(r, θ, z) calculated using equation (12) is maximum for each block 46 in the RT tomographic image 412 corresponding to each coordinate z. The extracted coordinate (r, θ, z) is the maximum gradient point G of the block 46. FIG. 11 schematically shows the maximum gradient points G of G1 to G8 extracted in each of the first block 461 to the eighth block 468 on the RT tomographic image 412.

[0099] The control unit 201 can convert the RT-type tomographic image 412 into the XY-type tomographic image 411 by known coordinate conversion. In Fig. 11, the maximum gradient points G1 to G8 are arranged in a substantially circular shape on the XY-type tomographic image 411.

[0100] The control unit 201 calculates the coordinates of the center of gravity C of maximum gradient point G1 to maximum gradient point G8 on the XY-type tomographic image 411. Since the method of calculating the coordinates of the center of gravity C from the coordinates of multiple points on a plane is well known, detailed explanation will be omitted. The control unit 201 calculates the distance D between the calculated center of gravity C and each maximum gradient point G. The control unit 201 calculates a median Dc that is a representative value of the calculated distances D.

[0101] Through the above process, a temporary representative line 422 is determined whose center is the center of gravity C of the maximum gradient point G and whose radius is the median value Dc of the distances D between the center of gravity C and each maximum gradient point G. The average value may be used as the representative value of the distance D.

[0102] Note that the extracted position of the maximum gradient point G may be inappropriate due to noise, artifacts, etc. in the tomographic image 41. To avoid adverse effects caused by the inclusion of inappropriate maximum gradient points G, it is desirable for the control unit 201 to perform processing to remove outliers of the maximum gradient points G.

[0103] For example, in the second or subsequent tomographic image 41, the control unit 201 calculates the distance D between the center of gravity C and each maximum gradient point G, and then eliminates any distance D whose difference from the representative value Dc of the distance D in the previous tomographic image 41 exceeds a predetermined threshold. The control unit 201 then calculates the representative value Dc of the remaining distances D. Through the above processing, the control unit 201 can determine a provisional representative line 422 that is less susceptible to the influence of inappropriate maximum gradient points G.

[0104] 12 , a description will be given of the processing performed after determining the provisional representative line 422 for each tomographic image 41. In the following description, the center of the provisional representative line 422 in the n-th tomographic image 41 will be referred to as Cn, and the radius of the provisional representative line 422 will be referred to as Dn.

[0105] The control unit 201 calculates moving average values ​​of the coordinate of the center C and the radius D of K tomographic images 41 before and after the nth tomographic image 41. For example, when K is 3, the control unit 201 calculates the average value Cnavg of the coordinate of the center C and the average value Dnavg of the radius D for the temporary representative line 422 of a total of seven tomographic images 41 from the (n-3)th tomographic image 41 to the (n+3)th tomographic image 41. The control unit 201 uses the calculated Cnavg and Dnavg as the center and radius of the representative line 42 in the nth tomographic image 41.

[0106] In the above description, the same number of tomographic images 41 are used to calculate the gradient explained using equations (12) and (13) and to calculate Cnavg and Dnavg, but different numbers of tomographic images 41 may be used. For example, the control unit 201 may calculate the gradient with K=3 and calculate Cnavg and Dnavg with K=7.

[0107] For the tomographic images 41 located near both ends of the set of tomographic images 41, for example, the center and radius of the provisional representative line 422 are used as the center and radius of the representative line 42. For the tomographic images 41 located near both ends of the set of tomographic images 41, the number of tomographic images 41 up to the endmost tomographic image 41 may be used as the value of K.

[0108] By the above processing, the influence of noise and artifacts in the tomographic images 41 is avoided and representative lines 42 that can constitute a smoothly connected three-dimensional shape in the three-dimensional space formed by a set of tomographic images 41 are created.

[0109] The methods for determining the representative line 42 described in Representative Line Determination-1 and Representative Line Determination-2 are merely examples and are not limited to these. For example, the representative line 42 may be determined on the XY-type tomographic image 411 by a method such as pattern matching.

[0110] 13A is an explanatory diagram illustrating the record layout of the tomographic image DB 36. The tomographic image DB 36 is a database in which tomographic images 41 created by three-dimensional scanning are recorded. The tomographic image DB 36 has a 3D scan ID field, a slice number field, and a tomographic image field. The tomographic image field has an XY format field and an RT format field.

[0111] The 3D scan ID field records a 3D scan ID assigned to each three-dimensional scan. The tomographic number field records a number indicating the order of the tomographic image 41 created by one three-dimensional scan. The XY format field records an XY format tomographic image 411. The RT format field records an RT format tomographic image 412. The tomographic image DB 36 has one record for each tomographic image 41.

[0112] Only the RT-format tomographic images 412 are recorded in the tomographic image DB 36, and the control unit 201 may create the XY-format tomographic images 411 by coordinate conversion as needed. Instead of the tomographic image DB 36 in which the tomographic images 41 are recorded, the control unit 201 may create the necessary tomographic images 41 as needed based on a database in which, for example, sound ray data or the like from a stage prior to the creation of the tomographic images 41 is recorded.

[0113] Instead of using the tomographic image number field, the tomographic image DB 36 may have a field for recording the distance from the starting point of the three-dimensional scan to the position where each tomographic image 41 was acquired. The control unit 211 can acquire the distance from the starting point of the three-dimensional scan to the position where each tomographic image 41 was acquired, for example, from the MDU 289 (see FIG. 25 ).

[0114] 13B is an explanatory diagram illustrating the record layout of the maximum gradient point DB 37. The maximum gradient point DB 37 is a database that records information about the maximum gradient point G described with reference to FIG. 11. The maximum gradient point DB 37 has a 3D scan ID field, a fault number field, a distance representative value field, a block number field, a maximum gradient position field, a distance field, and a flag field. The maximum gradient position field has an R field and a T field.

[0115] The 3D scan ID field records a 3D scan ID assigned to each three-dimensional scan. The slice number field records a number indicating the order of the slice image 41 created by one three-dimensional scan. The distance representative value field records a representative value of the distance D between the center of gravity C and each maximum gradient point G described with reference to FIG. 11 .

[0116] The block number field records the number of the block 46. The R field records the R coordinate of the maximum gradient point G in the block 46, i.e., the distance from the center of the image acquisition catheter 28. The T field records the T coordinate of the maximum gradient point G in the block 46, i.e., the scan angle.

[0117] The distance field records the distance D between the center of gravity C and the maximum gradient position G. The flag field records a flag indicating whether the maximum gradient position G is an outlier. A value of "1" means that the maximum gradient position G is not an outlier and is used to calculate the representative value recorded in the distance representative value field. A value of "0" means that the maximum gradient position G is an outlier and is not used to calculate the representative value recorded in the distance representative value field. The maximum gradient point DB 37 has one record for each block 46.

[0118] 13C is an explanatory diagram illustrating the record layout of the representative line DB 38. The representative line DB 38 is a database that records information about the representative line 42. The representative line DB 38 has a 3D scan ID field, a fault number field, a temporary representative line field, and a representative line field. The temporary representative line field and the representative line field each have a centroid field and a distance representative value field. The temporary representative line field and the centroid field of the representative line field each have an X field and a Y field.

[0119] The 3D scan ID field records a 3D scan ID assigned to each three-dimensional scan. The slice number field records a number indicating the order of the slice image 41 created by one three-dimensional scan. The gravity center field of the temporary representative line field records the X and Y coordinates of the gravity center C of the maximum gradient point G described with reference to FIG. 11. The distance representative value field of the temporary representative line field records a representative value Dc of the distance D.

[0120] The center of gravity field of the representative line field records the moving average values ​​of the X and Y coordinates of the center of gravity Cnavg, which is the moving average value of the center of gravity C described using Fig. 12. The distance representative value field of the representative line field records the distance representative value Dnavg, which is the moving average of the distance representative value D described using Fig. 12. The representative line DB 38 has one record for one tomographic image 41.

[0121] 14 is an explanatory diagram illustrating the record layout of the index value DB 39. The index value DB 39 is a database that records index values ​​corresponding to each scanning line 45. The index value DB 39 has a 3D scan ID field, a slice number field, a scanning line number field, an intersection field, and an index value field.

[0122] The intersection field has an R1 field and an L field. The index value field has a first index value field, a second index value field, and a third index value field. Note that the index value field does not have to have subfields. The index value field may have four or more subfields.

[0123] The 3D scan ID field records a 3D scan ID assigned to each three-dimensional scan. The slice number field records a number indicating the order of the slice image 41 created by one three-dimensional scan. The scan line number field records the number of the scan line. The R1 field records the distance R1 between the center of the image acquisition catheter 28 and the intersection point 48. The L field records the length L from the reference intersection point 489 to the intersection point 48 along the representative line 42. The first index value field, the second index value field, and the third index value field record a first index value, a second index value, and a third index value, respectively.

[0124] 15 is a flowchart illustrating the processing flow of the program. The control unit 201 acquires one tomographic image 41 from the tomographic image DB 36 (step S501). The control unit 201 starts a subroutine for calculating a temporary representative line (step S502). The subroutine for calculating a temporary representative line is a subroutine for calculating the center and radius of the temporary representative line 422 described using FIG. 11. The processing flow of the subroutine for calculating a temporary representative line will be described later.

[0125] The control unit 201 determines whether processing of the tomographic images 41 included in the set of tomographic images 41 has been completed (step S504). If it is determined that processing has not been completed (NO in step S504), the control unit 201 returns to step S501. If it is determined that processing has been completed (YES in step S504), the control unit 201 starts a representative line calculation subroutine (step S505). The representative line calculation subroutine is a subroutine that calculates the center and radius of the representative line 42 based on the moving average value of the temporary representative line 422, as described using FIG. 12 . The processing flow of the representative line calculation subroutine will be described later.

[0126] The control unit 201 selects a tomographic image 41 for calculating an index value (step S506). The tomographic image 41 selected in step S506 is one of the tomographic images 41 acquired in the loop from step S501 to step S504.

[0127] The control unit 201 selects one of the scanning lines 45 constituting the tomographic image 41 selected in step S506 (step S507). Note that, every time step S507 is executed, the control unit 201 selects one scanning line 45 clockwise from the reference intersection 489 described with reference to FIGS. 9A and 10 .

[0128] The control unit 201 starts a subroutine for calculating an index value (step S508). The subroutine for calculating an index value is a subroutine for calculating an index value based on one scanning line 45. The processing flow of the subroutine for calculating an index value will be described later.

[0129] The control unit 201 extracts a record corresponding to the scanning line 45 selected in step S507 from the index value DB 39. The control unit 201 records the distance R1, length L, and index value calculated in the index value calculation subroutine in the R1 field, L field, and index value field, respectively (step S510).

[0130] The control unit 201 determines whether or not the processing of the scanning lines 45 constituting the tomographic image 41 acquired in step S506 has been completed (step S511). If it is determined that the processing has not been completed (NO in step S511), the control unit 201 returns to step S507. If it is determined that the processing has been completed (YES in step S511), the control unit 201 determines whether or not the processing of one set of tomographic images 41 has been completed (step S512).

[0131] If it is determined that the process has not ended (NO in step S512), the control unit 201 returns to step S506. If it is determined that the process has ended (YES in step S512), the control unit 201 starts a display subroutine (step S513). The display subroutine is a subroutine that displays the index value image 44 based on the index value DB 39. The processing flow of the display subroutine will be described later. Thereafter, the control unit 201 ends the process.

[0132] 16 is a flowchart illustrating the processing flow of the temporary representative line calculation subroutine, which calculates the center and radius of the temporary representative line 422 described with reference to FIG. 11 based on one tomographic image 41.

[0133] As described with reference to FIG. 11 , the control unit 201 divides the tomographic image 41 being processed into a predetermined number of blocks 46 (step S521). The control unit 201 selects one block 46 (step S522). The control unit 201 calculates the coordinates of the maximum gradient point G in the block 46 (step S523). Specifically, the control unit 201 calculates the gradient of brightness at each position in the block 46 based on equation (12). The control unit 201 extracts the coordinates of the position where the calculated gradient of brightness is maximum.

[0134] The control unit 201 determines whether or not the processing of the block 46 divided in step S521 has been completed (step S524). If it is determined that the processing has not been completed (NO in step S524), the control unit 201 returns to step S522.

[0135] If it is determined that the process has ended (YES in step S524), the control unit 201 calculates the coordinates of the center of gravity C of the multiple maximum gradient points G (step S525).The control unit 201 then calculates the distance D between each maximum gradient point and the center of gravity C (step S526).

[0136] The control unit 201 determines whether the tomographic image 41 being processed is the first of a set of tomographic images 41 (step S527). If it is determined that it is the first (YES in step S527), the control unit 201 calculates a representative value Dc of the distance D calculated in step S526 (step S528). The representative value Dc is, for example, the median.

[0137] If it is determined that the image is not the first one (NO in step S527), the control unit 201 removes outliers from the distance D calculated in step S526 (step S531). An outlier is, for example, a distance D whose difference from a representative value Dc calculated for the immediately previous tomographic image 41 exceeds a predetermined threshold. The control unit 201 calculates a representative value Dc of the distance D (step S532).

[0138] After step S528 or step S532 is completed, the control unit 201 records the center of gravity C calculated in step S525 and the representative value Dc calculated in step S528 or step S532 in the representative line DB 38 (step S533).

[0139] Specifically, the control unit 201 extracts a record from the representative line DB 38 that corresponds to the tomographic image 41 being processed. The control unit 201 records the X and Y coordinates of the center of gravity C calculated in step S525 in the center of gravity field of the temporary representative line field of the extracted record. The control unit 201 records the representative value Dc calculated in step S528 or step S532 in the representative value field of the temporary representative line field of the extracted record. Thereafter, the control unit 201 ends the processing.

[0140] 17 is a flowchart illustrating the flow of processing in the representative line calculation subroutine. The control unit 201 acquires information recorded in the provisional representative line field for a set of tomographic images 41 from the representative line DB 38 (step S541). The control unit 201 selects the tomographic image number of the tomographic image 41 to be processed (step S542). For example, the control unit 201 increments the tomographic image number by one each time step S542 is executed.

[0141] The control unit 201 determines whether the tomographic image number selected in step S542 indicates the vicinity of both ends of the set of tomographic images 41 (step S543). Specifically, as described with reference to Fig. 12, when using a moving average value of K frames before and after the tomographic image 41, if the tomographic image number selected in step S542 is K or less and if it is (the total number of tomographic images 41 - K) or more, the control unit 201 determines that the selected tomographic image number indicates the vicinity of both ends.

[0142] If it is determined that the vicinity of both ends is indicated (YES in step S543), the control unit 201 records the same information as the information recorded in the provisional representative line field in the representative line field (step S544).If it is determined that the vicinity of both ends is not indicated (NO in step S543), the control unit 201 calculates the center of gravity C and the moving average of the representative value Dc using data on a predetermined number of tomography numbers before and after the tomography number being processed (step S545).

[0143] The control unit 201 records the calculated moving average in the representative line field of the representative line DB 38 (step S546). Specifically, the control unit 201 extracts a record from the representative line DB 38 that corresponds to the tomographic image 41 being processed. The control unit 201 records the moving average of the center of gravity C calculated in step S545 in the center of gravity field of the representative line field of the extracted record. The control unit 201 records the moving average of the representative value Dc calculated in step S545 in the representative value field of the representative line field of the extracted record.

[0144] The control unit 201 determines whether or not the processing for the temporary representative line 422 acquired in step S541 has been completed (step S547). If it is determined that the processing has not been completed (NO in step S547), the control unit 201 returns to step S542. If it is determined that the processing has been completed (YES in step S547), the control unit 201 terminates the processing.

[0145] 18 is a flowchart illustrating the flow of processing in the subroutine for calculating index values. The subroutine for calculating index values ​​is a subroutine for calculating index values ​​based on one scanning line 45.

[0146] The control unit 201 calculates the coordinates of the intersection 48 between the scanning line 45 currently being processed and the representative line 42 (step S551). Here, the control unit 201 calculates the coordinates of the intersection 48 in both the XY coordinate system and the RT coordinate system. The control unit 201 calculates ΔL, which is the length of the arc from the intersection 48 on the previous scanning line 45, using equation (10) (step S552).

[0147] The control unit 201 calculates the arc length L from the reference intersection point 489, which is the sum of the ΔL calculated previously (step S553). Note that for scanning lines 45 with a scanning angle of 180 degrees or more, the control unit 201 calculates the arc length L by subtracting the sum of ΔL from the perimeter of the representative line 42. Step S553 realizes the process of converting the horizontal axis of the angle-index value graph 54 described with reference to FIG. 3 to calculate the position-index value graph 55.

[0148] The control unit 201 acquires scanning line data for the scanning line 45 currently being processed (step S554). The control unit 201 then starts a subroutine for calculating a first index value (step S555). The subroutine for calculating a first index value is a subroutine for calculating the first index value described using FIGS. 4A, 4B, and 5A. The processing flow of the subroutine for calculating a first index value will be described later.

[0149] The control unit 201 starts a subroutine for calculating a second index value (step S556). The subroutine for calculating a second index value is a subroutine for calculating the second index value described using Figures 6A and 6B. The processing flow of the subroutine for calculating a second index value will be described later.

[0150] The control unit 201 starts a subroutine for calculating a third index value (step S557). The subroutine for calculating a third index value is the subroutine for calculating the third index value described using Figures 7A, 7B, and 8. The processing flow of the subroutine for calculating a third index value will be described later. Thereafter, the control unit 201 ends the processing.

[0151] 19 is a flowchart illustrating the processing flow of the subroutine for calculating the first index value. The subroutine for calculating the first index value is the subroutine for calculating the first index value described using FIGS. 4A, 4B, and 5A.

[0152] The control unit 201 generates the Gaussian function described using FIG. 4B and equation (2) (step S561). The control unit 201 calculates the weighted luminance Bw weighted by the Gaussian function described using equation (3) (step S562). The control unit 201 extracts data included in the calculation range 47 described using FIG. 5A from the weighted luminance Bw (step S563).

[0153] The control unit 201 calculates a first index value based on the data included in the calculation range 47 (step S564). As described above, the first index value is, for example, the average value of the weighted brightness Bw. Thereafter, the control unit 201 ends the process.

[0154] 20 is a flowchart illustrating the processing flow of the subroutine for calculating the second index value. The subroutine for calculating the second index value is a subroutine for calculating the second index value described using FIGS. 6A and 6B.

[0155] The control unit 201 extracts data included in the calculation range 47 described with reference to FIG. 6A from the luminance data (step S571). The control unit 201 calculates the second index value based on equation (8) (step S572). Thereafter, the control unit 201 ends the process.

[0156] 21 is a flowchart illustrating the process flow of the subroutine for calculating the third index value. The subroutine for calculating the third index value is the subroutine for calculating the third index value described using FIGS. 7A, 7B, and 8.

[0157] The control unit 201 extracts data included in the calculation range 47 described with reference to Fig. 7A from the luminance data (step S581). The control unit 201 calculates the frequency distribution of the extracted data (step S582). Since the process of calculating the frequency distribution based on a large amount of data is well known, detailed description will be omitted.

[0158] The control unit 201 extracts data from the low-brightness region 531 described with reference to FIG. 7B. The control unit 201 calculates two statistics, the mean value μL and the standard deviation σL, of the extracted data (step S583). The control unit 201 calculates a second threshold value, which is a threshold value on the high-brightness side, based on equation (9) (step S584).

[0159] The control unit 201 extracts data of the high brightness area 532 described with reference to Fig. 8 from the brightness data. The control unit 201 calculates a third index value, which is the standard deviation σH of the extracted data (step S585). Thereafter, the control unit 201 ends the process.

[0160] 22 is a flowchart illustrating the process flow of the display subroutine, which displays the index value image 44 based on the index value DB 39.

[0161] The control unit 201 acquires a first index value for the scanning lines 45 that make up a set of tomographic images 41 from the first index value field of the index value DB 39. The control unit 201 normalizes the acquired first index value to an integer between 0 and 255 (step S591).

[0162] The control unit 201 acquires the second index value for the scanning line 45 constituting the set of tomographic images 41 from the second index value field of the index value DB 39. The control unit 201 normalizes the acquired second index value to an integer between 0 and 255 (step S592).

[0163] The control unit 201 acquires the third index value for the scanning lines 45 constituting the set of tomographic images 41 from the third index value field of the index value DB 39. The control unit 201 normalizes the acquired third index value to an integer between 0 and 255 (step S593).

[0164] The control unit 201 converts the normalized first to third index values ​​into a color code for each scanning line 45 (step S594). Specifically, if the index value assigned to R (red), G (green), and B (blue) among the first, second, and third index values ​​is represented by AR, AG, and AB, respectively, the color code can be expressed as RGB (AR, AG, AB).

[0165] The control unit 201 sets a color represented by a color code at a coordinate position determined by the length L recorded in the L field of the index value DB 39 and the fault position Z obtained by multiplying the fault number recorded in the fault number field by the interval T for each scanning line 45. The control unit 201 determines the color code of the coordinate position corresponding to the gap between the scanning lines 45 by interpolation (step S595).

[0166] The control unit 201 displays an image generated using the color codes of the respective coordinate positions on the display unit 205 (step S596). The control unit 201 rotates, enlarges, reduces, etc. the image in response to instructions from the user. The control unit 201 may also display indicators, handles, etc. that the user can use to manipulate the image on the display unit 205. Thereafter, the control unit 201 ends the processing.

[0167] According to this embodiment, it is possible to generate an image that includes information about the size of the scanning line 45. The index value image 44 displays indices that emphasize information about the portion of the scanning line 45 near the representative line 42 in a manner similar to a development view, allowing a doctor or other user to intuitively grasp the portion to which they want to pay attention.

[0168] For example, when the representative lines 42 are set at positions corresponding to the vascular lumen, the length of each representative line 42 is expressed as the vertical length of the index value image 44 shown in Figure 1, so the user can easily understand whether the area of ​​interest is narrowed or dilated.

[0169] The user can grasp a lot of information in a short time by assigning three indices, which are calculated using different methods, to R, G, and B and displaying them on a single index value image 44. The types of indices to be used and the combinations of indices and colors may be changed by the user as appropriate.

[0170] The control unit 201 may display the index value image 44 and the tomographic image 41 side by side on the display unit 205. When the user clicks a point on the index value image 44, the control unit 201 may display the tomographic image 41 including the clicked position on the display unit 205.

[0171] [Variation 2] The control unit 201 may determine the lengths L and ΔL along the tomographic image 41, as described using Fig. 10, based on the center point of the representative line 42. Specifically, the control unit 201 calculates index values ​​for the intersections between each scanning line 45 and the representative line 42, such as the first intersection 481 and the second intersection 482, as described using Fig. 10.

[0172] The control unit 201 calculates the center point of the representative line 42. The control unit 201 determines ΔL at each fixed angle from the center point to generate index value stripes 43. In other words, ΔL is a constant value over the entire representative line 42.

[0173] According to this modification, it is possible to provide a catheter system 10 that clearly displays changes in blood vessel size due to stenosis or the like.

[0174] The angle formed between the first scanning line 451 and the third scanning line 453 is indicated by Δθ. Δθ is constant for all scanning lines 45 constituting the XY-type tomographic image 411. For example, when one tomographic image 41 is composed of 512 scanning lines 45, Δθ is approximately 0.7 degrees. In FIG. 10 , Δθ is schematically indicated by a large angle. The length from the first intersection 481 to the third intersection 483 along the representative line 42 is indicated by ΔL.

[0175] Because Δθ is a small angle, ΔL can be approximated by equation (10): ΔL=R11×sin Δθ (10) Similarly, the arc length ΔLn between the nth scanning line 45 and the (n+1)th scanning line 45 can be approximated by equation (11): ΔLn=R1n×sin Δθ (11) R1n is the distance between the intersection 48 of the nth scanning line 45 and the representative line 42 and the center of the image acquisition catheter 28.

[0176] The length L1 along the representative line 42 can be calculated by adding ΔL shown in equation (10) while sequentially moving the scanning line 45 from the reference line to the first scanning line 451. Through the above processing, the control unit 201 can generate a position-index value graph 55 based on the angle-index value graph 54. By converting the index value shown on the vertical axis of the position-index value graph 55 into a color as described above, the control unit 201 can generate a single index value stripe 43 corresponding to one tomographic image 41.

[0177] [Modification 3] In this modification, the area is used instead of the length L along the arc of the representative line 42. Figures 23A to 23C are explanatory diagrams illustrating the relationship between angle and area in this modification.

[0178] 23A schematically shows an image in which a representative line 42 is superimposed on an XY-type tomographic image 411. In this modification, instead of the length L1 from the reference intersection 489 to the first intersection 481, an area S1 of a substantially sector-shaped area surrounded by the representative line 42, the first scanning line 451, and the Y axis is used. Similarly, instead of the length L2 from the reference intersection 489 to the second intersection 482, an area S2 of a substantially sector-shaped area surrounded by the representative line 42, the second scanning line 452, and the Y axis is used.

[0179] Figure 23B shows an angle-index value graph 54 similar to Figure 9B. Figure 23C shows an area-index value graph 56. The horizontal axis of Figure 23 is the area described using Figure 23A. The vertical axis of Figure 23 is the index value.

[0180] As is clear from Figure 23A, there is a one-to-one correspondence between the scanning angle and the area. Area S1 can be calculated, for example, by multiplying the number of pixels included in the portion hatched downward to the left in Figure 23A by the area of ​​one pixel. Therefore, control unit 201 can convert the horizontal axis of angle-index value graph 54 into area, and create area-index value graph 56.

[0181] By generating index value stripes 43 based on the area-index value graph 56, an index value image 44 can be generated in which index values ​​corresponding to each scanning line 45 are set at coordinate positions determined by the area S and the tomographic position Z.

[0182] [Modification 4] In this modification, index value stripes 43 of the same length are generated for each tomographic image 41, and then the entire image is expanded or contracted based on the value of the representative line .

[0183] 3, the control unit 201 does not create a position-index value graph 55, but generates index value stripes 43 based on an angle-index value graph 54. The length of the generated index value stripes 43 corresponds to an angle of 360 degrees, and index value stripes 43 of the same length are generated for all tomographic images 41.

[0184] The control unit 201 calculates a value related to the representative line 42 for each tomographic image 41. The value related to the representative line 42 is, for example, the length of the representative line 42, the area inside the representative line 42, the average diameter of the representative line 42, etc. The control unit 201 expands or contracts the index value stripes 43 based on the value related to the representative line 42. Through the above procedure, the control unit 201 generates index value stripes 43 of different lengths for each tomographic image 41, as described using FIG. 1 .

[0185] [Variation 5] The control unit 201 may display an index on the index value image 44 based on the value related to the representative line 42. For example, the control unit 201 displays an index corresponding to the position of the tomographic image 41 where the average diameter of the representative line 42 is smallest, on the edge of the index value image 44. The control unit 201 may display the index value stripes 43 corresponding to the tomographic image 41 where the average diameter of the representative line 42 is smallest, in a color different from that of the other index value stripes 43.

[0186] [Variation 6] The control unit 201 may create multiple representative lines 42 and calculate index values. For example, the control unit 201 creates two representative lines 42: one corresponding to the vascular lumen and the other corresponding to the external elastic lamina. The control unit 201 subtracts the area of ​​the representative line 42 corresponding to the vascular lumen from the area of ​​the representative line 42 corresponding to the external elastic lamina. In this way, the control unit 201 can calculate an index value corresponding to the plaque area for each tomographic image 41.

[0187] 24 is an example of a screen according to embodiment 2. The control unit 201 sets a color corresponding to an index value at a coordinate position determined by a three-dimensional space in which the length L and the tomographic position Z are added to a third axis U. The index value image 44 is displayed in the form of a curved surface in the three-dimensional space.

[0188] 9A, that is, the distance from the center of the image acquisition catheter 28 to the intersection point 48. Any other parameter may be set for the third axis U.

[0189] According to this modification, it is possible to provide an information processing device 200 that displays an index value image 44 that simultaneously expresses a larger amount of information.

[0190] [Third Embodiment] This embodiment relates to a catheter system 10 that acquires a tomographic image 41 in real time and displays an index value image 44. Explanation of parts common to the first embodiment will be omitted.

[0191] 25 is an explanatory diagram illustrating the configuration of a catheter system 10 according to embodiment 3. The catheter system 10 includes an image processing device 210, a catheter control device 27, an MDU (Motor Driving Unit) 289, and an image acquisition catheter 28. The image acquisition catheter 28 is connected to the image processing device 210 via the MDU 289 and the catheter control device 27.

[0192] The image processing device 210 includes a control unit 211, a main memory device 212, an auxiliary memory device 213, a communication unit 214, a display unit 215, an input unit 216, and a bus. The control unit 211 is an arithmetic and control device that executes the program of this embodiment. The control unit 211 uses one or more CPUs, GPUs, multi-core CPUs, or the like. The control unit 211 is connected to each hardware unit that constitutes the image processing device 210 via the bus.

[0193] The main memory device 212 is a memory device such as an SRAM, a DRAM, a flash memory, etc. The main memory device 212 temporarily stores information required during processing performed by the control unit 211 and programs currently being executed by the control unit 211.

[0194] The auxiliary storage device 213 is a storage device such as an SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 213 stores the tomographic image DB 36, maximum gradient point DB 37, representative line DB 38, index value DB 39, programs executed by the control unit 211, and various data required for executing the programs. The tomographic image DB 36, maximum gradient point DB 37, representative line DB 38, and index value DB 39 may be stored in an external mass storage device connected to the image processing device 210. The external mass storage device may be so-called cloud storage.

[0195] The communication unit 214 is an interface for communication between the image processing device 210 and a network. The display unit 215 is, for example, a liquid crystal display panel or an organic EL panel. The input unit 216 is, for example, a keyboard and a mouse. The input unit 216 may be stacked on the display unit 215 to form a touch panel. The display unit 215 may be a display device connected to the image processing device 210.

[0196] The image processing device 210 is a general-purpose personal computer, tablet, mainframe computer, or virtual machine running on a mainframe computer. The image processing device 210 may be configured with hardware such as multiple personal computers or mainframe computers that perform distributed processing. The image processing device 210 may also be configured with a cloud computing system. The image processing device 210 and the catheter control device 27 may be configured as an integrated piece of hardware.

[0197] The image acquisition catheter 28 has a sheath 281, a shaft 283 inserted inside the sheath 281, and a sensor 282 arranged at the tip of the shaft 283. The MDU 289 rotates and advances and retracts the shaft 283 and the sensor 282 inside the sheath 281.

[0198] The sensor 282 may be, for example, an ultrasonic transducer that transmits and receives ultrasonic waves, or a transmitter / receiver for OCT (Optical Coherence Tomography) that emits near-infrared light and receives reflected light. In the following description, the image acquisition catheter 28 will be described as an example of an IVUS (Intravascular Ultrasound) catheter used to capture ultrasonic cross-sectional images from inside the circulatory system.

[0199] The catheter control device 27 creates one tomographic image 41 for each rotation of the sensor 282. The catheter control device 27 successively creates multiple tomographic images 41 that are approximately perpendicular to the sheath 281 by the MDU 289 rotating the sensor 282 while pulling or pushing it in the axial direction.

[0200] The control unit 211 sequentially acquires tomographic images 41 from the catheter control device 27 and records them in the tomographic image DB 36. In this manner, a so-called three-dimensional scan is performed, and a set of tomographic images 41 is recorded in the tomographic image DB 36.

[0201] The operation of advancing and retracting the sensor 282 includes both an operation of advancing and retracting the entire image acquisition catheter 28 and an operation of advancing and retracting the sensor 282 inside the sheath 281. The operation of advancing and retracting may be performed automatically at a predetermined speed by the MDU 289 or manually by the user.

[0202] The imaging catheter 28 is not limited to a mechanical scanning type that mechanically rotates and moves back and forth, but may be, for example, an electronic radial scanning type imaging catheter 28 that uses a sensor 282 in which multiple ultrasonic transducers are arranged in a ring shape.

[0203] The control unit 211 performs the processing described in the first embodiment using the tomographic image DB 36 recorded by the above operations, and displays the index value image 44 on the display unit 215 .

[0204] 26 is an explanatory diagram illustrating the configuration of an information processing device 200 according to a fourth embodiment. This embodiment relates to a form in which the information processing device 200 according to this embodiment is realized by combining and operating a general-purpose computer 90 and a program 97. Explanations of parts common to the first embodiment will be omitted.

[0205] The computer 90 includes a reading unit 209 in addition to the control unit 201, main memory unit 202, auxiliary memory unit 203, communication unit 204, display unit 205, input unit 206, and bus.

[0206] The program 97 is recorded on the portable recording medium 96. The control unit 201 reads the program 97 via the reading unit 209 and stores it in the auxiliary storage device 203. The control unit 201 may also read the program 97 stored in a semiconductor memory 98, such as a flash memory, implemented in the computer 90. Furthermore, the control unit 201 may download the program 97 from another server computer (not shown) connected via the communication unit 204 and a network (not shown) and store it in the auxiliary storage device 203.

[0207] The program 97 is installed as a control program for the computer 90, and is loaded into the main storage device 202 and executed. In this way, the information processing device 200 described in the first embodiment is realized. The program 97 in this embodiment is an example of a program product.

[0208] The technical features (constituent elements) described in each embodiment and modification can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should be considered not to be limiting. The scope of the present invention is defined by the claims, not by the meaning described above, and is intended to include all modifications within the meaning and scope of the claims.

[0209] 10 Catheter system 200 Information processing device 201 Control unit 202 Main memory device 203 Auxiliary memory device 204 Communication unit 205 Display unit 206 Input unit 209 Reading unit 210 Image processing device 211 Control unit 212 Main memory device 213 Auxiliary memory device 214 Communication unit 215 Display unit 216 Input unit 27 Catheter control device 28 Image acquisition catheter 281 Sheath 282 Sensor 283 Shaft 289 MDU 36 Tomographic image DB 37 Maximum gradient point DB 38 Representative line DB 39 Index value DB 41 Tomographic image 411 XY type tomographic image 412 RT type tomographic image 42 Representative line 422 Temporary representative line 43 Index value stripe 44 Index value image 45 Scanning line 451 First scanning line 452 Second scanning line 453 Third scanning line 46 Block 461 First block 468 Eighth block 47 Calculation range 48 Intersection 481 First intersection 482 Second intersection 489 Reference intersection 51 Gaussian function graph 52 Rayleigh distribution graph 53 Brightness histogram 531 Low brightness area 532 High brightness area 54 Angle-index value graph 55 Position-index value graph 56 Area-index value graph 59 Brightness graph 90 Computer 96 Portable recording medium 97 Program 98 Semiconductor memory

Claims

1. Acquiring a plurality of tomographic images generated by three-dimensional scanning using an imaging catheter that acquires images while moving a scanning plane in an axial direction; determining a representative line for each of the tomographic images; Calculating an index value of the scanning line in a calculation range determined based on an intersection between each scanning line constituting each of the tomographic images and the representative line; A color determined based on the index value is displayed at a coordinate determined by the position of the tomographic image and the position of the intersection. A program that causes a computer to carry out processing.

2. The representative line is a closed curve that surrounds the imaging catheter. The program according to claim 1.

3. The representative line is a circle surrounding the image acquisition catheter. The program according to claim 1.

4. The coordinates are determined based on the position of the tomographic image and a value related to the representative line. The program according to claim 1.

5. The value related to the representative line is the length along the representative line from a reference point defined on the representative line to the intersection point. The program according to claim 1.

6. The index value is determined based on one or more of a first index value, a second index value, and a third index value that are calculated by different calculation methods. The program according to claim 1.

7. The first index value is weighting the luminance values ​​in the scanning lines by a Gaussian function having a peak position at the intersection point; The weighted luminance value is averaged within the calculation range. The program according to claim 6.

8. The second index value is a function of a maximum likelihood estimate of a parameter of a Rayleigh distribution estimated based on the luminance values ​​in the calculation range of the scan line. The program according to claim 6.

9. The third index value is Calculating a frequency distribution of luminance values ​​in the scanning line; calculating a first statistic of the luminance values ​​for a range of the frequency distribution in which the luminance values ​​are lower than a first threshold value; Calculating a second threshold value based on the calculated first statistical amount; a second statistic calculated for a range of the frequency distribution in which the luminance value is higher than the second threshold value; The program according to claim 6.

10. the first statistic is a mean and a standard deviation; The second statistic is the standard deviation The program according to claim 9.

11. The color determined based on the index value is: normalizing the first index value, the second index value, and the third index value to integers ranging from 0 to 255; The normalized first index value, the second index value, and the third index value are generated by assigning the normalized first index value, the second index value, and the third index value to the luminance of each of a plurality of color channels constituting a color image in an arbitrary order without overlapping. The program according to claim 6.

12. The calculation range is a range including the intersection point. The program according to claim 1.

13. When the distance between the center of the imaging catheter and the intersection point is R1 and a predetermined constant is b, the calculation range is a distance from the center of the imaging catheter that is equal to or greater than (R1-b / 2) and equal to or less than the maximum value of the image acquisition range. The program according to claim 1.

14. When the distance between the center of the imaging catheter and the intersection point is R1 and a predetermined constant is b, the calculation range is a distance from the center of the imaging catheter that is equal to or greater than (R1-b / 2) and equal to or less than (R1+b). The program according to claim 1.

15. A color determined based on the index value is displayed at coordinates in a three-dimensional space determined by a third axis in addition to the positions of the tomographic images and the intersections. The program according to claim 1.

16. The third axis is the distance between the imaging catheter and the intersection point. The program according to claim 15.

17. Acquiring a plurality of tomographic images generated by three-dimensional scanning using an imaging catheter that acquires images while moving a scanning plane in an axial direction; determining a representative line for each of the tomographic images; Calculating an index value of the scanning line in a calculation range determined based on an intersection between each scanning line constituting each of the tomographic images and the representative line; A color determined based on the index value is displayed at a coordinate determined by the position of the tomographic image and the position of the intersection. An information processing method in which processing is performed by a computer.

18. An information processing device including a control unit, The control unit is Acquiring a plurality of tomographic images generated by three-dimensional scanning using an imaging catheter that acquires images while moving a scanning plane in an axial direction; determining a representative line for each of the tomographic images; Calculating an index value of the scanning line in a calculation range determined based on an intersection between each scanning line constituting each of the tomographic images and the representative line; A color determined based on the index value is displayed at a coordinate determined by the position of the tomographic image and the position of the intersection. Information processing device.