Information processing device, information processing system, information processing method, and computer program

Abstract

This information processing device comprises a control unit that: acquires at least one frame image, which is generated by using line data showing the intensity of a reflected wave with respect to an ultrasonic wave radially transmitted from a ultrasonic vibrator moving inside living tissue; and extracts a specific region from the at least one frame image, on the basis of the ratio between the average luminance in a region of interest contained in the at least one frame image and the dispersion of the luminance in the region of interest.

Description

[Technical Field]

[0001] This disclosure relates to an information processing device, an information processing system, an information processing method, and a computer program. [Background technology]

[0002] Patent documents 1 to 3 describe techniques related to the processing of intravascular ultrasound (IVUS) images. "IVUS" is an abbreviation for Intravascular Ultrasound. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Special Publication No. 2002-521168 [Patent Document 2] Special Publication No. 2015-503363 [Patent Document 3] Special Publication No. 2016-511043 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] IVUS images are obtained by passing a catheter, which has an imaging core that transmits and receives ultrasound waves, rotatably and axially movable at its tip, through the lumen of a blood vessel and analyzing the reflected ultrasound signals received by the imaging core. However, various blood cells such as red blood cells, white blood cells, lymphocytes, and platelets exist within blood vessels. Therefore, in IVUS images, reflected signals mainly from blood cells become noise, making it difficult to distinguish and observe in detail the areas where blood is flowing from the areas of objects to be observed, such as blood vessel branches, guidewires, and stents.

[0005] The purpose of this disclosure is to identify objects with higher accuracy based on the results of observing cross-sections of biological tissue using ultrasound. [Means for solving the problem]

[0006] An information processing device in one aspect of the present disclosure includes a control unit that acquires at least one frame image generated using line data indicating the intensity of reflected waves to ultrasonic waves radially transmitted from an ultrasonic transducer moving inside biological tissue, and extracts a specific region from the at least one frame image based on the ratio of the average brightness in a region of interest included in the at least one frame image to the variance of brightness in the region of interest.

[0007] In one embodiment, the control unit acquires a plurality of frame images as the at least one frame image, and extracts the specific region based on the ratio of the average brightness in the region of interest to the variance of the brightness in the region of interest, as well as at least one of the feature quantities in the time direction and the feature quantities in other spatial directions of the plurality of frame images.

[0008] In one embodiment, the control unit extracts a first region from the plurality of frame images based on the ratio of the average brightness of the first region of interest included in the plurality of frame images to the variance of the brightness in the first region of interest, extracts a second region from the plurality of frame images based on at least one of the feature quantities in the second region of interest included in the plurality of frame images, and determines that the region included in both the first region and the second region is the specified region.

[0009] In one embodiment, the control unit determines that a first pixel of interest is included in the first region if the ratio of the average brightness in the first region of interest for a first pixel of interest included in the plurality of frame images to the variance of the brightness in the first region of interest is less than a predetermined first value.

[0010] In one embodiment, the control unit calculates the correlation of brightness between different frame images in the second region of interest for a second pixel of interest included in the plurality of frame images as a feature quantity in the time direction.

[0011] As one embodiment, when the feature amount in the time direction is less than a predetermined second value, the control unit determines that the second pixel of interest is included in the second region.

[0012] As one embodiment, the control unit determines that the specific region is a blood flow region.

[0013] As one embodiment, the control unit enhances the contrast between the region included in the specific region and the region not included in the specific region in the at least one frame image.

[0014] As one embodiment, the control unit suppresses the luminance of the region included in the specific region in the at least one frame image.

[0015] As one embodiment, the control unit differentiates between the region included in the specific region and the region not included in the specific region, and causes the display unit to display the at least one frame image.

[0016] As one embodiment, the control unit causes the display unit to display the at least one frame image by making the display colors of the region included in the specific region and the region not included in the specific region different. <\

[0017] As one embodiment, the control unit causes the display unit to display the at least one frame image by increasing the transparency of the region included in the specific region compared to the transparency of the region not included in the specific region.

[0018] As one embodiment, based on whether the ratio of the average luminance to the variance of the luminance in the first region of interest for the first pixel of interest included in the at least one frame image belongs to any of a plurality of predetermined numerical ranges, the control unit extracts a plurality of types of the specific regions from the at least one frame image.

[0019] In one embodiment, the control unit extracts multiple types of specific regions from the multiple frame images based on which of a predetermined set of numerical ranges the ratio of the average brightness in a first region of interest for a first pixel of interest included in the multiple frame images to the variance of the brightness in the first region of interest belongs to, and which of a predetermined set of numerical ranges the at least one of the feature quantities in a second region of interest for a second pixel of interest included in the multiple frame images belongs to, and which of a predetermined set of numerical ranges the at least one of the feature quantities in a second region of interest for a second pixel of interest included in the multiple frame images belongs to.

[0020] An information processing system as one aspect of the present disclosure comprises the information processing device and the probe having the ultrasonic transducer.

[0021] An information processing method in one aspect of the present disclosure involves a control unit of an information processing device acquiring at least one frame image generated using line data indicating the intensity of reflected waves to ultrasonic waves transmitted radially from an ultrasonic transducer moving inside biological tissue, and extracting a specific region from the at least one frame image based on the ratio of the average brightness in a region of interest included in the at least one frame image to the variance of brightness in the region of interest.

[0022] A computer program in one aspect of the present disclosure causes the computer to perform the following processes: acquire at least one frame image generated using line data indicating the intensity of reflected waves to ultrasonic waves transmitted radially from an ultrasonic transducer moving inside biological tissue; and extract a specific region from the at least one frame image based on the ratio of the average brightness in a region of interest included in the at least one frame image to the variance of brightness in the region of interest. [Effects of the Invention]

[0023] According to one embodiment of this disclosure, the object to be observed can be identified with higher accuracy based on the results of observing a cross-section of biological tissue using ultrasound. [Brief explanation of the drawing]

[0024] [Figure 1] This is a perspective view of a diagnostic support system according to one embodiment of the present disclosure. [Figure 2] This is a perspective view of a probe and drive unit according to one embodiment of the present disclosure. [Figure 3] This is a block diagram showing the configuration of a diagnostic support device according to one embodiment of the present disclosure. [Figure 4A] This figure shows an example in which an IVUS image is acquired by a diagnostic support device according to one embodiment of the present disclosure. [Figure 4B] This figure shows an example in which an IVUS image is acquired by a diagnostic support device according to one embodiment of the present disclosure. [Figure 4C] This figure shows an example in which an IVUS image is acquired by a diagnostic support device according to one embodiment of the present disclosure. [Figure 4D] This figure shows an example in which an IVUS image is acquired by a diagnostic support device according to one embodiment of the present disclosure. [Figure 4E] This figure shows an example in which an IVUS image is acquired by a diagnostic support device according to one embodiment of the present disclosure. [Figure 5A] This is a flowchart showing the operation of a diagnostic support device according to one embodiment of this disclosure. [Figure 5B] This is a flowchart showing the operation of a diagnostic support device according to one embodiment of this disclosure. [Figure 6A] This figure shows an example of spatiotemporal features in IVUS images. [Figure 6B] This figure shows an example of spatiotemporal features in IVUS images. [Figure 7] This figure shows an example of extracting blood flow regions from IVUS images based on multiple spatiotemporal features. [Figure 8] This figure shows an example of applying post-processing to IVUS images. [Figure 9] This figure shows an example of extracting multiple regions from an IVUS image based on multiple spatiotemporal features. [Modes for carrying out the invention]

[0025] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0026] (System Configuration) Referring to Figure 1, the configuration of the diagnostic support system 10 as an information processing system according to this embodiment will be described. Figure 1 is a perspective view of the diagnostic support system 10 according to this embodiment. The diagnostic support system 10 comprises a diagnostic support device 11 as an information processing device, a cable 12, a drive unit 13, a keyboard 14, a pointing device 15, and a display 16.

[0027] In this embodiment, the diagnostic support device 11 is a dedicated computer specialized for image diagnosis, but it may also be a general-purpose computer such as a PC, WS, or tablet terminal. "PC" is an abbreviation for Personal Computer. WS is an abbreviation for Work Station.

[0028] Cable 12 is used to connect the diagnostic support device 11 and the drive unit 13 to transmit and receive information.

[0029] The drive unit 13 is used in conjunction with the probe 20, which will be described later, and is a device that drives the probe 20. The drive unit 13 is also called an MDU. "MDU" is an abbreviation for Motor Drive Unit. The probe 20 is used for IVUS. The probe 20 is also called an IVUS catheter or a diagnostic imaging catheter. Details of the drive unit 13 and probe 20 will be described later with reference to Figure 2.

[0030] The keyboard 14, pointing device 15, and display 16 are connected to the diagnostic support device 11 via any cable or wirelessly. The display 16 is, for example, an LCD, an organic EL display, or an HMD. "LCD" is an abbreviation for Liquid Crystal Display. "EL" is an abbreviation for Electro Luminescence. "HMD" is an abbreviation for Head-Mounted Display.

[0031] The diagnostic support system 10 further includes, as an option, a connection terminal 17 and a cart unit 18. The connection terminal 17 is used to connect the diagnostic support device 11 to an external device. The connection terminal 17 is, for example, a USB terminal. "USB" is an abbreviation for Universal Serial Bus. The external device is, for example, a recording medium such as a magnetic disk drive, magneto-optical disk drive, or optical disk drive.

[0032] The cart unit 18 is a cart with casters for mobility. The cart body of the cart unit 18 houses the diagnostic support device 11, cables 12, and drive unit 13. The top table of the cart unit 18 houses the keyboard 14, pointing device 15, and display 16.

[0033] Referring to Figure 2, the configuration of the probe 20 and drive unit 13 according to this embodiment will be described. Figure 2 is a perspective view of the probe 20 and drive unit 13 according to this embodiment. The probe 20 comprises a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic transducer 25, and a relay connector 26.

[0034] The drive shaft 21 extends through a sheath 23, which is inserted into the body cavity of a living organism, and an outer tube 24 connected to the proximal end of the sheath 23, into the interior of a hub 22 provided at the proximal end of the probe 20. The drive shaft 21 is rotatably mounted inside the sheath 23 and outer tube 24, with an ultrasonic transducer 25 for transmitting and receiving signals at its tip. The relay connector 26 connects the sheath 23 and the outer tube 24.

[0035] The hub 22, drive shaft 21, and ultrasonic transducer 25 are connected to each other so that they move integrally in the axial direction of the probe 20. Therefore, for example, when the hub 22 is pushed toward the tip, the drive shaft 21 and ultrasonic transducer 25 move toward the tip inside the sheath 23. For example, when the hub 22 is pulled toward the base, the drive shaft 21 and ultrasonic transducer 25 move toward the base inside the sheath 23, as indicated by the arrows in Figure 2.

[0036] The drive unit 13 comprises a scanner unit 31, a slide unit 32, and a bottom cover 33. The scanner unit 31 is connected to the diagnostic support device 11 via a cable 12.

[0037] The scanner unit 31 includes a probe connection part 34 that connects to the probe 20 and a scanner motor 35 that is a drive source for rotating the drive shaft 21. The probe connection part 34 is detachably connected to the probe 20 via an insertion port 36 of a hub 22 provided at the base end of the probe 20. Inside the hub 22, the base end of the drive shaft 21 is rotatably supported, and the rotational force of the scanner motor 35 is transmitted to the drive shaft 21. Signals are also transmitted and received between the drive shaft 21 and the diagnostic support device 11 via the cable 12. The diagnostic support device 11 generates tomographic images of the biological lumen and performs image processing based on the signals transmitted from the drive shaft 21.

[0038] The slide unit 32 mounts the scanner unit 31 so that it can move forward and backward, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp section 37, a slide motor 38, and a switch group 39.

[0039] The probe clamp portion 37 is located at a tip-side and coaxial position with the probe connection portion 34, and supports the probe 20 connected to the probe connection portion 34.

[0040] The slide motor 38 is a drive source that generates axial driving force for the probe 20. The scanner unit 31 moves back and forth when driven by the slide motor 38, and consequently the drive shaft 21 moves back and forth in the axial direction of the probe 20. The slide motor 38 is, for example, a servo motor.

[0041] The switch group 39 includes, for example, a forward switch and a pullback switch that are pressed when the scanner unit 31 is moved forward or backward, and a scan switch that is pressed when image rendering is started and stopped. The switch group 39 may include various switches as needed, but is not limited to the examples given here.

[0042] When the forward switch is pressed, the slide motor 38 rotates forward, and the scanner unit 31 moves forward. This pushes the hub 22 connected to the probe connection part 34 of the scanner unit 31 toward the tip, and the drive shaft 21 and ultrasonic transducer 25 move toward the tip inside the sheath 23. On the other hand, when the pullback switch is pressed, the slide motor 38 rotates backward, and the scanner unit 31 moves backward. This sweeps the hub 22 connected to the probe connection part 34 toward the base, and the drive shaft 21 and ultrasonic transducer 25 move toward the base inside the sheath 23.

[0043] When the scan switch is pressed, image rendering begins, and the scanner motor 35 and slide motor 38 are driven to retract the scanner unit 31. The user, such as the surgeon, connects the probe 20 to the scanner unit 31 in advance, and when image rendering begins, the drive shaft 21 rotates around the central axis of the probe 20 and moves axially towards the base end. The scanner motor 35 and slide motor 38 stop when the scan switch is pressed again, and image rendering ends.

[0044] The bottom cover 33 covers the entire circumference of the bottom surface and the side surface on the bottom side of the slide unit 32, and can move freely in and out of the way of the bottom surface of the slide unit 32.

[0045] Referring to Figure 3, the configuration of the diagnostic support device 11 according to this embodiment will be described. The diagnostic support device 11 includes components such as a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

[0046] The control unit 41 is one or more processors. The processors are general-purpose processors such as CPUs or GPUs, or dedicated processors specialized for specific processing. "CPU" is an abbreviation for Central Processing Unit. "GPU" is an abbreviation for Graphics Processing Unit. The control unit 41 may include one or more dedicated circuits, or one or more processors in the control unit 41 may be replaced with one or more dedicated circuits. Dedicated circuits are, for example, FPGAs or ASICs. "FPGA" is an abbreviation for Field-Programmable Gate Array. "ASIC" is an abbreviation for Application Specific Integrated Circuit. The control unit 41 controls each part of the diagnostic support system 10, including the diagnostic support device 11, and performs information processing related to the operation of the diagnostic support device 11.

[0047] The memory unit 42 is one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. Semiconductor memories are, for example, RAM or ROM. "RAM" is an abbreviation for Random Access Memory. "ROM" is an abbreviation for Read Only Memory. RAM is, for example, SRAM or DRAM. "SRAM" is an abbreviation for Static Random Access Memory. "DRAM" is an abbreviation for Dynamic Random Access Memory. ROM is, for example, EEPROM. "EEPROM" is an abbreviation for Electrically Erasable Programmable Read Only Memory. The memory unit 42 functions, for example, as a main memory, an auxiliary memory, or a cache memory. The memory unit 42 stores information used for the operation of the diagnostic support device 11 and information obtained by the operation of the diagnostic support device 11.

[0048] The communication unit 43 is one or more communication interfaces. The communication interface is a wired LAN interface, a wireless LAN interface, or an image diagnostic interface that receives and A / D converts IVUS signals. "LAN" is an abbreviation for Local Area Network. "A / D" is an abbreviation for Analog to Digital. The communication unit 43 receives information used for the operation of the diagnostic support device 11 and transmits information obtained by the operation of the diagnostic support device 11. In this embodiment, the drive unit 13 is connected to the image diagnostic interface included in the communication unit 43.

[0049] The input unit 44 is one or more input interfaces. The input interface is, for example, a USB interface or an HDMI® interface. "HDMI" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 accepts operations to input information used for the operation of the diagnostic support device 11. In this embodiment, the keyboard 14 and the pointing device 15 are connected to the USB interface included in the input unit 44, but the keyboard 14 and the pointing device 15 may be connected to the wireless LAN interface included in the communication unit 43.

[0050] The output unit 45 is one or more output interfaces. The output interface is, for example, a USB interface or an HDMI® interface. The output unit 45 outputs information obtained by the operation of the diagnostic support device 11. In this embodiment, the display 16 is connected to the HDMI® interface included in the output unit 45.

[0051] The functions of the diagnostic support device 11 are realized by executing a diagnostic support program (computer program) according to this embodiment on the processor included in the control unit 41. In other words, the functions of the diagnostic support device 11 are realized by software. The diagnostic support program is a program that causes the computer to execute the processing of steps included in the operation of the diagnostic support device 11, thereby realizing the functions corresponding to the processing of those steps. In other words, the diagnostic support program is a program that causes the computer to function as the diagnostic support device 11.

[0052] The program can be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs on which the program is recorded. "DVD" is an abbreviation for Digital Versatile Disc. "CD-ROM" is an abbreviation for Compact Disc Read Only Memory. The program may also be distributed by storing it in server storage and transferring it from the server to other computers via a network. The program may also be provided as a program product.

[0053] A computer, for example, stores a program recorded on a portable storage medium or a program transferred from a server in its main memory. Then, the computer reads the program stored in the main memory with its processor and executes the processing according to the read program. The computer may also read a program directly from a portable storage medium and execute the processing according to the program. The computer may also execute the processing according to the received program sequentially each time a program is transferred to it from a server. Processing may also be performed by a so-called ASP-type service that does not transfer programs from the server to the computer, but realizes its function only through execution instructions and result acquisition. "ASP" is an abbreviation for Application Service Provider. A program includes information used for processing by an electronic computer and information equivalent to a program. For example, data that is not a direct instruction to the computer but has the nature of defining the computer's processing falls under "information equivalent to a program".

[0054] Some or all of the functions of the diagnostic support device 11 may be implemented by a dedicated circuit included in the control unit 41. In other words, some or all of the functions of the diagnostic support device 11 may be implemented by hardware. Furthermore, the diagnostic support device 11 may be implemented by a single information processing device or by the cooperation of multiple information processing devices.

[0055] (Acquisition of IVUS images) The principle of acquiring IVUS images will be explained with reference to Figures 4A to 4E. Figures 4A to 4E show examples of how IVUS images are acquired by the diagnostic support device 11 according to this embodiment.

[0056] As described above, during IVUS image acquisition, the drive shaft 21 rotates around the central axis of the probe 20 and is pulled back towards the proximal end where the drive unit 13 is located, in accordance with the retraction of the scanner unit 31. The ultrasonic transducer 25, located at the tip of the drive shaft 21, also rotates around the central axis of the probe 20 in conjunction with the drive shaft 21, while transmitting and receiving ultrasonic signals inside biological tissue such as blood vessels, and moves towards the proximal end of the probe 20.

[0057] Figure 4A schematically shows a plane perpendicular to the central axis of the probe 20 that passes through the location where the ultrasonic transducer 25 is located. The ultrasonic transducer 25 rotates at a constant velocity around the central axis of the probe 20 within the biological tissue, transmitting ultrasonic pulse waves radially and receiving reflected pulse waves from each direction to generate a received signal. Figures 4A to 4E show an example in which the ultrasonic transducer 25 transmits 512 pulse waves at equal intervals while rotating 360 degrees, and generates received signals from reflected waves from each direction. In this way, each direction in which ultrasonic pulses are transmitted around the entire circumference of the ultrasonic transducer 25 is called a line (scanning line), and the data indicating the intensity of the ultrasonic reflected wave at each line is called line data.

[0058] Figure 4B shows an example of acquiring a luminance image by converting the received signal of the reflected wave in each line to luminance. As shown in Figure 4B, the received signal in each line is converted into a detection waveform that shows its amplitude, and then logarithmically converted. Then, luminance conversion is performed to associate the signal strength with the luminance in the image. The control unit 41 of the diagnostic support device 11 constructs luminance data for one line by assigning 256 levels of grayscale color according to the intensity of the reflected signal, for example. In each figure of Figure 4B, the horizontal axis represents time, and the vertical axis represents the signal strength for the received signal, detection waveform, and logarithmically converted signal. Since the speed at which ultrasound travels is constant, the time from transmitting the ultrasound to receiving the reflected wave is proportional to the distance between the ultrasonic transducer 25 and the object that reflected the ultrasound. Therefore, the line data obtained by the processing in Figure 4B is data that shows the intensity of the reflected signal according to the distance from the ultrasonic transducer 25 in terms of the luminance of the image.

[0059] Figure 4C shows an example of stacking line data from lines 1 to 512 that have undergone brightness conversion. By transforming this image into polar coordinates, a cross-sectional image of biological tissue, as shown in Figure 4D, is obtained. A single cross-sectional image is generated from 512 line data acquired while the ultrasonic transducer 25 rotates 360 degrees around the central axis of the probe 20. Such a single cross-sectional image is called a frame image, and its data is called frame data. The position of each pixel included in the frame image related to the IVUS image is determined by the distance r from the ultrasonic transducer 25 and the rotation angle θ from the reference angle of the ultrasonic transducer 25.

[0060] As mentioned above, IVUS images are acquired while the ultrasonic transducer 25 is pulled back towards the proximal end of the probe 20. Therefore, as shown in Figure 4E, when the ultrasonic transducer 25 moves within the blood vessel, the imaging catheter (probe 20) is swept (pulled back) in the longitudinal direction of the blood vessel, and continuous frame data is acquired. Here, since the ultrasonic transducer 25 acquires frame data one by one as it moves, the longitudinal direction of the blood vessel in the multiple frames of data acquired consecutively corresponds to the time direction. Hereafter, to distinguish it from the time direction, two orthogonal directions within the same frame image will be called spatial directions. In Figure 4E, the x and y directions correspond to spatial directions, and the z direction corresponds to the time direction. Figures 4A to 4E illustrate an example where 512 line data are acquired each time the probe 20 rotates, but the number of line data is not limited to 512; for example, it could be 256 or 1024.

[0061] (Example of operation) As described above, the IVUS image is acquired as at least one frame image generated using line data indicating the intensity of reflected waves to ultrasound transmitted radially from an ultrasonic transducer 25 moving inside biological tissue. The diagnostic support device 11 according to this embodiment extracts a specific region from the at least one frame image based on the ratio of the average brightness in the region of interest included in the at least one frame image to the variance of the brightness in the region of interest. The operation of the diagnostic support system 10 according to this embodiment will be described with reference to Figures 5A and 5B. The operation of the diagnostic support device 11 described with reference to Figures 5A and 5B corresponds to the information processing method according to this embodiment, and the operation of each step is executed based on the control of the control unit 41.

[0062] Before the start of the flow shown in Figures 5A and 5B, the user inserts the probe 20 into the probe connection part 34 and probe clamp part 37 of the drive unit 13, connecting and securing it to the drive unit 13. Then, the probe 20 is inserted into the target site within the blood vessel. The diagnostic support system 10 then rotates the ultrasound transducer 25 around the central axis of the probe 20 and pulls it back to begin acquiring IVUS images.

[0063] In step S11 of Figure 5A, the control unit 41 generates frame data of the IVUS image. In this embodiment, an example is described in which the diagnostic support device 11 generates and acquires frame data before polar coordinate transformation and performs the processing in each step of Figure 5A and Figure 5B, but similar processing may be performed on frame data after polar coordinate transformation. Hereinafter, the frame image to which the processing in steps S12 to S30 is performed will be referred to as the "target frame".

[0064] Next, the control unit 41 performs the processing in steps S12 to S17 for each pixel included in the target frame and determines whether the pixel is included in a specific region such as a blood flow region. Hereinafter, the pixel to be processed will be referred to as the "pixel of interest".

[0065] In step S12, the control unit 41 determines whether the brightness of the pixel of interest is greater than a predetermined threshold T1. If it is greater (YES in step S12), the process proceeds to step S16; otherwise, it proceeds to step S13. The threshold T1 can be determined, for example, according to the type of catheter or frequency mode.

[0066] In step S13, the control unit 41 extracts a region of interest (ROI) from the frame data, centering on the pixel of interest. Here, we describe an example where the region of interest is 15 × 15 × 15 pixels in two spatial directions and one temporal direction, but the shape and size of the region of interest are not limited to this and can be set appropriately according to the image resolution, etc. Furthermore, the shape and size of the region of interest may be different depending on the type of spatiotemporal feature (spatiotemporal feature quantity) calculated in step S14.

[0067] In step S14, the control unit 41 calculates two spatiotemporal features a and b based on the luminance values ​​of the pixels included in the region of interest. In this embodiment, an example is described in which the correlation coefficient R between frame images is used as such spatiotemporal feature a, and the ratio q of the luminance average of neighboring pixels to the luminance variance of neighboring pixels is used as such spatiotemporal feature b.

[0068] Figure 6A shows an example of calculating the correlation coefficient between frame images. In Figure 6A, f represents the target frame acquired at time t, and g - is the previous frame image acquired at time (t-1), g + This represents the next frame image acquired at time (t+1). The correlation coefficient R(d) of the pixel of interest is... x d y ) is calculated using the following formula.

number

[0069] however,

number

[0070] Also, (d x d y ) is the coordinate of the pixel of interest in the target frame. In the example in Figure 6A, the pixel of interest is at the center, with height h and width w as the window size, and adjacent frame images (target frame image f and adjacent frame image g) are included.+ or g - The correlation coefficient between the two is calculated. In this embodiment, the window size is 15 × 15, similar to the spatial size of the region of interest, but is not limited to this. The window size can be, for example, the distance between two speckle noise peaks. This distance can be calculated from the duration of the pulse waveform, which is calculated as the full width at half maximum of the autocorrelation coefficient including the noise.

[0071] In general, the presence of various blood cells in the blood flow region results in low uniformity of ultrasound images and a small correlation coefficient between frames. On the other hand, stents and calcified areas are uniformly formed, resulting in high uniformity of ultrasound images and a large correlation coefficient between frames. The correlation coefficient between frames in tissue and fibrous plaque areas is known to be somewhere in between these two extremes.

[0072] Figure 6B shows an example of calculating the ratio of the average brightness to the variance of neighboring pixels. The ratio q0(t) of the average brightness to the variance of neighboring pixels for the pixel of interest is calculated by the following formula.

number

[0073] however,

number

[0074] As shown in Equation 3, q0(t) is calculated by dividing the square root of the brightness variance of the region of interest (ROI) by the brightness mean of the region of interest. The ratio of brightness mean to brightness variance is known to be low in blood flow regions, high in tissue and fibrous plaque regions, and intermediate in stents and calcified areas.

[0075] As described above, it is known that the values of the spatio-temporal features calculated in step S14 differ depending on the type of region where the target pixel exists. Specifically, in the blood flow region, it is known that both the spatio-temporal feature a (correlation coefficient between frame images) and the spatio-temporal feature b (ratio of the luminance average of neighboring pixels to the luminance variance of neighboring pixels) become small. Therefore, in the present embodiment, the control unit 41 determines a region where both the spatio-temporal features a and b are below a predetermined threshold as the blood flow region. Specifically, as shown in FIG. 7, the control unit 41 generates maps of the spatio-temporal features a and b from the IVUS image before polar coordinate conversion, determines a range where the values of both the spatio-temporal features a and b are small as blood flow region pixels, and determines the other range as non-blood flow region pixels. Then, the control unit 41 generates a tissue property map M(r i ,θ j ) consisting of the blood flow region (pixel value = 1) in FIG. 7 and the other region (pixel value = 0). Here, r i is the distance from the ultrasonic vibrator 25, and θ j is the rotation angle from the reference angle.

[0076] In step S15, the control unit 41 determines whether the spatio-temporal feature a (correlation coefficient between frame images) is below a predetermined threshold T2 and whether the spatio-temporal feature b (ratio of the luminance average of neighboring pixels to the luminance variance of neighboring pixels) is below a predetermined threshold T3. If the spatio-temporal feature a is not below the threshold T2 or the spatio-temporal feature b is not below the threshold T3 (NO in step S15), the process proceeds to step S16. If the spatio-temporal feature a is below the threshold T2 and the spatio-temporal feature b is below the threshold T3 (YES in step S15), the process proceeds to step S17.

[0077] In step S16, the control unit 41 determines that the target pixel is not in the blood flow region, sets M(r i ,θ j ) = 0, and proceeds to step S18. In step S17, the control unit 41 determines that the target pixel is in the blood flow region, sets M(r i ,θ j ) = 1, and proceeds to step S18.

[0078] In step S18, the control unit 41 determines whether the processing in steps S12 to S17 has been performed for all pixels of the target frame. If it has not been performed (NO in step S18), it returns to step S12 and performs the processing in steps S12 to S17 with the unprocessed pixels as the pixels of interest. If it has been performed (YES in step S18), it proceeds to step S19 in Figure 5B.

[0079] In steps S19 to S23, the control unit 41 determines that the tissue characteristics map is a blood flow region (M(r i ,θ j )=1), but exclude regions of extremely small size from the blood flow region (i.e., M(r i ,θ j The process is performed to set )=0).

[0080] In step S19, the control unit 41 counts the number of pixels included in blood flow regions that are continuously connected in the tissue characteristics map. In step S20, the control unit 41 determines whether the counted value N exceeds a predetermined threshold T4. If it does not exceed the threshold (NO in step S20), the process proceeds to step S21; if it does exceed the threshold (YES in step S20), the process proceeds to step S22.

[0081] In step S21, the control unit 41 sets the value M(r) of the tissue characteristics map for the pixel of interest. i ,θ j The value of the tissue properties map M(r i ,θ j Set ) to 1 and proceed to step S23.

[0082] In step S23, the control unit 41 determines whether the processing in steps S19 to S22 has been performed on all pixels of the target frame. If it has been performed (YES in step S23), the process proceeds to step S24. If it has not been performed (NO in step S23), the process returns to step S19 and performs the processing from step S19 onwards on the unprocessed pixels.

[0083] In step S24, the control unit 41 scans each line in the R direction and checks the R coordinate r of the non-blood flow region pixel with the highest brightness value. l This is defined as the boundary of the blood flow region. Next, in step S25, the control unit 41 defines everything outside the boundary of the blood flow region as a non-blood flow region. That is, r i ≧r l In this case, the control unit 41 controls the tissue properties map M(r i ,θ j The process is performed to set ) = 0.

[0084] Then, in step S26, the control unit 41 determines whether or not the processes in steps S24 and S25 have been performed for all lines. If they have been performed (YES in step S26), the process proceeds to step S27; otherwise, the process returns to step S24 and the same process is performed for the unprocessed lines.

[0085] In steps S27 to S29, the control unit 41 determines that the tissue characteristics map is a blood flow region (M(r i ,θ j )=1) For each pixel, a minimum value filter is applied to suppress the brightness value. Figure 8 shows an example of images before and after brightness value suppression in the blood flow region. By performing post-processing such as brightness value suppression, users can easily visually distinguish between the blood flow region and other areas.

[0086] In step S27, the control unit 41, based on the processing up to step S26, determines the blood flow region (M(r) in the tissue properties map. i ,θ j It is determined whether or not it is set as )=1). If it is set as a blood flow region (YES in step S27), proceed to step S28; if it is not set as a blood flow region (NO in step S27), proceed to step S29. In step S28, the control unit 41 applies a minimum value filter to suppress the brightness value and proceeds to step S29.

[0087] In step S29, the control unit 41 determines whether the processing in steps S27 and S28 has been performed on all pixels of the target frame. If it has been performed (YES in S29), the process proceeds to step S30; otherwise, it returns to step S27.

[0088] In step S30, the control unit 41 processes the blood cell noise reduction image I that was processed in steps S12 to S29. out(t) The output is then generated. In step S31, the control unit 41 determines whether or not there is a next frame image. If there is (YES in step S31), the process returns to step S12; otherwise, the process ends.

[0089] As described above, in this embodiment, a specific region, such as a blood flow region, is extracted from an IVUS image based on the ratio of the average brightness in the region of interest included in the IVUS image to the variance of the brightness in that region of interest. Therefore, it becomes possible to identify the object to be observed with higher accuracy from the results of observing a cross-section of biological tissue using ultrasound.

[0090] In this embodiment, an example was described in which the biological tissue to be observed is a blood vessel, but it may also be an organ such as the heart, or other biological tissues. Also, in this embodiment, an example was described in which the region to be distinguished as a specific region is the blood flow region of the object to be observed, but it is not limited to this, and may also be a stent, guidewire, blood vessel wall, calcified lesion, plaque, or other reflective material.

[0091] Furthermore, in this embodiment, the control unit 41 acquires multiple frame images as IVUS images. The control unit 41 then extracts a specific region based on the ratio of the average brightness in the region of interest to the variance of brightness in that region of interest, as well as at least one of the temporal and spatial features of the multiple frame images. Specifically, the control unit 41 extracts a specific region based on the correlation of brightness between the frame images, in addition to the ratio with the variance of brightness. Therefore, by using multiple features rather than just one, it is possible to extract a specific region with higher accuracy.

[0092] In this embodiment, examples have been described in which the ratio of the average brightness in the region of interest to the variance of the brightness in the region of interest, and the correlation coefficient between frame images are used as spatiotemporal features for extracting a specific region, but the embodiment is not limited to these. The control unit 41 may also use, for example, frequency features (such as the power spectral ratio of low-frequency components to high-frequency components), texture features (such as LBP features), etc. Furthermore, in addition to these, the control unit 41 may also use general image features such as image statistics (normal distribution, Rayleigh distribution), histogram skewness, kurtosis, and second moment. "LBP" is an abbreviation for Local Binary Pattern.

[0093] As explained with reference to Figure 7, in this embodiment, the control unit 41 extracts a first region from multiple frame images based on the ratio of the average brightness of the first region of interest contained in the multiple frame images to the variance of the brightness in the first region of interest. Furthermore, the control unit 41 extracts a second region from the multiple frame images based on at least one feature quantity in the second region of interest contained in the multiple frame images. The control unit 41 then determines that a region commonly contained in the first and second regions is a specific region such as a blood flow region. Therefore, it is possible to extract a specific region with higher accuracy by using multiple feature quantities.

[0094] In this embodiment, it is determined whether a pixel of interest is included in a specific region, such as a blood flow region, or a candidate for such a region, based on whether spatiotemporal features such as the ratio of luminance dispersion in the region of interest and the luminance correlation between frame images are below predetermined thresholds (first value, second value). Therefore, according to this embodiment, specific regions can be extracted efficiently.

[0095] In this embodiment, in steps S27 to S29, the contrast between the area included in the specific region and the area not included in the specific region is enhanced in at least one frame image. Specifically, the brightness of the area included in the specific region is suppressed in at least one frame image. Therefore, according to this embodiment, by displaying an image that has undergone such processing on the display 16, the user can easily distinguish and visually identify specific regions such as blood flow areas.

[0096] The method of distinguishing between areas included in a specific region and areas not included in it, and displaying at least one frame image on the display unit, is not limited to contrast adjustment. For example, the control unit 41 may display at least one frame image on the display unit by using different display colors for areas included in the specific region and areas not included in the specific region. Alternatively, the control unit 41 may display at least one frame image on the display unit by increasing the transparency of the areas included in the specific region compared to the transparency of the areas not included in the specific region. By distinguishing between areas included in a specific region and areas not included in this way, the user can easily distinguish between the specific region and other areas by visual inspection.

[0097] (modified version) In the configuration described with reference to Figures 5A and 5B, a specific region, such as a blood flow region, is extracted based on the ratio of the average brightness in the region of interest for a pixel of interest included in the IVUS image to the variance of brightness in that region of interest. However, the specific region to be extracted is not limited to one type; multiple types of specific regions may be extracted (see Figure 9).

[0098] As mentioned above, it is generally known that the correlation coefficient between frame images is small in the blood flow region, large in stents and calcified areas, and intermediate in tissue and fibrous plaque. Therefore, the control unit 41 may extract multiple types of specific regions from the frame image based on which of a predetermined set of numerical ranges the ratio of the average brightness in the region of interest to the variance of the brightness in that region of interest for a pixel of interest included in the frame image belongs to. In the example in Figure 9, the control unit 41 compares the ratio of the average brightness to the variance of the brightness in the region of interest for each pixel of interest with the magnitudes of thresholds 3 and 4 to determine which of numerical ranges D, E, and F it belongs to. Based on such determination results for each pixel of interest, the control unit 41 may then extract specific regions of blood cells, stents / calcification, and tissue / fibrous plaque.

[0099] Furthermore, as mentioned above, the ratio of the average luminance to the luminance variance is known to be low in the blood flow region, high in the tissue and fibrous plaque region, and intermediate in the stent and calcified areas. Therefore, the control unit 41 may extract multiple specific regions not only by using the ratio of the average luminance to the luminance variance in the region of interest for the aforementioned pixel of interest, but also by using these properties for multiple spatiotemporal features in combination.

[0100] In other words, the control unit 41 determines which of a predetermined set of numerical ranges (first numerical range) the ratio of the average brightness in a first region of interest for a first pixel of interest included in a plurality of frame images to the variance of the brightness in the first region of interest belongs to. Furthermore, the control unit 41 determines which of a predetermined set of numerical ranges (second numerical range) at least one of the feature quantities in a second region of interest for a second pixel of interest included in a plurality of frame images belongs to. Based on these determination results, the control unit 41 may extract multiple specific regions from multiple types of frame images. This makes it possible to extract multiple specific regions with high accuracy based on multiple spatiotemporal features. In the example in Figure 9, for each pixel of interest, the control unit 41 compares the ratio of the average brightness to the variance of the brightness in the region of interest with the magnitudes of threshold 3 and threshold 4 to determine which of the numerical ranges D, E, and F it belongs to. Furthermore, the control unit 41 compares the correlation coefficient between frames in the region of interest for each pixel of interest with the magnitudes of threshold 1 and threshold 2 to determine which of the numerical ranges A, B, or C it belongs to. Based on these determination results, the control unit 41 may then extract specific regions of blood cells, stents / calcifications, and tissue / fibrous plaques.

[0101] As spatiotemporal features, correlation coefficients between frame images, frequency features, texture features, etc., may be used. Furthermore, in addition to these, general image features such as image statistics, histogram skewness, kurtosis, and second moment may also be used in combination.

[0102] This disclosure is not limited to the embodiments described above. For example, multiple blocks described in the block diagram may be combined, or a single block may be divided. Instead of executing multiple steps described in the flowchart in chronological order as described, they may be executed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary. Other modifications are possible without departing from the spirit of this disclosure. [Explanation of Symbols]

[0103] 10. Diagnostic support system 11. Diagnostic support device 12 Cables 13 Drive Unit 14-key keyboard 15 Pointing devices 16 displays 17 Connection terminals 18 Cart Units 20 probes 21 Drive shaft 22 Hubs 23 Sheath 24 Outer tube 25 Ultrasonic transducer 26 relay connectors 31 Scanner Unit 32 Slide Units 33 Bottom Cover 34 Probe connection section 35 Scanner motor 36 outlets 37 Probe clamp section 38 Slide motor 39 Switch Group 41 Control Unit 42 Storage section 43 Communications Department 44 Input section 45 Output section

Claims

1. Multiple frame images are obtained using line data that shows the intensity of reflected waves to ultrasound waves transmitted radially from an ultrasonic transducer moving within biological tissue. Based on the ratio of the average brightness in a first region of interest contained in the plurality of frame images to the variance of brightness in the first region of interest, the first region is extracted from the plurality of frame images. Based on at least one of the temporal features and spatial features in the second region of interest contained in the plurality of frame images, which is obtained by excluding the ratio of the mean luminance to the variance luminance, the second region is extracted from the plurality of frame images. The region that is included in both the first region and the second region is extracted as a specific region. An information processing device equipped with a control unit.

2. The information processing apparatus according to claim 1, wherein the control unit determines that a first pixel of interest is included in the first region if the ratio of the average brightness in the first region of interest for a first pixel of interest included in the plurality of frame images to the variance of brightness in the first region of interest is less than a predetermined first value.

3. The information processing apparatus according to claim 1 or 2, wherein the control unit calculates the correlation of brightness between different frame images in the second region of interest for a second pixel of interest included in the plurality of frame images as a feature quantity in the time direction.

4. The information processing apparatus according to claim 3, wherein the control unit determines that the second pixel of interest is included in the second region when the time-direction feature quantity is less than a predetermined second value.

5. The information processing apparatus according to any one of claims 1 to 4, wherein the control unit determines the specific region to be a blood flow region.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the control unit enhances the contrast between the region included in the specific region and the region not included in the specific region in the plurality of frame images.

7. The information processing apparatus according to claim 6, wherein the control unit suppresses the brightness of the region included in the specific region in the plurality of frame images.

8. The information processing apparatus according to any one of claims 1 to 7, wherein the control unit distinguishes between areas included in the specific area and areas not included in the specific area, and displays the plurality of frame images on the display unit.

9. The information processing apparatus according to claim 8, wherein the control unit displays the plurality of frame images on the display unit with different display colors for areas included in the specific area and areas not included in the specific area.

10. The information processing apparatus according to claim 8 or 9, wherein the control unit increases the transparency of the area included in the specific area to be higher than the transparency of the area not included in the specific area, and displays the plurality of frame images on the display unit.

11. The information processing apparatus according to claim 1, wherein the control unit extracts a plurality of types of specific regions from the plurality of frame images based on which of a plurality of predetermined numerical ranges the ratio of the average brightness in a first region of interest for a first pixel of interest included in the plurality of frame images to the variance of the brightness in the first region of interest belongs to.

12. The information processing apparatus according to claim 1, wherein the control unit extracts a plurality of specific regions from the plurality of frame images based on which of a plurality of predetermined numerical ranges the ratio of the average brightness in a first region of interest for a first pixel of interest included in the plurality of frame images to the variance of the brightness in the first region of interest includes the first pixel of interest, and which of a plurality of predetermined numerical ranges the at least one feature quantity in a second region of interest for a second pixel of interest included in the plurality of frame images belongs to.

13. An information processing device according to any one of claims 1 to 12, The probe having the ultrasonic transducer An information processing system equipped with the following features.

14. The control unit of the information processing device, Multiple frame images are obtained using line data that shows the intensity of reflected waves to ultrasound waves transmitted radially from an ultrasonic transducer moving within biological tissue. Based on the ratio of the average brightness in a first region of interest contained in the plurality of frame images to the variance of brightness in the first region of interest, the first region is extracted from the plurality of frame images. Based on at least one of the temporal features and spatial features in the second region of interest contained in the plurality of frame images, which is obtained by excluding the ratio of the mean luminance to the variance luminance, the second region is extracted from the plurality of frame images. The region that is included in both the first region and the second region is extracted as a specific region. Information processing methods.

15. A process for acquiring multiple frame images generated using line data indicating the intensity of reflected waves to ultrasound waves transmitted radially from an ultrasonic transducer moving within biological tissue, A process to extract a first region from the plurality of frame images based on the ratio of the average brightness of the first region of interest contained in the plurality of frame images to the variance of the brightness in the first region of interest, A process to extract a second region of interest from the plurality of frame images based on at least one of the temporal features and spatial features in the second region of interest contained in the plurality of frame images, which is obtained by subtracting the ratio of the mean brightness to the variance of brightness. A process for extracting a specific region that is included in both the first region and the second region. A computer program that causes a computer to execute a command.

Images