Medical image processing device, method of operating a medical image processing device, program, and recording medium

Abstract

The present invention provides a medical image processing device for displaying B-mode images from an endoscopic ultrasound device, a method for operating the medical image processing device, a program, and a recording medium. [Solution] A medical image processing device equipped with a processor that displays a B-mode image of an ultrasound endoscope on a display unit, wherein the processor displays multiple class information corresponding to multiple areas of interest in the B-mode image on the display unit, and one of the multiple areas of interest is an organ region.

Description

Technical Field

[0001] The present invention relates to a medical image processing apparatus, a method of operating a medical image processing apparatus, a program, and a recording medium, and particularly relates to a technique for notifying a user of useful information during observation of a medical image.

Background Art

[0002] Conventionally, as an image processing apparatus having this type of function, the one described in Patent Document 1 has been proposed.

[0003] In the endoscopic image processing apparatus described in Patent Document 1, observation images of a subject are sequentially input, a process for detecting a lesion candidate region is performed from the observation images, and when a lesion candidate region is continuously detected, for the observation image of the subject input after the elapse of the first hour from the timing when the detection of the lesion candidate region was started, the lesion candidate region is emphasized by a marker image. Further, when the second hour elapses after the first hour has elapsed, the emphasis process is terminated, and a notification process of adding a notification image (an icon indicating a flag) to a region outside the observation image in the display image is performed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The marker image described in Patent Document 1 emphasizes and displays the detected lesion candidate region by surrounding it, and is for preventing overlooking of the lesion candidate region, and is not information regarding the class that classifies the attention region in the medical image. Further, the marker image emphasizes and displays the detected lesion candidate region by surrounding it with a frame such as a rectangle, and is not superimposed on the position of the lesion candidate region.

[0006] Incidentally, when detecting the location and classified class of a region of interest from a time-series medical image and displaying information about the location and class of the region of interest, it is preferable to overlay the class information on the location of the region of interest so that it is easy for the user to see.

[0007] However, if information about the class of a region of interest is superimposed on the location of the region of interest after detection, the display of class information may interfere with the user's observation of the region of interest.

[0008] This invention has been made in view of these circumstances, and aims to provide a medical image processing device, a method for operating the medical image processing device, a program, and a recording medium that can present information on the position and classified class of a region of interest detected from a time-series medical image to the user in an easy-to-understand manner, without interfering with the user's observation. [Means for solving the problem]

[0009] To achieve the above objective, the invention according to the first embodiment provides a medical image processing apparatus equipped with a processor, wherein the processor performs a medical image acquisition process for sequentially acquiring time-series medical images, a first display control for sequentially displaying the medical images on a display unit, a process for recognizing information regarding the position of a region of interest within a medical image based on the sequentially acquired medical images, a process for classifying the region of interest into one of a plurality of classes based on the sequentially acquired medical images, and a second display control for superimposing class information indicating the classified class onto the position of the region of interest in the medical image displayed on the display unit, and the second display control changes the relative position of the class information with respect to the superimposed region of interest according to the elapsed time since the region of interest was recognized.

[0010] According to a first aspect of the present invention, class information indicating the classification class of a region of interest detected from a time-series medical image is superimposed on the location of the detected region of interest, thereby clearly presenting the location of the region of interest and its classified class to the user. Furthermore, the relative position of the superimposed class information to the region of interest is changed according to the elapsed time since the region of interest was recognized, so that the class information does not interfere with the user's observation.

[0011] In a medical image processing apparatus according to a second aspect of the present invention, the second display control preferably changes the relative position of the class information in a direction away from the position of the area of ​​interest when changing the relative position of the class information according to the elapsed time. This makes it possible to prevent the class information from interfering with the user's observation of the area of ​​interest.

[0012] In a medical image processing apparatus according to a third aspect of the present invention, the second display control preferably fixes the relative position of the class information indicating one or more specific classes from among a plurality of classes with respect to the area of ​​interest.

[0013] For example, in the case of a large area of ​​interest, overlaying class information on the location of that area of ​​interest does not interfere with observation, or interferes with observation to a small extent. Therefore, the class information indicating the class (specific class) of such an area of ​​interest can be fixed regardless of the time elapsed since recognition.

[0014] In a medical image processing apparatus according to a fourth aspect of the present invention, the second display control preferably changes the manner of change according to the classified class when changing the relative position of the class information according to the elapsed time.

[0015] The regions of interest detected from medical images differ in their surrounding context (e.g., their positional relationship to other regions of interest). Therefore, when changing the class information superimposed on the location of a region of interest according to the elapsed time, it is preferable to change the manner of change (e.g., the direction in which the class information is moved) so as not to interfere with the observation of other regions of interest.

[0016] In a medical image processing apparatus according to a fifth aspect of the present invention, the second display control preferably changes the display format of the class information to another display format when changing the relative position of the class information according to the elapsed time.

[0017] For example, when a region of interest is first detected, class information indicating the classification of the region of interest can be displayed as a marker, and then the marker can be changed to text information as time passes. Alternatively, when a region of interest is first detected, class information indicating the classification of the region of interest can be displayed as text information, and then the text information can be changed to a marker as time passes.

[0018] In a medical image processing apparatus according to a sixth aspect of the present invention, the class information includes at least one of character information, a marker, and a graphic that indicates the classified class.

[0019] In a medical image processing apparatus according to a seventh aspect of the present invention, the second display control preferably involves displaying emphasis information that highlights the area of ​​interest on the display unit and fixing the relative position of the emphasis information with respect to the area of ​​interest. The emphasis information that highlights the area of ​​interest can, for example, be a rectangular frame surrounding the area of ​​interest. Furthermore, by setting the color of the rectangular frame to a color corresponding to the class of the area of ​​interest, class information can be added to the rectangular frame.

[0020] In a medical image processing apparatus according to the eighth aspect of the present invention, the processor performs a reception process to receive a freeze instruction from the user operation unit, the first display control, upon receiving the freeze instruction, performs a process to switch the sequential display of medical images to be displayed on the display unit to a fixed display of a single medical image, and the second display control preferably fixes the relative position of the class information with respect to the area of ​​interest during the period when the display is switched to fixed display.

[0021] In the medical image processing apparatus according to the ninth aspect of the present invention, the medical image is preferably an ultrasound image. In the case of ultrasound images, multiple areas of interest (e.g., organs, large blood vessels, etc.) are detected simultaneously, and multiple class information corresponding to each of the multiple areas of interest is displayed, so the multiple class information tends to interfere with the user's observation. Therefore, in the case of ultrasound images, it is effective to change the relative position of the class information with respect to the area of ​​interest according to the elapsed time since the area of ​​interest was detected.

[0022] A medical image processing method according to a tenth aspect of the present invention includes the steps of sequentially acquiring time-series medical images, a first display control step of sequentially displaying the medical images on a display unit, a step of recognizing information regarding the location of a region of interest within a medical image based on the sequentially acquired medical images, a step of classifying the region of interest into one of a plurality of classes based on the sequentially acquired medical images, and a second display control step of superimposing class information indicating the classified class onto the location of the region of interest in the medical image displayed on the display unit, wherein the second display control step changes the relative position of the superimposed class information to the region of interest according to the elapsed time since the region of interest was recognized, and a processor executes the processing of each step.

[0023] In the medical image processing method according to the 11th aspect of the present invention, when changing the relative position of the class information in the second display control step according to the elapsed time, it is preferable to change it in a direction away from the position of the area of ​​interest.

[0024] In the medical image processing method according to the twelfth aspect of the present invention, in the second display control step, for class information indicating one or more specific classes among a plurality of classes, it is preferable to fix the relative position with respect to the attention area of the class information indicating the specific class.

[0025] In the medical image processing method according to the thirteenth aspect of the present invention, in the second display control step, when changing the relative position of the class information according to the elapsed time, it is preferable to change the mode of change according to the classified class.

[0026] In the medical image processing method according to the fourteenth aspect of the present invention, in the second display control step, when changing the relative position of the class information according to the elapsed time, it is preferable to also change the display form of the class information to another display form.

[0027] In the medical image processing method according to the fifteenth aspect of the present invention, the class information includes at least one of character information, a marker, and a figure indicating the classified class.

[0028] In the medical image processing method according to the sixteenth aspect of the present invention, in the second display control step, it is preferable to display emphasis information for emphasizing the attention area on the display unit and fix the relative position of the emphasis information with respect to the attention area.

[0029] In the medical image processing method according to the seventeenth aspect of the present invention, it includes a step of receiving a freeze instruction from the user operation unit. In the first display control step, when receiving the freeze instruction, the sequential display of the medical images displayed on the display unit is switched to a fixed display by one medical image. In the second display control step, it is preferable to fix the relative position of the class information with respect to the attention area during the period when the display is switched to the fixed display.

[0030] In the medical image processing method according to the eighteenth aspect of the present invention, the medical image is preferably an ultrasonic image.

[0031] The invention according to the 19th aspect is a medical image processing program that causes a processor to execute the processing of each step in the medical image processing method according to the 10th to 18th aspects. [Effects of the Invention]

[0032] According to the present invention, class information indicating the classification class of a region of interest detected from a time-series medical image is superimposed on the location of the detected region of interest, thereby clearly presenting the location of the region of interest and its classified class to the user. Furthermore, the relative position of the superimposed class information to the region of interest is changed according to the elapsed time since the region of interest was recognized, so that the class information does not interfere with the user's observation. [Brief explanation of the drawing]

[0033] [Figure 1] Figure 1 is a schematic diagram showing the overall configuration of an ultrasound endoscope system including a medical image processing device according to the present invention. [Figure 2] Figure 2 is a block diagram showing an embodiment of an ultrasound processor device that functions as a medical image processing device according to the present invention. [Figure 3] Figure 3 shows a first example of the display of medical images and class information on the monitor. [Figure 4] Figure 4 shows a second example of the display of medical images and class information on the monitor. [Figure 5] Figure 5 shows a third example of the display of medical images and class information on the monitor. [Figure 6] Figure 6 shows a fourth example of how medical images and class information are displayed on the monitor. [Figure 7] Figure 7 is a flowchart showing a first embodiment of the medical image processing method according to the present invention. [Figure 8] Figure 8 is a flowchart showing a second embodiment of the medical image processing method according to the present invention. [Figure 9]Figure 9 is a flowchart showing a third embodiment of the medical image processing method according to the present invention. [Modes for carrying out the invention]

[0034] Hereinafter, preferred embodiments of the medical image processing apparatus, the operating method of the medical image processing apparatus, the program, and the recording medium according to the present invention will be described with reference to the attached drawings.

[0035] [Overall configuration of an endoscopic ultrasound system including medical image processing equipment] Figure 1 is a schematic diagram showing the overall configuration of an ultrasound endoscope system including a medical image processing device according to the present invention.

[0036] As shown in Figure 1, the ultrasound endoscope system 2 comprises an ultrasound scope 10, an ultrasound processor device 12 for generating ultrasound images, an endoscope processor device 14 for generating endoscope images, a light source device 16 for supplying illumination light to the ultrasound scope 10 to illuminate the inside of the body cavity, and a monitor 18 for displaying ultrasound images and endoscope images.

[0037] The ultrasound scope 10 comprises an insertion section 20 inserted into the body cavity of the subject, a handheld control section 22 connected to the proximal end of the insertion section 20 for operation by the operator, and a universal cord 24 with one end connected to the handheld control section 22. The other end of the universal cord 24 is provided with an ultrasound connector 26 connected to an ultrasound processor device 12, an endoscope connector 28 connected to an endoscope processor device 14, and a light source connector 30 connected to a light source device 16.

[0038] The ultrasound scope 10 is detachably connected to the ultrasound processor unit 12, the endoscope processor unit 14, and the light source unit 16 via these connectors 26, 28, and 30. In addition, a tube 32 for air and water supply and a tube 34 for suction are connected to the light source connector 30.

[0039] The monitor 18 receives the respective video signals generated by the ultrasound processor unit 12 and the endoscope processor unit 14 and displays the ultrasound image and the endoscope image. The display of the ultrasound image and the endoscope image can be switched between as needed and displayed on the monitor 18, or both images can be displayed simultaneously.

[0040] The handheld control unit 22 is equipped with an air supply / water supply button 36 and a suction button 38 side by side, as well as a pair of angle knobs 42 and a treatment instrument insertion port 44.

[0041] The insertion section 20 has a tip, a base, and a longitudinal axis 20a. Starting from the tip, it is composed of a tip body 50 made of a rigid material, a curved section 52 connected to the base of the tip body 50, and a slender, long, flexible flexible section 54 connecting the base of the curved section 52 to the tip of the hand-operated section 22. Specifically, the tip body 50 is located on the tip side of the insertion section 20 in the direction of the longitudinal axis 20a. The curved section 52 is remotely bent by rotating a pair of angle knobs 42 provided on the hand-operated section 22. This allows the tip body 50 to be directed in a desired direction.

[0042] The tip body 50 is equipped with an ultrasonic transducer 62 and a bag-shaped balloon 64 that encloses the ultrasonic transducer 62. The balloon 64 can be inflated or deflated by water being supplied from the water tank 70 or by the water inside the balloon 64 being sucked out by the suction pump 72. The balloon 64 is inflated until it contacts the inner wall of the body cavity in order to prevent attenuation of ultrasound and ultrasonic echoes (echo signals) during ultrasound observation.

[0043] Furthermore, the tip body 50 is equipped with an endoscope observation unit (not shown), which has an observation unit and illumination unit, and includes an objective lens and an image sensor. The endoscope observation unit is located behind the ultrasonic probe 62 (on the handheld control unit 22 side).

[0044] [Medical image processing device] Figure 2 is a block diagram showing an embodiment of an ultrasound processor device that functions as a medical image processing device according to the present invention.

[0045] The ultrasound processor device 12 shown in Figure 2 recognizes a region of interest within a medical image based on sequentially acquired time-series medical images, and informs the user of information indicating the recognition result of the region of interest. In particular, the information indicating the recognition result, which is displayed together with the medical image, is displayed in a format that changes according to the elapsed time since the region of interest was recognized.

[0046] The ultrasonic processor device 12 shown in Figure 2 consists of a transmitting / receiving unit 100, an image generation unit 102, a CPU (Central Processing Unit) 104, a focus area recognition unit 106, a time measurement processing unit 108, a display control unit 110, and a memory 112, and the processing of each unit is realized by one or more processors.

[0047] The CPU 104 operates based on various programs, including the medical image processing program according to the present invention, stored in the memory 112, and comprehensively controls the transmitting / receiving unit 100, the image generation unit 102, the area of ​​interest recognition unit 106, the time measurement processing unit 108, and the display control unit 110, and also functions as a part of each of these units.

[0048] The transmitting / receiving unit 100 and the image generation unit 102, which function as medical image acquisition processing units, sequentially acquire medical images in a time series.

[0049] The transmitting unit of the transmitting / receiving unit 100 generates multiple drive signals to be applied to multiple ultrasonic transducers of the ultrasonic probe 62 of the ultrasonic scope 10, and applies the multiple drive signals to the multiple ultrasonic transducers by assigning a delay time to each of the multiple drive signals based on a transmission delay pattern selected by a scanning control unit (not shown).

[0050] The receiver of the transmitting / receiving unit 100 amplifies multiple detection signals output from multiple ultrasonic transducers of the ultrasonic probe 62 and converts the analog detection signals into digital detection signals (also called RF (Radio Frequency) data). This RF data is input to the image generation unit 102.

[0051] The image generation unit 102 performs reception focus processing by assigning delay times to multiple detection signals represented by RF data based on the reception delay pattern selected by the scanning control unit, and then adding these detection signals together. This reception focus processing forms sound line data in which the focus of the ultrasonic echo is narrowed.

[0052] The image generation unit 102 further applies STC (Sensitivity Timegain Control) to the sound line data to correct for attenuation due to distance according to the depth of the ultrasonic reflection position, and then generates envelope data by performing envelope detection processing using a low-pass filter or the like, and stores the envelope data for one frame, more preferably multiple frames, in a cine memory (not shown). The image generation unit 102 then applies preprocessing such as log (logarithmic) compression and gain adjustment to the envelope data stored in the cine memory to generate a B-mode image.

[0053] In this way, the transmitting / receiving unit 100 and the image generation unit 102 sequentially acquire time-series B-mode images (hereinafter referred to as "medical images").

[0054] The area of ​​interest recognition unit 106 performs the process of recognizing information about the location of the area of ​​interest within a medical image based on the medical image, and the process of classifying the area of ​​interest into one of several classes based on the medical image. This can be done, for example, by AI (Artificial Intelligence).

[0055] The areas of interest in this example are various organs within medical images (B-mode tomographic images), such as the pancreas, main pancreatic duct, spleen, splenic vein, splenic artery, and gallbladder.

[0056] The area of ​​interest recognition unit 106, upon receiving a series of medical images, recognizes the location of the area of ​​interest for each input medical image, outputs information about that location, and also recognizes which of several classes the area of ​​interest belongs to, and outputs information indicating the recognized class (class information).

[0057] The location of the region of interest can be, for example, the center of the rectangle surrounding the region of interest. The class information, in this example, indicates the type of organ.

[0058] The time measurement processing unit 108 measures the elapsed time from when the area of ​​interest is detected by the area of ​​interest recognition unit 106 until that area of ​​interest is no longer detected. If the area of ​​interest recognition unit 106 detects (recognizes) multiple areas of interest of different types, the time measurement processing unit 108 individually measures the elapsed time indicating the detection period for each area of ​​interest.

[0059] The display control unit 110 consists of a first display control unit 110A that displays time-series medical images on the monitor 18, which is the display unit, and a second display control unit 110B that displays information related to the area of ​​interest on the monitor 18.

[0060] The first display control unit 110A displays medical images sequentially acquired by the transmitting / receiving unit 100 and the image generation unit 102 on the monitor 18. In this example, a video showing an ultrasound tomography image is displayed on the monitor 18.

[0061] Furthermore, the first display control unit 110A receives a freeze command from the handheld operation unit 22 (user operation unit) of the ultrasound scope 10. For example, when the freeze button on the handheld operation unit 22 is operated and a freeze command is received, the first display control unit 110A switches the sequential display of medical images on the monitor 18 to a fixed display of a single medical image (the medical image at the current time).

[0062] The second display control unit 110B superimposes class information indicating the classification of the area of ​​interest recognized by the area of ​​interest recognition unit 106 onto the position of the area of ​​interest in the medical image displayed on the monitor 18. The second display control unit 110B changes the relative position of the superimposed class information to the area of ​​interest according to the elapsed time since the area of ​​interest was recognized, as measured by the time measurement processing unit 108.

[0063] Furthermore, when the second display control unit 110B receives a freeze command, it also fixes the relative position of the class information with respect to the area of ​​interest for the duration that the medical image displayed on the monitor 18 is fixed as a still image.

[0064] The following describes examples of displays such as class information indicating the classification of the area of ​​interest displayed on the monitor 18 by the second display control unit 110B.

[0065] <1st display example> Figure 3 shows a first example of the display of medical images and class information on the monitor.

[0066] Figure 3(a) shows the screen of monitor 18 when the target regions R1 and R2 are detected, and Figure 3(b) shows the screen of monitor 18 after a certain period of time has elapsed since the detection of the target regions R1 and R2.

[0067] In Figure 3, region R1 is the pancreas, and region R2 is the splenic vein.

[0068] As shown in Figure 3(a), when the target regions R1 and R2 are detected (or when less than a certain amount of time has passed since detection), the class information "Panc.", an abbreviation for pancreas, is superimposed at the location of the target region R1 (center or nearby), and the class information "SV", an abbreviation for splenic vein, is superimposed at the location of the target region R2.

[0069] In this way, when areas of interest R1 and R2 are detected, class information indicating the classification of the detected areas of interest R1 and R2 can be notified to the user. In particular, since the precise location of areas of interest R1 and R2 is required at the time of detection, it is preferable to superimpose the class information near the center of each area of ​​interest R1 and R2.

[0070] Subsequently, as shown in Figure 3(b), after a certain period of time (for example, a few seconds (about 1-2 seconds)) has elapsed since the detection of the regions of interest R1 and R2, the display of class information ("Panc.", "SV") indicating the classification of the regions of interest R1 and R2 is changed (moved) away from the positions of the regions of interest R1 and R2. In the example in Figure 3(b), the class information ("Panc.", "SV") is moved outside the regions of interest R1 and R2, respectively.

[0071] Displaying class information near the center of the area of ​​interest at all times can be problematic as it interferes with the diagnosis of the area of ​​interest. However, this problem can be solved by moving the class information indicating the classification of the area of ​​interest to the outside of the area of ​​interest after a certain period of time has elapsed since the area of ​​interest was detected.

[0072] Furthermore, the method for moving the class information can be either by discretely changing its position after a certain time (a fixed period of time) as described above, or by continuously shifting its position over time. Also, in the example shown in Figure 3, the class information is textual information indicating an abbreviation of the organ type in the region of interest, but it may also be textual information indicating the full name of the organ type in the region of interest.

[0073] <Second display example> Figure 4 shows a second example of the display of medical images and class information on the monitor.

[0074] Figure 4(a) shows the screen of monitor 18 when the target regions R1 and R3 are detected, and Figures 4(b) to 4(d) show the screen of monitor 18 after a certain period of time has elapsed since the detection of the target regions R1 and R3, respectively.

[0075] In Figure 4, region R1 is the pancreas, and region R3 is the main pancreatic duct.

[0076] As shown in Figure 4(a), when the areas of interest R1 and R3 are detected, the class information "Panc." (an abbreviation for pancreas) is superimposed at the location of area of ​​interest R1, and the class information "MPD" (an abbreviation for main pancreatic duct) is superimposed at the location of area of ​​interest R3.

[0077] In the example shown in Figure 4(b), after a certain period of time has elapsed since the detection of the regions of interest R1 and R3, the class information ("Panc.", "MPD") indicating the classification of the regions of interest R1 and R3 is moved outside of the regions of interest R1 and R3.

[0078] In this case, when changing the position of the class information ("Panc.", "MPD") according to the elapsed time, it is preferable to change the manner of change according to the classified class. That is, in the example shown in Figure 4(b), the class information ("Panc.") indicating the classification of the region of interest R1 is moved to the upper left of the region of interest R1 in Figure 4(b), and the class information ("MPD") for the region of interest R3 is moved to the lower right of the region of interest R3 in Figure 4(b). This is to ensure that when multiple pieces of class information are moved, these pieces of class information do not come into close proximity.

[0079] Furthermore, in the examples shown in Figures 4(c) and (d), after a certain period of time has elapsed since the detection of the regions of interest R1 and R3, the class information ("MPD") for region of interest R3 is moved outside of region of interest R3, while the relative position of the class information ("Panc.") indicating the classification of region of interest R1 relative to region of interest R3 remains fixed. In other words, for class information indicating the classification of one or more specific classes among multiple classes (in this example, ("Panc.")), the relative position of the class information indicating the classification of specific classes relative to the region of interest is fixed without being moved.

[0080] As shown in Figure 4(a), the areas of interest R1 and R3 differ in size; R1 is relatively large, while R3 is small. When R3 is small, overlaying class information ("MPD") indicating the classification of the area of ​​interest onto its location significantly reduces its visibility. On the other hand, when R1 is large, overlaying class information ("Panc.") indicating the classification of the area of ​​interest onto its location results in less reduction in the visibility of R3.

[0081] In this way, even if class information indicating the classification of the area of ​​interest is superimposed on the area of ​​interest, certain class information that does not significantly reduce the visibility of the area of ​​interest may be left in place even after a certain period of time has elapsed since the area of ​​interest was detected.

[0082] Note that the display example shown in Figure 4(d) differs from the display example shown in Figure 4(c) in that, after the movement of the class information ("MPD"), the region of interest R3 and the class information ("MPD") are connected by a leader line, clearly indicating the relationship between the two.

[0083] <3rd display example> Figure 5 shows a third example of the display of medical images and class information on the monitor.

[0084] Figure 5(a) shows the screen of monitor 18 when the regions of interest R1 and R2 are detected, and Figure 5(b) shows the screen of monitor 18 after a certain period of time has elapsed since the detection of the regions of interest R1 and R2.

[0085] The third display example shown in Figure 5 differs from the first display example shown in Figure 3 in the way the class information is displayed when the areas of interest R1 and R2 are detected.

[0086] As shown in Figure 5(a), when the areas of interest R1 and R2 are detected, a marker (circle) indicating the pancreas is superimposed as class information at the location of area R1, and a marker (star) indicating the splenic vein is superimposed as class information at the location of area R2. The type of class information indicating the classification of the area of ​​interest and the type of marker are pre-associated, allowing the user to confirm the class information by visually identifying the type of marker superimposed at the location of the area of ​​interest.

[0087] Furthermore, as shown in Figure 5(b), after a certain period of time has elapsed since the detection of the regions of interest R1 and R2, the markers indicating the class information are erased, and class information ("Panc.", "SV") indicating the classification of the regions of interest R1 and R2 is displayed outside the regions of interest R1 and R2, similar to Figure 3(b).

[0088] In other words, in the third display example shown in Figure 5, after a certain period of time has elapsed, the class information is moved, and the display format of the class information is also changed from a marker to text information.

[0089] As shown in the third display example in Figure 5, the marker can be displayed smaller than the text information, allowing for a highly visible and accurate display of the location information of the area of ​​interest. Furthermore, by changing the class information from a marker to text information after a certain period of time, detailed information about the class can be presented in an easily interpretable manner.

[0090] <4th display example> Figure 6 shows a fourth example of how medical images and class information are displayed on the monitor.

[0091] Figure 6(a) shows the screen of monitor 18 when the target regions R1 and R2 are detected, and Figure 6(b) shows the screen of monitor 18 after a certain period of time has elapsed since the detection of the target regions R1 and R2.

[0092] The fourth display example shown in Figure 6 differs from the first display example shown in Figure 3 in that it further displays highlighting information that emphasizes the areas of interest R1 and R2, respectively.

[0093] The highlighting information that emphasizes the areas of interest R1 and R2, as shown in Figure 6, is displayed with a fixed relative position to the areas of interest R1 and R2, regardless of the time elapsed since the areas of interest R1 and R2 were detected. In other words, when the areas of interest R1 and R2 move within a moving medical image, the highlighting information also moves along with the areas of interest R1 and R2, but the relative position of the highlighting information to the areas of interest R1 and R2 remains fixed.

[0094] The emphasis information used to highlight the areas of interest R1 and R2, as shown in Figure 6, is the bounding box surrounding the area of ​​interest. The bounding box surrounding the area of ​​interest can be obtained from the area of ​​interest recognition unit 106. The intersection of the diagonals of the bounding box can be used as the center position of the area of ​​interest.

[0095] The bounding box may be the same color (e.g., white) and line type regardless of the class information indicating the classification of the area of ​​interest, or it may be different in color and / or line type depending on the class information indicating the classification of the area of ​​interest. In the latter case, the bounding box will include the class information.

[0096] The bounding box can be displayed so that it inscribes the regions of interest R1 and R2, or slightly larger than the regions of interest R1 and R2. This does not reduce the visibility of the regions of interest, so it can be displayed in a fixed position regardless of the time elapsed since the regions of interest R1 and R2 were detected.

[0097] Furthermore, the highlighting information used to emphasize each area of ​​interest is not limited to bounding boxes. For example, it could include circles or ellipses surrounding the area of ​​interest, markers or shapes placed at the center of the area of ​​interest (which should be the same shape regardless of the area's class), or arrows indicating the location of the area of ​​interest.

[0098] The first to fourth display examples shown in Figures 3 to 6 display class information in an identifiable manner using text information (including abbreviations), markers, etc., but class information may also be displayed in an identifiable manner using shapes. For example, bounding boxes color-coded according to class information are included in shapes. Furthermore, the display forms of markers and shapes may not be clearly distinguishable; for example, the markers (circle, star) shown in Figure 5(a) can also be considered shapes.

[0099] Furthermore, various methods are possible for displaying class information and for transitioning the position of class information over time since detection, not limited to the first to fourth display examples.

[0100] [Medical Image Processing Methods] <First Embodiment> Figure 7 is a flowchart showing a first embodiment of the medical image processing method according to the present invention, illustrating the processing procedure for each part of the ultrasound processor device 12 shown in Figure 2.

[0101] In Figure 7, the CPU 104 first resets the count value of timer Ti, which indicates the elapsed time since each of the multiple regions of interest (in this example, multiple organs) was recognized, to zero (Ti=0) (step S10). Here, i is a parameter that indicates the types of organs from 1 to n, where n is the number of organ types (number of classes). Therefore, T1 is the timer corresponding to the first type of organ, and Tn is the timer corresponding to the nth type of organ.

[0102] The transmitting / receiving unit 100 and the image generation unit 102, which function as a medical image acquisition processing unit, sequentially acquire time-series medical images (step S12). If the frame rate of the time-series medical images is, for example, 30 fps (frames per second), one frame of medical image is acquired every 1 / 30 (second).

[0103] Next, the focus region recognition unit 106 performs recognition processing of focus regions (organs) Ri (i=1~n) present in the medical image based on the medical image acquired in step S12 (step S14). That is, the focus region recognition unit 106 recognizes the location of the focus region Ri present in the medical image and recognizes the class (type of organ) that indicates the classification of that focus region Ri. If there are multiple focus regions in the medical image, the unit recognizes the location of each focus region and the class of each focus region.

[0104] In parallel with the recognition of the area of ​​interest Ri by the area of ​​interest recognition unit 106, the first display control unit 110A displays the medical image acquired in step S12 on the monitor 18 (step S16 (first display control step S16)).

[0105] The CPU 104 determines in step S14 whether or not the region of interest Ri has been recognized, and if the region of interest Ri has been recognized, it proceeds to step S20 (second display control step S20) (step S18).

[0106] In step S20, the second display control unit 110B superimposes class information Ci (i=1~n) indicating the classification of the area of ​​interest Ri onto the position of the area of ​​interest Ri in the medical image displayed on the monitor 18. For example, in the first display example shown in Figure 3, the area of ​​interest R1 representing the pancreas and the area of ​​interest R2 representing the splenic vein are recognized, and class information C1 ("Panc.") indicating the classification of the area of ​​interest R1 is superimposed near the center of the area of ​​interest R1, and class information C2 ("SV") indicating the classification of the area of ​​interest R2 is superimposed near the center of the area of ​​interest R2.

[0107] Next, the time measurement processing unit 108 increments the count value of timer Ti (i=1~n) by 1 (step S22). In this example, the count value of timer Ti corresponds to the number of frames in which the same region of interest Ri is detected consecutively in the time-series medical image from the time the region of interest Ri was first detected. Therefore, if the frame rate is 30fps, a count value of timer T1 of 30 when the region of interest R1 is detected consecutively means 1 second. In other words, the count value of timer Ti indicates the elapsed time since the region of interest Ri was recognized.

[0108] Next, the CPU 104 determines whether the count value of timer Ti is greater than or equal to the threshold Th (step S24). The threshold Th can be, for example, 45. Converting the threshold Th (=45) to time at a frame rate of 30fps gives 1.5 seconds.

[0109] The CPU 104 proceeds to step S12 if the count value of timer Ti is less than the threshold Th. This causes the processing from step S12 to step S24 to be repeated.

[0110] On the other hand, if the count value of timer Ti is greater than or equal to the threshold Th, CPU 104 proceeds to step S26.

[0111] In step S26, the second display control unit 110B moves the relative position of the class information Ci with respect to the area of ​​interest Ri that was superimposed in step S20 in a direction away from the position near the center of the area of ​​interest Ri.

[0112] For example, in the first display example shown in Figure 3, the class information ("Panc.", "SV") indicating the classification of the areas of interest R1 and R2, which is superimposed near the center of the areas of interest R1 and R2 as shown in Figure 3(a), is moved to the outside of the areas of interest R1 and R2 as shown in Figure 3(b).

[0113] In this way, after a certain period of time (1.5 seconds in this example) has elapsed since the detection of the region of interest Ri, the class information Ci indicating the classification of the region of interest Ri is moved outside of the region of interest Ri. This prevents the display of the class information Ci from interfering with the diagnosis of the region of interest Ri.

[0114] On the other hand, if it is determined in step S18 that the region of interest Ri is not recognized ("No"), the CPU 104 further determines whether or not the region of interest Ri was recognized in the previous (1 frame prior) medical image (step S28).

[0115] If CPU104 determines that the region of interest Ri was not recognized in the previous medical image, it proceeds to step S12. If CPU104 determines that the region of interest Ri was recognized in the previous medical image, it proceeds to step S30.

[0116] In step S30, although the region of interest Ri was recognized in the previous medical image, the region of interest Ri is not recognized in the current medical image. Therefore, the second display control unit 110B terminates the display of the class information Ci on the monitor 18.

[0117] As soon as the display of class information Ci ends, CPU 104 resets the timer Ti count value to zero (step S32) and proceeds to step S12.

[0118] Furthermore, in step S20, when class information Ci is superimposed on the location of the region of interest Ri, the class information Ci is not limited to textual information indicating the name or abbreviation of the type of organ that is the region of interest Ri, but may also be other display forms such as markers or figures indicating the type of organ.

[0119] Furthermore, in step S26, after a certain period of time has elapsed since the area of ​​interest Ri was recognized (when the timer Ti count value is equal to or greater than the threshold Th), the class information Ci displayed near the center of the area of ​​interest Ri is moved outside the area of ​​interest Ri. However, the direction of movement can be changed depending on the class information Ci indicating the classification of the area of ​​interest Ri (see Figure 4(b)).

[0120] Furthermore, as shown in Figures 4(c) and (d), the class information for a specific class ("Panc." indicating the pancreas) may be kept in place from the location of the region of interest even after a certain period of time has elapsed. Also, as shown in Figure 4(d), the moved class information ("MPD") and the region of interest R3 can be connected with a leader line to clarify the relationship between the two.

[0121] Furthermore, as shown in Figure 5, the display format may be changed from a marker indicating class information Ci before a certain period of time has elapsed to a display format using text information indicating class information Ci after a certain period of time has elapsed.

[0122] <Second Embodiment> Figure 8 is a flowchart showing a second embodiment of the medical image processing method according to the present invention.

[0123] In Figure 8, steps that are common to the flowchart of the first embodiment shown in Figure 7 are given the same step numbers, and their detailed explanations are omitted.

[0124] The medical image processing method of the second embodiment shown in Figure 8 differs from the medical image processing method of the first embodiment shown in Figure 7 in that the processing in step S40 is added.

[0125] Step S40 (second display control step S40) shown in Figure 8 determines, after a certain period of time has elapsed since the area of ​​interest Ri was recognized (when the timer Ti count value is equal to or greater than the threshold Th), whether the class information Ci indicating the classification of the area of ​​interest Ri is of a specific class or not.

[0126] In step S40, if the class information Ci indicating the classification of the region of interest Ri is determined to be class information of a specific class ("Yes"), the process proceeds to step S12. In this case, the process does not proceed to step S26, so the position of the class information indicating the specific class remains fixed regardless of the elapsed time since the detection of the region of interest of the specific class.

[0127] For example, in the display examples shown in Figures 4(c) and (d), the class information indicating a specific class ("Panc." indicating the pancreas) is kept in place from the location of the area of ​​interest even after a certain period of time has elapsed.

[0128] In the case of relatively large regions of interest, such as the pancreas, overlaying class information onto the region of interest does not interfere with observation of the region, or at least does not interfere with observation to a significant degree.

[0129] <Third Embodiment> Figure 9 is a flowchart showing a third embodiment of the medical image processing method according to the present invention.

[0130] In Figure 9, steps that are common to the flowchart of the first embodiment shown in Figure 7 are given the same step numbers, and their detailed explanations are omitted.

[0131] The medical image processing method of the third embodiment shown in Figure 9 differs from the medical image processing method of the first embodiment shown in Figure 7 in that the processes in steps S50 and S52 are added.

[0132] Step S50, shown in Figure 9, determines whether or not a freeze command has been received from the handheld control unit 22 of the ultrasonic scope 10.

[0133] If it is determined that a freeze command has been received ("Yes"), the process proceeds to step S52, where it is determined whether or not a freeze release command has been received. If it is determined in step S52 that a freeze release command has not been received ("No"), the process returns to step S50. If it is determined that a freeze release command has been received ("Yes"), the process proceeds to step S22.

[0134] In other words, once a freeze command is received, the system will not proceed to step S12 until a freeze release command is received. As a result, the acquisition of new medical images in step S12 and the sequential display of new medical images in step S16 are stopped, and the monitor 18 displays the medical image at the time the freeze command was received (fixed as a still image).

[0135] Furthermore, when a freeze command is received, the class information Ci, which indicates the classification of the region of interest Ri recognized from the medical image at the time the freeze command was received, will be fixed and displayed until a freeze release command is received.

[0136] When a freeze command is received in this manner, the medical image displayed on monitor 18 is fixed to the medical image at the time the freeze command was received, and the display switches from video to still image until the freeze command is released. In addition, if a region of interest Ri is recognized from the medical image at the time the freeze command was received, the display of class information Ci indicating the classification of that region of interest Ri is also fixed.

[0137] [others] In this embodiment, the ultrasound processor device 12 has the function of a medical image processing device according to the present invention, but it is not limited to this, and a personal computer or the like, separate from the ultrasound processor device 12, may acquire time-series medical images from the ultrasound processor device 12 and function as a medical image processing device according to the present invention.

[0138] Furthermore, the time-series medical images are not limited to ultrasound images; for example, they may also be time-series endoscopic images captured by an objective lens and image sensor provided on the tip body 50 of the ultrasound scope 10 and output from the endoscopic processor device 14.

[0139] Furthermore, the area of ​​interest within a medical image is not limited to various organs; for example, it could be a lesion. In this case, possible classifications of lesions include tumors, non-tumorous areas, and others.

[0140] Furthermore, the hardware structure that performs various controls on the ultrasound processor device (medical image processing device) of the above embodiment consists of various processors as shown below. These various processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various control units; Programmable Logic Devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing; and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing.

[0141] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple control units may be composed of a single processor. Examples of composing multiple control units with a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as multiple control units, as is typical of computers such as clients and servers. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple control units, on a single IC (Integrated Circuit) chip, as is typical of a System on a Chip (SoC). Thus, various control units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0142] Furthermore, the present invention includes a medical image processing program that, when installed on a computer, causes the computer to function as a medical image processing device according to the present invention, and a non-volatile storage medium on which this medical image processing program is recorded.

[0143] Furthermore, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention. [Explanation of Symbols]

[0144] 2. Ultrasound Endoscopy System 10. Ultrasound scope 12. Ultrasonic processor device 14 Endoscope processor device 16 Light source device 18 monitors 20 Insertion section 20a Long axis 22 Handheld control unit 24 Universal Code 26 Ultrasonic Connectors 28 Endoscope connectors 30 Light source connectors 32, 34 tubes 36 Air / Water Supply Button 38 Suction button 42 Angle knob 44. Insertion port for treatment instruments 50 Tip body 52 Curved section 54 Soft part 62 Ultrasonic probe 64 Balloons 70 Water supply tank 72 Suction pump 100 Transmitter / Receiver 102 Image generation unit 104 CPU 106 Area of ​​Interest Recognition Unit 108-hour measurement processing unit 110 Display Control Unit 110A First Display Control Unit 110B Second Display Control Unit 112 memory C1, C2, Ci Class Information R1, R2, R3, Ri attention area Steps S10-S32, S40, S50, S52

Claims

1. A medical image processing device equipped with a processor that displays B-mode images from an ultrasound endoscope on a display unit, The aforementioned processor, Multiple class information corresponding to multiple areas of interest within the B-mode image is displayed on the display unit. One of the aforementioned areas of interest is the organ region. Medical image processing equipment.

2. One of the aforementioned areas of interest is the lesion area. The medical image processing apparatus according to claim 1.

3. The class information of the organ region is information indicating the type of organ in that organ region. The class information of the lesion region is information indicating the type of lesion in that lesion region. The medical image processing apparatus according to claim 2.

4. The aforementioned processor, The class information is superimposed and displayed on the corresponding location of the area of ​​interest on the B-mode image. The relative position of the class information with respect to the superimposed area of ​​interest is changed, according to the elapsed time since the area of ​​interest was recognized, from a position in the B-mode image superimposed on the area of ​​interest to a position outside the area of ​​interest. A medical image processing apparatus according to any one of claims 1 to 3.

5. The processor fixes the relative position of the class information representing one or more specific classes among the plurality of class information with respect to the region of interest. The medical image processing apparatus according to claim 4.

6. The processor changes the relative position of the class information according to the elapsed time, and changes the manner of the change according to the class information. The medical image processing apparatus according to claim 4 or 5.

7. The processor changes the display format of the class information to another display format when changing the relative position of the class information according to the elapsed time. A medical image processing apparatus according to any one of claims 4 to 6.

8. The class information includes at least one of the following: character information, markers, and graphics, which represent the class information. A medical image processing apparatus according to any one of claims 1 to 7.

9. The processor causes the display unit to display emphasis information that highlights the area of ​​interest, and fixes the relative position of the emphasis information with respect to the area of ​​interest. A medical image processing apparatus according to any one of claims 4 to 8.

10. The aforementioned processor, When a freeze command is received from the user control unit, the B-mode image displayed on the display unit is switched to a fixed display of a single medical image that constitutes the B-mode image. During the period when the display is switched to the fixed display, the relative position of the class information with respect to the area of ​​interest is fixed. A medical image processing apparatus according to any one of claims 4 to 9.

11. A method for operating a medical image processing device equipped with a processor and for displaying B-mode images of an ultrasound endoscope on a display unit, The processor performs the step of displaying the B-mode image on the display unit, The processor includes the step of causing the display unit to display a plurality of class information corresponding to a plurality of areas of interest in the B-mode image, One of the aforementioned areas of interest is the organ region. How to operate a medical image processing device.

12. One of the aforementioned areas of interest is the lesion area. A method for operating the medical image processing device according to claim 11.

13. The class information of the organ region is information indicating the type of organ in that organ region. The class information of the lesion region is information indicating the type of lesion in that lesion region. A method for operating the medical image processing device according to claim 12.

14. The processor superimposes the class information onto the corresponding area of ​​interest on the B-mode image. The processor changes the relative position of the class information with respect to the superimposed area of ​​interest, from a position in the B-mode image superimposed on the area of ​​interest to a position outside the area of ​​interest, according to the elapsed time since the area of ​​interest was recognized. A method for operating a medical image processing device according to any one of claims 11 to 13.

15. A medical image processing program that causes the processor to execute the method of operating a medical image processing device according to any one of claims 11 to 14.

16. A non-temporary and computer-readable recording medium on which the program described in claim 15 is recorded.

Images