Medical image processing apparatus, medical image processing method, and program

Abstract

To automatically correct a registration relation between a coordinate system of an ultrasonic system having been established before surgery and a coordinate system of a modality other than the ultrasonic system.SOLUTION: A medical image processing device includes a generation part, an acquisition part, a calculation part, and a correction part. The generation part generates registration information representing a registration relation between a coordinate system of an ultrasonic system and a coordinate system of another modality other than the ultrasonic system. The acquisition part acquires two-dimensional ultrasonic images of an analyte at prescribed time intervals in a process of moving a probe, and extracts feature groups respectively from the acquired two-dimensional ultrasonic images. The calculation part calculates a distance between a feature group at a current time and a feature group at an immediately preceding time. The correction part corrects the registration information based on the distance calculated by the calculation part.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in the present specification and drawings relate to a medical image processing apparatus, a medical image processing method, and a program.

Background Art

[0002] In the field of medical image imaging, when performing examination or treatment, it is necessary to perform registration between three-dimensional data (hereinafter referred to as volume data) of an examination site of a subject obtained during the examination or treatment and volume data acquired before the examination or treatment.

[0003] For example, before performing diagnosis or surgery on an examination site of a subject, CT scanning or MR scanning may be performed in advance on the examination site of the subject to obtain CT (Computed Tomography) volume data or MR (Magnetic Resonance) volume data with favorable anatomical views. Thereafter, before performing diagnosis or surgery on the examination site of the subject, three-dimensional ultrasonic scanning is performed on the examination site of the subject to obtain US (Ultra-Sonic) volume data of the current position. Then, by registering the CT volume data or MR volume data with the ultrasonic volume data, when performing diagnosis or surgery, the anatomical plane of CT volume data or MR volume data with high resolution can be quickly found corresponding to the anatomical plane of real-time two-dimensional ultrasonic volume data of the examination site, thereby facilitating accurate analysis and judgment by doctors, and enabling accurate processing in accurate diagnosis or surgery.

[0004] The registration between the different modalities (CT, MR, ultrasound) described above is called multi-modality registration, and this multi-modality registration is a key technique in image-guided interventional surgery. Multi-modality registration allows for the acquisition of a transformation matrix between the ultrasound system's coordinate system and the CT or MR coordinate system. This transformation matrix enables the registration of data from one modality to the data of another modality. Multi-modality images mapped through this registration simultaneously possess the advantages of high resolution from CT or MR images and the real-time capabilities of ultrasound images.

[0005] However, the aforementioned multimodality registration has a problem: if the patient's body moves during surgery, the coordinate system of the ultrasound system and the patient's coordinate system cannot correspond, resulting in an inaccurate correspondence between the patient's ultrasound image and the CT or MR image. This necessitates the surgeon to stop the surgery, reacquire the ultrasound volume data, and re-establish the correspondence between the ultrasound volume data and the CT or MR volume data. Considering that acquiring ultrasound volume data requires stopping the surgery, the acquisition process is complex, and the re-establishment process takes a long time, there is a need for a technology that can automatically determine whether correction is necessary and automatically correct for any movement of the patient, while avoiding the need to reacquire ultrasound volume data. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Special Publication No. 2009-500110 [Overview of the project] [Problems that the invention aims to solve]

[0007] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to automatically correct the registration relationship between the coordinate system of an ultrasound system already established before surgery and the coordinate system of other modalities besides the ultrasound system. However, the problems solved by the embodiments disclosed herein and in the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0008] The medical image processing apparatus according to this embodiment comprises a generation unit, an acquisition unit, a calculation unit, and a correction unit. The generation unit generates registration information indicating the registration relationship between the coordinate system of the ultrasound system and the coordinate system of a modality other than the ultrasound system. The acquisition unit acquires two-dimensional ultrasound images of a subject at predetermined time intervals during the probe's movement and extracts feature groups from each acquired two-dimensional ultrasound image. The calculation unit calculates the distance between the feature group at the current time and the feature group at the previous time. The correction unit corrects the registration information based on the distance calculated by the calculation unit. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the configuration of a medical image processing apparatus according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing the processing procedure (medical image processing method) by the medical image processing apparatus according to the first embodiment. [Figure 3] Figure 3 is a flowchart showing the processing procedure (medical image processing method) by the medical image processing apparatus according to the second embodiment. [Modes for carrying out the invention]

[0010] The following describes a medical image processing apparatus, a medical image processing method, and a computer program capable of implementing the medical image processing method, with reference to the drawings. The embodiments described below are merely examples and are not limited to those embodiments. Furthermore, the content described in one embodiment is, in principle, applicable to other embodiments as well.

[0011] (First embodiment) Before performing surgery on a patient, a three-dimensional scan is usually performed on the examination site to obtain ultrasound volume data and CT or MR volume data of the examination site, and then the CT or MR volume data is registered with the ultrasound volume data. During the surgical procedure, if the patient does not move, the coordinate system of the ultrasound system and the coordinate system of the patient always correspond. However, if the patient moves, the actual coordinate system of the patient may no longer correspond to the coordinate system of the ultrasound system.

[0012] Therefore, the medical image processing device according to this embodiment automatically corrects the registration relationship between the coordinate system of the ultrasound system already established before the surgery and the coordinate system of another modality other than the ultrasound system (three-dimensional CT coordinate system or three-dimensional MR coordinate system) without interrupting the surgery, as configured as described later.

[0013] Figure 1 is a block diagram showing the configuration of a medical image processing apparatus 1 according to the first embodiment. For example, as shown in Figure 1, the medical image processing apparatus 1 includes an input interface 20, a communication interface 21, a display 22, a processing circuit 23, and a storage circuit 12.

[0014] The input interface 20 is implemented by a trackball for various settings, switch buttons, a mouse, a keyboard, a touchpad for input operations by touching the operating surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, and an audio input circuit. The input interface 20 is connected to the processing circuit 23 and converts the input operations received from the operator into electrical signals and outputs them to the processing circuit 23. In Figure 1, the input interface 20 is provided inside the medical image processing device 1, but it may also be provided externally.

[0015] The display 22 is connected to the processing circuit 23 and displays various information and image data output from the processing circuit 23. For example, the display 22 can be implemented as an LCD monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, etc. For example, the display 22 displays a GUI (Graphical User Interface) for receiving operator instructions, various display images, and various processing results from the processing circuit 23. The display 22 is an example of a display unit. In Figure 1, the display 22 is provided inside the medical image processing device 1, but it may be provided externally.

[0016] The communication interface 21 is a NIC (Network Interface Card) or the like, and communicates with other devices. For example, the communication interface 21 is connected to the processing circuit 23 and collects image data from ultrasound diagnostic equipment, which is an ultrasound system, or from other modalities such as X-ray CT (Computed Tomography) equipment and MRI (Magnetic Resonance Imaging) equipment, and outputs it to the processing circuit 23.

[0017] The memory circuit 12 is connected to the processing circuit 23 and stores various types of data. For example, the memory circuit 12 can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by means of a hard disk or optical disc. The memory circuit 12 also stores programs corresponding to each processing function executed by the processing circuit 23. In Figure 1, the memory circuit 12 is located inside the medical image processing device 1, but it may also be located externally.

[0018] The processing circuit 23 controls each component of the medical image processing device 1 in response to input operations received from the operator via the input interface 20.

[0019] For example, the processing circuit 23 is implemented by a processor. As shown in Figure 1, the processing circuit 23 performs the generation function 10, the acquisition function 11, the calculation function 13, the determination function 14, the correction function 15, and the display control function 16. Here, each processing function performed by the components of the processing circuit 23 shown in Figure 1—the generation function 10, the acquisition function 11, the calculation function 13, the determination function 14, the correction function 15, and the display control function 16—is recorded in the memory circuit 12 of the medical image processing device 1 in the form of a program that can be executed by a computer, for example. The processing circuit 23 is a processor that reads each program from the memory circuit 12 and executes it to realize the processing function corresponding to each program. In other words, the processing circuit 23 in the state where each program has been read has each of the functions shown in the processing circuit 23 of Figure 1.

[0020] The term "processor" used in the above description refers to circuits such as, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), and a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). When the processor is, for example, a CPU, the processor implements its functions by reading and executing programs stored in the storage circuit 12. On the other hand, when the processor is, for example, an ASIC, instead of storing a program in the storage circuit 12, the program is directly incorporated into the circuit of the processor. Note that each processor of the present embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining a plurality of independent circuits to implement its functions. Furthermore, a plurality of components in FIG. 1 may be integrated into one processor to implement the functions thereof.

[0021] Next, the processing contents of the generation function 10, acquisition function 11, calculation function 13, determination function 14, correction function 15, and display control function 16 executed by the processing circuit 23 will be described.

[0022] For example, the display control function 16 causes the display 22 to display volume data in real time.

[0023] The generation function 10 generates registration information indicating a registration relationship between a coordinate system of an ultrasound system and a coordinate system of another modality other than the ultrasound system. The coordinate system of the ultrasound system may be obtained from ultrasound volume data scanned before surgery. The coordinate system of another modality other than the ultrasound system is a coordinate system of volume data, and may be obtained from CT volume data or MR volume data scanned before surgery. The generation function 10 is an example of a generation unit.

[0024] The acquisition function 11 has a function of acquiring an ultrasound image, a function of acquiring a feature group from the ultrasound image, and a function of acquiring a transformation matrix that is registration information indicating a registration relationship between a coordinate system of an ultrasound system and a coordinate system of another modality other than the ultrasound system. For example, depending on the object to be acquired, the acquisition function 11 is divided into an ultrasound image acquisition function 11A, a feature group acquisition function 11B, and a transformation matrix acquisition function 11C.

[0025] The ultrasound image acquisition function 11A acquires two-dimensional ultrasound images at predetermined time intervals as an ultrasound probe (hereinafter referred to as a probe) moves after the start of surgery. Here, the movement process of the probe is acquired from a magnetic sensor attached to the probe. For example, the magnetic sensor attached to the probe detects its own position (coordinates) and orientation (angle) as position information, whereby the movement process of the probe is acquired. The two-dimensional ultrasound images include a reference two-dimensional ultrasound image and a series of two-dimensional ultrasound images acquired at predetermined time intervals after the reference two-dimensional ultrasound image. The reference two-dimensional ultrasound image is a group of two-dimensional ultrasound images collected while the subject is not moving at the stage immediately after the registration of pre-operative ultrasound volume data with CT volume data or MR volume data is completed and the surgery starts, and the position of the coordinate system in the ultrasound system between the reference two-dimensional ultrasound image and the pre-operative ultrasound volume data does not change.

[0026] Regarding the selection of a reference two-dimensional ultrasound image, for example, a two-dimensional ultrasound image under ideal conditions, such as holding one's breath, can be selected. Furthermore, regarding the examination site, the user can select different sites depending on the clinical context; for example, in cardiac scanning, the user can select different cardiac chambers as the examination site and scan reference ultrasound images. The reference two-dimensional ultrasound image includes the following characteristic groups.

[0027] The feature group acquisition function 11B extracts feature groups from the ultrasound image acquired by the ultrasound image acquisition function 11A. The feature group is a collection of regions that have features in the examination site. In this embodiment, the feature group is a collection of multiple feature points that have anatomical features. For example, if the liver is the examination site, the feature points may be branching points of hepatic blood vessels. In other words, in this embodiment, information about the feature points (for example, coordinate information in the coordinate system of the ultrasound system) is extracted from the two-dimensional ultrasound image containing the feature points.

[0028] The transformation matrix acquisition function 11C acquires a transformation matrix between the coordinate system of the ultrasound system and the CT coordinate system or MR coordinate system, obtained by registering ultrasound volume data and CT volume data or MR volume data before surgery. This transformation matrix represents the registration relationship between the coordinate system of the ultrasound system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system, and when the transformation matrix is ​​updated, the registration relationship between the coordinate system of the ultrasound system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system is updated accordingly.

[0029] The memory circuit 12 has a first memory area 12A and a second memory area 12B. The first memory area 12A is used to store information about the feature group at the previous time, and in this embodiment, it stores information about the feature point at the previous time. The second memory area 12B is used to store information about the transformation matrix. The information in the first memory area 12A and the second memory area 12B is updated as the subject's body moves. Note that the first memory area 12A and the second memory area 12B may be provided externally as separate memory circuits, rather than being located within the memory circuit 12.

[0030] Specifically, the ultrasound image acquisition function 11A acquires a reference two-dimensional ultrasound image, and the feature group acquisition function 11B extracts information about feature points (for example, coordinate information of feature points in the coordinate system of the ultrasound system). At this time, the feature group acquisition function 11B stores the coordinate information of feature points in the reference two-dimensional ultrasound image as the coordinate information of feature points at the immediately preceding time in the first memory area 12A. When automatic correction occurs, the coordinate information of feature points in the reference two-dimensional ultrasound image stored in this first memory area 12A is replaced with the coordinate information of feature points at the time the correction occurred. Each time automatic correction occurs, the coordinate information of feature points at the time the correction occurred is stored in the first memory area 12A, replacing the coordinate information of feature points at the immediately preceding time.

[0031] Furthermore, the transformation matrix acquisition function 11C acquires a transformation matrix between the coordinate system of the ultrasound system and the CT coordinate system or MR coordinate system, obtained by registering the ultrasound volume data and the CT volume data or MR volume data before surgery. At this time, the transformation matrix acquisition function 11C stores the acquired transformation matrix in the second memory area 12B as the initial transformation matrix. If automatic correction occurs as described later, the transformation matrix is ​​updated, and the updated transformation matrix is ​​stored in the second memory area 12B. Then, each time automatic correction occurs, the updated transformation matrix is ​​stored in the second memory area 12B in place of the transformation matrix stored in the second memory area 12B.

[0032] The calculation function 13 calculates the distance between the feature point at the current time extracted by the feature group acquisition function 11B and the feature point at the previous time stored in the first memory area 12A. This distance may be the average of the Euclidean distances calculated using multiple corresponding feature point pairs, or it may be the maximum distance among multiple corresponding feature point pairs. To compare with a predetermined distance range, the calculated distance refers to a parameter that represents the coordinate difference between multiple corresponding feature point pairs as a whole. In addition, it is also possible to obtain the distance by other methods.

[0033] The judgment function 14 determines the relationship between the distance calculated by the calculation function 13 and a predetermined distance range. This distance range can be set by the user based on the time interval of ultrasound image sampling. The judgment function 14 determines whether the distance calculated by the calculation function 13 is below the lower limit of the distance range, within the distance range, or above the upper limit of the distance range. For example, when examining the liver region, the distance range is set to 5 mm to 30 mm. In this case, the judgment function 14 determines whether the distance calculated by the calculation function 13 is below 5 mm, within the 5 mm to 30 mm range, or above 30 mm.

[0034] For example, if the distance calculated by the calculation function 13 falls below the lower limit of the distance range, the judgment function 14 determines that the subject's movement is extremely small, and based on this determination, the correction function 15 does not need to perform the automatic correction described later. In other words, if the distance calculated by the calculation function 13 falls below the lower limit of the distance range, the correction function 15 does not correct the registration information that shows the registration relationship between the coordinate system of the ultrasound system and the coordinate system of other modalities other than the ultrasound system.

[0035] For example, if the distance calculated by the calculation function 13 is within the distance range, the determination function 14 determines that the subject's movement range is appropriate for the automatic correction described later, and based on the result of this determination, the correction function 15 performs the automatic correction described later. In other words, if the distance calculated by the calculation function 13 is within the distance range, the correction function 15 generates the correction information described later and corrects the registration information based on the generated correction information.

[0036] For example, if the distance calculated by the calculation function 13 exceeds the upper limit of the distance range, the determination function 14 needs to further determine the cause of the distance exceeding the upper limit of the distance range.

[0037] Specifically, if the determination function 14 determines that the distance calculated by the calculation function 13 exceeds the upper limit of the distance range, it further determines whether the characteristic points of the current time were obtained in the calculation by the calculation function 13.

[0038] For example, if the judgment function 14 determines that a feature point for the current time has been obtained, it may determine that the distance calculated by the calculation function 13 exceeds the upper limit of the distance range because the subject's movement is too large. In this case, automatic correction is inappropriate, and the judgment function 14 notifies the correction function 15 to stop the automatic correction process. At this time, the correction function 15 outputs the message "Large movement occurred, automatic correction impossible" by voice. Alternatively, the correction function 15 controls the display control function 16 to display the message on the display 22.

[0039] On the other hand, if the determination function 14 determines that a feature point for the current time has not been obtained, it can determine that this is due to a stop command input by the user. In this case, the determination function 14 notifies the correction function 15 to stop the automatic correction process. At this time, the correction function 15 outputs the message "Automatic correction has stopped based on the user's stop command" audibly. Alternatively, the correction function 15 controls the display control function 16 to display the message on the display 22.

[0040] Although we have described an example where the judgment function 14 is separate from the correction function 15, it is also possible to omit the judgment function 14 and have the correction function 15 perform the functions of the judgment function 14.

[0041] The correction function 15 has the functions to generate a correction matrix, which is correction information; to update the feature group for the previous time; and to update the transformation matrix for the current time (the time the correction occurs), and performs automatic correction. For example, depending on the function that performs automatic correction, the correction function 15 may be divided into a correction matrix generation function 15A, a feature group update function 15B, and a transformation matrix update function 15C.

[0042] The correction matrix generation function 15A generates a three-dimensional correction matrix based on the coordinate information of the corresponding feature points at the current time and the previous time, if the determination function 14 determines that the distance calculated by the calculation function 13 is within a predetermined distance range.

[0043] The feature group update function 15B updates the coordinate information of the feature points at the previous time to the coordinate information of the feature points at the current time, after the three-dimensional correction matrix has been generated by the correction matrix generation function 15A. Specifically, the feature group update function 15B replaces the coordinate information of the feature points at the current time with the coordinate information of the feature points at the previous time, and stores the replaced coordinate information of the feature points at the current time in the first memory area 12A. The coordinate information of the feature points stored in the first memory area 12A is used as the coordinate information of the feature points at the previous time when the next ultrasound image sampling is performed.

[0044] The transformation matrix update function 15C updates the transformation matrix using the three-dimensional correction matrix generated by the correction matrix generation function 15A and the transformation matrix stored in the second memory area 12B. By updating the transformation matrix, the transformation matrix update function 15C corrects the registration information that shows the registration relationship between the coordinate system of the ultrasound system already established before surgery and the coordinate system of other modalities other than the ultrasound system. The transformation matrix update function 15C also stores the updated transformation matrix as the current transformation matrix in the second memory area 12B. That is, after the registration information is corrected, the transformation matrix update function 15C replaces the registration information stored in the second memory area 12B with the corrected registration information.

[0045] Thus, the medical image processing device 1 according to this embodiment can automatically correct the registration relationship between the coordinate system of the ultrasound system already established before surgery and the coordinate system of other modalities (three-dimensional CT coordinate system or three-dimensional MR coordinate system) by updating the transformation matrix. For this reason, the medical image processing device 1 according to this embodiment can improve the accuracy of registration while saving time until registration is performed.

[0046] The process of automatic correction of volume data will be explained below with reference to Figure 2. Figure 2 is a flowchart showing the processing procedure (medical image processing method) by the medical image processing apparatus 1 according to the first embodiment.

[0047] In step S100 of Figure 2, a reference two-dimensional ultrasound image of one group is collected at the time the surgery begins.

[0048] Specifically, in step S100 of Figure 2, the ultrasound image acquisition function 11A acquires one group of reference two-dimensional ultrasound images, and in step S101 of Figure 2, the feature group acquisition function 11B extracts the coordinate information of feature points in the reference two-dimensional ultrasound image and stores it in the first memory area 12A as the coordinate information of the feature points at the immediately preceding time. Here, the reference two-dimensional ultrasound image is a two-dimensional ultrasound image collected after the start of surgery while the patient was stationary, and its coordinate system is the same as that of the ultrasound volume data obtained before surgery.

[0049] In step S200 of Figure 2, the ultrasound image acquisition function 11A obtains one group of two-dimensional ultrasound images at the current time. While the probe is moving, one group of two-dimensional ultrasound images is obtained at intervals of Δt (for example, Δt = 250 ms). In this way, the ultrasound image acquisition function 11A acquires multiple groups of two-dimensional ultrasound images at predetermined sampling intervals, so that each time one group of ultrasound images is acquired at intervals of Δt, the judgment function 14 can determine whether or not to perform automatic correction. The sampling interval Δt for the two-dimensional ultrasound images can be set by the user.

[0050] In step S300 of Figure 2, the feature group acquisition function 11B extracts a feature group from the two-dimensional ultrasound image obtained at the current time that represents a set of regions with features in the examination area, for example, a feature group that represents a set of multiple feature points having anatomical features.

[0051] Specifically, the feature group acquisition function 11B extracts coordinate information of feature points in the two-dimensional ultrasound image obtained at the current time. For example, in this embodiment, in step S300, multiple anatomical features (e.g., bifurcation points of hepatic blood vessels) are extracted from the two-dimensional ultrasound image. Subsequently, the coordinate information of the feature points at the current time and the coordinate information of the feature points at the previous time are used to calculate the distance in step S400 and to generate the correction matrix in step S500.

[0052] In step S400 of Figure 2, the calculation function 13 calculates the distance between each feature point at the current time extracted by the feature group acquisition function 11B and each feature point at the previous time stored in the first memory area 12A. The calculated distance may be, but is not limited to, the average of the Euclidean distances calculated using multiple corresponding feature point pairs, or the maximum distance of multiple corresponding feature point pairs.

[0053] In step S500 of Figure 2, the determination function 14 determines the relationship between the distance calculated by the calculation function 13 and a predetermined distance range.

[0054] If the calculated distance falls within a predetermined distance range (for example, 5mm to 30mm for the liver), the process proceeds to step S600. In this case, the correction function 15 performs automatic correction, and a correction matrix is ​​generated.

[0055] Furthermore, if the calculated distance falls below a predetermined lower limit of the distance range (for example, 5 mm for the liver), the automatic correction function 15 will not be performed.

[0056] Furthermore, if no feature points are detected, or if the calculated distance exceeds the upper limit of a predetermined distance range (for example, 30 mm for the liver), the judgment function 14 notifies the correction function 15 to stop the automatic correction process. At this time, the correction function 15 presents information regarding the cause of the stop. For example, the correction function 15 may present "Automatic correction stopped based on user's stop command" or "Large movement occurred, automatic correction impossible."

[0057] The distance ranges mentioned above are merely examples; specific distance ranges can be set by the user based on the sampling interval of the two-dimensional ultrasound images.

[0058] In step S600 of Figure 2, the correction matrix generation function 15A of the correction function 15 generates a three-dimensional correction matrix based on the feature points of the current time and the feature points of the previous time.

[0059] Specifically, the correction matrix generation function 15A extracts coordinate information of corresponding feature points from the two-dimensional ultrasound images of the current time and the previous time, and uses this coordinate information to generate a three-dimensional correction matrix. The correction matrix represents the registration relationship between multiple corresponding feature point pairs and can be generated to minimize the distance between corresponding feature points, but the method of generating the correction matrix is ​​not limited to this.

[0060] In step S700 of Figure 2, the transformation matrix update function 15C updates the transformation matrix based on the current transformation matrix stored in the second memory area 12B and the three-dimensional correction matrix generated by the correction matrix generation function 15A.

[0061] Specifically, in step S600 of Figure 2, the transformation matrix update function 15C obtains a three-dimensional correction matrix using the correction matrix generation function 15A. Then, in step S601 of Figure 2, it combines the three-dimensional correction matrix with the initial transformation matrix (between the ultrasound system coordinate system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system) to obtain an updated transformation matrix, which is stored in the second memory area 12B. In subsequent automatic correction, the updated transformation matrix is ​​used as the current transformation matrix for the next automatic correction. In step S700 of Figure 2, the transformation matrix update function 15C updates the transformation matrix based on the current transformation matrix stored in the second memory area 12B and the three-dimensional correction matrix generated by the correction matrix generation function 15A. Then, in step S701 of Figure 2, the transformation matrix update function 15C corrects the registration relationship between the ultrasound system coordinate system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system, which was already established before the surgery, by updating the transformation matrix.

[0062] Each time the transformation matrix is ​​updated, the registration relationship between the ultrasound system's coordinate system and the 3D CT coordinate system or 3D MR coordinate system is also updated accordingly, completing this automatic correction process.

[0063] In step S800 of Figure 2, the feature group update function 15B updates the coordinate information of each feature point from the previous time to the coordinate information of each feature point from the current time, after a three-dimensional correction matrix has been generated by the correction matrix generation function 15A.

[0064] Specifically, in step S800 in Figure 2, the feature group update function 15B replaces the coordinate information of each feature point at the current time with the coordinate information of each feature point at the previous time stored in the first memory area 12A. Then, in step S101 in Figure 2, the feature group update function 15B stores the coordinate information of the feature points at the previous time in the first memory area 12A. In the subsequent automatic correction, the replaced coordinate information of each feature point is used as the coordinate information of each feature point at the previous time for the next automatic correction.

[0065] In step S900 of Figure 2, when the determination function 14 receives a stop command from the user, it stops the automatic correction process based on the stop command.

[0066] Specifically, the judgment function 14 notifies the correction function 15 to stop the automatic correction process based on a stop command from the user. At this time, the correction function 15 presents "Automatic correction stopped based on the user's stop command" as information regarding the cause of the stop. Normally, after automatic correction starts, the process moves to a looping process (automatic correction loop), and each time a new feature point (feature point at the current time) is collected in steps S200 and S300, in step S400, the calculation function 13 calculates the distance between the feature point at the current time and the feature point at the previous time, and in step S500, the judgment function 14 determines whether it is necessary to generate a correction matrix and correct the registration relationship between the ultrasound system coordinate system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system by determining the relationship between the calculated distance and a predetermined distance range. Here, the automatic correction process can only be stopped when the calculated distance exceeds the upper limit of the predetermined distance range. On the other hand, the user can also stop the automatic correction process voluntarily as needed.

[0067] For example, when the judgment function 14 receives a stop command from the user, it discards the feature group, i.e., the feature points, of the two-dimensional ultrasound image extracted at the current time. In this case, since there are no feature points at the current time, the distance calculated in step S400 exceeds the upper limit of a predetermined distance range. Therefore, in the judgment in step S500, the judgment function 14 stops the automatic correction process and indicates that the automatic correction is complete. Here, one example of a means to stop the automatic correction process is to discard the collected feature points when a stop command from the user is received, but it is not limited to this; it may also be to stop the collection of two-dimensional ultrasound images when a stop command from the user is received. Furthermore, if the user needs to scan another location, they can stop the automatic correction of the currently scanned location by inputting a stop command, and then acquire a reference two-dimensional ultrasound image of the other location.

[0068] As described above, in the medical image processing apparatus 1 according to this embodiment, first, the generation function 10 generates registration information that shows the registration relationship between the coordinate system of the ultrasound system and the coordinate system of other modalities other than the ultrasound system. Next, the acquisition function 11 acquires a two-dimensional ultrasound image of the subject at predetermined time intervals during the probe movement process and extracts feature groups from the two-dimensional ultrasound image. Then, the calculation function 13 calculates the distance between the feature group at the current time and the feature group at the previous time, and the determination function 14 and correction function 15 correct the registration information based on this distance. As a result, the registration relationship between the coordinate system of the ultrasound system and the coordinate system of other modalities other than the ultrasound system (three-dimensional CT coordinate system or three-dimensional MR coordinate system), which was already established before the surgery, can be automatically corrected without interrupting the surgery. Therefore, according to the medical image processing apparatus 1 according to this embodiment, the time required for registration can be saved while improving the accuracy of registration. Furthermore, according to the medical image processing apparatus 1 according to this embodiment, artificial interference can be eliminated by automating the entire process of correcting the registration relationship, thereby improving convenience.

[0069] (Second embodiment) The following describes the medical image processing apparatus 1 according to the second embodiment.

[0070] In the first embodiment, a method is described as extracting feature points using a two-dimensional B-mode ultrasound image and automatically correcting the registration relationship based on the feature points.

[0071] On the other hand, in the second embodiment, ultrasound images from scan modes other than two-dimensional B-mode can also be used for automatic correction. That is, ultrasound images from multiple scan modes can be used for feature group extraction and distance calculation. Examples of ultrasound images from other scan modes include ultrasound images from Doppler mode (also called Doppler images) and ultrasound images from SMI (Super Micro-vascular Imaging) mode (also called SMI images), which represent hemodynamics (blood flow rate, blood flow velocity, etc.).

[0072] A Doppler image is generated by extracting components corresponding to, for example, blood flow or tissue from a received signal, and then calculating blood flow or tissue information for multiple points based on the extracted components. The Doppler image includes power, average velocity, variance, etc., of the components corresponding to blood flow or tissue.

[0073] Ultra-micro blood flow imaging technology enables super-resolution imaging of microvessels by positioning and accumulating time for a single microbubble flowing through microvessels, thereby eliminating the effects caused by acoustic diffraction.

[0074] In the second embodiment, ultrasound images from other scan modes can be superimposed on the acquired two-dimensional B-mode ultrasound image. In other words, by superimposing data from a Doppler image or SMI image containing blood flow information onto the two-dimensional B-mode ultrasound image, ultrasound images from multiple scan modes can be superimposed and displayed.

[0075] The difference between the second embodiment and the first embodiment is that in the first embodiment, two-dimensional B-mode is used as the base scan mode for the entire period of automatic correction, and ultrasound images in two-dimensional B-mode are used for feature group extraction, distance calculation, and automatic correction. On the other hand, in the second embodiment, ultrasound images from multiple scan modes (B-mode as the base scan mode, and other scan modes such as Doppler mode or SMI mode) are used, of which ultrasound images in B-mode are used for feature group extraction and distance calculation, and ultrasound images in Doppler mode or SMI mode are used for feature group extraction, distance calculation, and automatic correction. Similar to two-dimensional B-mode ultrasound images, feature groups in Doppler mode or SMI mode may be sets of feature points with anatomical features, and when Doppler mode is used, example of a feature group extracted from a Doppler image is a vascular bifurcation.

[0076] Figure 3 is a flowchart showing the processing procedure (medical image processing method) by the medical image processing apparatus 1 according to the second embodiment.

[0077] Compared to the first embodiment, the second embodiment replaces step S201 and step S202 with step S200 of the first embodiment. That is, in the first embodiment, a two-dimensional B-mode ultrasound image of the current time is acquired, while in the second embodiment, in addition to a two-dimensional B-mode ultrasound image of the current time, an SMI image or Doppler image of the current time is further acquired.

[0078] Furthermore, in the second embodiment, feature group extraction and distance calculation can be performed on ultrasound images from other scan modes (SMI mode or Doppler mode) each time a sample is taken. However, this is time-consuming, so it is desirable to save time in this part. For this reason, it is conceivable to use only two-dimensional B-mode ultrasound images for feature point extraction and distance calculation.

[0079] Information about blood flow or tissue contained in Doppler and SMI images is useful for correction of blood flow or tissue. When the area of ​​interest is blood flow or tissue, correction using Doppler and SMI images is more effective.

[0080] To summarize the above, in order to save time on automatic correction in response to actual detection requests, feature points are extracted from the Doppler image or SMI image only when it is determined that the subject has moved, and in that case, it is determined whether movement has occurred using a two-dimensional B-mode ultrasound image. Taking the detection of the liver as an example, feature points are obtained from the Doppler image or SMI image only if the distance between the feature point at the current time and the feature point at the previous time in B-mode is within the correction range of 5 mm to 30 mm, and a correction matrix is ​​generated using the extracted feature points.

[0081] The automatic correction loop in the second embodiment will now be described with reference to Figure 3.

[0082] In step S100 of Figure 3, the ultrasound image acquisition function 11A acquires a reference two-dimensional ultrasound image, and in step S101 of Figure 3, the feature group acquisition function 11B extracts the coordinate information of feature points in the reference two-dimensional ultrasound image and stores it in the first memory area 12A as the coordinate information of feature points at the immediately preceding time. Here, the reference two-dimensional ultrasound image includes a two-dimensional B-mode ultrasound image and an SMI-mode ultrasound image or a Doppler-mode ultrasound image.

[0083] In step S201 of Figure 3, the ultrasound image acquisition function 11A acquires a two-dimensional B-mode ultrasound image of the current time.

[0084] In step S202 of Figure 3, the ultrasound image acquisition function 11A acquires an SMI mode ultrasound image or a Doppler mode ultrasound image for the current time.

[0085] In step S300 of Figure 3, the feature group acquisition function 11B extracts coordinate information of feature points obtained at the current time from the two-dimensional B-mode ultrasound image or ultrasound image of another scan mode (SMI mode or Doppler mode) at the current time.

[0086] In step S400 of Figure 3, the calculation function 13 calculates the distance between the feature point at the current time extracted by the feature group acquisition function 11B and the feature point at the previous time stored in the first memory area 12A.

[0087] In step S500 of Figure 3, the determination function 14 determines the relationship between the distance calculated by the calculation function 13 and a predetermined distance range.

[0088] If the calculated distance falls below the lower limit of a predetermined distance range, the automatic correction by the correction function 15 is not performed. If the calculated distance exceeds the upper limit of a predetermined distance range, the judgment function 14 notifies the correction function 15 to stop the self-correction process. At this time, the correction function 15 indicates the reason for stopping. If the calculated distance is within the predetermined distance range, the process moves to step S600, and the automatic correction is performed by the correction function 15.

[0089] In step S600 of Figure 3, the correction matrix generation function 15A of the correction function 15 generates a three-dimensional correction matrix based on the coordinates of feature points extracted from ultrasound images of other scan modes (SMI mode or Doppler mode).

[0090] In step S700 of Figure 3, the transformation matrix update function 15C updates the current transformation matrix based on the correction matrix generated by the correction matrix generation function 15A and the transformation matrix stored in the second memory area 12B.

[0091] Specifically, in step S600 of Figure 3, the transformation matrix update function 15C obtains a three-dimensional correction matrix using the correction matrix generation function 15A. Then, in step S601 of Figure 3, it combines the three-dimensional correction matrix with the initial transformation matrix (between the ultrasound system coordinate system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system) to obtain an updated transformation matrix, which is stored in the second memory area 12B. In subsequent automatic correction, the updated transformation matrix is ​​used as the current transformation matrix for the next automatic correction. In step S700 of Figure 3, the transformation matrix update function 15C updates the transformation matrix based on the current transformation matrix stored in the second memory area 12B and the three-dimensional correction matrix generated by the correction matrix generation function 15A. Then, in step S701 of Figure 3, the transformation matrix update function 15C corrects the registration relationship between the ultrasound system coordinate system and the three-dimensional CT coordinate system or three-dimensional MR coordinate system, which was already established before the surgery, by updating the transformation matrix.

[0092] In step S800 of Figure 3, the feature group update function 15B replaces the coordinate information of feature points from other scan modes at the current time with the coordinate information of feature points from other scan modes at the previous time, which is stored in the first memory area 12A. Then, in step S101 of Figure 2, the feature group update function 15B stores the coordinate information of feature points from the previous time in the first memory area 12A.

[0093] In step S900 of Figure 3, the judgment function 14 stops the automatic correction process based on a stop command from the user.

[0094] Furthermore, the parts not described above are the same as those in the first embodiment, and therefore, redundant explanations are omitted here.

[0095] (Other embodiments) In addition to the embodiments described above, new embodiments can be obtained by adding or removing different constituent elements in each embodiment, or by combining them with each other. These new embodiments are also included in the spirit of this embodiment.

[0096] Furthermore, the configuration requirements of each device illustrated in this embodiment are functional concepts and do not necessarily have to be physically configured as shown. That is, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and unified in any unit according to various loads and usage conditions. Moreover, all or any part of the processing functions performed by each device can be realized by a CPU and a program that is analyzed and executed by that CPU, or by hardware using wired logic.

[0097] Furthermore, the names and combinations of information shown in the above diagram are merely examples and can be changed as needed.

[0098] Furthermore, the medical image processing method described in this embodiment can be implemented by executing a pre-prepared program on a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet. Alternatively, this program can be stored on a computer-readable non-temporary storage medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by being read from the storage medium by a computer.

[0099] According to at least one embodiment described above, the registration relationship between the coordinate system of the ultrasound system already established before surgery and the coordinate system of other modalities other than the ultrasound system can be automatically corrected.

[0100] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0101] 1. Medical image processing device 10 Generation function 11. Acquisition function 13. Calculation Function 15. Correction Function

Claims

1. A generation unit that generates registration information showing the registration relationship between the coordinate system of the ultrasonic system and the coordinate system of other modalities other than the ultrasonic system, During the probe's movement, an acquisition unit acquires two-dimensional ultrasound images of the subject at predetermined time intervals, and extracts feature groups, which are sets of multiple feature points having anatomical characteristics, from each acquired two-dimensional ultrasound image. A calculation unit calculates the Euclidean distance for each of a plurality of feature point pairs, each consisting of a plurality of feature points included in the feature group at the current time and a corresponding plurality of feature points included in the feature group at the previous time, and calculates the average or maximum value of the calculated plurality of Euclidean distances as the distance between the feature group at the current time and the feature group at the previous time. A correction unit corrects the registration information based on the distance calculated by the calculation unit, A medical image processing device equipped with [a specific feature].

2. Ultrasound images generated in multiple scan modes are used for extracting the feature group and calculating the distance. The medical image processing apparatus according to claim 1.

3. The aforementioned feature group is a collection of multiple feature points in a Doppler image or an ultramicroblood flow imaging image. The medical image processing apparatus according to claim 2.

4. If the distance calculated by the calculation unit falls within a predetermined distance range, the correction unit generates correction information and corrects the registration information based on the generated correction information. A medical image processing apparatus according to any one of claims 1 to 3.

5. If the distance calculated by the calculation unit falls below the lower limit of a predetermined distance range, the correction unit does not correct the registration information. A medical image processing apparatus according to any one of claims 1 to 3.

6. If the distance calculated by the calculation unit exceeds the upper limit of a predetermined distance range, the correction unit stops the correction of the registration information and presents information regarding the cause of the stoppage. A medical image processing apparatus according to any one of claims 1 to 3.

7. The correction unit generates the correction information to minimize the distance between corresponding feature points. The medical image processing apparatus according to claim 4.

8. The correction unit stops correcting the registration information based on a stop command from the user. The medical image processing apparatus according to claim 1.

9. After generating the correction information, the correction unit replaces the feature group for the current time with the feature group for the previous time. The medical image processing apparatus according to claim 4.

10. A display control unit that displays volume data from modalities other than the ultrasonic system in real time on the display unit. The medical image processing apparatus according to claim 1, further comprising the above.

11. A first storage unit that stores information about the feature group for the immediately preceding time, A second storage unit that stores information relating to the registration information, Furthermore, The correction unit, After the correction information is generated, the information regarding the feature group at the immediately preceding time stored in the first storage unit is replaced with the information regarding the feature group at the current time. After the registration information has been corrected, the registration information stored in the second storage unit is replaced with the corrected registration information. The medical image processing apparatus according to claim 9.

12. The acquisition unit acquires a two-dimensional ultrasound image including the feature group as a reference image. The medical image processing apparatus according to claim 1.

13. Other modalities besides the aforementioned ultrasound system include CT (Computed Tomography) devices or MRI (Magnetic Resonance Imaging) devices. The medical image processing apparatus according to claim 1.

14. The aforementioned correction information is a correction matrix, The aforementioned registration information is a transformation matrix. The medical image processing apparatus according to claim 4.

15. A generation step of generating registration information that shows the registration relationship between the coordinate system of the ultrasonic system and the coordinate system of other modalities other than the ultrasonic system, During the probe's movement, a two-dimensional ultrasound image of the subject is acquired at predetermined time intervals, and a feature group, which is a collection of multiple feature points having anatomical characteristics, is extracted from each acquired two-dimensional ultrasound image. A calculation step of calculating the Euclidean distance for each of a plurality of feature point pairs consisting of a plurality of feature points included in the feature group at the current time and a corresponding plurality of feature points included in the feature group at the previous time, and calculating the average or maximum value of the calculated plurality of Euclidean distances as the distance between the feature group at the current time and the feature group at the previous time, A correction step of correcting the registration information based on the calculated distance, A medical image processing method including [a specific term].

16. This generates registration information that shows the registration relationship between the coordinate system of the ultrasonic system and the coordinate systems of other modalities besides the ultrasonic system. During the probe's movement, two-dimensional ultrasound images of the subject are acquired at predetermined time intervals, and from each acquired two-dimensional ultrasound image, a feature group, which is a collection of multiple feature points possessing anatomical characteristics, is extracted. For each of the multiple feature point pairs, each consisting of multiple feature points included in the feature group at the current time and corresponding multiple feature points included in the feature group at the previous time, the Euclidean distance is calculated, and the average or maximum value of the calculated multiple Euclidean distances is calculated as the distance between the feature group at the current time and the feature group at the previous time. Based on the calculated distance, the registration information is corrected. A program that instructs a computer to perform a process.

Images