Medical image display device and medical image display method

Abstract

To create a combined image using medical image data of a plurality of frames.SOLUTION: A medical image display apparatus comprises processing circuitry configured to: receive a plurality of frames each containing medical image data; prioritize the plurality of frames; on the basis of the prioritization, select regions of the medical image data based on one or more of the frames; and generate combined image data by combining the selected regions of the medical image data such that different regions of the combined image data contain the medical image data based on different ones of the frames.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments disclosed in the present specification and drawings relate to a medical image display apparatus and a medical image display method.

[0002] Currently, medical volume imaging techniques capable of generating three-dimensional (3D) medical image data using various imaging modalities such as CT, PET, MRI, ultrasound, and X-ray are widely used for imaging and diagnostic purposes.

[0003] A dataset showing motion over time is also referred to as a time dataset. In current medical imaging, the use of time datasets is increasing more and more.

[0004] In general, 3D data is more complex to display than 2D data. Similarly, the display and integration of 4D data is even more complex than that of 3D data.

[0005] An action sequence is a process of synthesizing a continuous sequence of moving images into a single image, and is a well-established concept in the field of photography. The action sequence can be obtained by synthesizing a plurality of images on the same screen.

[0006] Figure 1 shows an example of an action sequence. The motion of a volleyball player is represented by synthesizing a plurality of temporally consecutively acquired images. Thereby, the motions of the volleyball player at respective instants from 10A to 10I are represented in one action sequence composite image. The volleyball player moves from left to right in Figure 1 as time elapses. The action sequence provides a method for representing motion in a still image.

[0007] Generation of an action sequence as shown in Figure 1 depends on spatial separation and separation of foreground and background. The volleyball player at each instant 10A to 10I is spatially separated. The background is extracted from the foreground of each of the image groups to be composited. The foreground includes the volleyball player and the ball. When images are composited to generate the action sequence image 10, a plurality of foregrounds are added to a common background.

[0008] In medical imaging, there is growing interest in creating displays that synthesize complex volume data into a single image. Examples include curved planar reformation (CPR), colon anatomy, and thoracic cage reformatting. Instead of searching through multiple images, users are provided with a single summary display.

[0009] Medical imaging includes creating video sequences, such as those used in ultrasound and angiography. While it is desirable to create an overview of the video sequence, there are very few image sequences that exhibit natural spatial separation for medical use. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Publication No. 2020-049204 [Overview of the Initiative] [Problems that the invention aims to solve]

[0011] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is the creation of a composite image using medical image data from multiple frames. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0012] The medical image display device according to the embodiment includes a processing circuit. The processing circuit receives a plurality of frames, each containing medical image data, defines a priority order for the plurality of frames, selects a region of medical image data based on one or more of the plurality of frames based on the priority order, and generates composite image data by compositing the selected regions of medical image data such that different regions of the composite image data contain medical image data based on different frames from the plurality of frames. The selection of the region includes determining a group of segment objects in each of the plurality of frames, and selecting the region based on the group of segment objects. [Brief explanation of the drawing]

[0013] Next, embodiments shown in the following drawings will be described as non-limiting examples. [Figure 1] Figure 1 shows an example of an operation sequence. [Figure 2] Figure 2 is a schematic diagram of an apparatus according to one embodiment. [Figure 3] Figure 3 is a flowchart illustrating the overview of a method according to one embodiment. [Figure 4A] Figure 4A shows an example of a drawing where a high-priority object is obscured by a low-priority object. [Figure 4B] Figure 4B shows an example of a drawing where a low-priority object is obscured by a high-priority object. [Figure 4C] Figure 4C schematically shows an example of rendering in which a portion of a lower-priority object that obscures a higher-priority object is represented with lower opacity than the portion that does not obscure the higher-priority object. [Figure 5] Figure 5 is a chart illustrating an example of an occlusion tree diagram of an object from three frames. [Figure 6] Figure 6 shows an example of segmentation in an image drawn from an ultrasonic volume using a method according to one embodiment. [Figure 7A] Figure 7A shows an example of an ultrasound image drawn from three frames by a method according to one embodiment. [Figure 7B] Figure 7B shows an example of an image in Figure 7A where different colors are represented with different shading. [Figure 8] Figure 8 shows an example of an image rendered by the method according to one embodiment. [Figure 9] Figure 9 shows the segmentation corresponding to the image of Figure 8. [Figure 10A] Figure 10A shows a first frame including two objects. [Figure 10B] Figure 10B shows the material region of the first frame. [Figure 10C] Figure 10C shows a second frame including two other objects. [Figure 10D] Figure 10D shows the material region of the second frame. [Figure 10E] Figure 10E shows the objects in the first and second frames before occlusion is identified. [Figure 10F] Figure 10F shows the objects in the first and second frames after occlusion is identified. DETAILED DESCRIPTION

[0014] A medical image display apparatus comprising a processing circuit according to one embodiment is provided. The processing circuit receives a plurality of frames each containing medical image data, defines priorities for the plurality of frames, selects regions of the medical image data based on one or more of the plurality of frames according to the priorities, and generates composite image data by compositing the selected regions of the medical image data such that different regions of the composite image data include medical image data based on different frames among the plurality of frames.

[0015] A method according to one embodiment comprises receiving a plurality of frames each containing medical image data, defining priorities for the plurality of frames, selecting regions of the medical image data based on one or more of the plurality of frames according to the priorities, and generating composite image data by compositing the selected regions of the medical image data such that different regions of the composite image data include medical image data based on different frames among the plurality of frames.

[0016] Figure 2 shows a schematic of the medical image processing device 20 according to the embodiment. The medical image processing device 20 is also referred to as a medical image display device. The medical image processing device 20 includes a computing device 22, in this example, a personal computer (PC) or workstation. The computing device 22 is connected to the scanner 24 via a data storage unit 30.

[0017] Furthermore, the medical image processing device 20 includes one or more display screens 26 and one or more input devices 28 such as a computer keyboard, mouse, or trackball.

[0018] In this embodiment, the scanner 24 is an ultrasound scanner configured to acquire ultrasound volume data using an ultrasound probe (not shown in Figure 2). The scanner 24 generates image data representing a volume that includes at least one anatomical region of a patient or other subject. The image data represents a plurality of voxels, each having an image data value.

[0019] In another embodiment, the scanner 24 may be configured to acquire two-dimensional, three-dimensional, or four-dimensional image data using any imaging modality. For example, the scanner 24 may include a magnetic resonance (MR) scanner, a computed tomography (CT) scanner, a cone-beam CT scanner, a positron emission tomography (PET) scanner, an X-ray scanner, an ultrasound scanner, and the like.

[0020] In this embodiment, the image dataset acquired by the scanner 24 is stored in the data storage unit 30 and provided to the arithmetic unit 22. In another embodiment, the image dataset is supplied from a remote data storage unit (not shown). The data storage unit 30 or the remote data storage unit may include any type of storage device as appropriate. In one embodiment, the medical image processing device 20 is not connected to the scanner.

[0021] The computing unit 22 includes a processing unit 32 for processing data. The processing unit 32 includes a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit 32 provides processing resources for automatically or semi-automatically processing medical image datasets. In another embodiment, the data to be processed is not limited to medical image data, but may include any image data.

[0022] The processing unit 32 includes a priority circuit 34 for determining the priority of multiple frames, a classification circuit 36 ​​for classifying voxel regions, a voxel processing circuit 38 for processing voxels and storing voxel information, and a rendering circuit 39 for performing image rendering.

[0023] In this embodiment, the priority circuit 34, the classification circuit 36, the voxel processing circuit 38, and the rendering circuit 39 are each implemented in the CPU and / or GPU by a computer program containing computer-readable instructions for performing the method of this embodiment. In another embodiment, these circuits may be implemented as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0024] Furthermore, the arithmetic unit 22 includes a hard drive and other PC components such as RAM, ROM, a data bus, an operating system with various device drivers, and a graphics card. For clarity, these components are not shown in Figure 2.

[0025] Figure 3 is a flowchart illustrating an embodiment of a method that can be performed using the apparatus of Figure 2. In the method of Figure 3, an image is drawn according to information based on multiple frames. These frames are combined according to priority and provided as an action sequence image 10.

[0026] In medical imaging, natural spatial separation as shown in Figure 1 may not be observed in consecutive frames. That is, objects are in different, non-overlapping positions within consecutive frames. Therefore, in the method shown in Figure 3, the contents within a frame are divided into object segmentations. Substances that obscure objects in higher-priority frames are treated differently from substances that do not obscure objects.

[0027] Figures 4A to 4C illustrate various methods for rendering objects in different frames, taking occlusion into account. Object 90 represents a region of matter from a higher-priority frame. Object 90 can be called a high-priority object. Object 92 represents a region of matter from a lower-priority frame. Object 90 can be called a low-priority object. The purpose of this rendering is to display information based on two frames within a single image.

[0028] In the drawing method shown in Figure 4A, the high-priority object 90 is obscured by the low-priority object 92. The high-priority object 90 is depicted as a solid. The low-priority object 92 is depicted as a solid that obscures the high-priority object 90. Since the high-priority object is partially hidden by the low-priority object, this drawing method is considered undesirable.

[0029] In the drawing method shown in Figure 4B, the low-priority object 92 is hidden by the high-priority object 90. Both the high-priority object 90 and the low-priority object 92 are depicted as solids. The drawing method shown in Figure 4B is considered appropriate.

[0030] In Figure 4C, the material obscuring the low-priority object 92 is separated from the material that does not obscure it, and the low-priority object 92 is divided into components 92A, 92B, 92C, 92D, 92E, and 92F. The material that does not obscure or is background material of components 92A, 92B, 92C, 92D, 92E, and 92F is depicted as solid. In other words, the part of the low-priority object 92 that does not obscure the high-priority object 90 is depicted as solid. Component 92F obscures the high-priority object. Component 92F may be made thin or only its outline remains to avoid occlusion. In Figure 4C, all of the high-priority object 90 is visible. The drawing method in Figure 4C can provide a useful way to display both the high-priority object 90 and the low-priority object 92 in a single image. The occlusion problem is considered to need correction.

[0031] The method shown in Figure 3 provides a drawing method for distinguishing between substances to be concealed and substances that are not concealed, as described below.

[0032] Returning to Figure 3, in step 40 of Figure 3, the priority circuit 34 receives a frame set. The frame set contains image data obtained by continuous image acquisition, for example, a set of frames of a four-dimensional image. The image data can be acquired by an appropriate scanning modality, such as ultrasound or CT. Each frame in the frame set represents a volume with an associated coordinate system. Each frame contains a volume image dataset, and the volume image dataset contains the voxel values ​​of the voxels of the volume. The frames correspond to a temporal order, which is the order in which the frames were acquired from the first frame to the last frame.

[0033] In other embodiments, the frame group may include frames acquired by other means, such as scanning or follow-up scanning. In one embodiment, for example, if the frame group contains frames acquired by other means, an image registration process is performed to align the images based on those frames. The image registration process includes rigid or non-rigid registration.

[0034] The voxel processing circuit 38 receives or determines the viewing direction relative to the coordinate system of multiple frames. The viewing direction may be determined according to the modality used to acquire the multiple frames, such as ultrasound or magnetic resonance. Alternatively, it may be input by the user or determined by an appropriate method.

[0035] In step 42, the priority circuit 34 determines the priority of each frame. In the embodiment shown in Figure 3, the priority of the frames is different from the chronological order of the frames. The priority circuit 34 selects and displays one of the frames as the highest priority frame. The highest priority frame is also referred to as the current frame. The highest priority frame can be any frame in the frame group. The highest priority frame is selected in response to user input, for example, when the user selects one frame from the frame group. In another embodiment, the priority circuit 34 may automatically select the highest priority frame, for example, by selecting the most important frame among several frames.

[0036] The highest priority frame is assigned a priority value of "1".

[0037] The highest priority frame is followed by the other frames in the frame group in order of priority. In the embodiment shown in Figure 3, frames are assigned priority values ​​as ascending integers. In another embodiment, an appropriate priority value system can be used as appropriate. For example, the priority value may be a number, a letter, or some other exponent.

[0038] In the embodiment shown in Figure 3, frames other than the highest priority frame are ranked so that future frames and past frames alternate. Future frames are frames acquired later in time than the highest priority frame, and past frames are frames acquired earlier in time than the highest priority frame.

[0039] Consider an example where five frames f1, f2, f3, f4, and f5 are acquired at times t1, t2, t3, t4, and t5, respectively. t1 is the earliest time, and t5 is the latest time.

[0040] Frame f3 is selected as the highest priority frame with the highest priority and is given a priority value of "1". Frame f4, the closest future frame to the highest priority frame, is selected as the second highest priority frame and is given a priority value of "2". Frame f2, the closest past frame to the highest priority frame, is selected as the third highest priority frame and is given a priority value of "3". The next future frame f5 is selected as the fourth highest priority frame and is given a priority value of "4". The next past frame f1 is selected as the fifth highest priority frame and is given a priority value of "5". Thus, the priority order is frames f3, f4, f2, f5, f1, which is different from the original temporal order of frames f1, f2, f3, f4, f5.

[0041] In the embodiment shown in Figure 3, past and future frames are alternated to maintain a constant temporal distance. In other embodiments, different priorities can be used.

[0042] In one embodiment, future frames are given a higher priority than past frames. The highest priority frame is the frame with the highest priority. Next, future frames are given a higher priority in chronological order. Past frames have a lower priority than future frames. For past frames, the chronological order may be reversed so that past frames that are temporally closer to the highest priority frame are given a higher priority than past frames that are farther away. For example, in the case of frames f1, f2, f3, f4, f5 in the temporal order where the highest priority frame is f3, the priority order in which future frames are given to past frames is f3, f4, f5, f2, f1.

[0043] In other embodiments, earlier frames may be given higher priority than later frames. For example, in the case of frames f1, f2, f3, f4, f5 in a temporal order where the highest priority frame is f3, the priority order for earlier frames over later frames would be f3, f2, f1, f5, f4.

[0044] In one embodiment, priority is determined for the highest priority frame and one or more past frames, excluding future frames. In another embodiment, priority is determined for the highest priority frame and one or more future frames, excluding past frames. In yet another embodiment, priority is determined for the highest priority frame, all past frames, and the last future frame.

[0045] In another embodiment, any priority can be used as appropriate. Any of the frames can be selected as the highest priority frame based on any selection method.

[0046] In step 44, the classification circuit 36 ​​selects one frame from the frame group to be processed. In steps 44 to 56 of Figure 3, the processing order of the frames is arbitrary. The classification circuit 36 ​​can select any of the frames from the frame group, for example, in order of priority or in chronological order. In another embodiment, the processing order, such as in order of priority or chronological order, may be fixed.

[0047] In step 46, the classification circuit 36 ​​classifies the regions of the volume image dataset for the selected frame. In the embodiment shown in Figure 3, voxels are classified into one of four categories. The first category is voxels belonging to the material region 48. The second category is voxels belonging to the external void 50. The third category is voxels belonging to the internal void 52. The fourth category is voxels belonging to the signal-deficient region 54.

[0048] The material region 48 is a region in which material, such as cell tissue, is identified. The material region 48 may include a single object or a collection of multiple objects. The external void 50 is, for example, the area surrounding the scanned patient's body. The internal void 52 is, for example, an air region inside the scanned patient's body, such as an air region in the colon. The signal deficiency area 54 is, for example, a region in which no signal is obtained or only a weak signal is obtained. For example, in ultrasound and magnetic resonance, there are regions in which no signal is obtained or only a weak signal is obtained.

[0049] In another embodiment, any category can be used as appropriate. The number of categories is arbitrary. Some of the categories such as material area 48, external void 50, internal void 52, and signal deficiency area 54 may be used. Certain categories from the categories such as material area 48, external void 50, internal void 52, and signal deficiency area 54 may be grouped together. One or more of the categories such as material area 48, external void 50, internal void 52, and signal deficiency area 54 may be divided into multiple categories.

[0050] In the embodiment shown in Figure 3, classification is performed by setting the image values ​​of voxels in a volume image dataset as thresholds and applying one or more morphological operators to segment regions of different categories. For example, a threshold may be used to distinguish material from voids, and one or more morphological operators may be used to distinguish external voids from internal voids. The outline of a material region may be identified to separate internal voids. In another embodiment, the classification may appropriately include any classification process, such as segmentation.

[0051] In step 56, the region classification of the selected frame is completed. If there are still frames to be classified within the frame group, the process in Figure 3 returns to step 44, and the classification circuit 36 ​​selects another frame to be processed. Once all classifications within the frame group are complete, the process in Figure 3 proceeds to step 58.

[0052] In step 58, the voxel processing circuit 38 selects frames to be processed according to the priority determined in step 42. At the beginning of step 58, the highest priority frame with a priority value of "1" is selected. Subsequent frames are processed in ascending order of priority value.

[0053] In step 60, the voxel processing circuit 38 selects a voxel to be processed from the selected frame. Initially, this is a voxel from the highest priority frame. The selected voxel is one of a group of voxels classified as a material region or internal cavity.

[0054] In the embodiment shown in Figure 3, the order in which voxels are selected for processing is determined by the viewing direction. Within the selected frame, voxels closer to the conceptual observer with respect to the viewing direction are processed before voxels further away. When calculating occlusion, as described below with reference to step 64, the calculated occlusion can be reused in an order that traverses the frame. When measuring more complex occlusion, this traversal calculation may differ from a linear traverse with respect to the viewing direction. Additional temporary storage may also be used. In another embodiment, voxels within the selected frame can be processed in any order as appropriate.

[0055] In step 62, the voxel processing circuit 38 determines whether or not there is a voxel value written to the voxel position corresponding to the selected voxel. If the first frame, which is the highest priority frame, is the target of processing, there is no voxel value written to the voxel position. The process in Figure 3 proceeds to step 64.

[0056] In step 64, the voxel processing circuit 38 identifies the occlusion level associated with the selected voxel. The occlusion level is described in detail below with reference to Figure 5. If the first frame is the one to be processed, there is no occlusion, and the occlusion level is zero.

[0057] The voxel processing circuit 38 writes the occlusion level of the voxel to the corresponding voxel location within the object occlusion index 66. In other words, the voxel processing circuit 38 assigns an occlusion index value.

[0058] In the case of the highest priority frame, there is no occlusion, so the voxel processing circuit 38 does not perform object partitioning according to occlusion. Object partitioning performed for subsequent frames is described below, referring to step 64.

[0059] In step 68, the voxel processing circuit 38 performs contour processing. Contour processing in step 68 is optional and may be omitted depending on the embodiment. When compositing multiple objects based on different frames, contour processing artificially inserts gaps between the objects.

[0060] In contour processing, the voxel processing circuit 38 determines whether the selected voxel is located at the boundary of the corresponding object. Voxels at the boundary are removed from the object by contour processing. By removing boundary voxels, gaps are created when drawing the object.

[0061] In another embodiment, voxels contained within the boundary are added to the boundary region and treated as separate objects. During drawing, the boundary region is used to create gaps between objects.

[0062] By artificially creating gaps between objects, the boundaries between objects based on different frames can be highlighted within the drawn image. Highlighting the boundaries between objects emphasizes that their content is based on different frames. Using contouring to emphasize differences between objects is particularly useful when there are no significant differences in the colors of the objects during rendering.

[0063] In step 70, the voxel processing circuit 38 writes the voxel value of the voxel to the corresponding voxel location in volume 72. The voxel processing circuit 38 also writes the label determined in step 68 to the corresponding voxel location in segmentation volume 74.

[0064] In step 76, the voxel processing is complete. If there are any voxels to be processed remaining in the material regions and internal cavities within the selected frame, the method in Figure 3 returns to step 60 and another voxel is selected. If all voxels in the material regions and internal cavities within the selected frame have been processed, the method in Figure 3 returns to step 58.

[0065] In step 58, the voxel processing circuit 38 selects the next frame to be processed according to the priority determined in step 42. The priority value of the next selected frame is 1, which is higher than that of the previous frame.

[0066] In step 60, the voxel processing circuit 38 selects a voxel from the next selected frame. The selected voxel is one of a group of voxels classified as material regions or internal cavities within the next frame. In the embodiment shown in Figure 3, the order in which voxels are selected for processing is determined by the viewing direction, as described above. Voxels closer to the conceptual observer are processed before voxels further away.

[0067] In step 62, the voxel processing circuit 38 determines, for example, by referring to volume 72, whether or not a voxel value has been written to the voxel location corresponding to the selected voxel. If a voxel value has been written to the voxel location, the method in Figure 3 proceeds to step 76, and processing for this voxel is completed.

[0068] If no voxel value has been written to a voxel location, the method in Figure 3 proceeds to step 64. For a second-priority frame, the written voxels are those in the material area or internal cavity of the first-priority frame. If a voxel in the material area or internal cavity of the second-priority frame is still in an unwritten voxel location, it is because the location of the material area or internal cavity in the second frame is different from that of the highest-priority frame. Therefore, the process in Figure 3 distinguishes the voxels in the second frame that represent movement from the highest-priority frame.

[0069] Similarly, for subsequent frames, voxel locations where voxel values ​​have not been written do not constitute part of any material region or internal void in a higher-priority frame.

[0070] In step 64, the voxel processing circuit 38 determines the occlusion level associated with the selected voxel. If it is determined that the voxel does not conceal, the occlusion level is zero. If it is determined that the voxel does conceal, the occlusion level is other than zero. The occlusion levels are explained below with reference to Figure 5.

[0071] Occlusion is determined using an occlusion detection method. This method infers from the visual direction the areas where lower-priority objects obscure higher-priority objects are located, and separates these areas as a segmentation hierarchy. The visual direction refers to the observer's position.

[0072] In the embodiment shown in Figure 3, the occlusion detection method utilizes the observer's direct line of sight. A voxel is determined to be obscured if it lies in the direct line of sight between the observer and one or more voxels of higher-priority objects. If there are one or more higher-priority objects in the line of sight, and a voxel lies in the line of sight between the observer and the foremost of the higher-priority objects, it can be determined that the voxel is obscured. In another embodiment, a voxel may be determined to be obscured if it lies in the direct line of sight between the observer and any of the higher-priority objects.

[0073] In another embodiment, the occlusion detection method determines that a voxel is obscuring a voxel if it lies within a defined angular range relative to a voxel of a higher-priority object, for example, within a cone extending from the voxel of the higher-priority object toward the observer.

[0074] When using parallel projection, the viewing direction of the parallel projection may be used as an occlusion detection method.

[0075] In modalities with a preferred line of sight axis or orientation, the concealment determination of an occlusion detection method may be limited to that preferred line of sight axis or orientation. In some situations, ultrasound or magnetic resonance imaging may have a preferred line of sight axis or orientation, for example, always oriented around the probe direction or the main scan direction. A voxel is determined to be concealed if it lies between the observer and a higher-priority object on the preferred line of sight axis.

[0076] In other embodiments, voxel occlusion determination may be performed by any method as appropriate.

[0077] If a voxel is determined to be a target for concealment based on the occlusion detection method used by the voxel, the voxel processing circuit 38 assigns an occlusion level according to what is being concealed. If the voxel conceals only multiple voxels of one or more higher-priority objects with an occlusion level of zero (not concealed), the voxel processing circuit 38 assigns an occlusion level of "1". If the voxel conceals one higher-priority object voxel with an occlusion level of "1", the voxel processing circuit 38 assigns an occlusion level of "2".

[0078] Furthermore, objects with a higher priority that are subject to occlusion testing are those obtained from past high-priority object frames.

[0079] In one embodiment, the occlusion level determined for one or more preceding voxels can be reused in determining the current voxel. For example, if occlusion is defined along a single axis, reusing known values ​​from past voxels in determining the current voxel may result in the voxel traversing that axis.

[0080] A voxel can be assigned to one of two or more locations on a given object, based on its occlusion level.

[0081] For example, a voxel in a material region where an occlusion level of "0" is determined in step 64 is assigned to one location within the material region, while a voxel in a material region where an occlusion level of "1" is determined is assigned to a different location within the material region. Different labels may be assigned to different locations. A voxel determined to obscure a first object is assigned to a different location and given a different level than a voxel determined to obscure a second object, such as an object based on a different frame.

[0082] In step 68, the voxel processing circuit 38 performs contour processing. Contour processing in step 68 is optional and can be omitted depending on the embodiment. Contour processing artificially inserts gaps between objects when compositing objects based on different frames.

[0083] In contour processing, the voxel processing circuit 38 determines whether the selected voxel is included in the boundary of the corresponding object. Voxels included in the boundary are removed from the object by contour processing. By removing boundary voxels, gaps are created between objects when the objects are drawn.

[0084] In another embodiment, voxels contained within the boundary are added to the boundary region and treated as separate objects. The boundary region is used to create gaps between objects during rendering.

[0085] Figure 5 is a schematic chart illustrating an example of an object occlusion tree diagram based on three frames. The object hierarchy can be constructed based on this occlusion tree diagram. The object hierarchy may also be determined using the occlusion determination in step 64.

[0086] Frame 1, code 101, is the highest priority frame and has the highest priority. Object 1, code 111, contains matter within frame 1. Since the voxels of object 1 do not occlude, it is assigned an occlusion level of "0".

[0087] Frame 2, code 102, is the second-highest priority frame. The matter within Frame 2 is divided into two objects. Object 2, code 112, contains part of the matter within Frame 2. Frame 2 is in front of Object 1 and is determined to obscure Object 1. The voxels of Object 2 obscure Object 1 and are therefore assigned an occlusion level of "1". Object 3, code 113, contains the remaining part of the matter within Frame 2. Object 3 is not determined to be in front of Object 1. The voxels of Object 3 do not obscure Object 1 and are therefore assigned an occlusion level of "0".

[0088] Frame 3, designated 103, is the third-priority frame. The material within frame 3 is divided into three objects. Object 4, designated 114, includes a portion of the material within frame 3 that is in front of object 2 and is judged to conceal object 2. Object 2 includes the material of frame 2 that is in front of object 1 and is judged to conceal object 1. In other words, object 4, designated 114, includes material that is in front of both object 1 and object 2 and conceals both of them.

[0089] The voxels of object 4 are assigned an occlusion level of "2" because they conceal object 2. Object 2 itself also conceals other objects, and its occlusion level is "1".

[0090] Object 5, designated 115, includes a portion of material within frame 3 that is in front of object 1 and is judged to conceal object 1. Frame 3 is not located in front of object 2. The voxels of object 5 are assigned an occlusion level of "1" because they conceal object 1. Object 1 is not the object being concealed.

[0091] Object 6, reference numeral 116, comprises the remaining portion of material within frame 3 that is not determined to be in front of object 1 or object 2. The voxels of object 6 are assigned an occlusion level of "0" because they do not obscure any other objects.

[0092] Generally, the occlusion level of a voxel of an object that is concealing something can be determined by adding 1 to the highest occlusion level of the object being concealed by other objects.

[0093] The occlusion tree diagram in Figure 5 may be used in subsequent drawing steps, such as drawing step 80 described below. The occlusion tree diagram may also be used to obtain an opacity formula to guarantee the contribution of pixels when drawing a group of occluding objects. For example, if object 2 occludes object 1, and object 4 occludes object 2, the opacity formula may specify that object 1 accounts for 50% of the opacity, object 2 for 25%, and object 4 for 25%.

[0094] Returning to Figure 3, after step 68, the method in Figure 3 proceeds to step 70. In step 70, the voxel processing circuit 38 writes the voxel value to the corresponding voxel location in volume 72. The voxel processing circuit 38 also writes the label determined in step 66 to the corresponding voxel location in segmentation volume 74.

[0095] In step 76, the voxel processing is complete. If there are any voxels remaining to be processed in the material regions or internal cavities within the selected frame, the method in Figure 3 returns to step 60 to select another voxel. Once all voxels in the material regions and internal cavities within the selected frame have been processed, the method in Figure 3 returns to step 58.

[0096] Steps 58 through 76 are repeated until all voxels in the material regions and internal cavities within all frames of the frame group have been processed. The voxel processing algorithm in Figure 3 terminates at step 78.

[0097] In step 80, the rendering circuit 39 draws an image using volume 72 and segmentation volume 74. Any drawing method can be used as appropriate. For example, the drawing method may include shaded volume rendering. The drawing method may include unshaded volume rendering. The drawing method may include general illumination.

[0098] An object may be divided into a concealed portion and an unconcealed portion, as described above, for example with reference to Figure 5. In the drawing of step 80, the concealed portion of the object may be drawn with higher transparency than the unconcealed portion. In another embodiment, the drawing of the concealed portion of the object may be omitted. In yet another embodiment, visual effects such as color, opacity, reflectivity, or texture may be applied to the object as appropriate. That is, drawing parameters such as color, opacity, reflectivity, or texture may be adjusted. Different visual effects may be used for the concealed portion and the unconcealed portion of the object. Different objects may be drawn using different visual effects.

[0099] The drawn image can be displayed on a screen, for example, display screen 26. The drawn image may also be displayed to a user, for example, a clinician.

[0100] In the method shown in Figure 3, frames are prioritized according to their importance. Segmentation combines data from multiple frames into a single image. Objects segmented from different frames are aligned within the composite image. Segmentation is performed based on occlusion.

[0101] Instead of compositing all frames, for example, image fusion combines frames in order of priority based on the algorithm shown in Figure 3. Voxels from multiple frames are combined into a single dataset. For each voxel, the algorithm is used to identify the frame from which the voxel value originates. Compared to methods that render multiple datasets at once, such as fusion rendering, rendering using a single dataset becomes faster and more efficient. However, in some situations, fusion can consume vast resources.

[0102] The method shown in Figure 3 uses segmentation to adjust which spaces to combine and which to leave for the next frame. To avoid unnecessary combinations, only suitable materials are selected.

[0103] Segmentation identifies which voxel belongs to which frame within a set of frames. The content of each frame is divided into segment objects.

[0104] Occlusion is addressed by using occlusion detection methods to separate the concealing material into its own segmentation hierarchy. The segmentation contains information about how objects conceal each other.

[0105] The method shown in Figure 3 provides a processing technique that can be used primarily for creating volume rendering operation sequences from time datasets. This process aims to avoid cluttered and difficult-to-view images by combining frames based on priority and adjusting segmentation to highlight areas that should be hidden.

[0106] Figure 6 shows an example of segmentation in an image drawn from an ultrasound volume. The ultrasound volume contains voxels from two frames, frame 1 and frame 2. Frame 1 has a higher priority than frame 2.

[0107] We will use a y-axis occlusion detection method where the occlusion direction is downwards in Figure 6. In Figure 6, the lower object is in front of the upper object.

[0108] Regions 120 and 122 are regions of material from frame 1, which is a higher-priority frame. Regions 120 and 122 are shaded with a first shading using a first color, such as red.

[0109] Region 124 is a region of material in frame 2 that is in front of region 120 of material in frame 1. Region 124 is shaded with a second shading using a second color, such as green. Considering the occlusion axis, region 124 is located between the observer and region 120. Region 124 is considered to be obscured and is given an occlusion level of "1".

[0110] Region 126 is another region of material in frame 2. Region 126 is shaded with a third color, such as blue, using third shading. Considering the occlusion axis, region 126 is located between the observer and region 120, but behind the foremost, high-priority region 122. In the occlusion detection method of Figure 6, a region is considered occluded if it is located between the observer and the foremost, high-priority region. Since region 126 is not located between the observer and the foremost region 122 of frame 1, it is considered not occluded and is given an occlusion level of "0".

[0111] Region 128 is yet another region of material in frame 2, located in front of a portion of region 120 and behind another portion of region 120. Region 128 is shaded the same way as region 126. Because region 128 is located in the foremost region in the observer's line of sight, i.e., behind region 120, it is not considered obscured and is given an occlusion level of "0".

[0112] Regions 124, 126, and 128 each contain material from Frame 2, but are treated as distinct objects.

[0113] Figure 6 shows an example of using the occlusion detection method to consider occlusion related to the foremost object. In another embodiment, the occlusion detection method may consider occlusion related to the rearmost object.

[0114] Adding further frames to frames 1 and 2 in Figure 6 may add other objects that obscure one or more of the regions shown in Figure 6. The region of material that obscures one or more of the regions shown in Figure 6 may be divided into multiple different objects based on the occlusion properties of each region.

[0115] Figures 7A and 7B show examples of ultrasound images drawn from three frames using the method in Figure 3. Different frames are represented in different colors. Figure 7A shows a color image version reproduced in grayscale. Figure 7B shows another version of Figure 7A, where different frames are represented using different colors and different shading effects.

[0116] In Figure 7B, the region of material from the highest priority frame is drawn in the first color and represented by a vertical line. The highest priority frame may also be called the top frame 130. In Figure 7B, the region of material from the next frame 132, which has the second highest priority, is drawn in the second color and represented by a diagonal line to the right. In Figure 7B, the region of material from the last frame 134, which has the lowest priority, is drawn in the third color and represented by a diagonal line to the left.

[0117] Images 7A and 7B show fetal movements. The arms are depicted in different colors, making their movement visible. Head movements can also be observed.

[0118] The image in Figure 8 shows how the sphere 140 deforms from left to right through the intermediate shape 142 into a cube 144. Based on the method in Figure 3, the movement between the three frames (sphere 140, intermediate shape 142, and cube 144) is combined into a single image or screen. For example, gaps are created between objects in the image according to the contour processing described with reference to step 68 in Figure 3.

[0119] Cube 144 is larger than sphere 140 and has the potential to conceal matter. Therefore, sphere 140 is defined as having the highest priority. By setting sphere 140 as the highest priority object, the entirety of sphere 140 becomes visible. Intermediate shape 142 is the second highest priority object, and cube 144 is the lowest priority object.

[0120] Opacity can be controlled based on occlusion characteristics. Using segmentation as described with reference to Figure 3, materials that conceal higher-priority objects may be removed.

[0121] Furthermore, simply combining the volumes of the sphere 140, the intermediate shape 142, and the cube 144 may result in the cube 144 in the final frame being less clear than in the other frames. In the method shown in Figure 3, the rendering circuit 39 can determine the rendering method by separating the material into new labels.

[0122] The method shown in Figure 3 makes it possible to combine the deformation of an object and the transformation from sphere 140 to cube 144 on a single screen.

[0123] Figure 9 shows the segmentation of the object in Figure 8. In Figure 8, if an object is above other objects in the drawing, it is considered to obscure the other objects. The first segment object 150 contains a voxel of a sphere 140. The second segment object 152 and the third segment object 154 contain a voxel of an intermediate shape 142. The second segment object 152 contains a voxel that obscures the first segment object 150. The third segment object 154 contains a voxel that does not obscure any of the segment objects.

[0124] The fourth segment object 156 and the fifth segment object 158 ​​contain voxels of cube 144. The fourth segment object 156 contains voxels that conceal the third segment object 154. The fifth segment object 158 ​​contains voxels that do not conceal any segment object.

[0125] In the segmentation shown in Figure 9, it can be confirmed that objects were generated to address the concealing substance using the method shown in Figure 3.

[0126] During rendering, the materials that conceal the third segment object 154 and the fifth segment object 158 ​​may be rendered with low transparency, or may be omitted entirely. In another embodiment, the concealing materials may be rendered using a different color or other image properties than the non-concealing materials in the same frame.

[0127] Figures 10A to 10F show combinations of frames containing multiple objects.

[0128] Figure 10A shows the first frame containing two objects 201 and 202. Object 201 has a void 203. Figure 10B shows the material region 205 of the first frame, which combines the materials of objects 201 and 202. Figure 10B also shows the region of the internal void 206, which includes the void 203 of object 201, and the region of the external void 217. The external void 217 is the space not concealed by objects 201 and 202.

[0129] Figure 10C shows a second frame containing two other objects 211 and 212. Figure 10D shows the material region 215 of the second frame, which combines the material of objects 211 and 212. Figure 10D also shows an external void 217, which is the space not obscured by objects 211 and 212.

[0130] Figure 10E shows objects 201, 202 and empty space 203 in the first frame and objects 211, 212 in the second frame before occlusion is determined. Objects 211, 212 in the second frame have higher priority than objects 201, 202 in the first frame.

[0131] Figure 10F shows the objects in the first and second frames after occlusion determination. The occlusion determination process divides the first object 201 into an object 223 that occludes it and an object 225 that does not occlude it, and divides the second object 202 into an object 224 that occludes it and an object 226 that does not occlude it.

[0132] When determining occlusion, the entire material region 215 of the second frame is considered. The concealing object may conceal one or both of the first and second objects 211 and 212. For example, the first part of the concealing object 224 may conceal object 211, and the second part of the concealing object 224 may conceal object 212.

[0133] However, the concealing objects 223 and 225 and the non-concealing objects 224 and 226 are each formed from one of the original objects 201 and 202. For example, the first concealing object 223 is formed from object 201, and the second concealing object 225 is formed from object 202.

[0134] The number of objects in the first frame does not need to be the same as the number of objects in subsequent frames. The number of objects in each frame is arbitrary.

[0135] In other embodiments, the data for each frame may be offset so that objects in different frames appear further apart. For example, if there is lateral movement between frames, the object may be shifted upward from that frame to further separate it. Adding the offset makes the objects appear further apart, making their individual movements easier to distinguish. For certain biological structures, the data for each frame may also be offset to create a slightly larger volume, causing the objects to appear further apart.

[0136] In the embodiments described above, a single-frame priority system was used, where one frame was the highest priority frame. In another embodiment, the priority system may be changed so that different frames take turns being the highest priority frames. For example, frames could be cycled from the future to the present, and then to the past. By cycling the frame priority system, it becomes possible to create a video in which the material emphasized changes from frame to frame, but the overall content volume is maintained.

[0137] In the embodiments described above, the frame group includes image data obtained by sequentially acquiring images such as frames of a four-dimensional image. In other embodiments, any frame group can be used as appropriate. The frame group may include frames acquired at different points in time, such as frames obtained by imaging the same patient at different times, weeks, months, or years. The frame group may also include frames obtained by imaging different patients or other subjects. Furthermore, the frame group may include image data obtained by acquiring gate images, such as gated heart rate phases synchronized with the heartbeat, or images of the phase of dynamic scan angiography showing the flow of contrast agent in blood vessels.

[0138] The method shown in Figure 3 can also be used to measure short-term differences (e.g., movement between frames in a 4D dataset) or long-term differences (e.g., disease progression between different scans).

[0139] In other embodiments, any image may be appropriately combined using the method shown in Figure 3. For example, when aligning the first image with the second image, the registration error may be shown by comparing the first and second images based on the method in Figure 3. The first image is also referred to as the target image. If the alignment is successful, there will be little difference between the images, and the image obtained by combination will look similar to the first image. If the alignment is unsuccessful, the differences between the images may be highlighted within the combined image.

[0140] In one embodiment, if the user rotates their gaze significantly, the volume may be reprocessed. For example, the reprocessing includes repeating steps 58 to 76 of the method in Figure 3 for the new viewing direction.

[0141] In the method shown in Figure 3, occlusion is based on the viewing direction. As the viewing direction changes, previously occluding voxels may become non-occluding voxels, and / or previously non-occluding voxels may become occluding voxels. Reprocessing creates an updated segmentation volume 74.

[0142] The rendering circuit 39 can draw the updated composite image using the updated segmentation volume 74. The updated composite image is an image viewed from a different viewing direction and may differ from the original composite image, including areas that are hidden.

[0143] In one embodiment, a threshold may be applied to changes in the visual direction. For example, reprocessing may be performed only when a large change exceeding the threshold occurs in the visual direction. If the change in the visual direction is less than the threshold, the rendering circuit 39 reuses the existing volume 72 and segmentation volume 74.

[0144] In another embodiment, depending on the modality of the medical image, there may be a primary viewing direction, and no change in the viewing direction may occur. In such an embodiment, the method shown in Figure 3 is performed only once.

[0145] The above-described embodiment includes determining data synthesis based on a plurality of frames, each containing medical image data such as ultrasound data. In another embodiment, any medical image data from one or more modalities can be used as appropriate. The medical image represents a human or animal subject.

[0146] In other embodiments, each frame may contain any type of data other than medical image data.

[0147] A medical image display device equipped with a processing circuit is provided based on one embodiment. The processing circuit receives a plurality of medical image time-phase data, defines parameters relating to the priority of each medical image data in the plurality of medical image time-phase data, selects one or more medical image data from the plurality of medical image time-phase data based on the parameters, and generates composite image data by combining the selected medical image data.

[0148] The composite image data may be generated by combining selected medical image data. The selected medical image data is aligned based on parameters.

[0149] A medical imaging method is provided based on one embodiment. The method comprises a temporal sequence of frames, a priority order of frames, and a set of segment objects for each frame indicating related / unrelated cellular tissues. The data represented by the segmentation of each frame is synthesized, and the contents of the combined volume segmentation are divided.

[0150] Occlusion detection methods can be used to infer from the visual direction areas where lower-priority objects obscure higher-priority objects, and these areas may be separated as a segmentation hierarchy.

[0151] Frame priority may be determined by the current frame's ranking relative to past frames, only considering the current frame and past frames.

[0152] Frame priority may be determined by the current frame's ranking relative to only the current and future frames.

[0153] Frame priority can be in the order of the current frame, past frames, and the last future frame.

[0154] Frame priority may maintain temporal distance, with the current frame and alternating past / future frames in order.

[0155] The occlusion detection method can also be a straight line of sight from the direction of visual observation.

[0156] The occlusion detection method may also be based on an angular range centered on the line of sight.

[0157] If the user rotates their gaze significantly, the volume may be reprocessed.

[0158] If the modality has a primary viewing direction, the process may be executed only once.

[0159] Features across frames can be separated even when there is no lateral movement by alternating frame offsets.

[0160] The gated heart rate phase may be shown. The dynamic scan angiography phase may be shown.

[0161] The same process may also be used to indicate alignment errors.

[0162] The display can be made into a video by using multiple processes that cycle through frames from the future to the present and then to the past.

[0163] A medical image processing apparatus is provided, comprising a processing circuit, based on one embodiment. The processing circuit receives a plurality of frame datasets, each representing a situation at a different time point in time, obtains segmentation of at least one cell tissue or other feature in at least a portion of the frames, and synthesizes the data from the plurality of frame datasets to create a single image representing a situation at multiple different time points in time. The data synthesis includes at least partially spatially separating the cell tissue or other features at different time points in the single image, and excluding at least a portion of the data in the synthesized image using segmentation.

[0164] Image synthesis to create a single image may include assigning priorities to segmentation and reducing or removing occlusion according to those priorities.

[0165] The processing circuit may set frame priorities. Segmentation acquisition may include acquiring a set of segment objects representing related / unrelated cell tissues for each frame.

[0166] Using an occlusion detection method, regions where low-priority objects obscure high-priority objects can be estimated from the visual direction, and these regions can be separated as a segmentation hierarchy.

[0167] Frame priority may include at least one of the following items: a) Only the current frame and past frames, in order from the current frame. b) Only the current frame and future frames, in order from the current frame. c) The current frame, past frames, and the last future frame. d) Maintain temporal distance between the current frame and alternating past and future frames.

[0168] The occlusion detection method may be a straight line of sight from the direction of visual observation. The occlusion detection method may also be an angular range centered on the line of sight.

[0169] The processing circuit may perform at least one of the following operations. a) Repeat segmentation and / or synthesis in response to the user's eye rotation. b) If there is a primary viewing direction, perform segmentation and / or synthesis only once. c) By alternately offsetting the frames, features across frames can be separated even when there is no lateral movement. d) Show gated cardiac phase or dynamic scan angiography phase. e) Indicates alignment error. f) Create a video display by cycling through frames from the future to the present and then to the past.

[0170] While specific circuits have been described in this specification, in other embodiments, one or more functions of these circuits can be implemented by a single processing resource or other component. Alternatively, functions implemented by a single circuit can be implemented by combining two or more processing resources or other components. A single circuit includes multiple components, whether or not they are spaced apart, that implement the functions of that circuit. Multiple circuits include single components that implement the functions of those circuits.

[0171] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. The novel methods and systems can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0172] 10. Motion sequence image 20 Medical Image Processing Equipment 22 Arithmetic unit 30 Data storage unit 24 Scanners 26 Display screen 28 Input devices 32 Processing Unit 34 Priority circuit 36 Classification circuit 38 Voxel Processing Circuit 39 Rendering Circuit

Claims

1. Receive multiple frames, each containing medical image data. The priority order of the aforementioned multiple frames is defined, Based on the aforementioned priority, a region of medical image data based on one or more frames from the plurality of frames is selected. A processing circuit generates composite image data by synthesizing composite image data from the selected region of the aforementioned medical image data, such that different regions of the composite image data include medical image data based on different frames from the plurality of frames. Equipped with, The selection of the region includes determining a group of segment objects in each of the plurality of frames, and selecting the region based on the group of segment objects. Medical image display device.

2. The aforementioned multiple frames represent data acquired at different points in time or phases of time. At least a portion of the different regions of the aforementioned composite image data represents different points in time or temporal phases. The medical image display device according to claim 1.

3. The processing circuit further draws an image from the synthesized image data. The medical image display device according to claim 1.

4. The selection of the region further includes inferring at least one concealment region in which, when viewed from the visual direction, a segment object based on a first frame among the plurality of frames conceals a segment object based on a second frame different from the first frame among the plurality of frames. The medical image display device according to claim 1.

5. The first frame has a lower priority than the second frame. The medical image display device according to claim 4.

6. The estimation of the concealed area is based on at least one of the straight line of sight from the viewing direction and the angular range around the line of sight from the viewing direction. The medical image display device according to claim 4.

7. The processing circuit further assigns an occlusion index value to each segment object, The occlusion index value is determined by whether the segment object obscures one or more other segment objects. The medical image display device according to claim 1.

8. The processing circuit further adjusts at least one drawing parameter of the concealed region based on the estimated concealed region. The medical image display device according to claim 4.

9. The aforementioned priority order is, Select the current frame and a group of past frames in order from the current frame. Select the current frame and a group of future frames in order from the current frame. Select the current frame and a group of past frames in order, and then select the final future frame. This includes at least one of the following: alternately selecting past and future frames so as to maintain a constant temporal interval from the current frame. The medical image display device according to claim 1.

10. The group of segment objects includes two or more of the following: material, internal voids, external voids, and signal deficiency locations. The medical image display device according to claim 1.

11. The processing circuit further performs reprocessing, which includes updating the concealed area estimated based on the updated visual direction in response to a change in the visual direction. The medical image display device according to claim 4.

12. The aforementioned viewing direction is the primary viewing direction of the imaging modality from which the multiple frames are acquired. The medical image display device according to claim 4.

13. A plurality of frames, each containing medical image data, are received. The priority order of the aforementioned multiple frames is defined, Based on the aforementioned priority, a region of medical image data based on one or more frames from the plurality of frames is selected. A processing circuit generates composite image data by synthesizing composite image data from the selected region of the aforementioned medical image data, such that different regions of the composite image data include medical image data based on different frames from the plurality of frames. Equipped with, The processing circuit further, An image is drawn from the aforementioned composite image data, By applying an offset to regions from different frames among the aforementioned multiple frames, the regions are visually separated within the drawn image. Medical image display device.

14. The processing circuit further aligns the positions of the multiple frames by performing rigid or non-rigid registration before combining the selected regions of the image data. The medical image display device according to claim 1.

15. The aforementioned multiple frames represent the gated heart rate phase or the dynamic scan angiography phase. The medical image display device according to claim 1.

16. The first of the aforementioned multiple frames represents the image to be aligned, The second frame among the aforementioned multiple frames represents an image for which alignment has already been performed. The aforementioned composite image data represents the alignment error. The medical image display device according to claim 1.

17. A plurality of frames, each containing medical image data, are received. The priority order of the aforementioned multiple frames is defined, Based on the aforementioned priority, a region of medical image data based on one or more frames from the plurality of frames is selected. A processing circuit generates composite image data by synthesizing composite image data from the selected region of the aforementioned medical image data, such that different regions of the composite image data include medical image data based on different frames from the plurality of frames. Equipped with, The processing circuit further, An image is drawn from the aforementioned composite image data, By applying different priorities to the aforementioned multiple frames and drawing the images, the current frame is cycled through to create a video display. Medical image display device.

18. Receiving multiple frames, each containing medical image data. To define the priority order of the aforementioned multiple frames, Based on the aforementioned priority, select a region of medical image data based on one or more frames from the plurality of frames. The method comprises generating composite image data by compositing selected regions of medical image data such that different regions of the composite image data contain medical image data based on different frames within the plurality of frames, The selection of the region includes determining a group of segment objects in each of the plurality of frames, and selecting the region based on the group of segment objects. Medical image display method.

19. Receiving a plurality of frames, each containing medical image data, To define the priority order of the aforementioned multiple frames, Based on the aforementioned priority, select a region of medical image data based on one or more frames from the plurality of frames. The method comprises generating composite image data by compositing selected regions of medical image data such that different regions of the composite image data contain medical image data based on different frames within the plurality of frames, moreover, An image is drawn from the aforementioned composite image data, By applying an offset to regions from different frames among the aforementioned multiple frames, the regions are visually separated within the drawn image. Medical image display method.

20. Receiving a plurality of frames, each containing medical image data, To define the priority order of the aforementioned multiple frames, Based on the aforementioned priority, select a region of medical image data based on one or more frames from the plurality of frames. The method comprises generating composite image data by compositing selected regions of medical image data such that different regions of the composite image data contain medical image data based on different frames within the plurality of frames, moreover, An image is drawn from the aforementioned composite image data, By applying different priorities to the aforementioned multiple frames and drawing the images, the current frame is cycled through to create a video display. Medical image display method.

Images