Image fusion method, image fusion device and storage medium

Abstract

The invention relates to an image fusion method, an image fusion device and a storage medium, and the method comprises the steps: determining a mask according to a first instruction; according to the mask, fusing the images of the multiple modes; adjusting the space parameters of the mask according to the second instruction to obtain an adjusted mask; and displaying the fused image according to the adjusted mask. Based on the method and the device, independent translation and rotation of the fusion mask can be supported, so that a user not only can transversely or longitudinally move the mask to explore a shielded area through intuitive dragging operation, but also can rotate the mask to enable the orientation of the mask to be aligned with a specific anatomical structure or a focus boundary, thereby realizing a flexible observation view angle, and improving the observation efficiency. The illness state can be determined as soon as possible.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to image fusion methods, image fusion apparatus, storage media, and program products. Background Technology

[0002] In modern precision medicine, multimodal medical image fusion has become an indispensable tool in clinical diagnosis, surgical planning, and radiotherapy. For example, it can overlay images acquired by different imaging devices (e.g., with different modalities) to provide doctors with more comprehensive patient information.

[0003] However, current image fusion technology still has some technical problems, such as poor flexibility, weak interactivity, and limited functionality, making it difficult to meet users' increasingly diverse and flexible needs for image fusion. Summary of the Invention

[0004] This disclosure provides an image fusion method, an image fusion apparatus, and a storage medium to address the shortcomings of related technologies.

[0005] According to a first aspect of the present disclosure, an image fusion method is provided, comprising: determining a mask according to a first instruction; fusing images of multiple modalities according to the mask; adjusting the spatial parameters of the mask according to a second instruction to obtain an adjusted mask; and displaying the fused image according to the adjusted mask.

[0006] According to a second aspect of the present disclosure, an image fusion apparatus is provided, comprising: a processing module configured to determine a mask according to a first instruction; fuse images of multiple modalities according to the mask; adjust the spatial parameters of the mask according to a second instruction to obtain an adjusted mask; and a display module configured to display the fused image according to the adjusted mask.

[0007] According to a third aspect of the present disclosure, a storage medium is provided that stores instructions that, when executed on a communication device, cause the communication device to perform the image fusion method described above.

[0008] According to a fourth aspect of the present disclosure, a program product is provided, which, when executed by a communication device, causes the communication device to perform the image fusion method.

[0009] In the embodiments of this disclosure, the mask units of the mask can be set based on physical dimensions. On the one hand, since physical dimensions are more intuitive for users to understand than reference dimensions, it is easier for users to select appropriate physical dimensions based on experience to determine the mask units of the mask. On the other hand, when the user adjusts the mask units, the change in the size of the mask units can be reflected based on the change in physical dimensions. Compared with adjusting the size of the mask units based on the image matrix, the change in the size of the mask units is more continuous and closer to a stepless change, which is beneficial to improving the user experience.

[0010] In the embodiments of this disclosure, users can form a custom closed curve by inputting a first instruction, and then determine a mask based on the closed curve, thereby realizing the shape of the user-defined mask. Accordingly, users can input any desired shape of closed curve as needed to accurately fit the complex human body structure shape, which is beneficial for fusion images to facilitate users to observe complex and irregular lesion areas.

[0011] In the embodiments of this disclosure, users can independently control the rotation and translation of the mask using only a single finger (e.g., touch operation) or a single key (e.g., clicking a mouse button to drag), without switching tools or using complex shortcut keys, significantly improving the smoothness of operation and clinical applicability in the multimodal image fusion process. Furthermore, both translation and rotation amounts can be mapped to a physical coordinate system (e.g., in millimeters), facilitating the ensuring anatomical scale consistency of the interactive results.

[0012] In the embodiments of this disclosure, users can adjust spatial parameters such as translation position and rotation angle of regular-shaped masks, such as checkerboard masks, horizontal stripe masks, and vertical stripe masks, and display the fused image in real time based on the user-adjusted mask. Unlike static checkerboard masks or those that only support overall translation in related technologies, this disclosure supports independent translation and rotation of the fused mask. This allows users to intuitively drag and drop the mask to explore obscured areas, and also rotate the mask to align its orientation with specific anatomical structures or lesion boundaries, thereby achieving an unprecedentedly flexible observation perspective.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0015] Figure 1 This is a schematic flowchart illustrating an image fusion method according to an embodiment of the present disclosure.

[0016] Figures 2A to 2C This is a schematic diagram of a mask unit shown according to an embodiment of the present disclosure.

[0017] Figure 3A This is a schematic diagram of a modal image according to an embodiment of the present disclosure.

[0018] Figure 3B This is a schematic diagram of another modal image shown according to an embodiment of the present disclosure.

[0019] Figure 4A This is a schematic diagram illustrating a fused image according to an embodiment of the present disclosure.

[0020] Figure 4B This is a schematic diagram illustrating another fused image according to an embodiment of the present disclosure.

[0021] Figure 4C This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0022] Figure 5A This is a schematic diagram of a checkerboard mask according to an embodiment of the present disclosure.

[0023] Figure 5B This is a schematic diagram of a horizontal stripe mask according to an embodiment of the present disclosure.

[0024] Figure 5C This is a schematic diagram of a vertical stripe mask according to an embodiment of the present disclosure.

[0025] Figure 6 This is a schematic diagram of yet another mask shown according to an embodiment of the present disclosure.

[0026] Figure 7 This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0027] Figure 8 This is a schematic diagram of an operating area according to an embodiment of the present disclosure.

[0028] Figure 9A This is a schematic diagram illustrating a fused image according to an embodiment of the present disclosure.

[0029] Figure 9B This is a schematic diagram illustrating another fused image according to an embodiment of the present disclosure.

[0030] Figure 9C This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0031] Figure 9DThis is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0032] Figure 10 This is a schematic diagram of a system framework according to an embodiment of the present disclosure.

[0033] Figure 11 This is a schematic flowchart illustrating another image fusion method according to an embodiment of the present disclosure.

[0034] Figure 12 This is a block diagram of an image fusion apparatus according to an exemplary embodiment. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0036] In some embodiments, multimodal image fusion technology can overlay and display images of different modalities acquired by different imaging devices to provide doctors with more comprehensive patient information.

[0037] In some embodiments, the multimodal image fusion method may include at least one of the following: Alpha fusion (also known as transparency blending): blending images by adjusting the transparency of different modalities; the disadvantage is that it can easily cause image confusion and make it difficult to distinguish which modal the information comes from. Checkerboard fusion: The screen is divided into alternating checkerboard areas to display images from different modalities; its advantage is that it can relatively clearly distinguish the source of information, and it is one of the more commonly used image fusion methods in clinical practice. Stripe fusion: such as horizontal stripe fusion or vertical stripe fusion, the principle is similar to checkerboard fusion, and it is suitable for observing structural changes in a specific direction (such as horizontal or vertical).

[0038] To improve diagnostic efficiency, the need to allow users to dynamically adjust fused images to observe different regions has become crucial. However, some medical imaging software (including open-source software, commercial workstations, and treatment planning systems) has significant limitations in interactive fusion displays, particularly in checkerboard and stripe fusion.

[0039] In some embodiments, taking open-source software as an example: For example, 3D Slicer: As one of the world's most popular open-source medical image computing platforms, 3DSlicer provides powerful multimodal image registration and fusion capabilities. Its fusion module supports Alpha fusion and checkerboard fusion. However, the grid width of its checkerboard fusion is fixed, usually determined by the image matrix size (e.g., 8×8, 16×16), and cannot be customized in physical dimensions (e.g., millimeters). Furthermore, the checkerboard pattern is static and does not support users dynamically adjusting the position and angle of the moving checkerboard, which greatly limits doctors' continuous observation of regions of interest.

[0040] For example, MITK (The Medical Imaging Interaction Toolkit): MITK is another important open-source medical imaging integration platform used in research and commercial product development. It also provides basic image fusion and display functions, including a checkerboard mode. However, similar to 3D Slicer, MITK's checkerboard fusion is typically static in implementation. The size of its checkerboard is bound to the image's inherent grid, does not support custom widths in millimeters, and lacks the ability to interactively move the checkerboard interface. Users who want to observe areas obscured by the grid lines must switch back to other display modes or re-segment the image, a cumbersome process.

[0041] For example, ITK-SNAP (Insight Segmentation and Registration Toolkit - Snake Automatic Partitioning): ITK-SNAP is known for its powerful image segmentation capabilities, and it also supports side-by-side and fused display of multimodal images. However, ITK-SNAP's fusion function is relatively basic, mainly focusing on alpha fusion, lacking checkerboard or stripe fusion capabilities, and does not support interactive movement of these patterns.

[0042] In some embodiments, taking a commercial imaging workstation and TPS system software as an example: For example, Philips IntelliSpace Portal: Philips' high-end image post-processing platform offers a wealth of post-processing and analysis tools, but it only has a transparency blending mode, and its blending function is limited.

[0043] For example, GE AW Server (General Medical Post-Processing Workstation): GE's commercial workstations also integrate multimodal image fusion capabilities, but do not support checkerboard fusion mode, let alone interaction based on it.

[0044] For example, Siemens syngo.via (Siemens team research platform): Siemens' platform is known for its extensive use in radiology, providing advanced image fusion and visualization solutions, but it also does not integrate checkerboard fusion functionality, nor does it support interaction on this basis.

[0045] In some embodiments, mainstream treatment planning systems (TPS) are used as examples, such as Varian Eclipse, Elekta Monaco, and Philips Pinnacle.

[0046] In mainstream TPS systems, image fusion functionality generally suffers from insufficient interactive flexibility. Specifically, most systems do not support checkerboard fusion; for example, Varian Eclipse only offers rectangular frame fusion and crosshair fusion as alternatives. While some systems, such as Elekta Monaco and RayStation, do have checkerboard fusion modes, their functionality is significantly limited: for instance, Monaco only supports fixed grid sizes (such as 2×2 or 4×4) and can only perform overall translation, unable to achieve arbitrary size settings or free adjustment of grid line positions; RayStation also does not support customizable checkerboard size or real-time movement.

[0047] It is evident that mainstream TPS systems cannot provide a truly flexible and interactive checkerboard fusion experience in terms of fusion display, making it difficult to meet the clinical needs for precise verification of specific areas.

[0048] Based on the above analysis, the limitations of current image fusion technology can be summarized as follows: Poor flexibility: Checkerboard fusion and stripe fusion typically have fixed granularity (e.g., checkerboard width, stripe spacing) or only a limited number of alternative levels, and are generally tied to image pixels. This prevents users from steplessly and arbitrarily setting the granularity in real physical dimensions (e.g., millimeters), making it difficult to adapt to the observation needs of anatomical structures at different scales.

[0049] Weak interactivity: In most systems, the fused images (checkerboard, stripes) are static, and users cannot move the image position in real time through intuitive drag-and-drop operations while maintaining the fusion mode. This makes it very inconvenient to examine critical areas obscured by pattern boundaries (e.g., when fused from MRI and PET images, where the MRI image is partially obscured by the PET image, and vice versa), requiring repeated switching of display modes or resetting, disrupting the continuity of diagnosis.

[0050] Limited functionality: The lack of a universal mechanism prevents users from defining the fusion display method based on regions of interest of arbitrary shapes (such as irregular tumors or specific organs). Regular patterns (e.g., checkerboard, stripes) are difficult to accurately conform to complex human anatomy, limiting the potential of fusion displays in precise diagnosis and surgical planning. This makes the aforementioned image fusion tools inadequate for observing complex, irregular lesion areas.

[0051] Therefore, the technical problems that this disclosure aims to solve include at least overcoming the shortcomings of the aforementioned image fusion techniques in terms of flexibility, interactivity, and functionality. This disclosure can provide an image fusion method capable of supporting movable checkerboard patterns and stripes of arbitrary width (mm), and further extended to support user-defined fusion regions of arbitrary shapes.

[0052] Figure 1 This is a schematic flowchart illustrating an image fusion method according to an embodiment of the present disclosure.

[0053] In some embodiments, the image fusion method may be performed by a display device.

[0054] For example, the display device can be used to receive instructions (e.g., a first instruction, a second instruction) input by the user, and to execute any step in the embodiments of this disclosure based on the instructions input by the user, and can also be used to display the fused image.

[0055] For example, display devices include at least one of the following: mobile phones, personal computers, tablet computers, televisions, and medical devices with display functions.

[0056] like Figure 1 As shown, the image fusion method may include the following steps: In step S101, a mask (also known as a mask) is determined according to the first instruction. In step S102, images of multiple modalities are fused according to the mask; In step S103, the spatial parameters of the mask are adjusted according to the second instruction to obtain the adjusted mask; In step S104, the fused image is displayed according to the adjusted mask.

[0057] It should be noted that this disclosure does not restrict the execution order of steps S102 and S103 above; for example, it can be performed as follows: Figure 1 As shown, step S103 is executed after step S102, or step S103 may be executed before step S102.

[0058] Where step S103 can be performed before step S102, step S102 can be omitted, that is, only steps S101, S103 and S104 need to be performed.

[0059] In addition, steps S103 and S104 can be executed repeatedly. For example, if the user can adjust the spatial parameters of the mask multiple times as needed, the fused image can be displayed in real time based on the adjusted mask each time.

[0060] In some embodiments, a user may input a first command and a second command to a display device. The input method of the first command and the second command is not limited in this disclosure. For example, they may be input through physical buttons, voice input, touch operation, or other methods.

[0061] In some embodiments, the granularity of the mask is determined based on the resolution of the image.

[0062] For example, the resolution of the images to be fused (or images to be fused) can be determined first, and then the smallest unit of the image, called a voxel, can be determined based on the resolution. The granularity of the mask can have the same size as the voxel (e.g., the projection of a voxel in a two-dimensional plane).

[0063] In some embodiments, the mask type includes at least one of the following: checkerboard mask; horizontal stripe mask; vertical stripe mask; custom shape mask.

[0064] In some embodiments, when the mask type includes at least one of a checkerboard mask, a horizontal stripe mask, and a vertical stripe mask, determining the mask according to the first instruction further includes: determining the size of the mask unit according to the first instruction, wherein the size of the mask unit is greater than or equal to the size of the granularity; and determining the value of the mask unit according to the relationship between the size of the mask unit and the coordinates of the mask unit.

[0065] In some embodiments, the size of the mask unit is characterized based on at least one of the following: physical size; the number of granularities.

[0066] For example, physical dimensions may include at least one of the following: meter, decimeter, centimeter, millimeter, micrometer.

[0067] For example, the number of granularities can be a predetermined (e.g., predefined) number or a number set by the user.

[0068] In some embodiments, the type of granularity value or mask unit value in the mask is the same as the type of mode. Accordingly, in a given mask, different granularity values ​​or mask unit positions can display images of different modes, so as to achieve the fusion of multiple modal images.

[0069] In some embodiments, the number of types of the multiple modalities is greater than or equal to two.

[0070] For example, a mask is used to fuse images of N modalities (e.g., the image of the first modality, the image of the second modality, ..., the image of the Nth modality). The values ​​of the mask units can be N, for example, values ​​from 0 to N-1.

[0071] For example, when N=2, the values ​​of the mask units can include 0 and 1.

[0072] The following embodiments mainly take N=2 as an example, that is, the fusion of two modal images, to illustrate the technical solution of this disclosure.

[0073] In some embodiments, the modality includes at least one of the following: Computed Tomography (CT). Magnetic Resonance Imaging (MRI). Positron emission tomography (PET). Single Photon Emission Computed Tomography (SPECT).

[0074] It should be noted that the modalities of the images involved in this disclosure are not limited to the above-mentioned types, and may also include other types of modalities, such as T1-weighted MRI images and T2-weighted MRI images.

[0075] In some embodiments, the checkerboard mask corresponds to a checkerboard pattern in the screen; and / or, the horizontal stripe mask corresponds to a horizontal stripe in the screen; and / or, the vertical stripe mask corresponds to a vertical stripe in the screen.

[0076] For example, taking a checkerboard mask as an example, the mask unit is square, and the size of the mask unit can refer to the length or width of the checkerboard. The size of the mask unit can be equal to the size of one or more granularities.

[0077] For example, taking a mask including a horizontal stripe mask as an example, the size of the mask unit can refer to the width of the horizontal stripe, and the size of the mask unit can be equal to the size of one or more granularities.

[0078] For example, taking a mask including a vertical stripe mask as an example, the size of the mask unit can refer to the width of the vertical stripe, and the size of the mask unit can be equal to the size of one or more granularities.

[0079] Figures 2A to 2C This is a schematic diagram of a mask unit shown according to an embodiment of the present disclosure.

[0080] For example, when the mask type is a checkerboard mask with a grain size of 8 millimeters (mm), the shape of the checkerboard mask can be as follows: Figure 2A As shown, it includes a 10-row × 10-column checkerboard pattern. The checkerboard pattern is square, and each checkerboard pattern contains 4 granules. Each granule is 8 mm long and 8 mm wide, and each checkerboard pattern is 16 mm long and 16 mm wide.

[0081] For example, when the mask type is a horizontal stripe mask and the mask grain size is 8 millimeters (mm), the shape of the horizontal stripe mask can be as follows: Figure 2B As shown, this includes 10 horizontal stripes. The width of each stripe is determined based on two granularities. Each granularity has a length and width of 8 mm, so the width of one horizontal stripe is 16 mm. The length of the horizontal stripes can be predefined (e.g., equal to the horizontal dimension of the screen or equal to the horizontal dimension of the image to be merged), or it can be set by the user; this disclosure does not limit this.

[0082] For example, when the mask type is a vertical stripe mask and the mask grain size is 8 millimeters (mm), the shape of the vertical stripe mask can be as follows: Figure 2C As shown, this includes 10 vertical stripes. The width of each stripe is determined based on two granularities. Each granularity has a length and width of 8 mm, so the width of one vertical stripe is 16 mm. The length of the vertical stripes can be predefined (e.g., equal to the screen's vertical dimension or equal to the vertical dimension of the image to be merged), or it can be set by the user; this disclosure does not limit this.

[0083] It should be noted that, Figures 2A to 2C The masks shown are just a few examples of this disclosure, and the masks are not limited to those described above. Figures 2A to 2C As shown, for example, in some embodiments, the mask unit may include only one granularity. In this case, the mask unit and the granularity are equivalent, and the size of the mask unit can be determined directly based on the granularity.

[0084] In the embodiments of this disclosure, the mask units of the mask can be set based on physical dimensions. On the one hand, since physical dimensions are more intuitive for users to understand than reference dimensions, it is easier for users to select appropriate physical dimensions based on experience to determine the mask units of the mask. On the other hand, when the user adjusts the mask units, the change in the size of the mask units can be reflected based on the change in physical dimensions. Compared with adjusting the size of the mask units based on the image matrix, the change in the size of the mask units is more continuous and closer to a stepless change, which is beneficial to improving the user experience.

[0085] The following examples illustrate, through several embodiments, the fusion of multiple modalities of images based on a mask.

[0086] In some embodiments, fusing images of multiple modalities according to the mask includes: determining an image of a modality corresponding to a certain granularity based on the granularity value in the mask; determining an image unit located at the coordinates of the granularity in the image of the corresponding modality; and displaying the image unit at the granularity-corresponding position.

[0087] Figure 3A This is a schematic diagram of a modal image according to an embodiment of the present disclosure. Figure 3B This is a schematic diagram of another modal image shown according to an embodiment of the present disclosure.

[0088] like Figure 3A As shown, an image of one modality (e.g., referred to as the first modality image) is a T1-weighted MRI image; as Figure 3B As shown, another modality of image (e.g., referred to as the second modality image) is a T2-weighted MRI image. The following examples primarily focus on... Figure 3A and Figure 3B Taking the fusion of two modalities of images as an example, the technical solution of this disclosure is illustrated by way of example.

[0089] It should be noted that the first modal image (e.g., a T1-weighted MRI image) and the second modal image (e.g., a T2-weighted MRI image) shown in the subsequent embodiments can be images with the same shape and size. However, if the first modal image and the second modal image have different sizes, the smaller image can be resampled to adjust its resolution so that it has the same resolution as the larger image.

[0090] In some embodiments, the mask and the image to be fused may be aligned, for example, the shape of the mask is the same as the shape of the image to be fused, and the size of the mask is the same as the size of the image to be fused.

[0091] In some embodiments, for example, the fused image can be denoted as fused, the first modal image can be denoted as A, the second modal image can be denoted as B, and the mask is denoted as Mask, whose value is the value corresponding to a certain mask unit or a certain granularity in the mask unit. Mask(x,y) is used to characterize the value of the mask unit or granularity located at position (x,y), where (x,y) represents the coordinates of the mask unit or the coordinates of the granularity.

[0092] For example, the fused image can be determined based on the following formula: (Formula 1); First, an empty merged image of the same size as the images to be merged can be output.

[0093] Then, for the fused image of the empty space, mask unit-level assignment or granular-level assignment can be performed.

[0094] For example, if the value of Mask(x,y) is 1, the empty fused image displays the image unit located at position (x,y) in the first modality image, for example, denoted as A(x,y); For example, if the value of Mask(x,y) is 0, the empty fused image displays the image unit located at position (x,y) in the second modality image, for example, denoted as B(x,y).

[0095] Accordingly, the fused image can display the content of the corresponding modal image at each location, thereby achieving the fusion of multiple modal images.

[0096] It should be noted that before fusing the images, the mask can be aligned with the images to be fused, and the image units in the images to be fused can be determined based on granularity. For example, in a mask such as... Figure 2A In the case shown, the image to be fused can contain 10×10 image units with a side length of 8 mm, which facilitates the subsequent determination of the image unit located at (x,y) in the image to be fused.

[0097] Figure 4A This is a schematic diagram illustrating a fused image according to an embodiment of the present disclosure. Figure 4B This is a schematic diagram illustrating another fused image according to an embodiment of the present disclosure. Figure 4C This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0098] For example, in the case of a checkerboard mask, based on the mask pair Figure 3A and Figure 3B The image obtained by fusing the images in the image can be as follows: Figure 4A As shown.

[0099] For example, in the case of a horizontal stripe mask, based on the mask pair Figure 3A and Figure 3B The image obtained by fusing the images in the image can be as follows: Figure 4B As shown.

[0100] For example, in the case of a vertical stripe mask, based on the mask pair Figure 3A and Figure 3B The image obtained by fusing the images in the image can be as follows: Figure 4C As shown.

[0101] The following examples illustrate how to determine the values ​​of mask units in a mask.

[0102] For example, the value of the granularity at position (x,y) is denoted as Mask(x,y), and the size of the mask unit is denoted as w (e.g., in millimeters).

[0103] When the mask type is a checkerboard mask, Mask(x,y) can be calculated based on the following formula: (Formula 2); The floor function represents rounding down.

[0104] Therefore, it can be ensured that the mask units (chessboard grid) with values ​​of 1 and 0 are alternately distributed with an interval of w in both the horizontal and vertical directions.

[0105] Figure 5A This is a schematic diagram of a checkerboard mask according to an embodiment of the present disclosure.

[0106] For example, in Figure 2A Based on the mask shown, the value of each granularity can be determined, thereby determining the values ​​of the mask unit containing four granularities. For example, let the size of the mask unit be denoted as w. Figure 2A The four granularities p1 (located in the first row, first column), p2 (located in the second row, first column), p3 (located in the first row, second column), and p4 (located in the second row, second column) contained in the mask unit in the upper left corner are denoted as (0,0), (0,8), (8,0), and (8,8) respectively. Based on Formula 2, it can be determined that the values ​​of these four granularities p1, p2, p3, and p4 are all 0. Therefore, the value of the mask unit containing these four granularities is 0, resulting in a mask as shown in Figure 2. Figure 5A As shown, each mask unit is alternately distributed at intervals of w (2 grains, or 16 millimeters).

[0107] When the mask type is a horizontal stripe mask, Mask(x,y) can be calculated based on the following formula: (Formula 3); The floor function represents rounding down.

[0108] Therefore, it can be ensured that mask units (horizontal stripes) with values ​​of 1 and 0 are alternately distributed in the longitudinal direction at intervals of w.

[0109] Figure 5B This is a schematic diagram of a horizontal stripe mask according to an embodiment of the present disclosure.

[0110] For example, in Figure 2B Based on the mask shown, the value of each granularity can be determined, and thus the values ​​of mask units containing multiple granularities can be determined. For example, taking the size of the mask unit as w, based on Formula 3, the mask can be determined as follows: Figure 5B As shown, each mask unit is alternately distributed longitudinally at intervals of w (2 grains, or 16 millimeters).

[0111] When the mask type is a vertical stripe mask, Mask(x,y) can be calculated based on the following formula: (Formula 4); The floor function represents rounding down.

[0112] Therefore, it can be ensured that mask units (vertical stripes) with values ​​of 1 and 0 are alternately distributed in the horizontal direction at intervals of w.

[0113] Figure 5C This is a schematic diagram of a vertical stripe mask according to an embodiment of the present disclosure.

[0114] For example, in Figure 2C Based on the mask shown, the value of each granularity can be determined, and thus the values ​​of mask units containing multiple granularities can be determined. For example, taking the size of the mask unit as w, based on Formula 4, the mask size can be determined as follows: Figure 5C As shown, each mask unit is alternately distributed laterally at intervals of w (2 grains, or 16 millimeters).

[0115] It should be noted that the above method of determining the corresponding values ​​of mask units and / or granularity based on formulas is only a partial example of this disclosure. The method of determining the corresponding values ​​of mask units and / or granularity in this disclosure is not limited to the above examples. For example, the user can manually set the corresponding values ​​of each mask unit and / or granularity as needed.

[0116] In some embodiments, when the type of the mask includes a custom shape mask, determining the mask according to the first instruction further includes: determining a closure curve according to the first instruction; and determining a value corresponding to the granularity according to the relationship between the granularity and the closure curve.

[0117] For example, if the screen supports touch functionality, the first instruction may include instructions generated by the user based on touch operations, that is, the user may draw a closed curve based on touch operations.

[0118] For example, a user can also draw a closed curve by dragging the mouse, and the first instruction includes the instruction generated by the user dragging the mouse.

[0119] For example, a user can also draw a closed curve by entering code to draw the graph; the first instruction includes the code for drawing the image entered by the user.

[0120] In the embodiments of this disclosure, users can form a custom closed curve by inputting a first instruction, and then determine a mask based on the closed curve, thereby realizing the shape of the user-defined mask. Accordingly, users can input any desired shape of closed curve as needed to accurately fit the complex human body structure shape, which is beneficial for fusion images to facilitate users to observe complex and irregular lesion areas.

[0121] It should be noted that the closed curve is not limited to being drawn by the user in real time. For example, the closed curve can also be drawn and stored by the user in advance. The user can then select the closed curve to use for the mask from the pre-stored closed curves as needed. For example, the closed curve can be a closed curve with a predefined shape, such as a closed curve corresponding to the shape of the liver, the shape of the heart, the shape of the lung, etc. The user can select the closed curve to use for the mask from the predefined closed curves as needed.

[0122] In some embodiments, determining the value corresponding to the granularity based on the relationship between the granularity and the closed curve may include any of the following methods.

[0123] For example, for a particle at position (x,y), a ray can be generated from a particle at position (0,y), with the ray pointing towards position (x,y). Of course, the ray can be parallel to the horizontal direction (e.g., denoted as the x-axis), parallel to the vertical direction (e.g., denoted as the y-axis), or any other direction.

[0124] This allows us to determine the number of intersections between the ray and the closed curve. For example, if the number of intersections is odd (e.g., 1), we can determine that the granularity at position (x,y) is inside the closed curve, and the corresponding value can be recorded as 1. If the number of intersections is even (e.g., 0, 2), we can determine that the granularity at position (x,y) is outside the closed curve, and the corresponding value can be recorded as 0.

[0125] For example, for a granularity located at (x,y), the proportion of the area inside the closed curve can be calculated. If the proportion of the area is greater than a threshold (e.g., 50%), the granularity is determined to be inside the closed curve; if the proportion of the area is less than the threshold (e.g., 50%), the granularity is determined to be outside the closed curve; if the proportion of the area is equal to the threshold (e.g., 50%), the determination of whether the granularity is inside or outside the closed curve can be set based on user needs.

[0126] Figure 6 This is a schematic diagram of yet another mask shown according to an embodiment of the present disclosure.

[0127] For example, a user can enter Figure 6 The shape shown is a closed curve. After determining the relationship between each particle size and the closed curve, the corresponding value for each particle size can be determined. For example, the particle size inside the closed curve corresponds to a value of 1, and the particle size outside the closed curve corresponds to a value of 0.

[0128] Figure 7 This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0129] For example, based on Formula 1 above, we can determine which modality of image needs to be displayed at position (x, y), for example, in Figure 6 Based on the mask shown, it can be determined that the granularity within the closed curve needs to display the second modal image, while the granularity outside the closed curve needs to display the first modal image. The fused image can... Figure 7 As shown: The images inside the closed curve are T2-weighted MRI images, and the images outside the closed curve are T1-weighted MRI images.

[0130] The following examples illustrate how adjusting the spatial parameters of a mask according to a second instruction to obtain an adjusted mask.

[0131] In some embodiments, the spatial parameters include at least one of the following: translational position; rotation angle.

[0132] For example, the second instruction can be used to translate the mask, thereby adjusting the translation position of the mask, such as translating it in any direction within a plane.

[0133] For example, the second instruction can be used to rotate the mask, thereby adjusting the rotation angle of the mask, such as rotating the mask by any angle in a clockwise or counterclockwise direction.

[0134] In some embodiments, the screen for displaying images may include two areas, such as a translation area and a rotation area. When a user drags the mouse or performs a swipe touch operation in the translation area, the mask can be translated; when a user drags the mouse or performs a swipe touch operation in the rotation area, the mask can be rotated.

[0135] Figure 8 This is a schematic diagram of an operating area according to an embodiment of the present disclosure.

[0136] like Figure 8 As shown, the translation area is located near the center of the screen, and the rotation area is located near the edge of the screen. Taking the example of a user controlling the translation and rotation of the mask by dragging the mouse: When a user places the mouse in the rotation area and drags it, the processor on the screen can calculate in real time the change in angle between the current position of the mouse and the center of the screen (or the center of the image or the center of the mask), and use this angle difference as the current rotation amount, for example, denoted as Δθ.

[0137] When a user places the mouse in the panning area and drags it, the processor on the screen can determine the mouse's two-dimensional displacement vector in real time, for example, denoted as (Δx, Δy), and directly map this vector to the translation amount of the mask.

[0138] In the embodiments of this disclosure, users can independently control the rotation and translation of the mask using only a single finger (e.g., touch operation) or a single key (e.g., clicking a mouse button to drag), without switching tools or using complex shortcut keys, significantly improving the smoothness of operation and clinical applicability in the multimodal image fusion process. Furthermore, both translation and rotation amounts can be mapped to a physical coordinate system (e.g., in millimeters), facilitating the ensuring anatomical scale consistency of the interactive results.

[0139] In some embodiments, when the mask type includes at least one of checkerboard mask, horizontal stripe mask, and vertical stripe mask, adjusting the spatial parameters of the mask according to the second instruction to obtain the adjusted mask includes: performing an inverse operation on the coordinates of the granularity in the adjusted mask according to the adjustment method of the spatial parameters to obtain the original coordinates; and determining the value corresponding to the granularity in the adjusted mask according to the relationship between the granularity and the original coordinates.

[0140] For example, when adjusting the spatial parameters of the mask according to the second instruction, the changes in the spatial parameters can be recorded. For example, the change in translation position can be represented by vector T, and the change in rotation angle can be represented by rotation matrix R.

[0141] For the granularity P in the adjusted mask, for example, the corresponding coordinates (x, y), the coordinates (x, y) can be inversely operated on using the spatial parameter adjustment method. For example, it can be implemented based on the following formula: (Formula 5); Where P represents the coordinates (x, y), P origin R represents the original coordinates corresponding to granularity P, and R-1 represents the inverse matrix of matrix R.

[0142] Furthermore, the value corresponding to the granularity in the adjusted mask can be determined based on the relationship between the granularity and the original coordinates. For example, in the case of a checkerboard mask, P can be... origin Substituting into Formula 2 above, the resulting value can be used as the value corresponding to the granularity P; for example, in the case of a horizontal stripe mask, P can be... origin Substituting into Formula 3 above, the resulting value can be used as the value corresponding to the granularity P; for example, in the case of a vertical stripe mask, P can be... origin Substituting into Formula 4 above, the resulting value can be used as the value corresponding to granularity P.

[0143] In the embodiments of this disclosure, P, P origin T and R can be defined in the real number space, so that regardless of whether Porigin is outside the range of the original image (e.g., the mask before adjusting the spatial parameters based on the second instruction), it can correctly return the value corresponding to the granularity P, supporting infinite extension and seamless connection.

[0144] Furthermore, in some embodiments, after adjusting the spatial parameters of the mask according to the second instruction to obtain the adjusted mask, the fused image can be displayed according to the adjusted mask. For example, the image mode to be displayed for each particle can be determined according to the value corresponding to each particle in the adjusted mask.

[0145] For example, for granularity P, the corresponding coordinates in the adjusted mask are (x, y), and the original coordinates Porigin corresponding to P can be denoted as (x0, y0). Taking a checkerboard mask as an example, (x0, y0) can be substituted into Formula 2 above, and the resulting value can be used as the value corresponding to granularity P. For example, if the resulting value is 0, it can be determined that the first modality image needs to be displayed. Then, the adjusted mask can be aligned with the first modality image, and the image unit with coordinates (x, y) obtained from the first modality image can be used as the content displayed at position (x, y) in the fused image.

[0146] Figure 9A This is a schematic diagram illustrating a fused image according to an embodiment of the present disclosure. Figure 9B This is a schematic diagram illustrating another fused image according to an embodiment of the present disclosure. Figure 9CThis is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0147] For example, in the case of a checkerboard mask, it is possible to... Figure 4A The checkerboard mask is shifted 15 mm and rotated 45 degrees clockwise, and the fused image is then displayed based on the adjusted mask. Figure 9A As shown.

[0148] For example, in the case of a horizontal stripe mask, it is possible to... Figure 4B The checkerboard mask is shifted 15 mm and rotated 15 degrees clockwise, and the fused image is then displayed based on the adjusted mask. Figure 9B As shown.

[0149] For example, in the case of a vertical stripe mask, it is possible to... Figure 4C The checkerboard mask is shifted 15 mm and rotated 15 degrees clockwise, and the fused image is then displayed based on the adjusted mask. Figure 9C As shown.

[0150] In the embodiments of this disclosure, users can adjust spatial parameters such as translation position and rotation angle of regular-shaped masks such as checkerboard masks, horizontal stripe masks, and vertical stripe masks, and display the fused image in real time based on the user-adjusted mask. Unlike the static or only overall translation-supporting checkerboard patterns in related technologies, this disclosure supports independent translation and rotation of the fused mask. This allows users to intuitively drag and drop the mask to explore obscured areas (for example, if a certain position has a value of 0 before mask adjustment, i.e., the first modality image is displayed, then the second modality image at that position is obscured by the first modality image; when the value of that position is 1 after mask adjustment, the second modality image can be displayed), and also rotate the mask to align its orientation with specific anatomical structures or lesion boundaries, thereby achieving a flexible observation perspective and facilitating rapid diagnosis of the condition.

[0151] In some embodiments, when the mask type includes a custom shape mask, adjusting the spatial parameters of the mask according to the second instruction to obtain an adjusted mask includes: adjusting the points on the closed curve according to the adjustment method of the spatial parameters to obtain an adjusted closed curve; and determining the value of the grain size in the adjusted mask according to the relationship between the grain size and the adjusted closed curve.

[0152] In some embodiments, when the mask type includes a custom shape mask, the closed curve can be adjusted according to the second instruction to obtain the adjusted mask.

[0153] For example, a closed curve can be composed of multiple points (which may be called control points), and each control point can be moved according to a second instruction.

[0154] For example, you can first translate the control point and then rotate it, or rotate it first and then translate it; for example, the reference point for the rotation operation can be the center of the screen, or the center of the closed figure formed by the closed curve.

[0155] Accordingly, the position and / or angle of the closed curve in the mask are adjusted, and the value corresponding to the granularity can be determined based on the relationship between the granularity and the adjusted closed curve, thereby obtaining the adjusted mask. For a specific implementation example, please refer to the previous example on determining the granularity value based on the relationship between the granularity and the closed curve; it will not be repeated here.

[0156] Furthermore, in some embodiments, after adjusting the spatial parameters of the mask according to the second instruction to obtain the adjusted mask, the fused image can be displayed according to the adjusted mask. For example, the image mode to be displayed for each particle can be determined according to the value corresponding to each particle in the adjusted mask.

[0157] For example, for granularity P, the corresponding coordinates in the adjusted mask are (x, y). For example, if it is located inside a closed curve, the corresponding value is 1. It can be determined that the second modal image needs to be displayed. Then, the image unit with coordinates (x, y) in the first modal image can be obtained as the content displayed at position (x, y) in the fused image.

[0158] Figure 9D This is a schematic diagram illustrating yet another fused image according to an embodiment of the present disclosure.

[0159] For example, when the mask type is a custom shape mask, it is possible to... Figure 7 The custom-shaped mask is translated by 30 mm and rotated 60 degrees clockwise (e.g., with the screen center as the reference point for rotation), and then the blended image is displayed based on the adjusted mask, as shown below. Figure 9D As shown.

[0160] In the embodiments of this disclosure, users can adjust spatial parameters such as the translation position and rotation angle of a custom shape mask, and the fused image can be displayed in real time based on the user-adjusted mask. This facilitates circumferential, multi-angle exploration of irregular lesions or key anatomical structures. Especially when observing complex, asymmetric structures, its flexibility and accuracy far surpass image fusion techniques in related technologies.

[0161] The following examples illustrate the implementation of this disclosure.

[0162] In some embodiments, this disclosure also proposes an image fusion system for performing the image fusion method described in any of the above embodiments. For example, the image fusion system may include the following three modules: User interaction module: Used to receive user input instructions, such as first instruction, second instruction, etc., and can also select the mask type based on the user input instructions.

[0163] Core processing module: Used for generating masks, adjusting masks, and performing related calculations for image fusion.

[0164] Data and Display Module: Used to manage source image data (such as images of multiple modalities) and render the fusion results to the display screen in real time.

[0165] Figure 10 This is a schematic diagram of a system framework according to an embodiment of the present disclosure.

[0166] like Figure 10 As shown, for example, the user interaction module can receive user input instructions to determine whether the user selects a standard type mask (e.g., a checkerboard mask, a horizontal stripe mask, a vertical stripe mask) or a custom shape mask. It can also determine the size of the mask unit, such as 16 mm (2 grains) as shown in the previous embodiment, or any physical size or any number of grains.

[0167] When the user selects a standard type of mask, the mask unit can be determined based on the user's input instructions. For example, a mask unit may include at least one granularity.

[0168] When the user selects a custom shape mask, the closed curve can be determined based on the user's input instructions.

[0169] In addition, the user interaction module can also determine the adjustment amount such as translation and rotation based on the user's input instructions.

[0170] For example, the core processing module can adjust the spatial parameters of the mask according to the adjustment amount, thereby obtaining the adjusted mask.

[0171] For the granularity in the standard type mask, the original coordinates can be determined based on Formula 5 above, and the original coordinates can be substituted into the corresponding formulas in Formulas 2, 3 and 4 above to obtain the corresponding value of the granularity in the adjusted mask, thereby generating the adjusted standard type mask.

[0172] For a custom shape mask, each control point on the closed curve can be moved according to the adjustment amount. Then, the value corresponding to the granularity can be determined based on the relationship between the granularity and the adjusted closed curve, thus obtaining the adjusted mask. For example, based on the embodiments described above, the value corresponding to the granularity can be determined by determining whether the granularity is inside or outside the closed curve; this will not be elaborated further here. Accordingly, an adjusted custom shape mask can be generated.

[0173] The data and display module can input the images to be fused (e.g., the first modal image and the second modal image) into the core processing module through the image data manager. The core processing module can fuse the first modal image and the second modal image according to the generated mask, and feed the fused image back to the data and display module. Then, the data and display module can render and display the fused image through the display rendering engine.

[0174] Figure 11 This is a schematic flowchart illustrating another image fusion method according to an embodiment of the present disclosure.

[0175] like Figure 11 As shown, users can first select the mask type according to their needs, such as standard type masks and custom shape masks.

[0176] Standard mask types can include subtypes such as checkerboard masks, horizontal stripe masks, and vertical stripe masks, allowing users to further select from these subtypes as needed. When a user selects a custom shape mask, they can draw closed curves as required.

[0177] Furthermore, users can adjust the spatial parameters of the mask, such as adjusting the mask's translation position and rotation angle; for standard types of masks, users can also set the mask elements (e.g., setting the size of the mask elements). Based on this, the mask generation is complete.

[0178] Then, the generated mask can be used to perform image fusion on the images to be fused, and the display device can render and display the fused image in real time.

[0179] The system can continue to listen to user input commands, such as when the user adjusts the spatial parameters of the mask again. The mask can be adjusted according to the translation position, rotation angle, etc., to obtain the adjusted mask. Then, the fused image can be redefined based on the adjusted mask, thereby realizing dynamic adjustment of the mask in the fused image.

[0180] Figure 12This is a block diagram illustrating an image fusion apparatus according to an exemplary embodiment. For example, the image fusion apparatus can be disposed in or applied to the display device described in any of the preceding embodiments; for example, the image fusion apparatus can be used to perform the image fusion method described in any of the preceding embodiments.

[0181] like Figure 12 As shown, the image fusion device includes a receiving module 1201, a processing module 1202, and a display module 1203.

[0182] In some embodiments, the receiving module is configured to receive a first instruction and / or a second instruction; the processing module is configured to determine a mask according to the first instruction; fuse images of multiple modalities according to the mask; adjust the spatial parameters of the mask according to the second instruction to obtain an adjusted mask; and the display module is configured to display the fused image according to the adjusted mask.

[0183] In some embodiments, the mask type includes at least one of the following: checkerboard mask; horizontal stripe mask; vertical stripe mask; custom shape mask.

[0184] In some embodiments, the processing module is configured to determine the granularity of the mask based on the resolution of the image.

[0185] In some embodiments, when the mask type includes at least one of a checkerboard mask, a horizontal stripe mask, and a vertical stripe mask, the processing module is configured to determine the size of the mask unit by the first instruction, wherein the size of the mask unit is greater than or equal to the size of the granularity; and determine the value of the mask unit based on the relationship between the size of the mask unit and the coordinates of the mask unit.

[0186] In some embodiments, the mask unit is characterized based on at least one of the following: physical size; number of particles.

[0187] In some embodiments, the checkerboard mask corresponds to a checkerboard pattern in the screen; and / or, the horizontal stripe mask corresponds to a horizontal stripe in the screen; and / or, the vertical stripe mask corresponds to a vertical stripe in the screen.

[0188] In some embodiments, the processing module is configured to perform an inverse operation on the coordinates of the granularity in the adjusted mask according to the adjustment method of the spatial parameters to obtain the original coordinates; and determine the value corresponding to the granularity in the adjusted mask according to the relationship between the granularity and the original coordinates.

[0189] In some embodiments, when the type of the mask includes a custom shape mask, the processing module is configured to determine a closure curve according to the first instruction; and to determine the value corresponding to the granularity according to the relationship between the granularity and the closure curve.

[0190] In some embodiments, the processing module is configured to adjust the points on the closed curve according to the adjustment method of the spatial parameters to obtain an adjusted closed curve; and to determine the value corresponding to the grain size in the adjusted mask according to the relationship between the grain size and the adjusted closed curve.

[0191] In some embodiments, the type of grain size value in the mask is the same as the type of mode.

[0192] In some embodiments, the processing module is configured to determine an image corresponding to the granularity based on the granularity value in the mask; determine an image unit located at the coordinates of the granularity in the image of the corresponding modality; and the display unit is configured to display the image unit at the granularity-corresponding position.

[0193] In some embodiments, the spatial parameters include at least one of the following: translational position; rotation angle.

[0194] In some embodiments, the modality includes at least one of the following: computed tomography (CT); magnetic resonance imaging (MRI); positron emission tomography (PET); and single-photon emission computed tomography (SPECT).

[0195] In some embodiments, the number of types of the multiple modalities is greater than or equal to two.

[0196] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0197] Embodiments of this disclosure also provide a storage medium storing instructions that, when executed on a communication device, cause the communication device to perform the image fusion method described in any of the above embodiments.

[0198] Embodiments of this disclosure also provide a program product that, when executed by a communication device, causes the communication device to perform the image fusion method described in any of the above embodiments.

[0199] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0200] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0201] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image fusion method, characterized in that, include: Determine the mask according to the first instruction; Images of multiple modalities are fused based on the mask; The spatial parameters of the mask are adjusted according to the second instruction to obtain the adjusted mask; The fused image is displayed based on the adjusted mask.

2. The method according to claim 1, characterized in that, The mask type includes at least one of the following: checkerboard mask; Horizontal stripe mask; Vertical stripe mask; Custom shape mask.

3. The method according to claim 2, characterized in that, Determining the mask according to the first instruction includes: The granularity of the mask is determined based on the resolution of the image.

4. The method according to claim 3, characterized in that, When the mask type includes at least one of a checkerboard mask, a horizontal stripe mask, and a vertical stripe mask, the step of determining the mask according to the first instruction further includes: The size of the mask unit is determined according to the first instruction, wherein the size of the mask unit is greater than or equal to the size of the particle size; The value of the mask unit is determined based on the relationship between the size of the mask unit and the coordinates of the mask unit.

5. The method according to claim 4, characterized in that, The size of the mask unit is characterized based on at least one of the following: Physical dimensions; The number of particle sizes.

6. The method according to claim 5, characterized in that, The checkerboard mask corresponds to a checkerboard pattern in the screen; and / or, The horizontal stripe mask corresponds to horizontal stripes in the mask unit on the screen; and / or, The vertical stripe mask corresponds to vertical stripes in the mask unit on the screen.

7. The method according to claim 5, characterized in that, The step of adjusting the spatial parameters of the mask according to the second instruction to obtain the adjusted mask includes: The original coordinates are obtained by inversely calculating the coordinates of the grain size in the adjusted mask according to the adjustment method of the spatial parameters. Based on the relationship between the particle size and the original coordinates, the value corresponding to the particle size in the adjusted mask is determined.

8. The method according to claim 3, characterized in that, In the case that the type of mask includes a custom shape mask, the step of determining the mask according to the first instruction further includes: Determine the closed curve according to the first instruction; The value corresponding to the particle size is determined based on the relationship between the particle size and the closed curve.

9. The method according to claim 8, characterized in that, The step of adjusting the spatial parameters of the mask according to the second instruction to obtain the adjusted mask includes: The points on the closed curve are adjusted according to the adjustment method of the spatial parameters to obtain the adjusted closed curve; Based on the relationship between the particle size and the adjusted closed curve, the value corresponding to the particle size in the adjusted mask is determined.

10. The method according to any one of claims 1 to 9, characterized in that, The process of fusing images of multiple modalities based on the mask includes: The image corresponding to the particle size mode is determined based on the particle size value in the mask; Determine the image unit located at the coordinates of the corresponding modality in the image at the specified granularity; The image unit is displayed at the position corresponding to the granularity.

Images