Medical image fusion method and equipment

Abstract

The present invention provides a medical image fusion method and device, wherein the method comprises the following steps: performing three-dimensional imaging acquisition on the patient's diseased part to obtain a three-dimensional imaging image; identifying the tumor tissue located in the three-dimensional imaging image, and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image; performing two-dimensional ultrasound acquisition on the diseased part based on the two-dimensional acquisition frame, and performing dimensional conversion based on the obtained two-dimensional ultrasound image to obtain a three-dimensional ultrasound image; mapping the tumor tissue located in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtaining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image; and determining the trajectory data of the electromagnetic navigation needle based on the morphological attributes of the tumor tissue. The present invention at least improves the corresponding medical diagnosis assistance.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a medical image fusion method and device. Background Art

[0002] Ablation therapy is a medical technique that destroys abnormal tissues through heat or cold. This treatment method is mainly used to treat tumors or other abnormal tissues, including liver tumors, kidney tumors, prostate tumors, etc. Common ablation techniques include radiofrequency ablation, microwave ablation, laser ablation and cryoablation.

[0003] The inventors discovered during their research that when medical staff use electromagnetic navigation needles to perform corresponding ablation operations, since the electromagnetic navigation needles are generally used in conjunction with corresponding ultrasound equipment, when the ultrasound equipment performs corresponding two-dimensional ultrasound acquisition of the diseased area, the acquired two-dimensional ultrasound images may not be able to clearly display the corresponding tumor tissue, making it impossible to determine the precise location of the tumor tissue and perform subsequent puncture operations. Summary of the invention

[0004] Based on the above problems, the present invention is proposed to provide a medical image fusion method and device that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a medical image fusion method is provided, comprising the following steps:

[0006] Performing three-dimensional imaging acquisition on the patient's diseased part to obtain a three-dimensional imaging image corresponding to the diseased part;

[0007] Identifying the tumor tissue in the three-dimensional imaging image, and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image;

[0008] Performing two-dimensional ultrasonic acquisition on the diseased part based on the two-dimensional acquisition framework, and performing dimensional conversion based on the obtained two-dimensional ultrasonic image to obtain a three-dimensional ultrasonic image;

[0009] Mapping the tumor tissue in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtaining the morphological properties of the tumor tissue in the three-dimensional ultrasound image;

[0010] The trajectory data of the electromagnetic navigation needle is determined based on the morphological properties of the tumor tissue.

[0011] Optionally, in the method according to the present invention, identifying the tumor tissue located in the three-dimensional imaging image, and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image, includes:

[0012] Inputting the three-dimensional imaging image into a trained image recognition model to determine the tumor tissue located in the three-dimensional imaging image and the organ tissue connected to the tumor tissue;

[0013] Determine a tissue connection surface between the tumor tissue and the organ tissue, and perform coordinate processing on the three-dimensional imaging image based on the tissue connection surface to obtain a first image coordinate system corresponding to the three-dimensional imaging image, wherein the tissue connection surface is parallel to a reference coordinate plane formed between an X-axis and a Y-axis in the first image coordinate system;

[0014] Determining each contour coordinate point constituting the tissue contour of the tumor tissue in the three-dimensional imaging image after coordinate processing;

[0015] Determine the coordinate points corresponding to the X-axis coordinate extreme value, the Y-axis coordinate extreme value, and the Z-axis coordinate extreme value in each contour coordinate point as the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, respectively;

[0016] A tumor region surrounding the tumor tissue is generated based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, and a two-dimensional acquisition frame centered on the tumor region and adapted to the ultrasound acquisition window is determined in the three-dimensional imaging image.

[0017] Optionally, in the method according to the present invention, generating a tumor region surrounding the tumor tissue based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point comprises:

[0018] Determine a Z-axis minimum value and a Z-axis maximum value based on the Z-axis extreme value coordinate point, and determine a Z-axis length based on the Z-axis minimum value and the Z-axis maximum value;

[0019] Taking the X-axis extreme value coordinate point as the starting point and extending along the direction parallel to the Y-axis, an X-axis contour line is obtained;

[0020] Taking the Y-axis extreme value coordinate point as a starting point and extending along a direction parallel to the X-axis, a Y-axis contour line intersecting with the X-axis contour line is obtained;

[0021] A tumor coordinate plane corresponding to the tumor tissue is determined based on the X-axis contour line and the Y-axis contour line, and a surface shape having the same length as the Z-axis length is extended toward the Z-axis extreme coordinate point based on the tumor coordinate plane to obtain a tumor area surrounding the tumor tissue.

[0022] Optionally, in the method according to the present invention, determining a two-dimensional acquisition frame centered on the tumor region and adapted to an ultrasound acquisition window in the three-dimensional imaging image includes:

[0023] Acquire a region center point located in the tumor region, and acquire images corresponding to the ultrasound acquisition window at multiple image viewing angles of the tumor region in a direction parallel to the reference coordinate plane as a reference direction, to obtain region schematic diagrams corresponding to different image viewing angles, wherein the image center point of each region schematic diagram coincides with the region center point;

[0024] Determine that each regional schematic diagram corresponds to each image portion of the tumor tissue, and determine the area of ​​each image portion to obtain each image area corresponding to each regional schematic diagram;

[0025] The schematic diagram of the region with the largest corresponding image area is determined as a two-dimensional acquisition frame adapted to the ultrasound acquisition window.

[0026] Optionally, in the method according to the present invention, performing two-dimensional ultrasound acquisition on the diseased part based on the two-dimensional acquisition framework, and performing dimensional conversion based on the obtained two-dimensional ultrasound image to obtain a three-dimensional ultrasound image, including:

[0027] Coordinate processing is performed on the diseased part, and a real coordinate system corresponding to the diseased part is formed;

[0028] Determining the first frame coordinate points corresponding to the two-dimensional acquisition frame as a first coordinate group in the first image coordinate system, and determining a coordinate conversion relationship between the real coordinate system and the first image coordinate system;

[0029] Based on the coordinate transformation relationship, coordinate transformation is performed on each first frame coordinate point in the first coordinate group, and each obtained second frame coordinate point is determined as a second coordinate group corresponding to the real coordinate system;

[0030] Performing two-dimensional ultrasonic acquisition on the diseased part based on the second coordinate group, and performing coordinate processing on the obtained two-dimensional ultrasonic image to obtain a second image coordinate system corresponding to the two-dimensional ultrasonic image;

[0031] The two-dimensional ultrasound image is dimensionally transformed based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

[0032] Optionally, in the method according to the present invention, performing dimension conversion on the two-dimensional ultrasound image based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to a third image coordinate system includes:

[0033] Acquire a region schematic diagram corresponding to the two-dimensional acquisition frame, and determine an image viewing angle corresponding to the region schematic diagram;

[0034] Determine a regional surface located in the tumor region and having a parallel relationship with the image viewing angle, and determine a regional thickness corresponding to the tumor region based on the regional surface;

[0035] Based on the region thickness, a Z axis having the same length as the region thickness is established in the second image coordinate system to obtain a third image coordinate system;

[0036] The two-dimensional ultrasound image is transformed based on the Z axis to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

[0037] Optionally, in the method according to the present invention, obtaining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image includes:

[0038] Determine all concave contour segments respectively having a concave trend located on the tumor tissue along the tissue contour of the tumor tissue, and obtain a first contour endpoint and a second contour endpoint located on the same concave contour segment;

[0039] Connecting the first contour endpoint and the second contour endpoint to obtain an endpoint connection line, and obtaining a line segment distance corresponding to the endpoint connection line;

[0040] Obtaining the concave depth corresponding to the concave contour segment, and retrieving the distance weight value and the depth weight value;

[0041] The normalized line segment distance and the depression depth are respectively multiplied by the distance weight value and the depth weight value to obtain a distance coefficient and a depth coefficient;

[0042] The distance coefficient and the depth coefficient are summed to obtain a concave evaluation coefficient corresponding to the concave contour segment, and each concave evaluation coefficient corresponding to each concave contour segment is compared with a preset threshold value;

[0043] The morphological attributes of the tumor tissue in the three-dimensional ultrasound image are determined based on the comparison result.

[0044] Optionally, in the method according to the present invention, determining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image based on the comparison result includes:

[0045] When the concave evaluation coefficients respectively corresponding to the concave contour segments are all less than or equal to the preset threshold, determining the morphological attribute of the tumor tissue as an independent attribute;

[0046] When the concave evaluation coefficient corresponding to any concave contour segment is greater than the preset threshold, the concave contour segment is determined as a first target contour segment, and a concave point position corresponding to the concave depth of the first target contour segment is obtained;

[0047] Taking the depression point as a starting point, determine other depression contour segments that are closest to the depression point along the tissue contour, and determine the depression contour segments as second target contour segments;

[0048] The concave points corresponding to the first target contour segment and the concave points corresponding to the second target contour segment are connected along the tissue contour to obtain tissue division lines that divide the tumor tissue into associated tumors, and the morphological attributes of the tumor tissue are determined as associated attributes.

[0049] Optionally, in the method according to the present invention, determining the trajectory data of the electromagnetic navigation needle based on the morphological attributes of the tumor tissue includes:

[0050] When the morphological attribute of the tumor tissue is an independent attribute, obtaining a tissue center point corresponding to the tumor tissue, and generating trajectory data passing through the tissue center point;

[0051] When the morphological attribute of the tumor tissue is a companion attribute, each companion tumor included in the tumor tissue is obtained, and each companion center point corresponding to each companion tumor is obtained respectively, and trajectory data of each companion center point corresponding to the electromagnetic navigation needle is generated respectively.

[0052] According to another aspect of the present invention, a medical image fusion device is provided, comprising:

[0053] A three-dimensional imaging module is configured to perform three-dimensional imaging acquisition on the diseased part of the patient to obtain a three-dimensional imaging image corresponding to the diseased part;

[0054] a frame determination module, configured to identify the tumor tissue in the three-dimensional imaging image, and determine a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image;

[0055] A two-dimensional ultrasound module is configured to perform two-dimensional ultrasound acquisition on the diseased part based on the two-dimensional acquisition framework, and perform dimensional conversion based on the obtained two-dimensional ultrasound image to obtain a three-dimensional ultrasound image;

[0056] a tissue mapping module, configured to map the tumor tissue in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtain the morphological properties of the tumor tissue in the three-dimensional ultrasound image;

[0057] The trajectory determination module is configured to determine the trajectory data of the electromagnetic navigation needle based on the morphological properties of the tumor tissue.

[0058] According to the solution of the present invention, after three-dimensional imaging of the diseased part, a corresponding three-dimensional imaging image including the tumor tissue can be obtained, so that the tumor tissue can be located, so that in the subsequent process, a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window can be determined in the three-dimensional imaging image, and further two-dimensional ultrasound acquisition is performed on the diseased part through the two-dimensional acquisition frame, and the corresponding two-dimensional ultrasound image is converted into a three-dimensional ultrasound image, and then the tumor tissue located in the three-dimensional imaging image can be mapped to the three-dimensional ultrasound image, so that the medical staff can learn the location and morphological properties of the corresponding tumor tissue based on the ultrasound principle, thereby improving the corresponding medical diagnosis assistance; at the same time, in order to facilitate the subsequent ablation operation, the present embodiment can also automatically generate trajectory data of the corresponding electromagnetic navigation needle based on the morphological properties of the tumor tissue, thereby improving the corresponding ablation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flowchart of a medical image fusion method according to an embodiment of the present invention is shown;

[0060] Figure 2 shows a structural block diagram of the ultrasound device in this embodiment;

[0061] Figure 3 A structural block diagram of a medical image fusion device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0062] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0063] Ablation therapy is a medical technique that destroys abnormal tissues through heat or cold. This treatment method is mainly used to treat tumors or other abnormal tissues, including liver tumors, kidney tumors, prostate tumors, etc. Common ablation techniques include radiofrequency ablation, microwave ablation, laser ablation and cryoablation.

[0064] The inventors discovered during their research that when medical staff use electromagnetic navigation needles to perform corresponding ablation operations, since the electromagnetic navigation needles are generally used in conjunction with corresponding ultrasound equipment, when the ultrasound equipment performs corresponding two-dimensional ultrasound acquisition of the diseased area, the acquired two-dimensional ultrasound images may not be able to clearly display the corresponding tumor tissue, making it impossible to determine the precise location of the tumor tissue and perform subsequent puncture operations.

[0065] To solve the problems existing in the above-mentioned prior art, the inventor proposes the solution of the present invention. An embodiment of the present invention provides a medical image fusion method, which can be executed in a computing device, wherein the computing device can be understood as a terminal with data processing function, such as a mobile phone or a computer.

[0066] Figure 1 FIG. 4 is a flow chart showing a medical image fusion method according to an embodiment of the present invention. Figure 1 As shown, the method starts at step S102, and in step S102, the following contents are included:

[0067] A three-dimensional imaging acquisition is performed on the patient's diseased part to obtain a three-dimensional imaging image corresponding to the diseased part.

[0068] For example, in this embodiment, in order to be able to determine the tumor tissue located in the patient's body, the patient's diseased part can be imaged and collected in three dimensions based on the three-dimensional imaging technology existing in existing medicine, so as to obtain a three-dimensional imaging image corresponding to the diseased part; here, the diseased part can be understood as a part of the human body corresponding to the tumor tissue, such as the chest, leg or hand, and the corresponding three-dimensional imaging technology can include CT scanning technology, MRI scanning technology and PET-CT scanning technology, etc.; by performing three-dimensional imaging acquisition of the diseased part through any of the above three-dimensional imaging technologies, a three-dimensional imaging image can be obtained, wherein the three-dimensional imaging image includes the organ tissue corresponding to the diseased part and the corresponding tumor tissue, and the organ tissue may include blood vessels, organs, etc.

[0069] In step S104, the following contents are included:

[0070] The tumor tissue in the three-dimensional imaging image is identified, and a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window is determined in the three-dimensional imaging image.

[0071] For example, in this embodiment, after obtaining the corresponding three-dimensional imaging image, the three-dimensional imaging image can be subjected to corresponding image recognition to determine the tumor tissue existing in the diseased part and obtain the location of the tumor tissue in the diseased part, so as to facilitate the corresponding electromagnetic navigation to perform the corresponding puncture operation on the tumor tissue in the subsequent process, and further perform treatment methods such as ablation; and because when the electromagnetic navigation needle is punctured, the electromagnetic navigation needle is generally used in conjunction with the corresponding ultrasound device, but when the ultrasound device performs the corresponding two-dimensional ultrasound acquisition on the diseased part, the acquired two-dimensional ultrasound image may not be able to clearly display the corresponding tumor tissue, so that the precise location of the tumor tissue cannot be determined, and the subsequent puncture operation cannot be performed. Based on this, it is necessary to display the tumor tissue located in the three-dimensional imaging image in the two-dimensional ultrasound image to achieve corresponding image fusion, so that the electromagnetic navigation needle can perform the corresponding puncture operation; therefore, it is necessary to first obtain the ultrasound acquisition window of the corresponding ultrasound device, and determine the frame of the three-dimensional imaging image based on the ultrasound acquisition window, obtain a two-dimensional acquisition frame adapted to the ultrasound acquisition window, and perform the subsequent image fusion process based on the obtained two-dimensional acquisition frame.

[0072] Here, in order to improve the corresponding fusion effect, the corresponding two-dimensional acquisition frame can be acquired with the tumor tissue as the center, thereby improving the image prominence of the tumor tissue.

[0073] It should be noted that, in this embodiment, Figure 2 As shown, the ultrasound device may include a display terminal and an ultrasound probe connected to the display terminal. When performing two-dimensional ultrasound acquisition on the patient's diseased part, the doctor will attach the corresponding ultrasound probe to the skin of the patient's diseased part to display the corresponding two-dimensional ultrasound image on the display terminal. When further treatment methods such as ablation are performed, the doctor will also insert the corresponding electromagnetic navigation needle into the patient's body, thereby determining the location of the electromagnetic navigation needle through the corresponding two-dimensional ultrasound image and completing the corresponding puncture operation. Since the two-dimensional ultrasound image displayed on the display terminal is formed based on the reflection of the ultrasound wave emitted by the ultrasound probe in the diseased part, the corresponding two-dimensional ultrasound image is an acquisition window with a certain size, that is, the ultrasound acquisition window mentioned above. Since the size of the ultrasound acquisition window depends on the hardware structure of the corresponding ultrasound device, different models of ultrasound devices may correspond to ultrasound acquisition windows of different sizes.

[0074] Furthermore, in this embodiment, the above-mentioned “identifying the tumor tissue located in the three-dimensional imaging image, and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image” may also include the following steps:

[0075] Inputting the three-dimensional imaging image into a trained image recognition model to determine the tumor tissue located in the three-dimensional imaging image and the organ tissue connected to the tumor tissue;

[0076] Determine a tissue connection surface between the tumor tissue and the organ tissue, and perform coordinate processing on the three-dimensional imaging image based on the tissue connection surface to obtain a first image coordinate system corresponding to the three-dimensional imaging image, wherein the tissue connection surface is parallel to a reference coordinate plane formed between an X-axis and a Y-axis in the first image coordinate system;

[0077] Determining each contour coordinate point constituting the tissue contour of the tumor tissue in the three-dimensional imaging image after coordinate processing;

[0078] Determine the coordinate points corresponding to the X-axis coordinate extreme value, the Y-axis coordinate extreme value, and the Z-axis coordinate extreme value in each contour coordinate point as the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, respectively;

[0079] A tumor region surrounding the tumor tissue is generated based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, and a two-dimensional acquisition frame centered on the tumor region and adapted to the ultrasound acquisition window is determined in the three-dimensional imaging image.

[0080] For example, in this embodiment, in order to obtain a two-dimensional acquisition frame that is adapted to the ultrasound acquisition window and corresponds to the tumor tissue, the following method steps may be used:

[0081] First, the three-dimensional imaging images obtained above can be input into the trained image recognition model accordingly, so that the image recognition model can perform image recognition on the three-dimensional imaging images, and determine the tumor tissue located in the three-dimensional imaging images and the organ tissue connected to the tumor tissue; it should be noted that since the tumor tissue generally cannot exist independently, that is, it needs to rely on corresponding tissue carriers to grow, such as blood vessels or organs, in order to be able to fully identify the tumor tissue, it is necessary to simultaneously identify the organ tissue connected to the tumor tissue;

[0082] Next, since there is a corresponding tissue connection surface between the tumor tissue and the organ tissue, corresponding coordinate processing can be performed on the three-dimensional imaging image based on the tissue connection surface, so as to obtain a first image coordinate system corresponding to the three-dimensional imaging image, wherein the first image coordinate system is a three-dimensional coordinate system, and the first image coordinate system includes an X-axis, a Y-axis, and a Z-axis, and a reference coordinate plane composed of the X-axis and the Y-axis is parallel to the corresponding tissue connection surface;

[0083] Then, since the tumor tissue in the three-dimensional imaging image has been identified by the corresponding trained image recognition model, each contour coordinate point constituting the tissue contour of the tumor tissue can be determined in the three-dimensional imaging image based on the first image coordinate system;

[0084] Then, since the shape of the formed tumor tissue is irregular, it may be approximately circular, approximately square, approximately polygonal, etc. Therefore, in order to be able to completely acquire the tumor tissue, the coordinate points corresponding to the X-axis coordinate extreme value, the Y-axis coordinate extreme value, and the Z-axis coordinate extreme value can be determined as the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point based on each contour coordinate point, so that the tumor area surrounding the tumor tissue can be further generated based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point;

[0085] Finally, a two-dimensional acquisition frame centered on the tumor region and adapted to the ultrasound acquisition window can be determined in the three-dimensional imaging image.

[0086] Here, this embodiment performs corresponding coordinate processing on the three-dimensional imaging image, and obtains each contour coordinate point of the tissue contour constituting the tumor tissue according to the obtained first image coordinate system, and then can generate a tumor area corresponding to the surrounding tumor tissue based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point and the Z-axis extreme value coordinate point located in each contour coordinate point. The formed tumor area corresponds to a regular shape, which is convenient for subsequent further image processing, reduces the corresponding data calculation amount, and can also improve the accuracy of tumor tissue recognition.

[0087] Furthermore, on the one hand, in this embodiment, the above-mentioned “generating a tumor region surrounding the tumor tissue based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point” may also include the following steps:

[0088] Determine a Z-axis minimum value and a Z-axis maximum value based on the Z-axis extreme value coordinate point, and determine a Z-axis length based on the Z-axis minimum value and the Z-axis maximum value;

[0089] Taking the X-axis extreme value coordinate point as the starting point and extending along the direction parallel to the Y-axis, an X-axis contour line is obtained;

[0090] Taking the Y-axis extreme value coordinate point as a starting point and extending along a direction parallel to the X-axis, a Y-axis contour line intersecting with the X-axis contour line is obtained;

[0091] A tumor coordinate plane corresponding to the tumor tissue is determined based on the X-axis contour line and the Y-axis contour line, and a surface shape having the same length as the Z-axis length is extended toward the Z-axis extreme coordinate point based on the tumor coordinate plane to obtain a tumor area surrounding the tumor tissue.

[0092] For example, in this embodiment, it can be known from the above content that in reality, the shape of the corresponding tumor tissue may be irregular. Therefore, a tumor region with a corresponding regular shape can be generated based on the tumor tissue based on a corresponding image processing operation to reduce the corresponding data processing amount. Here, the corresponding image processing operation can be implemented based on the following method steps:

[0093] First, since the corresponding tumor region is formed based on three dimensions, the corresponding tumor region can be generated based on the corresponding X-axis extreme value coordinate point, the Y-axis extreme value coordinate point and the Z-axis extreme value coordinate point. Correspondingly, the Z-axis extreme value coordinate point includes a Z-axis minimum value coordinate point and a Z-axis maximum value coordinate point. The corresponding Z-axis length can be determined based on the Z-axis minimum value of the corresponding Z-axis minimum value coordinate point and the Z-axis maximum value of the corresponding Z-axis maximum value coordinate point. Here, the corresponding Z-axis length can be understood as the size of the tumor tissue in the direction parallel to the Z-axis.

[0094] Next, the X-axis extreme value coordinate point can be used as the starting point and extended in a direction parallel to the Y-axis to obtain the corresponding X-axis contour line;

[0095] Next, after the corresponding X-axis contour line is formed, the Y-axis extreme value coordinate point can be used as a starting point and extended in a direction parallel to the X-axis to obtain a Y-axis contour line intersecting with the X-axis contour line; it should be noted that since the Y-axis extreme value coordinate point includes the Y-axis maximum value coordinate point corresponding to the Y-axis maximum value and the Y-axis minimum value coordinate point corresponding to the Y-axis minimum value, and the X-axis extreme value coordinate point also includes the X-axis maximum value coordinate point corresponding to the X-axis maximum value and the X-axis minimum value coordinate point corresponding to the X-axis minimum value, therefore, the correspondingly generated X-axis contour lines and Y-axis contour lines also correspond to two, and the two X-axis contour lines and Y-axis contour lines can form a plane with a rectangular shape accordingly;

[0096] Finally, the corresponding rectangular plane can be determined as the tumor coordinate plane corresponding to the tumor tissue, and the tumor coordinate plane is extended based on the aforementioned acquired Z-axis length, thereby obtaining a tumor region including the tumor tissue.

[0097] It should be noted that, in this embodiment, the tumor region obtained by the above method is specifically in the shape of a three-dimensional rectangle, which can not only fully surround the tumor tissue, but also reduce the data processing amount and improve the corresponding processing efficiency due to its simple shape.

[0098] Furthermore, on the other hand, in this embodiment, the above-mentioned “determining a two-dimensional acquisition frame centered on the tumor area and adapted to the ultrasound acquisition window in the three-dimensional imaging image” may also include the following steps:

[0099] Acquire a region center point located in the tumor region, and acquire images corresponding to the ultrasound acquisition window at multiple image viewing angles of the tumor region in a direction parallel to the reference coordinate plane as a reference direction, to obtain region schematic diagrams corresponding to different image viewing angles, wherein the image center point of each region schematic diagram coincides with the region center point;

[0100] Determine that each regional schematic diagram corresponds to each image portion of the tumor tissue, and determine the area of ​​each image portion to obtain each image area corresponding to each regional schematic diagram;

[0101] The schematic diagram of the region with the largest corresponding image area is determined as a two-dimensional acquisition frame adapted to the ultrasound acquisition window.

[0102] For example, in this embodiment, the acquisition of the two-dimensional acquisition frame can be achieved based on the following method steps:

[0103] First, since the tumor region corresponds to a regular shape based on three-dimensional, the center point of the region located in the tumor region can be quickly determined by interlacing the corresponding vertices, and further, the image of the corresponding ultrasound acquisition window of the tumor region at multiple image angles is acquired with the direction parallel to the reference coordinate plane obtained above as the reference direction, so as to obtain the regional schematic diagrams corresponding to different image angles, and the image center point of each regional schematic diagram also overlaps with the corresponding regional center point respectively; here, since the tumor tissue has a tissue connection relationship with the corresponding organ tissue, in order to ensure that the tumor tissue can be fully displayed after the subsequent image fusion, it is necessary to eliminate the interference effect of the organ tissue on the tumor tissue as much as possible;

[0104] Here, the elimination method can be to obtain the corresponding multi-image perspectives of the tumor area by taking the reference coordinate plane as the reference direction, so as to obtain the front view, rear view, left view, right view, top view and bottom view of the corresponding reference coordinate plane. Since the reference coordinate plane is parallel to the tissue connection plane, the corresponding rear view and front view are obtained based on the image perspective parallel to the tissue connection plane, so as to ensure that one of the views can be obtained from the image perspective of the tumor tissue being located in front of the organ tissue, and the other view can be obtained from the image perspective of the tumor tissue being located behind the organ tissue. Since the organ tissue and the tumor tissue are located on both sides of the tissue connection plane, it can be further ensured that the interference effect of the organ tissue on the view obtained from the image perspective of the tumor tissue being located in front of the organ tissue is minimal, and the interference effect of the view obtained from the image perspective of the tumor tissue being located behind the organ tissue is maximized, thereby achieving the adjustment of the interference influence.

[0105] Next, after completing the regional schematic diagrams for different image viewing angles, it is possible to determine the image parts of the tumor region corresponding to each regional schematic diagram; here, since the corresponding tumor tissue is in an irregular shape, after obtaining the regional schematic diagrams corresponding to different image viewing angles, the image area of ​​the image part of the tumor tissue corresponding to each regional schematic diagram may be different. In order to ensure that the tumor tissue is fully displayed as much as possible during the subsequent two-dimensional ultrasound acquisition process to improve the subsequent puncture accuracy, it is necessary to determine the area of ​​each image part, and then obtain the image areas corresponding to each regional schematic diagram;

[0106] Finally, after the image area corresponding to each regional schematic diagram is determined, the regional schematic diagram corresponding to the largest image area can be determined as the two-dimensional acquisition frame adapted to the ultrasound acquisition window.

[0107] It should be noted that after the acquisition of schematic diagrams of each area corresponding to different image viewing angles is completed, since the front view and the rear view are symmetrical to each other, the left view and the right view are symmetrical to each other, and the top view and the bottom view are symmetrical to each other, without considering the interference of other organ tissues on the tumor tissue, the image areas corresponding to the two views showing a symmetrical relationship should be the same. Therefore, either one of them can be selected as the corresponding two-dimensional acquisition frame; at the same time, since one of the front view and the rear view is acquired from the image perspective of the tumor tissue being located in front of the organ tissue, and the other view is acquired from the image perspective of the tumor tissue being located behind the organ tissue, in order to eliminate the interference of the organ tissue on the tumor tissue as much as possible, the view acquired from the image perspective of the tumor tissue being located behind the organ tissue can be removed, and the corresponding two-dimensional acquisition frame can be determined based on all remaining views.

[0108] In step S106, the following contents are included:

[0109] The diseased part is subjected to two-dimensional ultrasonic acquisition based on the two-dimensional acquisition framework, and dimension conversion is performed based on the obtained two-dimensional ultrasonic image to obtain a three-dimensional ultrasonic image.

[0110] For example, in this embodiment, after the two-dimensional acquisition frame for the three-dimensional imaging image is determined based on the tumor tissue, two-dimensional ultrasound acquisition based on the two-dimensional acquisition frame can be synchronously performed on the diseased part. Here, the two-dimensional ultrasound acquisition based on the two-dimensional acquisition frame can be understood as finding the corresponding position area in the patient's part according to the two-dimensional acquisition frame determined above and performing two-dimensional ultrasound acquisition, so that the obtained two-dimensional ultrasound image can be the same size as the two-dimensional acquisition frame, and the position and shape of the corresponding tumor tissue are also the same. For example, when the schematic diagram of the area corresponding to the two-dimensional acquisition frame is a front view based on the reference coordinate plane, the corresponding two-dimensional ultrasound image is also a front view obtained based on the reference coordinate diagram;

[0111] After completing the corresponding two-dimensional ultrasound acquisition and obtaining the corresponding two-dimensional ultrasound image, since the two-dimensional ultrasound image may not be able to clearly display the corresponding tumor tissue, in order to facilitate the subsequent insertion of the electromagnetic navigation needle so that the electromagnetic navigation can puncture the tumor tissue, it is necessary to convert the dimension of the two-dimensional ultrasound image, that is, convert the original two-dimensional into three-dimensional, so as to obtain the corresponding three-dimensional ultrasound image, so as to facilitate the subsequent fusion of the three-dimensional-based tumor tissue into the corresponding three-dimensional ultrasound image.

[0112] Furthermore, in this embodiment, the above-mentioned “performing two-dimensional ultrasonic acquisition on the diseased part based on the two-dimensional acquisition framework, and performing dimensional conversion based on the obtained two-dimensional ultrasonic image to obtain a three-dimensional ultrasonic image” may also include the following steps:

[0113] Coordinate processing is performed on the diseased part, and a real coordinate system corresponding to the diseased part is formed;

[0114] Determining the first frame coordinate points corresponding to the two-dimensional acquisition frame as a first coordinate group in the first image coordinate system, and determining a coordinate conversion relationship between the real coordinate system and the first image coordinate system;

[0115] Based on the coordinate transformation relationship, coordinate transformation is performed on each first frame coordinate point in the first coordinate group, and each obtained second frame coordinate point is determined as a second coordinate group corresponding to the real coordinate system;

[0116] Performing two-dimensional ultrasonic acquisition on the diseased part based on the second coordinate group, and performing coordinate processing on the obtained two-dimensional ultrasonic image to obtain a second image coordinate system corresponding to the two-dimensional ultrasonic image;

[0117] The two-dimensional ultrasound image is dimensionally transformed based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

[0118] For example, in order to perform corresponding two-dimensional ultrasound acquisition of the diseased part based on the two-dimensional acquisition framework obtained above, and further perform dimensional conversion to obtain a corresponding three-dimensional ultrasound image, in this embodiment, the following method steps can be used to achieve this:

[0119] First, corresponding coordinate processing can be performed based on the diseased part, so as to obtain a real coordinate system corresponding to the diseased part. Here, the real coordinate system is established based on three dimensions;

[0120] Next, since the two-dimensional acquisition frame is acquired based on the aforementioned three-dimensional imaging image, each first frame coordinate point corresponding to the two-dimensional acquisition frame can be determined based on the first image coordinate system corresponding to the three-dimensional imaging image, and each first frame coordinate point can be correspondingly determined as a first coordinate group; here, since the three-dimensional imaging image is acquired based on the diseased part, the coordinate conversion relationship between the real coordinate system and the first image coordinate system can be determined based on the corresponding relationship between the diseased part and the three-dimensional imaging image;

[0121] Next, after the corresponding coordinate transformation relationship is obtained, the first frame coordinate points in the first coordinate group can be transformed accordingly based on the coordinate transformation relationship, thereby obtaining the second frame coordinate points corresponding to the real coordinate system, and then the second frame coordinate points can be determined as the second coordinate group;

[0122] Then, after the acquisition of the second coordinate group is completed, the area position corresponding to the need for two-dimensional ultrasound acquisition can be determined at the diseased part, and the corresponding two-dimensional ultrasound image can be obtained by performing two-dimensional ultrasound acquisition on the area position, and the two-dimensional ultrasound image is further processed by coordinates to obtain a second image coordinate system corresponding to the two-dimensional ultrasound image; here, the second image coordinate system is obtained based on two dimensions;

[0123] Finally, the acquired second image coordinate system can be transformed from the original two-dimensional to three-dimensional, so as to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system. Here, the third image coordinate system is obtained based on the three-dimensional.

[0124] Furthermore, in this embodiment, the above-mentioned “converting the dimension of the two-dimensional ultrasound image based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system” may also include the following steps:

[0125] Performing dimension conversion on the two-dimensional ultrasound image based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system includes:

[0126] Acquire a region schematic diagram corresponding to the two-dimensional acquisition frame, and determine an image viewing angle corresponding to the region schematic diagram;

[0127] Determine a regional surface located in the tumor region and having a parallel relationship with the image viewing angle, and determine a regional thickness corresponding to the tumor region based on the regional surface;

[0128] Based on the region thickness, a Z axis having the same length as the region thickness is established in the second image coordinate system to obtain a third image coordinate system;

[0129] The two-dimensional ultrasound image is transformed based on the Z axis to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

[0130] For example, in this embodiment, since it is necessary to map the three-dimensional tumor tissue to the corresponding two-dimensional ultrasound image in the subsequent process to complete the corresponding image fusion process, it is necessary to ensure that the three-dimensional ultrasound image after the dimension conversion can fully surround the three-dimensional tumor tissue. In this case, it can be achieved based on the following method steps:

[0131] First, since the two-dimensional acquisition frame is obtained based on the regional schematic diagram corresponding to the corresponding image area, the regional schematic diagram corresponding to the two-dimensional acquisition frame can be obtained, and the image viewing angle corresponding to the regional schematic diagram can be determined. Here, the image viewing angle can be a front view angle, a rear view angle, a left view angle, a right view angle, a top view angle, and a bottom view angle;

[0132] Next, after the image viewing angle corresponding to the regional schematic diagram is determined, a regional surface that is parallel to the image viewing angle can be determined in the tumor region, and the regional thickness corresponding to the tumor region can be further determined based on the regional surface; here, the regional thickness can be obtained by determining a vertical line that is perpendicular to the regional surface in the tumor region based on the regional surface, and determining the length of the vertical line as the corresponding regional thickness;

[0133] Next, since the second image coordinate system is established based on the two-dimensional ultrasound image, that is, the second image coordinate system corresponds to two dimensions, and in order to be able to map the tumor tissue based on three dimensions, a Z axis having the same length as the regional thickness can be established in the second image coordinate system based on the acquired regional thickness, thereby obtaining a corresponding third image coordinate system;

[0134] Finally, the two-dimensional ultrasound image can be transformed based on the Z axis to obtain a three-dimensional ultrasound image of the corresponding point image coordinate system. Here, the dimension transformation can be understood as a three-dimensional transformation based on the two-dimensional ultrasound image.

[0135] In step S108, the following contents are included:

[0136] The tumor tissue in the three-dimensional imaging image is mapped to the three-dimensional ultrasound image, and the morphological properties of the tumor tissue in the three-dimensional ultrasound image are obtained.

[0137] For example, in this embodiment, after completing the dimensional conversion of the two-dimensional ultrasound image and obtaining the corresponding three-dimensional ultrasound image, the tumor tissue located in the three-dimensional imaging image can be mapped to the three-dimensional ultrasound image, thereby realizing the corresponding display of the tumor tissue based on the ultrasound image, so as to facilitate subsequent ablation operations; it should be noted that since tumor tissue generally presents an irregular shape, and some tumor tissues may correspond to independent existence, while some tumor tissues may correspond to associated existence, the corresponding associated tumor tissue generally includes multiple associated tumors, in order to be able to plan different trajectory data according to different types of tumor tissues, it is necessary to determine the morphological properties of each tumor tissue.

[0138] Furthermore, in this embodiment, the above-mentioned “obtaining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image” may also include the following steps:

[0139] Determine all concave contour segments respectively having a concave trend located on the tumor tissue along the tissue contour of the tumor tissue, and obtain a first contour endpoint and a second contour endpoint located on the same concave contour segment;

[0140] Connecting the first contour endpoint and the second contour endpoint to obtain an endpoint connection line, and obtaining a line segment distance corresponding to the endpoint connection line;

[0141] Obtaining the concave depth corresponding to the concave contour segment, and retrieving the distance weight value and the depth weight value;

[0142] The normalized line segment distance and the depression depth are respectively multiplied by the distance weight value and the depth weight value to obtain a distance coefficient and a depth coefficient;

[0143] The distance coefficient and the depth coefficient are summed to obtain a concave evaluation coefficient corresponding to the concave contour segment, and each concave evaluation coefficient corresponding to each concave contour segment is compared with a preset threshold value;

[0144] The morphological attributes of the tumor tissue in the three-dimensional ultrasound image are determined based on the comparison result.

[0145] It should be noted that, since the corresponding associated tumor tissue includes parts of the tissue connected between the associated tumors, and corresponding depressions will appear at the corresponding connecting positions, based on this feature, all the concave contour segments with a concave trend on the tissue contour of the tumor tissue can be obtained. Since the tumor tissue itself generally presents an irregular shape, some concave contour segments may be naturally formed. Therefore, it is necessary to screen all the obtained concave contour segments to determine whether there are corresponding ones caused by the associated tumors.

[0146] Here, the corresponding judgment methods include the following:

[0147] First, since the depression caused by the associated tumor generally has a large depth and length, the first contour endpoint and the second contour endpoint located in the same depression contour segment can be obtained, and the two are connected to obtain an endpoint connection line;

[0148] Next, the line segment distance of the corresponding endpoint connection line and the depression depth of the corresponding depression contour segment can be obtained respectively, by calling the corresponding distance weight value and depth weight value, and multiplying the line segment distance and the depression depth with the distance weight value and the depth weight value respectively, to obtain the corresponding distance coefficient and depth coefficient;

[0149] Finally, by summing up the distance coefficient and the depth coefficient, the concavity evaluation coefficient of the corresponding concave contour segment can be obtained, and the corresponding concavity evaluation coefficient is compared with the retrieved preset threshold. According to the comparison result, the morphological properties of the tumor tissue in the three-dimensional ultrasound image can be determined.

[0150] It should be noted that, here, the larger the corresponding concavity evaluation coefficient is, the higher the possibility that the concavity contour segment is generated based on the associated tumor is.

[0151] Furthermore, in this embodiment, the above-mentioned “determining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image based on the comparison result” may also include the following steps:

[0152] When the concave evaluation coefficients respectively corresponding to the concave contour segments are all less than or equal to the preset threshold, determining the morphological attribute of the tumor tissue as an independent attribute;

[0153] When the concave evaluation coefficient corresponding to any concave contour segment is greater than the preset threshold, the concave contour segment is determined as a first target contour segment, and a concave point position corresponding to the concave depth of the first target contour segment is obtained;

[0154] Connecting the concave point corresponding to the first target contour segment and the concave point corresponding to the second target contour segment along the tissue contour to obtain a tissue dividing line for dividing the tumor tissue into associated tumors, and determining the morphological attribute of the tumor tissue as an associated attribute;

[0155] The concave point corresponding to the first target contour segment is connected with the concave point corresponding to the second target contour segment to obtain a tissue division line for dividing the tumor tissue into associated tumors, and the morphological attributes of the tumor tissue are determined as associated attributes.

[0156] For example, in this embodiment, when the concave evaluation coefficients of all concave contour segments corresponding to the tumor tissue are less than or equal to the preset threshold, it indicates that all the concave contour segments are likely to be naturally formed and are not formed based on the associated tumor. Therefore, the morphological attribute of the tumor tissue can be determined as an independent attribute; and when the concave evaluation coefficient corresponding to any concave contour segment is greater than the preset threshold, it indicates that the generation of the concave contour segment is likely to be based on the associated tumor. At this time, it is necessary to divide the tumor tissue accordingly to determine the associated tumor located in the tumor tissue, wherein the corresponding division method can be described as follows:

[0157] First, when a concave contour segment is generated based on an associated tumor, other parts of the tumor tissue should have other concave contour segments that are symmetrical to the concave contour segment. Therefore, the concave contour segment whose corresponding concave evaluation coefficient is greater than a preset threshold can be determined as the first target segment, and the corresponding concave point position can be obtained;

[0158] Then, the intersection point can be used as a starting point to determine other concave contour segments that are closest to the concave point along the tissue contour, and the concave contour segments can be determined as second target contour segments;

[0159] Finally, the corresponding tissue division line can be obtained by connecting the concave points of the first target contour segment and the concave points of the second target contour segment along the tissue contour, wherein the tissue division line can divide the tumor tissue into various associated tumors, so that the morphological attributes of the tumor tissue can be determined as associated attributes.

[0160] It should be noted that, in this embodiment, the tissue demarcation line obtained above can be understood as a tissue line segment corresponding to two adjacent associated tumors, or can also be listed as a tissue line segment corresponding to one associated tumor. By obtaining the corresponding tissue demarcation line, the tumor tissue can be divided to obtain the corresponding associated tumors.

[0161] Here, from the above content, it can be seen that the depression evaluation coefficient is obtained by summing the distance coefficient and the depth coefficient. The distance coefficient is obtained by multiplying the line segment distance by the distance weight value, and the depth coefficient is obtained by multiplying the depression depth by the depth weight value. Based on this, since the morphological attributes determined by the numerical comparison results may have a certain deviation from the actual situation, in order to improve the corresponding determination accuracy, the corresponding weight values ​​can be updated and trained in real time to realize the process of updating and iterating each weight value, thereby continuously improving the corresponding determination accuracy.

[0162] Here, the update iteration of each weight can be achieved through the following steps:

[0163] Acquire the actual attribute of the corresponding tumor tissue sent by the management end based on the tumor tissue, and compare the actual attribute with the morphological attribute;

[0164] If the actual attribute is an independent attribute and the morphological attribute is a companion attribute, the distance weight value and the depth weight value are respectively reduced by training;

[0165] If the actual attribute is a companion attribute and the morphological attribute is an independent attribute, the distance weight value and the depth weight value are respectively increased by training;

[0166] The distance weight value and depth weight value after training are obtained by the following formula:

[0167] ,

[0168] in, is the distance weight value Increase the number of training sessions. is the distance weight value The training constant value of is the distance weight value Reduce the number of training sessions. is the distance weight value after training, is the depth weight value Increase the number of training sessions. is the depth weight value The training constant value of is the depth weight value Reduce the number of training sessions. is the depth weight value after training.

[0169] In step S110, the following contents are included:

[0170] The trajectory data of the electromagnetic navigation needle is determined based on the morphological properties of the tumor tissue.

[0171] For example, in this embodiment, after the corresponding morphological attributes are determined, the corresponding trajectory data may be determined, thereby guiding the electromagnetic navigation needle to puncture the tumor tissue based on the corresponding trajectory data.

[0172] Furthermore, in this embodiment, the above-mentioned “determining the trajectory data of the electromagnetic navigation needle based on the morphological attributes of the tumor tissue” may also include the following steps:

[0173] When the morphological attribute of the tumor tissue is an independent attribute, obtaining a tissue center point corresponding to the tumor tissue, and generating trajectory data passing through the tissue center point;

[0174] When the morphological attribute of the tumor tissue is a companion attribute, each companion tumor included in the tumor tissue is obtained, and each companion center point corresponding to each companion tumor is obtained respectively, and trajectory data of each companion center point corresponding to the electromagnetic navigation needle is generated respectively.

[0175] For example, in this embodiment, when generating corresponding trajectory data, in order to ensure the corresponding ablation effect, it is necessary to determine whether the electromagnetic navigation needle can pass through the tissue center point of the tumor tissue for puncture. If the morphological attribute of the tumor tissue is a companion attribute, the corresponding trajectory data can be customized based on each companion tumor to indicate that the electromagnetic navigation needle can pass through the companion center point of the companion tumor for puncture based on the corresponding trajectory data, thereby improving the corresponding ablation effect.

[0176] According to the scheme of this embodiment, after three-dimensional imaging of the diseased part, a corresponding three-dimensional imaging image including the tumor tissue can be obtained, so that the tumor tissue can be located, so that in the subsequent process, a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window can be determined in the three-dimensional imaging image, and further two-dimensional ultrasound acquisition is performed on the diseased part through the two-dimensional acquisition frame, and the corresponding two-dimensional ultrasound image is converted into a three-dimensional ultrasound image, and then the tumor tissue located in the three-dimensional imaging image can be mapped to the three-dimensional ultrasound image, so that the medical staff can learn the location and morphological properties of the corresponding tumor tissue based on the ultrasound principle, thereby improving the corresponding medical diagnosis assistance; at the same time, in order to facilitate the subsequent ablation operation, this embodiment can also automatically generate the trajectory data of the corresponding electromagnetic navigation needle based on the morphological properties of the tumor tissue, thereby improving the corresponding ablation efficiency.

[0177] Another embodiment of the present invention provides a medical image fusion device, Figure 2 As the corresponding structural block diagram, the device includes:

[0178] A three-dimensional imaging module is configured to perform three-dimensional imaging acquisition on the diseased part of the patient to obtain a three-dimensional imaging image corresponding to the diseased part;

[0179] a frame determination module, configured to identify the tumor tissue in the three-dimensional imaging image, and determine a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image;

[0180] A two-dimensional ultrasound module is configured to perform two-dimensional ultrasound acquisition on the diseased part based on the two-dimensional acquisition framework, and perform dimensional conversion based on the obtained two-dimensional ultrasound image to obtain a three-dimensional ultrasound image;

[0181] a tissue mapping module, configured to map the tumor tissue in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtain the morphological properties of the tumor tissue in the three-dimensional ultrasound image;

[0182] The trajectory determination module is configured to determine the trajectory data of the electromagnetic navigation needle based on the morphological properties of the tumor tissue.

[0183] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the preferred embodiment of the present invention.

[0184] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0185] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0186] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0187] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of submodules or subunits or subcomponents.

[0188] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present invention and to form different embodiments.

[0189] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0190] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0191] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, rather than for explaining or limiting the subject matter of the present invention.

Claims

1. A medical image fusion method, characterized in that: The following steps are involved: Performing three-dimensional imaging acquisition on the patient's diseased part to obtain a three-dimensional imaging image corresponding to the diseased part; Identifying the tumor tissue in the three-dimensional imaging image, and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image; Performing two-dimensional ultrasonic acquisition on the diseased part based on the two-dimensional acquisition framework, and performing dimensional conversion based on the obtained two-dimensional ultrasonic image to obtain a three-dimensional ultrasonic image; Mapping the tumor tissue in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtaining the morphological properties of the tumor tissue in the three-dimensional ultrasound image; Determining trajectory data of the electromagnetic navigation needle based on the morphological attributes of the tumor tissue; Wherein, obtaining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image includes: Determine all concave contour segments respectively having a concave trend located on the tumor tissue along the tissue contour of the tumor tissue, and obtain a first contour endpoint and a second contour endpoint located on the same concave contour segment; Connecting the first contour endpoint and the second contour endpoint to obtain an endpoint connection line, and obtaining a line segment distance corresponding to the endpoint connection line; Obtaining the concave depth corresponding to the concave contour segment, and retrieving the distance weight value and the depth weight value; The normalized line segment distance and the depression depth are respectively multiplied by the distance weight value and the depth weight value to obtain a distance coefficient and a depth coefficient; The distance coefficient and the depth coefficient are summed to obtain a concave evaluation coefficient corresponding to the concave contour segment, and each concave evaluation coefficient corresponding to each concave contour segment is compared with a preset threshold value; The morphological attributes of the tumor tissue in the three-dimensional ultrasound image are determined based on the comparison result.

2. The medical image fusion method according to claim 1, characterized in that: The method includes identifying a tumor tissue located in the three-dimensional imaging image and determining a two-dimensional acquisition frame centered on the tumor tissue and adapted to an ultrasound acquisition window in the three-dimensional imaging image, including: Inputting the three-dimensional imaging image into a trained image recognition model to determine the tumor tissue located in the three-dimensional imaging image and the organ tissue connected to the tumor tissue; Determine a tissue connection surface between the tumor tissue and the organ tissue, and perform coordinate processing on the three-dimensional imaging image based on the tissue connection surface to obtain a first image coordinate system corresponding to the three-dimensional imaging image, wherein the tissue connection surface is parallel to a reference coordinate plane formed between an X-axis and a Y-axis in the first image coordinate system; Determining each contour coordinate point constituting the tissue contour of the tumor tissue in the three-dimensional imaging image after coordinate processing; Determine the coordinate points corresponding to the X-axis coordinate extreme value, the Y-axis coordinate extreme value, and the Z-axis coordinate extreme value in each contour coordinate point as the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, respectively; A tumor region surrounding the tumor tissue is generated based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point, and a two-dimensional acquisition frame centered on the tumor region and adapted to the ultrasound acquisition window is determined in the three-dimensional imaging image.

3. The medical image fusion method according to claim 2, characterized in that: Generating a tumor region surrounding the tumor tissue based on the X-axis extreme value coordinate point, the Y-axis extreme value coordinate point, and the Z-axis extreme value coordinate point includes: Determine a Z-axis minimum value and a Z-axis maximum value based on the Z-axis extreme value coordinate point, and determine a Z-axis length based on the Z-axis minimum value and the Z-axis maximum value; Taking the X-axis extreme value coordinate point as the starting point and extending along the direction parallel to the Y-axis, an X-axis contour line is obtained; Taking the Y-axis extreme value coordinate point as a starting point and extending along a direction parallel to the X-axis, a Y-axis contour line intersecting with the X-axis contour line is obtained; A tumor coordinate plane corresponding to the tumor tissue is determined based on the X-axis contour line and the Y-axis contour line, and a surface shape having the same length as the Z-axis length is extended toward the Z-axis extreme coordinate point based on the tumor coordinate plane to obtain a tumor area surrounding the tumor tissue.

4. The medical image fusion method according to claim 2, characterized in that: Determining a two-dimensional acquisition frame centered on the tumor region and adapted to an ultrasound acquisition window in the three-dimensional imaging image includes: Acquire a region center point located in the tumor region, and acquire images corresponding to the ultrasound acquisition window at multiple image viewing angles of the tumor region in a direction parallel to the reference coordinate plane as a reference direction, to obtain region schematic diagrams corresponding to different image viewing angles, wherein the image center point of each region schematic diagram coincides with the region center point; Determine that each regional schematic diagram corresponds to each image portion of the tumor tissue, and determine the area of ​​each image portion to obtain each image area corresponding to each regional schematic diagram; The schematic diagram of the region with the largest corresponding image area is determined as a two-dimensional acquisition frame adapted to the ultrasound acquisition window.

5. The medical image fusion method according to claim 4, characterized in that: Performing two-dimensional ultrasonic acquisition on the diseased part based on the two-dimensional acquisition framework, and performing dimensional conversion based on the obtained two-dimensional ultrasonic image to obtain a three-dimensional ultrasonic image, including: Coordinate processing is performed on the diseased part, and a real coordinate system corresponding to the diseased part is formed; Determining the first frame coordinate points corresponding to the two-dimensional acquisition frame as a first coordinate group in the first image coordinate system, and determining a coordinate conversion relationship between the real coordinate system and the first image coordinate system; Based on the coordinate transformation relationship, coordinate transformation is performed on each first frame coordinate point in the first coordinate group, and each obtained second frame coordinate point is determined as a second coordinate group corresponding to the real coordinate system; Performing two-dimensional ultrasonic acquisition on the diseased part based on the second coordinate group, and performing coordinate processing on the obtained two-dimensional ultrasonic image to obtain a second image coordinate system corresponding to the two-dimensional ultrasonic image; The two-dimensional ultrasound image is dimensionally transformed based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

6. The medical image fusion method according to claim 5, characterized in that: Performing dimension conversion on the two-dimensional ultrasound image based on the second image coordinate system to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system includes: Acquire a region schematic diagram corresponding to the two-dimensional acquisition frame, and determine an image viewing angle corresponding to the region schematic diagram; Determine a regional surface located in the tumor region and having a parallel relationship with the image viewing angle, and determine a regional thickness corresponding to the tumor region based on the regional surface; Based on the region thickness, a Z axis having the same length as the region thickness is established in the second image coordinate system to obtain a third image coordinate system; The two-dimensional ultrasound image is transformed based on the Z axis to obtain a three-dimensional ultrasound image corresponding to the third image coordinate system.

7. The medical image fusion method according to claim 1, characterized in that: Determining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image based on the comparison result includes: When the concave evaluation coefficients respectively corresponding to the concave contour segments are all less than or equal to the preset threshold, determining the morphological attribute of the tumor tissue as an independent attribute; When the concave evaluation coefficient corresponding to any concave contour segment is greater than the preset threshold, the concave contour segment is determined as a first target contour segment, and a concave point position corresponding to the concave depth of the first target contour segment is obtained; Taking the depression point as a starting point, determine other depression contour segments that are closest to the depression point along the tissue contour, and determine the depression contour segments as second target contour segments; The concave points corresponding to the first target contour segment and the concave points corresponding to the second target contour segment are connected along the tissue contour to obtain tissue division lines that divide the tumor tissue into associated tumors, and the morphological attributes of the tumor tissue are determined as associated attributes.

8. The medical image fusion method according to claim 7, characterized in that: Determining trajectory data of the electromagnetic navigation needle based on the morphological attributes of the tumor tissue includes: When the morphological attribute of the tumor tissue is an independent attribute, obtaining a tissue center point corresponding to the tumor tissue, and generating trajectory data passing through the tissue center point; When the morphological attribute of the tumor tissue is a companion attribute, each companion tumor included in the tumor tissue is obtained, and each companion center point corresponding to each companion tumor is obtained respectively, and trajectory data of each companion center point corresponding to the electromagnetic navigation needle is generated respectively.

9. A medical image fusion device, characterized in that: include: A three-dimensional imaging module is configured to perform three-dimensional imaging acquisition on the diseased part of the patient to obtain a three-dimensional imaging image corresponding to the diseased part; a frame determination module, configured to identify the tumor tissue in the three-dimensional imaging image, and determine a two-dimensional acquisition frame centered on the tumor tissue and adapted to the ultrasound acquisition window in the three-dimensional imaging image; A two-dimensional ultrasound module is configured to perform two-dimensional ultrasound acquisition on the diseased part based on the two-dimensional acquisition framework, and perform dimensional conversion based on the obtained two-dimensional ultrasound image to obtain a three-dimensional ultrasound image; a tissue mapping module, configured to map the tumor tissue in the three-dimensional imaging image to the three-dimensional ultrasound image, and obtain the morphological properties of the tumor tissue in the three-dimensional ultrasound image; A trajectory determination module, configured to determine trajectory data of the electromagnetic navigation needle based on the morphological properties of the tumor tissue; Wherein, obtaining the morphological attributes of the tumor tissue in the three-dimensional ultrasound image includes: Determine all concave contour segments respectively having a concave trend located on the tumor tissue along the tissue contour of the tumor tissue, and obtain a first contour endpoint and a second contour endpoint located on the same concave contour segment; Connecting the first contour endpoint and the second contour endpoint to obtain an endpoint connection line, and obtaining a line segment distance corresponding to the endpoint connection line; Obtaining the concave depth corresponding to the concave contour segment, and retrieving the distance weight value and the depth weight value; The normalized line segment distance and the depression depth are respectively multiplied by the distance weight value and the depth weight value to obtain a distance coefficient and a depth coefficient; The distance coefficient and the depth coefficient are summed to obtain a concave evaluation coefficient corresponding to the concave contour segment, and each concave evaluation coefficient corresponding to each concave contour segment is compared with a preset threshold value; The morphological attributes of the tumor tissue in the three-dimensional ultrasound image are determined based on the comparison result.

Images