An intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery

Abstract

The present invention belongs to the field of medical image processing technology, specifically an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery, comprising the following steps: Step S1, measuring key geometric size parameters of the nodule; Step S2, establishing an approximate three-dimensional model of the nodule; Step S3, aligning the approximate three-dimensional model with the ultrasound image; Step S4, completing the acquisition of nodule edge position data of the current ultrasound image; Step S5: calculating the nodule edge position data; Step S6, creating a high-precision three-dimensional digital model; Step S7, creating a three-dimensional intraoperative navigation and positioning system, and performing block ablation on the nodule. The present invention solves the technical problems of excessive and insufficient ablation that have always existed in thyroid nodule thermal ablation surgery. It optimizes the existing operating methods of thyroid nodule thermal ablation surgery, provides the surgeon with a powerful intraoperative navigation and positioning tool, and significantly improves the accuracy and success rate of thyroid nodule thermal ablation surgery.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and specifically relates to an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery. Background Art

[0002] Thermal ablation of thyroid nodules is a minimally invasive thyroid treatment technique primarily used to treat benign thyroid nodules and some low-risk papillary carcinomas. This procedure typically involves puncturing the thyroid nodule with an ablation needle under ultrasound guidance. Microwaves or high-frequency currents are used to heat the tumor or nodule tissue, causing coagulative necrosis, thereby shrinking and shedding tissue cells and achieving aseptic necrosis. The advantage of thermal ablation of thyroid nodules lies in its minimally invasive nature, which preserves thyroid function while causing minimal damage to surrounding tissues. It is also simple, safe, and effective.

[0003] Although thermal ablation of thyroid nodules has many advantages, it also has certain risks. Since there are abundant nerves and blood vessels around the thyroid gland, the thermal effect during ablation treatment may damage these tissues, leading to postoperative complications such as coughing when drinking water, swallowing difficulties, hoarseness, or damage to adjacent blood vessels causing intraoperative bleeding. In addition, if malignant nodules are not completely eliminated, there may be a risk of recurrence or metastasis. Due to the vaporization phenomenon produced during the ablation process, the boundaries of the thyroid nodules will become blurred and difficult to identify under ultrasound images. The surgeon's experience is required to judge the effective range of the thermal effect, which can easily lead to excessive or insufficient ablation. Therefore, how to accurately locate the three-dimensional spatial position of the nodule and accurately capture the edge of the nodule during surgery, and present it to the surgeon in an intuitive way to avoid excessive or insufficient ablation, has become a technical problem that needs to be solved urgently in clinical practice. Summary of the Invention

[0004] The purpose of the present invention is to provide an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery, so as to solve the problem in the prior art proposed in the background technology, in which the vaporization phenomenon generated during the ablation process makes the edge of the thyroid nodule blurred under the ultrasound image and difficult to identify. It is necessary to rely on the operator's experience to judge the effective range of the thermal effect, which easily leads to excessive or insufficient ablation.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery includes the following steps:

[0007] Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule;

[0008] Step S2, establishing an approximate three-dimensional model of the nodule using the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model, which serves as a basic model for subsequent high-precision digital nodule modeling;

[0009] Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the ultrasound image;

[0010] Step S4: projecting the approximate three-dimensional model onto the current ultrasound image, and using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, thereby completing the nodule edge position data acquisition of the current ultrasound image;

[0011] Step S5, repeating step S4 to solve the nodule edge position data on each layer of the ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, generating high-precision three-dimensional point cloud data of the nodule;

[0012] Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5;

[0013] In step S7, the high-precision three-dimensional digital model of the nodule created in step S6 is used in combination with the location of the real-time ultrasound image during the operation to generate the outline of the nodule and superimpose it on the ultrasound image to enhance the ultrasound image in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and grid-based partitioned ablation is performed layer by layer until the entire nodule is ablated.

[0014] According to the above technical solution, in step S1, the key geometric size parameters of the nodule include the maximum longitudinal diameter W, the maximum transverse diameter H and the maximum anterior-posterior diameter L of the nodule.

[0015] According to the above technical solution, the volume V of the nodule is calculated based on the key geometric size parameters of the nodule measured in step S1 as follows:

[0016]

[0017] Where W is the maximum long diameter, H is the maximum transverse diameter, and L is the maximum front-back diameter.

[0018] According to the above technical solution, in step S3, the nodule center registration method is specifically as follows:

[0019] Step S301 , establishing a Cartesian coordinate system (X, Y, Z) with the center of the approximate three-dimensional model as the origin. In this coordinate system, the center coordinates of the nodule are (0, 0, 0);

[0020] Step S302: Extract the closed edge of the nodule on the ultrasound image and calculate the center coordinates of the nodule. The specific method is as follows: first, perform ROI segmentation, filtering, and contrast enhancement on the ultrasound image to make the nodule edge clearer for subsequent processing. Then, use a pre-trained neural network based on a deep learning method (such as Mask R-CNN, etc.) to detect the image and identify the thyroid nodule area. These algorithms are often trained with a large amount of labeled data and can accurately identify the characteristics of thyroid nodules and automatically detect similar structures in new ultrasound images. After the nodule is detected, the nodule is segmented from the background image using image segmentation techniques (such as the level set algorithm, etc.). Then, an edge detection algorithm (such as the Canny edge detection operator, the non-maximum suppression algorithm, etc.) is used to extract the edge coordinate point set of the nodule. The point set is shown below:

[0021]

[0022] Finally, the center coordinates of the nodule (C x , C y , C z ):

[0023]

[0024] Step S303: Move the center of the thyroid nodule on the ultrasound image to the center of the approximate three-dimensional model of the nodule to complete the registration of the nodule center. The specific method is: first construct a translation and rotation matrix (homogeneous coordinates are generally used in three-dimensional space). The translation matrix is:

[0025]

[0026] Assume that during the registration process, the image is rotated around the x, y, and z axes by , the rotation matrices in the three directions are , as shown below:

[0027]

[0028] The center coordinates of the nodule on the ultrasound image (C x , C y , C z ) transformation to align its center with the center (0, 0, 0) of the nodule approximate 3D model. The transformation is as follows:

[0029]

[0030] Furthermore, the intersection line between the ultrasound image and the approximate three-dimensional model of the nodule is calculated as the initial contour edge of the nodule.

[0031] According to the above technical solution, in step S4, the projection curve is fitted to the nodule edge of the ultrasound image using an image algorithm as follows:

[0032] Step S401: first extract the region containing the nodule on the ultrasound image as the ROI region;

[0033] Step S402: Filter the ROI area using a Gaussian convolution kernel to eliminate the influence of noise, and then solve the partial derivatives in the x and y directions of the filtered image to extract the vertical and horizontal edge information of the ultrasound image. Then calculate the gradient of the image. :

[0034]

[0035] Where, represents the image gradient, represents the partial derivative in the vertical direction of the image, Represents the partial derivative of the image in the horizontal direction; the gradient amplitude of the ultrasound image can be expressed as the following formula.

[0036]

[0037] Where, is the image gradient amplitude; The larger the value is, the greater the gray value on the gradient image is, and the greater the possibility that the pixel is the edge of the nodule. At the same time, the gradient direction can be expressed as the following formula:

[0038]

[0039] Where, is the image gradient direction, indicating the direction of the nodule edge;

[0040] Step S403: Initialize the contour curve and its control points and construct the energy equation of the contour curve; specifically:

[0041] Step S4031: The intersection line of the approximate 3D model and the ultrasound image after the nodule center is aligned in step S303 is used as the initial edge contour of the nodule. , and set equidistant control points on the initial contour curve;

[0042] In step S4032, the edge contour of the nodule can be expressed as a functional form in the following form:

[0043]

[0044] Where s is the length of the curve described in Fourier transform form; and Represent the coordinate position of each control point in the image, and then construct the energy functional form based on the control points, specifically:

[0045]

[0046] Where E represents the total energy of the nodule contour edge, E i represents the image energy, E s represents the bending energy, E e represents elastic energy;

[0047] Step S404, finding the minimum value of the energy function; specifically comprising the following steps:

[0048] Step S4041, calculating the initial energy E0;

[0049] Step S4042, calculating the moving direction of the current control point and moving the control point;

[0050] Step S4043, recalculate the contour curve energy and determine whether the calculated contour curve energy is less than the threshold value. If yes, end the process; if not, return to step S4042 and recalculate.

[0051] According to the above technical solution, in step S5, high-precision three-dimensional point cloud data of nodules is generated as follows:

[0052] After completing the fitting of the contour curve of the current ultrasound image of the approximate three-dimensional model to the edge of the nodule, the ultrasound probe is moved to the next layer of the approximate three-dimensional model, and the method in step S4 is repeated until the fitting of the nodule edges on all layers is completed, thereby obtaining high-precision three-dimensional point cloud data C of the nodule;

[0053]

[0054] Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set consisting of the edge points of all layers of the nodule on the ultrasound image (assuming a total of k points).

[0055] According to the above technical solution, in step S6, the calculated high-precision three-dimensional point cloud data of the nodule is gridded to obtain a three-dimensional geometric model of the nodule; specifically, a rectangular area is used to contain the entire point cloud space, and the point cloud is divided into several small rectangular areas. The points in each small rectangular area are continuous, and the point cloud in each small rectangular area is gridded, and then all areas are assembled together to form a triangulated mesh model of the entire nodule.

[0056] According to the above technical solution, meshing the point cloud within each small rectangular area includes the following steps:

[0057] Step S601: Select an initial point, denoted as P1, search for the nearest point of P1 within the region, denoted as P2, connect P1 and P2, and select a third point, P3, to form the initial triangle. To ensure the quality of the triangle, P3 should meet the following requirements:

[0058]

[0059] Step S602: After the initial triangular face is generated, step S601 is repeated for each edge in the normal direction, so that the face mesh continues to grow outward until the mesh reconstruction of the entire nodule edge point cloud is completed.

[0060] According to the above technical solution, in step S7, the layer of nodules is evenly divided into multiple grids, and ablation is performed grid by grid, specifically including the following steps:

[0061] Step S701: Locate the current ablation position. Since the two-dimensional ultrasound scanning used for thermal ablation of thyroid nodules is 2D imaging, only ultrasound images of a certain layer of the nodule can be observed through two-dimensional ultrasound scanning. Therefore, it is necessary to first locate one layer for ablation, and then move to the next layer after ablation is completed.

[0062] Step S702: generating a nodule outline based on the current ultrasound image position and the high-precision three-dimensional digital model of the nodule, and superimposing the outline on the ultrasound image to enhance the ultrasound image in real time;

[0063] Step S703, using the enhanced ultrasound image to perform precise ablation;

[0064] Step S704: locate the next layer and repeat steps S701 to S703 until the ablation of the entire nodule is completed.

[0065] According to the above technical solution, the ultrasound image is enhanced; the specific method is:

[0066] Step S7021: After locating the ablation position, the intersection line between the current ablation layer and the high-precision three-dimensional digital model of the nodule is calculated to obtain the navigation contour line of the nodule edge in the current ultrasound image;

[0067] In step S7022, assuming that the transverse direction is the Z direction, the current ultrasound image plane can be expressed by the following formula:

[0068]

[0069] Where c is a constant;

[0070] In step S7023, since the high-precision three-dimensional digital model of the nodule is composed of triangular surfaces, the surface of the high-precision three-dimensional digital model of the nodule cannot be expressed by an explicit parameterized equation. The nodule edge surface can be expressed as a point set, specifically:

[0071]

[0072] Step S7024, solving the discrete nodule edge curve on the current ultrasound image, specifically:

[0073]

[0074] in, It represents the intersection of the plane where the ultrasound image is located and the three-dimensional model of the nodule, that is, the contour line of the nodule on the current ultrasound image, and p is a specific point on the nodule contour line;

[0075] Step S7025: Divide the ablation area into multiple independent small areas using a fixed spacing to obtain an auxiliary ablation navigation image, and superimpose the auxiliary ablation navigation image on the original ultrasound image to obtain an enhanced ultrasound image.

[0076] Compared with the prior art, the present invention has the following beneficial effects:

[0077] The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a 3D digital model of the thyroid nodule through ultrasound scanning before the operation, and further creates an intraoperative navigation system based on the 3D model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge curve of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0079] Figure 2 This is a flowchart of the precise thyroid ablation procedure using enhanced ultrasound images according to the present invention;

[0080] Figure 3 This is a schematic diagram of the key dimensions of nodules measured using two-dimensional ultrasound in the present invention; Figure 3 (A) is a schematic diagram of the maximum long diameter W and maximum transverse diameter H of the nodule measured by two-dimensional ultrasound images. Figure 3 (B) is a schematic diagram of the maximum anteroposterior diameter L of the nodule measured using a two-dimensional ultrasound image;

[0081] Figure 4 This is a standard 3D model diagram of a thyroid nodule according to the present invention;

[0082] Figure 5 This is a flowchart of the registration of the nodule 3D model and ultrasound image of the present invention;

[0083] Figure 6 The standard ellipsoid model of thyroid nodules and the ultrasound image registration diagram of the present invention;

[0084] Figure 7 This is a flow chart of nodule edge curve fitting in the present invention;

[0085] Figure 8 This is the ROI area map of the thyroid nodule of the present invention;

[0086] Figure 9 Schematic diagram of the initial contour curve and control points of the present invention;

[0087] Figure 10 Schematic diagram of the moving direction of the control point of the present invention;

[0088] Figure 11 This is a flow chart of iterative calculation of contour curve energy of the present invention;

[0089] Figure 12 This is a schematic diagram of gridding point cloud data of the present invention;

[0090] Figure 13 This is a flowchart of the precise ablation based on enhanced ultrasound images of the present invention;

[0091] Figure 14 Schematic diagram of projection of node cloud onto ultrasound image according to the present invention;

[0092] Figure 15 The least square method is used to fit the nodule edge curve graph in the present invention;

[0093] Figure 16 This is the auxiliary ablation navigation map of the present invention;

[0094] Figure 17 Ultrasound images are enhanced for the present invention. DETAILED DESCRIPTION

[0095] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0096] Example 1

[0097] like Figure 1As shown, an intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery includes the following steps:

[0098] Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule;

[0099] Step S2, establishing an approximate three-dimensional model of the nodule using the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model, which serves as a basic model for subsequent high-precision digital nodule modeling;

[0100] Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the ultrasound image;

[0101] Step S4: projecting the approximate three-dimensional model onto the current ultrasound image, and using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, thereby completing the nodule edge position data acquisition of the current ultrasound image;

[0102] Step S5, repeating step S4 to solve the nodule edge position data on each layer of the ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, generating high-precision three-dimensional point cloud data of the nodule;

[0103] Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5;

[0104] In step S7, the high-precision three-dimensional digital model of the nodule created in step S6 is used in combination with the location of the real-time ultrasound image during the operation to generate the outline of the nodule and superimpose it on the ultrasound image to enhance the ultrasound image in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and grid-based partitioned ablation is performed layer by layer until the entire nodule is ablated.

[0105] The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a three-dimensional digital model of the thyroid nodule through ultrasound scanning before the operation, and further creates an intraoperative navigation system based on the three-dimensional model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge contour of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery.

[0106] Example 2

[0107] This invention utilizes mathematical modeling and image analysis techniques to rapidly reconstruct thyroid nodules in three dimensions and enhances intraoperative ultrasound images based on the reconstructed digital nodule model. The enhanced ultrasound image and virtual digital nodule model allow for spatial tracking, localization, and visualization of the thyroid nodule's edge contours throughout the ablation process, effectively resolving the technical challenges of over- and under-ablation while also providing the surgeon with more precise surgical navigation.

[0108] The specific implementation route of the method of the present invention is as follows Figure 2 As shown in the figure, the process consists of seven main steps. S1: A preoperative 2D ultrasound scan is performed on the patient to measure the three key geometric parameters of the nodule. S2: Using the key parameters obtained in step S1, an approximate ellipsoid model of the nodule is constructed, serving as the basis for subsequent high-precision digital nodule model construction. S3: The center of the ellipsoid model is aligned with the nodule center on the ultrasound image. S4: The approximate ellipsoid model of the nodule is projected onto the current ultrasound image. Since the geometric center of the projection curve coincides with the center of the nodule on the ultrasound image, an image algorithm is used to fit the projection curve to the nodule boundary curve on the ultrasound image, completing the nodule contour data acquisition for the current ultrasound image. S5: Move to the next layer and repeat step S4 until the 3D contour spatial data of the entire nodule is acquired, generating high-precision point cloud data for the nodule. S6: Using the point cloud data generated in step S5, a high-precision 3D digital model of the nodule is created. S7: Positioning the first layer (the layer with the largest diameter), the nodule region in this layer is evenly divided into multiple grids, guided by the 3D digital model of the nodule, and ablation is performed grid by grid. Because the contour edge of the nodule layer can be calculated through the 3D digital model of the nodule, the contour boundary of the nodule can be clearly outlined on the enhanced ultrasound image, thus avoiding excessive or insufficient ablation. This step is then repeated for each layer until the entire nodule is ablated.

[0109] S1, preoperative diagnosis, measurement of key size parameters of nodules by ultrasound images.

[0110] To create a three-dimensional digital model of a nodule, we first need to obtain the nodule's key geometric parameters. Since an ellipsoid is often used to approximate the nodule's shape in clinical practice, measuring the nodule's maximum longitudinal diameter (W), maximum transverse diameter (H), and maximum anteroposterior diameter (L) is sufficient. These parameters can be obtained using two-dimensional ultrasound imaging. The steps are as follows:

[0111] (1) Patient preparation: The patient lies supine with the head tilted back to fully expose the thyroid area of ​​the neck.

[0112] (2) Ultrasound equipment preparation: Select a high-resolution two-dimensional ultrasound probe and adjust appropriate ultrasound parameters.

[0113] (3) Nodule localization: The ultrasound probe is used to find and locate the nodule in the thyroid region. On ultrasound images, the nodule usually appears as a low-echo or isoechoic mass.

[0114] (4) Measure the maximum length / transverse diameter: The ultrasound probe is placed in the transverse position. On the two-dimensional ultrasound image, use a measuring tool (such as an electronic cursor) to measure the maximum length W and maximum transverse diameter H of the nodule, such as Figure 3 As shown in (A).

[0115] (5) Measure the maximum anteroposterior diameter: The ultrasound probe is in the sagittal position and the maximum anteroposterior diameter L of the nodule is measured. Figure 3 As shown in (B).

[0116] S2, build an approximate three-dimensional model of the nodule based on the nodule size parameters and calculate its volume.

[0117] According to the maximum longitudinal diameter W, maximum transverse diameter H and maximum anteroposterior diameter L of the nodule obtained in step S1 above, an approximate three-dimensional ellipsoid model of the nodule is established, as shown in FIG. Figure 4 The volume V of a nodule is generally calculated using the following formula.

[0118]

[0119] Where W is the maximum long diameter, H is the maximum transverse diameter, and L is the maximum front-back diameter.

[0120] S3, Registration of the nodule approximate 3D model with the ultrasound image.

[0121] In this step, the center of the ellipsoid model of the thyroid nodule is aligned with the ultrasound image. A Cartesian coordinate system (X, Y, Z) is established with the center of the ellipsoid model as the origin, so the center coordinates of the nodule are (0, 0, 0). All subsequent operations on the ultrasound image and nodule model are performed in this coordinate system. Figure 5 As shown, this step is divided into two sub-steps. S31 extracts the closed boundary of the thyroid nodule on the ultrasound image and calculates the center coordinates. S32 calculates the rotation and translation matrix to complete the alignment of the image center. The specific method is: first, the ultrasound image is segmented, filtered, and contrast enhanced to make the nodule edge clearer for subsequent processing; then, the image is detected by a pre-trained neural network based on a deep learning method (such as MaskR-CNN, etc.) to identify the thyroid nodule area; these algorithms are often trained with a large amount of labeled data and can accurately identify the characteristics of the thyroid nodule and automatically detect similar structures in new ultrasound images. After the nodule is detected, the nodule is segmented from the background image using image segmentation technology (such as the level set algorithm, etc.); then, an edge detection algorithm (such as the Canny edge detection operator, the non-maximum suppression algorithm, etc.) is used to extract the edge coordinate point set of the nodule. The point set is shown below:

[0122]

[0123] Finally, the center coordinates of the nodule (C x , C y , C z ):

[0124]

[0125] The center of the thyroid nodule on the ultrasound image is moved to the center of the nodule approximate 3D model to complete the nodule center registration. The specific method is: first construct the translation and rotation matrix, where the translation matrix is:

[0126]

[0127] Assume that during the registration process, the image is rotated around the x, y, and z axes by , the rotation matrices in the three directions are , as shown below:

[0128]

[0129] The center coordinates of the nodule on the ultrasound image (C x , C y , C z ) transformation to align its center with the center (0, 0, 0) of the nodule approximate 3D model. The transformation is as follows:

[0130]

[0131] Furthermore, the intersection line between the ultrasound image and the nodule approximate 3D model is calculated as the initial contour edge of the nodule; the image after registration is as follows Figure 6 shown.

[0132] S4, Use an image algorithm to fit the projection curve of the standard model to the edge of the nodule.

[0133] On ultrasound images, the edge of a thyroid nodule generally presents a visible elliptical closed outline, such as Figure 3 As shown in Figure 2. The contour has a clear gradient on the ultrasound image, so the edge detection method can be used to extract the nodule edge to make it more obvious. Figure 5 ) as the initial contour, and use the numerical iterative algorithm to make this curve gradually approach the edge of the nodule until it finally converges and completely coincides with the edge of the nodule. This step is divided into four sub-steps, such as Figure 7 shown.

[0134] S401, image preprocessing (ROI extraction, noise reduction, calculation of image gradient amplitude).

[0135] The ultrasound image of thyroid examination contains a lot of tissue information, including CCA (common carotid artery), esophagus, lymph nodes, etc. In order to reduce the computational workload, the region containing the thyroid nodule is first extracted as the ROI region, such as Figure 8 shown.

[0136] Filtering the ROI region using a Gaussian convolution kernel can eliminate the effects of noise. The partial derivatives of the filtered image in the x and y directions are then calculated to calculate the image gradient magnitude. To speed up the calculation, it is preferable to first calculate the partial derivatives of the Gaussian filter to obtain a Gaussian partial derivative filter. The Gaussian partial derivative filter is then convolved with the image to calculate the image gradient. Formula (5) describes the process of convolving the Gaussian partial derivative filter with the image to calculate the gradient.

[0137]

[0138] In the above formula, express Figure 8 The ultrasound image shown, Represents the Gaussian convolution kernel. Formula (5) indicates that first performing a Gaussian filter on the image and then calculating the partial derivative is equivalent to first calculating the partial derivative of the Gaussian filter and then performing convolution with the image, but it can greatly reduce the amount of calculation. Formula (6) is the formula for calculating the image gradient.

[0139]

[0140] Formulas (7) and (8) are the calculation formulas for gradient magnitude and gradient direction respectively. The larger it is, the greater the gray value at that location on the gradient image, and the greater the possibility that the pixel is the edge of a nodule. Indicates the direction of the edge of the nodule at that location.

[0141]

[0142] S402 , setting an initialization contour curve S0 and its control points, and constructing an energy equation of the contour curve.

[0143] In this step, the contour line of the nodule standard ellipsoid model generated in step S32 is used as the initial contour curve S0, and equidistant control points are set, such as Figure 9 shown.

[0144] The contour curve can be expressed as a functional form in the form of the following equation (9), where s is the length of the curve described in Fourier transform form. and Represent the coordinate position of each control point in the image.

[0145]

[0146] The energy functional can be constructed based on the control points of formula (9), as shown in formula (10):

[0147]

[0148] Among them, E represents the total energy of the contour curve, which consists of three parts of energy: image energy E i , bending energy E s and elastic energy E e . Among them, the image energy E i is the external energy, which indicates the degree of fitting between the nodule contour edge curve S0 and the nodule edge, and is used to control the contour edge curve to approach the target edge on the ultrasound image; the image energy E i The smaller the value, the better the fit; elastic energy E e and bending energy E s It is used to control the elastic deformation of the contour curve and maintain the continuity and smoothness of the contour curve. The minimum value of E is solved by an iterative algorithm to obtain the stable contour curve, that is, the edge curve of the nodule.

[0149] An iterative algorithm can be used to find the minimum value of E (such as Figure 11 As shown in the figure), the stable contour curve, that is, the edge contour of the nodule, can be obtained. Formula (11) is the calculation formula for image energy. is a Gaussian filter, Find the gradient, and then convolve this gradient operator with the control points on the curve to solve the image energy. Note that the curve will evolve in the direction of the maximum gradient, so in order to solve To obtain the minimum value of , the energy needs to be inverted.

[0150]

[0151] However, in the process of initial curve evolution, the curve should be kept smooth and elastic as much as possible, so two constraints E are introduced. e and E s , see formula (9). E e The first-order derivative of the contour functional is used to maintain the elasticity of the curve, E s The second derivative of the contour functional is used to keep the curve smooth.

[0152]

[0153] Formula (12) is a partial differential equation, which is discretized at each control point to obtain a computable form:

[0154]

[0155] Substitute formula (11) and formula (13) into formula (10), and then introduce elastic energy and bending energy Their respective influence coefficients and , we get the final calculable energy formula:

[0156]

[0157] S403, finding the minimum value of the energy function.

[0158] For each control point in the above formula (14), calculate its moving direction and iteratively solve the minimum value of the energy function until the contour curve coincides with the edge of the thyroid nodule. Figure 10 middle, are three consecutive control points on the contour curve. Assume that the control points Move only one pixel at a time. They are 8 possible movement directions. To connect control points and The vector of To connect control points and Vector. Respectively with vector and The vector with the smallest cross product is the control point Direction of movement .

[0159]

[0160] S5, solve the nodule edge contour curve on each layer of ultrasound image and create a three-dimensional point cloud of the nodule.

[0161] After completing the fitting of the contour curve of the current ultrasound image and the edge of the thyroid nodule, the ultrasound probe is moved to the next layer and the method of step S4 above is repeated until the fitting of the thyroid nodule edge on all layers is completed to obtain the three-dimensional point cloud C of the nodule. The calculation of the three-dimensional point cloud C is shown in the following formula.

[0162]

[0163] Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set consisting of the edge points of all layers of the nodule on the ultrasound image (assuming a total of k points).

[0164] S6, Build a 3D model of the thyroid gland using point cloud data.

[0165] To visualize the thyroid nodule's three-dimensional model more intuitively, the calculated nodule edge point cloud data is meshed to create a three-dimensional geometric model of the nodule. Using a modular meshing method allows for multi-CPU parallel computing, accelerating the meshing process. Specifically, a rectangular region encompasses the entire point cloud space, dividing the point cloud into several small cubes. The points within each small region are continuous. Meshing and reassembling the point cloud within each small region forms a triangulated mesh of the entire nodule.

[0166] The meshing method within each region consists of the following two steps:

[0167] Step 1: Select an initial point, denoted as P1, search for the closest point to P1 within the region, denoted as P2, connect P1 and P2, and select a third point, P3, to form the initial triangle. To ensure the quality of the triangle, the following requirements (17) and (18) must be met simultaneously. Figure 12 Schematic diagram of generating a mesh from point cloud data.

[0168]

[0169] Step 2: After the initial triangle surface is generated, repeat this step for each edge in the normal direction until the mesh reconstruction of the entire nodule edge point cloud is completed.

[0170] S7, created a three-dimensional navigation system to complete precise ablation of the entire nodule based on enhanced ultrasound images.

[0171] In this step, the 3D model of the thyroid nodule generated in the previous steps is used to create a navigation system, enhance the original ultrasound image, and perform precise ablation based on the enhanced ultrasound image. This step is completed in four substeps: S71, locate the current ablation position; S72, generate a navigation system based on the current ultrasound image position and the nodule 3D template, and enhance the ultrasound image; S73, use the enhanced ultrasound image to perform precise ablation; S74, locate the next layer, and repeat the above steps until the entire nodule is ablated. Figure 13 The flowchart of this step is shown in Figure 2.

[0172] S71, positioning the current ablation layer.

[0173] Thermal ablation of thyroid nodules is typically performed under ultrasound guidance. Because ultrasound uses 2D imaging, the surgeon can only visualize a specific layer of the nodule. Therefore, the surgeon will target a specific layer for ablation, then move on to the next layer after completing the current ultrasound image, until the entire nodule is ablated. During the procedure, the surgeon typically begins ablation at the layer with the largest diameter in the transverse plane.

[0174] S72: Generate a navigation system based on the current ultrasound image position and the three-dimensional model of the nodule to enhance the ultrasound image.

[0175] After locating the ablation position, the intersection line between the current ablation layer and the generated 3D model of the thyroid nodule is calculated to obtain the thyroid margin navigation curve of the current ultrasound image. Assuming that the transverse direction is the Z direction, the current ultrasound image plane can be expressed as the following formula (19), where c is a constant.

[0176]

[0177] Since the 3D model of a thyroid nodule consists of triangular surfaces generated from a point cloud, the 3D surface of the nodule cannot be represented by an explicit parameterized equation. The edge surface of the nodule can be represented as a point set, as shown in the following equation (20).

[0178]

[0179] Combining equations (19) and (20) can solve the discrete nodule edge curve on the current ultrasound image, as shown in equation (21). The greater the density of the point cloud, the more accurate the nodule edge curve.

[0180]

[0181] Preferably, a dynamically adjustable threshold value is set to represent the distance from the layer where the current ultrasound image is located. Points within the threshold value are projected onto the ultrasound image to obtain a set of edge points of the nodule. Figure 14 Indicates that the points within the threshold interval are projected onto the ultrasound image.

[0182] For the nodule edge point set projected onto the ultrasound image, if these points are regular and continuous, they can be directly connected with straight lines to generate edge curves. Otherwise, the least squares method can be used to fit the edge curve of the nodule, such as Figure 15 shown.

[0183] Furthermore, the ablation area is divided into multiple independent small areas using a fixed spacing, such as Figure 16 shown.

[0184] The auxiliary ablation navigation image is superimposed on the original ultrasound image to obtain an enhanced ultrasound image, such as Figure 17 During the ablation process, the surgeon ablates each divided area one by one. The real-time displayed edge curve can also greatly avoid the problem of blurred nodule edges caused by vaporization, thereby avoiding the problems of over-ablation and under-ablation.

[0185] S73, locate to the next layer and complete the ablation of the entire thyroid nodule.

[0186] After completing the ablation of the current ultrasound image, the surgeon locates the next layer and repeats the steps of the previous layer until the precise ablation of the entire thyroid nodule is completed.

[0187] It should be noted that, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0188] The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a three-dimensional digital model of the thyroid nodule through ultrasound scanning before the operation, and further creates an intraoperative navigation system based on the three-dimensional model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge contour of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery.

[0189] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery, characterized by: The assisted navigation and positioning system uses an ultrasound probe to perform an ultrasound scan on the patient and then assists the surgery during the operation. The system includes the following steps: Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule; Step S2, establishing an approximate three-dimensional model of the nodule using the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model, which serves as a basic model for subsequent high-precision digital nodule modeling; Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the scanned ultrasound image; Step S4: projecting the approximate three-dimensional model onto the current ultrasound image, and using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, thereby completing the nodule edge position data acquisition of the current ultrasound image; In step S4, the projection curve is fitted to the edge of the nodule in the ultrasound image using an image algorithm. Specifically, the following steps are performed: Step S401: first extract the region containing the nodule on the ultrasound image as the ROI region; Step S402: Filter the ROI region using a Gaussian convolution kernel to eliminate the influence of noise, and then solve the partial derivatives in the x and y directions of the filtered image to extract the vertical and horizontal edge information on the ultrasound image respectively; then calculate the gradient of the image : Where, represents the image gradient, represents the partial derivative in the vertical direction of the image, Represents the partial derivative of the image in the horizontal direction; the gradient amplitude of the ultrasound image can be expressed as the following formula; Where, is the gradient amplitude of the ultrasound image; The larger the value is, the greater the gray value of the extracted image on the gradient image is, and the greater the possibility that the extracted image is the edge of the nodule is. At the same time, the gradient direction can be expressed as the following formula: Where, is the image gradient direction, which indicates the direction of the nodule edge of the extracted image; Step S403: Initialize the contour curve and its control points and construct the energy equation of the contour curve; specifically: Step S4031, taking the intersection line of the approximate three-dimensional model after the nodule center is aligned in step S303 and the ultrasound image as the initial edge contour S0 of the nodule, and setting equidistant control points on the initial contour curve; In step S4032, the edge contour of the nodule can be expressed as a functional form in the following form: Where s is the length of the curve described in Fourier transform form; and 、 Represent the coordinate position of each control point in the image, and then construct the energy functional form based on the control points, specifically: Where E represents the total energy of the nodule contour edge, E i represents the image energy, E s represents the bending energy, represents elastic energy; Step S404, finding the minimum value of the energy function; specifically comprising the following steps: Step S4041, calculating the initial energy E0; Step S4042, calculating the moving direction of the current control point and moving the control point; Step S4043, recalculate the contour curve energy and determine whether the calculated contour curve energy is less than the threshold value. If yes, then end the process; if not, return to step S4042 and recalculate. Step S5, repeating step S4 to solve the nodule edge position data on each layer of the ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, generating high-precision three-dimensional point cloud data of the nodule; Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5; Step S7: Using the high-precision three-dimensional digital model of the nodule created in step S6, combined with the location of the real-time ultrasound image during the operation, the contour of the nodule is generated and superimposed on the ultrasound image, thereby enhancing the ultrasound image in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and ablation is performed layer by layer in a grid-like manner until the entire nodule is completely ablated. In step S7, the nodule region of the extracted image layer is evenly divided into multiple grids, and ablation is performed on each grid one by one. Specifically, the following steps are included: Step S701: Locate the current ablation position. Since the two-dimensional ultrasound scanning used for thermal ablation of thyroid nodules is 2D imaging, only ultrasound images of a certain layer of the nodule can be observed through two-dimensional ultrasound scanning. Therefore, it is necessary to first locate one layer for ablation, and then move to the next layer after ablation is completed. Step S702: generating a nodule outline based on the current ultrasound image position and the high-precision three-dimensional digital model of the nodule, and superimposing the outline on the ultrasound image to enhance the ultrasound image in real time; Step S703, using the enhanced ultrasound image to perform precise ablation; Step S704: locate the next layer and repeat steps S701 to S703 until the ablation of the entire nodule is completed; Enhance the ultrasound image; the specific method is: Step S7021: After locating the ablation position, the intersection line between the current ablation layer and the high-precision three-dimensional digital model of the nodule is calculated to obtain the navigation contour line of the nodule edge in the current ultrasound image; In step S7022, assuming that the transverse direction is the Z direction, the current ultrasound image plane can be expressed by the following formula: Where c is a constant; In step S7023, since the high-precision three-dimensional digital model of the nodule is composed of triangular surfaces, the surface of the high-precision three-dimensional digital model of the nodule cannot be expressed by an explicit parameterized equation. The nodule edge surface can be expressed as a point set, specifically: Step S7024, solving the discrete nodule edge curve on the current ultrasound image, specifically: in, It represents the intersection of the plane where the ultrasound image is located and the three-dimensional model of the nodule, that is, the contour line of the nodule on the current ultrasound image, and p is a specific point on the nodule contour line; Step S7025: Divide the ablation area into multiple independent small areas using a fixed spacing to obtain an auxiliary ablation navigation image, and superimpose the auxiliary ablation navigation image on the original ultrasound image to obtain an enhanced ultrasound image.

2. The intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 1, characterized in that: In step S1 , the key geometrical parameters of the nodule include the maximum longitudinal diameter W, the maximum transverse diameter H, and the maximum anteroposterior diameter L of the nodule.

3. The intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 2, characterized in that: The volume V of the nodule is calculated based on the key geometric size parameters of the nodule measured in step S1 as follows: V=W×H×L×π / 6 Where W is the maximum long diameter, H is the maximum transverse diameter, and L is the maximum front-back diameter.

4. The intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 2, characterized in that: In step S3, the nodule center registration method is specifically as follows: Step S301 , establishing a Cartesian coordinate system (X, Y, Z) with the center of the approximate three-dimensional model as the origin. In this coordinate system, the center coordinates of the nodule are (0, 0, 0); Step S302: Extract the closed edge of the nodule on the ultrasound image and calculate the center coordinates of the nodule. The specific method is as follows: first, perform ROI segmentation, filtering, and contrast enhancement on the ultrasound image to make the nodule edge clearer for subsequent processing; then, detect the image through pre-training based on deep learning methods to identify the thyroid nodule area; after detecting the nodule, use image segmentation technology to segment the nodule from the background image; then use the edge detection algorithm to extract the edge coordinate point set of the nodule, which is shown as follows: Where x i ,y i represents the edge coordinates of the nodule; Finally, the center coordinates of the nodule (C x , C y , C z ): Step S303: Move the center of the thyroid nodule on the ultrasound image to the center of the approximate three-dimensional model of the nodule to complete the registration of the nodule center. The specific method is as follows: first, construct a translation and rotation matrix, where the translation matrix is: Assume that during the registration process, the image is rotated around the x, y, and z axes by , the rotation matrices in the three directions are , as shown below: The center coordinates of the nodule on the ultrasound image (C x , C y , C z ) transformation to align its center with the center (0, 0, 0) of the nodule approximate 3D model. The transformation is as follows: In the formula, (C x , C y , C z ) represents the coordinates of the center point of the nodule, Represents a rotation matrix.

5. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 4, characterized in that: In step S5, high-precision three-dimensional point cloud data of the nodule is generated as follows: After completing the fitting of the contour curve of the current ultrasound image of the approximate three-dimensional model to the edge of the nodule, the ultrasound probe is moved to the next layer of the approximate three-dimensional model, and the method in step S4 is repeated until the fitting of the nodule edges on all layers is completed, thereby obtaining high-precision three-dimensional point cloud data C of the nodule; Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set consisting of the edge points of all layers of the nodule on the ultrasound image.

6. The intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 5, characterized in that: In step S6, the calculated high-precision three-dimensional point cloud data of the nodule is gridded to obtain a three-dimensional geometric model of the nodule; specifically, a rectangular area is used to contain the entire point cloud space, and the point cloud is divided into several small rectangular areas. The points in each small rectangular area are continuous. The point cloud in each small rectangular area is gridded, and then all areas are assembled together to form a triangulated mesh model of the entire nodule.

7. The intraoperative assisted navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 6, characterized in that: Meshing the point cloud within each small rectangular area includes the following steps: Step S601: Select an initial point, denoted as P1, search for the nearest point of P1 within the extracted image area, denoted as P2, connect P1 and P2, and then select a third point, P3, to form an initial triangular face. To ensure the quality of the triangular face, P3 should meet the following requirements: Step S602: After the initial triangular face is generated, step S601 is repeated for each edge in the normal direction, so that the face mesh continues to grow outward until the mesh reconstruction of the entire nodule edge point cloud is completed.

Images