Intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation operation

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery, which comprises the following steps: S1, measuring key geometric dimension parameters of nodules; s2, establishing an approximate three-dimensional model of the nodule; s3, registering the approximate three-dimensional model with the ultrasonic image; s4, completing nodule edge position data acquisition of the current ultrasonic image; s5, calculating nodule edge position data; s6, creating a high-precision three-dimensional digital model; and S7, creating a three-dimensional intra-operative navigation and positioning system, and performing block ablation on the nodule. The thyroid nodule thermal ablation device solves the technical problems of excessive ablation and insufficient ablation always existing in a thyroid nodule thermal ablation operation. The operation method of an existing thyroid nodule thermal ablation operation is optimized, a powerful intraoperative navigation and positioning tool is provided for an operator, and the accuracy and the success rate of the thyroid nodule thermal ablation operation are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery. Background Art

[0002] Thermal ablation of thyroid nodules is a minimally invasive thyroid treatment technique, mainly used to treat benign thyroid nodules and some low-risk papillary carcinomas. This procedure usually uses an ablation needle to puncture the thyroid nodule under ultrasound guidance, and uses microwaves or high-frequency currents to generate heat in the tumor or nodule tissue and cause coagulative necrosis, thereby achieving the purpose of tissue cell shrinkage and shedding and aseptic necrosis. The advantage of thermal ablation of thyroid nodules is that it is minimally invasive, can preserve thyroid function while causing less damage to surrounding tissues, and is easy to operate, safe and effective.

[0003] Although thermal ablation of thyroid nodules has many advantages, it also has certain risks. Since there are abundant nerves and blood vessels around the thyroid gland, the thermal effect during ablation treatment may damage these tissues, leading to postoperative complications such as choking when drinking water, swallowing difficulties, hoarseness, or damage to adjacent blood vessels causing intraoperative bleeding. In addition, if malignant nodules are not completely eliminated, there may be a risk of recurrence or metastasis. Due to the vaporization phenomenon produced during the ablation process, the boundaries of thyroid nodules will become blurred and difficult to identify under ultrasound images. The effective range of the thermal effect must be judged based on the surgeon's experience, which can easily lead to excessive or insufficient ablation. Therefore, how to accurately locate the three-dimensional spatial position of the nodule and accurately capture the edge of the nodule during surgery, and present it to the surgeon in an intuitive way to avoid excessive and insufficient ablation, has become a technical problem that needs to be solved urgently in clinical practice. Summary of the invention

[0004] The purpose of the present invention is to provide an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery, so as to solve the problem in the prior art proposed in the background technology that the edge of the thyroid nodule becomes blurred and difficult to identify under ultrasound imaging due to the vaporization phenomenon produced during the ablation process, and the effective range of the thermal effect needs to be judged based on the operator's experience, which easily leads to excessive or insufficient ablation.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: An intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery comprises the following steps: Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule; Step S2, establishing an approximate three-dimensional model of the nodule through the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model as a basic model for subsequent high-precision digital nodule modeling; Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the ultrasound image; Step S4, projecting the approximate three-dimensional model onto the current ultrasound image, using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, and completing the nodule edge position data acquisition of the current ultrasound image; Step S5, repeating step S4 to solve the nodule edge position data on each layer of ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, and high-precision three-dimensional point cloud data of the nodule is generated; Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5; Step S7, using the high-precision three-dimensional digital model of the nodule created in step S6, combined with the location of the real-time ultrasound image during the operation, the outline of the nodule is generated and superimposed on the ultrasound image, and the ultrasound image is enhanced in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and grid-based partition ablation is performed layer by layer until the ablation of the entire nodule is completed.

[0006] According to the above technical solution, in step S1, the key geometric size parameters of the nodule include the maximum long diameter W, the maximum transverse diameter H and the maximum anterior-posterior diameter L of the nodule.

[0007] According to the above technical solution, the volume V of the nodule is calculated according to the key geometric size parameters of the nodule measured in step S1 as follows: Where W is the maximum long diameter, H is the maximum transverse diameter and L is the maximum front-back diameter.

[0008] According to the above technical solution, in step S3, the nodule center registration method is specifically as follows: Step S301: Establish a Cartesian coordinate system with the center of the approximate three-dimensional model as the origin. , in this coordinate system the center coordinates of the nodule are ; Step S302, extract the closed edge of the nodule on the ultrasound image and calculate the center coordinates of the nodule; the specific method is: first, perform ROI segmentation, filtering, and contrast enhancement on the ultrasound image to make the nodule edge clearer for subsequent processing; then detect the image through a pre-trained neural network based on a deep learning method (such as Mask R-CNN, etc.) to identify the thyroid nodule area; these algorithms are often trained with a large amount of labeled data and can accurately identify the characteristics of thyroid nodules and automatically detect similar structures in new ultrasound images. After the nodule is detected, the nodule is segmented from the background image using image segmentation technology (such as the level set algorithm, etc.); then an edge detection algorithm (such as the Canny edge detection operator, the non-maximum suppression algorithm, etc.) is used to extract the edge coordinate point set of the nodule, and the point set is shown below: Finally, the center point coordinates of the nodule are calculated by the following formula : Step S303, move the center of the thyroid nodule on the ultrasound image to the center of the nodule approximate three-dimensional model to complete the registration of the nodule center; the specific method is: first construct a translation and rotation matrix (homogeneous coordinates are generally used in three-dimensional space). The translation matrix is: Assume that during the registration process, the image is rotated around the x, y, and z axes respectively. , the rotation matrices in the three directions are , as shown below: The center coordinates of the nodule on the ultrasound image Transformation can be used to align its center with the center of the nodule approximation 3D model Aligned, the transformations are as follows: Furthermore, the intersection line between the ultrasound image and the approximate three-dimensional model of the nodule is calculated as the initial contour edge of the nodule.

[0009] According to the above technical solution, in step S4, the projection curve is fitted to the nodule edge of the ultrasound image using an image algorithm as follows: Step S401, first extracting the region containing the nodule on the ultrasound image as the ROI region; Step S402: filter the ROI region using a Gaussian convolution kernel to eliminate the influence of noise, and then solve the partial derivatives in the x and y directions of the filtered image to extract the vertical and horizontal edge information on the ultrasound image. Then calculate the gradient of the image : In the formula, represents the image gradient, represents the partial derivative in the vertical direction of the image, Represents the partial derivative of the image in the horizontal direction; the gradient amplitude of the ultrasound image can be expressed as the following formula.

[0010] In the formula, is the image gradient amplitude; The larger the value is, the greater the gray value on the gradient image is, and the greater the possibility that the pixel is the edge of the nodule. At the same time, the gradient direction can be expressed as the following formula: In the formula, is the image gradient direction, indicating the direction of the nodule edge; Step S403, setting the initialization contour curve and its control points, and constructing the energy equation of the contour curve; specifically: Step S4031: taking the intersection line of the approximate 3D model and the ultrasound image after the nodule center is aligned in step S303 as the initial edge contour of the nodule , and set equidistant control points on the initial contour curve; Step S4032, the edge contour of the nodule can be expressed as a functional form in the following form: Where s is the length of the curve described in Fourier transform form; and Respectively represent the coordinate position of each control point in the image, and then construct the energy functional form based on the control points, specifically: Where E represents the total energy of the nodule contour edge, represents the image energy, represents the bending energy, represents elastic energy; Step S404, solving the minimum value of the energy function; specifically comprising the following steps: Step S4041, calculate initial energy ; Step S4042, calculating the moving direction of the current control point, and moving the control point; Step S4043, recalculate the contour curve energy, and determine whether the calculated contour curve energy is less than the threshold value, if yes, then end, if not, return to step S4042, and recalculate.

[0011] According to the above technical solution, in step S5, the high-precision three-dimensional point cloud data of the nodule is generated as follows: After completing the fitting of the contour curve of the current ultrasound image of the approximate three-dimensional model with the edge of the nodule, the ultrasound probe is moved to the next layer of the approximate three-dimensional model, and the method in step S4 is repeated until the fitting of the nodule edges on all layers is completed, thereby obtaining high-precision three-dimensional point cloud data C of the nodule; Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set consisting of edge points of all layers of the nodule on the ultrasound image (assuming there are k points in total).

[0012] According to the above technical solution, in step S6, the calculated high-precision three-dimensional point cloud data of the nodule is gridded to obtain a three-dimensional geometric model of the nodule; specifically, a rectangular area is used to contain the entire point cloud space, and the point cloud is divided into several small rectangular areas. The points in each small rectangular area are continuous, and the point cloud in each small rectangular area is gridded, and then all areas are assembled together to form a triangulated mesh model of the entire nodule.

[0013] According to the above technical solution, meshing the point cloud in each small rectangular area includes the following steps: Step S601, select an initial point and record it as ,search The closest point in this region is denoted by ,connect and , then select the third point Construct the initial triangular surface. To ensure the quality of the triangular surface, The following requirements should be met at the same time: Step S602: After the initial triangular face is generated, step S601 is repeated for each edge in the normal direction, so that the face mesh continues to grow outward until the mesh reconstruction of the entire nodule edge point cloud is completed.

[0014] According to the above technical solution, in step S7, the layer of nodule area is evenly divided into multiple grids, and ablation is performed grid by grid, specifically including the following steps: Step S701, locating the current ablation position; since the two-dimensional ultrasound scanning used in thermal ablation of thyroid nodules is 2D imaging, only the ultrasound image of a certain layer of the nodule can be observed through two-dimensional ultrasound scanning, so it is necessary to first locate a layer for ablation, and then move to the next layer after the ablation is completed; Step S702, generating a nodule contour line based on the current ultrasound image position and the high-precision three-dimensional digital model of the nodule, superimposing the contour line on the ultrasound image, and enhancing the ultrasound image in real time; Step S703, using the enhanced ultrasound image to perform precise ablation; Step S704, locate the next layer, and repeat steps S701 to S703 until the ablation of the entire nodule is completed.

[0015] According to the above technical solution, the ultrasonic image is enhanced; the specific method is: Step S7021, after locating the ablation position, calculating the intersection line between the current ablation layer and the high-precision three-dimensional digital model of the nodule, and obtaining the nodule edge navigation contour line of the current ultrasound image; Step S7022, assuming that the transverse direction is the Z direction, the current ultrasound image plane can be expressed by the following formula: Where c is a constant; Step S7023: Since the high-precision three-dimensional digital model of the nodule is composed of triangular surfaces, the surface of the high-precision three-dimensional digital model of the nodule cannot be represented by an explicit parameterized equation. The edge surface of the nodule can be represented as a point set, which is specifically: Step S7024, solving the discrete nodule edge curve on the current ultrasound image, specifically: in, It represents the intersection of the plane where the ultrasound image is located and the three-dimensional model of the nodule, that is, the contour line of the nodule on the current ultrasound image, and p is a specific point on the contour line of the nodule; Step S7025, using a fixed spacing to divide the ablation area into multiple independent small areas, to obtain an auxiliary ablation navigation image, and to superimpose the auxiliary ablation navigation image on the original ultrasound image to obtain an enhanced ultrasound image.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a 3D digital model of thyroid nodules through ultrasound scanning before surgery, and further creates an intraoperative navigation system based on the 3D model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge curve of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 This is a flowchart of the precise thyroid ablation surgery using enhanced ultrasound images of the present invention; Figure 3 This is a schematic diagram of the key dimensions of nodules measured using two-dimensional ultrasound in the present invention; Figure 3 (A) is a schematic diagram of the maximum long diameter W and maximum transverse diameter H of the nodule measured by two-dimensional ultrasound images. Figure 3 (B) is a schematic diagram of the maximum anterior-posterior diameter L of the nodule measured using two-dimensional ultrasound images; Figure 4 This is a standard 3D model diagram of a thyroid nodule of the present invention; Figure 5 The flowchart of the nodule 3D model and ultrasound image registration of the present invention; Figure 6 The standard ellipsoid model of thyroid nodules and the ultrasound image registration diagram of the present invention; Figure 7 This is a flow chart of nodule edge curve fitting in the present invention; Figure 8 This is the ROI area map of the thyroid nodule of the present invention; Fig. 9 It is a schematic diagram of the initial contour curve and control points of the present invention; Fig.10 A schematic diagram of the moving direction of the control point of the present invention; Fig.11 It is a flow chart of iterative calculation of contour curve energy of the present invention; Fig.12 This is a schematic diagram of gridding point cloud data of the present invention; Fig.13 This is a flowchart of the precise ablation based on enhanced ultrasound images of the present invention; Fig.14 It is a schematic diagram of the projection of the node cloud to the ultrasound image of the present invention; Fig.15 The least square method is used to fit the nodule edge curve graph in the present invention; Fig.16 It is the auxiliary ablation navigation map of the present invention; Fig.17 Ultrasound images are enhanced for the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1 like Figure 1 As shown, an intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery includes the following steps: Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule; Step S2, establishing an approximate three-dimensional model of the nodule through the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model as a basic model for subsequent high-precision digital nodule modeling; Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the ultrasound image; Step S4, projecting the approximate three-dimensional model onto the current ultrasound image, using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, and completing the nodule edge position data acquisition of the current ultrasound image; Step S5, repeating step S4 to solve the nodule edge position data on each layer of ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, and high-precision three-dimensional point cloud data of the nodule is generated; Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5; Step S7, using the high-precision three-dimensional digital model of the nodule created in step S6, combined with the location of the real-time ultrasound image during the operation, the outline of the nodule is generated and superimposed on the ultrasound image, and the ultrasound image is enhanced in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and grid-based partition ablation is performed layer by layer until the ablation of the entire nodule is completed.

[0020] The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a three-dimensional digital model of thyroid nodules through ultrasound scanning before surgery, and further creates an intraoperative navigation system based on the three-dimensional model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge contour of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery.

[0021] Embodiment 2 The present invention is a method of using mathematical modeling and image analysis technology to quickly reconstruct thyroid nodules in three dimensions, and to enhance the intraoperative ultrasound image based on the reconstructed digital nodule model. The enhanced ultrasound image and virtual digital nodule model are used to achieve spatial position tracking, positioning, and edge contour visualization of thyroid nodules during the entire ablation process, thereby effectively solving the technical difficulties of excessive and insufficient ablation, and also providing more accurate surgical navigation for the surgeon.

[0022] The specific implementation route of the method of the present invention is as follows Figure 2 As shown in the figure, it includes 7 main steps. Among them, S1: Perform a two-dimensional ultrasound scan on the patient before the operation to measure the three key geometric size parameters of the nodule; S2: Establish an approximate ellipsoid model of the nodule through the key parameters obtained in step S1, which is used as the basic model for subsequent high-precision digital nodule modeling; S3: Align the center of the ellipsoid model of the nodule with the center of the nodule on the ultrasound image; S4: Project the approximate ellipsoid model of the nodule onto the current ultrasound image. Since the geometric center surrounded by its projection curve coincides with the center of the nodule on the ultrasound image, the projection curve can be fitted to the nodule boundary curve on the ultrasound using an image algorithm to complete the nodule contour data acquisition of the current ultrasound image. S5: Move to the next layer and repeat step S4 until the three-dimensional contour space data acquisition of the entire nodule is completed, generating high-precision point cloud data of the nodule. S6: Use the point cloud data generated in step S5 to create a high-precision three-dimensional digital model of the nodule. S7: Position the first layer (the largest diameter layer), and under the guidance of the three-dimensional digital model of the nodule, evenly divide the nodule area of ​​this layer into multiple grids, and ablate each grid one by one. Since the contour edge of the nodule can be calculated through the three-dimensional digital model of the nodule, the contour boundary of the nodule can be clearly outlined on the enhanced ultrasound image, thereby avoiding excessive or insufficient ablation. Then repeat this step in each layer until the ablation of the entire nodule is completed.

[0023] S1, preoperative diagnosis, measurement of key size parameters of nodules by ultrasound images.

[0024] In order to establish a three-dimensional digital model of the nodule, the key geometric parameters of the nodule need to be obtained first. Since an ellipsoid is often used to approximate the shape of the nodule in clinical practice, it is sufficient to measure the maximum long diameter W, maximum transverse diameter H, and maximum anterior-posterior diameter L of the nodule. These parameters can be obtained through two-dimensional ultrasound imaging. The steps are as follows: (1) Patient preparation: The patient lies in supine position with the head tilted back to fully expose the thyroid area of ​​the neck.

[0025] (2) Preparation of ultrasound equipment: Select a high-resolution two-dimensional ultrasound probe and adjust appropriate ultrasound parameters.

[0026] (3) Nodule localization: The ultrasound probe is used to find and locate the nodule in the thyroid region. On ultrasound images, the nodule usually appears as a low-echo or isoechoic mass.

[0027] (4) Measuring the maximum length / transverse diameter: The ultrasound probe is placed in a transverse position, and a measuring tool (such as an electronic cursor) is used to measure the maximum length W and maximum transverse diameter H of the nodule on the two-dimensional ultrasound image. Figure 3 (A) shown.

[0028] (5) Measure the maximum anteroposterior diameter: The ultrasound probe is placed in the sagittal position to measure the maximum anteroposterior diameter L of the nodule. Figure 3 (B) shown.

[0029] S2, build an approximate three-dimensional model of the nodule based on the nodule size parameters and calculate its volume.

[0030] According to the maximum longitudinal diameter W, maximum transverse diameter H and maximum anterior-posterior diameter L of the nodule obtained in step S1 above, an approximate three-dimensional ellipsoid model of the nodule is established, as shown in Figure 4 The volume V of a nodule is generally calculated using the following formula.

[0031] Where W is the maximum long diameter, H is the maximum transverse diameter and L is the maximum front-back diameter.

[0032] S3, nodule approximate 3D model is registered with ultrasound image.

[0033] In this step, the center of the ellipsoid model of the thyroid nodule is aligned with the ultrasound image. A Cartesian coordinate system is established with the center of the ellipsoid model as the origin. , so the center coordinates of the nodule are All subsequent operations on ultrasound images and nodule models are completed in this coordinate system. Figure 5As shown, this step is divided into two sub-steps. S31 extracts the closed boundary of the thyroid nodule on the ultrasound image and calculates the center coordinates. S32 calculates the rotation and translation matrix to complete the registration of the center of the image. The specific method is: first, the ultrasound image is segmented, filtered, and contrast enhanced to make the edge of the nodule clearer for subsequent processing; then the image is detected by a pre-trained neural network based on a deep learning method (such as Mask R-CNN, etc.) to identify the thyroid nodule area; these algorithms are often trained with a large amount of labeled data and can accurately identify the characteristics of the thyroid nodule and automatically detect similar structures in new ultrasound images. After the nodule is detected, the nodule is segmented from the background image using image segmentation technology (such as the level set algorithm, etc.); then the edge detection algorithm (such as the Canny edge detection operator, the non-maximum suppression algorithm, etc.) is used to extract the edge coordinate point set of the nodule, and the point set is shown below: Finally, the center point coordinates of the nodule are calculated by the following formula : Move the center of the thyroid nodule on the ultrasound image to the center of the approximate three-dimensional model of the nodule to complete the registration of the nodule center; the specific method is: first construct the translation and rotation matrix, where the translation matrix is: Assume that during the registration process, the image is rotated around the x, y, and z axes respectively. , the rotation matrices in the three directions are , as shown below: The center coordinates of the nodule on the ultrasound image Transformation can be used to align its center with the center of the nodule approximation 3D model Aligned, the transformations are as follows: Furthermore, the intersection line between the ultrasound image and the approximate three-dimensional model of the nodule is calculated as the initial contour edge of the nodule; the image after registration is as follows Figure 6 shown.

[0034] S4, the projection curve of the standard model is fitted to the edge of the nodule using an image algorithm.

[0035] On ultrasound images, the edge of a thyroid nodule generally presents a visible elliptical closed outline, such as Figure 3As shown in Figure 2. The contour has a more obvious gradient on the ultrasound image, so the edge detection method can be used to extract the edge of the nodule to make it more obvious. Furthermore, the contour of the standard ellipse model of the nodule generated by the above steps (see Figure 5 ) as the initial contour, and use the numerical iteration algorithm to make this curve gradually approach the edge of the nodule until it finally converges and completely coincides with the edge of the nodule. This step is divided into four sub-steps, such as Figure 7 shown.

[0036] S401, image preprocessing (ROI extraction, noise reduction, calculation of image gradient amplitude).

[0037] The ultrasound images of thyroid examination contain a lot of tissue information, including CCA (common carotid artery), esophagus, lymph nodes, etc. In order to reduce the computational workload, the area containing the thyroid nodules is first extracted as the ROI area, such as Figure 8 shown.

[0038] The above ROI area is filtered using a Gaussian convolution kernel to eliminate the influence of noise, and then the partial derivatives in the x and y directions of the filtered image are solved respectively to calculate the gradient amplitude of the image. In order to speed up the calculation, preferably, the partial derivative of the Gaussian filter can be first calculated to obtain a Gaussian partial derivative filter, and then the Gaussian partial derivative filter is convolved with the image to calculate the gradient of the image. Formula (5) is the process of solving the gradient by convolving the Gaussian partial derivative filter with the image.

[0039] In the above formula, express Figure 8 The ultrasound image shown, Represents the Gaussian convolution kernel. Formula (5) indicates that the image is first Gaussian filtered and then the partial derivative is calculated, which is equivalent to first calculating the partial derivative of the Gaussian filter and then convolving the image, but it can greatly reduce the amount of calculation. Formula (6) is the calculation formula for the image gradient.

[0040] Formulas (7) and (8) are the calculation formulas for the gradient magnitude and gradient direction, respectively. The larger the value is, the greater the gray value is on the gradient image, and the greater the possibility that the pixel is the edge of a nodule. Indicates the direction of the edge of the nodule.

[0041] S402, setting initialization contour curve and its control points, construct the energy equation of the contour curve.

[0042] In this step, the contour line of the nodule standard ellipsoid model generated in step S32 is used as the initial contour curve , and set equidistant control points, such as Fig. 9 shown.

[0043] The contour curve can be expressed in a functional form as follows: (9), where s is the length of the curve described in Fourier transform form. and Respectively represent the coordinate position of each control point in the image.

[0044] The energy functional can be constructed based on the control points of formula (9), as shown in formula (10): Among them, E represents the total energy of the contour curve, which consists of three parts of energy: image energy , bending energy and elastic energy Among them, the image energy is the external energy, representing the edge curve of the nodule contour The degree of fitting with the edge of the nodule is used to control the contour edge curve to approach the target edge on the ultrasound image; image energy The smaller the value, the better the fit; elastic energy and bending energy It is used to control the elastic deformation of the contour curve and maintain the continuity and smoothness of the contour curve. By using an iterative algorithm to solve the minimum value of E, the contour curve in a stable state, that is, the edge curve of the nodule, can be obtained.

[0045] An iterative algorithm can be used to find the minimum value of E (such as Fig.11 As shown in the figure), the stable contour curve, that is, the edge contour of the nodule, can be obtained. Formula (11) is the calculation formula for image energy. is a Gaussian filter, Find the gradient, and then convolve this gradient operator with the control points on the curve to solve the image energy. Note that the curve will evolve in the direction of the maximum gradient, so in order to solve To achieve the minimum value of , the energy needs to be inverted.

[0046] However, in the process of initial curve evolution, the curve should be kept smooth and elastic as much as possible, so two constraints are introduced: and , see formula (9). Use the first-order derivative of the contour functional to maintain the elasticity of the curve, The second derivative of the contour functional is used to keep the curve smooth.

[0047] Formula (12) is a partial differential equation, which is discretized at each control point to obtain a computable form: Substitute formula (11) and formula (13) into formula (10), and then introduce elastic energy and bending energy The respective influence coefficients and , we get the final calculable energy formula: S403, finding the minimum value of the energy function.

[0048] For each control point in the above formula (14), calculate its moving direction and iteratively solve the minimum value of the energy function until the contour curve coincides with the edge of the thyroid nodule. Fig.10 middle, are three consecutive control points on the contour curve. Assume that the control points Move only one pixel at a time. They are 8 possible movement directions. To connect control points and The vector of To connect control points and Vector. Respectively with vector and The vector with the smallest cross product is the control point. Direction of movement .

[0049] S5, solving the nodule edge contour curve on each layer of ultrasound image to create a three-dimensional point cloud of the nodule.

[0050] After completing the fitting of the contour curve of the current ultrasound image and the edge of the thyroid nodule, the ultrasound probe is moved to the next layer, and the method of step S4 above is repeated until the fitting of the edges of the thyroid nodules on all layers is completed to obtain the three-dimensional point cloud C of the nodule. The calculation of the three-dimensional point cloud C is shown in the following formula.

[0051] Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set consisting of edge points of all layers of the nodule on the ultrasound image (assuming there are k points in total).

[0052] S6, Build a 3D model of the thyroid gland using point cloud data.

[0053] In order to observe the three-dimensional model of the thyroid nodule in a more intuitive way, the point cloud data of the nodule edge obtained by the above calculation is meshed to obtain the three-dimensional geometric model of the nodule. The modular meshing method can call multi-CPU parallel computing to accelerate the meshing process. Specifically, a rectangular area is used to contain the entire point cloud space, and the point cloud is divided into several small cubes. The points in each small area are continuous. The point cloud in each small area is meshed and reassembled to form a triangulated mesh of the entire nodule.

[0054] The meshing method inside each region consists of the following two steps: Step 1: Choose an initial point ,search The closest point in this region is denoted by ,connect and , select the third point To ensure the quality of the triangle, the following equations (17) and (18) should be met at the same time. Fig.12 Schematic diagram of generating a mesh from point cloud data.

[0055] Step 2: After the initial triangular surface is generated, repeat this step for each edge in the normal direction until the mesh reconstruction of the entire nodule edge point cloud is completed.

[0056] S7, created a three-dimensional navigation system to complete precise ablation of the entire nodule based on enhanced ultrasound images.

[0057] In this step, the navigation system is created using the three-dimensional model of the thyroid nodule generated in the above steps, the original ultrasound image is enhanced, and precise ablation is performed based on the enhanced ultrasound image. This step is completed in four sub-steps. S71, locate the current ablation position; S72, generate a navigation system based on the current ultrasound image position and the nodule three-dimensional template, and enhance the ultrasound image; S73, use the enhanced ultrasound image for precise ablation; S74, locate the next layer, and repeat the above steps until the ablation of the entire nodule is completed. Fig.13 The flowchart of this step is shown in Figure 2.

[0058] S71, positioning the current ablation layer.

[0059] Thermal ablation of thyroid nodules is usually performed under ultrasound guidance. Since ultrasound uses 2D imaging, the surgeon can only observe the image of a certain layer of the nodule through ultrasound. Therefore, the surgeon will locate a certain layer for ablation, and move to the next layer after completing the current ultrasound image until the ablation of the entire nodule is completed. During the operation, the surgeon generally starts ablation from the largest diameter layer in the transverse position.

[0060] S72, generating a navigation system based on the current ultrasound image position and the three-dimensional model of the nodule to enhance the ultrasound image.

[0061] After locating the ablation position, the intersection line between the current ablation layer and the three-dimensional model of the thyroid nodule generated above is calculated to obtain the thyroid edge navigation curve of the current ultrasound image. Assuming that the cross-section direction is the Z direction, the current ultrasound image plane can be expressed as the following formula (19), where c is a constant.

[0062] Since the 3D model of thyroid nodules is composed of triangular surfaces generated by point clouds, the 3D surface of the nodule cannot be expressed by explicit parameterized equations. The edge surface of the nodule can be expressed as a point set, as shown in the following equation (20).

[0063] Combining equations (19) and (20) can solve the discrete form of the nodule edge curve on the current ultrasound image, as shown in equation (21). The greater the density of the point cloud, the more accurate the nodule edge curve.

[0064] Preferably, a dynamically adjustable threshold value is set to represent the distance from the layer where the current ultrasound image is located. Points within the threshold value are projected onto the ultrasound image to obtain an edge point set of the nodule. Fig.14 It indicates that the points within the threshold interval are projected onto the ultrasound image.

[0065] For the nodule edge point set projected onto the ultrasound image, if these points are regular and continuous, they can be directly connected with straight lines to generate edge curves. Otherwise, the least squares method can be used to fit the edge curve of the nodule, such as Fig.15 shown.

[0066] Furthermore, the ablation area is divided into multiple independent small areas using a fixed spacing, such as Fig.16 shown.

[0067] The auxiliary ablation navigation image is superimposed on the original ultrasound image to obtain an enhanced ultrasound image, such as Fig.17During the ablation process, the surgeon performs ablation one by one according to the divided areas, and the edge curve displayed in real time can also largely avoid the problem of blurred nodule edges caused by vaporization, thereby avoiding the problems of over-ablation and under-ablation.

[0068] S73, locate to the next layer and complete the ablation of the entire thyroid nodule.

[0069] After completing the ablation of the current ultrasound image, the operator locates the next layer and repeats the steps of the previous layer until the precise ablation of the entire thyroid nodule is completed.

[0070] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0071] The method proposed in the present invention solves the technical problems of excessive and insufficient ablation that have always existed in thermal ablation surgery for thyroid nodules. The present invention uses computer vision and image processing technology to perform real-time analysis and modeling of ultrasound images, creates a three-dimensional digital model of thyroid nodules through ultrasound scanning before surgery, and further creates an intraoperative navigation system based on the three-dimensional model to enhance the ultrasound image in real time. The enhanced ultrasound image can display the edge contour of the thyroid nodule in real time, assisting the surgeon to accurately judge the ablation range and the edge of the nodule, thereby avoiding excessive and insufficient ablation to a great extent. The beneficial effect brought about by the present invention is that it significantly optimizes the existing operating method of thermal ablation surgery for thyroid nodules, provides the surgeon with a powerful intraoperative navigation tool, and can significantly improve the success rate of ablation surgery.

[0072] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery, characterized by: The following steps are involved: Step S1, performing a two-dimensional ultrasound scan on the patient using an ultrasound probe before surgery to measure key geometric size parameters of the nodule; Step S2, establishing an approximate three-dimensional model of the nodule through the key geometric size parameters obtained in step S1, and calculating the volume of the approximate three-dimensional model as a basic model for subsequent high-precision digital nodule modeling; Step S3, registering the center of the approximate three-dimensional model with the center of the nodule on the scanned ultrasound image; Step S4, projecting the approximate three-dimensional model onto the current ultrasound image, using an image algorithm to fit the projection curve to the nodule edge shown in the ultrasound image, and completing the nodule edge position data acquisition of the current ultrasound image; Step S5, repeating step S4 to solve the nodule edge position data on each layer of ultrasound image until the three-dimensional spatial data collection of the entire nodule is completed, and high-precision three-dimensional point cloud data of the nodule is generated; Step S6, creating a high-precision three-dimensional digital model of the nodule using the high-precision three-dimensional point cloud data generated in step S5; Step S7, using the high-precision three-dimensional digital model of the nodule created in step S6, combined with the location of the real-time ultrasound image during the operation, the outline of the nodule is generated and superimposed on the ultrasound image, and the ultrasound image is enhanced in real time; the enhanced ultrasound image is used to assist in navigation and positioning of the nodule, and grid-based partition ablation is performed layer by layer until the ablation of the entire nodule is completed.

2. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 1, characterized in that: In step S1, the key geometrical dimension parameters of the nodule include the maximum long diameter W, the maximum transverse diameter H and the maximum anterior-posterior diameter L of the nodule.

3. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 2, characterized in that: The volume V of the nodule is calculated according to the key geometric size parameters of the nodule measured in step S1 as follows: Where W is the maximum long diameter, H is the maximum transverse diameter and L is the maximum front-back diameter.

4. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 2, characterized in that: In step S3, the nodule center registration method is specifically as follows: Step S301: Establish a Cartesian coordinate system with the center of the approximate three-dimensional model as the origin. , in this coordinate system the center coordinates of the nodule are ; Step S302, extract the closed edge of the nodule on the ultrasound image and calculate the center coordinates of the nodule; the specific method is: first, perform ROI segmentation, filtering, and contrast enhancement on the ultrasound image to make the nodule edge clearer for subsequent processing; then detect the image through pre-training based on the deep learning method to identify the thyroid nodule area therein; after the nodule is detected, use the image segmentation technology to segment the nodule from the background image; then use the edge detection algorithm to extract the edge coordinate point set of the nodule, the point set is shown as follows: In the formula, represents the edge coordinates of the nodule; Finally, the center point coordinates of the nodule are calculated by the following formula ; Step S303, moving the center of the thyroid nodule on the ultrasound image to the center of the nodule approximate three-dimensional model to complete the registration of the nodule center; the specific method is: first constructing a translation and rotation matrix, wherein the translation matrix is: Assume that during the registration process, the image is rotated around the x, y, and z axes respectively. , the rotation matrices in the three directions are , as shown below: The center coordinates of the nodule on the ultrasound image Transformation can be used to align its center with the center of the nodule approximation 3D model Aligned, the transformations are as follows: In the formula, represents the coordinates of the center point of the nodule, Represents a rotation matrix.

5. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 4, characterized in that: In step S4, the projection curve is fitted to the edge of the nodule in the ultrasound image using an image algorithm, specifically: Step S401, first extracting the region containing the nodule on the ultrasound image as the ROI region; Step S402: filter the ROI region using a Gaussian convolution kernel to eliminate the influence of noise, and then solve the partial derivatives in the x and y directions of the filtered image to extract the vertical and horizontal edge information on the ultrasound image; then calculate the gradient of the image. : In the formula, represents the image gradient, represents the partial derivative in the vertical direction of the image, Represents the partial derivative of the image in the horizontal direction; the gradient amplitude of the ultrasound image can be expressed as the following formula; In the formula, is the image gradient amplitude; The larger the value is, the greater the gray value on the gradient image is, and the greater the possibility that the pixel is the edge of the nodule. At the same time, the gradient direction can be expressed as the following formula: In the formula, is the image gradient direction, indicating the direction of the nodule edge; Step S403, setting the initialization contour curve and its control points, and constructing the energy equation of the contour curve; specifically: Step S4031: taking the intersection line of the approximate 3D model and the ultrasound image after the nodule center is aligned in step S303 as the initial edge contour of the nodule , and set equidistant control points on the initial contour curve; Step S4032, the edge contour of the nodule can be expressed as a functional form in the following form: Where s is the length of the curve described in Fourier transform form; and Respectively represent the coordinate position of each control point in the image, and then construct the energy functional form based on the control points, specifically: Where E represents the total energy of the nodule contour edge, represents the image energy, represents the bending energy, represents elastic energy; Step S404, solving the minimum value of the energy function; The specific steps include: Step S4041, calculate initial energy ; Step S4042, calculating the moving direction of the current control point, and moving the control point; Step S4043, recalculate the contour curve energy, and determine whether the calculated contour curve energy is less than the threshold value, if yes, then end, if not, return to step S4042, and recalculate.

6. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 5, characterized in that: In step S5, high-precision three-dimensional point cloud data of nodules is generated as follows: After completing the fitting of the contour curve of the current ultrasound image of the approximate three-dimensional model with the edge of the nodule, the ultrasound probe is moved to the next layer of the approximate three-dimensional model, and the method in step S4 is repeated until the fitting of the nodule edges on all layers is completed, thereby obtaining high-precision three-dimensional point cloud data C of the nodule; Where C represents the high-precision three-dimensional point cloud data of the nodule, which is a point set composed of edge points of all layers of the nodule on the ultrasound image.

7. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 6, characterized in that: In step S6, the calculated high-precision three-dimensional point cloud data of the nodule is gridded to obtain a three-dimensional geometric model of the nodule; specifically, a rectangular area is used to contain the entire point cloud space, and the point cloud is divided into several small rectangular areas. The points in each small rectangular area are continuous, and the point cloud in each small rectangular area is gridded, and then all areas are assembled together to form a triangulated mesh model of the entire nodule.

8. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 7, characterized in that: Meshing the point cloud in each small rectangular area includes the following steps: Step S601, select an initial point and record it as ,search The closest point in this region is denoted by ,connect and , then select the third point Construct the initial triangular surface. To ensure the quality of the triangular surface, The following requirements should be met at the same time: Step S602: After the initial triangular face is generated, step S601 is repeated for each edge in the normal direction, so that the face mesh continues to grow outward until the mesh reconstruction of the entire nodule edge point cloud is completed.

9. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 1, characterized in that: In step S7, the layer of nodule area is evenly divided into multiple grids, and ablation is performed grid by grid, specifically including the following steps: Step S701, locating the current ablation position; since the two-dimensional ultrasound scanning used in thermal ablation of thyroid nodules is 2D imaging, only the ultrasound image of a certain layer of the nodule can be observed through two-dimensional ultrasound scanning, so it is necessary to first locate a layer for ablation, and then move to the next layer after the ablation is completed; Step S702, generating a nodule contour line based on the current ultrasound image position and the high-precision three-dimensional digital model of the nodule, superimposing the contour line on the ultrasound image, and enhancing the ultrasound image in real time; Step S703, using the enhanced ultrasound image to perform precise ablation; Step S704, locate the next layer, and repeat steps S701 to S703 until the ablation of the entire nodule is completed.

10. The intraoperative auxiliary navigation and positioning system for thyroid nodule thermal ablation surgery according to claim 9, characterized in that: Enhance the ultrasound image; the specific method is: Step S7021, after locating the ablation position, calculating the intersection line between the current ablation layer and the high-precision three-dimensional digital model of the nodule, and obtaining the nodule edge navigation contour line of the current ultrasound image; Step S7022, assuming that the transverse direction is the Z direction, the current ultrasound image plane can be expressed by the following formula: Where c is a constant; Step S7023: Since the high-precision three-dimensional digital model of the nodule is composed of triangular surfaces, the surface of the high-precision three-dimensional digital model of the nodule cannot be represented by an explicit parameterized equation. The edge surface of the nodule can be represented as a point set, which is specifically: Step S7024, solving the discrete nodule edge curve on the current ultrasound image, specifically: in, It represents the intersection of the plane where the ultrasound image is located and the three-dimensional model of the nodule, that is, the contour line of the nodule on the current ultrasound image, and p is a specific point on the contour line of the nodule; Step S7025, using a fixed spacing to divide the ablation area into multiple independent small areas, to obtain an auxiliary ablation navigation image, and to superimpose the auxiliary ablation navigation image on the original ultrasound image to obtain an enhanced ultrasound image.

Images