Thoracic surgery lung puncture navigation system and method

Abstract

The invention provides a thoracic surgery lung puncture navigation system and method. The thoracic surgery lung puncture navigation system comprises medical imaging equipment, a model construction module, a positioning patch module, intelligent head display equipment, a puncture needle and a puncture guide plate. The medical imaging equipment generates a lung tissue image containing the positioning patch; the puncture guide plate comprises a plurality of photosensitive patches and a guide hole which penetrates through the main panel and is used for guiding a puncture needle; the model building module is used for building a virtual model comprising lung tissue and a planned puncture path and a puncture guide plate axis model according to the lung tissue image; and the intelligent head display equipment identifies the positioning patch to determine a coordinate transformation matrix between the virtual model and an actual scene, and projects the virtual model according to the transformation matrix, so that a doctor can coincide an axis model of the puncture guide plate with a planned virtual path, a puncture needle can be quickly guided in, and puncture is quickly realized.

Description

Technical Field

[0001] The present application relates to the field of surgical navigation, and in particular, to a thoracic surgery lung puncture navigation system and method. Background Art

[0002] The lung puncture positioning needle currently used in clinical practice is a percutaneous puncture positioning needle under CT guidance. When the puncture needle reaches the lesion, the needle tip releases the inner needle with a barb and fixes it to the lesion inside the lung, guiding the doctor to complete the lobectomy under the visual field of the thoracoscope.

[0003] A pulmonary nodule puncture positioning needle usually consists of the following main parts:

[0004] Needle body: The main body of the puncture needle, usually made of a hard material such as stainless steel. The length and diameter vary depending on the patient's body shape and the depth of the nodule.

[0005] Positioning catheter: This part ensures that the needle reaches the target location accurately. The catheter may have scales or markings to help the doctor determine the depth of the puncture.

[0006] Needle: This is the tip at the end of a needle, typically a sharp point, used to penetrate tissue and collect samples. Needle designs may vary depending on the intended use; common designs include cutting needles and vacuum needles.

[0007] Scale and depth indicator: The positioning needle is usually equipped with a scale or depth indicator to help the doctor judge the depth of the puncture and avoid puncturing too deep or too shallow.

[0008] Positioning systems: These include CT-guided, ultrasound-guided, or AR-assisted systems to help doctors accurately locate lung nodules. The systems may include positioning markers or guide devices to ensure needle accuracy.

[0009] In existing AR-assisted lung puncture procedures, the model building module constructs a virtual model of the lung tissue and the planned puncture path based on CT or ultrasound images. The AR device then uses the recognition and positioning module (commonly consisting of multiple metal balls, which are scanned along with the body by CT) to superimpose the virtual puncture path on the body. While existing AR-assisted puncture technology can provide puncture navigation, the doctor still needs to refer to the puncture path and correct the puncture angle in real time during the puncture. Summary of the Invention

[0010] The purpose of this application is to provide a thoracic surgery lung puncture navigation system and method for quickly and accurately achieving navigation of thoracic surgery lung puncture surgery.

[0011] In the first aspect, the present application provides a thoracic surgery lung puncture navigation system, including a medical imaging device, a model building module, a positioning patch module, an intelligent head display device, a puncture needle and a puncture guide; the positioning patch module includes a plurality of positioning patches, the medical imaging device is used to scan human lung tissue and the positioning patch module, and generate a lung tissue image including the positioning patches; the puncture guide includes a main panel, the main panel includes a plurality of photosensitive patches and a guide hole penetrating the main panel for guiding the puncture needle; the model building module constructs a virtual model including lung tissue and a planned puncture path according to the lung tissue image, and The position of the positioning patch in the lung tissue image determines the coordinates of the positioning patch in the virtual model, and constructs the axis model of the guide hole according to the shape of the puncture guide; the smart head display device identifies the actual position of the positioning patch on the positioning patch module, and determines the coordinate transformation matrix between the virtual model and the actual scene according to the position of the positioning patch in the virtual model and the actual position coordinates, and projects the virtual model according to the transformation matrix. The smart head display device identifies the spatial position and direction of the puncture guide according to the photosensitive patch, and adjusts the position and direction of the axis model according to the spatial position and direction of the puncture guide and projects it through the coordinate transformation matrix.

[0012] Optionally, the positioning patch module includes a base layer, on which a groove is provided for placing the positioning patch; the positioning patch includes a metal layer and a photosensitive layer, and the photosensitive layer is located above the metal layer; the photosensitive layer is used for identification of the smart head display device, and the metal layer is used for identification of the medical imaging device.

[0013] Optionally, the positioning patch further includes a filter film for filtering interfering light.

[0014] Optionally, the base layer further includes a hollow area for the puncture needle to pass through.

[0015] Optionally, the number of positioning patches is 5, and they are distributed on at least two straight lines.

[0016] Optionally, at least three circular photosensitive patches and one triangular photosensitive patch are provided on the puncture guide. The circular photosensitive patches are used for the smart head display device to identify the position of the puncture guide, and the triangular photosensitive patch is used for the smart head display device to identify the spatial posture of the puncture guide.

[0017] Optionally, a rectangular photosensitive patch is also provided on the puncture guide.

[0018] In a second aspect, the present application provides a thoracic surgery lung puncture navigation method, which uses the thoracic surgery lung puncture navigation system of the first aspect to assist in thoracic surgery lung surgery.

[0019] Optionally, the thoracic surgery lung puncture navigation method provided in this application further includes:

[0020] The positioning patch image in the image captured by the smart head display device is separated from the background to form the positioning patch outline;

[0021] Fitting is performed according to the positioning patch contour to obtain an elliptical curve of the positioning patch contour;

[0022] The coordinates of the center point of the fitted elliptic curve are used as the position coordinates of the positioning patch to obtain the actual coordinate positions of all patches.

[0023] Optionally, the thoracic surgery lung puncture navigation method provided in this application further includes:

[0024] Obtain the preoperative position of the localization patch, calculate the distance between each localization patch and other localization patches, construct a distance matrix of the reference model, and based on the distance matrix, arrange the distances from each patch to other patches in order to form a reference feature vector;

[0025] Collect the image features of the positioning patches, obtain the real-time position of each positioning patch during the operation, calculate the real-time distance between each positioning patch and other positioning patches, construct an actual distance matrix, and based on the actual distance matrix, arrange the actual distances from each patch to other patches in order to form an actual feature vector;

[0026] Perform one-to-one matching between the reference eigenvector and the actual eigenvector, and find the matching relationship that minimizes the difference between the actual eigenvector and the reference eigenvector;

[0027] According to the matching relationship between the reference feature vector and the reference feature vector, the mapping relationship between the positioning patch identified during the operation and the positioning patch before the operation is output.

[0028] The beneficial effects of this application are as follows:

[0029] In this application, by setting a puncture guide and importing the axis model of the puncture guide, the posture and position of the puncture guide can be quickly determined. When the doctor performs thoracic surgery lung puncture, he only needs to overlap the axis model of the puncture guide with the planned virtual path to quickly introduce the puncture needle, thereby quickly achieving puncture without the need to correct the puncture angle in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0031] Figure 1A framework diagram of a thoracic surgery lung puncture navigation system provided in an embodiment of the present application;

[0032] Figure 2 A schematic structural diagram of a positioning patch module in a thoracic surgery lung puncture navigation system provided in an embodiment of the present application;

[0033] Figure 3 This is a schematic structural diagram of a puncture guide in a thoracic surgery lung puncture navigation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following, in conjunction with the accompanying drawings, provides a clear and complete description of the technical solutions in the embodiments of this application. Obviously, the described embodiments represent only a portion of the embodiments of this application and do not constitute a complete set of embodiments. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this application without inventive effort are intended to fall within the scope of protection of this application. Unless otherwise defined, technical or scientific terms used in this application should have the same ordinary meanings as those understood by persons of ordinary skill in the art. The terms "first," "second," and similar expressions used in this application do not denote any order, quantity, or importance; they are merely used to distinguish between different components. Terms such as "include" or "comprising" mean that the element or object preceding the term includes the elements or objects listed after the term, and their equivalents, without excluding other elements or objects. Terms such as "connect," "couple," or "connected" are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used solely to indicate relative positional relationships. When the absolute position of the described objects changes, the relative positional relationships may also change accordingly.

[0035] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0036] The thoracic surgery lung puncture navigation system provided in the embodiment of the present application is as follows: Figure 1As shown, the system may include: medical imaging equipment, a model building module, a connection network, a puncture device, and a smart head-mounted display device. The model building module can be a physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The model building module converts images from the medical imaging device into a three-dimensional model and sends it to the smart head-mounted display device via the connection network. The medical imaging device may be a computed tomography (CT) scanner: which uses X-rays and complex computer processing to generate cross-sectional images of the body. CT scans can provide more detailed image information than ordinary X-rays. Magnetic resonance imaging (MRI): uses powerful magnetic fields and radio waves to create detailed images of organs and structures in the body. Ultrasound imaging equipment: uses high-frequency sound waves to produce real-time images of internal structures. Positron emission tomography (PET): A nuclear medicine imaging technology that generates three-dimensional images by detecting the distribution of radioactive substances in the body. The CT device used in the embodiments of this application. Before the CT device performs a scan, it is necessary to place a positioning patch on the surface of the patient's lesion skin. The CT scan is performed together with the positioning patch. The CT or MR medical image is processed through the model building module and converted into a three-dimensional model of organs, lesions, etc. The smart head display device receives the three-dimensional model and the planned puncture path from the server. The doctor wears the smart head display device, and the camera on the smart head display device scans the positioning patch and identifies the position of the positioning patch. Based on the spatial position of the positioning patch, the actual position relationship of the smart head display device relative to these markers is calculated. With accurate spatial information, the system can accurately superimpose pre-designed virtual images (such as anatomical structures, surgical paths, etc.) to the corresponding positions in the actual scene. For example, in surgical operations, it can help doctors see the specific location of blood vessels or organs hidden under the skin, thereby improving the accuracy of the operation.

[0037] The thoracic surgery lung puncture navigation system of the present embodiment also includes a positioning patch module, a smart head-mounted display device, a puncture needle, and a puncture guide. During thoracic surgery, medical imaging equipment typically uses a CT scanner. The CT scanner captures images of lung tissue and the positioning patch, generating an image of the lung tissue including the positioning patch, providing the data foundation for 3D modeling.

[0038] Positioning patch module, used for identification of optical and medical imaging equipment, uses flexible silicone pads to connect five positioning patches together, and the silicone pad is connected to the patient's body. The bottom of the silicone pad is connected to the patient's body with medical grade tape. The specific structure is as follows Figure 2As shown, the positioning patch module includes a base layer 1 with a groove for positioning patch 12. The base layer is annular with a hollow area 13 in the middle for the puncture needle to pass through. From bottom to top, the positioning patch 12 consists of a metal layer, a photosensitive layer, and a filter membrane. The photosensitive layer is used for identification by the smart head-mounted display device and is generally made of a material with good light reflectivity. The metal layer is used for identification by the CT device. The CT scan captures the metal layer and locates a specific location on the patient's body. When 3D modeling is performed based on the CT data, these positioning marks are also included on the 3D model. The smart head-mounted display device can detect the location of the positioning patch during scanning and then overlay the 3D model on the patient's body for matching. The filter membrane can filter visible light. During surgery, the positioning patch is exposed to the shadowless lamp. The filter membrane filters the filtered light to prevent reflection from the shadowless lamp light, thereby preventing it from affecting the doctor's vision and operation, and also preventing interference with the smart head-mounted display camera. In this embodiment of the application, there are five positioning patches, distributed on two parallel lines. The positioning patches are circular in shape, which facilitates the identification of the center point.

[0039] The identification method for positioning patches is as follows: First, preliminary noise reduction processing is performed on the images captured by AR glasses to remove noise from the image. Denoising can be performed using mean filtering, median filtering, bilateral filtering, wavelet transform, adaptive filtering, and deep learning-based methods. Mean filtering is a simple linear filtering method that replaces the value of a pixel by taking the average of the values of its neighborhood. This method effectively removes Gaussian noise. Median filtering is very effective for randomly appearing black and white dots in an image. It replaces the central pixel value with the median of all the pixels in its neighborhood, effectively removing noise while preserving edge information. Bilateral filtering combines spatial proximity and pixel value variance to perform a weighted average, preserving edges while reducing noise. It is suitable for situations where edge preservation is critical. Wavelet transform decomposes the image into detail and approximate components at different scales. The detail components are then thresholded to remove noise before reconstructing the image. Adaptive filtering adjusts filter parameters based on local image characteristics. For example, adaptive Wiener filtering dynamically adjusts the filter strength based on the information surrounding each pixel to achieve better denoising while preserving image features. Convolutional neural network (CNN)-based methods have also been applied to image denoising. These methods are able to learn complex noise patterns and effectively restore clear images from noisy images.

[0040] The next step is to separate the image of the positioning patch from the background. Because the patch contains a photosensitive layer, it will appear as a distinct light spot under the operating room lighting. Therefore, binarization is the preferred method for this separation. The specific separation method is as follows: First, the positioning patch image must be converted to grayscale. This is because color information is not important for detecting brighter light spots, and grayscale images simplify subsequent processing. If the original image has low contrast, contrast can be enhanced through methods such as histogram equalization or adaptive histogram equalization to make the light spots more distinct. Selecting an appropriate threshold is a key step in the binarization process. A suitable threshold, T, is required to separate pixels into two groups: pixels with a brightness greater than T (considered part of the light spot) and pixels with a brightness less than or equal to T (considered background). Either a global threshold or a local (adaptive) threshold can be selected. Global thresholding is suitable for uniform backgrounds. A threshold can be set manually or automatically calculated using an algorithm. Local / adaptive thresholding is suitable for uneven backgrounds, dynamically adjusting the threshold based on the local information surrounding each pixel. Finally, a binary image is generated by applying a selected threshold to the image, converting all pixel values to 0 or 1 (or 0 and 255 in the case of an 8-bit grayscale image). In this image, the foreground (i.e., the light spot) is usually white (1 or 255) and the background is black (0). To improve the quality of the binary image, some morphological operations can be performed, such as: erosion is used to eliminate small noise points. Dilation is used to fill small holes within the foreground object or connect adjacent objects. Opening (erosion followed by dilation) is used to improve smooth object boundaries and disconnect narrow connections. Closing (dilation followed by erosion) is used to help fill small holes inside objects. Connected domain analysis (also known as component labeling) can be used to identify and label independent regions in the binary image. The true light spot regions can then be screened based on features such as area and shape, and those small regions that do not meet the criteria or misjudged regions can be removed. The final step is to extract the light spot regions of interest from the processed image. This may involve further image processing steps such as contour detection and bounding box drawing to facilitate subsequent analysis or display.

[0041] The third step is to smooth the jagged edges of the positioning patch image through Gaussian filtering to form the positioning patch outline. The specific steps are as follows: obtain the edge gradient distribution map of the positioning patch image; dynamically calculate the standard deviation σ of the Gaussian kernel function based on the local curvature radius. The calculation formula of the standard deviation σ is:

[0042]

[0043] Here, κ is the local curvature value, k∈[0.5,1.5] is the adjustment coefficient, and ε=0.01 is the zero division prevention constant. A spatially adaptive anisotropic Gaussian filter is constructed, whose principal axis is aligned with the edge tangent direction. The anisotropic Gaussian filter is constructed as follows: a two-dimensional Gaussian kernel is decomposed into a tangential component G_t and a normal component G_n, where the standard deviation of the normal component σ_n=2σ_t, and σ_t=σ is the base standard deviation. The kernel function is expressed as: G(x,y)=G_t(x')·G_n(y'), where x' and y' are the projections of the point (x,y) in the image along the tangent-normal coordinate system. Multi-scale Gaussian convolution is performed along the normal direction of the curve, and weighted fusion is performed simultaneously across color channels. The multi-scale Gaussian convolution operation includes parallel filtering at three scales (σ = 0.8, 1.2, and 1.6). The final result is synthesized using a Laplacian pyramid fusion algorithm, where the fusion weights are positively correlated with the edge sharpness response value in scale space. After completing these steps, the output curve can be verified. The edge oscillation index (EI) of the smoothed image can be calculated. When the EI exceeds a preset threshold, the σ value is automatically increased and filtering is re-applied until the EI is ≤ 0.1 × the initial value and the structural similarity (SSIM) is ≥ 0.98. Finally, an image of the localized patch contour with smoothed edges is output.

[0044] The fourth step is to fit the elliptical curve. Although the positioning patch is circular, due to the angle problem, the shape of the positioning patch captured by the AR glasses is actually closer to an ellipse. Therefore, it is more accurate to use an elliptical curve to simulate the positioning patch shape than a circular curve. Therefore, the elliptical curve is fitted in this application. The following is a specific fitting method for the elliptical curve. Fitting is performed according to the contour of the positioning patch to preliminarily fit the elliptical curve of the positioning patch contour. The specific fitting process is as follows: Obtain a discrete point set {P i (x i ,y i ), i = 1, 2, ..., n}, Pi represents the i-th discrete point, xi, yi represent the horizontal and vertical coordinates of the i-th discrete point respectively; the ellipse fitting objective function is constructed based on the algebraic distance minimization criterion. The general quadratic curve equation of the ellipse in the two-dimensional plane is: Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0, where B 2 -4AC<0, A, B, C, D, E, and F are elliptic curve coefficients. For a discrete point P i (x i ,y i ), whose algebraic distance to the ellipse is defined as: d i =Ax i 2 +Bx i y i +Cyi 2 +Dx i +Ey i +F. The algebraic distance directly represents the residual when a point is substituted into the ellipse equation. The objective function aims to minimize the sum of the squares of the algebraic distances of all points, which can be solved by the least squares method. The objective function is as follows:

[0045]

[0046] In the embodiment of the present application, the dynamic weight coefficient is introduced to optimize the ellipse parameter solution process, and the weight coefficient w i =1 / (1+αd i 2 ), d i is the radial distance from the discrete point set to the current ellipse estimate, where α is the robustness adjustment factor, with a value range of α∈[0.1,1.0]. Finally, the ellipse parametric equation and error evaluation index that meet the preset fitting accuracy are output. The accuracy of the ellipse parametric equation is verified by the error evaluation index, which includes:

[0047] Average algebraic error: ε_alg=(Σw_i(Ax i 2 +Bx i y i +Cy i 2 +Dx i +Ey i +F) 2 ) / n;

[0048] Maximum geometric error: ε_geo=max{|d i -(a+b) / 2|}, where a and b are the major and minor axes of the ellipse;

[0049] Goodness-of-fit indices: y i Coordinate mean.

[0050] The positioning patch position extraction method in the embodiment of the present application accurately extracts the center position of the positioning patch by filtering out interference from indoor lighting and reflections from surgical instruments. Based on elliptical curve fitting, the center position of the patch can be accurately calculated regardless of the surgeon's position or the angle at which the positioning patch is collected. Binarization can be used to quickly and accurately extract the light spot of the positioning patch. Furthermore, the jagged edges of the light spot are smoothed, which helps to fit the closest elliptical curve for the light spot.

[0051] After identifying the center position of the positioning patch, the positioning patch is identified so that the position of each positioning patch can be distinguished. The specific identification method is as follows:

[0052] Get the preoperative positions of the positioning patches Q1, Q2, Q3, Q4, and Q5, calculate the distance between each positioning patch and other positioning patches, and construct the distance matrix of the reference model. If the coordinates of the five points are as follows: Q1 (0, 0, 0), Q2 (1, 0, 0), Q3 (2, 0, 0), Q4 (1, 1, 0), Q5 (2, 1, 0), then the distance matrix table of the five points is generated as shown in Table 1:

[0053] Table 1

[0054]

[0055] Based on the distance matrix, the distances from each patch to the other four patches are arranged in ascending order to form a reference feature vector V ref , for example, the eigenvector of the reference patch Q1 is V ref,1 , if the distances from positioning patch Q1 to the other four positioning patches are d 12 ,d 13 ,d 14 ,d 15 , and d 12 <d 13 <d 14 <d 15 , then V ref,1 =[d 12 ,d 13 ,d 14 ,d 15 ]. Reference feature vector set: F ref ={V ref,1 ,V ref,2 ,…,V ref,5}.

[0056] After constructing the positioning patch, the preoperative reference feature vector V ref Afterwards, AR medical glasses are used to collect data in real time during the operation. The real-time position of each positioning patch during the operation is obtained, the real-time distance between each positioning patch and other positioning patches is calculated, and an actual distance matrix is constructed. Based on the actual distance matrix, the actual distance from each patch to other patches is arranged in order to form an actual feature vector V curr Similarly, F curr ={V curr,1 ,V curr,2 ,…,V curr,5}.

[0057] Perform a one-to-one match between the reference eigenvector and the actual eigenvector, searching for the matching relationship that minimizes the difference between the actual and reference eigenvectors. A cost matrix consisting of the squared Euclidean distances between the reference and actual eigenvectors can be constructed. Using the Hungarian algorithm, matrix transformations and a covering strategy are employed to solve the minimum global cost matching problem using a linear sum assignment problem. This gradually finds the matching relationship that minimizes the difference between the actual and reference eigenvectors.

[0058] The specific method is as follows: construct a 5×5 matrix C, where each element C[i][j] represents the reference eigenvector V ref,i With the actual eigenvector V curr,j Difference of: C[i][j]=Diff(V ref,i ,V curr,j ).set up

[0059] Vref,1=[1.0,1.0,1.0,0.866],actual eigenvector Vcurr,2=[1.0,1.0,1.0,0.866],then:

[0060]

[0061] The Hungarian algorithm uses the following steps to find the matching with the lowest total cost: Row reduction: Subtract the minimum value of each row from the elements in that row, ensuring that there is at least one zero in each row. This simplifies the matrix to facilitate subsequent covering of zero elements. For example, suppose a row has the elements [3, 1, 4, 2, 5] and the minimum value of that row is 1. After reduction, the matrix becomes [2, 0, 3, 1, 4]. Column reduction: Subtract the minimum value of each column from the elements in that column (if there are already zeros in the column, they can be skipped). This further simplifies the matrix and increases the number of zero elements. Covering zero elements: Cover all zero elements with the minimum number of rows or columns. Mark the locations of all zero elements. Use a greedy strategy or DFS to find the minimum covering line (row or column). Covering rule: If the number of covering lines equals the matrix dimension, proceed directly to the step to determine the optimal matching; otherwise, proceed to the step to adjust the matrix. Determine the optimal matching: In the adjusted matrix, use backtracking or DFS to find a unique matching for each zero element, ensuring that there is only one matching per row and column. Output the matching list Matching = {(i, j)}, indicating that the reference patch i corresponds to the actual patch j.

[0062] In addition, you can also use brute force search and matching to traverse all possible permutations and combinations to find the permutation that minimizes the error between the two eigenvector matrices.

[0063] Finally, according to the matching relationship between the reference feature vector and the reference feature vector, the mapping relationship between the positioning patch identified during the operation and the positioning patch before the operation is output.

[0064] In addition, based on the mapping relationship between the positioning patch identified during the operation and the positioning patch before the operation, the transformation matrix between the two can be calculated, and the coordinates of the positioning patch identified during the operation can be projected to the coordinate system of the positioning patch before the operation to verify whether the projection error is less than the preset threshold. If the error is within the tolerance range, the identity of the actual patch is output; otherwise, re-matching is triggered. The covariance matrix can be calculated by singular value decomposition to capture the linear correlation between the two point sets, and the optimal rotation matrix can be obtained. The translation vector is calculated by the center of mass offset, and the actual coordinate system is projected to the reference coordinate system using the rotation matrix and the translation vector. Specifically, it can be implemented through the Kabsch algorithm, which includes the following steps: calculating the center of mass of the two position point sets of the positioning patch before and during the operation, translating the point set to the origin, calculating the covariance matrix, SVD decomposition, and calculating the translation. Regarding the transformation matrix, it is a mature technology in the field of AR technology, and the specific calculation process will not be repeated.

[0065] like Figure 3 As shown, the puncture guide 2 includes a main panel 21, which includes multiple photosensitive patches and a guide hole 26 that passes through the main panel for guiding the puncture needle. The guide hole 26 is formed by a cylindrical sleeve 25. At least three circular photosensitive patches 22, a triangular photosensitive patch 23 and a rectangular photosensitive patch 24 are set on the puncture guide. The circular photosensitive patch 22 is used by the smart head display device to identify the position of the puncture guide, and the triangular photosensitive patch 23 is used by the smart head display device to identify the spatial posture of the puncture guide along the axis, such as the axial angle, direction and angle with the horizontal plane. The triangle is composed of three non-collinear vertices, which are rotationally invariant and unique. Even if it is partially blocked, the shape can still be reconstructed through the remaining vertices. The triangle can quickly determine the direction through the difference in vertex coordinates (such as side length, angle), and assist in calculating the rotation angle and spatial posture of the object. The four corner points of the rectangular positioning patch provide more spatial reference data, which can reduce noise interference and improve positioning accuracy.

[0066] The steps for building a 3D model of lung tissue in the model building module are as follows:

[0067] CT image acquisition and preprocessing: High-resolution CT (typically slice thickness ≤ 1 mm) is used to ensure clear details in continuous slice images. Voltage 120 kV, current 200-300 mA, and pitch ≤ 1.0 are used to minimize motion artifacts. This approach preserves edges while reducing noise and enhancing the contrast between the lungs and surrounding tissues.

[0068] Lung tissue segmentation: Quantified Hounsfield Unit (HU) values are used to distinguish between air (lung parenchyma HU ≈ -600 to -900) and soft tissue (chest wall HU ≈ +30 to +70) for coarse segmentation. Closing operations are used to fill holes created by small blood vessels within the lungs. Erosion and expansion are used to remove interference from the trachea and bronchi. A U-Net model is used to train a segmentation network. CT slices are input and the network outputs pixel-level labels for lung lobes, nodules, or tumors.

[0069] 3D reconstruction: The segmented binary mask is converted into a triangular mesh using the Marching Cubes algorithm, reducing the number of triangles to adapt to the real-time rendering capabilities of AR devices. Ray casting directly visualizes the transparency and color of the original CT data, preserving internal structural details, but the computational complexity is high and requires GPU acceleration.

[0070] Model optimization and annotation: Hole filling ensures that the lung lobe surface is closed and undamaged. Gaussian smoothing or Laplace filtering is used to eliminate jagged edges. The left and right lung lobes, bronchi, and blood vessels are distinguished, and the location of lesions is marked, with different structures assigned different colors (e.g., red = tumor, blue = blood vessels).

[0071] The model building module constructs a virtual model of the lung tissue and the planned puncture path based on the lung tissue image. The doctor determines the location of the lesion by analyzing the CT image and can directly add the puncture path associated with the lung component to the model building module. The puncture path can be a straight line or a series of points distributed along the line. In this embodiment, the puncture path uses two points, which together define a straight line.

[0072] Because the CT scan includes both lung tissue and the positioning patch, the relative position of the positioning patch and the lung tissue can be determined. Therefore, the model construction module determines the coordinates of the positioning patch on the virtual model based on the relative position of the positioning patch and the lung tissue image. The model construction module can also construct an axis model for the guide hole based on the shape of the puncture guide. The model construction module can be a platform located on a local server or a network server.

[0073] When doctors perform lung puncture during thoracic surgery, they only need to overlap the axis model with the planned virtual path to quickly insert the puncture needle and complete the puncture quickly.

[0074] The smart head display device in the embodiment of the present application adopts AR glasses. The AR glasses can have a built-in data processing module or an external data processing module (connected to the data processing module on the server through the network). The position calculation of the patch, image processing, etc. are realized through the data processing module. In the embodiment of the present application, whether the AR glasses use a built-in or external data processing module, they all belong to the category of smart head display devices. In the embodiment of the present application, taking the AR glasses with a built-in data processing module as an example, the AR glasses are used to identify the actual position of the positioning patch on the positioning patch module, and the coordinate transformation matrix between the virtual model and the actual scene is determined according to the position of the positioning patch in the virtual model and the actual position coordinate relationship, and the virtual model is projected according to the transformation matrix. It should be noted that the AR glasses also identify the spatial position and spatial posture of the puncture guide based on the photosensitive patch, and adjust the position and direction of the axis model in real time according to the spatial position and posture of the puncture guide and project it through the coordinate transformation matrix.

[0075] AR medical glasses include a visual imaging device, a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The visual imaging device is used to capture the positioning patch image and send it to the processor; the memory is used to store computer programs; and the processor is used to implement the above-mentioned thoracic surgery lung puncture navigation method when executing the computer program stored in the memory.

[0076] The functions of each functional unit of the AR medical glasses provided in the embodiment of the present application can be achieved through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in the AR medical glasses provided in the embodiment of the present application will not be repeated here.

[0077] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0078] The communication interface is used for communication between the above electronic device and other devices.

[0079] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0080] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0081] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and systems according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] The computer program for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The computer program can be executed entirely on the target object computing device, partially on the target object device, as a separate software package, partially on the target object computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the target object computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0085] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0086] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0087] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods or systems. Therefore, the present application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware.

[0088] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0089] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A thoracic surgery lung puncture navigation system, characterized in that: The invention comprises a medical imaging device, a model building module, a positioning patch module, an intelligent head display device, a puncture needle and a puncture guide; the positioning patch module comprises a plurality of positioning patches; the medical imaging device is used to scan human lung tissue and the positioning patch module to generate a lung tissue image containing the positioning patches; the puncture guide comprises a main panel, the main panel comprises a plurality of photosensitive patches and a guide hole penetrating the main panel for guiding the puncture needle; the model building module constructs a virtual model including lung tissue and a planned puncture path according to the lung tissue image, and locates the puncture path according to the positioning patch in the lung tissue image. The position of the positioning patch on the virtual model is determined according to the position of the positioning patch, and the axis model of the guide hole is constructed according to the shape of the puncture guide; the smart head display device identifies the actual position of the positioning patch on the positioning patch module, determines the coordinate transformation matrix between the virtual model and the actual scene according to the position and actual position of the positioning patch in the virtual model, and projects the virtual model according to the coordinate transformation matrix; the smart head display device identifies the spatial position and direction of the puncture guide according to the photosensitive patch, and adjusts the position and direction of the axis model according to the spatial position and direction of the puncture guide and projects it through the coordinate transformation matrix.

2. The thoracic surgery lung puncture navigation system according to claim 1, characterized in that: The positioning patch module includes a base layer, on which a groove is provided for placing the positioning patch; the positioning patch includes a metal layer and a photosensitive layer, and the photosensitive layer is located above the metal layer; the photosensitive layer is used for identifying the smart head display device, and the metal layer is used for identifying the medical imaging device.

3. The thoracic surgery lung puncture navigation system according to claim 2, characterized in that: The positioning patch also includes a filter film for filtering interfering light.

4. The thoracic surgery lung puncture navigation system according to claim 3, characterized in that: The base layer also includes a hollow area for a puncture needle to pass through.

5. The thoracic surgery lung puncture navigation system according to claim 4, characterized in that: There are 5 positioning patches, and they are distributed on at least two straight lines.

6. The thoracic surgery lung puncture navigation system according to claim 1, characterized in that: At least three circular photosensitive patches and one triangular photosensitive patch are provided on the puncture guide plate. The circular photosensitive patches are used by the smart head display device to identify the position of the puncture guide plate, and the triangular photosensitive patch is used by the smart head display device to identify the spatial posture of the puncture guide plate.

7. The thoracic surgery lung puncture navigation system according to claim 6, characterized in that: A rectangular photosensitive patch is also provided on the puncture guide plate.

8. A thoracic surgery lung puncture navigation method, characterized by: A thoracic surgery lung puncture navigation system as claimed in any one of claims 1 to 7 is used to assist thoracic surgery lung surgery.

9. The thoracic surgery lung puncture navigation method according to claim 8, characterized in that: Also includes: The positioning patch image in the image captured by the smart head display device is separated from the background to form the positioning patch outline; Fitting is performed according to the positioning patch contour to obtain an elliptical curve of the positioning patch contour; The coordinates of the center point of the fitted elliptic curve are used as the position coordinates of the positioning patch to obtain the actual coordinate positions of all patches.

10. The thoracic surgery lung puncture navigation method according to claim 9, characterized in that: Also includes: Obtain the coordinates of the positioning patches in the constructed 3D model, calculate the distance between each positioning patch and other positioning patches in the 3D model, construct a distance matrix, and based on the distance matrix, arrange the distances of each patch to other patches in the 3D model in order to form a reference feature vector; The AR glasses collect the image features of the positioning patches, obtain the real-time position of each positioning patch on the AR glasses, calculate the real-time distance between each positioning patch and other positioning patches, construct an actual distance matrix, and based on the actual distance matrix, arrange the actual distances from each patch to other patches in order to form an actual feature vector. Perform one-to-one matching between the reference eigenvector and the actual eigenvector, and find the matching relationship that minimizes the difference between the actual eigenvector and the reference eigenvector; According to the matching relationship between the reference feature vector and the reference feature vector, the mapping relationship between the positioning patch identified during the operation and the positioning patch in the three-dimensional model is output.

Images