An augmented reality glasses assisted preoperative virtual localization system for pulmonary nodules

Abstract

The present application relates to the technical field of medical data processing, in particular to a kind of lung nodule preoperative virtual positioning system assisted by augmented reality glasses, including memory and processor, the processor executes the computer program stored in the memory, to realize: the construction of lung lobe tubular structure lung lobe tubular structure chart;According to the distance distribution between nodes and blood vessel attribute information, the basic attribute parameter of each edge is obtained, the current surface gripping point and lesion center point are acquired;Obtain transverse attenuation factor and axial projection factor, determine the deformation resistance weight of each edge in combination with basic attribute parameter;According to the deformation resistance weight of the edge accumulated between the nodes corresponding to the current surface gripping point and lesion center point, the deep displacement transmission degree is obtained;According to deep displacement transmission degree and the pulling direction of surgical instrument, the lesion center point position is corrected to obtain the corrected lesion position.The present application ensures that the virtual nodule rendered in AR glasses is accurately matched with the position of deep real lesion.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, specifically to an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules. Background Technology

[0002] Thoracoscopic lung nodule resection, as the mainstream minimally invasive treatment for lung nodules, relies on a three-dimensional model constructed from preoperative CT images. An augmented reality (AR) navigation system is used to overlay the virtual nodule onto the real-time intraoperative field of view to guide precise lesion resection. However, the core logic of existing AR navigation systems is "rigid registration," which assumes that lung tissue is a rigid object with a fixed relative position. It simply synchronizes the displacement of the lung surface markers to the deep lesions, ignoring the fact that lung tissue is actually an anisotropic complex of high-rigidity vascular bronchi and low-rigidity lung parenchyma. During surgery, when the surgeon pulls on the lung surface, the force transmission follows the law of forward force transmission and lateral blockage, that is, the force is mainly transmitted along the vascular bundle in a tunnel-like manner. If the pulling direction is perpendicular to the direction of the blood vessels or the force point is far away from the main blood vessels, the force will be quickly dissipated by the soft lung parenchyma, resulting in nonlinear deformation with large surface displacement and almost no movement of the deep lesions. This contradiction between the real mechanical properties and the rigidity assumption causes a positive overshoot error in the virtual nodule position. Summary of the Invention

[0003] To address the technical problem of virtual nodule positioning errors caused by neglecting the true mechanical properties of lung tissue in existing technologies, the present invention aims to provide an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules. The specific technical solution adopted is as follows: An augmented reality glasses-assisted preoperative virtual positioning system for lung nodules includes a memory and a processor, wherein the processor executes a computer program stored in the memory to perform the following steps: Target identification is performed based on the patient's chest CT scan data to construct a lung lobe tubular structure map, where the nodes in the lung lobe tubular structure map are the intersections or endpoints of the tubular structures; Based on the distance distribution between nodes and vascular attribute information in the lung lobe tubular structure diagram, the basic attribute parameters of each edge are obtained, and the current surface gripping point of the surgical instrument and the marked lesion center point in the CT scan data are obtained. Based on the basic attribute parameters of each side in the lung lobe tubular structure diagram and the distance distribution between the center point of each side and the traction direction of the surgical instrument, the transverse attenuation factor and axial projection factor of each side are analyzed. Combined with the basic attribute parameters of each side, the deformation resistance weight of each side in the lung lobe tubular structure diagram is determined. The degree of deep displacement transmission is obtained by accumulating the deformation resistance weight between the corresponding nodes of the current surface gripping point and the lesion center point; the position of the lesion center point is corrected according to the degree of deep displacement transmission and the traction direction of the surgical instrument, and the corrected lesion position is obtained.

[0004] Preferably, obtaining the basic attribute parameters of each edge based on the distance distribution between nodes and vascular attribute information in the lung lobe tubular structure diagram specifically includes: Based on the direction of blood flow between every two nodes in the lung lobe tubular structure diagram, the axial vector of the edge corresponding to each pair of nodes is determined; based on the distance distribution between the two nodes corresponding to each edge and the distribution of the blood vessel diameter corresponding to the edge, the deformation resistance coefficient of each edge is obtained; the basic attribute parameters of each edge in the lung lobe tubular structure diagram include the axial vector and the deformation resistance coefficient.

[0005] Preferably, the step of obtaining the deformation resistance coefficient of each edge based on the distance distribution between two nodes corresponding to each edge and the distribution of blood vessel diameters corresponding to the edge specifically includes: Obtain the blood vessel diameter corresponding to each edge. Based on the blood vessel diameter of each edge and the Euclidean distance between the two nodes of each edge, obtain the deformation resistance coefficient of each edge. The blood vessel diameter and the deformation resistance coefficient are positively correlated, and the Euclidean distance between the two nodes is negatively correlated with the deformation resistance coefficient.

[0006] Preferably, the step of analyzing the lateral attenuation factor and axial projection factor of each side based on the basic attribute parameters of each side in the lung lobe tubular structure diagram and the distance distribution between the center point of each side and the traction direction of the surgical instrument specifically includes: The traction principal axis vector is determined by pointing from the current position of the surface gripping point to the position of the hilum anchoring center point; the hilum anchoring center point is the centroid of all nodes corresponding to the root of the lung lobe and the blood vessel diameter being greater than a preset diameter threshold. Based on the distance distribution between the center point of each side in the lung lobe tubular structure diagram and the line containing the traction principal axis vector, as well as the magnitude of the traction principal axis vector, the lateral attenuation factor of each side is obtained. The axial projection factor of each side is obtained by using the unit vector of the axial vector of each side and the unit vector of the principal axial amount of traction in the diagram of the tubular structure of the lung lobe.

[0007] Preferably, the step of obtaining the lateral attenuation factor for each side based on the distance distribution between the center point of each side in the lung lobe tubular structure diagram and the straight line containing the traction principal axis vector, and the magnitude of the traction principal axis vector, specifically includes: The vertical Euclidean distance from the center point of each side in the lung lobe tubular structure diagram to the line containing the traction principal axis vector is obtained as the axial deviation distance of each side; the product between the preset bundle width ratio coefficient and the modulus of the traction principal axis is used as the force-bearing bundle width parameter. A decay function is constructed based on the axial offset distance and the force-bearing width parameter of each side to obtain the lateral decay factor of each side.

[0008] Preferably, the step of obtaining the axial projection factor of each side based on the unit vector of the axial vector of each side in the lung lobe tubular structure diagram and the unit vector of the principal axial amount of traction specifically includes: The absolute value of the dot product of the unit vector of the axial vector of each side and the unit vector of the main axial amount of the pull is raised to a preset power to obtain the axial projection factor of each side.

[0009] Preferably, the method for obtaining the deformation resistance weight of each side in the lung lobe tubular structure diagram specifically includes: The product of the deformation resistance coefficient, lateral attenuation factor, and axial projection factor of each edge is used as the current conduction admittance index of each edge; the negative correlation coefficient of the current conduction admittance index is used as the deformation retardation weight of each edge.

[0010] Preferably, the step of obtaining the degree of deep displacement transmission based on the deformation resistance weight accumulated between the corresponding nodes of the current surface gripping point and the lesion center point specifically includes: Based on the deformation resistance weight of the preferred path between the current surface gripping point and the corresponding node of the lesion center point, the average resistance of the preferred path is obtained; the degree of deep displacement transmission is obtained by negatively correlating the total resistance of the conduction path.

[0011] Preferably, the method of obtaining the average resistance of the preferred path based on the deformation resistance weight of the corresponding nodes between the current surface gripping point and the lesion center point specifically includes: Using a path planning algorithm, the optimal path is obtained when the sum of the deformation resistance weights of all edges is minimized, with the node closest to the current surface gripping point as the starting node and the node closest to the lesion center point as the ending node. The sum of the deformation resistance weights of all edges on the optimal path is then used as the total resistance of the conduction path.

[0012] Preferably, the step of correcting the position of the lesion center point based on the degree of deep displacement transmission and the traction direction of the surgical instruments to obtain the corrected lesion position specifically includes: Obtain the vector from the initial gripping point of the surgical instrument to the current gripping point on the surface, and form the surface displacement vector; The product between the surface displacement vector and the degree of deep displacement transmission is calculated, and the coordinates of the lesion center point are calculated and the product result is accumulated to obtain the corrected coordinates of the lesion location.

[0013] The embodiments of the present invention have at least the following beneficial effects: This invention first transforms the tubular structures of lung lobes from a patient's chest CT scan into a discretized diagram of these structures. By identifying the intersections and endpoints of the tubular structures as nodes, a computable digital anatomical framework is constructed, overcoming the limitation of quantitative analysis of continuous lung tissue and providing a fundamental topological carrier for subsequent mechanical property extraction and force transmission simulation. Then, on one hand, by using information such as node distances and vessel diameters, the axial vector and deformation resistance coefficient of each vessel edge are extracted, endowing the network diagram with mechanical analysis capabilities. On the other hand, the surface gripping points of surgical instruments are obtained to represent the dynamic force-bearing ends, and the preoperatively calibrated lesion center points are obtained to represent the static target ends, providing core data input for subsequent force transmission efficiency calculations. Furthermore, by integrating static mechanical properties with intraoperative dynamic traction constraints, deformation resistance weights that reflect the real-time force transmission laws within the lung tissue are generated, accurately characterizing the force transmission potential of each vessel segment and providing a quantitative basis for subsequent displacement transmission efficiency assessment. Finally, by accumulating the deformation resistance weights of the path between the surface gripping point and the lesion center point, the degree of deep displacement transmission is quantified. Then, combined with the traction direction, the original lesion coordinates are nonlinearly corrected, ultimately outputting a corrected lesion location consistent with the actual tissue deformation. This transforms abstract mechanical analysis into concrete visual navigation information, ensuring accurate matching between the virtual nodules rendered in the AR glasses and the actual deep lesion location, providing reliable guidance for determining the surgical resection range. Attached Figure Description

[0014] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a structural block diagram of an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules provided by the present invention; Figure 2 This is a flowchart of the steps of a preoperative virtual positioning method for lung nodules assisted by augmented reality glasses provided by the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] The following description, in conjunction with the accompanying drawings, details a specific scheme for an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules provided by this invention. For example... Figure 1 As shown, this embodiment of the invention provides an augmented reality glasses-assisted preoperative virtual positioning system for lung nodules, including a memory and a processor. The processor executes a computer program stored in the memory to implement the steps of an augmented reality glasses-assisted preoperative virtual positioning method for lung nodules.

[0019] Please see Figure 2 The diagram illustrates a flowchart of a preoperative virtual localization method for lung nodules assisted by augmented reality glasses, according to an embodiment of the present invention. The method includes the following steps: Step S100: Based on the CT scan data of the patient's chest, target recognition is performed to construct a lung lobe tubular structure map, wherein the nodes in the lung lobe tubular structure map are the intersections or endpoints of the tubular structures.

[0020] First, CT scan data of the patient's chest is acquired. In this embodiment, to provide a unified spatial measurement benchmark, a CT scan coordinate system is defined. This coordinate system serves as the absolute spatial benchmark for the entire process, and all subsequent anatomical structures and intraoperative instrument coordinates are uniformly transformed to this coordinate system. Specifically, the CT scan coordinate system is a coordinate system directly defined based on the geometric space of the CT image data itself, and it is usually also the absolute world coordinate system or reference coordinate system in all medical image processing workflows.

[0021] Then, in this embodiment, a region growing algorithm based on grayscale thresholds is used to identify targets in the CT scan data, obtaining the tubular structures in the target lung lobe of the patient, including the bronchial, pulmonary artery, and pulmonary vein regions. After segmentation, the centerlines of the obtained target lung lobe tubular structures are extracted using a morphological thinning algorithm. The intersections and endpoints of each centerline are used as nodes. Based on the actual connection relationships between the nodes and the centerlines, a lung lobe tubular structure map is constructed. It should be understood that the edges in the lung lobe tubular structure map represent the centerlines of the tubular structures between intersections or endpoints.

[0022] It should be noted that by setting the CT value range of the tubular structure of the target lung lobe as the grayscale threshold and using a region growing algorithm to divide the CT scan data, the extraction of the tubular structure of the lung lobe can be achieved. This is a well-known technique and will not be elaborated here. In some embodiments, the tubular structure of the target lung lobe can also be segmented from the CT scan data using a pre-trained U-Net neural network model, which will not be elaborated here either.

[0023] It should be understood that each node in the lung lobe tubular structure diagram corresponds to a three-dimensional coordinate in the CT scan coordinate system, and an edge between two nodes that have an edge connection relationship constitutes a blood vessel segment. This graph structure simplifies continuous biological tissue into a computable discrete topological network.

[0024] Step S200: Based on the distance distribution between nodes in the lung lobe tubular structure diagram and the vascular attribute information, obtain the basic attribute parameters of each edge, and acquire the current surface gripping point of the surgical instrument and the center point of the lesion marked in the CT scan data.

[0025] The main purpose of this step is to build a bridge between the preoperative static anatomical model and the real-time operation during the operation. By quantifying the mechanical properties of the vascular network and identifying key anatomical and operational points, core data support is provided for subsequent dynamic simulation of force transmission efficiency and accurate correction of lesion localization deviations.

[0026] Lung tissue, as an anisotropic complex composed of high-stiffness vascular bronchi and low-stiffness lung parenchyma, exhibits stress transmission highly dependent on the geometric orientation and stiffness characteristics of the vascular skeleton, with significant differences in the deformation resistance of different vascular layers. This characteristic necessitates clarifying the force transmission attributes of vascular segments to accurately simulate the force flow distribution under traction. Therefore, this step first extracts the axial vector (quantifying the force transmission direction) and deformation resistance coefficient (quantifying stiffness differences) of each vascular segment based on the distance distribution between nodes and vascular diameter attributes in the lung lobe tubular structure diagram, forming fundamental attribute parameters characterizing the vascular mechanical properties. Simultaneously, by tracking the real-time position of surgical instruments using augmented reality glasses, the intraoperative surface gripping points (dynamic force-bearing ends) are determined, and the lesion center point marked in the preoperative CT scan data (static target end) is retrieved to construct a mechanical transmission analysis framework between the force-bearing point and the target point. This series of operations not only builds upon the results of preoperative static anatomical modeling but also captures key information from intraoperative dynamic operations, laying a data foundation with clear attributes and precise locations for subsequent dynamic impedance field calculations and nonlinear displacement compensation.

[0027] Specifically, the basic attribute parameters of each side in the lung lobe tubular structure diagram include the axial vector and the deformation resistance coefficient.

[0028] The first step is to determine the axial vector of the corresponding edge between each pair of nodes based on the direction of blood flow in the lung lobe tubular structure diagram.

[0029] As a concrete example, for any edge corresponding to a blood vessel segment, obtain the two nodes of the edge, and then normalize the spatial vector from the proximal node to the distal node to obtain the axial vector of the edge.

[0030] It should be understood that the axial vector represents the macroscopic direction from the proximal end to the distal end, and the actual shape of the blood vessel may be curved (e.g., serpentine). However, in tensile mechanics analysis, the focus is on the overall directional capacity to bear tensile force. Simplifying the curved pipe segment into a straight beam vector connecting the two ends is a common approach in finite element analysis and simplified mechanical modeling, which can effectively characterize the guiding effect of the pipe segment on a macroscopic scale.

[0031] It should be noted that, for the two nodes corresponding to an edge, the proximal node is the node closer to the heart, and the distal node is the node farther from the heart. More specifically, the entrance of the main pulmonary artery or main bronchus is identified as the global root node in the topological graph. The root distance of a node is defined by calculating the shortest path length from any node in the graph along an edge to that root node. For any edge, the node with the smaller root distance between its two endpoints is the proximal node, and the node with the larger root distance is the distal node. The normalization of spatial vectors is performed by the ratio of the vector to its magnitude; normalization is to obtain a unit vector representing the direction.

[0032] Force is highly efficient when transmitted along the vascular direction (anterograde), but is easily dissipated by the lung parenchyma when transmitted laterally. The orientation of the blood vessels directly determines the main transmission path of force within the lung, and the axial vector of each side (vascular segment) is essentially a quantitative definition of the directionality of force transmission within the blood vessels.

[0033] The second step is to obtain the deformation resistance coefficient of each edge based on the distance distribution between the two nodes corresponding to each edge and the distribution of the blood vessel diameter corresponding to the edge.

[0034] Significant differences in stiffness exist between different vascular segments. Robust main vessels exhibit strong resistance to deformation and serve as the core channels for force transmission; conversely, small, terminal vessels have low stiffness and weak force transmission efficiency. This difference in stiffness directly determines the distribution of force transmission efficiency within the vascular network. Therefore, calculating the deformation resistance coefficient by combining the length (distance between nodes) and diameter distribution of vascular segments is crucial for transforming the geometric characteristics of blood vessels into a mechanical stiffness index, enabling an objective differentiation of the deformation resistance of different vascular segments.

[0035] Specifically, the diameter of the blood vessel corresponding to each edge is obtained, and the deformation resistance coefficient of each edge is obtained based on the blood vessel diameter of each edge and the Euclidean distance between the two nodes of each edge. The blood vessel diameter and the deformation resistance coefficient are positively correlated, and the Euclidean distance between the two nodes is negatively correlated with the deformation resistance coefficient.

[0036] As a concrete example, taking any edge as an illustration, sampling points are obtained at equal intervals along the center line corresponding to this edge. Using each sampling point as the center, the diameter of the largest inscribed sphere within the region containing each sampling point along this line is obtained. The arithmetic mean of the diameters of all sampling points along this edge is taken as the diameter of the blood vessel corresponding to that edge. It should be noted that the location of the sampling points along the center line corresponding to this edge is determined based on a preset sampling density (e.g., one sampling point every 5 mm) or a fixed number of sampling points (e.g., 10). There are no restrictions here; the implementer can choose according to the specific real-time scenario.

[0037] As a concrete example, based on the nonlinear mechanical properties of biological soft tissue, the vascular bundle, as the primary stress-supporting structure, depends mainly on its cross-sectional moment of inertia (proportional to the fourth power of the diameter) and geometric length (inversely proportional to the length). Therefore, this embodiment constructs comprehensive stiffness attribute characteristic parameters, and the method for obtaining the deformation resistance coefficient of the vascular segment corresponding to each edge can be expressed by the formula: ,in, This represents the deformation resistance coefficient of the edge. This indicates the diameter of the blood vessel on that side. This represents the Euclidean distance between the two nodes of the edge. This represents the normalization function; for example, the minimization normalization method can be used.

[0038] The deformation resistance coefficient of each side in the vascular network diagram is used to numerically distinguish between large main blood vessels (high stiffness, main force transmission channels) and small terminal microvessels (low stiffness, easily deformable areas).

[0039] Furthermore, the current surface gripping point of the surgical instruments and the marked center point of the lesion in the CT scan data are obtained.

[0040] It should be noted that in the CT scan coordinate system, the centroid coordinates of the lung nodule to be removed are marked by the doctor and marked as the center point of the lesion. This coordinate represents the absolute position of the nodule in the state of not being subjected to external traction.

[0041] Since the movement of surgical instruments occurs in the camera coordinate system, the anatomical model is located in the CT scan coordinate system, and the force has a clear direction, a unified spatial reference is needed.

[0042] The first step is to calculate the image-visual registration matrix by matching patient chest markers or using magnetic localization registration technology.

[0043] Specifically, the coordinates of preoperative chest markers are recorded to form a first coordinate set A. The same set of markers on the patient's body is then identified using the AR glasses' camera or probe, and their coordinates in the camera coordinate system are recorded as a second coordinate set B. The chest markers can be selected from locations with obvious anatomical bony landmarks. Further, singular value decomposition is used to calculate the rotation matrix and translation vector that map each coordinate point in the second coordinate set B to each coordinate point in the first coordinate set A. These are combined to obtain a transformation matrix, which is the image-visual registration matrix. This matrix is ​​used to convert the real-time coordinate data from the augmentation glasses (camera coordinate system) to the CT scan coordinate system. It should be noted that singular value decomposition is a well-known technique and will not be elaborated upon here.

[0044] The second step is to obtain the original coordinates of the tip of the surgical instrument output by the augmented reality glasses at the current moment, and to perform spatial transformation using the image-visual registration matrix to obtain the coordinates of the instrument tip in the CT scan coordinate system. By monitoring the state of the surgical instrument, when the instrument is closed and located on the lung surface, the spatial point corresponding to the coordinates of the instrument tip is taken as the current surface gripping point.

[0045] Step S300: Based on the basic attribute parameters of each side in the lung lobe tubular structure diagram and the distance distribution between the center point of each side and the traction direction of the surgical instrument, analyze the lateral attenuation factor and axial projection factor of each side, and determine the deformation resistance weight of each side in the lung lobe tubular structure diagram by combining the basic attribute parameters of each side.

[0046] The main purpose of this step is to integrate the static mechanical properties of blood vessels with the dynamic traction constraints during surgery, quantify the degree of force transmission blockage in each blood vessel segment, and generate deformation blockage weights that reflect the mechanical distribution inside the lung tissue in real time. This provides a core basis for subsequent searching of the minimum impedance force transmission path and accurate calculation of deep displacement transmission rate.

[0047] Lung tissue, as an anisotropic complex composed of high-stiffness vascular bronchi and low-stiffness lung parenchyma, exhibits significant anterograde unimpeded force transmission and lateral obstruction. Force transmission efficiency depends not only on the mechanical properties of the blood vessels themselves but also on the dynamic influence of the traction direction and the spatial position and orientation of the vessels. Therefore, this step first clarifies the force transmission direction and stiffness potential of the blood vessels based on the fundamental attribute parameters (axial vector, deformation resistance coefficient) of each side in the lung lobe tubular structure diagram. Then, combining the distance distribution between the center point of each side and the traction direction, the spatial deviation of the blood vessel relative to the traction axis is analyzed, and the lateral attenuation factor (quantifying the dissipation effect of spatial position on force transmission) is calculated. Simultaneously, based on the matching degree between the vascular axial vector and the traction direction, the axial projection factor (quantifying the promoting effect of directional consistency on force transmission) is derived. Finally, by integrating the static fundamental attribute parameters and the dynamic dual factors, the inherent stiffness of the blood vessel segment and the force transmission efficiency under current traction are transformed into calculable deformation obstruction weights. This process achieves deep coupling between static anatomical models and dynamic surgical operations, accurately reproducing the nonlinear laws of force transmission within lung tissue, and providing crucial mechanical quantitative support for solving the positioning deviation problem caused by skin movement without muscle movement.

[0048] In this regard, the sub-steps of step S300 can be implemented by steps S301 to S304.

[0049] Step S301: Determine the traction principal axis vector by pointing from the current position of the surface gripping point to the position of the hilum anchoring center point; the hilum anchoring center point is the centroid of all nodes corresponding to the root of the lung lobe and the blood vessel diameter is greater than the preset diameter threshold.

[0050] First, nodes located at the root of the lung lobe are obtained from the lung lobe tubular structure diagram. From these nodes, all nodes with a vessel diameter greater than a preset diameter threshold are selected, and the centroid of all these nodes is used as the anchoring center point of the lung hilum. The diameter threshold can be set to 10 mm, which can be adjusted by the implementer based on the vessel distribution in the specific implementation scenario.

[0051] It should be understood that the hilar anchoring center point is the equivalent geometric anchor point of the rigid connection area between the lung root and the mediastinum. It corresponds to the anatomical region containing large structures such as the main bronchus, pulmonary arteries and veins. During traction operations, it exhibits almost no displacement and serves as a fixed benchmark for mechanical calculations. The hilar anchoring center point provides a fixed boundary condition with zero displacement for the lung tissue traction mechanical model, ensuring the convergence of the force field calculation.

[0052] Then, the direction from the current surface gripping point to the lung hilum anchoring center point is taken as the direction of the vector, and the Euclidean distance between the current surface gripping point and the lung hilum anchoring center point is taken as the magnitude of the vector, to obtain the traction principal axis vector. The direction of the traction principal axis vector represents the main direction of the current force transmission.

[0053] To prevent calculation divergence caused by misoperation leading to the grasping point coinciding with the hilum of the lung, if the modulus of the traction axis is less than a preset safety threshold (e.g., 20 mm), it is determined that there is no effective traction operation. In this case, all subsequent calculation steps that depend on the traction direction are skipped, and the degree of deep displacement transmission R(t) is directly set to 0, meaning that the lesion location is not corrected.

[0054] Step S302: Based on the distance distribution between the center point of each side in the lung lobe tubular structure diagram and the line containing the traction principal axis vector, as well as the magnitude of the traction principal axis vector, the lateral attenuation factor of each side is obtained.

[0055] This step quantifies the effect of spatial location on force dissipation. Lung parenchyma, as a porous viscoelastic medium, exhibits a significant damping effect on stresses deviating from the main conduction path. The farther the vascular segment is from the main traction axis, the more pronounced the force dissipation and the lower the force transmission efficiency. By combining the distance distribution between the center point of the vascular segment and the main traction axis, as well as the modulus of the main traction axis (simulating stress diffusion effects), the lateral attenuation factor can be calculated. This accurately characterizes the spatial force transmission potential of the blood vessel, effectively shielding weak force-transmitting vascular segments far from the main axis, ensuring that subsequent weight calculations focus on the core force-transmitting region.

[0056] Specifically, the first step is to obtain the vertical Euclidean distance from the center point of each side in the lung lobe tubular structure diagram to the line containing the traction principal axis vector as the axial deviation distance of each side.

[0057] As a concrete example, the midpoint between the two nodes of each edge is taken as the center point of each edge. Then, based on the distance calculation method from point to line, the axial offset distance of each edge is obtained, which reflects the vertical distance from the center point of the corresponding blood vessel segment to the main traction axis. This distance feature is independent of the traction direction and only focuses on the spatial distance.

[0058] The second step is to use the product of the preset beam width ratio coefficient and the modulus of the pulling main axis as the force beam width parameter.

[0059] The process of obtaining the force beam width parameter simulates the law that the longer the traction distance, the wider the force wave range. For example, when traction is applied to the distal end of the lung lobe, blood vessels at a slightly more distant point can also participate in force transmission, which is more in line with real anatomical mechanics.

[0060] As a concrete example, the preferred range for the bundle width ratio factor is 0.15 to 0.25, an empirical estimate based on the aspect ratio of the human lung lobe. This bundle width ratio factor is set to simulate the stress diffusion effect under Saint-Venant's Principle. In soft tissue traction, stress transfer is not solely along a single axis, but rather diffuses towards the root in a "cone-like" pattern. A larger modulus of the principal axial moment of traction results in a longer traction distance and a wider cross-section affected by stress upon reaching the root.

[0061] The third step is to construct an attenuation function based on the axial offset distance and the force beam width parameter of each side to obtain the lateral attenuation factor of each side.

[0062] As a concrete example, taking the i-th edge as an illustration, the method for obtaining the lateral attenuation factor of the i-th edge in the lung lobe tubular structure diagram can be expressed by the formula: in, denoted by , which represents the lateral attenuation factor of the i-th edge in the lung lobe tubular structure diagram, and t represents the current time, which is also the acquisition time of the current surface gripping point. This represents the axial offset distance of the i-th side in the diagram of the tubular structure of the lung lobe. Indicates the force-bearing width parameter. This represents an exponential function with the natural constant e as its base.

[0063] Considering that force dissipation is nonlinear, it is slow at close range and accelerates dramatically at long range. (Square term) It can amplify the attenuation effect over long distances, achieving nonlinear attenuation that is slow in the near term and steep in the far term, which is more in line with the physical characteristics of lung parenchyma.

[0064] This embodiment is based on the core form of the Gaussian function, and is optimized and derived by combining the axial offset distance and the force-bearing width parameters. As a core molecular term, it amplifies the attenuation effect over long distances, achieving nonlinear dissipation simulation with a gradual decrease in the near term and a steeper decrease in the far term. As the denominator, through the force-bearing width parameter Adapting to attenuation rates in different traction scenarios, The larger the value, the smoother the decay. The smaller the value, the steeper the decay.

[0065] The lateral attenuation factor for each side quantifies the lateral damping effect of the vessel segment. A larger value indicates that the vessel segment is close to the traction axis, with low impedance and high displacement transmissibility. A smaller value indicates that the vessel segment is farther away from the traction axis, with high impedance and low displacement transmissibility.

[0066] Step S303: Based on the unit vector of the axial vector of each side in the lung lobe tubular structure diagram and the unit vector of the traction principal axis, obtain the axial projection factor of each side.

[0067] This step quantifies the promoting effect of directional matching on force transmission. The mechanical properties of vascular bundles are similar to reinforcing fibers, exhibiting high force transmission efficiency only under axial tension, and easily dissipating force under lateral tension. By calculating the unit vector projection relationship between the axial vector of the vascular segment and the principal axial amount of tension, the directional consistency of the two can be quantified. Force transmission inhibition is applied to vascular segments with perpendicular directions, while force transmission weight is enhanced for vascular segments with parallel directions, accurately reconstructing the micromechanical mechanism of unidirectional force transmission and lateral isolation.

[0068] Specifically, the absolute value of the dot product of the unit vector of the axial vector of each side and the unit vector of the pulling principal axis is raised to a preset power to obtain the axial projection factor of each side.

[0069] As a concrete example, taking the i-th edge as an illustration, the method for obtaining the axial projection factor of the i-th edge in the lung lobe tubular structure diagram can be expressed by the formula: in, denoted by , which represents the axial projection factor of the i-th edge in the lung lobe tubular structure diagram, and t represents the current time, which is also the acquisition time of the current surface gripping point. Let be the unit vector representing the axial vector of the i-th side in the diagram of the tubular structure of the lung lobe. This represents the unit vector of the traction principal axis at the current gripping point on the surface. This represents a preset coefficient, with a preferred value of 2 or 3.

[0070] This indicates that the dot product of two unit vectors yields the cosine of the angle between them, directly quantifying the degree of parallelism in the corresponding directions. This indicates that the vector dot product operation result is subjected to exponentiation. The core function of setting the preset coefficient is to apply nonlinear penalty to the directional deviation between the axial vector and the traction principal axis vector, so that the force transmission efficiency of the nearly vertical blood vessel drops to 0 quickly, which is more in line with the force transmission law of real lung tissue. That is, if the blood vessel deviates slightly from the traction direction, the force transmission efficiency will decrease significantly.

[0071] It should be noted that, since the axial vector of each side obtained in this embodiment is a unit vector that only represents the direction, it can be directly used in the calculation of the axial projection factor. The method for obtaining the unit vector of the traction principal axis can be the ratio of the traction principal axis vector to its magnitude, which will not be elaborated here.

[0072] The axial projection factor of the vessel segment corresponding to each edge represents the degree of directional matching between the vessel segment and the traction axis. The larger the value, the more parallel the vessel orientation is to the traction axis, and the higher the force transmission efficiency of the vessel. When the value is closer to 0, it means that the vessel orientation is almost perpendicular to the traction axis, and the vessel is subjected to lateral force, resulting in low force transmission efficiency.

[0073] Step S304: Based on the deformation resistance coefficient, lateral attenuation factor and axial projection factor of each side in the lung lobe tubular structure diagram, obtain the deformation resistance weight of each side.

[0074] Specifically, the product of the deformation resistance coefficient, lateral attenuation factor, and axial projection factor of each edge is used as the current conduction admittance index of each edge; the negative correlation coefficient of the current conduction admittance index is used as the deformation retardation weight of each edge. This quantization process aims to integrate static stiffness with dynamic geometric factors, transforming them into edge weights that characterize the degree of force transmission retardation.

[0075] As a concrete example, the method for obtaining the deformation retardation weight can be expressed as: ,in, This represents the deformation resistance weight of the i-th edge in the diagram of the lung lobe tubular structure. represents the current conduction admittance index of the i-th side in the lung lobe tubular structure diagram, and t represents the current time, which refers to the parameter calculated under the traction principal axis vector corresponding to the current surface gripping point obtained at the current time. Wherein, For example, the smallest positive number (e.g.) (), used to prevent division by zero errors.

[0076] Deformation retardation weight A larger value indicates that the path is less likely to transmit displacement under the current force. Furthermore, after calculating the vascular segments corresponding to all edges in the lung lobe tubular structure diagram, all deformation resistance weights are updated to the corresponding edge attributes of the lung lobe tubular structure diagram, generating the deformation resistance weight map for the current moment. This map reflects in real time the distribution of the "mechanical stiffness" of every inch of tissue inside the lung under a specific traction operation.

[0077] Step S400: Based on the deformation resistance weight accumulated between the corresponding nodes of the surface gripping point and the lesion center point, the degree of deep displacement transmission is obtained; based on the degree of deep displacement transmission and the traction direction of the surgical instrument, the position of the lesion center point is corrected to obtain the corrected lesion position.

[0078] The main purpose of this step is to transform the abstract distribution of mechanical resistance into a concrete lesion localization correction. By quantifying the transmission efficiency of surface traction displacement to deep lesions, the corrected lesion location that conforms to the actual tissue deformation law is output, solving the localization deviation problem of "skin moves but flesh does not" caused by traditional rigid registration, and providing accurate visual guidance for surgical navigation.

[0079] As an anisotropic complex, lung tissue does not uniformly transmit displacement caused by surface traction to deeper tissues. As force is transmitted along the vascular skeleton, it gradually dissipates due to the accumulation of resistance weights, and the transmission efficiency is negatively correlated with the total resistance along the path. Therefore, firstly, taking the vascular node corresponding to the current surface gripping point and the lesion center point as the starting and ending point, along the path of least impedance force transmission within the lung tissue, the deformation resistance weights along all vascular edges are accumulated. This total resistance is then converted into the degree of deep displacement transmission; that is, the smaller the resistance, the higher the transmission degree, and the easier it is for the lesion to follow surface displacement; conversely, the transmission degree is low, and the lesion remains almost stationary.

[0080] Then, combining the traction direction of the surgical instruments (i.e., the directional attribute of the traction principal axis vector), the surface displacement is nonlinearly attenuated according to the degree of transmission, and finally superimposed on the coordinates of the lesion center point calibrated before surgery to obtain the corrected lesion position that conforms to the real-time tissue deformation. This process realizes a closed loop of mechanical analysis → displacement quantification → visual compensation, ensuring that the virtual nodule rendered in the AR glasses is always consistent with the location of the deep real lesion, providing core technical support for the precise resection of lung nodules.

[0081] The method for obtaining the degree of deep displacement transmission is implemented through the following steps.

[0082] The first step is to obtain the average resistance of the preferred path based on the deformation resistance weight of the nodes corresponding to the current surface gripping point and the lesion center point.

[0083] Specifically, using a path planning algorithm, the node closest to the current surface gripping point is taken as the starting node, and the node closest to the lesion center point is taken as the ending node. The optimal path is obtained when the sum of the deformation resistance weights of all edges is minimized. The sum of the deformation resistance weights of all edges on the optimal path is taken as the total resistance of the conduction path.

[0084] As a concrete example, using Dijkstra's algorithm or A* algorithm, in the deformation stagnation weight graph at the current moment, we search for the path connecting the starting node and the ending node. The path with the minimum sum of deformation stagnation weights of all edges traversed is recorded as the preferred path.

[0085] A smaller total blockage value in the preferred conduction path indicates that the path is a direct pathway composed of robust blood vessels and oriented in the same direction as the traction (anterograde traction). A larger value indicates that the path contains a large number of small blood vessels, or the path is perpendicular to the traction direction, or the path is forced to cross non-vascular areas (lateral traction).

[0086] The second step is to perform negative correlation processing on the total resistance of the conduction path to obtain the degree of deep displacement transmission.

[0087] As a concrete example, the method for obtaining the degree of deep displacement transfer can be expressed by the formula: ,in, This indicates the degree of deep displacement transmission at the current moment. This represents the total obstruction of the conduction path; t represents the current time. It should be noted that in this embodiment, t is used to represent the total obstruction of the conduction path. The parameters identified represent the parameters calculated based on the traction principal axis vector corresponding to the current surface gripping point obtained at the current moment.

[0088] The total resistance of the conduction path characterizes the cumulative impedance along the traction path of the transmitted force. The quantification of deep displacement transmission essentially uses mathematical methods to simulate force transmission loss, allowing the displacement correction of the virtual nodule to no longer rigidly follow, but rather transmit on demand, matching the actual deformation pattern of lung tissue. Since the displacement transmission efficiency of soft tissue is negatively correlated with cumulative resistance and exhibits nonlinear attenuation characteristics, the cumulative resistance is transformed into a dimensionless transmission characteristic value.

[0089] When there is a smooth force transmission channel, the total obstruction of the conduction path is small, the degree of deep displacement transmission is large, and the deep lesion almost completely follows the surface movement; when the force transmission is obstructed, the total obstruction of the conduction path is large, the degree of deep displacement transmission is small, and the deep lesion remains stationary, that is, the phenomenon of "skin moving but flesh not moving" occurs.

[0090] Furthermore, the method for obtaining the corrected lesion location is as follows: The first step is to obtain the vector from the initial gripping point of the surgical instrument to the current gripping point on the surface, thus forming the surface displacement vector.

[0091] As a concrete example, the initial gripping point refers to the position where the surgical instrument first stably grips the surface of the lung lobe at the start of this traction operation, denoted as t=0. The surface displacement vector is the vector pointing from the initial gripping point at t=0 to the surface gripping point at the current time t.

[0092] The surface displacement vector characterizes the total deformation amplitude and direction applied to the lung surface by the physician's traction operation. The direction of the surface displacement vector refers to the direction from the initial gripping point to the current surface gripping point, and the magnitude of the surface displacement vector refers to the Euclidean distance between the current surface gripping point and the initial gripping point.

[0093] The second step is to calculate the product between the surface displacement vector and the degree of deep displacement transmission, and then calculate the coordinates of the lesion center point and sum the product results to obtain the corrected coordinates of the lesion location.

[0094] in, This represents the coordinates of the corrected lesion location. This indicates the degree of deep displacement transmission at the current moment. This represents the coordinates of the center point of the lesion. This represents the surface displacement vector.

[0095] The surface displacement vector is the total displacement in three-dimensional space (including components in the x, y, and z directions). The degree of deep displacement transmission is a coefficient with a value range of 0-1. Multiplying the two results in the displacement components in each direction being attenuated according to the transmission rate. Multiplying the two preserves the directional characteristics of the displacement, ensuring that the displacement direction of the lesion is consistent with the surface traction direction, with only the amplitude attenuated according to mechanical laws.

[0096] The product result represents the effective displacement after correction by mechanical laws. Then, by using the vector spoofing rule, the original lesion location can be added to the effective displacement after correction by mechanical laws to obtain the true location of the nodule.

[0097] The virtual nodule position is no longer a simple rigid follower, but a dynamic position that conforms to the actual tissue mechanical response, based on the current traction location and direction.

[0098] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An augmented reality glasses-assisted preoperative virtual positioning system for lung nodules, comprising a memory and a processor, characterized in that, The processor executes the computer program stored in the memory to perform the following steps: Target identification is performed based on the patient's chest CT scan data to construct a lung lobe tubular structure map, where the nodes in the lung lobe tubular structure map are the intersections or endpoints of the tubular structures; Based on the distance distribution between nodes and vascular attribute information in the lung lobe tubular structure diagram, the basic attribute parameters of each edge are obtained, and the current surface gripping point of the surgical instrument and the marked lesion center point in the CT scan data are obtained. Based on the basic attribute parameters of each side in the lung lobe tubular structure diagram and the distance distribution between the center point of each side and the traction direction of the surgical instrument, the transverse attenuation factor and axial projection factor of each side are analyzed. Combined with the basic attribute parameters of each side, the deformation resistance weight of each side in the lung lobe tubular structure diagram is determined. The degree of deep displacement transmission is obtained by accumulating the deformation resistance weight between the corresponding nodes of the current surface gripping point and the lesion center point. Based on the degree of deep displacement and the traction direction of the surgical instruments, the position of the lesion center point is corrected to obtain the corrected lesion position.

2. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 1, characterized in that, The basic attribute parameters of each edge are obtained based on the distance distribution between nodes and vascular attribute information in the lung lobe tubular structure diagram, specifically including: Based on the direction of blood flow between every two nodes in the lung lobe tubular structure diagram, the axial vector of the edge corresponding to each pair of nodes is determined; based on the distance distribution between the two nodes corresponding to each edge and the distribution of the blood vessel diameter corresponding to the edge, the deformation resistance coefficient of each edge is obtained; the basic attribute parameters of each edge in the lung lobe tubular structure diagram include the axial vector and the deformation resistance coefficient.

3. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 2, characterized in that, The deformation resistance coefficient of each edge is obtained based on the distance distribution between two nodes corresponding to each edge and the distribution of blood vessel diameters corresponding to the edge. Specifically, this includes: Obtain the blood vessel diameter corresponding to each edge. Based on the blood vessel diameter of each edge and the Euclidean distance between the two nodes of each edge, obtain the deformation resistance coefficient of each edge. The blood vessel diameter and the deformation resistance coefficient are positively correlated, and the Euclidean distance between the two nodes is negatively correlated with the deformation resistance coefficient.

4. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 2, characterized in that, The method involves analyzing the lateral attenuation factor and axial projection factor of each side based on the basic attribute parameters of each side in the lung lobe tubular structure diagram and the distance distribution between the center point of each side and the traction direction of the surgical instrument. Specifically, this includes: The traction principal axis vector is determined by pointing from the current position of the surface gripping point to the position of the hilum anchoring center point; the hilum anchoring center point is the centroid of all nodes corresponding to the root of the lung lobe and the blood vessel diameter being greater than a preset diameter threshold. Based on the distance distribution between the center point of each side in the lung lobe tubular structure diagram and the line containing the traction principal axis vector, as well as the magnitude of the traction principal axis vector, the lateral attenuation factor of each side is obtained. The axial projection factor of each side is obtained by using the unit vector of the axial vector of each side and the unit vector of the principal axial amount of traction in the diagram of the tubular structure of the lung lobe.

5. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 4, characterized in that, The method for obtaining the lateral attenuation factor for each side based on the distance distribution between the center point of each side in the lung lobe tubular structure diagram and the line containing the traction principal axis vector, as well as the magnitude of the traction principal axis vector, specifically includes: The vertical Euclidean distance from the center point of each side in the lung lobe tubular structure diagram to the line containing the traction principal axis vector is obtained as the axial deviation distance of each side; the product between the preset bundle width ratio coefficient and the modulus of the traction principal axis is used as the force-bearing bundle width parameter. A decay function is constructed based on the axial offset distance and the force-bearing width parameter of each side to obtain the lateral decay factor of each side.

6. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 4, characterized in that, The axial projection factor for each side is obtained by combining the unit vector of the axial vector of each side in the lung lobe tubular structure diagram with the unit vector of the principal axial amount of traction. Specifically, this includes: The absolute value of the dot product of the unit vector of the axial vector of each side and the unit vector of the main axial amount of the pull is raised to a preset power to obtain the axial projection factor of each side.

7. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 2, characterized in that, The method for obtaining the deformation resistance weight of each side in the lung lobe tubular structure diagram specifically includes: The product of the deformation resistance coefficient, lateral attenuation factor, and axial projection factor of each edge is used as the current conduction admittance index of each edge; the negative correlation coefficient of the current conduction admittance index is used as the deformation retardation weight of each edge.

8. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 2, characterized in that, The degree of deep displacement transmission is obtained by accumulating the deformation resistance weight between the corresponding nodes of the current surface gripping point and the lesion center point, specifically including: Based on the deformation resistance weight of the preferred path between the current surface gripping point and the corresponding node of the lesion center point, the average resistance of the preferred path is obtained; the degree of deep displacement transmission is obtained by negatively correlating the total resistance of the conduction path.

9. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 8, characterized in that, The step of obtaining the average resistance of the preferred path based on the deformation resistance weight of the corresponding nodes between the current surface gripping point and the lesion center point specifically includes: Using a path planning algorithm, the optimal path is obtained when the sum of the deformation resistance weights of all edges is minimized, with the node closest to the current surface gripping point as the starting node and the node closest to the lesion center point as the ending node. The sum of the deformation resistance weights of all edges on the optimal path is then used as the total resistance of the conduction path.

10. The augmented reality glasses-assisted preoperative virtual positioning system for lung nodules according to claim 2, characterized in that, The process of correcting the lesion's center point position based on the degree of deep displacement transmission and the traction direction of the surgical instruments to obtain the corrected lesion position specifically includes: Obtain the vector from the initial gripping point of the surgical instrument to the current gripping point on the surface, and form the surface displacement vector; The product between the surface displacement vector and the degree of deep displacement transmission is calculated, and the coordinates of the lesion center point are calculated and the product result is accumulated to obtain the corrected coordinates of the lesion location.

Images