Photoacoustic microscopic surgical navigation method and system based on simultaneous localization and mapping
Patent Information
- Application Number
- CN202310281413.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-22
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2043-03-22
AI Technical Summary
[0004]发明目的在于针对现有技术不足，提出一种基于同步定位与建图的光声显微手术导航方法及系统，通过手持的自由扫描构建手术部位完整血管网络，实现光声显微手术导航在术中的实时成像扩展更新，可解决光声显微成像中成像视场小、扫描自由度低，以及术中成像更新难的问题
[0039] 1. The method and system of this invention are mainly applied to surgical navigation in photoacoustic microscopy. Addressing the problems of low scanning freedom and small imaging field of view in existing photoacoustic microscopy, which makes it difficult to achieve global observation of lesions, this invention employs a method based on synchronous positioning and mapping to locate the scanner pose and construct a large-field-of-view three-dimensional photoacoustic vascular network. This enables photoacoustic microscopy imaging that can be freely scanned with a handheld scanner and whose field of view can be updated and expanded in real time during surgery, improving the clinical application value of photoacoustic microscopy and providing a new solution for surgical navigation.
Smart Images
Figure CN116459006B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer-aided medical technology, specifically relating to a photoacoustic microsurgical navigation method and system based on synchronous positioning and mapping. Background Technology
[0002] Photoacoustic microscopy is an emerging medical imaging technology that generates photoacoustic signals by specifically exciting tissues with non-invasive nanosecond pulsed lasers. Imaging is achieved by exploiting the differences in laser absorption by different tissues, combining the high resolution of optical imaging with the deep penetration of acoustic imaging. Surgical navigation methods based on modalities such as MRI and CT require the injection of contrast agents in surgeries involving blood vessels and suffer from problems such as poor vascular resolution and ionizing radiation. In contrast, photoacoustic microscopy can image microvascular and other minute biological tissue structures with high resolution without contrast agents, offering significant advantages. Therefore, it is increasingly being applied in the field of surgical navigation, for example: in tumor resection surgery, it displays the microvascular network at the tumor margin to guide surgeons in determining the resection margin; in minimally invasive cosmetic surgery, it displays the three-dimensional vascular distribution of the target area to guide the needle to avoid facial arteries.
[0003] Temporal and spatial resolution are often mutually exclusive in photoacoustic microscopy. Mechanically scanned photoacoustic microscopy, using a two-degree-of-freedom motor to scan along a fixed trajectory, offers excellent resolution; however, its slow imaging speed and low-degree-of-freedom scanning method make it difficult to apply in clinical surgical scenarios. New high-speed photoacoustic microscopy methods based on MEMS, galvanometers, or polygon scanners offer near real-time imaging speeds, and these systems are easier to use for handheld scanning. However, they mostly have a small field of view, making it difficult to achieve global observation of lesions. Clinical surgical navigation scenarios often require medical images with a sufficient field of view to scan the entire area to identify and distinguish different parts of the tissue for overall structural analysis. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a photoacoustic microsurgical navigation method and system based on simultaneous positioning and mapping. By constructing a complete vascular network of the surgical site through handheld free scanning, the photoacoustic microsurgical navigation can achieve real-time imaging expansion and updates during the operation, which can solve the problems of small imaging field of view, low scanning freedom, and difficulty in updating imaging during the operation in photoacoustic microscopy.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] In a first aspect, the present invention provides a photoacoustic microsurgical navigation method based on simultaneous localization and mapping, comprising the following steps:
[0007] A scanner is used to perform photoacoustic 3D imaging of the surgical site. The photoacoustic signals are acquired by a data acquisition card and loaded into the system in real time.
[0008] The photoacoustic signal is preprocessed and then converted into three-dimensional point cloud data, which is then further preprocessed.
[0009] Point cloud matching is performed based on the 3D structural information between the point clouds of the previous frame and the current frame, and the pose of the photoacoustic scanner is located. The point cloud matching includes coarse matching and fine matching. The coarse matching uses the NDT algorithm, and the fine matching uses the Point-to-Plane ICP algorithm. The pose matrix calculated by coarse matching is used as the initial pose for fine matching. Based on this, the precise relative pose between the two point clouds and the scanner pose are obtained through further iteration.
[0010] Motion distortion correction is performed based on the estimated scanner movement speed;
[0011] Incorporating the prior knowledge of vascular morphology in the photoacoustic mode into graph optimization improves the global consistency of vascular network construction.
[0012] Construct or update a large field-of-view three-dimensional photoacoustic vascular network for intraoperative navigation.
[0013] As a preferred technical solution, when using a scanner to perform photoacoustic three-dimensional imaging of the surgical site, the nanosecond pulsed laser reaches the surgical site through the reflection of the two-degree-of-freedom rotating galvanometer group or polygonal mirror group in the scanner, generating a photoacoustic echo signal that is received by the ultrasonic transducer and acquired by the data acquisition card. The photoacoustic signal, timestamp, and scanning angle information are loaded into the system in real time.
[0014] As a preferred technical solution
[0015] The photoacoustic signal preprocessing specifically includes:
[0016] One-dimensional sliding window mean filtering is used to smooth noise interference. The width of the sliding window is defined, the sliding window is placed at the beginning of the signal and the values of all signal sampling points in the window are summed. The result is divided by the window width, and the mean value is used as the new value after filtering. Then the window slides and the filtering calculation is repeated until it slides to the end of the signal.
[0017] The signal envelope is extracted to improve the signal quality and reliability. The Hilbert transform of the original signal is calculated, with the original signal as the real part and the signal after the Hilbert transform as the imaginary part. An analytic signal is constructed, and the magnitude of the analytic signal is calculated, which is the envelope of the original signal.
[0018] The photoacoustic signal is specifically converted into three-dimensional point cloud data as follows:
[0019] The sampling time of each sampling point on the signal is obtained by the sampling period of the data acquisition card and the index of each sampling point on the signal. The depth of the sampling point is obtained by the sampling time of each sampling point and the propagation speed of ultrasound in the tissue. The depth of each sampling point and the scanning rotation angle can be used to calculate three-dimensional point cloud data, where each point contains three-dimensional coordinate information and intensity information. The three-dimensional point set converted from all photoacoustic signals acquired in a complete scan is a frame of point cloud.
[0020] As a preferred technical solution, the preprocessing of the three-dimensional point cloud data includes removing outliers and downsampling;
[0021] When removing outliers, the number of other points within a 50μm radius around each point in the point cloud is counted. If the number is less than the threshold of 10, the point is determined to be an outlier that cannot provide effective vascular structure information and is removed. The radius and threshold parameters can be adjusted according to the vascular morphology of the surgical site.
[0022] The downsampling process projects the point cloud onto the maximum value along the scanning axis, retaining the points in the point cloud that best represent the contour structure of the blood vessel surface, thereby improving the speed of subsequent localization and mapping operations while ensuring accuracy. The downsampled point cloud data and the unsampled complete point cloud data are stored in the system simultaneously. The downsampled point cloud data will be used for subsequent inter-frame matching and global optimization, while the unsampled complete point cloud data will be used for the construction of a three-dimensional photoacoustic blood vessel network.
[0023] As a preferred technical solution, the coarse matching specifically includes:
[0024] The point cloud is divided into cubic meshes, and the mean and covariance matrices within the meshes are calculated. The current frame point cloud is transformed into the coordinate system of the previous frame point cloud based on the initial estimate of the relative pose, and the matching degree is calculated through joint probability. The matching probability is optimized, and the optimal relative pose matrix is solved.
[0025] The precise matching specifically refers to:
[0026] The pose matrix obtained from coarse matching is used as the initial transformation matrix for iteration; the nearest point between two frame point clouds is searched to form a match, and its normal vector is calculated; the current frame point cloud is transformed to the coordinate system of the previous frame point cloud through the transformation matrix, and the distance from each point to the local plane of the matching point is calculated as the residual; the optimal relative pose matrix is solved by minimizing the sum of squares of the residuals; the optimization is iterated continuously until the relative pose matrix is less than the threshold m; the optimal pose matrix for inter-frame matching is obtained, and the localization of the photoacoustic scanner is achieved at the same time.
[0027] As a preferred technical solution, in the motion distortion correction, the relative pose matrix between frames obtained by fine matching of point clouds is differentiated with respect to time to obtain the average moving speed Jocobian matrix of the scanner, and then the motion distance of each point in the frame is calculated and remapped to the corresponding position. After obtaining the point cloud frame after correcting motion distortion, inter-frame matching is performed again to correct the matching result.
[0028] As a preferred technical solution, the prior vascular morphology of the photoacoustic mode is incorporated into graph optimization. All scanner poses and vascular network information are represented as a graph structure, where nodes represent scanner pose estimates or point cloud positions, and edges represent the relationships and constraints between them. Prior constraints on vascular morphology of photoacoustic imaging are added, a least squares problem is constructed, and the Gauss-Newton algorithm is used to solve the optimization problem.
[0029] As a preferred technical solution, the prior constraints of the vascular morphology in the photoacoustic imaging are vascular morphology filtering based on the photoacoustic mode of the Hessian matrix. The Hessian matrix calculates the curvature, orientation and other features of the vascular structure, which can distinguish linear vascular structures from other non-linear structures. The algorithm will use these constraints to optimize the estimation, thereby improving the accuracy of vascular network construction.
[0030] As a preferred technical solution, the construction or updating of a large field-of-view three-dimensional photoacoustic vascular network for intraoperative navigation is achieved by fusing point cloud frames based on the optimization results.
[0031] Secondly, the present invention provides a photoacoustic microsurgical navigation system based on synchronous positioning and mapping, for implementing the photoacoustic microsurgical navigation method based on synchronous positioning and mapping, including a data acquisition module, a data preprocessing module, a front-end matching module, a motion distortion correction module, a back-end optimization module, and a vascular mapping module;
[0032] The data acquisition module is used to perform photoacoustic three-dimensional imaging of the surgical site using a scanner. The photoacoustic signals are acquired by the data acquisition card and loaded into the system in real time.
[0033] The data preprocessing module is used to preprocess the photoacoustic signal and then convert it into three-dimensional point cloud data, and to preprocess the three-dimensional point cloud data.
[0034] The front-end matching module is used to perform point cloud matching based on the three-dimensional structural information between the point clouds of the previous frame and the current frame, and to locate the pose of the photoacoustic scanner. The point cloud matching includes coarse matching and fine matching. The coarse matching uses the NDT algorithm, and the fine matching uses the Point-to-Plane ICP algorithm. The pose matrix calculated by the coarse matching is used as the initial pose for the fine matching. Based on this, the precise relative pose between the two point clouds and the scanner pose are obtained through further iteration.
[0035] The motion distortion correction module is used to correct motion distortion based on the estimated scanner movement speed.
[0036] The backend optimization module is used to incorporate the prior knowledge of vascular morphology in the photoacoustic mode into the graph optimization, thereby improving the global consistency of vascular network construction.
[0037] The vascular mapping module is used to construct or update a large field-of-view three-dimensional photoacoustic vascular network that can be used for intraoperative navigation.
[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0039] 1. The method and system of this invention are mainly applied to surgical navigation in photoacoustic microscopy. Addressing the problems of low scanning freedom and small imaging field of view in existing photoacoustic microscopy, which makes it difficult to achieve global observation of lesions, this invention employs a method based on synchronous positioning and mapping to locate the scanner pose and construct a large-field-of-view three-dimensional photoacoustic vascular network. This enables photoacoustic microscopy imaging that can be freely scanned with a handheld scanner and whose field of view can be updated and expanded in real time during surgery, improving the clinical application value of photoacoustic microscopy and providing a new solution for surgical navigation.
[0040] 2. This invention utilizes the maximum projection of photoacoustic point clouds for inter-frame matching, effectively reducing the massive point cloud data, retaining key information about vascular structures in the point cloud while filtering out invalid information, greatly reducing the computational load and storage space requirements, making subsequent matching and optimization processing more efficient and accurate, and making the system more lightweight and easy to use during surgery.
[0041] 3. This invention utilizes graph optimization with added vascular morphology priors, and uses the Hessian matrix to calculate the curvature, orientation, and other features of vascular structures in photoacoustic modes. It distinguishes linear vascular structures from other non-linear structures in the point cloud, and uses this as a vascular morphology prior constraint for photoacoustic imaging, resulting in an accurate and globally consistent photoacoustic vascular network, which helps improve the accuracy and safety of surgery. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a flowchart of the photoacoustic microsurgical navigation method based on synchronous positioning and mapping according to the present invention.
[0044] Figure 2 This is a flowchart of the data preprocessing module of the present invention.
[0045] Figure 3(a) is a sparse three-dimensional photoacoustic vascular network constructed by the downsampling point cloud of the present invention.
[0046] Figure 3(b) shows a dense three-dimensional photoacoustic vascular network constructed from the complete point cloud. Detailed Implementation
[0047] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.
[0048] Example
[0049] The method and system proposed in this invention enable real-time imaging updates and expansion during photoacoustic microsurgery navigation, constructing a complete vascular network of the surgical area. This solves the problems of limited field of view, limited scanning freedom, and difficulty in updating images during surgery inherent in photoacoustic microsurgery. (Refer to...) Figure 1 The present invention includes the following steps:
[0050] S1, photoacoustic microscopy imaging and signal loading, specifically:
[0051] A handheld high-speed galvanometer scanner or a polygon scanner is used to perform photoacoustic 3D imaging of the surgical site. In this embodiment, a collimated nanosecond pulsed laser is reflected by a two-degree-of-freedom rotating galvanometer group or polygon mirror group in the scanner, and then focused by a lens to reach the surgical site. The scanning range is approximately 4*4mm, and the time required for a complete scan is approximately 0.2s. The photoacoustic echo signal is received by a flat-field ultrasonic transducer and acquired by a data acquisition card. The photoacoustic signal, timestamp, and scanning angle information are loaded into the system in real time.
[0052] S2, Data preprocessing, such as Figure 2As shown, specifically:
[0053] S21. Preprocess the signal.
[0054] One-dimensional sliding window mean filtering is used to smooth noise interference. The sliding window width is defined as 5. The sliding window is placed at the beginning of the signal and the values of all signal sampling points within the window are summed. The result is divided by the window size of 5, and the mean value is used as the new value after filtering. Then the window slides and the filtering calculation is repeated until it slides to the end of the signal.
[0055] Enveloping the signal improves its quality and reliability. The Hilbert transform of the original signal is calculated, with the original signal as the real part and the transformed signal as the imaginary part. An analytic signal is constructed, and its magnitude is calculated; this magnitude is the envelope of the original signal.
[0056] S22, Signal decomposition into 3D point cloud
[0057] The sampling time of each sampling point on the signal is obtained by using the sampling period of the data acquisition card and the index of each sampling point on the signal.
[0058] Furthermore, the depth of the sampling point can be determined by the sampling time of each sampling point and the propagation speed of ultrasound in the tissue. The propagation speed can be set to approximately 1540 m / s, which is the typical speed of sound in tissue.
[0059] Furthermore, the depth and scanning rotation angle of each sampling point can be used to calculate three-dimensional point cloud data, where each point contains three-dimensional coordinate information and intensity information;
[0060] Furthermore, the three-dimensional point set converted from all photoacoustic signals acquired in a complete scan is a single frame of point cloud.
[0061] S23. Preprocess the point cloud data.
[0062] Outlier removal in point clouds. Outliers in the point cloud are removed to eliminate the impact of noise on mapping.
[0063] For each point in the point cloud, find its nearest neighbor within a radius of 50 μm;
[0064] Furthermore, calculate the number of neighboring points of each point within a given radius;
[0065] Furthermore, the number of neighboring points is compared with a pre-set threshold of 10. If the number of neighboring points of a point is less than this threshold, then the point is considered an outlier and needs to be removed from the point cloud data.
[0066] In this embodiment, the radius and threshold parameters can be adjusted according to the vascular morphology of the surgical site.
[0067] Furthermore, point cloud downsampling is performed using maximum value projection to improve the speed of subsequent localization and mapping calculations while maintaining accuracy. The maximum intensity value of each point in the point cloud is calculated along the scanning axis, retaining only the points with the highest intensity value. Compared to typical point cloud processing methods such as uniform downsampling, voxel downsampling, and random downsampling, maximum value projection is more suitable for data reduction and information preservation in photoacoustic images. It can remove invalid and inefficient information from the point cloud to the greatest extent possible, and the retained points are those that best represent the contour structure of the blood vessel surface.
[0068] The downsampled point cloud data and the unsampling complete point cloud data are stored in the system at the same time. The downsampled point cloud data will be used for subsequent inter-frame matching and global optimization, while the unsampling complete point cloud data will be used for the construction of a three-dimensional photoacoustic vascular network.
[0069] S3, front-end matching, specifically:
[0070] S31. Use the NDT (Normal Distributions Transform) algorithm for coarse inter-frame matching. Divide the point cloud into a cubic mesh and calculate the mean and covariance matrix within the mesh. Transform the current frame's point cloud into the coordinate system of the previous frame's point cloud based on the initial estimate of the relative pose, and calculate the matching degree through joint probability. Optimize the matching probability and solve for the optimal relative pose matrix.
[0071] S32. The Point-to-Plane ICP algorithm is used for fine inter-frame matching. The pose matrix obtained from coarse matching is used as the initial transformation matrix for iteration; the nearest point between two frame point clouds is searched to form a match, and its normal vector is calculated; the current frame point cloud is transformed to the coordinate system of the previous frame point cloud through the transformation matrix, and the distance from each point to the local plane of the matching point is calculated as the residual; the optimal relative pose matrix is solved by minimizing the sum of squares of the residuals; the optimization is iterated until the relative pose matrix is less than the threshold m. The optimal pose matrix for inter-frame matching is obtained, and the localization of the photoacoustic scanner is achieved simultaneously.
[0072] S4. Correction of motor distortion, specifically:
[0073] In surgical navigation, the surgeon's hand is usually in motion when using a photoacoustic scanner to scan the surgical site, and the focus of the laser scan is also moving, resulting in motion distortion that needs to be corrected. The Jacobian matrix of the scanner's average movement speed is obtained by taking the time derivative of the relative pose matrix between point cloud frames; this can be efficiently calculated using an automatic differentiation algorithm in C++. The Jacobian matrix is then multiplied with the point cloud coordinates to calculate the movement distance of each point within the frame. Based on the calculated movement distance, each point in the original point cloud is remapped to correct the motion distortion. After completing the motion distortion correction, the point cloud frames are rematched to refine the matching results, as described in S3.
[0074] S5. Backend graph optimization with added blood vessel morphology constraints, specifically:
[0075] S51. Construct a pose graph. Represent the scanner pose and vascular network information as a graph structure, where nodes represent the pose estimate or point cloud position of each scanner, and edges represent the relationships and constraints between them.
[0076] S52. Extracting Vascular Morphological Features. By analyzing the point cloud data, the morphological features of blood vessels are extracted using the Hessian matrix. The Hessian matrix can capture the second derivative information of the three-dimensional structure of blood vessels; its eigenvalues and eigenvector directions effectively describe the morphological features of blood vessels, distinguishing vascular structures from non-vascular structures. These features can be used as constraints in the optimization process.
[0077] S53. Construct a graph optimization problem with vascular morphology constraints. Within the graph optimization framework, add vascular morphology constraint edges connecting two pose nodes and including the corresponding vascular morphology features. During optimization, the system needs to consider these constraint edges to ensure global consistency of the optimized vascular morphology features.
[0078] S54. Construct a least squares problem and solve the optimization problem using the Gauss-Newton algorithm.
[0079] S6. Based on the optimization results, all point cloud frames are fused to construct or update a large field-of-view three-dimensional photoacoustic vascular network. Figures 3(a) and 3(b) show a set of test results for this embodiment. Figure 3(a) shows a sparse three-dimensional photoacoustic vascular network constructed from downsampled point clouds, and Figure 3(b) shows a dense three-dimensional photoacoustic vascular network constructed from complete point clouds.
[0080] Another embodiment of the present invention provides a photoacoustic microsurgical navigation system based on synchronous positioning and mapping, including: a data acquisition module, a data preprocessing module, a front-end matching module, a motion distortion correction module, a back-end optimization module, and a vascular mapping module;
[0081] The data acquisition module is used to perform photoacoustic three-dimensional imaging of the surgical site using a scanner. The photoacoustic signals are acquired by the data acquisition card and loaded into the system in real time to provide raw data for subsequent processing.
[0082] The data preprocessing module is used to preprocess the photoacoustic signal and convert it into three-dimensional point cloud data, and then further preprocess the three-dimensional point cloud data. This module is responsible for preprocessing the photoacoustic signal to improve signal quality. The preprocessed signal is converted into three-dimensional point cloud data and further preprocessed, such as removing outliers and downsampling, for use in subsequent modules.
[0083] The front-end matching module is used to perform point cloud matching based on the three-dimensional structural information between the point clouds of the previous frame and the current frame, and to locate the pose of the photoacoustic scanner. The point cloud matching includes coarse matching and fine matching. The coarse matching uses the NDT algorithm, and the fine matching uses the Point-to-Plane ICP algorithm. The pose matrix calculated by the coarse matching is used as the initial pose for the fine matching. Based on this, the precise relative pose between the two point clouds and the scanner pose are obtained through further iteration.
[0084] The motion distortion correction module is used to correct motion distortion based on the estimated scanner movement speed. This process involves calculating the time derivative of the inter-frame relative pose matrix obtained from fine-matching of the point cloud to obtain the scanner's average movement speed Jacobian matrix. Then, the motion distance of each point within the frame is calculated and remapped to its corresponding position to obtain the point cloud frame after motion distortion correction. Finally, inter-frame matching is performed again to correct the matching results.
[0085] The backend optimization module is used to incorporate the prior knowledge of blood vessel morphology in the photoacoustic mode into the graph optimization to improve the global consistency of the blood vessel network construction; furthermore, a blood vessel morphology constraint graph optimization method based on the Hessian matrix is adopted to achieve optimization.
[0086] The vascular mapping module is used to construct or update a large field-of-view three-dimensional photoacoustic vascular network that can be used for intraoperative navigation.
[0087] This embodiment integrates the processed and corrected point cloud data obtained from the aforementioned modules, enabling real-time construction or updating of vascular networks. The constructed large-field-of-view three-dimensional photoacoustic vascular network can provide detailed and accurate vascular information, thereby providing precise navigation for surgeons during operations.
[0088] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include those elements, but also includes other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0089] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0090] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A photoacoustic microsurgical navigation method based on simultaneous localization and mapping, characterized in that, Includes the following steps: A scanner is used to perform photoacoustic 3D imaging of the surgical site. The photoacoustic signals are acquired by a data acquisition card and loaded into the system in real time. The photoacoustic signal is preprocessed and then converted into three-dimensional point cloud data, which is then further preprocessed. Specifically, the photoacoustic signal preprocessing involves: One-dimensional sliding window mean filtering is used to smooth noise interference. The width of the sliding window is defined, the sliding window is placed at the beginning of the signal and the values of all signal sampling points in the window are summed. The result is divided by the window width, and the mean value is used as the new value after filtering. Then the window slides and the filtering calculation is repeated until it slides to the end of the signal. The signal envelope is extracted to improve the signal quality and reliability. The Hilbert transform of the original signal is calculated, with the original signal as the real part and the signal after the Hilbert transform as the imaginary part. An analytic signal is constructed, and the magnitude of the analytic signal is calculated, which is the envelope of the original signal. The photoacoustic signal is specifically converted into three-dimensional point cloud data as follows: The sampling time of each sampling point on the signal is obtained by the sampling period of the data acquisition card and the index of each sampling point on the signal. The depth of the sampling point is obtained by the sampling time of each sampling point and the propagation speed of ultrasound in the tissue. The depth of each sampling point and the scanning rotation angle can be used to calculate three-dimensional point cloud data. Each point contains three-dimensional coordinate information and intensity information. The three-dimensional point set converted from all photoacoustic signals acquired in a complete scan is a frame of point cloud. Point cloud matching is performed based on the 3D structural information between the point clouds of the previous frame and the current frame, and the pose of the photoacoustic scanner is located. The point cloud matching includes coarse matching and fine matching. The coarse matching uses the NDT algorithm, and the fine matching uses the Point-to-Plane ICP algorithm. The pose matrix calculated by coarse matching is used as the initial pose for fine matching. Based on this, the precise relative pose between the two point clouds and the scanner pose are obtained through further iteration. Motion distortion correction is performed based on the estimated scanner movement speed; Incorporating the prior knowledge of vascular morphology in the photoacoustic mode into graph optimization improves the global consistency of vascular network construction. Construct or update a large field-of-view three-dimensional photoacoustic vascular network for intraoperative navigation; The coarse matching specifically refers to: The point cloud is divided into cubic meshes, and the mean and covariance matrices within the meshes are calculated. The current frame point cloud is transformed into the coordinate system of the previous frame point cloud based on the initial estimate of the relative pose, and the matching degree is calculated through joint probability. The matching probability is optimized, and the optimal relative pose matrix is solved. The precise matching specifically refers to: The pose matrix obtained from coarse matching is used as the initial transformation matrix for iteration; the nearest point between two frames of point clouds is searched to form a match, and its normal vector is calculated; the current frame point cloud is transformed to the coordinate system of the previous frame point cloud through the transformation matrix, and the distance from each point to the local plane of the matching point is calculated as the residual; the optimal relative pose matrix is solved by minimizing the sum of squares of the residuals; the optimization is iterated continuously until the relative pose matrix is less than the threshold m; The optimal pose matrix for inter-frame matching is obtained, and the photoacoustic scanner is located simultaneously. In the motion distortion correction process, the relative pose matrix between frames obtained by fine matching of point clouds is differentiated with respect to time to obtain the average moving speed Jocobian matrix of the scanner. Then, the motion distance of each point in the frame is calculated and remapped to the corresponding position to obtain the point cloud frame after motion distortion correction. Then, the inter-frame matching is performed again to correct the matching result. The prior vascular morphology of the photoacoustic mode is incorporated into graph optimization. All scanner poses and vascular network information are represented as a graph structure, where nodes represent scanner pose estimates or point cloud positions, and edges represent the relationships and constraints between them. Prior constraints on vascular morphology of photoacoustic imaging are added, a least squares problem is constructed, and the optimization problem is solved using the Gauss-Newton algorithm. The prior constraints for vascular morphology in photoacoustic imaging are vascular morphology filtering based on the photoacoustic mode of the Hessian matrix. The Hessian matrix calculates the curvature and orientation features of the vascular structure, which can distinguish linear vascular structures from other non-linear structures. The algorithm uses these constraints to optimize the estimation, thereby improving the accuracy of vascular network construction.
2. The photoacoustic microsurgical navigation method based on simultaneous localization and mapping according to claim 1, characterized in that, When using a scanner to perform photoacoustic 3D imaging of the surgical site, nanosecond pulsed laser light is reflected by a two-degree-of-freedom rotating galvanometer group or polygonal mirror group in the scanner to reach the surgical site, generating photoacoustic echo signals that are received by the ultrasonic transducer and acquired by the data acquisition card. The photoacoustic signals, timestamps, and scanning angle information are loaded into the system in real time.
3. The photoacoustic microsurgical navigation method based on simultaneous localization and mapping according to claim 1, characterized in that, Preprocessing of the 3D point cloud data includes outlier removal and downsampling; When removing outliers, the radius around each point in the statistical point cloud is 50. μm If the number of other points within the range is less than the threshold of 10, the point is determined to be an outlier that cannot provide effective vascular structure information and is removed; the radius and threshold parameters can be adjusted according to the vascular morphology of the surgical site. The downsampling process projects the point cloud onto the maximum value along the scanning axis, retaining the points in the point cloud that best represent the contour structure of the blood vessel surface, thereby improving the speed of subsequent localization and mapping operations while ensuring accuracy. The downsampled point cloud data and the unsampled complete point cloud data are stored in the system simultaneously. The downsampled point cloud data will be used for subsequent inter-frame matching and global optimization, while the unsampled complete point cloud data will be used for the construction of a three-dimensional photoacoustic blood vessel network.
4. The photoacoustic microsurgical navigation method based on simultaneous localization and mapping according to claim 1, characterized in that, The construction or updating of the large field-of-view three-dimensional photoacoustic vascular network that can be used for intraoperative navigation is based on the optimization results, which fuse point cloud frames to construct or update the large field-of-view three-dimensional photoacoustic vascular network.
5. A photoacoustic microsurgical navigation system based on simultaneous localization and mapping, characterized in that, The photoacoustic microsurgical navigation method based on synchronous positioning and mapping as described in any one of claims 1-4 includes a data acquisition module, a data preprocessing module, a front-end matching module, a motion distortion correction module, a back-end optimization module, and a vascular mapping module. The data acquisition module is used to perform photoacoustic three-dimensional imaging of the surgical site using a scanner. The photoacoustic signals are acquired by the data acquisition card and loaded into the system in real time. The data preprocessing module is used to preprocess the photoacoustic signal and then convert it into three-dimensional point cloud data, and to preprocess the three-dimensional point cloud data. The front-end matching module is used to perform point cloud matching based on the three-dimensional structural information between the point clouds of the previous frame and the current frame, and to locate the pose of the photoacoustic scanner. The point cloud matching includes coarse matching and fine matching. The coarse matching uses the NDT algorithm, and the fine matching uses the Point-to-Plane ICP algorithm. The pose matrix calculated by the coarse matching is used as the initial pose for the fine matching. Based on this, the precise relative pose between the two point clouds and the scanner pose are obtained through further iteration. The motion distortion correction module is used to correct motion distortion based on the estimated scanner movement speed. The backend optimization module is used to incorporate the prior knowledge of vascular morphology in the photoacoustic mode into the graph optimization, thereby improving the global consistency of vascular network construction. The vascular mapping module is used to construct or update a large field-of-view three-dimensional photoacoustic vascular network that can be used for intraoperative navigation.