Puncture needle track prediction method, system and device and medium

Abstract

The invention discloses a puncture needle track prediction method, system and device and a medium. The method comprises the following steps: acquiring continuous photoacoustic imaging image data when a puncture needle punctures in tissue; processing the photoacoustic imaging image data, extracting a needle point position, and converting each frame of image data into a feature vector about the needle point position; a Transformer model is established, the input quantity of the model is the feature vector of past image data about the position of the needle tip and priori knowledge, and the output quantity of the model is the center coordinate of the needle tip of future image data and the insertion angle of the needle tip in the surrounding tissue. Through data enhancement and constraint injection, a pure data driven model can fuse physical laws and clinical experience, finally real-time and accurate prediction of the motion trail of the puncture needle is achieved, medical employees can be helped to more accurately control the path of the puncture needle in a complex operation environment, the misoperation risk is reduced, and the operation efficiency is improved. And the safety and the accuracy of medical puncture operation are obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intersection of medical imaging and machine learning, and in particular to a method for predicting the dynamic trajectory of a puncture needle by combining photoacoustic imaging and a Transformer model. Background Art

[0002] Puncture technology is a widely used fundamental procedure in modern medical diagnosis and treatment. Puncture needles are used in a variety of medical procedures, such as tissue sampling, drug injection, drainage, and local anesthesia. Precise puncture techniques are crucial for diagnostic and treatment effectiveness and patient safety. However, due to the complexity and variability of human tissue, accurate positioning and trajectory control of puncture needles present numerous challenges.

[0003] Traditional ultrasound imaging technology is widely used in medical punctures. It provides real-time imaging guidance, enabling medical personnel to perform procedures with visual assistance. However, ultrasound imaging has certain limitations in terms of noise interference, resolution, and precise positioning of the puncture needle tip. Artifacts and noise that may appear in ultrasound images can degrade image quality and affect visualization of the puncture needle. Furthermore, the physical properties of ultrasound limit its penetration and resolution in soft tissues, particularly for visualization of deep tissues.

[0004] Photoacoustic imaging (PAI) is an emerging hybrid imaging technology that combines the high contrast of optical imaging with the high spatial resolution of ultrasound imaging. PAI uses laser excitation to generate a photoacoustic effect in tissue, capturing the tissue's photoacoustic signals to generate high-contrast images. Because it can provide clearer visualization of tissue structure and puncture needles, PAI has shown significant advantages in the field of medical imaging. However, during dynamic puncture procedures, real-time prediction and control of the puncture needle's trajectory is difficult to achieve using only static images from PAI.

[0005] Therefore, there is an urgent need for an efficient and dynamic puncture needle trajectory prediction solution to improve the accuracy and safety of medical puncture operations. Summary of the Invention

[0006] The embodiments of the present application solve the problems of inaccurate and insufficient real-time performance of puncture needle trajectory prediction in the prior art by providing a puncture needle trajectory prediction method, system, device, and computer-readable storage medium.

[0007] In order to solve the above problems, in a first aspect, an embodiment of the present application provides a puncture needle trajectory prediction method, comprising the following steps:

[0008] Acquiring continuous photoacoustic imaging image data when the puncture needle punctures the tissue;

[0009] Processing the photoacoustic imaging image data, extracting the needle tip position, and converting each frame of image data into a feature vector related to the needle tip position;

[0010] A Transformer model was established. The input of the model was the feature vector of the needle tip position in the past T1 frame image data and prior knowledge. The output of the model was the needle tip center coordinates (x, y, z) in the future T2 frame image data and the insertion angle θ of the needle tip in the surrounding tissue. T1 and T2 were both positive integers.

[0011] Preferably, the photoacoustic imaging image data is collected in real time by a photoacoustic imaging device;

[0012] Preferably, the photoacoustic imaging image data shows the needle tip and surrounding tissue structures.

[0013] Preferably, the method for processing photoacoustic imaging image data includes:

[0014] Adaptive threshold segmentation algorithm is used to separate the needle tip from the surrounding tissue and extract the needle tip contour. Based on the physical characteristics of photoacoustic signal attenuation with depth, the needle tip signal intensity is depth-corrected.

[0015] Use an edge detection algorithm to determine the coordinates (x, y, z) of the needle tip center and the insertion angle θ of the needle tip in the surrounding tissue; where x, y, and z are the coordinate values of the length, width, and depth axes, respectively;

[0016] The needle tip center coordinates (x, y, z), the needle tip insertion angle θ and the corrected needle tip signal intensity I z Normalize to the interval [0,1] to eliminate dimension differences; normalize the timestamp t to relative time; obtain the feature vector [x, y, z, θ, I z ,t].

[0017] Furthermore, when extracting the needle tip contour, the needle tip features are highlighted through weighted enhancement based on the fact that the needle tip signal intensity is higher than that of the surrounding tissue.

[0018] Preferably, the prior knowledge includes:

[0019] Photoacoustic signal attenuation: Encoded depth correction vector Where μ is the attenuation coefficient, Z1,…,Z T1 Respectively represent the depth when the input sequence is 1,…,T1;

[0020] Tip contrast: Encoded intensity weight vector v I =[I Z1 / I avg ,...,I ZT1 / I avg]; among them, I Z1 ,…,I ZT1 , respectively represent the tip signal strength when the input sequence is 1,…,T1, I avg is the average needle tip signal intensity of the T1 input sequences;

[0021] Speed constraint: Encoding speed upper limit vector v v =[min(v t ,10) / 10,...,min(v T ,10) / 10]; where v t is the current instantaneous frame speed, v T is the maximum inter-frame velocity in each time segment;

[0022] Angle constraint: Encodes the upper limit vector v of the angle change θ =[min(|θ t -θ t-1 |,30) / 30,...,min(|θ T -θ T-1 |,30) / 30]; where θ t -θ t-1 Represents the angle change between the current frame and the previous frame, θ T -θ T-1 It is the maximum angle change between two adjacent frames in each time segment.

[0023] Preferably, the training loss function of the Transformer model includes:

[0024] Position prediction loss: Among them, [x t ,y t ,z t ] is the real location, is the predicted position, N = 10;

[0025] Angle prediction loss: Among them, θ t is the true angle, is the prediction angle;

[0026] Signal attenuation constraint, used to penalize the signal strength at the predicted location that violates the attenuation model: Among them, I t is the signal strength at the predicted location, Zt is the depth of the predicted location, ε I is the tolerance threshold;

[0027] Speed constraint, used to penalize predicted speed exceeding the limit:

[0028] Angle constraint, used to penalize excessive angle changes:

[0029] Total loss: L = L MSE +0.5·L angle +0.2·(L decay +L speed +L angle ).

[0030] In a second aspect, an embodiment of the present application further provides a puncture needle trajectory prediction system, the system comprising:

[0031] A data acquisition module is used to acquire continuous photoacoustic imaging image data when the puncture needle punctures the tissue;

[0032] a data processing module, configured to process the photoacoustic imaging image data, extract the needle tip position, and convert each frame of image data into a feature vector relating to the needle tip position;

[0033] The model building module is used to build the Transformer model. The input of the model is the feature vector of the needle tip position and prior knowledge of the past T1 frame image data. The output of the model is the needle tip center coordinates (x, y, z) of the future T2 frame image data and the insertion angle θ of the needle tip in the surrounding tissue. T1 and T2 are both positive integers.

[0034] Preferably, the system further includes one or more of the following modules:

[0035] A visual interface that renders the puncture needle trajectory in real time, displaying the needle tip position, predicted confidence interval, and signal strength;

[0036] The trajectory deviation warning module is used to trigger a highlight prompt when the deviation between the predicted trajectory and the target path exceeds a set threshold;

[0037] Speed / angle abnormality warning module, used to issue an audible alarm when the predicted speed or angle change exceeds the set threshold;

[0038] The operation suggestion module is used to prompt adjustment of the insertion angle or speed based on the predicted trajectory.

[0039] In a third aspect, an embodiment of the present application further provides a puncture needle trajectory prediction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the puncture needle trajectory prediction method described above are implemented.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned puncture needle trajectory prediction method when executed by a processor.

[0041] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0042] This application provides a method for predicting puncture needle trajectories that combines photoacoustic imaging and a Transformer model. This method uses photoacoustic imaging to acquire real-time position data of the puncture needle and processes this data using a Transformer model to accurately predict the needle's trajectory in real time. This invention will help medical practitioners more accurately control the puncture needle's path in complex operating environments, reduce the risk of misoperation, and significantly improve the safety and accuracy of medical puncture procedures.

[0043] 2. During data processing, based on the physical property of exponential decay of photoacoustic signals with tissue depth, the needle tip signal is depth-corrected to enhance feature extraction in deep tissue and improve the model's adaptability to complex anatomical structures. A weighting factor is also used to emphasize the needle tip signal, addressing the lack of contrast between the needle tip and soft tissue in traditional photoacoustic imaging.

[0044] 3. Through input augmentation, the Transformer model simultaneously receives raw data and prior knowledge. Prior knowledge serves as a bridge between the Transformer model and domain knowledge. Through data augmentation and constraint injection, the purely data-driven Transformer model is able to integrate physical laws with clinical experience, ultimately achieving high-precision and high-reliability prediction of puncture needle trajectories. By incorporating prior knowledge, the model can still generate reasonable predictions even when data is insufficient or noisy, improving model generalization. Prior knowledge explicitly introduces physical rules into the model, making the prediction results more consistent with medical logic. This design significantly improves the practicality and safety of the model in complex medical scenarios.

[0045] 4. The training loss function of the Transformer model adopts position prediction loss and angle prediction loss, and integrates weighted losses of signal attenuation constraint, speed constraint and angle constraint. It can reduce the risk of overfitting, limit the model parameter space by the loss function, avoid excessive dependence on training data, and ensure that the model prediction results are consistent with the data rules and meet the actual limitations of clinical operations.

[0046] 5. The system provided in this application reduces the operator's cognitive load by integrating photoacoustic imaging, trajectory prediction, physical parameters and tissue characteristics into a unified interface. Through a closed-loop process of real-time prediction → visualization → alarm → recommendation, a dynamic adjustment mechanism during surgery is formed, which significantly reduces the risk of misoperation. The interface supports personalized configuration to adapt to the operating habits of different doctors. The explicit display of confidence intervals and physical constraints makes the prediction results consistent with medical logic and enhances doctors' trust in AI models. Through real-time data rendering, intelligent risk warnings and dynamic decision support, complex AI prediction results are converted into intuitive and actionable clinical information. The synergistic effect of the two not only improves the safety and efficiency of puncture operations, but also promotes the practical application of AI technology in clinical scenarios, providing key technical support for precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] Figure 1 This is a flowchart of the puncture needle trajectory prediction method provided in Example 1 of the present application;

[0049] Figure 2 Shown is a schematic diagram of a photoacoustic imaging image of a puncture surgery process acquired in real time by a photoacoustic imaging device;

[0050] Figure 3 The figure shows the comparison between the prediction result of the puncture needle trajectory prediction method provided in this embodiment and the actual trajectory;

[0051] Figure 4 This is a structural block diagram of the puncture needle trajectory prediction system provided in Example 2 of the present application;

[0052] Figure 5 This is a structural block diagram of the puncture needle trajectory prediction device provided in Example 3 of the present application.

[0053] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0054] The embodiments of the present application provide a puncture needle trajectory prediction method, system, device and computer-readable storage medium to solve the problems of inaccurate and insufficient real-time performance of puncture needle trajectory prediction in the prior art.

[0055] To better understand the above technical solutions, exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numbers in different drawings represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of devices consistent with certain aspects of the present application, as detailed in the appended claims.

[0056] Example 1

[0057] Advances in machine learning have brought new opportunities to medical imaging and biopsy. In particular, the Transformer model, a deep learning architecture, has achieved remarkable results in natural language processing and computer vision in recent years. With its powerful sequence modeling capabilities and self-attention mechanism, the Transformer model effectively captures complex patterns and long-range dependencies in sequential data. These properties make the Transformer excellent for processing time series and forecasting tasks.

[0058] However, existing technologies have not yet effectively combined photoacoustic imaging and the Transformer model to address the challenge of dynamic needle trajectory prediction. This paper provides a high-precision, low-latency method for predicting the dynamic trajectory of puncture needles by fusing the real-time data of photoacoustic imaging with the sequential prediction capabilities of the Transformer.

[0059] This method uses modeling to predict the dynamic motion trajectory of the puncture needle. The Transformer model can provide real-time trajectory prediction and decision support, improving the safety and accuracy of puncture operations. This puncture needle trajectory prediction method based on photoacoustic imaging and the Transformer model will provide more reliable technical support for clinical surgery and promote the development of precision medicine.

[0060] Figure 1 This is a flow chart of the puncture needle trajectory prediction method provided in this embodiment. The puncture needle trajectory prediction method includes the following steps:

[0061] Step S1: Acquire continuous photoacoustic imaging image data when the puncture needle punctures the tissue.

[0062] Specifically, the photoacoustic imaging image data can be collected in real time by a photoacoustic imaging device. The photoacoustic imaging image data needs to clearly show the needle tip and surrounding tissue structures.

[0063] In an optional embodiment, the photoacoustic imaging device is configured as follows: laser wavelength range 600-900 nm (preferably 780 nm, suitable for deep tissue), pulse frequency 10-50 Hz, energy density 10-20 mJ / cm 2 , ultrasonic sensor frequency 5-10MHz, frame rate 30fps, resolution 0.1mm.

[0064] Step S2: Processing the photoacoustic imaging image data, extracting the needle tip position, and converting each frame of image data into a feature vector regarding the needle tip position.

[0065] Specifically, the processing method includes:

[0066] S21: Adaptive threshold segmentation algorithm (e.g., Otsu method) is used to separate the needle tip from surrounding tissue and extract the needle tip contour. Based on the physical property of photoacoustic signal attenuation with depth, the needle tip signal intensity is depth-corrected to enhance deep image features.

[0067] The calibration model used is:

[0068] I z =I0e (-μz)

[0069] Where I0 is the initial tip signal strength at depth 0, I z is the corrected signal intensity of the needle tip at depth z, and μ is the attenuation coefficient.

[0070] μ reflects the rate at which the photoacoustic signal decays with increasing depth in the medium. This attenuation is primarily due to factors such as absorption and scattering of the medium during the propagation of the photoacoustic signal. In one optional embodiment, μ = 0.1-0.5 cm⁻¹, preferably 0.3 cm⁻¹.

[0071] In this embodiment, to enhance deep image features, depth correction is performed on the needle tip signal intensity. This correction compensates for the weakening of deep signal intensity due to attenuation, allowing the image features of deep tissue to more accurately reflect their actual state. This correction restores deep signal intensity to its pre-attenuation level, thereby improving the contrast and clarity of deep images and enabling a clearer display of the structure and characteristics of deep tissue.

[0072] Furthermore, the needle tip signal intensity is typically 2-5 times higher than that of the surrounding tissue, and the needle tip features are highlighted by weighted enhancement. In an optional embodiment, the weighting factor is 2.0.

[0073] Furthermore, Gaussian filtering is applied to remove high-frequency noise in the photoacoustic image and retain the needle tip signal, making the needle tip signal easier to identify.

[0074] S22: Using the Canny edge detection algorithm, determine the coordinates (x, y, z) of the needle tip center and the insertion angle θ of the needle tip in the surrounding tissue; where x, y, and z are the coordinate values of the length, width, and depth axes, respectively.

[0075] S23: The needle tip center coordinates (x, y, z), the needle tip insertion angle θ and the corrected needle tip signal intensity I z Normalize to the interval [0,1] to eliminate dimension differences; normalize the timestamp t to relative time (with the start frame of the sequence as 0); obtain the feature vector [x, y, z, θ, I z ,t].

[0076] Step S3: Build a Transformer model. The model inputs are the feature vector of the needle tip position from the past T1 frame image data and prior knowledge. The model outputs are the needle tip center coordinates (x, y, z) and the insertion angle θ of the needle tip in the surrounding tissue from the future T2 frame image data. T1 and T2 are both positive integers. Typically, T1>T2.

[0077] In a preferred embodiment, the Transformer model includes a 6-layer encoder and a 6-layer decoder, each layer has 8 self-attention heads, a hidden layer dimension of 512, a feedforward network dimension of 2048, and a dropout rate of 0.1. The input sequence length T1 = 20, and the output sequence length T2 = 10.

[0078] Specifically, the model input consists of two parts:

[0079] (1) Feature vectors of the past 20 frames of image data. Each frame of feature vector contains the position of the needle tip (x, y, z), angle θ, signal strength I z and timestamp t, that is, the feature vector is [x,y,z,θ,I z ,t]. After normalization and noise filtering, these data form an input sequence X with a length of 20 t .

[0080] (2) Prior knowledge. This includes:

[0081] Photoacoustic signal attenuation: Encoded depth correction vector Where μ is the attenuation coefficient, where μ takes the average value of 0.3 cm -1 . Z1,…,Z T1 They represent the depth when the input sequence is 1,…,T1 respectively.

[0082] Tip contrast: Encoded intensity weight vector v I =[I Z1 / Iavg ,...,I ZT1 / I avg ]. Among them, I Z1 ,…,I ZT1 , respectively represent the tip signal strength when the input sequence is 1,…,T1, I avg is the average tip signal intensity of the T1 input sequences.

[0083] Speed constraint: Encoding speed upper limit vector v v =[min(v t ,10) / 10,...,min(v T ,10) / 10]. Among them, v t is the current instantaneous inter-frame speed (in mm / s), which represents the inter-frame speed at time point t. min(v t ,10) / 10 is the instantaneous frame speed v t Perform upper limit constraint and take v t The smaller value of 10 is used, and is evenly divided into 10 intervals to ensure that the predicted puncture needle speed does not exceed the safety threshold (10 mm / s), avoiding operational risks caused by excessive speed. T is the maximum inter-frame speed in each time segment. The ellipsis omits the upper limit constraint of the inter-frame speed between the current instantaneous inter-frame speed and the maximum inter-frame speed in the time segment. v Indicates the maximum movement speed in each time segment.

[0084] Angle constraint: Encodes the upper limit vector v of the angle change θ =[min(|θ t -θ t-1 |,30) / 30,...,min(|θ T -θ T-1 |,30) / 30]. Where θ t is the angle of the current frame, θ t-1 is the angle of the previous frame, θ t -θ t-1 Indicates the angle change between the current frame and the previous frame, min(|θ t -θ t-1 |,30) / 30 means that the upper limit of the angle is determined by the smaller value between the change amount and 30°, and is evenly divided into 30 intervals. T -θ T-1 Is the maximum angle change between two adjacent frames in each time segment. The ellipsis omits the upper limit constraint on the angle change from the angle change between the current frame and the previous frame to the maximum angle change between two adjacent frames in the time segment. θ Indicates the maximum allowed angle change in each time segment.

[0085] Here, the photoacoustic signal attenuation is used to highlight the effect of coding depth on the signal, the tip contrast is used to highlight the tip features, and the velocity constraint and angle constraint are used to limit the physical rationality of the prediction. prior =[v μ ,v I ,v v ,v θ ] is mapped to 512 dimensions through a fully connected layer, and the input sequence X t Splicing to form enhanced input X′ T =[X t ,v prior ].

[0086] Through the splicing operation, the model receives both raw data and prior knowledge, which enhances the representation ability of the model input.

[0087] The Transformer's encoder-decoder architecture automatically learns the relationship between the original data and prior knowledge through a self-attention mechanism when processing augmented input. For example, the signal attenuation vector helps the model understand the effect of depth on the photoacoustic signal, avoiding predictions of unreasonable needle tip positions; and the velocity / angle constraint vector guides the model to generate trajectories that conform to physical constraints.

[0088] The model output is: the needle tip position [x, y, z] and insertion angle θ for the next 10 frames (about 0.5 seconds, frame rate 30fps).

[0089] The prediction results include: the mean (predicted value) of the position and angle, and the confidence interval (based on the Gaussian distribution assumption, 95% confidence level). If the predicted trajectory deviation exceeds a threshold (such as 0.5mm) or violates the speed / angle constraint, the system triggers a real-time alarm.

[0090] In this embodiment, the training of the Transformer model not only relies on the mean square error (MSE) loss, but also forces the model to comply with domain rules through prior constraint loss.

[0091] Specifically, the training loss function of the Transformer model includes:

[0092] Position prediction loss: Among them, [x t ,y t ,z t ] is the real location, is the predicted position, N=10.

[0093] Angle prediction loss: Among them, θ t is the true angle, is the prediction angle.

[0094] Prior constraint losses, including:

[0095] Signal attenuation constraint, used to penalize the signal strength at the predicted location that violates the attenuation model: Among them, I t is the signal strength at the predicted location, Zt is the depth of the predicted location, ε I =0.05 is the tolerance threshold.

[0096] Speed constraint, used to penalize predicted speed exceeding the limit: Among them, v t is the frame rate.

[0097] Angle constraint, used to penalize excessive angle changes: Among them, θ t is the angle of the current frame, θ t-1 is the angle of the previous frame.

[0098] Total loss: L = L MSE +0.5·L angle +0.2·(L decay +L speed +L angle )

[0099] In summary, the training loss function of the Transformer model adopts position prediction loss and angle prediction loss, and integrates weighted losses of signal attenuation constraint, speed constraint and angle constraint, which can ensure that the model prediction results are consistent with the data rules and meet the actual limitations of clinical operations.

[0100] In a preferred embodiment, the optimizer of the Transformer model adopts the Adam optimizer, with an initial learning rate of 0.001, a batch size of 32, and 200 training rounds.

[0101] Furthermore, in a preferred embodiment, the Transformer model also adopts the following training strategy:

[0102] Use learning rate decay (halved every 50 rounds) to prevent overfitting;

[0103] Early stopping (validation set loss does not decrease for 10 consecutive epochs).

[0104] The method provided in this example combines the advantages of photoacoustic imaging technology and the Transformer model to achieve accurate, real-time prediction of the puncture needle trajectory. Photoacoustic imaging acquires high-resolution images in real time. The Transformer model, as the main framework for sequence prediction, uses a self-attention mechanism to capture complex dependencies in time series data, enabling dynamic prediction of the puncture needle trajectory.

[0105] During data processing, the needle tip signal is depth-corrected based on the physical property of exponential decay of photoacoustic signals with tissue depth. This enhances the ability to extract features from deep tissue and improves the model's adaptability to complex anatomical structures. A weighting factor is also used to emphasize the needle tip signal, addressing the lack of contrast between the needle tip and soft tissue in traditional photoacoustic imaging.

[0106] Prior knowledge also bridges the gap between the Transformer model and domain knowledge. Through data augmentation and constraint injection, the purely data-driven Transformer model is able to integrate physical laws with clinical experience, ultimately achieving highly accurate and reliable needle trajectory prediction. By incorporating prior knowledge, the model can still generate reasonable predictions even when data is insufficient or noisy, improving model generalization. Prior knowledge explicitly introduces physical rules into the model, making predictions more consistent with medical logic. This design significantly enhances the model's practicality and safety in complex medical scenarios.

[0107] In addition, the training loss function of the Transformer model uses position prediction loss and angle prediction loss, and integrates weighted losses of signal attenuation constraints, speed constraints, and angle constraints. This can reduce the risk of overfitting, limit the model parameter space through the loss function, avoid excessive dependence on training data, and ensure that the model prediction results are consistent with the data laws and meet the actual limitations of clinical operations.

[0108] The accuracy of the above model is verified through specific experiments below.

[0109] In the clinical trial, several patients were selected for puncture operation.

[0110] Subject selection: Patients who require biopsy surgery were selected to ensure a representative sample. All patients underwent a detailed preoperative evaluation to exclude factors that may affect the trial results.

[0111] During the operation, the insertion path of the puncture needle, needle tip positioning time, total operation time, needle tip positioning error, trajectory deviation, and the success rate of the operation were recorded.

[0112] Photoacoustic imaging equipment is used to collect photoacoustic imaging images of the puncture process in real time. The images should clearly show the needle tip and surrounding tissue structures, such as Figure 2 shown.

[0113] Figure 3 The figure shows the comparison between the prediction result and the true trajectory of the puncture needle trajectory prediction method provided by this embodiment. The blue dot line (blue with circle mark) represents the "True Trajectory", that is, the true trajectory of the puncture needle. The orange square line (orange with square mark) represents the "Tracking Result", that is, the trajectory result predicted by the prediction method of the present invention. Figure 3 It can be seen that the curves of the true trajectory and the tracking result are basically consistent in overall trends, indicating that the prediction method of the present invention can well capture the movement direction and path of the target.

[0114] In most cases, the procedure time using the present invention was shortened by approximately 30% compared to conventional methods. This was primarily due to the system's rapid and accurate prediction of the needle tip position, which reduced the number of repeated adjustments.

[0115] The real-time trajectory prediction method of the present invention significantly reduces the risk of accidental injury during puncture operations, especially when puncturing complex tissue structures. The trajectory prediction method provided by the present invention can help operators make quick adjustments and avoid potential tissue damage.

[0116] Medical staff gave positive comments on their experience of using the method of the present invention, believing that the method provides strong support in complex surgical environments, reduces operational difficulty, and improves patient safety.

[0117] Example 2

[0118] Based on the same inventive concept as the puncture needle trajectory prediction method in the aforementioned embodiment 1, this embodiment also provides a puncture needle trajectory prediction system, such as Figure 4 As shown, the system includes:

[0119] A data acquisition module is used to acquire continuous photoacoustic imaging image data when the puncture needle punctures the tissue;

[0120] a data processing module, configured to process the photoacoustic imaging image data, extract the needle tip position, and convert each frame of image data into a feature vector relating to the needle tip position;

[0121] The model building module is used to build the Transformer model. The input of the model is the feature vector of the needle tip position and prior knowledge of the past T1 frame image data. The output of the model is the needle tip center coordinates (x, y, z) of the future T2 frame image data and the insertion angle θ of the needle tip in the surrounding tissue. T1 and T2 are both positive integers, and T1>T2.

[0122] In a preferred embodiment, the puncture needle trajectory prediction system also includes a visualization interface, for example, a 3D visualization interface developed using WebGL, which renders the puncture needle trajectory in real time (the green solid line is the actual path, and the red dotted line is the predicted path), displays the needle tip position (highlighted mark), the predicted confidence interval (transparent area) and the signal strength (color coding).

[0123] Dynamic rendering supports multiple view switching (cross-section, sagittal, and coronal). Highlighting enhances the operator's ability to capture key information. Confidence interval display intuitively reflects the reliability of prediction results.

[0124] In a preferred embodiment, the puncture needle trajectory prediction system further includes:

[0125] The multimodal data fusion display module is used to overlay the real-time image of photoacoustic imaging with the trajectory prediction results to assist the operator in understanding the spatial relationship between the puncture needle and the surrounding tissue.

[0126] The parameter panel is used to display key parameters in real time, including current speed (mm / s), insertion angle (°), signal strength (dB) and prediction error (mm).

[0127] The view operation module supports interactions such as zooming, rotating, and panning, facilitating multi-angle observation during surgery.

[0128] The historical playback module can replay the trajectory data of the entire operation process for post-operative analysis and training.

[0129] In a preferred embodiment, the puncture needle trajectory prediction system further includes:

[0130] Trajectory deviation warning module: When the deviation between the predicted trajectory and the target path is greater than 0.5mm, a red highlight prompt is triggered.

[0131] Speed / angle abnormality warning module: When the predicted speed is greater than 10mm / s or the angle change is greater than 30°, an audible alarm is issued.

[0132] Intelligent Operation Suggestion Module: Generates operation suggestions based on prediction results, prompting users to adjust the insertion angle or speed. For example, "The current angle deviates from the target by 5°, so it is recommended to fine-tune it to the left by 3°" or "Deep tissue resistance increases, so it is recommended to reduce the insertion speed."

[0133] Risk level assessment module: Combines confidence intervals and prediction errors to grade current operational risks (low risk - green, medium risk - yellow, high risk - red).

[0134] The various specific processes and specific examples of the puncture needle trajectory prediction method in the aforementioned embodiment 1 are also applicable to the puncture needle trajectory prediction system of this embodiment. Through the detailed description of embodiment 1, those skilled in the art can clearly understand the implementation method of the puncture needle trajectory prediction system in this embodiment, so for the sake of brevity of the specification, they will not be described in detail here.

[0135] In this embodiment, the cognitive load of the operator is reduced by integrating photoacoustic imaging, trajectory prediction, physical parameters (speed, angle) and tissue characteristics (signal strength) into a unified interface. Through a closed-loop process of real-time prediction → visualization → alarm → recommendation, a dynamic adjustment mechanism is formed during surgery, which significantly reduces the risk of misoperation. The interface supports personalized configuration (such as alarm threshold adjustment and view layout customization) to adapt to the operating habits of different doctors. The explicit display of confidence intervals and physical constraints makes the prediction results consistent with medical logic and enhances doctors' trust in AI models.

[0136] Clinical trials have shown that using this system reduces needle positioning time by an average of 30% and the number of adjustments by 50%. Real-time early warnings have also reduced the incidence of intraoperative complications (such as misplaced vascular puncture) by 40%. In scenarios such as tumor biopsy and nerve block, the system helps doctors achieve submillimeter precision, improving the success rate of diagnosis and treatment.

[0137] In summary, the puncture needle trajectory prediction system provided in this embodiment transforms complex AI prediction results into intuitive and actionable clinical information through real-time data rendering, intelligent risk warnings, and dynamic decision support. The synergistic effect of these two factors not only improves the safety and efficiency of puncture procedures but also promotes the practical application of AI technology in clinical scenarios, providing key technical support for precision medicine.

[0138] Example 3

[0139] Based on the same inventive concept as the puncture needle trajectory prediction method in the aforementioned embodiment 1, this embodiment also provides a puncture needle trajectory prediction device, such as Figure 5 As shown, a computer program is stored thereon, and when the program is executed by the processor, the steps of the puncture needle trajectory prediction method described in Example 1 are implemented.

[0140] Among them, Figure 5In the embodiment of the present invention, a bus architecture (represented by bus 300) is shown. Bus 300 may include any number of interconnected buses and bridges. Bus 300 links various circuits together, including one or more processors represented by processor 302 and memory represented by memory 304. Bus 300 may also link various other circuits together, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are not described further herein. Bus interface 306 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 may be the same component, namely a transceiver, which provides a means for communicating with various other devices over a transmission medium.

[0141] The processor 302 is responsible for managing the bus 300 and general processing, while the memory 304 may be used to store data used by the processor 302 when performing operations.

[0142] Example 4

[0143] Based on the same inventive concept as the puncture needle trajectory prediction method in the aforementioned embodiment 1, this embodiment further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the puncture needle trajectory prediction method described in embodiment 1.

[0144] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0148] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0149] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A puncture needle trajectory prediction method, characterized in that: The steps include: Acquiring continuous photoacoustic imaging image data when the puncture needle punctures the tissue; Processing the photoacoustic imaging image data, extracting the needle tip position, and converting each frame of image data into a feature vector related to the needle tip position; A Transformer model was established. The input of the model was the feature vector of the needle tip position in the past T1 frame image data and prior knowledge. The output of the model was the needle tip center coordinates (x, y, z) in the future T2 frame image data and the insertion angle θ of the needle tip in the surrounding tissue. T1 and T2 were both positive integers.

2. The puncture needle trajectory prediction method according to claim 1, wherein: The photoacoustic imaging image data is collected in real time by a photoacoustic imaging device; and / or The photoacoustic imaging image data shows the needle tip and surrounding tissue structures.

3. The puncture needle trajectory prediction method according to claim 1, wherein: The method for processing photoacoustic imaging image data comprises: Adaptive threshold segmentation algorithm is used to separate the needle tip from the surrounding tissue and extract the needle tip contour. Based on the physical characteristics of photoacoustic signal attenuation with depth, the needle tip signal intensity is depth-corrected. Use an edge detection algorithm to determine the coordinates (x, y, z) of the needle tip center and the insertion angle θ of the needle tip in the surrounding tissue; where x, y, and z are the coordinate values of the length, width, and depth axes, respectively; The needle tip center coordinates (x, y, z), the needle tip insertion angle θ and the corrected needle tip signal intensity I z Normalize to the interval [0,1] to eliminate dimension differences; normalize the timestamp t to relative time; obtain the feature vector [x, y, z, θ, I z ,t].

4. The puncture needle trajectory prediction method according to claim 3, wherein: When extracting the needle tip contour, the needle tip features are highlighted through weighted enhancement based on the fact that the signal intensity of the needle tip is higher than that of the surrounding tissue.

5. The puncture needle trajectory prediction method according to claim 1, wherein: The prior knowledge includes: Photoacoustic signal attenuation: Encoded depth correction vector Where μ is the attenuation coefficient, Z1,…,Z T1 Respectively represent the depth when the input sequence is 1,…,T1; Tip contrast: Encoded intensity weight vector v I =[I Z1 / I avg ,...,I ZT1 / I avg ]; among them, I Z1 ,…,I ZT1 , respectively represent the tip signal strength when the input sequence is 1,…,T1, I avg is the average needle tip signal intensity of the T1 input sequences; Speed constraint: Encoding speed upper limit vector v v =[min(v t ,10) / 10,...,min(v T ,10) / 10]; where v t is the current instantaneous frame speed, v T is the maximum inter-frame velocity in each time segment; Angle constraint: encoding the upper limit vector v of the angle change θ =[min(|θ t -θ t-1 |,30) / 30,...,min(|θ T -θ T-1 |,30) / 30]; where θ t -θ t-1 Represents the angle change between the current frame and the previous frame, θ T -θ T-1 It is the maximum angle change between two adjacent frames in each time segment.

6. The puncture needle trajectory prediction method according to claim 5, characterized in that: The training loss function of the Transformer model includes: Position prediction loss: Among them, [x t ,y t ,z t ] is the real location, is the predicted position, N = 10; Angle prediction loss: Among them, θ t is the true angle, is the prediction angle; Signal attenuation constraint, used to penalize the signal strength at the predicted location that violates the attenuation model: Among them, I t is the signal strength at the predicted location, Zt is the depth of the predicted location, ε I is the tolerance threshold; Speed constraint, used to penalize predicted speed exceeding the limit: Angle constraint, used to penalize excessive angle changes: Total loss: L = L MSE +0.5·L angle +0.2·(L decay +L speed +L angle ).

7. A puncture needle trajectory prediction system, characterized in that: The system comprises: A data acquisition module is used to acquire continuous photoacoustic imaging image data when the puncture needle punctures the tissue; a data processing module, configured to process the photoacoustic imaging image data, extract the needle tip position, and convert each frame of image data into a feature vector relating to the needle tip position; The model building module is used to build the Transformer model. The input of the model is the feature vector of the needle tip position and prior knowledge of the past T1 frame image data. The output of the model is the needle tip center coordinates (x, y, z) of the future T2 frame image data and the insertion angle θ of the needle tip in the surrounding tissue. T1 and T2 are both positive integers.

8. The puncture needle trajectory prediction system according to claim 7, wherein: The system also includes one or more of the following modules: A visual interface that renders the puncture needle trajectory in real time, displaying the needle tip position, predicted confidence interval, and signal strength; A multimodal data fusion display module is used to overlay the real-time image of photoacoustic imaging with the trajectory prediction results to assist the operator in understanding the spatial relationship between the puncture needle and surrounding tissues; The view operation module supports zooming, rotating, and panning interactions, facilitating multi-angle observation during surgery; The historical playback module can playback the trajectory data of the entire operation process for postoperative analysis and training; The trajectory deviation warning module is used to trigger a highlight prompt when the deviation between the predicted trajectory and the target path exceeds a set threshold; Speed / angle abnormality warning module, used to issue an audible alarm when the predicted speed or angle change exceeds the set threshold; An intelligent operation suggestion module is used to generate operation suggestions based on the prediction results, prompting users to adjust the insertion angle or speed; The risk level assessment module is used to classify the current operational risk by combining the confidence interval and the prediction error.

9. A puncture needle trajectory prediction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the puncture needle trajectory prediction method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the puncture needle trajectory prediction method according to any one of claims 1 to 6 are implemented.

Images