Ultrasound-guided navigation system based on multi-modal detection and cnb catapult mechanism

Abstract

The application discloses an ultrasound-guided navigation system based on a multimodal detection and CNB ejection mechanism, and belongs to the technical field of ultrasound-assisted diagnosis. The multimodal detection simultaneously loads a directional bounding box model and a segmentation model during system initialization. The directional bounding box model performs directional detection on a puncture needle in an input frame. In a result extraction stage, puncture needle information is extracted from the directional bounding box model detection result, and a lesion mask is extracted from a segmentation result. The lesion mask is adjusted to the same size as the input frame. In the extraction of the lesion mask, the validity of the segmentation result is verified, and then the first valid mask data is extracted from the multi-class segmentation result. The mask data is transmitted from GPU memory to CPU, and the floating-point probability value is converted into an integer binary mask, so that the puncture needle direction detection and multi-class anatomical structure segmentation are realized. The CNB ejection mechanism establishes a mathematical model of the ejection process through an ejection trajectory calculation function, thereby providing a reliable mathematical basis for CNB ejection operation diagnosis.

Description

Technical Field

[0001] This invention relates to the field of ultrasound-assisted diagnostic technology, and in particular to an ultrasound-guided navigation system based on multimodal detection and CNB ejection mechanism. Background Technology

[0002] A puncture needle is a medical device used in minimally invasive surgery to sample and inject tissue from various organs, including the kidneys, liver, lungs, breast, thyroid, prostate, pancreas, testes, uterus, ovaries, and the body surface. Ultrasound-guided navigation is a medical technique that uses ultrasound technology to achieve precise positioning and diagnosis (or identification). It assists doctors in planning puncture paths or injecting drugs through real-time imaging, significantly improving the accuracy and safety of diagnosis.

[0003] In existing technologies, ultrasound-guided puncture needle navigation systems mostly use a single YOLO detection model, which cannot simultaneously perform directional puncture needle structure detection and anatomical target segmentation. At the same time, traditional bounding box detection is difficult to accurately capture the direction of the puncture needle, which can easily lead to trajectory calculation errors. Furthermore, existing segmentation models cannot distinguish between multiple types of anatomical structures in real time, posing numerous safety hazards.

[0004] Furthermore, during the ejection of the puncture needle, the ejection distance is predetermined and cannot be adjusted in real time after firing. At the same time, the corresponding firing trajectory is also predetermined. Therefore, the accuracy of the ejection trajectory calculation and the ejection safety assessment and risk management of the CNB ejection mechanism are essential core technologies. Summary of the Invention

[0005] This application provides an ultrasound-guided navigation system based on multimodal detection and CNB ejection mechanism, which can simultaneously detect the direction of the puncture needle and segment multiple anatomical structures in real-time ultrasound imaging, and can provide a reliable mathematical basis for CNB ejection operation.

[0006] This application provides an ultrasound-guided navigation system based on multimodal detection and CNB ejection mechanism. For multimodal detection, the navigation system is implemented using a parallel dual-model detection architecture, including a detection phase and a result extraction phase. In the detection phase, the system sets unified inference parameters. During system initialization, two YOLOv11 models are loaded simultaneously: the first is an oriented bounding box model used for detecting the directionality of the puncture needle in the input frame; the second is a segmentation model used for segmenting seven types of anatomical structures. In the result extraction phase, the detection system extracts puncture needle information from the oriented bounding box model detection results and extracts information from the segmentation results by category. The system extracts all valid masks or their contour / center information for anatomical structures and adapts the mask size for each category to the same size as the input frame. The segmentation model establishes a hierarchical safety system based on medical priorities, with category mapping consistent with the training configuration. When extracting masks for each category, the system first verifies the validity of the segmentation results, then extracts all valid mask data from each category of the multi-class segmentation results. The mask data is transferred from GPU memory to the CPU and binarized using a preset segmentation threshold, converting floating-point probability values ​​into integer binary masks. When the mask size does not match the target frame size, the system uses a nearest neighbor interpolation algorithm for size adjustment.

[0007] The navigation system also establishes a mathematical model of the ejection process through an ejection trajectory calculation function, including:

[0008] Physical coordinate transformation converts the needle tip position and target center from the pixel coordinate system to the physical coordinate system. Then, the direction vector of the puncture needle is calculated based on the needle angle, converting the angle from degrees to radians, and the unit vector components in the x and y directions are calculated using trigonometric functions.

[0009] Direction vector calculation, used for converting angles to direction vectors;

[0010] The ejection endpoint is calculated by using the physical coordinates of the starting position of the needle tip and the ejection distance to calculate the physical coordinates of the ejection endpoint.

[0011] Target distance calculation is used to calculate the straight-line distance from the tip of the needle to the center of the target using the physical coordinates of the target center.

[0012] In one possible implementation, navigation path planning is performed using an enhanced A* algorithm with medical constraints. During system initialization, a medical grid space is established as the computational basis for the A* algorithm. The continuous image space is discretized into a searchable node network through gridding. Pixel spacing parameters are set for physical-to-pixel coordinate transformation. The medical constraint parameter system includes:

[0013] Angle variation limits: The recommended maximum angle variation is 30.0 degrees, and the clinical limit angle threshold is 45.0 degrees, simulating the physical limitations of needle rotation in real puncture operations;

[0014] Step size control, with a minimum step size of 1 grid unit and a maximum step size of 3 grid units, ensures the operability of the navigation path;

[0015] Grid resolution, default 4 pixels / grid unit, balances calculation accuracy and processing efficiency through configurable resolution parameters;

[0016] The connectivity method employs 8-connected neighborhood search, providing path selection in 8 directions;

[0017] The weighting strategy uses a risk weight factor of 5.0 and a path smoothing weight factor of 2.0.

[0018] In one possible implementation, a continuous risk field is constructed in the pixel space of the original ultrasound image, and then the pixel-level risk values ​​are aggregated to grid nodes through spatial mapping to realize the transformation from the continuous risk field to the discrete search space. In turn, the final generated risk field provides guidance for the navigation path planning algorithm.

[0019] The detection system first creates a zero-value-initialized floating-point risk field matrix based on the input image size, and then constructs boundary risk regions and anatomical structure risk regions in layers;

[0020] The boundary risk area is defined as follows: boundary constraints are established within a 5-pixel range of the top edge, bottom edge, left edge, and right edge of the image.

[0021] The anatomical structure risk area is defined as follows: For each detected anatomical structure object, the system extracts the segmentation mask or geometric contour, center coordinates, critical risk threshold D_crit, warning threshold D_warn, and avoidance priority parameters for each anatomical structure object; the risk partition distance is uniformly calculated using the boundary safety distance d_boundary, and an equidistant hierarchical risk model is adopted to establish a two-level risk area system, in which the high-risk area is assigned an extremely high risk value of 1000.0 × priority, and the warning area is assigned a medium risk value of 200.0 × priority.

[0022] In one possible implementation, a movement cost architecture is integrated into the navigation path planning process. The detection system establishes a three-layer cost structure, namely enhanced movement cost, which includes basic movement cost, risk cost, and smoothness cost. The basic movement cost calculates the geometric distance from the current grid point to adjacent grid points using the Euclidean distance formula to ensure the spatial efficiency of the path. The risk cost is calculated by multiplying the risk value of adjacent points by the risk weight factor in medical constraints and applying a standardization coefficient of 0.001 for numerical adjustment. The smoothness cost is based on the angle changes of navigation path segments. When the current grid point has a parent node, the smoothness cost calculation algorithm calculates the angle changes of adjacent path segments: obtained by the vector dot product and vector magnitude. The cosine of the included angle is used, and then the inverse cosine function is applied to obtain the change in angle and convert it from radians to degrees. The system evaluates all angle changes: when the angle change is ≤10 degrees, a negative reward is given to encourage a smooth path; when the angle change is between 10 and 20 degrees, a linearly increasing penalty is applied; when the angle change is between 20 and 30 degrees, a quadratic penalty is applied; when the angle change is >30 degrees, the penalty intensity increases sharply, and a cubic or infinite penalty is applied to reinforce the cost of uncomfortable turning. The weighting factors in the three-layer cost structure are optimized based on clinical practice: the risk weight factor of 5.0 ensures that the algorithm prioritizes avoiding high-risk areas, based on the principle of the highest priority of medical safety; the standardization coefficient of 0.001 is used to unify the numerical range and ensure the reasonable superposition of costs of different dimensions.

[0023] In one possible implementation, an angle change penalty mechanism is also integrated into the navigation path planning process. The detection system establishes an angle change threshold mechanism based on medical constraints. When the angle change between path segments is greater than 30 degrees, the angle change penalty strategy algorithm applies an angle change penalty. The penalty intensity is determined by the product of the angle change amount and the smoothness weight factor, and a coefficient of 0.1 is applied for numerical optimization.

[0024] The angle change penalty strategy algorithm includes: constructing a five-level medical safety threshold system, establishing a graded penalty mechanism based on the physical limitations of clinical puncture operations, and setting five evaluation levels according to the medical safety level of the angle change: comfortable operation range (0-10 degrees), clinically acceptable range (10-20 degrees), high-risk operation range (20-30 degrees), extremely dangerous operation range (30-45 degrees), and technically prohibited range (≥45 degrees). Among them, 30 degrees is the recommended maximum upper limit of angle change. If the angle change exceeds 30 degrees, searching is still allowed, but the penalty intensity increases sharply. 45 degrees is the clinical limit threshold. When the angle change is ≥45 degrees, an infinite penalty value is set to prohibit the path from expanding.

[0025] In one possible implementation, the navigation system establishes a geometric distance calculation algorithm based on parameterized straight line equations, with the calculation formula as follows:

[0026] d1 = √[(px - projection_x)² + (py - projection_y)²], where:

[0027] d1 is the shortest distance from the target point to the line segment;

[0028] px and py are the coordinates of the target point;

[0029] projection_x = x1 + t × (x2 - x1), projection_y = y1 + t × (y2 - y1), where x1 and y1 are the coordinates of the starting point of the line segment, x2 and y2 are the coordinates of the ending point of the line segment, and t is the projection parameter, which represents the relative position of the projection point on the line segment. t = max(0, min(1, [(px - x1) × (x2 - x1) + (py - y1) × (y2 - y1)] / line_length²)), where the value of t is in the range of 0-1. line_length is the length of the line segment, line_length = √[(x2 - x1)² + (y2 - y1)²]. max and min are the maximum and minimum value functions, respectively, to ensure that the projection point is within the range of the line segment.

[0030] The shortest distance from the target point to the line segment is combined with the determination of whether the object is inside or outside the structure to obtain the boundary safety distance d_boundary, where d_boundary≤0 indicates that the ejection trajectory intersects with or intrudes into the boundary of the anatomical structure.

[0031] In one possible implementation, the system establishes a multi-dimensional safety assessment framework including an overall safety score and a list of key risks. The overall safety score uses a standardized scoring system from 1.0 to 0.0, where 1.0 represents the safest state and 0.0 represents the most dangerous state. Based on the status of key risks in the list of key risks, the system automatically triggers an adjustment suggestion generation process. Adjustment suggestions include adjusting the ejection distance, changing the needle insertion angle, and selecting different needle insertion points. The overall safety score is calculated using a cumulative penalty mechanism. The base score is 1.0 in the safest state, representing no risk. The score is determined by taking the minimum value of the penalty factors for all risk sources, achieving a veto for the most severe risk.

[0032] Overall safety score = base score × penalty;

[0033] Penalty is the penalty coefficient, which is set according to the risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safety penalty = 1.0. When multiple risks exist at the same time, the penalty is the minimum value of the penalty coefficient of each risk.

[0034] In one possible implementation, the multi-dimensional security assessment framework further includes a warning message list. This list outputs a graded hierarchy of the gap distance (distance_to_trajectory) based on the detected anatomical structure type and its preset risk distance threshold, with each level mutually exclusive.

[0035] (1) Arteries: distance_to_trajectory≤5.0mm is "critical risk"; 5.0mm<distance_to_trajectory≤8.0mm is "warning"; distance_to_trajectory>8.0mm is "safe";

[0036] (2) Veins: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe";

[0037] (3) Esophagus: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe";

[0038] (4) Diaphragm: distance_to_trajectory≤2.0mm is "critical risk"; 2.0mm<distance_to_trajectory≤4.0mm is "warning"; distance_to_trajectory>4.0mm is "safe";

[0039] (5) Skeleton: As a physical obstacle zone, when distance_to_trajectory = 0.0mm (i.e., the boundary safety distance d_boundary≤0, indicating that the ejection trajectory intersects, contacts, or invades the interior of the skeleton boundary), it is a "critical risk"; when 0.0mm<distance_to_trajectory≤1.0mm, it is a "warning"; when distance_to_trajectory>1.0mm, it is a "safe";

[0040] (6) Lesions and lymph nodes: These are the target areas and are not to be avoided. Their critical_clearance and warning_clearance are both 0.0 mm, and they are marked as "safe / crossable" by default. When multiple risks exist at the same time, the warning message list will output multiple risk level labels and prompts simultaneously, with the highest priority risk as the main prompt. The risk level is adjusted for the overall safety score using a post-processing penalty coefficient.

[0041] Final safety score = Base score × Penalty;

[0042] The base score is 1.0 in the safest state with no risk; the penalty coefficient is determined according to the risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safe penalty = 1.0; when multiple risks exist at the same time, the penalty coefficient is the minimum of the penalty coefficients of each risk; in obstacle areas, skeletal structures are used as physical obstacles and are completely bypassed during path planning.

[0043] In one possible implementation, the navigation system establishes differentiated technical alternatives based on seven types of anatomical structures, including artery-specific treatment, intermediate-risk structure treatment, and obstacle structure treatment. For the artery-specific treatment, Doppler ultrasound is used to confirm the artery location before CNB is performed. For the intermediate-risk structure treatment, the needle insertion path is adjusted to avoid the intermediate-risk structure. For the obstacle structure treatment, the needle insertion path or point is modified.

[0044] In one possible implementation, the system establishes a ballistic error calculation model for the ejection trajectory, including position error calculation, relative error calculation, and angle error calculation;

[0045] In the CNB trajectory validator, the position error is calculated using the Euclidean distance formula:

[0046] Position E = √[(predicted x - actual x)² + (predicted y - actual y)²] × k; where...

[0047] E position: Position error, in millimeters (mm);

[0048] x-prediction, y-prediction: The coordinates of the launch endpoint predicted by the algorithm (pixel coordinates).

[0049] x actual, y actual: The actual recorded coordinates of the needle tip impact point (pixel coordinates).

[0050] k=0.208998: Pixel pitch calibration coefficient, in mm / pixel. This coefficient is based on the standard image resolution and physical size calibration of the ultrasound equipment to ensure the accuracy of coordinate transformation.

[0051] Relative error reflects the proportional relationship between prediction error and launch distance:

[0052] E relative = (E position / D ejection) × 100;

[0053] E-relative: Relative error, expressed as a percentage (%);

[0054] D ejection: ejection distance, set to 22.0mm in the CNB system;

[0055] The angle error is obtained by calculating the angle between the predicted trajectory vector and the actual impact trajectory vector:

[0056] Angle E = arccos[cos(θ)] × (180° / π), cos(θ) = (Vpredicted·Vactual) / (|Vpredicted| × |Vactual|), where the cosine value cos(θ) needs to be protected by the numerical domain, i.e., cos(θ) = max(-1.0, min(1.0, cos(θ))). The numerical domain protection mechanism forces cos(θ) to be within the range of [-1.0, 1.0] to prevent floating-point operation errors from causing the inverse cosine function calculation to fail, thus improving the numerical stability of the algorithm. When |Vpredicted| = 0 or |Vactual| = 0, the denominator is 0, and the system marks the angle error as unusable and skips the calculation (or sets it to 0 according to preset rules), thereby avoiding division by zero.

[0057] V-prediction: The vector pointing from the tip of the needle to the predicted ejection endpoint, V-prediction = (x-prediction - x-tip, y-prediction - y-tip);

[0058] Vactual: The vector from the needle tip to the actual point of impact, Vactual = (xactual - xneedle tip, yactual - yneedle tip).

[0059] E angle: angular error, in degrees (°);

[0060] x needle tip, y needle tip: Coordinates of the needle tip position at the start of ejection;

[0061] ·: Vector dot product operation;

[0062] |Vprediction|: Euclidean magnitude of the prediction vector;

[0063] |Vactual|: Euclidean magnitude of the actual vector;

[0064] max, min: Numerical constraint functions that ensure the cosine value is within the valid range [-1, 1].

[0065] Beneficial Effects: Compared with existing technologies, the ultrasound-guided navigation system based on multimodal detection and CNB ejection mechanism provided in this application adopts a YOLOv11 dual-model architecture for target detection and anatomical structure segmentation. Combined with the medical A* path planning algorithm, it can achieve puncture needle direction detection and recognition of multiple types of anatomical structures. The system verifies detection accuracy using standard evaluation methods. Angle calculation is based on geometric relationships, and distance measurement is converted through pixel spacing calibration, supporting multi-scale detection of puncture needles of different specifications. System performance indicators may vary depending on the actual test environment and hardware configuration. Specific detection accuracy, angle measurement accuracy, and segmentation quality indicators should be based on the test results of the actual deployment environment.

[0066] By calculating the straight-line distance from the needle tip to the center of the target, and through precise mathematical modeling and high-precision numerical calculation, a reliable mathematical basis can be provided for CNB ejection operation diagnosis.

[0067] It can deeply integrate AI detection results of anatomical structures with medical statistical models to provide scientific risk quantification basis for CNB ejection operations;

[0068] Furthermore, it can achieve deep integration of the system with medical safety requirements, providing scientific, practical, and operable safety optimization guidance for CNB ejection operations through standardized adjustment strategies and technical alternatives based on anatomical structure classification, ensuring the safety and effectiveness of clinical operations.

[0069] These and other objects, features and advantages of the present invention will become fully apparent from the following detailed description. Attached Figure Description

[0070] Figure 1 The diagram shows the hierarchical architecture of the ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism of this application. Detailed Implementation

[0071] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0072] To avoid ambiguity in terminology and to ensure consistency among the various parts of this specification, the following terms and symbols are agreed to be used consistently:

[0073] Anatomical structure objects / target structure objects: Each category of instances output by the segmentation model (e.g., arteries, veins, bones, esophagus, diaphragm, lesions, lymph nodes, etc.). The system extracts the mask (or contour / center information) for each category and each instance and outputs it.

[0074] Treatment target structure: The target tissue (e.g., lesion, lymph node) corresponding to the planned endpoint;

[0075] Non-therapeutic target structures: Anatomical structures other than therapeutic target structures.

[0076] Mask / contour / center: These refer to the binary segmented region, its geometric boundary curve, and the center coordinates obtained from the mask's geometric moments, respectively.

[0077] Boundary safety distance d_boundary: The minimum signed distance between the ejection trajectory and the geometric boundary of the anatomical structure; d_boundary≤0 indicates that the trajectory intersects with the boundary or intrudes, d_boundary>0 indicates that the trajectory is outside the boundary and there is a safety gap.

[0078] Critical threshold D_crit and warning threshold D_warn: Distance thresholds configured based on structure category, both with the same dimension as d_boundary; risk determination uses d_boundary as the sole comparison and sole judgment metric.

[0079] The two-tier risk zone system consists of a high-risk zone (d_boundary < D_crit) and a warning zone (D_crit ≤ d_boundary < D_warn).

[0080] Risk value: The baseline value for boundary risk is 100; anatomical structures are assigned a tiered value: High-risk zone = 1000.0 × priority p, Warning zone = 200.0 × priority p, Normal zone = 0.0.

[0081] Obstacle threshold T_obs: The threshold used to mark high-risk areas of non-treatment target structures as hard obstacles; preferably configured as T_obs=β×(200.0×p_target), where p_target is the priority of the treatment target structure (default 1), and β>1 is the scaling factor.

[0082] d_display: The non-negative distance used for visualizing / displaying the risk field. Take d_display=max(0,d_boundary), but threshold comparison and risk judgment are still based on d_boundary.

[0083] In ultrasound-guided puncture needle navigation detection systems, a single YOLO detection model is often used. This model has the problem of not being able to simultaneously detect directional puncture needle structures and segmented anatomical targets with high precision. In particular, traditional bounding box detection cannot accurately capture the direction of the puncture needle, which makes the trajectory calculation error easy. Furthermore, existing segmentation models cannot distinguish between multiple types of anatomical structures in real time, which poses a safety hazard and makes it difficult to achieve accurate positioning, diagnosis or identification of puncture needles.

[0084] In view of this, embodiments of this application provide an ultrasound-guided navigation system based on multimodal detection and CNB ejection mechanism, which can simultaneously achieve high-precision puncture needle direction detection and segmentation of multiple anatomical structures in real-time ultrasound imaging, while maintaining a computational efficiency of less than 50ms per frame. It can also integrate YOLOv11-OBB (directed bounding box) detection for puncture needle structure with YOLOv11-SEG (segmentation) for 7 types of anatomical structure recognition, avoiding model interference and maintaining detection accuracy above 90%. In addition, it can implement a medical A* pathfinding algorithm that combines anatomical structure risk field, multi-objective optimization and real-time constraint satisfaction for ultrasound-guided surgery. Furthermore, it can construct a dynamic risk field that reflects different safety requirements of different anatomical structures (arteries, veins, esophagus, diaphragm, bones, lesions, lymph nodes) and has an appropriate distance-based risk gradient.

[0085] Specifically, the multimodal detection system for ultrasound-guided puncture needle navigation (hereinafter referred to as the detection system) is implemented using a parallel dual-model detection architecture. The collaborative operation of two YOLOv11 models is achieved through the ModelDetector class, including a detection phase and a result extraction phase. In the detection phase, a parallel inference strategy is employed to achieve optimal performance. The detection system sets unified inference parameters, including a confidence threshold of 0.25 (optimized for medical imaging), an intersection-over-union (IoU) threshold of 0.45 (to prevent duplicate detection), and detailed output control. During system initialization, two dedicated YOLOv11 models are loaded simultaneously. The first is an Oriented Bounding Box (OBB) model used for directional detection of the puncture needle in the input frame; the second is a Segmentation model used for segmenting seven types of anatomical structures. The parallel execution of these two processes further improves processing efficiency.

[0086] During the result extraction stage, the detection system extracts puncture needle information from the directional bounding box (OBB) model detection results, including needle tip position, angle, and bounding box parameters; it extracts the effective masks (or their contour / center information) of all anatomical structures from the segmentation results one by one according to category, and adapts the size of each mask to the same size as the input frame; the mask extraction process applies a segmentation threshold of 0.5 to each category to generate binary masks to ensure the accuracy of the segmentation boundaries.

[0087] Combination Figure 1 Example 1: Detailed technical implementation of the parallel dual-model detection architecture

[0088] This embodiment details the specific structural design, training method, and technical parameters of the dual YOLOv11 model.

[0089] 1.1 Technical Implementation of the YOLOv11-OBB Oriented Bounding Box Detection Model:

[0090] The backbone network adopts a hierarchical feature extraction architecture, which includes five main downsampling stages: P1 layer output stride 2 (feature map size is 1 / 2 of the input), P2 layer output stride 4 (feature map size is 1 / 4 of the input), P3 layer output stride 8 (feature map size is 1 / 8 of the input), P4 layer output stride 16 (feature map size is 1 / 16 of the input), and P5 layer output stride 32 (feature map size is 1 / 32 of the input).

[0091] The feature fusion network adopts a feature pyramid network design, which achieves multi-scale feature fusion through upsampling and feature stitching. It integrates the Spatial Pyramid Pooling Module (SPPF) to expand the receptive field and introduces the Position Aware Attention Module (C2PSA) to enhance the detection capability of slender objects.

[0092] Special adjustments for puncture needle detection include: setting the number of categories to 1 for single-class detection of puncture needles; expanding the output parameters to 6 dimensions including center point coordinates x and y, width w, height h, rotation angle θ, and confidence; limiting the angle parameter θ to a range of 0 to π radians, corresponding to 0 to 180 degrees; and optimizing the receptive field for feature extraction of slender objects.

[0093] Multi-scale detection adaptation strategy: The P3 detection layer handles smaller needles with a length range of 30-60 mm, the P4 detection layer handles medium-length needles with a length range of 60-120 mm, and the P5 detection layer handles long needles with a length range of 120-180 mm.

[0094] 1.2 Technical Implementation of the YOLOv11-SEG Seven-Class Anatomical Structure Segmentation Model:

[0095] Category Definitions and Medical Stratification: Arteries (Category 0) are the highest priority medical contraindication areas, veins (Category 1) are the second-level contraindication areas, lymph nodes (Category 2) are the target tissues, bones (Category 3) are the areas of physical barriers, lesions (Category 4) are the primary treatment targets, esophagus (Category 5) are the second-level contraindication areas, and diaphragm (Category 6) are the second-level contraindication areas.

[0096] The segmentation head network adopts a prototype network architecture with 32 prototypes and 256 mask feature dimensions. It outputs multi-level segmentation masks to support instance-level accurate segmentation.

[0097] The medical priority weighting is based on clinical medical standards, setting risk weight coefficients for different anatomical structures: artery weight coefficient 9.0 (highest risk), vein weight coefficient 7.0, lymph node weight coefficient 1.0 (target tissue), bone weight coefficient 8.0 (physical obstacle), lesion weight coefficient 1.0 (primary target), esophagus weight coefficient 7.0, and diaphragm weight coefficient 6.0.

[0098] 1.3 Parallel Inference Architecture Implementation:

[0099] The dual-model parallel computing adopts an independent model loading method to avoid parameter conflicts. The unified inference parameter settings include a confidence threshold of 0.25 and an IoU threshold of 0.45. The input image is uniformly standardized to 640×640 pixels. FP32 precision is used to ensure the computational accuracy of medical applications.

[0100] The parallel execution mechanism utilizes a thread pool executor to achieve true parallel inference. The maximum number of worker threads is set to 2 to handle OBB detection and segmentation tasks respectively. Parallel efficiency is ensured through asynchronous submission and result retrieval mechanisms.

[0101] Memory optimization strategies include pre-allocating GPU memory buffers to reduce dynamic memory allocation overhead, and using CUDA streams for asynchronous data transfer and computation overlap.

[0102] Example 2: Detailed implementation of training dataset composition and annotation methods:

[0103] 2.1 Design of the puncture needle detection dataset:

[0104] Dataset size statistics: a total of 3230 images, 2584 images in the training set (80%), and 646 images in the validation set (20%); the image size distribution includes primary size 1920×1080 pixels (ultra-high-definition ultrasound images), secondary size 1256×894 pixels (high-definition ultrasound images), and standard size 720×576 pixels (standard ultrasound images).

[0105] Statistics on the distribution of puncture needle specifications: 14G needles account for 35%, 16G needles account for 35%, 18G needles account for 20%, and 20G needles account for 10%; Needle length distribution: short needles (30-60 mm) account for 25%, medium length needles (60-120 mm) account for 45%, and long needles (120-180 mm) account for 30%.

[0106] X-AnyLabeling annotation tool application: Supports directional bounding box annotation, provides an angle adjustment interface to precisely control the annotation direction, supports batch annotation and quality inspection, and outputs YOLO format annotation files to directly adapt to the training process;

[0107] OBB annotation format specification: Each annotation file contains a category identifier 0 (representing a puncture needle) and the normalized coordinates of the four vertices of the oriented bounding box arranged in counterclockwise order. Annotation example: 0 0.4005 0.3301 0.6907 0.2489 0.6866 0.2261 0.3964 0.3072, where the meaning of each parameter is as follows: 0 is the category ID (puncture needle), followed by the normalized coordinates of the four vertices arranged in a counterclockwise direction, forming an oriented bounding box polygon that accurately describes the direction and position information of the puncture needle;

[0108] 2.2 Design of a seven-class anatomical structure segmentation dataset:

[0109] Dataset size and distribution: Total number of images: 5960, training set: 4768 images (80%), validation set: 1192 images (20%); Distribution of samples by category: arteries: 2949 cases, veins: 1872 cases, lymph nodes: 4790 cases, bones: 3496 cases, lesions: 5409 cases, esophagus: 2104 cases, diaphragm: 1355 cases;

[0110] X-AnyLabeling segmentation and annotation process: Pixel-level precise annotation standards are adopted. A conservative annotation strategy is used to avoid over-annotation in areas with blurred boundaries. Overlapping areas are annotated according to medical priorities. The annotation is carried out by three senior physicians to reduce subjective differences in annotation.

[0111] YOLO segmentation format output: The annotation output adopts the YOLO segmentation format, which includes a category ID to identify one of the seven anatomical structures, and a normalized polygon vertex coordinate sequence of multiple vertex coordinate pairs arranged in order. Each vertex contains normalized x and y coordinates. The annotation example format is category ID x1 y1 x2 y2 x3 y3 ... xn yn. All coordinate values ​​are normalized relative to the image size and range from 0 to 1.

[0112] 2.3 Medical-Specific Data Augmentation Strategies:

[0113] Enhancement configuration tailored to the characteristics of ultrasound images: Geometric transformation parameters include rotation angle ±5 degrees (to maintain correct anatomical orientation), translation ratio 5% (to reduce displacement amplitude), scaling range 20% (to adapt to different patient body types), shear angle ±2 degrees (for slight deformation), and fluoroscopic transformation 0.0001 (for minimal fluoroscopic distortion).

[0114] Color space adjustment: Hue adjustment 1% (ultrasound images are mainly grayscale), saturation adjustment 30%, brightness adjustment 20% (to adapt to different device gain settings);

[0115] Ultrasonic-specific noise simulation: Base noise level 5%, speckle noise enabled (simulating ultrasound-specific artifacts), gain change 15% (simulating equipment parameter changes), probe shadow probability 10% (simulating probe occlusion effect).

[0116] Spatial enhancement strategies: mosaic enhancement 80% probability, mixed enhancement 10% probability (reduced to maintain the realism of medical images), copy and paste 5% probability (only used for lesion target enhancement);

[0117] Example 3: Detailed implementation of key training parameters and loss function design:

[0118] 3.1 Training hyperparameter configuration:

[0119] Basic training parameter settings: 300 training rounds (medical applications require more thorough training), batch size 32 (to balance GPU memory and training stability), input image size 640×640 pixels, early stop patience value 50 rounds (to increase patience to avoid stopping too early), checkpoint save interval save every 10 rounds;

[0120] Learning rate scheduling strategy: Initial learning rate 0.001 (reduced to improve training stability), final learning rate ratio 0.01, SGD momentum 0.937, weight decay 0.0005 (L2 regularization coefficient), 5 warm-up rounds (adding a warm-up phase), warm-up momentum 0.8, cosine learning rate scheduling enabled;

[0121] Optimizer selection: Optimizer type AdamW (more suitable for medical image features), mixed precision training enabled, gradient cropping enabled and threshold set to 10.0;

[0122] 3.2 Implementation of Independent Training Mode:

[0123] The training strategy employs a two-stage independent approach: The first stage involves training the OBB (Obstruction Bypass) detection model for puncture needles, lasting 200 epochs. The dedicated dataset is the puncture needle detection dataset (3230 images), and the loss function is the OBB-specific loss function. Validation metrics include metrics / precision(B), metrics / recall(B), metrics / mAP50(B), and metrics / mAP50-95(B). The second stage involves training the anatomical structure segmentation model, lasting 300 epochs. The dedicated dataset is the anatomical structure segmentation dataset (5960 images), and the loss function is the medical segmentation-specific loss function. Validation metrics include metrics / precision(B), metrics / recall(B), metrics / mAP50(B), metrics / mAP50-95(B), metrics / precision(M), metrics / recall(M), metrics / mAP50(M), and metrics / mAP50-95(M).

[0124] Independence guarantee mechanism: The two models use different weights to initialize the seeds, the training process is completely separated with no parameter sharing, and each has its own independent verification and testing process, and saves the optimal model weights separately;

[0125] 3.3 Detailed Design of Loss Function:

[0126] 3.3.1 Loss function for OBB detection of puncture needle:

[0127] The overall loss function is: L_total = λ_box × L_box + λ_cls × L_cls + λ_dfl × L_dfl, where the weight parameters include λ_box (boundary box loss weight set to 7.5), λ_cls (classification loss weight set to 0.5), and λ_dfl (distribution focus loss weight set to 1.5).

[0128] The regression loss for rotated bounding boxes uses probabilistic IoU loss: L_box=1-ProbIoU(pred_rboxes,target_rboxes), where ProbIoU is the probabilistic intersection-union ratio that specifically handles the overlap calculation of rotated bounding boxes and can effectively handle angle information without a separate angle loss term;

[0129] The classification loss uses binary cross-entropy loss: L_cls=BCEWithLogitsLoss(pred_scores,target_scores);

[0130] The distributed focus loss uses DFL loss to improve boundary localization accuracy: L_dfl=DFLoss(pred_dist,target_dist), where DFLoss transforms continuous bounding box regression into a discrete distribution learning problem to improve localization accuracy;

[0131] 3.3.2 Anatomical Structure Segmentation Loss Function:

[0132] The overall loss function is composed of: L_total = λ_box × L_box + λ_seg × L_seg + λ_cls × L_cls + λ_dfl × L_dfl, where the weight parameters include λ_box as the bounding box loss weight of 7.5, λ_seg as the segmentation loss weight of 7.5 (same as box), λ_cls as the classification loss weight of 0.5, and λ_dfl as the distribution focus loss weight of 1.5;

[0133] The segmentation mask loss is calculated using binary cross-entropy loss to determine the mask prediction error: L_seg=BCEWithLogitsLoss(pred_masks, gt_masks), where pred_masks is obtained by multiplying the prototype mask and coefficients: pred_masks=torch.einsum('in,nhw->ihw',pred_coeffs, proto_masks);

[0134] Other loss components: The calculation methods for bounding box loss, classification loss, and DFL loss are the same as those for the detection task, ensuring consistency between detection and segmentation tasks;

[0135] 3.4 Verification indicator system:

[0136] Validation metrics for OBB detection of puncture needles: Training process monitoring metrics / precision(B) bounding box precision represents the proportion of true positives among those predicted as positives; metrics / recall(B) bounding box recall represents the proportion of true positives correctly predicted; metrics / mAP50(B) average precision with an IoU threshold of 0.5; metrics / mAP50-95(B) average precision with an IoU threshold from 0.5 to 0.95 (step size 0.05). These metrics are calculated using the ap_per_class function based on statistics of true positives, confidence, predicted class, and target class.

[0137] Anatomical segmentation validation metrics: Training process monitoring bounding box metrics (labeled B) include metrics / precision (B) bounding box precision, metrics / recall (B) bounding box recall, metrics / mAP50 (B) bounding box mAP at IoU=0.5, and metrics / mAP50-95 (B) bounding box mAP at IoU=0.5:0.95; Segmentation mask metrics (labeled M) include metrics / precision (M) mask precision, metrics / recall (M) mask recall, metrics / mAP50 (M) mask mAP at IoU=0.5, and metrics / mAP50-95 (M) mask mAP at IoU=0.5:0.95. All metrics are calculated using standard methods of the Metric class to ensure consistency and comparability of the evaluation.

[0138] Example 4: System Overall Architecture and Component Interaction Design:

[0139] In ultrasound-guided puncture needle navigation detection systems, a single YOLO detection model is often used. This model has the problem of not being able to simultaneously detect directional puncture needle structures and segmented anatomical targets with high precision. In particular, traditional bounding box detection cannot accurately capture the direction of the puncture needle, which makes the trajectory calculation error easy. Furthermore, existing segmentation models cannot distinguish between multiple types of anatomical structures in real time, which poses a safety hazard and makes it difficult to achieve accurate positioning, diagnosis or identification of puncture needles.

[0140] The relevant technical parameters (based on the actual configuration of the project code) include:

[0141] - Dual-model input unified specification: 640×640 pixel standardized input (supports automatic detection and adaptation of dynamic model size through the _get_model_input_size method);

[0142] - Confidence threshold: 0.25, set in detector initialization and main program parameter configuration, specifically optimized for low-contrast features in medical imaging;

[0143] - Cross-Union Ratio (IoU) threshold: 0.45, set in the detector parameter configuration to balance detection accuracy and repetition suppression;

[0144] - Segmentation binarization threshold: 0.5, set in the detector segmentation parameter configuration to ensure accurate segmentation of anatomical structure boundaries;

[0145] - Global pixel pitch: 0.208998 mm / pixel.

[0146] Pixel spacing parameters can be configured in the main configuration file and in each functional module.

[0147] Detailed output control includes:

[0148] 1. Control of the level of detail in test results:

[0149] -Basic mode: Outputs only bounding box coordinates and confidence level;

[0150] - Detailed mode: Outputs complete detection information, including:

[0151] - The precise coordinates of the detection box (x, y, width, height);

[0152] - Confidence score for each detected target;

[0153] - Category labels and category probability distribution;

[0154] - Rotation bounding box angle information (for OBB models);

[0155] -Pixel-level information of the segmentation mask (for the SEG model);

[0156] 2. Information output control during the reasoning process:

[0157] - Timestamp records: Time information for processing each frame;

[0158] -Performance metrics: inference time, memory usage;

[0159] -Model status: GPU utilization, model loading status;

[0160] - Error diagnosis: Detailed error messages when inference fails;

[0161] 3. Medical-specific output control:

[0162] For the specific needs of ultrasonic navigation systems:

[0163] - Anatomical structure identification: Detailed classification of anatomical structures (arteries, veins, esophagus, diaphragm, lesions, lymph nodes, bones);

[0164] - Risk level information: Medical risk classification for each detection target;

[0165] - Spatial relationship description: the distance and relative positional relationship between targets;

[0166] - Puncture suitability assessment: Feasibility analysis of the puncture procedure;

[0167] 4. Debugging and development support outputs:

[0168] - Intermediate Feature Map: Visualization of features at each layer of the network;

[0169] - Confidence heatmap: Detects the spatial distribution of confidence levels;

[0170] - Anchor box information: Candidate box generation and filtering process;

[0171] - Post-processing details: Detailed steps of the NMS (Non-maximum suppression) process;

[0172] 5. Control of Clinical Record Output:

[0173] - Operation log: Complete record of detection sequences;

[0174] - Keyframe markers: timestamps of important medical events;

[0175] -Quality assessment: Image quality and detection reliability scores;

[0176] -DICOM compatible output: Metadata conforming to medical imaging standards;

[0177] Technical implementation level:

[0178] In the inference parameters of the YOLO model:

[0179] -verbose=True / False: Controls the level of verbosity in the terminal output;

[0180] -save_txt=True: Saves the test results to a text file;

[0181] -save_conf=True: Include confidence information in the output;

[0182] -save_crop=True: Saves the cropped image of the detected target;

[0183] -show_labels=True: Display label information in the visualization;

[0184] -show_conf=True: Display confidence levels in the visualization;

[0185] In medical navigation systems, detailed output control ensures that the system can: provide clinicians with complete diagnostic information, support post-operative medical analysis and quality control, facilitate system debugging and performance optimization, meet medical record and auditing requirements, and support continuous algorithm improvement and validation.

[0186] Therefore, detailed output control is a comprehensive parameter setting mechanism that determines the richness and format of the AI ​​detection model's output information to meet the specific needs of different application scenarios (real-time navigation, offline analysis, clinical recording, system debugging).

[0187] Specific implementation examples:

[0188] Assuming the input ultrasound image size is 1920×1080 pixels, the system processing flow is as follows:

[0189] 1. Image preprocessing: Scale the 1920×1080 image to a standard 640×640 input;

[0190] 2. Dual-model parallel inference: The OBB model and the SEG model process the normalized image simultaneously;

[0191] 3. Confidence Score Screening: Detection results with a confidence score > 0.25 are retained. The calculation formula is as follows:

[0192] Conf_valid={detection|conf(detection)>0.25};

[0193] in:

[0194] Conf_valid: The set of valid detection results;

[0195] detection: A single detection result object;

[0196] conf(detection): The confidence score of the detection result, with a value range of [0,1].

[0197] 0.25: Confidence threshold; detection results below this value are discarded.

[0198] 4. Non-maximum suppression: An IoU threshold of 0.45 eliminates duplicate detections. The calculation formula is as follows:

[0199] IoU=Area_intersection / Area_union;

[0200] in:

[0201] -IoU: Intersection over Union (IoU), used to measure the degree of overlap between two bounding boxes;

[0202] -Area_intersection: The area of ​​the intersection region between the two bounding boxes;

[0203] -Area_union: The area of ​​the union region of the two bounding boxes;

[0204] - When IoU > 0.45, retain the detection boxes with higher confidence and suppress duplicate detection;

[0205] 5. Post-segmentation processing: Apply a threshold of 0.5 to generate a binary mask:

[0206] Mask_binary(x,y)={1,if prob(x,y)>0.5;0,otherwise};

[0207] in:

[0208] -Mask_binary(x,y): The binary mask value at pixel coordinates (x,y);

[0209] prob(x,y): The probability value that the pixel coordinate (x,y) belongs to the target category, with a value range of [0,1].

[0210] 0.5: Segmentation threshold; pixels exceeding this value are marked as target regions.

[0211] 6. Result Restoration: Restore the 640×640 detection result to the original size of 1920×1080.

[0212] In a further optimization scheme, the detection system integrates TensorRT engine optimization technology, automatically determining whether to enable hardware acceleration based on the detected model file format. When a TensorRT-optimized OBB model is detected, the system automatically loads the GPU-accelerated orientation detection engine; when an optimized segmentation model is detected, the corresponding segmentation acceleration engine is activated, which can improve inference speed.

[0213] The detection system can accurately detect and diagnose seven key anatomical structures. The segmentation model establishes a hierarchical safety system based on medical priorities, with class mapping consistent with the classes7-seg.yaml training configuration. The correspondence between the seven anatomical structures is as follows:

[0214] Category 0: Artery, Level 1 Contraindication Area, Critical Risk Threshold 5.0 mm, Warning Threshold 8.0 mm, Priority 9 (Highest Risk) (defined in the Anatomical Structure Risk Configuration Module).

[0215] Category 1: Vein, Level 2 contraindication area, critical risk threshold 3.0 mm, warning threshold 5.0 mm, priority 7 (defined in the anatomical structure risk configuration module).

[0216] Category 2: Lymph node, target area (puncture target), critical risk threshold 0.0 mm, warning threshold 0.0 mm, priority 1 (treatment target) (defined in the anatomical structure risk configuration module);

[0217] Category 3: Bone, Barrier Area (Physical Obstacle), Critical Risk Threshold 0.0mm (Absolutely No Penetration), Warning Threshold 1.0mm, Priority 8 (defined in the Anatomical Structure Risk Configuration Module);

[0218] Category 4: Lesion / mass, Target area (puncture target), Critical risk threshold 0.0 mm, Warning threshold 0.0 mm, Priority 1 (Primary treatment target) (defined in the Anatomical Structure Risk Configuration Module).

[0219] Category 5: Esophagus, secondary contraindication area, critical risk threshold 3.0 mm, warning threshold 5.0 mm, priority 7 (defined in the anatomical structure risk configuration module).

[0220] Category 6: Diaphragm, Level 2 contraindication area, critical risk threshold 2.0 mm, warning threshold 4.0 mm, priority 6 (defined in the anatomical structure risk configuration module).

[0221] Specific implementation examples of segmentation algorithms for segmentation models:

[0222] Taking the detection of arterial structures using 1920×1080 ultrasound images as an example:

[0223] 1. Input processing: The original image is scaled to 640×640 pixels;

[0224] 2. Segmentation reasoning: YOLOv11-SEG outputs a 640×640×7 multi-class probability graph;

[0225] 3. Thresholding: Apply a threshold of 0.5 to the artery category (class_id=0):

[0226] Artery_mask(x,y)={1, if P_artery(x,y)>0.5;0,otherwise};

[0227] in:

[0228] Artery_mask(x,y): The binary mask of the artery at pixel coordinates (x,y);

[0229] P_artery(x,y): The probability value that the pixel coordinate (x,y) belongs to the artery category;

[0230] 0.5: Segmentation threshold to ensure clear boundaries;

[0231] 4. Geometric Analysis: Calculate the contour area and equivalent radius:

[0232] Area=Σ(Artery_mask(x,y)=1);

[0233] R_equiv=√(Area / π)×pixel_spacing;

[0234] in:

[0235] Area: Total number of pixels in the arterial region;

[0236] R_equiv: Equivalent radius, in millimeters;

[0237] π: Pi, approximately equal to 3.14159;

[0238] pixel_spacing: pixel spacing, 0.208998 mm / pixel;

[0239] 5. Risk parameter settings:

[0240] Risk_radius=R_equiv+Safety_margin;

[0241] in:

[0242] Risk_radius: Arterial risk radius;

[0243] Safety_margin: Safety margin distance, 8.0mm for arteries.

[0244] Mask extraction and size adaptation algorithm: The system first verifies the validity of the segmentation output; then, it iterates through the segmentation results according to the anatomical structure category, extracts the mask for each valid instance under each category, and simultaneously calculates / outputs the geometric contour and center coordinates (or any combination thereof) of the instance; after the mask data is transferred from GPU memory to CPU, it is uniformly binarized using a preset segmentation threshold (e.g., 0.5), converting floating-point probability values ​​into integer binary masks; when the mask size does not match the target frame size, the system uses the nearest neighbor interpolation algorithm to adjust the size to maintain the sharpness and accuracy of the segmentation boundary; through the above extraction strategy by category and by instance, the system can output the spatial information of all anatomical structure objects, providing complete input for subsequent risk assessment and path planning;

[0245] The navigation system also establishes a mathematical model of the ejection process through an ejection trajectory calculation function, including:

[0246] Physical coordinate transformation converts the needle tip position and target center from the pixel coordinate system to the physical coordinate system. The transformation ratio is determined by the pixel spacing parameter (default 0.208998mm / pixel). Then, the direction vector of the puncture needle is calculated based on the needle angle, converting the angle from degrees to radians, and the unit vector components in the x and y directions are calculated through trigonometric functions.

[0247] Assuming an input ultrasound image of 1920×1080, a needle tip position of (320, 240) pixels, a needle angle of 45°, a target center of (400, 300) pixels, and a standard ejection distance of 22.0 mm:

[0248] Physical coordinate transformation:

[0249] - Needle tip physical coordinates: (320×0.208998, 240×0.208998) = (66.88mm, 50.16mm);

[0250] - Target physical coordinates: (400×0.208998, 300×0.208998) = (83.60mm, 62.70mm).

[0251] The direction vector is calculated using the following formulas: angle_rad = needle_angle × π / 180; direction_x = cos(angle_rad); direction_y = sin(angle_rad), where angle_rad is the angle of the needle in radians, needle_angle is the angle of the needle, π is pi, direction_x is the unit vector component in the x-direction, direction_y is the unit vector component in the y-direction, and cos and sin are trigonometric functions used for converting angles to direction vectors.

[0252] The calculation formula for the ejection endpoint is: fire_end_x = tip_x + direction_x × throw_distance; fire_end_y = tip_y + direction_y × throw_distance, where fire_end_x, fire_end_y are the physical coordinates of the ejection endpoint, tip_x, tip_y are the physical coordinates of the starting position of the needle tip, and throw_distance is the ejection distance; the specific calculation is: fire_end_x = 66.88 + 0.7071 × 22.0 = 82.44 mm, fire_end_y = 50.16 + 0.7071 × 22.0 = 65.72 mm.

[0253] The target distance is calculated using the formula: distance_to_target = √[(target_x - tip_x)² + (target_y - tip_y)²], where distance_to_target is the straight-line distance from the tip to the center of the target, and target_x and target_y are the physical coordinates of the target center. Specifically, distance_to_target = √[(83.60 - 66.88)² + (62.70 - 50.16)²] = √[279.22 + 157.35] = 20.89 mm.

[0254] Based on the accuracy analysis of the actual parameters in the code: the coordinate transformation error is ±0.075mm, and the trigonometric function calculation accuracy in the direction vector is ±1×10⁻¹. 5 (Double-precision floating-point precision);

[0255] Overall accuracy error analysis, sources of cumulative error:

[0256] 1. Pixel quantization error: ±0.075mm;

[0257] 2. Floating-point operation error: ±1×10⁻¹ 5 mm (negligible);

[0258] 3. Angle discretization error: The upper limit of the lateral offset at the endpoint is Δ≈D·sin(Δθ). For example, when the launch distance D=22.0mm and the angular offset step length Δθ=1°, the lateral discretization error at the endpoint is approximately 22·sin(1°)≈0.38mm;

[0259] 4. Launch distance setting error: ±0.1mm (equipment mechanical precision);

[0260] Overall theoretical error: √(0.075²+0.38²+0.1²)≈±0.40mm (This mainly comes from the angle step size; it can be further reduced by using a finer angle step size or continuous angle input).

[0261] Improvements over traditional approximation methods:

[0262] Error sources in traditional methods:

[0263] 1. Approximate value of π (3.14159): Error ~10⁻ 5 ;

[0264] 2. Angle table lookup: error ~1°;

[0265] 3. Single-precision floating-point: Precision ~7 significant digits;

[0266] The improvement method adopted in this system is as follows:

[0267] 1. Use the math.pi constant: improve calculation accuracy;

[0268] 2. Real-time angle calculation: Eliminates discretization error;

[0269] 3. Double precision operation: Improves the accuracy of numerical calculations.

[0270] The system calculates the straight-line distance from the needle tip to the target center, serving as a fundamental parameter for evaluating ejection effectiveness. Through precise mathematical modeling and high-precision numerical calculations, the system achieves millimeter-level trajectory calculation accuracy, providing a reliable mathematical foundation for CNB ejection operations.

[0271] In one embodiment, navigation path planning is performed using an enhanced A* (MedicalAStar) algorithm with medical constraints. During system initialization, a medical grid space is established as the computational basis for the A* (MedicalAStar) algorithm. The continuous image space is discretized into a searchable network of nodes through gridding. Pixel spacing parameters (globally configured by default 0.208998 mm / pixel, obtained from DICOM metadata, supports system-level configuration adjustments, and can be configured to other values ​​such as 0.15 to adapt to different devices) are set for physical-pixel coordinate transformation. The medical constraint parameter system includes:

[0272] For angle variation limitations, it is recommended to set the maximum angle variation (comfort / operability limit) to 30.0 degrees and the clinical limit angle threshold to 45.0 degrees. In the medical constraint parameter configuration, it is defined that: if the angle variation exceeds 30.0 degrees, searching is still allowed but the penalty intensity increases sharply; when the angle variation is ≥45.0 degrees, an infinite penalty value is set to prevent the path from expanding.

[0273] Step size control, with a minimum step size of 1 grid unit and a maximum step size of 3 grid units, is configured in the medical constraint parameters to ensure the operability of the navigation path;

[0274] The grid resolution in the optimized navigator has been adjusted to 4 pixels per grid cell to balance computational accuracy and processing efficiency.

[0275] In terms of connectivity, the medical A* algorithm module employs 8-connected neighborhood search, providing path selection in 8 directions;

[0276] The weighting strategy, with a risk weight factor of 5.0 and a path smoothing weight factor of 2.0, is set in the medical constraint parameters to achieve a balance between safety and operability.

[0277] The Medical Grid space is a discretized computational grid space that implements the MedicalGrid class to divide ultrasound images into regular grids. Its function is to serve as a data structure for the A* path search algorithm. The coordinate system is a grid coordinate system (grid_x, grid_y). The resolution is a configurable grid resolution (default 4 pixels / grid).

[0278] In one possible implementation, a continuous risk field is constructed in the pixel space of the original ultrasound image, and then pixel-level risk values ​​are aggregated to grid nodes through spatial mapping to realize the transformation from the continuous risk field to the discrete search space. The resulting risk field then provides guidance for the navigation path planning algorithm.

[0279] The pixel space, also known as the image space (Risk Field image space), is a continuous pixel-level risk field space. It implements the risk_field array, which corresponds one-to-one with the pixels of the original ultrasound image. Its function is to store the risk value of each pixel (based on the distance to the anatomical structure). The coordinate system is the pixel coordinate system (pixel_x, pixel_y). The resolution is the same as that of the original ultrasound image (1:1 pixel correspondence).

[0280] The relationship and transformation between grid space and image space are as follows:

[0281] Image space -> Mesh space:

[0282] grid_x, grid_y=self.grid.pixel_to_grid(pixel_x, pixel_y);

[0283] Grid space -> Image space:

[0284] pixel_x, pixel_y=self.grid.grid_to_pixel(grid_x, grid_y);

[0285] Risk field mapped to grid:

[0286] def set_risk_field(self, risk_field: np.ndarray):

[0287] Mapping continuous risk field values ​​to discrete grid points:

[0288] risk_region=risk_field[pixel_y_start:pixel_y_end, pixel_x_start:pixel_x_end];

[0289] avg_risk=np.mean(risk_region) # Get the average value of the region;

[0290] self.grid[x][y].risk_value=avg_risk.

[0291] In simple terms, image space is a continuous risk field at the pixel level, while grid space is a discrete computational grid used for path search. The two are linked through coordinate transformation and value mapping.

[0292] More preferably, the detection system first creates a zero-value-initialized floating-point risk field matrix based on the input image size, and then constructs boundary risk regions and anatomical structure risk regions in layers;

[0293] The boundary risk area is defined as follows: The system sets a standardized boundary risk value of 100.0 at the image edge. Specifically, boundary constraints are established within 5 pixels of the top edge, bottom edge, left edge, and right edge of the image. This is implemented in the medical grid risk field construction module to prevent the planned path from exceeding the effective imaging area.

[0294] The anatomical structure risk region is defined as follows: For each detected anatomical structure object, the system extracts the segmentation mask or geometric contour, center coordinates (for annotation and indexing), critical risk threshold D_crit, warning threshold D_warn, and avoidance priority parameters for each anatomical structure object. The risk zone distance is uniformly calculated using the boundary safety distance d_boundary (which can be obtained by distance transformation of the mask boundary), not the center Euclidean distance. An equidistant hierarchical risk model (equidistant risk bands based on the boundary safety distance d_boundary) is adopted to establish a two-level risk region system, where the high-risk zone (distance < critical threshold) is assigned an extremely high risk value of 1000.0 × priority, and the warning zone (critical threshold ≤ distance < warning threshold) is assigned a medium risk value of 200.0 × priority.

[0295] Specific implementation examples of risk field construction algorithms:

[0296] Taking the detection of arterial structures in a 1920×1080 ultrasound image as an example (center coordinates (500, 400), critical threshold 5.0 mm, warning threshold 8.0 mm, priority 9):

[0297] 1. Physical to pixel conversion:

[0298] The critical threshold pixel = 5.0mm ÷ 0.208998mm / pixel ≈ 23.9 pixels;

[0299] Warning threshold pixels = 8.0mm ÷ 0.208998mm / pixel ≈ 38.3 pixels;

[0300] 2. Tiered risk calculation:

[0301] High-risk area: Distance < critical threshold, risk value = 1000.0 × 9 = 9000.0;

[0302] Warning zone: Critical threshold ≤ Distance < Warning threshold, Risk value = 200.0 × 9 = 1800.0;

[0303] 3. Distance calculation (using boundary safety distance as the standard):

[0304] The system extracts the geometric boundary of the artery based on the artery segmentation mask and calculates the minimum signed distance d_boundary(x,y) from any pixel (x,y) to the boundary.

[0305] Where d_boundary(x,y)>0 indicates that the pixel is located outside the boundary, d_boundary(x,y)=0 indicates that it is located on the boundary, and d_boundary(x,y)<0 indicates that it is located inside the boundary.

[0306] When constructing a risk field for pixel-level cost display, d_display(x,y)=max(0,d_boundary(x,y)) can be used as the display distance, but threshold comparison and risk judgment are based on d_boundary.

[0307] 4. Risk Allocation:

[0308] When d_boundary(x,y) < the critical threshold, the pixel (x,y) belongs to the high-risk area and is assigned a value of 9000.0;

[0309] When the critical threshold ≤ d_boundary(x,y) < warning threshold, the pixel (x,y) belongs to the warning zone and is assigned a value of 1800.0;

[0310] When d_boundary(x,y) ≥ the warning threshold, the risk value remains at 0.0.

[0311] 5. Obstacle marking (resolving soft / hard constraint conflicts):

[0312] The system determines obstacles based on a combination of anatomical structure type and risk value. Areas with a risk value exceeding the preset obstacle threshold T_obs and that are not treatment target structures are marked as obstacles in the medical grid system.

[0313] Since the treatment target structure (lesion, lymph node) has a priority of 1, its warning area risk value (200.0×1=200) is lower than the obstacle threshold, so it will not be incorrectly marked as an obstacle, ensuring that the path can be correctly planned to the treatment target;

[0314] Wherein, the obstacle threshold T_obs is a configurable parameter, preferably with dimensions and magnitude consistent with the risk value system of this specification (boundary risk 100, warning zone 200×priority, high-risk zone 1000×priority), and can be determined in the following manner:

[0315] T_obs = β × (200.0 × p_target), where p_target is the priority of the target structure (p_target = 1 in this embodiment), and β is a scaling factor greater than 1 (preferably 2.0 to 5.0, more preferably β = 2.5, corresponding to T_obs = 500).

[0316] This design allows for the simultaneous adjustment of β or direct configuration of T_obs when priority settings, risk benchmarks, or clinical safety strategies are adjusted, thereby maintaining consistency in the access decision strategy of "target tissues are accessible, while non-target high-risk areas are inaccessible".

[0317] 6. Mesh mapping:

[0318] In the optimized version of Navigator, the risk field is mapped to the grid system in 4×4 pixel blocks.

[0319] In one possible implementation, a movement cost architecture is integrated into the navigation path planning process. The detection system establishes a three-layer cost structure, namely enhanced movement cost, which includes basic movement cost, risk cost, and smoothness cost. The basic movement cost calculates the geometric distance from the current grid point to adjacent grid points using the Euclidean distance formula to ensure the spatial efficiency of the path. The risk cost is calculated by multiplying the risk value of adjacent points by the risk weight factor in the medical constraints and adjusting the value using a standardization coefficient of 0.001. The smoothness cost is based on the angle change of the navigation path segment. When the current grid point has a parent node, the smoothness cost calculation algorithm calculates the angle change of adjacent path segments: the cosine value of the included angle is obtained by the vector dot product and the vector magnitude, and then the inverse cosine function is applied to obtain the angle change and convert it from radians to degrees. The system evaluates all angle changes: negative rewards are given when the angle change is ≤10 degrees to encourage smooth paths; linearly increasing penalties are applied when the angle change is between 10-20 degrees; quadratic penalties are applied when the angle change is between 20-30 degrees; and the penalty intensity increases sharply when the angle change is >30 degrees, applying cubic or infinite penalties to reinforce the cost of uncomfortable turning. The weighting factors in the three-layer cost structure are optimized based on clinical practice: a risk weighting factor of 5.0 ensures that the algorithm prioritizes avoiding high-risk areas, based on the highest priority principle of medical safety; a standardization coefficient of 0.001 is used to unify the numerical range and ensure the reasonable superposition of costs of different dimensions; a smoothness weighting factor of 2.0 is based on the smoothness requirements of puncture operation to ensure that the path is within the clinically operable range; and an adjustment coefficient of 0.1 balances the weights of each cost component to avoid a single factor dominating path selection.

[0320] The enhanced mobility cost calculation formula is: total_cost = base_cost + risk_cost + smoothness_cost;

[0321] Where, base_cost: basic movement cost, with a diagonal movement cost of √2≈1.414;

[0322] risk_cost=neighbor.risk_value×risk_weight×scale_factor,

[0323] Where risk_cost is the risk cost, neighbor.risk_value is the risk value of the neighbor node, risk_weight is the risk weight factor, and scale_factor is the scale factor.

[0324] smoothness_cost: Smoothness cost, calculated for all angle changes; when the angle change is greater than 30 degrees, an additional penalty term or penalty coefficient transition is triggered. The calculation formula is: smoothness_cost = angle_change × smoothness_weight × penalty_factor, where angle_change: the amount of change in path turning angle (degrees), smoothness_weight: smoothness weight factor, and penalty_factor: penalty factor.

[0325] Basic moving cost calculation example:

[0326] When evaluating a move from grid point A (x=10, y=15) to adjacent grid point B (x=11, y=16), the geometric distance is calculated using the Euclidean distance formula. Specifically, the base movement cost is calculated as follows: Base movement cost = √[(11-10)² + (16-15)²] = √[1² + 1²] = √2 ≈ 1.414. For orthogonal movements (e.g., from A to C (x=11, y=15)), the base cost is 1.0; for diagonal movements, the base cost is 1.414, reflecting the additional distance cost of diagonal movement.

[0327] Example of risk cost calculation:

[0328] Assuming target grid point B is located in the artery warning zone and the artery avoidance priority is 9, then the risk value of this node is 200.0 × 9 = 1800.0; the risk weight factor in the medical constraints is set to 5.0, and the scale factor is set to 0.001, then the risk cost is calculated as follows:

[0329] Risk cost = 1800.0 × 5.0 × 0.001 = 9.0.

[0330] This value represents the additional path cost incurred due to proximity to high-risk anatomical structures, guiding the algorithm to prioritize avoiding dangerous areas.

[0331] Example of comprehensive cost calculation:

[0332] Combining the two components mentioned above, the total movement cost from grid point A to grid point B is: Total cost = Basic movement cost + Risk cost = 1.414 + 9.0 = 10.414.

[0333] In contrast, the cost of moving to a high-risk area of ​​the arteries (risk value of 1000.0 × 9 = 9000.0) is: 1.414 + (9000.0 × 5.0 × 0.001) = 1.414 + 45.0 = 46.414;

[0334] The cost of moving to the warning zone of the treatment target structure (priority 1, risk value 200.0) is: 1.414 + (200.0 × 5.0 × 0.001) = 1.414 + 1.0 = 2.414;

[0335] The cost of moving to a non-risk area (risk value of 0) is only 1.414 + (0 × 5.0 × 0.001) = 1.414, which reflects the algorithm's preference for safe paths.

[0336] In one possible implementation, an angle change penalty mechanism is also integrated into the navigation path planning process. The detection system establishes an angle change threshold mechanism based on medical constraints. When the angle change between path segments exceeds the preset recommended maximum angle change limit (30 degrees), the angle change penalty strategy algorithm applies the angle change penalty. The penalty intensity is determined by the product of the angle change amount and the smoothness weight factor, and a coefficient of 0.1 is applied for numerical optimization.

[0337] The angle change penalty strategy algorithm includes: constructing a five-level medical safety threshold system, establishing a graded penalty mechanism based on the physical limitations of clinical puncture operations, and setting five assessment levels according to the medical safety level of the angle change: comfortable operation range (0-10 degrees), clinically acceptable range (10-20 degrees), high-risk operation range (20-30 degrees), extremely dangerous operation range (30-45 degrees), and technically prohibited range (≥45 degrees).

[0338] The corresponding tiered penalty strategy is as follows:

[0339] Comfortable operating range processing: When the angle change is within the ideal operating range, the system gives a negative reward factor to actively encourage the generation of smooth paths;

[0340] Clinically acceptable range treatment: A linearly increasing function is used, and the intensity of the penalty is proportional to the angle that exceeds the comfort range;

[0341] Handling of high-risk operation ranges: Implement a power-law penalty mode, with the penalty value increasing by a power of two based on the angle deviation;

[0342] Handling of extremely dangerous operating ranges: Implement a power-law-increasing penalty mode, with the penalty value increasing by a power of three based on the angle deviation;

[0343] Technically prohibited interval handling: Set an infinite penalty value.

[0344] Example of an angle change penalty strategy: Comfort range penalty example: When the angle change is 5°, the system determines that it is within the comfortable operating range (0°-10°) and gives a negative reward: Penalty value = -0.5 × adjustment coefficient 0.1 = -0.05. The negative value indicates a positive encouragement for a smooth path, effectively guiding the algorithm to generate a clinically appropriate path.

[0345] To introduce the aforementioned negative rewards in A*-based path search without compromising convergence and optimality, the system adopts a search strategy that allows negative edge weights and performs potential-based shaping on the cost function: a potential function Φ(·) is introduced into the search node, transforming the edge cost c(u,v) into c'(u,v)=c(u,v)+Φ(u)-Φ(v), thereby ensuring that the search process satisfies the preset lower limit constraint of cumulative cost while maintaining the path ranking relationship.

[0346] Example of acceptable linear penalty: When the angle changes by 15°, it exceeds the comfort range by 5°. The system applies a linear penalty: Exceeding angle = 15° - 10° = 5°; Penalty value = 5 × 2.0 (smoothness weight) × 0.1 × 0.5 = 0.5;

[0347] The penalty value is moderate, allowing the path to proceed but increasing the corresponding cost.

[0348] Example of high-risk range power penalty: When the angle changes by 25°, the system enters a high-risk operating range.

[0349] Basic penalty = (20° - 10°) × 2.0 × 0.1 × 0.5 = 1.0; Second power penalty = (25° - 20°)² × 2.0 × 0.1 × 2.0 = 5² × 0.4 = 10.0; Total penalty value = 1.0 + 10.0 = 11.0;

[0350] Extremely dangerous range (30°-45°), base penalty = cumulative penalty of preceding range;

[0351] Cubic power penalty = (angle change - 30°)³ × smoothness weight × adjustment coefficient × 10.0;

[0352] Forbidden range (≥45°), penalty value = ∞ (complete blockage).

[0353] A substantial penalty value can effectively prevent the algorithm from choosing a high-risk path.

[0354] Medical safety correction example: In addition to the above 25° angle change, the system further considers medical constraints:

[0355] - Deep puncture correction: (25-15°)×0.3×1.2=3.6;

[0356] - Correction for proximity to critical structures: (25°-12°)×0.2×1.0=2.6;

[0357] - Needle specification correction (16G): (25°-20°)×0.4×1.8=3.6;

[0358] Total medical safety correction: 3.6 + 2.6 + 3.6 = 9.8;

[0359] Example of final penalty value: Total penalty value = base angle penalty + medical safety correction = 11.0 + 9.8 = 20.8.

[0360] Medical safety correction calculation formula: A triple correction formula is implemented through a medical safety correction algorithm:

[0361] Deep puncture correction, if angle change > 15°: Deep correction = (angle change - 15°) × 0.3 × depth factor (1.2); Adjacent critical structure correction, if angle change > 12°: Adjacent correction = (angle change - 12°) × 0.2 × adjacency factor (1.0); Needle specification correction, if angle change > 20°: Specification correction = (angle change - 20°) × 0.4 × needle flexibility factor;

[0362] Needle dexterity factor calculation formula: The specification mapping formula is realized through the needle dexterity factor acquisition algorithm: Needle dexterity factor = {14G: 2.0, / / Coarse needle, difficult to turn; 16G: 1.8, 18G: 1.5, 20G: 1.2, 22G: 1.0, / / Standard reference; 25G: 0.8 / / Fine needle, relatively flexible};

[0363] Risk field mapping calculation formula: The risk value mapping formula is implemented through the risk field mapping algorithm:

[0364] Grid risk value = mean(pixel risk field[y_start:y_end,x_start:x_end]);

[0365] Where: x_start = grid x × grid resolution; y_start = grid y × grid resolution;

[0366] x_end = min(x_start + grid resolution, risk field width); y_end = min(y_start + grid resolution, risk field height);

[0367] The final enhanced A* cost function is: F(n) = G(n) + H(n); where: G(n): the actual cumulative cost function, representing the actual cumulative path cost from the starting point to the current node n, calculated as follows:

[0368] G(n) = G(parent) + move_cost(parent, n); This can be further elaborated as follows:

[0369] G(n) = Parent node G value + Enhanced movement cost = Parent node G value + (Basic movement distance + Medical risk cost + Smoothness penalty cost);

[0370] Here, the parent node G value is defined as the cumulative actual cost from the starting point s to the parent node parent, that is, G(parent) represents the total cost of the path already traversed. In particular, the starting point G value is initialized to G(s) = 0, and the G value of each subsequent node is calculated recursively.

[0371] The base distance in move_cost(parent, n) refers to the geometric distance between two adjacent grid nodes. For an 8-connected grid, the base distance for orthogonal movement is 1.0, and the base distance for diagonal movement is √2≈1.414. The calculation formula is: base_distance=√[(x_n - x_parent)²+(y_n-y_parent)²];

[0372] Where (x_parent, y_parent) and (x_n, y_n) are the grid coordinates of the parent node and the current node, respectively, and the two are adjacent (distance ≤ √2).

[0373] H(n): Heuristic cost estimation function, representing the estimated residual cost from the current node n to the target node goal. The Euclidean distance is used as the heuristic function, and the calculation formula is: H(n)=√[(x_goal-x_current)²+(y_goal-y_current)²];

[0374] Where (x_current, y_current) are the grid coordinates of the current node n, and (x_goal, y_goal) are the grid coordinates of the target node. H(n) represents the straight-line distance from the current position to the target position, used to estimate the minimum possible cost of the remaining path and guide the search towards the target.

[0375] F(n): The total estimated cost function combines the actual cumulative cost and the estimated residual cost, used for priority queue sorting in the A* algorithm: F(n) = G(n) + H(n) = actual cumulative cost + estimated residual cost. The A* algorithm improves search efficiency by prioritizing the expansion of nodes with the smallest F(n) value, ensuring the optimal path is found. This value effectively reflects the comprehensive impact of multi-dimensional medical constraints on path selection, ensuring the clinical safety and operability of the generated path.

[0376] Assume the starting point is s(0,0), the current node is n(10,10), the neighboring node is m(11,11), and the target node is goal(100,100):

[0377] 1. The G value of the current node n: G(n) represents the cumulative actual cost from the starting point (0,0) to node (10,10), assuming G(n) = 20.5;

[0378] 2. Calculate the cost of moving to neighbor node m:

[0379] - Basic movement distance: √[(11-10)²+(11-10)²]=√2≈1.414 (adjacent diagonal movement);

[0380] - Enhanced movement cost: move_cost = 1.414 + risk cost + smoothness cost, assumed to be 2.0;

[0381] - The G value of the neighboring node: G(m) = G(n) + move_cost = 20.5 + 2.0 = 22.5;

[0382] 3. Calculate the heuristic cost of neighbor node m:

[0383] - H(m)=√[(100-11)²+(100-11)²]=√(89²+89²)≈125.9 (straight-line distance to the target);

[0384] 4. Calculate the total estimated cost of neighbor node m:

[0385] - F(m)=G(m)+H(m)=22.5+125.9=148.4.

[0386] In the example above, the base moving distance of 1.414 and the heuristic distance of 125.9 describe completely different geometric relationships. The former is used to accumulate the G value, and the latter is used to estimate the H value. Together, they constitute the F value to guide the A* search.

[0387] In one embodiment, the navigation system establishes a geometric distance calculation algorithm based on parameterized straight line equations, with the calculation formula as follows:

[0388] d1 = √[(px - projection_x)² + (py - projection_y)²], where: d1 is the shortest distance from the target point to the line segment; px and py are the coordinates of the target point;

[0389] projection_x = x1 + t × (x2 - x1), projection_y = y1 + t × (y2 - y1), where x1 and y1 are the coordinates of the starting point of the line segment, x2 and y2 are the coordinates of the ending point of the line segment, and t is the projection parameter, which represents the relative position of the projection point on the line segment. t = max(0, min(1, [(px - x1) × (x2 - x1) + (py - y1) × (y2 - y1)] / line_length²)), where the value of t is in the range of 0-1. line_length is the length of the line segment, line_length = √[(x2 - x1)² + (y2 - y1)²]. max and min are the maximum and minimum value functions, respectively, to ensure that the projection point is within the range of the line segment.

[0390] With a target radius of 5.0mm, the parameters of the ejection trajectory line segment are as follows: starting point coordinates are (66.88mm, 50.16mm), ending point coordinates are (82.44mm, 65.72mm), and line segment length is: √[(82.44-66.88)²+(65.72-50.16)²]=√[242.21+242.21]=22.0mm.

[0391] Assuming the target point coordinates are (75.0mm, 60.0mm), calculate the shortest distance from it to the trajectory line segment:

[0392] 1. Calculation of projection parameters:

[0393] t=[(75.0-66.88)×(82.44-66.88)+(60.0-50.16)×(65.72-50.16)] / (22.0)²;

[0394] =[8.12×15.56+9.84×15.56] / 484.0;

[0395] =[126.35+153.11] / 484.0=279.46 / 484.0=0.577.

[0396] 2. Constraint Check:

[0397] t=max(0,min(1,0.577))=0.577 (within the range [0,1], no constraints are needed).

[0398] 3. Calculation of projection points:

[0399] projection_x=66.88+0.577×(82.44-66.88)=66.88+0.577×15.56=66.88+8.98=75.86mm;

[0400] projection_y=50.16+0.577×(65.72-50.16)=50.16+0.577×15.56=50.16+8.98=59.14mm.

[0401] 4. Shortest distance calculation:

[0402] d1=√[(75.0-75.86)²+(60.0-59.14)²]=√[0.74+0.74]=√1.48=1.22mm.

[0403] Since the distance of 1.22mm is less than the target radius of 5.0mm, the target is determined to have been hit by the trajectory. Calculating the shortest distance from the target point to the line segment ensures that the CNB needle can accurately collect pathological samples, thus assessing the clinical effectiveness of the ejection operation—that is, evaluating whether valuable pathological samples can be successfully obtained.

[0404] In addition, the boundary safety distance d_boundary is obtained from the shortest signed distance from the ejection trajectory to the boundary of the anatomical structure (negative distances are allowed, and d_boundary≤0 indicates intersection / intrusion). As a qualitative risk classification rather than a probabilistic damage prediction, it can assess the risk of the ejection trajectory to dangerous anatomical structures (such as arteries and veins). As a safety assessment, it determines whether ejection is allowed to miss any risk factors, thereby ensuring that the CNB ejection process does not accidentally damage important anatomical structures.

[0405] In one possible implementation, the system establishes a multi-dimensional safety assessment framework that includes an overall safety score and a list of key risks. The overall safety score uses a standardized scoring system from 1.0 to 0.0, where 1.0 represents the safest state and 0.0 represents the most dangerous state. Based on the status of key risks in the list of key risks, the system automatically triggers an adjustment suggestion generation process. The adjustment suggestions include adjusting the ejection distance, changing the needle insertion angle, and selecting different needle insertion points, thereby establishing a complete ejection safety control system. This allows for the deep integration of the seven types of anatomical structure recognition results from multimodal AI detection with the physical characteristics of CNB ejection, achieving intelligent ejection safety control and providing comprehensive safety assurance for clinical CNB operations.

[0406] The overall safety score is calculated based on a cumulative penalty mechanism. The base score is 1.0, representing the safest state with no risk. When a critical risk is detected, the score is multiplied by a penalty factor of 0.1; when a warning risk is detected, the score is multiplied by a penalty factor of 0.7. The calculation formula is as follows:

[0407] Overall safety score = base score × penalty;

[0408] Here, "penalty" is a penalty coefficient set according to risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safety penalty = 1.0. This cumulative penalty mechanism ensures that any critical risk will significantly reduce the overall safety score. When multiple risk sources exist simultaneously, the penalty factor is the minimum value of each risk penalty factor, thus giving the most severe risk a veto effect on the overall safety score, forming a strict safety constraint system.

[0409] Specifically, adjusting the ejection distance involves the system selecting the option with the highest overall safety score from the set of feasible ejection distances that satisfy both the hit constraint and the safety threshold constraint. When a critical risk arises (e.g., the existence of intersections / intrusions with d_boundary ≤ 0), a shorter feasible ejection distance is prioritized to reduce potential damage risk. The system evaluates the safety of each candidate distance option based on a cumulative penalty mechanism to ensure that the option with the lowest risk is selected from multiple distance options.

[0410] Example: Assume the current ejection distance is 22.0mm (standard distance), and the candidate adjustable distances are 15, 18, 22, 25, 29, and 32mm. After safety assessment, the current plan's boundary safety distance at the artery, d_boundary, is -2.0mm (intersection / intrusion, d_boundary ≤ 0), which is determined to be a critical risk. The system prioritizes searching for shorter distance plans in the candidate set and re-evaluates them. If the 15mm plan meets the hit constraint and the corresponding d_boundary > 0, an adjustment suggestion is generated: "It is recommended to adjust the ejection distance to 15mm to eliminate the intersection / intrusion risk and improve the overall safety score." This ejection distance adjustment suggestion is a parameter / gear selection suggestion before firing; no further adjustment is made after firing.

[0411] Specifically, the selection of different needle entry points involves determining the feasible search space based on the current distance from the needle tip to the target and the maximum needle length limit, including grid search and radial search.

[0412] Specifically, changing the needle insertion angle involves the system combining the angle constraints in the medical A* algorithm and providing the operator with angle adjustment guidance based on the recommended maximum angle change limit (30 degrees, soft constraint) in the medical constraints.

[0413] Direct angle calculation: α=arctan2(Δy,Δx)×180° / π, where α is the needle entry angle, Δx=target x coordinate - needle entry point x coordinate, Δy=target y coordinate - needle entry point y coordinate.

[0414] Feasibility verification of angle adjustment:

[0415] The system incorporates angle constraints from the medical A* algorithm to ensure that the recommended angle adjustment range conforms to actual operational limitations: max_angle_change_recommend=30.0° (upper limit of recommended maximum angle change, soft constraint); max_angle_change_limit=45.0° (clinical limit, hard constraint).

[0416] recommended_angle_range=[current_angle-30°, current_angle+30°] (recommended adjustment range); search_angle_range=[current_angle-45°, current_angle+45°] (upper bound of the algorithm's search range), where the angle can still be searched beyond 30° but the penalty in the cost function increases sharply; when the angle change is ≥45°, an infinite penalty value is set to prevent the path from expanding.

[0417] The specific application of the cumulative penalty calculation is as follows: when the overall safety score is below 0.3, the system automatically generates high-priority adjustment suggestions, including "suggesting to change the needle insertion point position" or "suggesting to adjust the needle insertion angle to the vertical direction"; when the overall safety score is in the range of 0.3-0.7, the system generates medium-priority suggestions, such as "considering to reduce the ejection distance" or "suggesting to confirm the location of the anatomical structure"; when the overall safety score is above 0.7, the system confirms that the current configuration is safe and feasible, and generates a confirmatory suggestion "the current ejection configuration is safe and the operation can be performed", thereby ensuring that the system can identify potential risks and provide appropriate safety reminders, while not completely preventing ejection operations.

[0418] Calculation example:

[0419] By aggregating multiple risk alerts into levels (critical / warning / info) and combining them with the overall security score to generate recommendations, the system can improve the alert level and recommendation priority in complex risk scenarios. This alert / label system is only used for risk explanation and interactive prompts and does not deduct from the overall security score.

[0420] In one embodiment, the multi-dimensional security assessment framework further includes a warning information list. This list outputs a graded distribution of the gap distance (distance_to_trajectory) based on the detected anatomical structure type and its preset risk distance threshold, with each grade mutually exclusive.

[0421] (1) Arteries: distance_to_trajectory≤5.0mm is "critical risk"; 5.0mm<distance_to_trajectory≤8.0mm is "warning"; distance_to_trajectory>8.0mm is "safe";

[0422] (2) Veins: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe";

[0423] (3) Esophagus: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe";

[0424] (4) Diaphragm: distance_to_trajectory≤2.0mm is "critical risk"; 2.0mm<distance_to_trajectory≤4.0mm is "warning"; distance_to_trajectory>4.0mm is "safe";

[0425] (5) Skeleton: As a physical obstacle zone, when distance_to_trajectory = 0.0mm (i.e., the boundary safety distance d_boundary≤0, indicating that the ejection trajectory intersects, contacts, or invades the interior of the skeleton boundary), it is a "critical risk"; when 0.0mm<distance_to_trajectory≤1.0mm, it is a "warning"; when distance_to_trajectory>1.0mm, it is a "safe";

[0426] (6) Lesions and lymph nodes: These are the target areas and are not to be avoided. Their critical_clearance and warning_clearance are both 0.0 mm, and they are marked as "safe / crossable" by default. When multiple risks exist at the same time, the warning message list will output multiple risk level labels and prompts simultaneously, with the highest priority risk as the main prompt. The risk level is adjusted for the overall safety score using a post-processing penalty coefficient.

[0427] Final safety score = Base score × Penalty;

[0428] The base score is 1.0 in the safest state with no risk; the penalty coefficient is determined according to the risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safe penalty = 1.0; when multiple risks exist at the same time, the penalty coefficient is the minimum of the penalty coefficients of each risk; in obstacle areas, skeletal structures are used as physical obstacles and are completely bypassed during path planning.

[0429] Rules for generating warning message list level labels:

[0430] When a critical risk exists (e.g., any structure whose distance_to_trajectory ≤ critical risk threshold), output the "critical risk" label and display it according to risk priority; when only a warning risk exists (critical risk threshold < distance_to_trajectory ≤ warning threshold), output the "warning" label; when all structures are in the safe zone (distance_to_trajectory > warning threshold), output the "safe" label.

[0431] When multiple risks exist simultaneously, the system outputs multiple risk labels and prompts at the same time, and displays them in order of preset priority. When a critical risk is detected, the overall security score will be reduced to 10% of the original value, and when a warning risk is detected, the overall security score will be reduced to 70% of the original value.

[0432] In one possible implementation, the navigation system establishes differentiated technical alternatives based on seven types of anatomical structures, including artery-specific treatment protocols, intermediate-risk structure treatment protocols, and obstacle structure treatment protocols. For the artery-specific treatment protocol, Doppler ultrasound is used to confirm the artery location before CNB (Needle-to-Blank Bone) is performed. For the intermediate-risk structure treatment protocol, the needle insertion path is adjusted to avoid the intermediate-risk structure. For the obstacle structure treatment protocol, the needle insertion path or point is modified. Specific examples of technical alternative implementations are provided below.

[0433] Assuming two key risk structures, arteries and bones, are detected:

[0434] Step 1: Artery risk management. The system detected a critical risk with structure_type 'artery' and automatically generated a Doppler ultrasound suggestion: "Consider using Doppler ultrasound to confirm the artery location before performing CNB".

[0435] Step 2: Handling skeletal obstructions. The system detected a critical risk with structure_type 'bone' and automatically generated a mandatory adjustment requirement: "Skeletal obstruction requires modification of the needle path or needle entry point."

[0436] Step 3: Generate a comprehensive recommendation list. The system integrates all types of adjustment suggestions into a complete recommended_adjustments list: recommended_adjustments=["Recommend reducing the ejection distance to 15mm to avoid high-risk structures", "Consider adjusting the needle insertion angle to avoid key anatomical structures", "Recommend choosing different needle insertion points to obtain a safer ejection path", "Consider using Doppler ultrasound to confirm the artery location before performing CNB", "Bone obstruction necessitates changing the needle path or needle insertion point"].

[0437] In one possible implementation, the navigation system establishes a ballistic error calculation model for the catapult trajectory, including position error calculation, relative error calculation, and angle error calculation, which can further improve the accuracy of the CNB catapult trajectory.

[0438] In the CNB trajectory validator, the position error is calculated using the Euclidean distance formula:

[0439] E position = √[(x predicted - x actual)² + (y predicted - y actual)²] × α; where, E position: position error, in millimeters (mm); x predicted, y predicted: coordinates of the ejection endpoint predicted by the algorithm; x actual, y actual: coordinates of the actual recorded needle impact point; α = 0.208998: pixel pitch calibration coefficient, in mm / pixel;

[0440] Calculation Example: Suppose the predicted endpoint of a certain ejection is (252, 218) pixels, and the actual endpoint is (250, 220) pixels, then the position error is calculated as follows:

[0441] Position E = √[(252-250)²+(218-220)²]×0.208998 = 2.83×0.208998 = 0.59mm.

[0442] Relative error reflects the proportional relationship between prediction error and launch distance:

[0443] E relative = (E position / D ejection) × 100;

[0444] E-relative: Relative error, expressed as a percentage (%);

[0445] D ejection: ejection distance, set to 22.0mm in the CNB system;

[0446] The relative error corresponding to the above-mentioned positional error of 0.59mm is:

[0447] E relative = (0.59 / 22.0) × 100 = 2.68%.

[0448] The angle error is obtained by calculating the angle between the predicted trajectory vector and the actual impact trajectory vector:

[0449] E angle = arccos[cos(θ)] × (180° / π), cos(θ) = (V predicted · V actual) / (|V predicted| × |V actual|), where the cosine value cos(θ) needs to be protected by the numerical domain, i.e., cos(θ) = max(-1.0, min(1.0, cos(θ))), where the cos(θ) on the left side of the equals sign is the corrected value, and the cos(θ) on the right side is the original value; V predicted: from the tip of the needle to the predicted The vector at the final point of the ejection is Vpredicted = (xpredicted - xtip, ypredicted - ytip); Vactual: the vector from the tip of the needle to the actual impact point, Vactual = (xactual - xtip, yactual - ytip); Eangle: the angle error, in degrees (°); xtip, ytip: the coordinates of the tip of the needle at the start of the ejection; ·: vector dot product operation; |Vpredicted|: the Euclidean modulus of the predicted vector; |Vactual|: the Euclidean modulus of the actual vector; max, min: numerical constraint functions to ensure that the cosine value is within the effective domain [-1, 1].

[0450] By protecting the numerical domain of the cosine value cos(θ), the calculation of the inverse cosine function can be prevented from failing when floating-point errors cause cos(θ) to slightly exceed [-1.0, 1.0]. It should be noted that numerical domain protection does not eliminate the risk of division by zero when the denominator |Vpredicted| × |Vactual| is 0: when |Vpredicted| = 0 or |Vactual| = 0, the angle error Eangle should be deemed unusable and skipped from statistics / alternatively labeled (or set to 0 and the reason recorded) to ensure the stability of the numerical calculation.

[0451] Assume the actual data from a certain CNB ejection operation:

[0452] - Tip starting position: (120, 150) pixels;

[0453] - Predicted ejection endpoint: (200, 180) pixels;

[0454] - Actual impact point: (205, 185) pixels;

[0455] Step 1: Construct the trajectory vector, Vpredicted = (200-120, 180-150) = (80, 30); Vactual = (205-120, 185-150) = (85, 35);

[0456] Step 2: Calculate the vector magnitude, |Vpredicted|=√(80²+30²)=√(6400+900)=√7300=85.44; |Vactual|=√(85²+35²)=√(7225+1225)=√8450=91.93;

[0457] Step 3: Calculate the vector dot product, Vpredicted·Vactual = 80×85 + 30×35 = 6800 + 1050 = 7850;

[0458] Step 4: Calculate the cosine value, cos(θ) = 7850 / (85.44 × 91.93) = 7850 / 7852.16 = 0.9997;

[0459] Step 5: Apply numerical protection, cos(θ)=max(-1.0,min(1.0,0.9997))=0.9997;

[0460] Step 6: Calculate the angle error, θ=arccos(0.9997)=0.0245 radians; E angle=0.0245×(180° / π)=1.40°.

[0461] Example of calculation results: Assuming the angle error of the ejection is 1.40°, it indicates that there is a small deviation between the actual ejection direction and the predicted direction, which is within the acceptable accuracy range.

[0462] The system also establishes a precision improvement mechanism based on the collaboration of three modules, including:

[0463] 1. Data acquisition mechanism of CNB trajectory verifier:

[0464] The trajectory validator, acting as an "intelligent medical recording system," is responsible for accurately recording and validating the predicted and actual results of each ejection. Its validation process is as follows:

[0465] 1.1 Prediction and Recording Stage: Record the needle tip position (x_tip, y_tip), needle angle θ, target point position (x_target, y_target), ejection distance D_throw, and predicted endpoint (x_pred, y_pred);

[0466] 1.2 Verification of actual results: Record the actual endpoint (x_actual, y_actual), surgical success status S, and sample quality grade Q;

[0467] 1.3 Error Calculation and Classification: Calculate position error, angle error and target offset distance, and classify and statistically analyze them according to conditions such as needle specifications, ejection distance, and target distance;

[0468] Instance verification method: The system has established a verification mechanism that compares prediction verification with actual results, and can calculate statistical data of indicators such as position error and angle error.

[0469] 2. Trend analysis algorithm of CNB error collector:

[0470] The error collector, acting as the "hospital quality control center," employs an intelligent time window adaptation algorithm for error trend analysis.

[0471] R_trend = [1 / (n-1)] × Σ(E(i+1)-Ei) (i=1 to n-1);

[0472] Symbol meanings: Rtrend: rate of change of trend, in mm / time window; Ei: average error of the i-th time window; n: number of time windows;

[0473] Time window adaptive selection strategy:

[0474] - When the time span is less than 1 hour: use minute-level grouping;

[0475] - When the time span is 1-24 hours: use hourly grouping;

[0476] - When the time span exceeds 24 hours: use daily grouping;

[0477] Case Study: The system can detect error change trends to determine whether accuracy is continuously improving. When an abnormal rate of change is detected, the system automatically limits it to a reasonable range.

[0478] 3. Systematic bias correction of the CNB calibration algorithm:

[0479] The calibration algorithm uses statistical methods to distinguish between systematic errors and random errors, and only corrects confirmed systematic deviations.

[0480] 3.1 Formula for systematic deviation test:

[0481] One-sample t-test: t = (X̄ - μ0) / (S / √n); Coefficient of variation criterion: CV = S / |X̄|;

[0482] Symbol meanings: X̄: Sample mean (e.g., distance ratio or angle offset); μ0: Expected value (distance ratio is 1.0, angle offset is 0°); S: Sample standard deviation; n: Sample size; CV: Coefficient of variation;

[0483] Systematic bias determination criteria: When |t|>2.0 and CV<0.5, a systematic bias is determined to exist.

[0484] 3.2 Distance Correction Algorithm:

[0485] For confirmed systematic deviations in distance, apply a scaling factor for correction:

[0486] D-correction = D-prediction × λ, where the scaling factor is calculated as follows:

[0487] λ=(1 / n)×Σ(Dactual,i / DPredicted,i) (i=1 to n);

[0488] Symbol meanings: D-correction: corrected ejection distance; D-prediction: original predicted distance; λ: distance scaling factor; D-actual,i: actual ejection distance of the i-th time; D-prediction,i: predicted ejection distance of the i-th time;

[0489] Safety constraint: 0.95≤λ≤1.05, which means a maximum distance correction of 5% is allowed.

[0490] 3.3 Angle Correction Algorithm:

[0491] For systematic angular deviations, apply offset correction:

[0492] θ correction = θ prediction + Δθ; where the angle offset is: Δθ = (1 / n) × Σ(θ actual,i - θ prediction,i) (i = 1 to n), where: θ correction: the corrected needle angle; θ prediction: the original predicted angle; Δθ: the angle offset correction amount; θ actual,i: the actual ejection angle of the i-th ejection; θ prediction,i: the predicted angle of the i-th ejection.

[0493] Safety constraint: |Δθ|≤2°, meaning a maximum angle correction of 2 degrees is allowed.

[0494] 4. Cross-validation to prevent overfitting:

[0495] K-fold cross-validation (K=5) was used to evaluate the correction effect:

[0496] R(k) = (Ēoriginal(k) - Ēcorrected(k)) / Ēoriginal(k);

[0497] Symbol meanings: R(k): improvement rate of the kth fold; E_original(k): mean original error of the kth fold; E_corrected(k): mean error after correction of the kth fold; overfit detection index: σR=√{(1 / K)×Σ[R(k)-R̄]²}, (k=1 to K);

[0498] When σR > 0.5 or R(k) < -0.3, it is determined that there is a risk of overfitting.

[0499] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the invention. The advantages of the present invention have been fully and effectively realized. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments; any variations or modifications can be made to the implementation of the present invention without departing from these principles.

Claims

1. An ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism, characterized in that, For multimodal detection, the navigation system is implemented using a parallel dual-model detection architecture, which includes a detection phase and a result extraction phase. In the detection phase, the detection system sets uniform inference parameters. During system initialization, two YOLOv11 models are loaded simultaneously. The first is an oriented bounding box model, which is used to detect the directionality of the puncture needle in the input frame. The second is a segmentation model, which is used to segment seven types of anatomical structures. In the result extraction stage, the detection system extracts puncture needle information from the detection results of the directional bounding box model, extracts the effective mask or its contour / center information of all anatomical structures from the segmentation results one by one according to the category, and adapts the size of each category mask to the same size as the input frame. The segmentation model establishes a hierarchical security system based on medical priorities, with category mapping consistent with the training configuration. When extracting masks for each category, the validity of the segmentation results is first verified. Then, all valid mask data are extracted from the multi-class segmentation results one by one according to the category. Specifically, one or more valid masks are extracted for each lesion category. When multiple valid lesion masks exist, the main lesion mask is determined according to the mask area / confidence. The main lesion mask is used for target center calculation and subsequent navigation / ejection constraint evaluation. The mask data is transferred from GPU memory to CPU and binarized using a preset segmentation threshold, converting floating-point probability values ​​into integer binary masks. When the mask size does not match the target frame size, the system uses a nearest neighbor interpolation algorithm for size adjustment. The navigation system also establishes a mathematical model of the ejection process through an ejection trajectory calculation function, including: Physical coordinate transformation converts the needle tip position and target center from the pixel coordinate system to the physical coordinate system. Then, the direction vector of the puncture needle is calculated based on the needle angle, converting the angle from degrees to radians, and the unit vector components in the x and y directions are calculated using trigonometric functions. Direction vector calculation, used for converting angles to direction vectors; The ejection endpoint is calculated by using the physical coordinates of the starting position of the needle tip and the ejection distance to calculate the physical coordinates of the ejection endpoint. Target distance calculation is used to calculate the straight-line distance from the tip of the needle to the center of the target using the physical coordinates of the target center.

2. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 1, characterized in that, Navigation path planning is performed using an enhanced A* algorithm with medical constraints. During system initialization, a medical grid space is established as the computational basis for the A* algorithm. The continuous image space is discretized into a searchable node network through gridding. Pixel spacing parameters are set for physical-to-pixel coordinate transformation. The medical constraint parameter system includes: Angle variation limits: The recommended maximum angle variation is 30.0 degrees, and the clinical limit angle threshold is 45.0 degrees, simulating the physical limitations of needle rotation in real puncture operations; Step size control, with a minimum step size of 1 grid unit and a maximum step size of 3 grid units, ensures the operability of the navigation path; Grid resolution, default 4 pixels / grid unit, balances calculation accuracy and processing efficiency through configurable resolution parameters; The connectivity method employs 8-connected neighborhood search, providing path selection in 8 directions; The weighting strategy uses a risk weight factor of 5.0 and a path smoothing weight factor of 2.

0.

3. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 2, characterized in that, A continuous risk field is constructed in the pixel space of the original ultrasound image. Then, the pixel-level risk values ​​are aggregated to grid nodes through spatial mapping to realize the transformation from the continuous risk field to the discrete search space. The final generated risk field then provides guidance for the navigation path planning algorithm. The detection system first creates a zero-value-initialized floating-point risk field matrix based on the input image size, and then constructs boundary risk regions and anatomical structure risk regions in layers; The boundary risk area is defined as follows: boundary constraints are established within a 5-pixel range of the top edge, bottom edge, left edge, and right edge of the image. The anatomical structure risk area is defined as follows: For each detected anatomical structure object, the system extracts the segmentation mask or geometric contour, center coordinates, critical risk threshold, warning threshold, and avoidance priority parameters for each anatomical structure object; the risk zone distance is uniformly calculated using the boundary safety distance, and an equidistant hierarchical risk model is adopted to establish a two-level risk area system, in which the high-risk area is assigned an extremely high risk value of 1000.0 × priority, and the warning area is assigned a medium risk value of 200.0 × priority.

4. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 3, characterized in that, The navigation path planning process integrates a movement cost architecture. The detection system establishes a three-layer cost structure, including basic movement cost, risk cost, and smoothness cost, i.e., enhanced movement cost. The basic movement cost calculates the geometric distance from the current grid point to adjacent grid points using the Euclidean distance formula, ensuring the spatial efficiency of the path. The risk cost is calculated by multiplying the risk value of adjacent points by the risk weight factor in medical constraints, and then applying a standardization coefficient of 0.001 for numerical adjustment. The smoothness cost is based on the angle changes of navigation path segments. When the current grid point has a parent node, the smoothness cost calculation algorithm calculates the angle changes of adjacent path segments. The system uses... The cosine of the included angle is obtained by multiplying the vector dot product with the vector magnitude. Then, the inverse cosine function is applied to obtain the angle change and convert it from radians to degrees. The system evaluates all angle changes: when the angle change is ≤10 degrees, a negative reward is given to encourage a smooth path; when the angle change is between 10 and 20 degrees, a linearly increasing penalty is applied; when the angle change is between 20 and 30 degrees, a quadratic penalty is applied; when the angle change is >30 degrees, the penalty intensity increases sharply, and a cubic penalty or an infinite penalty is applied to reinforce the cost of uncomfortable turning. The weighting factors in the three-layer cost structure are optimized based on clinical practice: the risk weighting factor of 5.0 ensures that the algorithm prioritizes avoiding high-risk areas, based on the principle of medical safety as the highest priority. A standardization factor of 0.001 is used to unify the numerical range and ensure the reasonable superposition of costs with different dimensions.

5. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 4, characterized in that, An angle change penalty mechanism is also integrated into the navigation path planning process. The detection system establishes an angle change threshold mechanism based on medical constraints. When the angle change between path segments is greater than 30 degrees, the angle change penalty strategy algorithm applies an angle change penalty. The penalty intensity is determined by the product of the angle change amount and the smoothness weight factor, and a 0.1 adjustment coefficient is applied for numerical optimization. The angle change penalty strategy algorithm includes: constructing a five-level medical safety threshold system, establishing a graded penalty mechanism based on the physical limitations of clinical puncture operations, and setting five assessment levels according to the medical safety level of the angle change: comfortable operation range, clinically acceptable range, high-risk operation range, extremely dangerous operation range, and technically prohibited range. Among them, the upper limit of the angle change in the high-risk operation range is 30 degrees, which is the recommended maximum angle change limit. If the angle change exceeds 30 degrees, searching is still allowed, but the penalty intensity increases sharply. Among them, the upper limit of the angle change in the extremely dangerous operation range is 45 degrees, which is the clinical limit threshold. When the angle change is ≥45 degrees, an infinite penalty value is set to prohibit path expansion.

6. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 1, characterized in that, The navigation system establishes a geometric distance calculation algorithm based on parametric straight line equations. The calculation formula is as follows: d1 = √[(px - projection_x)² + (py - projection_y)²], where: d1 is the shortest distance from the target point to the line segment; px and py are the coordinates of the target point; projection_x = x1 + t × (x2 - x1), projection_y = y1 + t × (y2 - y1), where x1 and y1 are the coordinates of the starting point of the line segment, x2 and y2 are the coordinates of the ending point of the line segment, and t is the projection parameter, which represents the relative position of the projection point on the line segment. t = max(0, min(1, [(px - x1) × (x2 - x1) + (py - y1) × (y2 - y1)] / line_length²)), where the value of t is in the range of 0-1. line_length is the length of the line segment, line_length = √[(x2 - x1)² + (y2 - y1)²]. max and min are the maximum and minimum value functions, respectively, to ensure that the projection point is within the range of the line segment. The shortest distance from the target point to the line segment is combined with the determination of whether the object is inside or outside the structure to obtain the boundary safety distance d_boundary. Here, d_boundary > 0 means that the ejection trajectory is outside the structure boundary and the minimum gap between it and the boundary is d_boundary. d_boundary ≤ 0 means that the ejection trajectory intersects / contacts with the boundary of the anatomical structure or intrudes into it.

7. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 6, characterized in that, The system establishes a multi-dimensional safety assessment framework including an overall safety score and a list of key risks. The overall safety score uses a standardized scoring system from 1.0 to 0.0, where 1.0 represents the safest state and 0.0 represents the most dangerous state. Based on the status of key risks in the list of key risks, the system automatically triggers an adjustment suggestion generation process. Adjustment suggestions include adjusting the ejection distance, changing the needle insertion angle, and selecting different needle insertion points. The overall safety score is calculated using a cumulative penalty mechanism. The base score is 1.0 in the safest state, representing no risk. The score is determined by taking the minimum value of the penalty factors for all risk sources, achieving a veto for the most severe risk. Overall safety score = base score × penalty; Penalty is the penalty coefficient, which is set according to the risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safety penalty = 1.

0. When multiple risks exist at the same time, the penalty is the minimum value of the penalty coefficient of each risk.

8. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 7, characterized in that, The multi-dimensional security assessment framework also includes a warning information list. This list outputs the gap distance (distance_to_trajectory) in a tiered manner based on the detected anatomical structure type and its preset risk distance threshold, with each tier mutually exclusive. (1) Arteries: distance_to_trajectory≤5.0mm is "critical risk"; 5.0mm<distance_to_trajectory≤8.0mm is "warning"; distance_to_trajectory>8.0mm is "safe"; (2) Veins: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe"; (3) Esophagus: distance_to_trajectory≤3.0mm is "critical risk"; 3.0mm<distance_to_trajectory≤5.0mm is "warning"; distance_to_trajectory>5.0mm is "safe"; (4) Diaphragm: distance_to_trajectory≤2.0mm is "critical risk"; 2.0mm<distance_to_trajectory≤4.0mm is "warning"; distance_to_trajectory>4.0mm is "safe"; (5) Skeleton: As a physical obstacle area, distance_to_trajectory = 0.0mm, that is, the boundary safety distance d_boundary≤0, indicates that the ejection trajectory intersects, contacts or invades the interior of the skeleton boundary, which is a "critical risk"; 0.0mm < distance_to_trajectory ≤ 1.0mm indicates a "warning"; distance_to_trajectory > 1.0mm indicates a "safe" condition. (6) Lesions and lymph nodes: As the target area, they are not to be avoided. Their critical_clearance and warning_clearance are both 0.0mm. They are marked as "safe / crossable" by default. When multiple risks exist at the same time, the warning information list will output multiple risk level labels and prompts at the same time, and the highest priority risk will be used as the main prompt. The risk level is adjusted for the overall safety score using a post-processing penalty factor: Final safety score = Base score × Penalty; The base score is 1.0 in the safest state with no risk; the penalty coefficient is determined according to the risk level: critical risk penalty = 0.1, warning risk penalty = 0.7, and safe penalty = 1.0; when multiple risks exist at the same time, the penalty coefficient is the minimum value of the penalty coefficients of each risk. In obstacle zones, skeletal structures act as physical barriers, and are completely bypassed during path planning.

9. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 6, characterized in that, The navigation system establishes differentiated technical alternatives based on seven types of anatomical structures, including artery-specific treatment, intermediate-risk structure treatment, and obstacle structure treatment. For the artery-specific treatment, Doppler ultrasound is used to confirm the artery location before CNB is performed. For the intermediate-risk structure treatment, the needle insertion path is adjusted to avoid the intermediate-risk structure. For the obstacle structure treatment, the needle insertion path or point is modified.

10. The ultrasonic guidance and navigation system based on multimodal detection and CNB ejection mechanism as described in claim 6, characterized in that, The system establishes a ballistic error calculation model for the ejection trajectory, including position error calculation, relative error calculation, and angle error calculation; In the CNB trajectory validator, the position error is calculated using the Euclidean distance formula: Position E = √[(predicted x - actual x)² + (predicted y - actual y)²] × k; where... E position: Position error, in millimeters; x-prediction, y-prediction: The coordinates of the launch endpoint predicted by the algorithm, in pixels; x_actual, y_actual: The actual recorded coordinates of the needle tip impact point, in pixels; k=0.208998: Pixel pitch calibration coefficient, in mm / pixel. This coefficient is based on the standard image resolution and physical size calibration of the ultrasound equipment to ensure the accuracy of coordinate transformation. Relative error reflects the proportional relationship between prediction error and launch distance: E relative = (E position / D ejection) × 100; E-relative: Relative error, expressed as a percentage (%); D ejection: ejection distance, set to 22.0mm in the CNB system; The angle error is obtained by calculating the angle between the predicted trajectory vector and the actual impact trajectory vector: Angle E = arccos[cos(θ)] × (180° / π), cos(θ) = (Vpredicted·Vactual) / (|Vpredicted| × |Vactual|), where the cosine value cos(θ) needs to be protected by the numerical domain, i.e., cos(θ) = max(-1.0, min(1.0, cos(θ))). The numerical domain protection mechanism forces cos(θ) to be within the range of [-1.0, 1.0] to prevent floating-point operation errors from causing the inverse cosine function calculation to fail, thus improving the numerical stability of the algorithm. When |Vpredicted| = 0 or |Vactual| = 0, the denominator is 0. The system marks the angle error as unusable and skips the calculation, or sets it to 0 according to the preset rules, thereby avoiding division by zero. V-prediction: The vector pointing from the tip of the needle to the predicted ejection endpoint, V-prediction = (x-prediction - x-tip, y-prediction - y-tip); Vactual: The vector from the needle tip to the actual point of impact, Vactual = (xactual - xneedle tip, yactual - yneedle tip). E angle: angular error, in degrees (°); x needle tip, y needle tip: Coordinates of the needle tip position at the start of ejection; ·: Vector dot product operation; |Vprediction|: Euclidean magnitude of the prediction vector; |Vactual|: Euclidean magnitude of the actual vector; max, min: Numerical constraint functions that ensure the cosine value is within the valid range [-1, 1].

Images