Intelligent testicular artery recognition system and method based on multi-modal fusion and dynamic learning
Through multimodal image acquisition and dynamic adaptive neural networks, combined with optical microscopy, ultrasonic blood flow and electrophysiological signals, accurate identification and real-time navigation of the testicular artery can be achieved, solving the problems of insufficient recognition accuracy and high mis-ligation rate in existing technologies, and improving surgical efficiency and safety.
Patent Information
- Application Number
- CN202510906128.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-01
- Publication Date
- 2025-09-30
AI Technical Summary
Existing technologies are inaccurate in identifying the testicular artery, have a high rate of misligation, and lack real-time optimization capabilities. They are unable to adapt to anatomical variations in different patients, limiting surgical efficiency and safety.
A multimodal image acquisition module was constructed, combining optical microscopy, ultrasonic blood flow and electrophysiological signals, and a dynamic adaptive neural network was used to identify the testicular artery. Real-time navigation and model updating were achieved through hardware synchronous acquisition, multi-branch network fusion, three-dimensional dynamic navigation and federated learning optimization.
It significantly improves the accuracy of testicular artery identification, reduces the rate of misligation, shortens operation time, improves operation efficiency and safety, and adapts to anatomical variations of different patients.
Smart Images

Figure CN120713472A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of medical image processing, surgical navigation and artificial intelligence technology, and specifically to a technology that integrates optical microscope images, ultrasonic blood flow signals and intraoperative electrophysiological data and realizes precise identification and real-time navigation of the testicular artery through a dynamic adaptive learning mechanism. Background Art
[0002] Varicocele is a common cause of male infertility, with surgical treatment being the primary treatment. Microscopic varicocele ligation is widely used due to its high precision and low complication rates. However, accurate identification and preservation of the testicular artery during surgery is crucial, as it supplies 90% of the testicular blood supply. Damage to the artery can lead to testicular atrophy and impaired spermatogenesis.
[0003] Currently, identification of the testicular artery relies primarily on the surgeon's subjective judgment, a traditional method with significant limitations. Its accuracy is significantly affected by factors such as surgeon experience, intraoperative bleeding, and vasospasm. Literature indicates that traditional methods have an accuracy rate of approximately 75%-85%, with a misligation rate as high as 5-8%, severely impacting surgical outcomes and patient outcomes.
[0004] Existing technologies also suffer from a single-modality dependency, relying solely on optical microscope images for recognition. When bleeding occurs during surgery, the optical features become blurred, significantly increasing the difficulty of recognition. Furthermore, existing deep learning models are mostly static, trained offline, lacking real-time intraoperative optimization capabilities. They are unable to adapt to vascular anatomical variations across patients, significantly increasing the rate of mis-ligation in patients with anatomical variations.
[0005] Furthermore, traditional systems only provide static annotations and fail to integrate hemodynamic and electrophysiological signals to build a multi-dimensional basis for decision-making. This makes it difficult to provide comprehensive and accurate decision support to the surgeon, thus limiting surgical efficiency and safety. Currently, no system has achieved real-time fusion analysis of optical imaging, ultrasound blood flow, and electrophysiological signals, and there is a lack of a mechanism for continuous model optimization based on postoperative feedback, which has become a technical challenge that needs to be addressed in this field. Summary of the Invention
[0006] This invention has developed a system and method for intelligent testicular artery identification that integrates optical microscopy imaging, ultrasonic hemodynamics, and vascular electrophysiological signals. Through a four-layer technical architecture comprised of "hardware synchronous acquisition - multi-branch network fusion - three-dimensional dynamic navigation - and federated learning optimization," this system achieves full-process intelligent artery identification during microscopic varicocele ligation. The system's core innovation lies in overcoming the physical limitations of traditional single-modality recognition. By leveraging multi-dimensional biosignal complementation and a dynamic learning mechanism, the accuracy of artery identification in complex surgical scenarios has been increased from 75%-85% with traditional methods to 97.3%, while also reducing the mis-ligation rate to below 0.8% and shortening surgical time by an average of 28% (from 49 minutes to 35 minutes).
[0007] The system composition and technical details are as follows:
[0008] The preferred multimodal image acquisition module includes an optical submodule using a Zeiss Pico core optical system, equipped with a 1920×1080 resolution CMOS image sensor, and supporting real-time acquisition at 30fps. It also features a built-in 808nm near-infrared laser module (with adjustable power range of 10-50mW) that uses a dichroic mirror to enable simultaneous acquisition of indocyanine green angiography and white-light imaging. The fluorescence imaging frame rate is consistent with that of white light, with a spectral response range of 700-900nm. The ultrasound submodule integrates a linear array ultrasound probe (model Vivid 90, GE Healthcare) with an operating frequency of 10-15MHz, an axial resolution of 0.08mm, a lateral resolution of 0.15mm, and a sampling depth dynamically adjusted (5-10mm) via an adaptive focusing algorithm. The probe is rigidly connected to the microscope optical path via a six-axis robotic arm (with a repeatability of 0.05mm), supporting ±30° pitch, ±15° yaw, and 0-50mm axial movement. The scanning angle and focus position are controlled in real time by an FPGA (Xilinx Zynq UltraScale+). The electrophysiology submodule utilizes an 8-channel flexible electrode array (50 μm thick, 100 μm diameter) coated with silver chloride to reduce contact impedance. It features a 1 kHz sampling rate, 24-bit analog-to-digital conversion accuracy, and a passband filter range of 0.1-1000 Hz, enabling real-time recording of vascular wall action potentials (feature extraction is triggered when the amplitude is ≥50 μV). The synchronous trigger system utilizes a hardware synchronization circuit based on a field-programmable gate array (FPGA). This system uses a 10 MHz crystal oscillator as the master clock, which is transmitted to each modality acquisition unit via differential signaling. This ensures a phase difference of ≤30 μs between the optical frame synchronization signal (VSYNC), ultrasound scan trigger signal (TRIG), and electrophysiological sampling clock (CLK), resulting in a timestamp accuracy of 1 μs.
[0009] The dynamic adaptive neural network model is as follows:
[0010] The preferred network architecture details are as follows: The optical branch uses a Swin-UNet++ backbone network consisting of a 4-layer encoder and a 4-layer decoder. Each encoder layer consists of two Swin Transformer blocks (window size 7×7). Overlapping sliding windows (128×128, stride 64) are introduced to process 2048×1536 resolution images. The multi-head self-attention mechanism is set with 8 heads, the embedding dimension is 384, and LayerNorm normalization is used to prevent gradient vanishing. The ultrasound branch uses a 5-layer 1D-CNN (convolution kernel size 3×3, stride 1) at the front end to extract blood flow time-frequency map features. The back end is connected to a dual-channel attention module (channel attention + spatial attention), focusing on strengthening the feature extraction of 1-2Hz pulse frequency components. The short-time Fourier transform uses a Hanning window with a window length of 256 points and a 75% overlap rate. The electrophysiology branch employed a "1D-CNN + bidirectional LSTM" architecture. The 1D-CNN consisted of three layers (5×5 convolution kernel, stride 1) to extract action potential waveform features. The bidirectional LSTM had a hidden layer dimension of 128 and focused on periodic electrical activity (frequency 1-3 Hz) through a temporal attention mechanism. An online learning mechanism employed an "uncertainty sampling + entropy minimization" strategy, screening samples with a predicted entropy ≥ 1.5 every 5 minutes (entropy calculation formula: H(p) = -∑p_i log(p_i)). Parameters were updated using an SGD optimizer (momentum 0.9, weight decay 1e-4). The learning rate decayed from 0.001 to 0.0005 using a cosine function, and gradient clipping (threshold 5.0) was performed every 100 iterations. Virtual adversarial training (VAT) was introduced to enhance model robustness, with a perturbation amplitude of ε = 1.0 and three iterations to ensure the model's robustness to intraoperative noise (e.g., electrophysiological baseline drift).
[0011] The three-dimensional dynamic navigation module includes the following processes:
[0012] Preferably, the spatiotemporal registration process: in the rigid registration stage, SIFT feature point detection (scale space extreme value detection, contrast threshold 0.04) is used, and the RANSAC algorithm (500 iterations) is used to eliminate mismatched points, and the transformation matrix solution error is ≤0.5 pixels; in the non-rigid registration stage, the B-spline free deformation field (control point grid spacing 5mm×5mm×5mm) is used, and the deformation field parameters are optimized based on the Demons algorithm. The registration error is evaluated by the mutual information (MI) indicator, and the final three-dimensional space error is ≤0.3mm.
[0013] Optional visualization and interaction: 3D reconstruction uses the marching cubes algorithm, with a voxel size of 0.1mm×0.1mm×0.1mm, and surface rendering uses the Phong illumination model. The holographic projection module (HoloLens2) has a resolution of 1280×720 per eye and a field of view of 52°. SLAM technology aligns the 3D probability cloud map with the microscope's physical space, with a registration error of ≤1mm. Dynamic annotation rules: When the confidence level is ≥95% and the ultrasound PI is ≥1.5, the artery is annotated with a red semi-transparent grid (with a transparency of α=0.7); when the confidence level is 80%-95%, the point cloud is displayed as a yellow point cloud; below 80%, the annotation is hidden to avoid distracting the operator.
[0014] The closed-loop optimization decision module includes the following framework:
[0015] The preferred multi-dimensional decision model uses the weighting function: Score = 0.5 × P_optical + 0.3 × (PI / 3.0) + 0.2 × (1-D_electro), where P_optical is the optical probability (range 0-1), PI is the ultrasonic pulsatility index (range 0-3), and D_electro is the Euclidean distance of the electrophysiological features (range 0-1). The decision threshold is set as follows: Score ≥ 0.85 confirms arterial status, 0.7-0.85 triggers enhanced verification, and < 0.7 is marked as non-arterial. The federated learning framework uses the FedAvg algorithm. Each hospital edge node retains local data (N = 200 cases / center), and a global model update is performed quarterly. Security mechanisms include homomorphic encryption for transmitting model parameter gradients, and differential privacy (ε = 1.0, δ = 1e-5) is implemented during local training to prevent patient data leakage.
[0016] The method steps and implementation details are as follows:
[0017] S1. Collaborative annotation of multimodal data (data construction)
[0018] Annotation process: 1. Using 4D Slicer 5.0.2 software, import synchronously acquired optical video (.mkv format, 1920×1080, 30fps), ultrasound sequence (.dcm format, 25 frames per second), and electrophysiological data (.edf format, 1kHz); 2. Synchronize the three modal data using the time axis alignment tool (accuracy 1 frame), and construct a vascular anatomical reference model based on preoperative CTA (.nii format, resolution 0.5mm×0.5mm×0.5mm); 3. Use a semi-supervised annotation strategy: first, three senior surgeons manually annotate 20% of the data (1000 frames), then automatically annotate the remaining 80% using pseudo-labeling technology (pseudo-labeling, threshold 0.9), and finally verify by experts (error rate ≤5%).
[0019] Dataset construction: Contains data from 200 patients, each of which includes: optical images (30 minutes, approximately 54,000 frames), ultrasound blood flow images (same time period, approximately 67,500 frames), electrophysiological signals (30 minutes, 1,800,000 sampling points), and CTA three-dimensional models; data enhancement: optical images are rotated (±15°), scaled (0.8-1.2 times), and enhanced with Gaussian noise (σ=0.05); ultrasound data are enhanced with time-frequency domain perturbations; and baseline drift simulation is added to electrophysiological signals (frequency 0.1-1 Hz, amplitude ±20 μV).
[0020] S2. Dynamic Fusion Training (Algorithm Optimization)
[0021] Pre-training stage: Cross-modal transfer learning was performed on the MONAI dataset (containing 1000 medical images), freezing the parameters of the first two layers of the encoder, with a learning rate of 0.001, training for 100 epochs, and using the Dice Loss + Cross-Entropy Loss loss function (weighted 1:1).
[0022] Intraoperative optimization stage: approximately 500 frames of unlabeled data (entropy value ≥ 1.5) were collected every 5 minutes, and the parameters were updated using the "model prediction-teacher forcing" mode. The teacher model was the preoperative pre-training model, and the student model was the online fine-tuning model. Convergence was accelerated through knowledge distillation (temperature parameter T = 2).
[0023] Postoperative optimization phase: Semen analysis data (PRM forward motility sperm rate, sperm concentration) and complication records of patients are collected 3 months after surgery. When the AUC value of the global model on the validation set increases by ≥2%, the model update is triggered. The update package size is ≤10MB and is transmitted to each hospital node via the 5G network.
[0024] S3. Real-time multimodal processing (hardware acceleration)
[0025] Signal processing pipeline: 1. Optical images are transmitted to a GPU (NVIDIA A10, 24GB of video memory) via PCIe 4.0 (8GB / s bandwidth) for preprocessing (normalization to [0, 1], channel reordering). 2. Ultrasound Doppler signals are processed in real time by an FPGA (Xilinx VU9P): FFT calculation (4096 points) generates time-frequency plots, and the PI value is calculated (formula: PI = (PSV - EDV) / TAMV). 3. Electrophysiological signals are input by a data acquisition card (NI USB-6366) and passed through a second-order Butterworth bandpass filter (0.1-100Hz) to extract the R wave peak (amplitude ≥ 50μV).
[0026] Feature fusion strategy: Cross-modal attention gate is used. The optical feature map (H×W×C1) and the ultrasonic feature map (H×W×C2) are channel-concatenated (C1+C2), and then subjected to 1×1 convolution to generate attention weights. The weights are dot-multiplied with the electrophysiological feature vector (dimension C3) and then fused with the 3D reconstruction module through upsampling.
[0027] S4. Closed-loop decision support (clinical application)
[0028] Intraoperative decision-making process: 1. When the confidence level of the three-dimensional probability cloud map is ≥90% and the ultrasound PI is ≥1.5, the system automatically triggers electrophysiological verification (lasting 5 seconds, with the sampling rate increased to 2kHz); 2. If the electrophysiological signal detects ≥3 complete action potential cycles (frequency 1-2Hz) and the Euclidean distance of the eigenvector is <0.8, it is confirmed to be an artery, and the navigation system highlights it in red; 3. The operator can manually trigger the "deep verification" mode using the foot switch, and the system uploads the previous 10 seconds of multimodal data to the cloud server (response time ≤2 seconds) for secondary verification.
[0029] Postoperative feedback mechanism: A correlation analysis model between decision-making results and clinical outcomes was established, and the Cox proportional hazard model was used to evaluate the risk factors for mispuncture. The decision threshold was updated every 50 surgeries. For example, when the postoperative testicular atrophy rate was >3%, the optical probability threshold was automatically increased from 90% to 92%.
[0030] The above technical solution can bring the following specific and clear technical effects:
[0031] 1. Significantly Improved Recognition Accuracy, Stronger Robustness in Complex Scenarios: By integrating trimodal fusion of optical imaging, ultrasonic blood flow signals, and electrophysiological data, combined with a dynamic adaptive neural network, the system improves the accuracy of testicular artery recognition during microscopic varicocele ligation from 75%-85% with traditional single-modality methods to 97.3%. In complex scenarios such as bleeding and vasospasm, ultrasonic blood flow signals (pulsatility index PI ≥ 1.5) and electrophysiological characteristics (action potential frequency 1-3Hz) can replace ambiguous optical features and continuously provide recognition information, increasing the recognition success rate from 68% to 94% and reducing the false ligation rate from 5-8% to below 0.8%.
[0032] 2. Dynamic adaptive learning enables continuous model evolution and adapts to individual differences: Online fine-tuning is performed every 5 minutes during surgery using high-uncertainty samples (entropy ≥ 1.5). Combined with postoperative federated learning (based on patients' semen quality improvement rates and complication data), the model dynamically adapts to vascular anatomical variations in different patients. In 20 patients with anatomical variations, the traditional static model had a 15% mispuncture rate, while this system reduced this rate to 0. When applied across centers, federated learning optimization reduced the difference in model recognition accuracy between hospitals to ≤3%, avoiding the accuracy loss caused by a "one-size-fits-all" model adaptation.
[0033] 3. 3D dynamic navigation and multi-dimensional decision-making shorten surgery time and improve operation safety: 3D probability cloud map superimposed by holographic projection (voxel 0.1mm 3 , update frequency ≥15fps) and a multi-dimensional weighted decision model (optical probability weight 0.5, ultrasound PI weight 0.3, electrophysiological feature weight 0.2), shortening the time for key vessel identification from 120 seconds using traditional methods to 45 seconds, and the average surgical time from 49 minutes to 35 minutes, improving efficiency by 28.6%. At the same time, the system provides three-dimensional decision support for the surgeon through real-time annotation (highlighting arteries when confidence level ≥95% and PI ≥1.5) and enhanced electrophysiological verification triggered by conflict indicators (sampling rate increased to 2kHz), significantly reducing the risk of testicular artery mispuncture and reducing the incidence of postoperative complications from 7.8% to 2.1%. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 This is a schematic diagram of the testicular artery intelligent recognition system based on multimodal fusion and dynamic learning in this invention.
[0035] Figure 2 This is a step diagram of the testicular artery intelligent identification method based on multimodal fusion and dynamic learning in this invention.
[0036] Figure 3 This is the real-time multimodal processing flow chart of the testicular artery intelligent recognition system based on multimodal fusion and dynamic learning of the patented invention.
[0037] Figure 4 This is a schematic diagram of the patented multimodal image acquisition module of this invention. DETAILED DESCRIPTION
[0038] The following is a combination of the embodiments of the present invention Figures 1 to 4 The technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0039] 1. Equipment Integration and Preoperative Preparation
[0040] 1. Hardware system construction
[0041] According to the technical solution of claim 1, a Zeiss pico microscope (resolution 1920×1080, frame rate 30fps) is connected to the NVIDIA Jetson AGX Orin edge computing platform (275TOPS computing power) via an HDMI interface. At the same time, a 10-15MHz high-frequency linear array ultrasound probe is integrated through a six-axis robotic arm bracket (positioning accuracy 0.1mm), achieving ±30° deflection and 0-50mm axial adjustment. The electrophysiological microelectrode array (8 channels, sampling rate 1kHz) is connected to the data acquisition card via a BNC interface, sharing the FPGA synchronization trigger circuit with the microscope and ultrasound probe to ensure that the time synchronization accuracy of the three-modal data is ≤30μs.
[0042] A near-infrared fluorescence imaging channel (excitation wavelength 808 nm) is integrated into the microscope optical path to support indocyanine green angiography.
[0043] 2. Software deployment and model pre-training
[0044] A dynamic adaptive neural network model based on the Swin-UNet++ architecture is deployed on the edge computing platform. The optical branch uses a 128×128 overlapping sliding window (step size 64) and a multi-head self-attention mechanism (head number 8). The ultrasound branch extracts blood flow features through short-time Fourier transform (window length 256 points, overlap rate 75%). The electrophysiological branch uses a CNN-LSTM network.
[0045] Before the operation, the 4D Slicer tool was used to construct a four-modal label library containing preoperative CTA data of 200 patients. Semi-supervised learning and pseudo-labeling technology were used to form a "image-ultrasound-electrophysiology-label" four-tuple training set, which increased the labeling efficiency by more than 3 times.
[0046] 2. Intraoperative Real-time Recognition and Navigation Process
[0047] 1.Synchronous acquisition of multimodal data
[0048] During surgery, a microscope captures optical images at 30fps, an ultrasound probe acquires blood flow signals at a sampling depth of 5-10mm, and electrophysiological electrodes simultaneously record electrical activity in the vessel wall. Hardware-synchronized triggering ensures data latency of less than 50ms (the latency indicator of "S3. Real-time multimodal processing" in Claim 5). If intraoperative bleeding occurs, near-infrared fluorescence imaging (excitation wavelength 808nm) is automatically activated to enhance vascular visualization.
[0049] 2. Dynamic neural network real-time processing
[0050] The model is fine-tuned online every 5 minutes using unlabeled data with an entropy value ≥ 1.5. The learning rate uses a cosine decay strategy (initial 0.001, lower limit 0.0005), and gradient clipping (threshold 5.0) is used to prevent overfitting. The optical branch uses overlapping sliding windows to improve the accuracy of vessel edge recognition (the "overlapping sliding window mechanism" described in Claim 3), and the ultrasound branch uses short-time Fourier transforms to identify 1-2 Hz pulsation characteristics.
[0051] 3. Three-dimensional navigation and multi-dimensional decision-making
[0052] After the multimodal features are registered in time and space (rigid + non-rigid transformation, spatial error ≤ 0.3mm), a three-dimensional probability cloud map (voxel 0.1mm) is generated. 3 ) is superimposed on the microscope screen via holographic projection, with an update frequency ≥15fps and a latency ≤100ms. Arteries are highlighted when the confidence level is ≥95% and the ultrasound PI is ≥1.5. Conflicting indicators trigger a 10-second electrophysiological monitoring (sampling rate 2kHz).
[0053] 3. Postoperative Model Optimization and Clinical Verification
[0054] 1. Federated learning drives model evolution
[0055] Postoperative data on semen quality improvement (ΔPRM ≥ 15%) and complications were collected, and the global model was updated using a federated learning framework (FedAvg algorithm). When the test set AUC value improved by ≥ 2%, the update package was automatically pushed to each hospital node, and five local update rounds were performed.
[0056] 2. Clinical effectiveness verification
[0057] A randomized controlled trial (n=400) showed that the testicular artery preservation rate in the experimental group was 98.5% (87.2% in the control group), the misligation rate was 0.8%, the average operation time was 35 minutes (shortened by 28.6%), and the complication rate was 2.1%.
[0058] IV. Correspondence between Technical Advantages and Claims
[0059] 1. Multimodal fusion improves robustness
[0060] In the hemorrhage scenario, ultrasound PI = 1.8 and electrophysiological features (Euclidean distance 0.6) make up for the recognition gap when the optical probability drops to 75%, and the artery is correctly marked.
[0061] 2. Dynamic Learning to Adapt to Anatomical Variation
[0062] During surgery for a patient with double testicular arteries, the model was fine-tuned online five times (with the learning rate decaying from 0.001 to 0.0007), improving the recognition confidence from 78% to 96%, thus avoiding mispuncture.
[0063] 3. 3D navigation improves surgical efficiency
[0064] The holographic projection three-dimensional probability cloud map shortens the time for identifying key blood vessels from 120 seconds to 45 seconds. The surgical time for a complex case is reduced by 22 minutes compared with traditional methods, corresponding to the efficient computing architecture of "S3. Real-time multimodal processing" in claim 5 (GPU+FPGA heterogeneous computing, delay ≤100ms).
[0065] This implementation strictly follows the technical features in the claims, and is highly consistent with the claims from hardware integration, algorithm flow to clinical verification, fully reflecting the technical advantages and implementation feasibility of "multimodal fusion, dynamic learning, and three-dimensional navigation."
[0066] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.
Claims
1. A testicular artery intelligent recognition system based on multimodal fusion and dynamic learning, characterized by: include: The multimodal image acquisition module integrates a Zeiss pico microscope (optical resolution 1920×1080, frame rate 30fps), a 10-15MHz high-frequency linear array ultrasound probe (axial resolution ≤0.1mm) and a 1kHz sampling rate electrophysiological microelectrode array (channel number ≥8). Through a hardware synchronization trigger circuit (synchronization accuracy ≤30μs), the module realizes the spatiotemporal synchronization of optical images, ultrasound Doppler blood flow signals (sampling depth 5-10mm) and vascular wall electrical activity signals. The module has a built-in near-infrared fluorescence imaging channel (excitation wavelength 808nm, power ≤50mW) to support indocyanine green angiography. The ultrasound probe is integrated into the microscope optical path through a six-axis robotic arm bracket with a positioning accuracy of 0.1mm, and can achieve ±30° deflection and 0-50mm axial adjustment. The dynamic adaptive neural network model is a three-branch parallel network constructed based on the Swin-UNet++ architecture. The optical feature branch uses an overlapping sliding window (window size 128×128, step size 64) combined with a multi-head self-attention mechanism (head number 8, Embedding dimension 384) extracts vascular texture and diameter change features. The ultrasound blood flow branch generates blood flow time-frequency maps through short-time Fourier transform (window length 256 points, overlap rate 75%) and combines convolutional attention module to identify arterial pulsation features (frequency domain peak 1-2Hz). The electrophysiology branch uses 1D-CNN (convolution kernel size 5×5) and bidirectional LSTM (hidden layer dimension 128) to extract action potential waveform features (amplitude ≥50μV, frequency 1-3Hz). The model supports the use of entropy value ≥1 every 5 minutes during the operation. .5 unlabeled high uncertainty samples are fine-tuned online by minimizing the cross entropy loss function. The learning rate adopts the cosine decay strategy (initial 0.001, lower limit 0.0005) and gradient clipping (threshold 5.0) is used to prevent overfitting. The three-dimensional dynamic navigation module uses a spatiotemporal registration technology that combines rigid transformation (based on SIFT feature matching) with non-rigid transformation (B-spline free deformation field) to map multimodal data to the DICOM standard three-dimensional coordinate system (spatial error ≤ 0.3mm). Based on NVIDIA A heterogeneous computing platform, built using a Jetson AGX Orin (275TOPS computing power) and a Xilinx UltraScale+ FPGA accelerator card (peak bandwidth 400GB / s), generates a 3D probability cloud map of the testicular artery (voxel size 0.1mm×0.1mm×0.1mm) in real time. This probability cloud map is overlaid on the microscope screen in pseudocolor using a holographic projection module (resolution 1920×1080, contrast 1000:1). The transparency dynamically adjusts with the confidence level (formula: α = min(1.0, confidence level × 1.2)). When the confidence level is ≥95% and the ultrasound pulsatility index (PI) is ≥1.5, the arterial contour is highlighted with a red grid line. A closed-loop optimization decision-making module is used to construct a multi-dimensional weighted decision-making model (weight coefficients were determined using preoperative training data from 200 RCTs: optical probability 0.5, ultrasound PI 0.3, and electrophysiological characteristics 0.2) Decision thresholds are optical probability heatmap confidence ≥ 90%, ultrasound PI ≥ 1.5, and electrophysiological feature vector Euclidean distance < 0.
8. Conflicting indicators automatically trigger 10 seconds of continuous electrophysiological signal monitoring (with a sampling rate increased to 2kHz). Postoperatively, a federated learning framework (using the FedAvg algorithm with five local update rounds) collects data on semen PRM improvement rates and complication rates. Local model parameters at each hospital are automatically updated when the global model's AUC on the test set improves by ≥ 2%. The system operates by synchronously acquiring optical, ultrasound, and electrophysiological data through a multimodal imaging module and transmitting them to an edge computing platform via hardware synchronization. A dynamic adaptive neural network model extracts features from each modality in parallel and fuses them using a cross-modal attention mechanism to generate an arterial probability distribution. The 3D navigation module maps the probability distribution into a 3D cloud map and registers it with the microscope image for real-time annotation. The decision module outputs a decision based on multi-dimensional indicators to assist the surgeon in identifying arteries. Postoperatively, the system automatically collects clinical data and optimizes the model through federated learning, forming a closed-loop mechanism of "data acquisition - feature fusion - real-time navigation - decision support - model evolution." .
2. The system according to claim 1, wherein: The ultrasonic probe of the multimodal image acquisition module is integrated into the microscope optical path through a mechanical arm bracket, with a positioning accuracy of 0.1 mm.
3. The system according to claim 1, wherein: The optical branch of the dynamic adaptive neural network model uses an overlapping sliding window mechanism to process high-resolution images and improve the accuracy of blood vessel edge recognition.
4. The system according to claim 1, wherein: The closed-loop optimization decision module automatically triggers a 10-second continuous electrophysiological signal monitoring process when there is a conflict in indicators (such as optical probability ≥90% but ultrasound PI <1.5).
5. A method for intelligently identifying the testicular artery based on the system according to any one of claims 1 to 4, characterized in that: include: S1. Collaborative multimodal data annotation: Using the 4D Slicer tool, we perform spatiotemporal alignment of synchronously acquired optical images, ultrasound blood flow maps, electrophysiological signals, and anatomical structures. We construct a quadrimodal label library containing preoperative CTA vascular reconstruction data from at least 200 patients as anatomical reference. Using semi-supervised learning strategies and pseudo-labeling techniques, we increase labeling efficiency by more than three times, creating a training set of the "image-ultrasound-electrophysiology-label" quadruple. S2. Dynamic fusion training: Cross-modal transfer pre-training is first performed on the MONAI public medical imaging dataset. During surgery, a "model prediction-decision verification-parameter update" cycle is followed. Online fine-tuning is performed every 5 minutes using newly acquired unlabeled data at an initial learning rate of 0.001 (using a cosine decay strategy with a lower limit of 0.0005). Postoperatively, the global model is updated using federated learning (FedAvg algorithm, with 5 local update rounds) based on the patient's semen quality improvement rate (ΔPRM ≥ 15%) and complication rate 3 months after surgery. S3. Real-time multimodal processing: Optical and ultrasound signals are acquired through hardware-synchronized triggering (latency <50ms). After parallel feature extraction via a multi-branch network, these signals are fused using an attention-gating mechanism. A 3D reconstruction algorithm is used to generate a vascular tree structure. Pulse correlation analysis is then performed in conjunction with electrophysiological signals to verify arterial properties. A GPU and FPGA heterogeneous computing architecture is employed (ultrasound Doppler signals are processed in real time by the FPGA, while optical image feature extraction is performed in parallel by the GPU), achieving a processing latency of ≤100ms. S4. Closed-loop decision support: When the confidence level of the three-dimensional probability cloud map is ≥90% and the ultrasound PI is ≥1.5, electrophysiological signal verification is triggered. If the Euclidean distance of the electrophysiological feature vector is <0.8, it is confirmed as the testicular artery and highlighted on the microscope screen. Complication data is automatically collected after surgery, and the decision weight coefficient is dynamically optimized through reinforcement learning for every 50 surgeries (the initial weight is determined by training data from 200 clinical trials before surgery).
6. The method according to claim 5, characterized in that The quad-modality label library in S1 contains preoperative CTA vascular reconstruction data of at least 200 patients as anatomical reference.
7. The method according to claim 5, characterized in that The multimodal data processing in S3 adopts a GPU and FPGA heterogeneous computing architecture, in which the ultrasound Doppler signal is processed in real time by the FPGA, and the optical image feature extraction is performed by parallel calculation on the GPU.
8. The method according to claim 5, characterized in that The initial weight coefficients of the weighted decision model in S4 are determined by training with 200 preoperative clinical trial data, and are subsequently dynamically optimized through reinforcement learning.
Citation Information
Cited By
Semen sample quality management system based on federated learning and dynamic time sequence prediction
CN121117522A
