A computer vision technology-based automatic evaluation method for blood vessel anastomosis skills

Abstract

The application provides a kind of blood vessel anastomosis skill automatic evaluation method based on computer vision technology, it is related to video processing technical field, the method is to utilize unlabelled blood vessel anastomosis operation video data, visual feature extraction model is trained, obtains pre-trained visual feature extraction model, blood vessel anastomosis operation video data is extracted, and blood vessel anastomosis skill visual feature is obtained;Blood vessel anastomosis skill visual feature is up-sampled using high-resolution pyramid, and high spatial resolution feature map is obtained;Using a variety of downstream task models, high spatial resolution feature map is operated action class identification, target segmentation and tip position identification;Action time consumption is calculated using operation action class, motion trajectory is obtained using instrument tip position, suture binary mask is identified using target segmentation, multidimensional index is calculated, and blood vessel anastomosis skill automatic evaluation result is obtained.The application solves the problem that the prior art is difficult to improve small target recognition accuracy.

Description

Technical Field

[0001] This specification relates to the field of video processing technology, and in particular to an automated assessment method for vascular anastomosis skills based on computer vision technology. Background Technology

[0002] Vascular anastomosis is a core surgical technique in cardiovascular surgery, microsurgery, and other fields, requiring surgeons to possess a high degree of precision and stability. Traditional training methods mainly rely on expert manual assessment, which suffers from problems such as strong subjectivity, inconsistent assessment standards, difficulty in quantification, and low training efficiency. In recent years, offline training systems, such as the low-cost plastic basin anastomosis simulation system developed in Japan, have assessed skills by quantifying the coefficient of variation (CV) of operation time, pitch, and bite, but still require manual measurement and scoring, and cannot achieve real-time feedback. In addition, there is a study that uses the Mask R-CNN model to detect and segment blood vessels and sutures in vascular anastomosis images (Quantitative Outcome-Oriented Assessment of Microsurgical Anastomosis), but it can only perform offline, one-dimensional "outcome assessment" of postoperative static photographs. There are also patents involving vascular anastomosis devices or auxiliary devices, but less attention is paid to vision-based automated training assessment. Although there are video target segmentation and video action recognition methods based on self-supervised learning, as well as techniques for analyzing the movement trajectory of surgical instruments, there is still no solution to systematically integrate these techniques and optimize them specifically for vascular anastomosis training scenarios. In particular, there is a lack of strategies for region locking centered on recipient vessels to improve the accuracy of small target recognition.

[0003] Current assessment techniques used for vascular anastomosis skills training have many limitations, failing to meet the requirements of modern surgical education for objectivity, real-time performance, and comprehensiveness. These limitations are mainly reflected in the following aspects: **High Subjectivity and Lack of Quantitative Standards:** Traditional assessments heavily rely on the on-site observation and subjective scoring of senior physicians (such as the OPRS scoring system). Inconsistent standards among assessors and susceptibility to personal experience and preferences result in a lack of objective, unified quantitative indicators, making horizontal comparisons and longitudinal tracking difficult. **Lagging Assessment and Inability to Provide Real-Time Feedback:** While existing mainstream offline training systems (such as those described in PMC6326066) incorporate quantitative indicators such as operation time, stitch length, and edge variation coefficient, manual measurement and calculation are still required after training. This lack of immediate, dynamic feedback during the trainee's operation significantly reduces learning efficiency. **Single and Incomplete Assessment Dimensions:** Some automated attempts, such as using the Mask R-CNN model to analyze static images, can only provide a "results-oriented" single-dimensional assessment of the final suture morphology. They fail to capture and analyze crucial information during the procedure, such as instrument movement trajectories and operational smoothness, which are essential for comprehensive skill improvement. Low technology integration and lack of targeted optimization: While advanced technologies in computer vision exist, such as video action recognition, self-supervised learning, and instrument trajectory tracking, these technologies have not been systematically integrated or specifically optimized for the high-precision surgical scenario of "vascular anastomosis." In particular, the lack of a dynamic region-locking mechanism centered on the recipient vessel results in insufficient accuracy in recognizing minute critical targets such as needle tips and sutures in complex backgrounds and rapid operations, making it difficult to support high-precision geometric parameter calculations (such as needle insertion angle). Strong data dependence and high cost: Many deep learning-based solutions require large amounts of precisely labeled video or image data for supervised training. However, obtaining high-quality surgical operation annotation data is extremely costly and time-consuming, severely restricting model development, iteration, and widespread application across different institutions. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides an automated evaluation method for vascular anastomosis skills based on computer vision technology. This method utilizes a "two-stage adaptive target region locking" strategy and a multi-task collaborative mechanism to solve the problem of existing technologies' difficulty in improving the accuracy of small target recognition.

[0005] To achieve the aforementioned objectives, the present invention employs the following technical solution: an automated assessment method for vascular anastomosis skills based on computer vision technology.

[0006] S1: Using unlabeled vascular anastomosis surgery video data, a self-supervised learning method was used to pre-train the visual feature extraction model to obtain the pre-trained visual feature extraction model.

[0007] S2: Use a pre-trained visual feature extraction model to extract features from vascular anastomosis surgery video data to obtain visual features of vascular anastomosis skills;

[0008] S3: Upsample the visual features of vascular anastomosis skills using a high-resolution pyramid to obtain a high spatial resolution feature map;

[0009] S4: Using multiple downstream task models, action recognition, target segmentation, and tip position recognition are performed on the high spatial resolution feature map to obtain the probability heatmaps of the action category, binary mask, and tip position in the current frame, respectively. The action category with the highest probability in the current frame is taken as the operation action category. Among them, multiple downstream task models are obtained through joint training based on the probability heatmaps of the action category, binary mask, and tip position in the current frame.

[0010] S5: Calculate the action time based on the type of operation action; analyze the instrument tip position based on the binary mask and the probability heat map of the tip position to calculate the motion trajectory and suture binary mask, calculate multi-dimensional indicators, obtain the automated evaluation results of vascular anastomosis skills, and complete the automated evaluation of vascular anastomosis skills.

[0011] The beneficial effects of the present invention are as follows: The present invention provides an automated evaluation method for vascular anastomosis skills based on computer vision technology. Through self-supervised pre-training, two-stage adaptive target region locking, multi-task collaborative neural network architecture and customized loss function system that integrates clinical prior knowledge, it realizes end-to-end, multi-dimensional, and real-time automated evaluation of operation process, suture geometry parameters and instrument movement trajectory. (1) Key point heatmap regression is used to replace bounding box detection to achieve sub-pixel level positioning and naturally support multi-instrument tip recognition. (2) Based on the action recognition results, a spatial attention map centered on the blood vessel edge is dynamically generated, and HR-FPN features are modulated to achieve semantic alignment between tasks and effectively suppress false detection. (3) Through the needle distance variance and edge distance deviation term, the model outputs a suture point distribution that conforms to clinical norms. By using surgical rules such as "the needle tip must be close to the blood vessel when inserting the needle", a differentiable supervision signal is constructed to improve the anatomical rationality of the detection results. (4) With the recipient blood vessel as the center, a fixed-size ROI is dynamically locked for all downstream task inputs, which significantly improves the signal-to-noise ratio of small targets. The ROI is further reduced according to the action recognition results, so as to realize the fine calculation of high-precision geometric parameters such as needle insertion / exit angle.

[0012] Further, S4 includes:

[0013] By utilizing the action recognition branch of the downstream task model, action recognition is performed on the visual features of vascular anastomosis skills to obtain the action category probability of the current frame.

[0014] By utilizing the target segmentation branch of the downstream task model, target segmentation is performed on the high spatial resolution feature map to obtain a binary mask.

[0015] Cross-task attention fusion is performed on the action category probabilities of the current frame to obtain a dynamically enhanced attention map;

[0016] Based on dynamic enhanced attention maps, the instrument head detection sub-model of the downstream task model is used to identify the tip position through key point heatmap regression, and obtain the probability heatmap of the tip position.

[0017] Furthermore, the expression for the loss function of the downstream task model is:

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] in, Indicates the total loss. This indicates the loss in action recognition. Indicates the target segmentation loss. This indicates the loss detected by the instrument head. Represents geometric constraint loss. This indicates cross-task consistency loss. The weights represent the action recognition loss. The weights represent the target segmentation loss. This indicates the weight of the detection loss by the instrument head. The weights representing the geometric constraint loss, The weights representing the cross-task consistency loss Represents the cross-entropy loss function. Indicates the actual action category, This indicates the action category output by the downstream task model. The loss represents the degree of overlap between the prediction and the actual region. This represents the actual label mask. This represents the mask used for predictions by the downstream task model. This represents the classification error for each pixel. This represents the number of samples in the batch, where i represents the i-th sample in the batch. This represents the true heatmap of the i-th sample. This represents the predicted heatmap for the i-th sample. Denotes the square of the L2 norm. This indicates the calculation of the variance of the set. This represents the predicted coordinates of the j-th seam point. This represents the predicted coordinates of the (j+1)th seam point. Represents Euclidean distance. Represents the average of the set. Represents Euclidean distance. Indicates the edge of the vascular anastomosis. This represents the ideal margin value. Indicates an indicator function, Indicates the first Frame action category prediction results Represents the ReLU function. This represents the predicted coordinates of the needle tip. This indicates the point on the edge of the blood vessel closest to the needle tip. This indicates the tolerance threshold.

[0025] Furthermore, the action recognition branch of the downstream task model is used to perform action recognition on the visual features of vascular anastomosis skills to obtain the action category probability of the current frame, including:

[0026] A pre-trained visual feature extraction model was used to analyze video data of vascular anastomosis surgery to obtain visual features of vascular anastomosis skills.

[0027] By utilizing the action recognition branch of the downstream task model, action recognition is performed on the visual features of vascular anastomosis skills to locate the center coordinates of the recipient vessel and obtain the recipient vessel localization result.

[0028] Based on the receptor vascular localization results, the region is expanded and locked to obtain the locked region;

[0029] Dynamic tracking and operation action recognition are performed on the locked area to obtain the action category probability of the current frame.

[0030] By utilizing the identified action categories, the ROI region is adaptively locked, improving the accuracy of downstream small target recognition subtasks.

[0031] Further, S5 includes:

[0032] Calculate the action time based on the action type;

[0033] Based on the binary mask and the probability heatmap of the tip position, the motion trajectory and suture binary mask are calculated by analyzing the tip position of the instrument. The motion trajectory, trajectory parameters and suture geometric parameters of the instrument tip are calculated for the operation action category, and the action score, motion score and geometric score are obtained respectively.

[0034] Based on action score, motion score, and geometric score, a comprehensive quality assessment index is calculated by weighting to obtain the automated assessment results of vascular anastomosis skills, thus completing the automated assessment of vascular anastomosis skills.

[0035] Furthermore, the trajectory parameters include path length and average speed, wherein the expression for path length is:

[0036] ;

[0037] The expression for average velocity is:

[0038] ;

[0039] The expression for the automated assessment result of vascular anastomosis skills is:

[0040] ;

[0041] in, Indicates the path length. and Represents the two-dimensional coordinates of the instrument tip at time t+1. and This represents the two-dimensional coordinates of the instrument tip at time t, where t represents the chronological index of the data point. Indicates average speed. This represents the total number of data points in the entire trajectory. Indicates the video frame rate. This indicates the results of the automated assessment of vascular anastomosis skills. Indicates action score, Indicates the sports score. Represents geometric scoring. This indicates the weight of the action score. Indicates the weight of the exercise score. This represents the weight of the geometric score. Attached Figure Description

[0042] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0043] Figure 1This is an exemplary flowchart illustrating an automated assessment method for vascular anastomosis skills based on computer vision technology, according to some embodiments of this specification.

[0044] Figure 2 This is a schematic diagram of the framework of an automated assessment method for vascular anastomosis skills based on computer vision technology, as shown in some embodiments of this specification. Detailed Implementation

[0045] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0046] Example

[0047] Figure 1 This is an exemplary flowchart illustrating an automated assessment method for vascular anastomosis skills based on computer vision technology, according to some embodiments of this specification. Figure 1 and Figure 2 As shown, the process includes the following steps. In some embodiments, the process may be executed by a processor.

[0048] S1: Using unlabeled vascular anastomosis surgery video data, a self-supervised learning method was used to pre-train the visual feature extraction model to obtain the pre-trained visual feature extraction model.

[0049] Video data of vascular anastomosis surgery is video data related to the vascular anastomosis surgery task.

[0050] The visual feature extraction model is a common visual feature extraction model such as Vision Transformer Base.

[0051] In some embodiments, the processor can crop the vascular anastomosis surgery video data to obtain a ROI cropped image of size 384×384×3, input it into the visual feature extraction model, initialize it with BYOL self-supervised pre-trained weights, and obtain 14×14 visual features of vascular anastomosis skills with 768 channels.

[0052] S2: Use a pre-trained visual feature extraction model to extract features from vascular anastomosis surgery video data to obtain visual features of vascular anastomosis skills.

[0053] The visual features of vascular anastomosis skills are feature maps (ViT) that are related to vascular anastomosis skills, have 768 channels, and are 14×14 in size.

[0054] S3: High-resolution pyramids are used to upsample the visual features of vascular anastomosis skills to obtain high spatial resolution feature maps.

[0055] High spatial resolution feature maps are feature maps with a resolution of 96×96×256.

[0056] In some embodiments, since the visual feature resolution of vascular anastomosis skills is low, which is not conducive to the localization of small targets, the high-resolution feature pyramid (HR-FPN) is used to upsample and align the outputs of the last 4 stages of ViT (different resolutions) to 1 / 4 of the original image resolution. The high spatial resolution feature map (96×96×256) is output by lateral connection + 3×3 convolution fusion, which facilitates subsequent segmentation and detection.

[0057] S4: Using multiple downstream task models, action recognition, target segmentation, and tip position recognition are performed on the high spatial resolution feature map to obtain the probability heatmaps of the action category, binary mask, and tip position in the current frame, respectively. The action category with the highest probability in the current frame is taken as the operation action category. Among them, multiple downstream task models are obtained through joint training based on the probability heatmaps of the action category, binary mask, and tip position in the current frame.

[0058] Downstream task models include the action recognition branch (Temporal Convolutional Network) (TCN model), the object segmentation branch (lightweight U-Net decoder), and the instrument head detection model (key point heatmap regression).

[0059] The probability of the action category in the current frame includes the probability of needle insertion and the probability of needle exit.

[0060] A binary mask is a mask that reflects the edge of a stitch.

[0061] The probability heatmap of the tip position is a probability heatmap of the peak coordinate position, used to reflect the probability of the instrument tip.

[0062] In some embodiments, the processor can utilize the action recognition branch of the downstream task model to perform action recognition on the visual features of vascular anastomosis skills to obtain the action category probability of the current frame; utilize the target segmentation branch of the downstream task model to perform target segmentation on the high spatial resolution feature map to obtain a binary mask; perform cross-task attention fusion on the action category probability of the current frame to obtain a dynamically enhanced attention map; and based on the dynamically enhanced attention map, utilize the instrument head detection sub-model of the downstream task model to perform tip position recognition through key point heatmap regression to obtain a probability heatmap of the tip position.

[0063] In some embodiments, the processor can utilize the action recognition branch of the downstream task model to analyze the visual features of vascular anastomosis skills, identify key operation steps such as "needle insertion," "needle withdrawal," "needle transfer," and "knotting," and obtain the action category probability of the current frame; utilize the target segmentation branch of the downstream task model (a lightweight U-Net decoder with 4 layers of upsampling) to perform target segmentation on the high spatial resolution feature map, accurately segment the "vascular anastomosis" region, and identify the "suture edge" for subsequent calculation of geometric parameters such as needle distance and edge distance, and obtain a 384×384 binary mask; utilize the instrument head detection sub-model of the downstream task model to perform action recognition on the action category probability and binary mask of the current frame through key point heatmap regression, detect and locate the tip position of key surgical instruments (such as "needle holder" and "forceps"), and obtain a probability heatmap (96×96×1) of the tip position; wherein, the key point heatmap regression includes 3 layers of Conv (256→128→64→1).

[0064] By utilizing a large number of unlabeled surgical videos, the ViT backbone network is pre-trained through BYOL self-supervised contrastive learning, which significantly reduces the dependence on expensive pixel-level / frame-level manually labeled data and improves the model's generalization ability and development efficiency. The resulting probabilistic heatmap of the tip position has sub-pixel accuracy, is robust to occlusion, and naturally supports multiple instrument tips (strong scalability).

[0065] In some embodiments, the processor can jointly train multiple downstream task models to obtain a trained downstream task model.

[0066] In some embodiments, the processor can select a spatial attention map from a preset attention template library that matches the semantics of the current operation based on the action category output by the action recognition model. For example, when the recognition result is 'needle insertion', the system automatically selects a Gaussian attenuation weight map centered on the edge of the blood vessel; when the recognition result is 'knot tying', it selects a weight map centered on the suture intersection area. This attention map is multiplied element-wise with the HR-FPN features to achieve dynamic enhancement of key regions before being fed into the segmentation / detection head.

[0067] By dynamically modulating the attention of the segmentation / detection branches using action recognition results, semantic alignment between tasks can be achieved, which can reduce false detections (such as mistaking tweezers for needle tips).

[0068] In some embodiments, the downstream task model can be obtained by jointly training multiple labeled training samples. For example, multiple labeled training samples can be input into the initial downstream task model, and a loss function can be constructed using the labels and the results of the initial downstream task model. Based on the loss function, the parameters of the initial downstream task model are iteratively updated using gradient descent or other methods. When a preset condition is met, the model training is complete, and the trained downstream task model is obtained. The preset condition may be that the loss function converges, the number of iterations reaches a threshold, etc.

[0069] In some embodiments, training samples may include historical vascular anastomosis surgery video data. Labels may be corresponding operative action categories. Labels may be manually annotated.

[0070] In some embodiments, the expression for the loss function of the downstream task model is:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] in, Indicates the total loss. This indicates the loss in action recognition. Indicates the target segmentation loss. This indicates the loss detected by the instrument head. Represents geometric constraint loss. This indicates cross-task consistency loss. The weights represent the action recognition loss. The weights represent the target segmentation loss. This indicates the weight of the detection loss by the instrument head. The weights representing the geometric constraint loss, The weights representing the cross-task consistency loss Represents the cross-entropy loss function. Indicates the actual action category, This indicates the action category output by the downstream task model. The loss represents the degree of overlap between the prediction and the actual region. This represents the actual label mask. This represents the mask used for predictions by the downstream task model. This represents the classification error for each pixel. This represents the number of samples in the batch, where i represents the i-th sample in the batch. This represents the true heatmap of the i-th sample. This represents the predicted heatmap for the i-th sample. Denotes the square of the L2 norm. This indicates the calculation of the variance of the set. This represents the predicted coordinates of the j-th seam point. This represents the predicted coordinates of the (j+1)th seam point. Represents Euclidean distance. Represents the average of the set. Represents Euclidean distance. Indicates the edge of the vascular anastomosis. This represents the ideal margin value. Indicates an indicator function, Indicates the first Frame action category prediction results Represents the ReLU function. This represents the predicted coordinates of the needle tip. This indicates the point on the edge of the blood vessel closest to the needle tip. This indicates the tolerance threshold.

[0078] Dice Loss focuses on global overlap, while BCE Loss focuses on local pixel classification. Combining the two can simultaneously optimize macroscopic and microscopic structures. When the foreground (such as sutures) occupies a very small proportion, BCE Loss may be dominated by the background, while Dice Loss forces the model to focus on the foreground region. BCE Loss provides stable gradients, while Dice Loss provides stronger semantic constraints; using them together can accelerate convergence and avoid getting trapped in local optima. Accurate segmentation of the "suture edge" is crucial. Dice Loss is sensitive to boundaries, while BCE Loss can refine the probability transition at the boundary. In clinical practice, the ideal stitch interval (SI) is typically 0.8–1.2 mm, requiring small differences between adjacent stitch intervals (i.e., high consistency). Large fluctuations in stitch interval can lead to uneven tension, bleeding, or stenosis. Therefore, a "low coefficient of variation" in stitch interval is itself an important indicator of skill proficiency. This encourages the model to predict suture points that are generally located at an ideal "bite" distance, avoiding them being too close or too far. In vascular anastomosis, the "edge distance" (ED) refers to the distance from the needle insertion point to the vessel wall, with an ideal value typically around 1 mm. Too large or too small a distance can affect the quality of the suture. The purpose of this loss term is to "pull" the model's predictions toward clinical standard values, making its output more consistent with actual operational procedures.

[0079] S3: Analyze the probability of the action category, the binary mask, and the probability heatmap of the tip position in the current frame to obtain the action category.

[0080] The operation category refers to the types of operations performed during vascular anastomosis surgery. For example, operation categories may include "needle insertion," "needle withdrawal," "needle transfer," and "knot tying," etc.

[0081] In some embodiments, the action recognition branch of the downstream task model is used to perform action recognition on the visual features of vascular anastomosis skills to obtain the action category probability of the current frame. This includes: analyzing vascular anastomosis surgery video data using a pre-trained visual feature extraction model to obtain visual features of vascular anastomosis skills; using the action recognition branch of the downstream task model to perform action recognition on the visual features of vascular anastomosis skills, locating the center coordinates of the recipient vessel, and obtaining the recipient vessel localization result; based on the recipient vessel localization result, performing region expansion and locking to obtain the locked region; and performing dynamic tracking and operation action recognition on the locked region to obtain the action category probability of the current frame.

[0082] The locked region is a square or rectangular area of ​​fixed size, extending outward from the located vessel center (cx, cy) (e.g., with a side length of 1 / 4 or 1 / 3 of the original image width). This region is the "Region of Interest" (ROI) for subsequent analysis. ROI = [cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2], where w and h are the preset width and height of the ROI.

[0083] In some embodiments, if the blood vessel may move slightly during the operation, a simple tracking algorithm (such as optical flow-based or Kalman filtering) can be introduced to predict the position of the blood vessel in the current frame based on the position of the blood vessel in the previous frame and dynamically adjust the ROI center to ensure that the blood vessel is always located near the ROI center.

[0084] In some embodiments, the processor can crop out the ROI region defined above for each frame of the video, and then input this ROI image (instead of the entire image) into the trained action recognition model. This greatly reduces background interference and allows small targets to occupy a larger relative proportion in the input image, thereby improving recognition accuracy. The ROI region is further adaptively adjusted according to the type of action being recognized. For example, when recognizing "needle insertion" and "needle withdrawal" actions, the ROI region needs to be further reduced for more refined determination of the needle insertion and withdrawal angles.

[0085] For tiny, critical targets such as needle tips and sutures, a two-stage adaptive target region locking strategy is proposed: the first stage dynamically locks and fixes the ROI centered on the recipient vessel to improve the signal-to-noise ratio; the second stage further focuses on the local region based on the action recognition results (such as "needle insertion"), supporting high-precision calculation of millimeter-level geometric parameters such as needle insertion / exit angles. Simultaneously, keypoint heatmap regression is used to replace traditional bounding box detection, and a high-resolution feature pyramid HR-FPN is introduced to significantly improve the localization accuracy of small targets.

[0086] S5: Based on computer vision technology, the index of operation action category is calculated to obtain the automated evaluation result of vascular anastomosis skills and complete the automated evaluation of vascular anastomosis skills.

[0087] The automated assessment results of vascular anastomosis skills are the results of automated scoring and feedback of the operational quality of vascular anastomosis skills.

[0088] In some embodiments, the processor can calculate the action time based on the type of operation action; analyze the instrument tip position to calculate the motion trajectory and suture binary mask based on the binary mask and the probability heat map of the tip position; calculate the instrument tip motion trajectory, trajectory parameters, and suture geometric parameters for the type of operation action to obtain the action score, motion score, and geometric score, respectively; and calculate the comprehensive quality assessment index by weighting the action score, motion score, and geometric score to obtain the automated assessment result of vascular anastomosis skills, thus completing the automated assessment of vascular anastomosis skills.

[0089] Motion scoring is a score calculated based on the movement trajectory of the instrument tip.

[0090] Motion scoring is a score calculated based on trajectory parameters.

[0091] Geometric scoring is a score calculated based on suture geometry parameters. These parameters can include stitch length, edge distance, and needle entry angle, among others.

[0092] In some embodiments, the processor can calculate kinematic parameters such as the trajectory, speed, and path length of instruments (e.g., needle holders, tweezers) based on the instrument tip position sequence output by the target detection model. Combining the motion recognition results with suture geometry parameters (stitch distance, edge distance), a comprehensive evaluation index is constructed to automatically score and provide feedback on the trainee's operational quality.

[0093] In some embodiments, the processor can obtain the pixel coordinates (x_t, y_t) of the "needle tip" and "tweezers tip" in each frame using an object detection model. Connecting the coordinates of consecutive frames forms the original motion trajectory sequence. To eliminate detection noise, the original trajectory is smoothed using algorithms such as moving average filtering, Savitzky-Golay filtering, or Kalman filtering.

[0094] In some embodiments, the trajectory parameters include path length and average speed, wherein the expression for path length is:

[0095] ;

[0096] in, Indicates the path length. and Represents the two-dimensional coordinates of the instrument tip at time t+1. and This represents the two-dimensional coordinates of the instrument tip at time t, where t represents the index of the data point in chronological order.

[0097] In some embodiments, the expression for average velocity is:

[0098] ;

[0099] in, Indicates average speed. This represents the total number of data points in the entire trajectory. This indicates the video frame rate.

[0100] In some embodiments, the processor can obtain the precise contour of the "suture edge" through a target segmentation model. The straight-line distance between two adjacent needle entry points (or exit points) is calculated as the stitch interval (SI). The distance from each needle entry point (or exit point) to the nearest "vascular anastomosis" edge is calculated as the edge distance (ED).

[0101] In some embodiments, the expression for the automated assessment result of vascular anastomosis skills is:

[0102] ;

[0103] in, This indicates the results of the automated assessment of vascular anastomosis skills. Indicates action score, Indicates the sports score. Represents geometric scoring. This indicates the weight of the action score. Indicates the weight of the exercise score. This represents the weight of the geometric score.

[0104] It can build an assessment process that requires no human intervention, abandoning the traditional method of relying on subjective scoring by experts, and providing objective, repeatable, and comparable scoring standards through quantifiable action recognition results, geometric parameters, and kinematic indicators.

[0105] By unifying action recognition, suture geometry quality, and other three dimensions into an end-to-end framework, and achieving semantic alignment between tasks through a cross-task attention fusion mechanism and customized loss functions (such as geometric constraint loss and cross-task consistency loss), the evaluation results are ensured to conform to both operational logic and clinical anatomy standards, providing trainees with comprehensive, accurate, and real-time training feedback.

[0106] In some embodiments of this specification, an automated assessment method for vascular anastomosis skills based on computer vision technology is provided. Through self-supervised pre-training, two-stage adaptive target region locking, multi-task collaborative neural network architecture, and a customized loss function system that integrates clinical prior knowledge, an end-to-end, multi-dimensional, and real-time automated assessment of the operation process, suture geometry parameters, and instrument movement trajectory is achieved. (1) Key point heatmap regression is used to replace bounding box detection to achieve sub-pixel-level localization and naturally support the recognition of multiple instrument tips. (2) Based on the action recognition results, a spatial attention map centered on the blood vessel edge is dynamically generated, and HR-FPN features are modulated to achieve semantic alignment between tasks and effectively suppress false detections. (3) Through the needle distance variance and edge distance deviation terms, the model outputs a suture point distribution that conforms to clinical norms. By using surgical rules such as "the needle tip must be close to the blood vessel when inserting the needle", a differentiable supervision signal is constructed to improve the anatomical rationality of the detection results. (4) With the recipient blood vessel as the center, a fixed-size ROI is dynamically locked for all downstream task inputs, which significantly improves the signal-to-noise ratio of small targets. The ROI is further reduced according to the action recognition results, so as to realize the fine calculation of high-precision geometric parameters such as needle insertion / exit angle.

Claims

1. An automated assessment method for vascular anastomosis skills based on computer vision technology, characterized in that, include: S1: Using unlabeled vascular anastomosis surgery video data, a self-supervised learning method was used to pre-train the visual feature extraction model to obtain the pre-trained visual feature extraction model. S2: Use a pre-trained visual feature extraction model to extract features from vascular anastomosis surgery video data to obtain visual features of vascular anastomosis skills; S3: Upsample the visual features of vascular anastomosis skills using a high-resolution pyramid to obtain a high spatial resolution feature map; S4: Using multiple downstream task models, action recognition, target segmentation, and tip position recognition are performed on the high spatial resolution feature map to obtain the probability heatmaps of the action category, binary mask, and tip position in the current frame, respectively. The action category with the highest probability in the current frame is taken as the operation action category. Among them, multiple downstream task models are obtained through joint training based on the probability heatmaps of the action category, binary mask, and tip position in the current frame. S5: Calculate the action time based on the type of operation action; analyze the instrument tip position based on the binary mask and the probability heat map of the tip position to calculate the motion trajectory and suture binary mask, calculate multi-dimensional indicators, obtain the automated evaluation results of vascular anastomosis skills, and complete the automated evaluation of vascular anastomosis skills.

2. The automated assessment method for vascular anastomosis skills based on computer vision technology according to claim 1, characterized in that, S4 includes: By utilizing the action recognition branch of the downstream task model, action recognition is performed on the visual features of vascular anastomosis skills to obtain the action category probability of the current frame. By utilizing the target segmentation branch of the downstream task model, target segmentation is performed on the high spatial resolution feature map to obtain a binary mask. Cross-task attention fusion is performed on the action category probabilities of the current frame to obtain a dynamically enhanced attention map; Based on dynamic enhanced attention maps, the instrument head detection sub-model of the downstream task model is used to identify the tip position through key point heatmap regression, and obtain the probability heatmap of the tip position.

3. The automated assessment method for vascular anastomosis skills based on computer vision technology according to claim 1, characterized in that, The expression for the loss function of the downstream task model is: ； ； ； ； ； ； in, Indicates the total loss. This indicates the loss in action recognition. Indicates the target segmentation loss. This indicates the loss detected by the instrument head. Represents geometric constraint loss. This indicates cross-task consistency loss. The weights represent the action recognition loss. The weights represent the target segmentation loss. This indicates the weight of the detection loss by the instrument head. The weights representing the geometric constraint loss, The weights representing the cross-task consistency loss Represents the cross-entropy loss function. Indicates the actual action category, This indicates the action category output by the downstream task model. The loss represents the degree of overlap between the prediction and the actual region. This represents the actual label mask. This represents the mask used for predictions by the downstream task model. This represents the classification error for each pixel. This represents the number of samples in the batch, where i represents the i-th sample in the batch. This represents the true heatmap of the i-th sample. This represents the predicted heatmap for the i-th sample. Denotes the square of the L2 norm. This indicates the calculation of the variance of the set. This represents the predicted coordinates of the j-th seam point. This represents the predicted coordinates of the (j+1)th seam point. Represents Euclidean distance. Represents the average of the set. Represents Euclidean distance. Indicates the edge of the vascular anastomosis. This represents the ideal margin value. Indicates an indicator function, Indicates the first Frame action category prediction results Represents the ReLU function. This represents the predicted coordinates of the needle tip. This indicates the point on the edge of the blood vessel closest to the needle tip. This indicates the tolerance threshold.

4. The automated assessment method for vascular anastomosis skills based on computer vision technology according to claim 2, characterized in that, The action recognition branch of the downstream task model is used to perform action recognition on the visual features of vascular anastomosis skills, and the probability of the action category in the current frame is obtained as follows: A pre-trained visual feature extraction model was used to analyze video data of vascular anastomosis surgery to obtain visual features of vascular anastomosis skills. By utilizing the action recognition branch of the downstream task model, action recognition is performed on the visual features of vascular anastomosis skills to locate the center coordinates of the recipient vessel and obtain the recipient vessel localization result. Based on the receptor vascular localization results, the region is expanded and locked to obtain the locked region; Dynamic tracking and operation action recognition are performed on the locked area to obtain the action category probability of the current frame.

5. The automated assessment method for vascular anastomosis skills based on computer vision technology according to claim 1, characterized in that, S5 includes: Calculate the action time based on the action type; Based on the binary mask and the probability heatmap of the tip position, the motion trajectory and suture binary mask are calculated by analyzing the tip position of the instrument. The motion trajectory, trajectory parameters and suture geometric parameters of the instrument tip are calculated for the operation action category, and the action score, motion score and geometric score are obtained respectively. Based on action score, motion score, and geometric score, a comprehensive quality assessment index is calculated by weighting to obtain the automated assessment results of vascular anastomosis skills, thus completing the automated assessment of vascular anastomosis skills.

6. The automated assessment method for vascular anastomosis skills based on computer vision technology according to claim 5, characterized in that, The trajectory parameters include path length and average speed, wherein the expression for path length is: ； The expression for average velocity is: ； The expression for the automated assessment result of vascular anastomosis skills is: ； in, Indicates the path length. and Represents the two-dimensional coordinates of the instrument tip at time t+1. and This represents the two-dimensional coordinates of the instrument tip at time t, where t represents the chronological index of the data point. Indicates average speed. This represents the total number of data points in the entire trajectory. Indicates the video frame rate. This indicates the results of the automated assessment of vascular anastomosis skills. Indicates action score, Indicates the sports score. Represents geometric scoring. This indicates the weight of the action score. Indicates the weight of the exercise score. This represents the weight of the geometric score.

Images