A Robotic Surgery Soft Tissue Tracking Method Based on Long-Short Time Cyclic Consistency

Abstract

This application relates to a robotic surgical soft tissue tracking method based on long-short-term cyclic consistency. In the offline training phase, the method extracts features by constructing a feature extraction network containing short-term channel correlation modules, which can enhance the recognition ability of low-contrast soft tissue and achieve accurate differentiation of target areas, thereby extracting surgical image features with a small number of parameters. At the same time, a multi-frame sampling strategy is adopted in the forward-backward tracking cycle to reduce errors caused by occlusion or artifacts, and the cyclic response error caused by long-term tracking is reduced by long-term cyclic consistency loss, thereby enhancing the temporal consistency of soft tissue over a long period of time.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a robotic surgical soft tissue tracking method based on long-short-term cyclic consistency. Background Technology

[0002] Currently, robot-assisted minimally invasive surgery is widely used in clinical practice due to its advantages such as less intraoperative bleeding, high surgical stability, and rapid postoperative recovery. During robotic surgical navigation, surgical instruments frequently interact with soft tissues to complete specific tasks. However, human organs and soft tissues lack distinct textures and are easily deformed. Furthermore, motion blur, changes in lighting, and artifacts such as blood during surgery often affect soft tissue tracking, making real-time localization and deformation estimation highly prone to errors. This situation can prevent surgeons from obtaining accurate information, increasing surgical risks and endangering the patient's life and health. Summary of the Invention

[0003] Therefore, it is necessary to provide a robotic surgical soft tissue tracking method based on long-short time cycle consistency, which includes: an offline training phase,

[0004] S1: Randomly select a video frame from the surgical video as the training video segment;

[0005] S2: Select soft tissue in the initial frame of the training video segment as the forward tracking target, and use a discriminative correlation filter to generate the corresponding first label for the forward tracking target;

[0006] S3: Input the forward-tracking target into a feature extraction network containing a short-time channel correlation module to obtain the first target image features; calculate the discriminative correlation filter of the initial frame based on the first target image features and the first label of the initial frame;

[0007] S4: For any subsequent frame in the training video segment, calculate the first response map of the current frame based on the discriminant correlation filter obtained from the previous frame and the first target image features extracted from the current frame; take the peak position in the first response map of the current frame as the position of the forward tracking target in the current frame and output it; take the first response map of the current frame as the first pseudo-label of the first target image features of the current frame, and calculate the discriminant correlation filter of the current frame with the first target image features of the current frame;

[0008] S5: Repeat S4 in the order of video frames in the training video segment to perform forward tracking until all video frames in the training video segment have been traversed; take the last frame of the training video segment as the initial frame for backward tracking, and take the forward tracking target and the corresponding first pseudo-label in the current frame determined by the last frame of the training video segment as the initial target and the corresponding initial first label for backward tracking, and perform backward tracking. The backward tracking is to repeat S4-S5 in reverse order of video frames in the training video segment until all video frames in the training video segment have been traversed.

[0009] S6: Based on the first response map obtained from forward tracking and the second response map obtained from backward tracking in each frame of the training video segment, construct a loss function and minimize the loss function to optimize the forward tracking process.

[0010] Preferably, generating a corresponding first label for the forward-tracking target using a discriminative correlation filter includes:

[0011] The discriminant correlation filter performs correlation operations on the first target image features to generate a Gaussian response map. Each pixel in the Gaussian response map represents the degree of matching between the corresponding position and the first target image features. The generated Gaussian response map is used as the first label corresponding to the first target image features.

[0012] Preferably, the feature extraction network includes a shallow network and a short-time channel correlation module;

[0013] The shallow network consists of two 3×3 convolutional layers, used to extract low-level features from the input image;

[0014] In the short-time channel correlation module, low-level features are passed through a global average pooling layer to obtain pooled features. The pooled features are then passed sequentially through parallel convolutional layers with kernel sizes of 3, 5, and 7. The results of each convolution are summed pixel by pixel, and the sum is activated to obtain activation weights. The activation weights are then multiplied element-wise with the low-level features, and the multiplication result is residually concatenated with the low-level features to obtain the output features of the short-time channel correlation module.

[0015] Preferably, the formula for calculating the discriminant correlation filter is:

[0016] ;

[0017] in, Indicates the first training video segment Discriminative correlation filter for frames; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first label corresponding to the first target image feature of the frame; This represents the first regularization parameter.

[0018] Preferably, the expression for the loss function is:

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] in, Represents the loss function; This indicates the number of video frames in the training video segment; Indicates smoothL1 loss; The backward tracing yields the first... The second response map of the frame; The first digit obtained by forward tracing The first response graph of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first target image feature of the frame; Indicates the first training video segment Discriminative correlation filter for frames; Indicates the first training video segment Discriminative correlation filter for frames; This indicates taking the absolute value of each element.

[0024] Preferably, the surgical videos are taken from the SurgT dataset and the OBRDataset dataset.

[0025] Preferably, it also includes an online tracking phase:

[0026] Step 1: Acquire the second surgical video to be detected, select the soft tissue target to be tracked and its corresponding label in the first frame of the second surgical video, and use the image patch around the soft tissue target as a negative sample;

[0027] Step 2: Based on the feature extraction network, extract the second target image features and negative sample image features from the soft tissue target and negative sample of the first frame, respectively, and calculate the neighborhood location correlation filter of the first frame based on the second target image features, label and negative sample image features of the first frame;

[0028] Step 3: For any subsequent frame of the second surgical video, calculate the third response map of the current frame based on the neighborhood position correlation filter obtained from the previous frame and the second target image features extracted from the current frame; take the peak position in the third response map of the current frame as the position of the soft tissue target in the current frame and output it; calculate the neighborhood position correlation filter of the current frame based on the third response map, the second target image features, and the negative sample image features.

[0029] Step 4: Repeat step 3 in the order of video frames until all video frames of the second surgical video have been traversed, and output the position of the soft tissue target in each frame to achieve online tracking of the soft tissue.

[0030] Preferably, the formula for calculating the neighborhood location correlation filter in the first frame is:

[0031] ;

[0032] ;

[0033] ;

[0034] in, This represents the neighborhood location correlation filter for the first frame; Represents the Discrete Fourier Transform; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; This represents the complex conjugate operation of the discrete Fourier transform; This represents the dot product operation; This represents the second target image feature in the first frame; This indicates the label corresponding to the second target image feature in the first frame; Indicates the number of negative samples; Indicates the first frame Negative sample image features of a negative sample; This represents the first regularization parameter; This represents the second regularization parameter.

[0035] Preferably, the formula for calculating the third response map of the current frame is:

[0036] ;

[0037] in, Indicates the first The third response map of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The second target image features of the frame; Indicates the first Frame neighborhood location correlation filter.

[0038] Preferably, the process of calculating the neighborhood location correlation filter for the current frame includes:

[0039] Use the third response map of the current frame as a pseudo-label for the second target image feature of the current frame.

[0040] Substitute the second target image features, pseudo-label, and negative sample image features of the current frame into the calculation formula of the neighborhood location correlation filter to obtain the neighborhood location correlation filter of the current frame.

[0041] Beneficial effects: In the offline training phase, this method constructs a feature extraction network containing short-term channel correlation modules to extract features, which can enhance the recognition ability of low-contrast soft tissues and achieve accurate differentiation of target regions, thereby extracting surgical image features with a small number of parameters. At the same time, a multi-frame sampling strategy is adopted in the forward-backward tracking cycle to reduce errors caused by occlusion or artifacts, and the long-term cycle consistency loss reduces the cycle response error caused by long-term tracking, thereby enhancing the temporal consistency of soft tissues over a long period of time. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of the offline training phase of the robotic surgical soft tissue tracking method based on long-short time cyclic consistency in the embodiments of this application.

[0044] Figure 2 This is a schematic diagram of the feature extraction network structure in an embodiment of this application.

[0045] Figure 3 This is a flowchart of the online tracking stage of the robotic surgical soft tissue tracking method based on long-short time cycle consistency in the embodiments of this application. Detailed Implementation

[0046] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0048] like Figure 1 As shown, this embodiment provides an offline training phase for a robotic surgical soft tissue tracking method based on long-short-time cyclic consistency, including:

[0049] S1: Randomly select a video frame from the surgical video as a training video segment (e.g., randomly select multiple video frames within a time range t from the surgical video as training video segments).

[0050] Existing methods mainly use discriminant correlation filters to perform forward-backward tracking through cyclic consistency verification between adjacent frames. However, these methods are prone to random bias and model fitting problems due to the large amount of tissue deformation or scale changes that occur during long-term surgeries.

[0051] To address the limitations of the aforementioned methods, an adaptive multi-frame sampling approach is proposed within the forward-backward tracking cycle. This systematically increases the temporal sampling density between surgical video frames, and iterative cross-validation of non-rigid targets is achieved through multi-frame consistency learning. By establishing a temporal correspondence between the adaptive multi-frame sampling and the reference frame, the error accumulation caused by tissue deformation or artifacts during long-term surgeries can be effectively mitigated, ensuring stable soft tissue tracking in long-term dynamic surgical environments.

[0052] In this embodiment, the surgical videos were taken from two publicly available datasets: the SurgT dataset and the OBRDataset. The SurgT dataset, released at the MICCAI 2022 EndoVis Challenge, consists of 157 videos from 20 clinical cases. The average video length is 30 seconds, with most videos having a frame rate of 25Hz and a few at 30Hz. The video resolution is primarily 1280p. 1280 pixels; The OBRDataset dataset contains 10 gastrointestinal endoscopic videos collected using Olympus NBI and Pentax i-scan endoscopes, totaling 7894 frames, with an image size of 640 pixels. With 480 pixels, the OBRDataset dataset is filled with textureless soft tissue compared to the SurgT dataset, making tracking more challenging. In this embodiment, the video frame images in the surgical video are obtained from the original video frame images through preprocessing methods such as cropping, normalization, and data augmentation, with a resolution of 224. 224 images were used to enhance the model's adaptability to input data and improve its generalization ability.

[0053] S2: Select soft tissue in the initial frame of the training video segment as the forward tracking target, and use a discriminative correlation filter to generate the corresponding first label for the forward tracking target.

[0054] Specifically, generating the corresponding first label for the forward-tracking target using a discriminative correlation filter includes:

[0055] The discriminant correlation filter performs correlation operations on the first target image features to generate a Gaussian response map. Each pixel in the Gaussian response map represents the degree of matching between the corresponding position and the first target image features. The generated Gaussian response map is used as the first label corresponding to the first target image features.

[0056] Discriminant Correlation Filter (DCF) is a commonly used target tracking algorithm. Its principle is to construct a filter by learning target features, so that the filter produces a high response in the target region and a low response in the background region. Specifically, in each frame of the image, DCF can generate a Gaussian response map by performing correlation operations on the features of the target region. Each pixel in the response map represents the degree of matching between the corresponding position and the target features, and its peak position is considered to be the position of the target in the current frame.

[0057] S3: Input the forward-tracking target into a feature extraction network containing a short-time channel correlation module to obtain the first target image features; calculate the discriminative correlation filter of the initial frame based on the first target image features and the first label of the initial frame.

[0058] Because soft tissue tracking in surgical videos requires real-time processing, feature extraction using encoder-decoder networks involves a large amount of parameter computation, making it difficult to meet real-time requirements. Existing methods use shallow networks for feature extraction, prioritizing computational speed but struggling to distinguish soft tissues with high texture similarity. In this embodiment, as... Figure 2 As shown, the feature extraction network includes a shallow network and a short-time channel correlation module;

[0059] The shallow network consists of two 3×3 convolutional layers, used to extract low-level features from the input image;

[0060] In the short-time channel correlation module, low-level features are passed through a global average pooling layer to obtain pooled features. These pooled features are then passed sequentially through parallel convolutional layers with kernel sizes of 3, 5, and 7. The convolutional results are summed pixel-wise, and the sum is used to obtain activation weights. These activation weights are then multiplied element-wise with the low-level features, and the multiplication result is residual-connected to the low-level features. The gradient is stabilized through the residual connection to obtain the output feature F of the short-time channel correlation module. The short-time channel correlation module can reduce local homogeneity by fusing inter-channel features in the feature extraction network with minimal parameter overhead, thereby enhancing the ability to distinguish similar texture structures.

[0061] Furthermore, the formula for calculating the discriminant correlation filter is:

[0062] ;

[0063] in, Indicates the first training video segment Discriminative correlation filter for frames; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first label corresponding to the first target image feature of the frame; This represents the first regularization parameter required to solve the ridge regression problem.

[0064] S4: For any subsequent frame in the training video segment, calculate the first response map of the current frame based on the discriminant correlation filter obtained from the previous frame and the first target image features extracted from the current frame; take the peak position in the first response map of the current frame as the position of the forward tracking target in the current frame and output it; take the first response map of the current frame as the first pseudo-label of the first target image features of the current frame, and calculate the discriminant correlation filter of the current frame with the first target image features of the current frame.

[0065] The formula for calculating the first response diagram is:

[0066] ;

[0067] in, Indicates the first The first response graph of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first training video segment Discriminative correlation filter for frames.

[0068] S5: Repeat S4 in the order of video frames in the training video segment to perform forward tracking until all video frames in the training video segment have been traversed; take the last frame of the training video segment as the initial frame for backward tracking, and take the forward tracking target and the corresponding first pseudo-label in the current frame determined by the last frame of the training video segment as the initial target and the corresponding initial first label for backward tracking, and perform backward tracking. The backward tracking is to repeat S4-S5 in reverse order of video frames in the training video segment until all video frames in the training video segment have been traversed.

[0069] S6: Based on the first response map obtained from forward tracking and the second response map obtained from backward tracking in each frame of the training video segment, construct a loss function and minimize the loss function to optimize the forward tracking process.

[0070] For each frame in the training video segment, the first response map of the search target during forward tracking should be as consistent as possible with the second response map computed during backward tracking. Therefore, to achieve unsupervised optimization of the tracker, a long-term cyclic consistency loss (LCCL) is designed for cyclic verification of multi-frame forward-backward tracking to maintain temporal consistency in tracking soft tissue changes and deformations. By comparing the similarity between the first response map of the search target in forward tracking and the second response map computed in backward tracking, the cyclic response error during tracker training is minimized using the Smooth L1 loss. The proposed long-term cyclic consistency loss function L is obtained by summing the losses of all frames in the forward-backward tracking cycle. Since the last frame cannot be used for backward tracking, the loss is calculated only for the first to the last frame in the training video segment. Summation is performed on the frames.

[0071] Specifically, the expression for the loss function is:

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] in, Represents the loss function; This indicates the number of video frames in the training video segment; Indicates smoothL1 loss; The backward tracing yields the first... The second response map of the frame; The first digit obtained by forward tracing The first response graph of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first target image feature of the frame; Indicates the first training video segment Discriminative correlation filter for frames; Indicates the first training video segment Discriminative correlation filter for frames; This indicates taking the absolute value of each element.

[0077] This loss function effectively suppresses biases caused by soft tissue changes or deformations by minimizing the cyclic response error generated during long-term tracking. Furthermore, the network parameters of the feature extraction network can be updated and optimized based on the training video segments and the loss function, thus completing the training of the feature extraction network.

[0078] like Figure 3 As shown, this embodiment provides an online tracking phase of a robotic surgical soft tissue tracking method based on long-short time cyclic consistency, including:

[0079] Step 1: Acquire the second surgical video to be detected, select the soft tissue target to be tracked and its corresponding label in the first frame of the second surgical video, and use the image patch around the soft tissue target as a negative sample.

[0080] Step 2: Based on the trained feature extraction network, extract the second target image features and negative sample image features from the soft tissue target and negative sample of the first frame, respectively, and calculate the neighborhood location correlation filter of the first frame based on the second target image features, label and negative sample image features of the first frame.

[0081] Specifically, the formula for calculating the neighborhood location correlation filter in the first frame is:

[0082] ;

[0083] ;

[0084] ;

[0085] in, This represents the neighborhood location correlation filter for the first frame; Represents the Discrete Fourier Transform; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; This represents the complex conjugate operation of the discrete Fourier transform; This represents the dot product operation; This represents the second target image feature in the first frame; This indicates the label corresponding to the second target image feature in the first frame; Indicates the number of negative samples; Indicates the first frame Negative sample image features of a negative sample; This represents the first regularization parameter; The second regularization parameter represents the domain-aware space.

[0086] During online tracking, the surgical scene is often unknown, and the soft tissue may be textureless or highly similar to its surroundings. Discriminative correlation filters (DCFs) are updated based on the training data, and using the original DCF makes it difficult for the tracker to effectively distinguish soft tissue from its surroundings in a new scene. Therefore, in online tracking, neighborhood location correlation filters (NPCFs) are used to help the tracker learn and understand the target's surrounding environment in new surgical scenes, improving its ability to track soft tissue online in unknown scenarios.

[0087] The neighborhood location correlation filter (NPCF) uses a second regularization parameter. Integrating NPCF into the ridge regression framework enhances traditional filters, enabling explicit differentiation of target soft tissue from anatomically adjacent regions during tracking. However, considering the discrete nature of input video frames during offline training, incorporating neighborhood context awareness might introduce unnecessary noise into the tracker; therefore, NPCF is only used in the online tracking phase.

[0088] Step 3: For any subsequent frame of the second surgical video, calculate the third response map of the current frame based on the neighborhood position correlation filter obtained from the previous frame and the second target image features extracted from the current frame; take the peak position in the third response map of the current frame as the position of the soft tissue target in the current frame and output it; calculate the neighborhood position correlation filter of the current frame based on the third response map, the second target image features, and the negative sample image features.

[0089] Specifically, the formula for calculating the third response map of the current frame is:

[0090] ;

[0091] in, Indicates the first The third response map of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The second target image features of the frame; Indicates the first Frame neighborhood location correlation filter.

[0092] Furthermore, the process of calculating the neighborhood location correlation filter for the current frame includes:

[0093] Use the third response map of the current frame as a pseudo-label for the second target image feature of the current frame.

[0094] Substitute the second target image features, pseudo-label, and negative sample image features of the current frame into the calculation formula of the neighborhood location correlation filter to obtain the neighborhood location correlation filter of the current frame.

[0095] Step 4: Repeat step 3 in the order of video frames until all video frames of the second surgical video have been traversed, and output the position of the soft tissue target in each frame to achieve long-term online tracking of soft tissue in unknown surgical scenarios.

[0096] The robotic surgical soft tissue tracking method based on long-short time cycle consistency provided in this embodiment has the following beneficial effects:

[0097] 1. This method enhances the recognition of low-contrast soft tissues by constructing a feature extraction network containing short-term channel correlation modules, achieving accurate differentiation of target regions and extracting surgical image features with a small number of parameters. Simultaneously, a multi-frame sampling strategy is employed within the forward-backward tracking cycle to reduce errors caused by occlusion or artifacts, and long-term cycle consistency loss is used to reduce cyclic response errors generated during long-term tracking, enhancing the temporal consistency of soft tissues over extended periods.

[0098] 2. This method selects the soft tissue target to be tracked and its corresponding label in the first frame of the second surgical video, and uses the image patches surrounding the soft tissue target as negative samples. Second target image features and negative sample image features are extracted from the soft tissue target and negative samples, respectively. Based on the second target image features, label, and negative sample image features of the first frame, a corresponding neighborhood location correlation filter is calculated. For each subsequent frame of the second surgical video, a third response map of the current frame is calculated based on the neighborhood location correlation filter obtained from the previous frame and the second target image features extracted from the current frame. The peak position in the third response map of the current frame is taken as the position of the soft tissue target in the current frame and output. Based on the third response map, target image features, and negative sample image features of the current frame, a neighborhood location correlation filter is calculated for the current frame. This process is repeated to achieve online tracking of soft tissue. This method, through the neighborhood location correlation filter, can effectively reduce background interference in unknown surgical scenarios and dynamically suppress interference similar to soft tissue textures, ensuring long-term stable tracking of soft tissue in dynamic surgical environments.

[0099] To verify the beneficial effects of the technical solution described in this application in surgical scenarios, comparative experiments were conducted using the publicly available datasets SurgT and OBRDataset. The following evaluation metrics were used in the experiments:

[0100] (1) Robustness: This refers to the ability of a tracking algorithm to successfully track a target even when the target is invisible or partially occluded. It is quantified by calculating the proportion of frames in which the target is successfully tracked out of all valid frames.

[0101] (2) Acc (Accuracy): Measures the degree of overlap between the target location predicted by the tracking algorithm and the actual target location. It is evaluated by calculating the average intersection-over-union (IoU) ratio of the predicted and ground truth boxes in all successfully tracked frames.

[0102] (3) Error: Represents the average deviation between the target position predicted by the tracking algorithm and the actual target position. It is quantified by calculating the average Euclidean distance between the center of the predicted box and the center of the actual box in all successfully tracked frames.

[0103] (4) FPS (Frame Rate): This indicates the speed at which the tracking algorithm processes video frames, i.e., the number of frames per second it can process. In real-time surgical video tracking applications, a higher FPS value (usually greater than 25Hz) is necessary to ensure that the algorithm can respond to changes in the video in a timely manner.

[0104] The experimental results are shown in Tables 1 and 2.

[0105] Table 1. Comparison of different tracking methods in SurgT

[0106] ;

[0107] Table 2 Comparison of different tracing methods in OBRDataset

[0108] ;

[0109] As shown in Tables 1 and 2, this application significantly outperforms the unsupervised comparison methods UDT and USOT in Rob, Acc, and Error metrics. Compared to the supervised method ODTrack, this application still exhibits considerable robustness and higher accuracy. Furthermore, while maintaining high robustness and accuracy, this application also demonstrates high real-time processing capabilities, achieving FPS of 26.37 and 27.31 Hz on the two datasets, respectively, which can meet the real-time tracking requirements in surgical scenarios.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A robotic surgical soft tissue tracking method based on long-short time cyclic consistency, characterized in that, include: Offline training phase, S1: Randomly select a video frame from the surgical video as the training video segment; S2: Select soft tissue in the initial frame of the training video segment as the forward tracking target, and use a discriminative correlation filter to generate the corresponding first label for the forward tracking target; S3: Input the forward-tracking target into a feature extraction network containing a short-time channel correlation module to obtain the first target image features; The discriminative correlation filter for the initial frame is calculated based on the first target image features and the first label of the initial frame; The feature extraction network includes a shallow network and a short-time channel correlation module; The shallow network consists of two 3×3 convolutional layers, used to extract low-level features from the input image; In the short-time channel correlation module, low-level features are passed through a global average pooling layer to obtain pooled features. The pooled features are then passed sequentially through parallel convolutional layers with kernel sizes of 3, 5, and 7. The results of each convolution are summed pixel by pixel, and the sum is activated to obtain activation weights. The activation weights are then multiplied element-wise with the low-level features, and the multiplication result is residually concatenated with the low-level features to obtain the output features of the short-time channel correlation module. S4: For any subsequent frame in the training video segment, calculate the first response map of the current frame based on the discriminant correlation filter obtained from the previous frame and the first target image features extracted from the current frame; take the peak position in the first response map of the current frame as the position of the forward tracking target in the current frame and output it; take the first response map of the current frame as the first pseudo-label of the first target image features of the current frame, and calculate the discriminant correlation filter of the current frame with the first target image features of the current frame; S5: Repeat S4 in the order of video frames in the training video segment to perform forward tracking until all video frames in the training video segment have been traversed; take the last frame of the training video segment as the initial frame for backward tracking, and take the forward tracking target and the corresponding first pseudo-label in the current frame determined by the last frame of the training video segment as the initial target and the corresponding initial first label for backward tracking, and perform backward tracking. The backward tracking is to repeat S4-S5 in reverse order of video frames in the training video segment until all video frames in the training video segment have been traversed. S6: Based on the first response map obtained from forward tracking and the second response map obtained from backward tracking in each frame of the training video segment, construct a loss function and minimize the loss function to optimize the forward tracking process.

2. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 1, characterized in that, The first label for the forward-tracking target is generated using a discriminative correlation filter, including: The discriminant correlation filter performs correlation operations on the first target image features to generate a Gaussian response map. Each pixel in the Gaussian response map represents the degree of matching between the corresponding position and the first target image features. The generated Gaussian response map is used as the first label corresponding to the first target image features.

3. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 1, characterized in that, The formula for calculating the discriminant correlation filter is: ； in, Indicates the first training video segment Discriminative correlation filter for frames; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first label corresponding to the first target image feature of the frame; This represents the first regularization parameter.

4. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 1, characterized in that, The expression for the loss function is: ； ； ； ； in, Represents the loss function; This indicates the number of video frames in the training video segment; Indicates smoothL1 loss; The backward tracing yields the first... The second response map of the frame; The first digit obtained by forward tracing The first response graph of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The first target image feature of the frame; Indicates the first The first target image feature of the frame; Indicates the first training video segment Discriminative correlation filter for frames; Indicates the first training video segment Discriminative correlation filter for frames; This indicates taking the absolute value of each element.

5. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 1, characterized in that, The surgical videos were taken from the SurgT dataset and the OBRDataset dataset.

6. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 1, characterized in that, It also includes an online tracking phase: Step 1: Acquire the second surgical video to be detected, select the soft tissue target to be tracked and its corresponding label in the first frame of the second surgical video, and use the image patch around the soft tissue target as a negative sample; Step 2: Based on the feature extraction network, extract the second target image features and negative sample image features from the soft tissue target and negative sample of the first frame, respectively, and calculate the neighborhood location correlation filter of the first frame based on the second target image features, label and negative sample image features of the first frame; Step 3: For any subsequent frame of the second surgical video, calculate the third response map of the current frame based on the neighborhood position correlation filter obtained from the previous frame and the second target image features extracted from the current frame; take the peak position in the third response map of the current frame as the position of the soft tissue target in the current frame and output it; calculate the neighborhood position correlation filter of the current frame based on the third response map, the second target image features, and the negative sample image features. Step 4: Repeat step 3 in the order of video frames until all video frames of the second surgical video have been traversed, and output the position of the soft tissue target in each frame to achieve online tracking of the soft tissue.

7. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 6, characterized in that, The formula for calculating the neighborhood location correlation filter in the first frame is: ； ； ； in, This represents the neighborhood location correlation filter for the first frame; Represents the Discrete Fourier Transform; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; This represents the complex conjugate operation of the discrete Fourier transform; This represents the dot product operation; This represents the second target image feature in the first frame; This indicates the label corresponding to the second target image feature in the first frame; Indicates the number of negative samples; Indicates the first frame Negative sample image features of a negative sample; This represents the first regularization parameter; This represents the second regularization parameter.

8. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 6, characterized in that, The formula for calculating the third response map of the current frame is: ； in, Indicates the first The third response map of the frame; Represents the inverse of the discrete Fourier transform; This indicates related operations in the frequency domain; Indicates the first The second target image features of the frame; Indicates the first Frame neighborhood location correlation filter.

9. The robotic surgical soft tissue tracking method based on long-short time cyclic consistency according to claim 6, characterized in that, The process of calculating the neighborhood location correlation filter for the current frame includes: Use the third response map of the current frame as a pseudo-label for the second target image feature of the current frame. Substitute the second target image features, pseudo-label, and negative sample image features of the current frame into the calculation formula of the neighborhood location correlation filter to obtain the neighborhood location correlation filter of the current frame.

Images