Fractal-driven laparoscope Toldt line positioning method and device and medium

Abstract

The invention discloses a fractal-driven laparoscope Toldt line positioning method and device and a medium. The method comprises the following steps: acquiring a plurality of laparoscope video frames; performing feature modeling on the plurality of laparoscope video frames according to a Toldt line segmentation model to obtain a multi-scale feature map; and performing multi-scale spatio-temporal feature fusion and aggregation processing on the multi-scale feature map, and inputting a processing result into a multi-scale decoder for prediction to obtain the position of the Toldt line in the abdominal cavity. According to the fractal-driven laparoscope Toldt line positioning method and device and the medium, the anatomical features of the Toldt line are captured through accurate feature modeling, the adaptability to the dynamic abdominal cavity environment is enhanced in combination with multi-scale spatial-temporal feature fusion, high-precision prediction is achieved through a multi-scale decoder, and therefore the position of the Toldt line in the abdominal cavity is recognized, and the positioning accuracy of the Toldt line in the abdominal cavity is improved. The problem that it is difficult to accurately recognize the position of the Toldt line in the abdominal cavity can be solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a fractal-driven laparoscopic Toldt's line positioning method, device and medium. Background Art

[0002] At the transition between the visceral peritoneum and the parietal peritoneum on the lateral margins of the ascending and descending colon, there is a constant line known as Toldt's line, also known in my country as the yellow-white junction line. Laparoscopy provides medical personnel with real-time observation of the internal anatomy of the abdominal cavity. During laparoscopic mesocolon resection, accurate identification of Toldt's line is crucial for surgical success, but the variability of anatomical location and the complexity of surrounding tissues make this identification challenging. Furthermore, with technological advances, deep learning models can annotate regions of interest in images and videos, helping surgical personnel quickly locate targets and reducing their workload. Current research focuses on efficient and high-precision detection and segmentation of intestinal polyps. To this end, several high-quality image and video polyp segmentation datasets have been constructed, such as the Kavsir-SEG and CVC series. Based on these datasets, various polyp segmentation algorithms have been proposed, such as PraNet, which utilizes reverse attention to enhance boundary cue capture, PNS-Net, which employs a regularized self-attention mechanism, and LDNet, which uses a dynamic kernel generation and update mechanism. Furthermore, Swin-UMamba, which utilizes a pre-trained architecture to improve segmentation accuracy, has advanced the state of intestinal polyp segmentation technology.

[0003] However, in the existing technology, deep learning video segmentation models trained based on medical datasets are not effective in identifying and segmenting Toldt's lines because their boundary features are blurred with the surrounding anatomical structures, which increases the difficulty of segmentation; models trained based on polyp segmentation datasets have difficulty capturing Toldt's line boundary clues and lack anatomical constraints, resulting in a unified strategy when processing different anatomical structures and an inability to cope with the challenges of complex anatomical structures and low-contrast target areas; therefore, existing technologies find it difficult to accurately identify the location of the Toldt's line area inside the abdominal cavity. Summary of the Invention

[0004] The present invention provides a fractal-driven laparoscopic Toldt's line positioning method, device and medium to solve the problem of difficulty in accurately identifying the position of Toldt's line in the abdominal cavity.

[0005] To achieve the above objectives, the present application provides a fractal-driven laparoscopic Toldt line positioning method, comprising: Acquire several laparoscopic video frames; performing feature modeling on the plurality of laparoscopic video frames according to a Toldt's line segmentation model to obtain a multi-scale feature map; wherein the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary through a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features; The multi-scale spatiotemporal feature fusion and aggregation processing is performed on the multi-scale feature map, and the processing result is input into the multi-scale decoder for prediction to obtain the position of Toldt's line inside the abdominal cavity.

[0006] The present invention first uses a Toldt's line segmentation model to model features in laparoscopic video frames. For the model, Toldt's lines, as part of the anatomical structure, possess unique geometric properties. A pre-set algorithm accurately quantifies these geometric properties, allowing the model to learn the unique shape and orientation of Toldt's line boundaries. By guiding spatial attention through fractal dimension features, the model can focus on regions most relevant to Toldt's line boundaries, ignoring background noise and interference. This focus improves recognition accuracy and stability. Spatiotemporal feature fusion and aggregation are then performed on the multi-scale feature maps. This process not only integrates spatial information at different scales but also captures dynamic changes in the temporal dimension, helping the model more accurately track and identify the location of Toldt's lines when processing dynamic abdominal videos. Finally, the fused features are input into a multi-scale decoder for prediction. The multi-scale decoder leverages this rich feature information for refined segmentation prediction, accurately outputting the location of Toldt's lines and helping the model maintain high accuracy when processing complex anatomical structures and low-contrast areas.

[0007] Compared with the existing technology, the present invention captures the anatomical features of Toldt's lines through precise feature modeling, combines multi-scale spatiotemporal feature fusion to enhance adaptability to the dynamic abdominal environment, and then uses a multi-scale decoder to achieve high-precision prediction, thereby identifying the position of Toldt's lines inside the abdominal cavity. Therefore, it can solve the problem of difficulty in accurately identifying the position of Toldt's lines inside the abdominal cavity.

[0008] As a preferred solution, multi-scale spatiotemporal feature fusion and aggregation processing is performed on the multi-scale feature map, and the processing results are input into a multi-scale decoder for prediction to obtain the position of Toldt's line inside the abdominal cavity, specifically: Performing cross-temporal convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map; Calculating the local fractal dimension of each pixel position in the spatiotemporal fusion feature map according to a fractal dimension estimation algorithm to obtain a fractal dimension feature map; Performing multiple feature extraction on the fractal dimension feature map to obtain a multi-scale enhanced feature map; The multi-scale enhanced feature map is input into the multi-scale decoder for prediction to obtain the position of the Toldt's line inside the abdominal cavity.

[0009] This preferred solution performs cross-temporal convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map, which can capture the temporal dependency between video frames and effectively fuse spatial and temporal features. The local fractal dimension of each pixel position in the spatiotemporal fusion feature map is calculated according to the fractal dimension estimation algorithm to obtain a fractal dimension feature map; as a geometric feature, the fractal dimension can reflect the complexity and irregularity of the anatomical structure. Multiple features are extracted from the fractal dimension feature map to obtain a multi-scale enhanced feature map. This multi-path design can capture local detail features and global contextual information, and can construct a more comprehensive and rich multi-scale enhanced feature map.

[0010] As a preferred solution, multiple feature extraction is performed on the fractal dimension feature map to obtain a multi-scale enhanced feature map, specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map; The first output feature map is input into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model for filtering to obtain the multi-scale enhanced feature map.

[0011] In this preferred solution, the local texture-aware convolution submodule focuses on the local texture features of the anatomical structure, extracting high-level semantics through adaptive convolution kernels to obtain the first output feature map. The fractal-guided anatomically consistent attention submodule, on the other hand, focuses on the global consistency of the anatomical structure and the distinction between foreground and background, filtering features through the attention mechanism to obtain a multi-scale enhanced feature map. These two data processing methods capture anatomical information at different scales, thereby constructing a more comprehensive and rich multi-scale enhanced feature map.

[0012] As a preferred solution, the spatiotemporal fusion feature map and the fractal dimension feature map are input into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map, specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model; splicing the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension to generate a joint feature tensor; Performing high-level semantic extraction on the joint feature tensor to obtain an expansion coefficient and an offset direction; The first output feature map of the local texture perception convolution submodule is calculated according to the expansion coefficient, the offset direction and the spatiotemporal fusion feature map.

[0013] This preferred solution generates a joint feature tensor by concatenating the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension. The fractal dimension feature map provides detailed information about the anatomical texture, while the spatiotemporal fusion feature map contains high-level semantic information. This process achieves multi-scale feature fusion. High-level semantic extraction of the joint feature tensor yields the expansion coefficient and offset direction, achieving the transformation from low-level features to high-level semantics. This high-level semantic information helps the model understand the overall layout and interrelationships of anatomical structures, providing strong support for subsequent segmentation tasks.

[0014] As a preferred solution, the first output feature map is input into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model for filtering to obtain the multi-scale enhanced feature map, specifically: Inputting the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model; Converting the fractal dimension feature map into a normalized distribution map, and extracting the index of the target foreground area and the index of the target background area from the distribution map to obtain an index set; Flatten the first output feature map and generate a triple embedding consisting of Query, Key and Value through matrix projection; Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map.

[0015] This preferred solution converts the fractal dimension feature map into a normalized distribution map, and extracts the indexes of the target foreground area and the target background area from it to obtain an index set. This process can accurately distinguish the foreground and background areas, providing accurate target areas for subsequent attention feature extraction.

[0016] As a preferred solution, data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map, specifically: In the triple embedding, the key-value pairs corresponding to the target foreground area and the target background area are respectively filtered according to the index set to obtain an embedding set containing only keys and values; Calculating a foreground attention result and a background attention result according to the query and the embedding set, and concatenating the foreground attention result and the background attention result to obtain an attention feature map; The number of channels of the attention feature map is adjusted to the number of channels of the spatiotemporal fusion feature map through convolution compression, and the attention feature map after channel alignment is fused to the spatiotemporal fusion feature map through jump connection to obtain the multi-scale enhanced feature map.

[0017] This preferred solution uses an index set to filter the key-value pairs corresponding to the target foreground and background regions, indicating that the global nature of the query must be maintained. Therefore, the query and filtered key-value pairs are used to calculate the foreground and background attention results, enabling the model to focus more on local key anatomical structures under the guidance of global information. The number of channels in the attention feature map is adjusted through convolution compression to align it with the number of channels in the spatiotemporal fusion feature map. This adjustment method not only retains the key information of the attention feature map, but also ensures the consistency of subsequent feature fusion.

[0018] As a preferred solution, the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary through a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features, specifically: Acquire a surgical video frame containing a Toldt's line area; Marking the area where the Toldt's line is located in the surgical video frame to obtain a laparoscopic Toldt's line segmentation dataset; According to the fractal dimension estimation algorithm and the preset module, the deep learning model is trained using the laparoscopic Toldt's line segmentation dataset to obtain the Toldt's line segmentation model; wherein the fractal dimension estimation algorithm is used to calculate the fractal dimension, and the preset module is used to extract anatomical features by guiding spatial attention through fractal dimension features.

[0019] This preferred solution constructs a high-quality laparoscopic Toldt's line segmentation dataset by marking the regions where Toldt's lines are located in surgical video frames. This helps deep learning models learn the accurate features of Toldt's lines, thereby improving recognition accuracy. Fractal dimension estimation algorithms generally have low computational complexity, and their application to the training process of deep learning models does not significantly increase the computational burden. By guiding spatial attention through fractal dimension features, the model can more accurately focus on areas with significant fractal properties, thereby extracting more accurate and relevant anatomical features.

[0020] As a preferred solution, feature modeling is performed on the laparoscopic video frames according to the Toldt line segmentation model to obtain a multi-scale feature map, specifically: Inputting target continuous frames from the plurality of laparoscopic video frames into the Swin-Transformer backbone network encoder of the Toldt's line segmentation model; According to the Swin-Transformer backbone network encoder, the local features and global features of the target continuous frames are modeled through a sliding window attention mechanism to obtain the multi-scale feature map.

[0021] In this preferred solution, the sliding window attention mechanism can capture global information through interactions between windows while processing local features. This enables the model to understand the overall structure while remaining sensitive to details, thereby accurately identifying key anatomical structures such as Toldt's line in complex laparoscopic video frames.

[0022] The present application also provides a fractal-driven laparoscopic Toldt's line positioning device, comprising a data module, a modeling module, and a prediction module; Wherein, the data module is used to obtain a number of laparoscopic video frames; The modeling module is configured to perform feature modeling on the plurality of laparoscopic video frames according to a Toldt's line segmentation model to obtain a multi-scale feature map; wherein the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary using a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features; The prediction module is used to perform multi-scale spatiotemporal feature fusion and aggregation processing on the multi-scale feature map, input the processing results into the multi-scale decoder for prediction, and obtain the position of Toldt's line inside the abdominal cavity.

[0023] As a preferred solution, the prediction module includes a convolution unit, a calculation unit, an extraction unit and a prediction unit; The convolution unit is used to perform cross-time convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map; The calculation unit is used to calculate the local fractal dimension of each pixel position in the spatiotemporal fusion feature map according to the fractal dimension estimation algorithm to obtain a fractal dimension feature map; The extraction unit is used to perform multiple feature extraction on the fractal dimension feature map to obtain a multi-scale enhanced feature map; The prediction unit is configured to input the multi-scale enhanced feature map into the multi-scale decoder for prediction, thereby obtaining the position of the Toldt's line inside the abdominal cavity.

[0024] As a preferred solution, the extraction unit includes an extraction subunit and a filtering subunit; The extraction subunit is configured to input the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map; The filtering subunit is used to input the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt line segmentation model for filtering to obtain the multi-scale enhanced feature map.

[0025] As a preferred solution, the extraction subunit is specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model; splicing the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension to generate a joint feature tensor; Performing high-level semantic extraction on the joint feature tensor to obtain an expansion coefficient and an offset direction; The first output feature map of the local texture perception convolution submodule is calculated according to the expansion coefficient, the offset direction and the spatiotemporal fusion feature map.

[0026] As a preferred solution, the filtering subunit is specifically: Inputting the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model; Converting the fractal dimension feature map into a normalized distribution map, and extracting the index of the target foreground area and the index of the target background area from the distribution map to obtain an index set; Flatten the first output feature map and generate a triple embedding consisting of Query, Key and Value through matrix projection; Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map.

[0027] As a preferred solution, data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map, specifically: In the triple embedding, the key-value pairs corresponding to the target foreground area and the target background area are respectively filtered according to the index set to obtain an embedding set containing only keys and values; Calculating a foreground attention result and a background attention result according to the query and the embedding set, and concatenating the foreground attention result and the background attention result to obtain an attention feature map; The number of channels of the attention feature map is adjusted to the number of channels of the spatiotemporal fusion feature map through convolution compression, and the attention feature map after channel alignment is fused to the spatiotemporal fusion feature map through jump connection to obtain the multi-scale enhanced feature map.

[0028] As a preferred solution, the modeling module includes a video unit, a labeling unit and a training unit; Wherein, the video unit is used to obtain a surgical video frame containing a Toldt's line area; The marking unit is used to mark the area where the Toldt's line is located in the surgical video frame to obtain a laparoscopic Toldt's line segmentation dataset; The training unit is used to train the deep learning model using the laparoscopic Toldt's line segmentation dataset according to the fractal dimension estimation algorithm and the preset module to obtain the Toldt's line segmentation model; wherein the fractal dimension estimation algorithm is used to calculate the fractal dimension, and the preset module is used to extract anatomical features by guiding spatial attention through fractal dimension features.

[0029] As a preferred solution, the modeling module includes an input unit and a construction unit; The input unit is configured to input the target continuous frames from the plurality of laparoscopic video frames into the Swin-Transformer backbone network encoder of the Toldt's line segmentation model; The construction unit is used to model the local features and global features of the target continuous frames through a sliding window attention mechanism according to the Swin-Transformer backbone network encoder to obtain the multi-scale feature map.

[0030] The present application also provides a storage medium having a computer program stored thereon. The computer program is called and executed by a computer to implement the fractal-driven laparoscopic Toldt's line positioning method as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 1 is a flow chart of a fractal-driven laparoscopic Toldt's line positioning method provided in an embodiment of the present application; Figure 2 Schematic diagram of the construction process of the Toldt's line segmentation model for laparoscopic video provided in an embodiment of the present application; Figure 3 Schematic diagram of the network architecture of FSA-Net provided in an embodiment of the present application; Figure 4 This is a flow chart of a fractal dimension estimation algorithm based on measure theory provided in an embodiment of the present application; Figure 5 Schematic diagram of a classification guidance feature aggregation module provided in an embodiment of the present application; Figure 6 This is a schematic diagram of the FSA-Net workflow provided by an embodiment of the present application; Figure 7This is a visual comparison diagram of the Toldt line segmentation effects of different segmentation models provided in the embodiments of the present application; Figure 8 This is a comparison chart of the quantitative results of Toldt's line segmentation performance of different segmentation models provided in the embodiments of the present application; Figure 9 Schematic diagram of the structure of a fractal-driven laparoscopic Toldt's line positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, unless otherwise specified, "several" means two or more.

[0034] The embodiment of the present application provides a fractal-driven laparoscopic Toldt's line positioning method, which is mainly used to assist colorectal surgical medical staff to accurately identify the Toldt's line area inside the abdominal cavity during surgery.

[0035] Example 1: See also Figure 1 The embodiment of the present application provides a fractal-driven laparoscopic Toldt line positioning method, including S1 to S3, and the specific implementation steps are as follows: S1. Acquire several laparoscopic video frames.

[0036] Step S1 of the embodiment of the present application is specifically as follows: Video frames were extracted from laparoscopic surgical video images at a rate of 2 frames per second. Subsequently, the size of these frames was downsampled to 384×384 pixels. Next, data augmentation processing was performed on each frame, including scaling, rotation, flipping, and random noise addition, to finally generate several processed laparoscopic video frames.

[0037] S2. Feature modeling is performed on several laparoscopic video frames based on the Toldt's line segmentation model to obtain a multi-scale feature map. The Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary through a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features.

[0038] Step S2 of the embodiment of the present application includes S2.1 to S2.2, wherein S2.1 is the process of establishing a Toldt line segmentation model, and S2.2 is the process of calculating a multi-scale feature map based on the Swin-Transformer backbone network encoder, specifically: S2.1. Obtain imaging data from laparoscopic videos of mesocolon resection surgeries and extract surgical video frames containing the Toldt line region from the imaging data. To obtain laparoscopic imaging data from mesocolon resection surgeries, data was first collected from the surgical procedures of 45 different patients. Each patient's surgery was recorded as several videos lasting approximately 1 to 2 hours. Subsequently, short video segments lasting 15 to 30 seconds were selected from each long video segment using a preset model or by professional surgeons. This ensured that each selected frame clearly displayed the Toldt line region. Ultimately, 150 high-quality laparoscopic surgical video segments were obtained, which laid a solid foundation for subsequent Toldt line segmentation model training. All clipped surgical video frames are extracted frame by frame, and each frame in the video is exported and saved as an image file in PNG format. Subsequently, these extracted frame sequences are downsampled to reduce the frame rate of the original video from 30FPS to 2FPS, that is, 2 frames are retained per second. After downsampling, a total of 5,000 images were obtained, and each image clearly showed the Toldt's line area. Then, the preset model or experienced colorectal surgical medical staff were used for manual annotation to accurately define the area where the Toldt's line was located by drawing a broken line to obtain a laparoscopic Toldt's line segmentation dataset. After the annotation work is completed, the preset model or colorectal surgery professor will perform a quality assessment to ensure that all annotation results in the laparoscopic Toldt's line segmentation dataset meet the preset standards; According to the fractal dimension estimation algorithm and preset modules, a deep learning model was trained using a laparoscopic Toldt's line segmentation dataset to obtain the Toldt's line segmentation model, which is a deep learning model based on the fractal-driven collaborative anatomical perception network (FSA-Net). Among them, the fractal dimension estimation algorithm is used to calculate the fractal dimension. The preset modules include a local texture-aware convolution (LTC) submodule and a fractal-guided anatomically consistent attention (FAA) submodule, which are used to extract anatomical features by guiding spatial attention through fractal dimension features, aiming to collaboratively extract internal and external anatomical features. In addition, the Toldt's line segmentation model includes a Swin-Transformer-based encoder (i.e., the Swin-Transformer backbone network encoder), a measure-theory-based fractal dimension estimation module, a fractal-guided feature aggregation module, and a Transformer-based multi-scale feature decoder.

[0039] In addition, the performance test results of the Toldt line segmentation model showed that it can process 30 to 40 frames of images per second. This processing speed fully matches the 30 frames per second frame rate of laparoscopic cameras in most current colorectal cancer surgeries, thus proving that the model has the ability to perform real-time segmentation of surgical video images.

[0040] To apply this application example, please refer to Figure 2-3 , Figure 2 This is a schematic diagram of the construction process of the Toldt's line segmentation model for the laparoscopic video provided in this application, which shows the construction process of the Toldt's line segmentation model; Figure 3 Schematic diagram of the network architecture of FSA-Net provided in this application, showing the network architecture of the Toldt line segmentation model.

[0041] In this embodiment S2.1, by marking the areas where Toldt's lines are located in the surgical video frames, a high-quality laparoscopic Toldt's line segmentation dataset can be constructed. This helps the deep learning model learn the accurate features of Toldt's lines, thereby improving recognition accuracy. Fractal dimension estimation algorithms generally have low computational complexity, and applying them to the training process of deep learning models does not significantly increase the computational burden. By guiding spatial attention through fractal dimension features, the model can more accurately focus on areas with significant fractal characteristics, thereby extracting more accurate and relevant anatomical features. In summary, segmenting Toldt's lines in videos is extremely challenging because their location and morphology vary significantly between individuals, and they have low contrast with surrounding tissue and blurred boundaries. To address this, this example innovatively proposes the FSA-Net framework, specifically designed for segmenting Toldt's linea alba in surgical laparoscopic images. FSA-Net first introduces a novel fractal dimension estimation algorithm to ensure computational accuracy. It then integrates two submodules: local texture-aware convolution (LTC) and fractal-guided anatomically consistent attention (FAA) to collaboratively capture anatomical features. Furthermore, this example constructs the first laparoscopic Toldt's line segmentation dataset to facilitate related research. Experimental validation on multiple datasets demonstrates that FSA-Net significantly improves Toldt's line segmentation accuracy, providing strong support for surgical procedures.

[0042] S2.2. Inputting target consecutive frames from a number of laparoscopic video frames into the Swin-Transformer backbone network encoder of the Toldt line segmentation model; wherein the target consecutive frames are a three-frame sequence consisting of a key frame and its preceding and following adjacent frames; According to the Swin-Transformer backbone network encoder, the local and global features of the target continuous frames are modeled through the sliding window attention mechanism to obtain multi-scale feature maps; among them, the multi-scale feature maps are feature maps of four scales {F1, F2, F3, F4}, and the resolution of these feature maps decreases successively, specifically to 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input size; and the Swin-Transformer backbone network encoder combines sliding window attention and global attention to effectively model the global and local features of the image at the same time.

[0043] In this embodiment S2.2, the sliding window attention mechanism can capture global information through interactions between windows while processing local features. This enables the model to understand the overall structure while remaining sensitive to details, thereby accurately identifying key anatomical structures such as Toldt's line in complex laparoscopic video frames.

[0044] S3. Perform multi-scale spatiotemporal feature fusion and aggregation processing on the multi-scale feature map, input the processed results into the multi-scale decoder for prediction, and obtain the position of Toldt's line inside the abdominal cavity.

[0045] Step S3 of the embodiment of the present application includes S3.1 to S3.4, wherein S3.1 is a process of estimating the fractal dimension of multi-scale features to obtain a pixel-level fractal dimension feature map, S3.2 is a process of extracting a first output feature map based on the local texture-aware convolution submodule, S3.3 is a process of extracting a multi-scale enhanced feature map based on the fractal-guided anatomically consistent attention submodule, and S3.4 is a process of performing a prediction based on the Transformer-based multi-scale feature decoder to obtain the position of Toldt's line inside the abdominal cavity, specifically: S3.1. At each scale, a spatiotemporal fusion module based on a 3D convolutional layer is used to perform cross-temporal convolution on the feature maps of three consecutive frames in the multi-scale feature map to obtain a spatiotemporal fusion feature map F. i Among them, F i The subscript i in [1] represents different feature map scales. Specifically, i={1,2,3,4} corresponds to feature map scales {1 / 4,1 / 8,1 / 16,1 / 32} of the original input, respectively. Furthermore, the spatiotemporal fusion module based on the 3D convolution layer aims to effectively utilize the temporal information between consecutive frames to improve the segmentation of Toldts lines. It is worth noting that 3D convolution, as a mature and widely used deep learning component, is often used to extract spatial features of 3D images and temporal information between consecutive video frames. According to the "Fractal dimension estimation module based on measure theory" in Toldt's line segmentation model, the spatiotemporal fusion feature map F is calculated according to the fractal dimension estimation algorithm. i The local fractal dimension of each pixel position in the pixel level is obtained to obtain the fractal dimension feature map D i Among them, the fractal dimension feature map D i {D1, D2, D3, D4}, whose size is the same as the "space-time fusion feature map F i "The same; fractal dimension can effectively characterize the texture complexity of different anatomical structures; using fractal dimension features as constraints helps the model better perceive different anatomical structures.

[0046] To apply this application example, please refer to Figure 4 , Figure 4 This is a flow chart of the fractal dimension estimation algorithm based on measure theory provided by this application, which shows the algorithm flow proposed in this implementation for estimating the fractal dimension of an image at the pixel level, aiming to more efficiently estimate the fractal dimension of a depth feature map; Reflect Padding represents a reflective padding operation, used to add edge pixels to the feature map. B(Fi[H,W],ε) represents the ε×ε neighborhood centered on Fi[H,W], and E[·] represents the mathematical expectation. q is a scaling exponent, which is set to 2 during fractal dimension estimation in this embodiment for ease of calculation. μ(x) represents the probability measure of the Fi pixel, and Dis(·) represents the Euclidean distance between two Fi pixels. The specific calculation process is as follows: ① Input feature map: Input a feature map Fi of size H×W.

[0047] ②Reflection filling: Perform reflection filling operation on the feature map Fi to increase edge pixels.

[0048] ③Define the neighborhood: For each pixel in the feature map Fi, define a neighborhood B(Fi[H,W],ε) with the pixel as the center and a size of ε×ε.

[0049] ④ Calculate the probability measure: For each pixel point, calculate its probability measure μ(x); and dμ(x) in the formula is used to represent the probability of each pixel point in the neighborhood B(Fi[H,W],ε) being sampled, and its size is a fixed value.

[0050] ⑤ Calculate distance and expectation: Calculate the Euclidean distance Dis(·) between a pixel x in the neighborhood B(Fi[H,W],ε) and any other pixel y in the neighborhood B(Fi[H,W],ε). The scaling factor q is not used in the calculation of the mathematical expectation, but is used in the calculation of (ε / E[·]). q-1 Calculation.

[0051] ⑥ Estimate fractal dimension: Use the measure-based fractal dimension estimation method, combined with the probability measure μ(x) and mathematical expectation E[·] calculated above, to estimate the fractal dimension of the feature map.

[0052] ⑦Output: Output the estimated fractal dimension.

[0053] In this embodiment, S3.1 convolves the multi-scale feature map across the temporal dimension to obtain a spatiotemporal fusion feature map, which can capture the temporal dependencies between video frames and effectively fuse spatial and temporal features. The local fractal dimension of each pixel position in the spatiotemporal fusion feature map is calculated using a fractal dimension estimation algorithm to obtain a fractal dimension feature map. Fractal dimension, as a geometric feature, can reflect the complexity and irregularity of anatomical structures. While box-counting dimension is effective in estimating the fractal dimension of various structures, the feature maps extracted by deep learning model backbones are often embedded in high dimensions, significantly increasing computational complexity. In contrast, the measure-based fractal dimension estimation methods used in this embodiment, such as the Rényi generalized dimension, demonstrate greater robustness when processing such high-dimensional data and can significantly alleviate the curse of dimensionality caused by direct geometric overlays.

[0054] S3.2, fusion of spatiotemporal feature map F i and fractal dimension feature map D i At the same time, the local texture-aware convolution (LTC) submodule is input; the local texture-aware convolution submodule belongs to the "fractal-guided feature aggregation module" in the Toldt line segmentation model, and the "fractal-guided feature aggregation module" also includes a fractal-guided anatomically consistent attention (FAA) submodule for subsequent extraction of multi-scale enhanced feature maps. In addition, the "fractal-guided feature aggregation module" combines the local texture-aware convolution module with the fractal-driven anatomically consistent attention, which helps FSA-Net collaboratively capture the contextual relationship between the internal anatomical structure and different anatomical structures, thereby improving the model's ability to distinguish different anatomical structures. The spatiotemporal fusion feature map F along the channel dimension i and fractal dimension feature map D i Perform splicing to generate a joint feature tensor; Perform high-level semantic extraction on the joint feature tensor to obtain the expansion coefficient E i and offset direction A i ; According to the expansion coefficient E i , offset direction A i and spatiotemporal fusion feature map F i The first output feature map output by the local texture perception convolution submodule is calculated.

[0055] To apply this application example, please refer to Figure 5 , Figure 5 This is a schematic diagram of the classification guidance feature aggregation module provided by this application, which shows the use of the classification guidance feature aggregation module to fusion feature map F i and fractal dimension feature map D i the process of data processing; The local texture-aware convolution submodule is designed to adaptively integrate features from different anatomical structures. Compared to standard deformable convolutional networks (DCNs), this implementation leverages anatomical knowledge embedded in fractal features to guide convolution kernel offsets, effectively preventing the model from learning unreasonable deformations.

[0056] Specific implementation such as Figure 5 As shown in the "(b) Local Texture Perceptual Convolution (LTC)" section, the spatiotemporal fusion feature map F i and fractal dimension feature map D i As input, they are used together to predict the offset of the DCN kernel. The offset generator consists of two 3×3 depth-separable convolution blocks, which are responsible for predicting the expansion coefficient E i and offset direction A i Then, using these predicted expansion coefficients and offset directions, the final offset can be calculated. i After obtaining the offset, combined with the original spatiotemporal fusion feature map F i and offset O i , the output features (i.e., the first output feature map) are calculated through expansion and deformable convolution operations, that is, the "expansion and deformable convolution layer" used will accept the "spatiotemporal fusion feature map F i and offset O i ” as a parameter to calculate the output feature map of the local texture-aware convolution submodule.

[0057] In this embodiment, S3.2, a joint feature tensor is generated by concatenating the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension. The fractal dimension feature map provides detailed information about the anatomical texture, while the spatiotemporal fusion feature map contains high-level semantic information. This process achieves multi-scale feature fusion. High-level semantics are extracted from the joint feature tensor to obtain the expansion coefficient and offset direction, achieving the transformation from low-level features to high-level semantics. This high-level semantic information helps the model understand the overall layout and interrelationships of anatomical structures, providing strong support for subsequent segmentation tasks.

[0058] S3.3, input the first output feature map into the fractal-guided anatomically consistent attention submodule; Use the linear transformation layer to transform the fractal dimension feature map D i Convert to normalized distribution map M i , and from the distribution map M i The index of the target foreground area and the index of the target background area are extracted to obtain an index set; the target foreground area refers to the anatomical structure or pathological area that needs to be focused on, which is usually directly related to the diagnosis and treatment goals; for example, the Toldt line-related area (as the anatomical boundary mark between the mesentery and retroperitoneal tissue in colon surgery, its accurate identification is crucial for surgical level separation and vascular protection); the target background area refers to the interfering area that is not the core diagnosis and treatment target, such as surgical instruments (which need to be shielded in intraoperative images to avoid affecting surgical field analysis) and distal tissues (such as adjacent organs or non-lesioned tissues, which usually need to be excluded from the analysis scope).

[0059] Flatten the first output feature map and generate a triple embedding consisting of Query, Key, and Value through matrix projection; In the triple embedding, the key-value pairs corresponding to the target foreground area and the target background area are filtered according to the index set to obtain an embedding set containing only keys and values; Based on the embedding set, according to the distribution graph M i Select the key-value pairs belonging to the foreground area, use the global query and the foreground keys to calculate the attention weight, and then aggregate it with the foreground values ​​to generate the foreground attention result. Similarly, generate the background attention result; splice the foreground attention result and the background attention result to obtain the attention feature map; Adjust the number of channels of the attention feature map to the spatiotemporal fusion feature map F by compressing the 1×1 convolution layer i The number of channels is fused into the spatiotemporal fusion feature map F through skip-connection. i , thereby updating the spatiotemporal fusion feature map F i , and obtain the multi-scale enhanced feature map output by the fractal-guided anatomically consistent attention sub-module.

[0060] Specific implementation such as Figure 5 As shown in the “(c) Fractal-guided Anatomy-Agreeable Attention (FAA)” section in

[15] , the core of this module is a fractal-guided key-value pair filtering strategy, which aims to enhance the model's ability to capture inter-anatomical information in a divide-and-conquer manner.

[0061] In this embodiment, S3.3 converts the fractal dimension feature map into a normalized distribution map, and extracts the indexes of the target foreground region and the target background region from it to obtain an index set. This process can accurately distinguish the foreground and background regions, providing an accurate target region for subsequent attention feature extraction. Furthermore, filtering the key-value pairs corresponding to the target foreground and background regions through the index set indicates the need to maintain the global nature of the query. Therefore, using the query and the filtered key-value pairs to calculate the foreground and background attention results enables the model to focus more on key local anatomical structures under the guidance of global information. Adjusting the number of channels in the attention feature map through convolutional compression to align it with the number of channels in the spatiotemporal fusion feature map not only preserves the key information of the attention feature map, but also ensures the consistency of subsequent feature fusion.

[0062] In general, in this embodiment S3.2-S3.3, multiple features are extracted from the fractal dimension feature map to obtain a multi-scale enhanced feature map. This multi-path design can capture local detail features and global context information, and can construct a more comprehensive and rich multi-scale enhanced feature map.

[0063] Furthermore, the local texture-aware convolution submodule focuses on local texture features of anatomical structures, extracting high-level semantics through adaptive convolution kernels to produce the first output feature map. The fractal-guided anatomically consistent attention submodule, on the other hand, prioritizes global anatomical consistency and distinguishes foreground from background, filtering features through an attention mechanism to produce a multi-scale enhanced feature map. These two data processing approaches capture anatomical information at different scales, thereby constructing a more comprehensive and rich multi-scale enhanced feature map.

[0064] S3.4. Input the multi-scale enhanced feature map into the Transformer-based multi-scale feature decoder of the Toldt's line segmentation model for prediction to obtain the location of the Toldt's line inside the abdominal cavity; The predicted position of Toldt's line is automatically outlined in the original surgical video image through the algorithm to achieve the visualization effect of segmentation.

[0065] The visualization effect of S3.4 of this embodiment provides surgical medical staff with intuitive and clear reference information, which can help them make more accurate decisions during the operation, thereby enhancing the safety of the operation and improving the overall effect of the operation.

[0066] To apply this application example, please refer to Figure 6-8 ; Figure 6 : is a schematic diagram of the FSA-Net workflow provided in this application, which shows the workflow of predicting the position of Toldt's line in the abdominal cavity based on the Toldt's line segmentation model (FSA-Net) in this embodiment.

[0067] Figure 7 This is a visual comparison of the Toldt's line segmentation effects of different segmentation models provided in this application, showing the Toldt's line segmentation results of different segmentation models. Among them, the longitudinal image corresponding to "Ours" is the prediction result of the position of Toldt's line in the abdominal cavity in Example 1; Figure 8 This is a comparison chart of the quantified results of Toldt's line segmentation performance of different segmentation models provided in this application, showing the quantified results of Toldt's line segmentation energy of different segmentation models. Among them, the horizontal data corresponding to "FSA-Net" is the quantified results of Toldt's line segmentation performance in the abdominal cavity of this embodiment 1; Figure 7-8As shown in FIG, the segmentation performance of FSA-Net in this embodiment is significantly better than that of the existing endoscopic image segmentation model.

[0068] Overall, this embodiment has the following beneficial effects: This application first uses a Toldt's line segmentation model to model features in laparoscopic video frames. For the model, Toldt's lines, as part of the anatomical structure, have unique geometric properties. By accurately quantifying these geometric properties through a pre-set algorithm, the model learns the unique shape and orientation of Toldt's line boundaries. By guiding spatial attention through fractal dimension features, the model focuses on regions most relevant to Toldt's line boundaries, ignoring background noise and interference. This focus improves recognition accuracy and stability. Spatiotemporal feature fusion and aggregation are then performed on multi-scale feature maps. This process not only integrates spatial information at different scales but also captures dynamic changes in the temporal dimension, helping the model more accurately track and identify the location of Toldt's lines when processing dynamic abdominal videos. Finally, the fused features are fed into a multi-scale decoder for prediction. The multi-scale decoder leverages this rich feature information for refined segmentation prediction, accurately outputting the location of Toldt's lines and helping the model maintain high accuracy when dealing with complex anatomical structures and low-contrast areas.

[0069] Example 2: See also Figure 9 , an embodiment of the present application provides a fractal-driven laparoscopic Toldt's line positioning device, comprising a data module 10, a modeling module 20 and a prediction module 30; Wherein, the data module 10 is used to obtain a number of laparoscopic video frames; a modeling module 20 for performing feature modeling on a plurality of laparoscopic video frames according to a Toldt's line segmentation model to obtain a multi-scale feature map; wherein the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary using a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features; The prediction module 30 is used to perform multi-scale spatiotemporal feature fusion and aggregation processing on the multi-scale feature map, input the processing results into the multi-scale decoder for prediction, and obtain the position of Toldt's line inside the abdominal cavity.

[0070] In one embodiment, the data module 10 is specifically: Video frames were extracted from laparoscopic surgical video images at a rate of 2 frames per second. Subsequently, the size of these frames was downsampled to 384×384 pixels. Next, data augmentation processing was performed on each frame, including scaling, rotation, flipping, and random noise addition, to finally generate several processed laparoscopic video frames.

[0071] In one embodiment, the modeling module 20 includes a video unit, a labeling unit, a training unit, an input unit, and a construction unit. The video unit, the labeling unit, and the training unit are processes for establishing a Toldt line segmentation model, and the input unit and the construction unit are processes for calculating a multi-scale feature map based on a Swin-Transformer backbone network encoder, specifically: The video unit is used to acquire laparoscopic video data from mesocolon resection surgeries and extract surgical video frames containing the Toldt line region from the image data. To obtain laparoscopic imaging data from mesocolon resection surgeries, data was first collected from the surgical procedures of 45 different patients. Each patient's surgery was recorded as several videos lasting approximately 1 to 2 hours. Subsequently, a preset model or professional surgeons selected short video segments lasting 15 to 30 seconds from each long video segment, ensuring that each selected frame clearly displayed the Toldt line region. Ultimately, 150 high-quality laparoscopic surgical video segments were obtained, which laid a solid foundation for subsequent Toldt line segmentation model training. The labeling unit is used to extract all the clipped surgical video frames frame by frame, export each frame in the video and save it as an image file in PNG format. Subsequently, these extracted frame sequences are downsampled to reduce the frame rate of the original video from 30FPS to 2FPS, that is, retaining 2 frames per second. After downsampling, a total of 5,000 images were obtained, and each image clearly showed the Toldt's line area. Then, the preset model or experienced colorectal surgical medical staff were used for manual labeling to accurately define the area where the Toldt's line is located by drawing a broken line to obtain a laparoscopic Toldt's line segmentation dataset. After the labeling work is completed, the preset model or the colorectal surgery professor will perform a quality assessment to ensure that all labeling results in the laparoscopic Toldt's line segmentation dataset meet the preset standards; A training unit is used to train a deep learning model using a laparoscopic Toldt's line segmentation dataset based on a fractal dimension estimation algorithm and a preset module to obtain a Toldt's line segmentation model, i.e., a deep learning model based on a fractal-driven collaborative anatomical perception network (FSA-Net). The fractal dimension estimation algorithm is used to calculate the fractal dimension. The preset module includes a local texture-aware convolution (LTC) submodule and a fractal-guided anatomically consistent attention (FAA) submodule, which are used to extract anatomical features by guiding spatial attention through fractal dimension features, aiming to collaboratively extract internal and external anatomical features. The Toldt's line segmentation model includes a Swin-Transformer-based encoder (i.e., a Swin-Transformer backbone network encoder), a measure-theoretic fractal dimension estimation module, a fractal-guided feature aggregation module, and a Transformer-based multi-scale feature decoder.

[0072] In addition, the performance test results of the Toldt line segmentation model showed that it can process 30 to 40 frames of images per second. This processing speed fully matches the 30 frames per second frame rate of laparoscopic cameras in most current colorectal cancer surgeries, thus proving that the model has the ability to perform real-time segmentation of surgical video images.

[0073] To apply this application example, please refer to Figure 2-3 , Figure 2 This is a schematic diagram of the construction process of the Toldt's line segmentation model for the laparoscopic video provided in this application, which shows the construction process of the Toldt's line segmentation model; Figure 3 Schematic diagram of the network architecture of FSA-Net provided in this application, showing the network architecture of the Toldt line segmentation model.

[0074] The video unit, labeling unit, and training unit of this embodiment can construct a high-quality laparoscopic Toldt's line segmentation dataset by labeling the area where Toldt's lines are located in the surgical video frames. This helps the deep learning model learn the accurate features of Toldt's lines, thereby improving recognition accuracy. Fractal dimension estimation algorithms generally have low computational complexity, and applying them to the training process of deep learning models does not significantly increase the computational burden. By guiding spatial attention through fractal dimension features, the model can more accurately focus on areas with significant fractal characteristics, thereby extracting more accurate and relevant anatomical features. In summary, segmenting Toldt's lines in videos is extremely challenging because their location and morphology vary significantly between individuals, and they have low contrast with surrounding tissue and blurred boundaries. To address this, this example innovatively proposes the FSA-Net framework, specifically designed for segmenting Toldt's linea alba in surgical laparoscopic images. FSA-Net first introduces a novel fractal dimension estimation algorithm to ensure computational accuracy. It then integrates two submodules: local texture-aware convolution (LTC) and fractal-guided anatomically consistent attention (FAA) to collaboratively capture anatomical features. Furthermore, this example constructs the first laparoscopic Toldt's line segmentation dataset to facilitate related research. Experimental validation on multiple datasets demonstrates that FSA-Net significantly improves Toldt's line segmentation accuracy, providing strong support for surgical procedures.

[0075] An input unit is configured to input target continuous frames from a plurality of laparoscopic video frames into a Swin-Transformer backbone network encoder of a Toldt line segmentation model; wherein the target continuous frames are a three-frame sequence consisting of a key frame and its preceding and following adjacent frames; A construction unit is used to model the local and global features of the target continuous frames through a sliding window attention mechanism based on the Swin-Transformer backbone network encoder to obtain a multi-scale feature map; wherein the multi-scale feature map is a feature map of four scales {F1, F2, F3, F4}, and the resolution of these feature maps decreases successively, specifically to 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input size; and the Swin-Transformer backbone network encoder combines sliding window attention and global attention to effectively model both the global and local features of the image.

[0076] In the input unit and construction unit of this embodiment, the sliding window attention mechanism can capture global information through interactions between windows while processing local features. This enables the model to understand the overall structure while remaining sensitive to details, thereby accurately identifying key anatomical structures such as Toldt's lines in complex laparoscopic video frames.

[0077] In one embodiment, the prediction module 30 includes a convolution unit, a calculation unit, an extraction subunit, a filtering subunit, and a prediction unit; Among them, the convolution unit and the calculation unit are the processes of estimating the fractal dimension of multi-scale features to obtain the fractal dimension feature map at the pixel level. The extraction subunit is the process of extracting the first output feature map based on the local texture perception convolution submodule. The filtering subunit is the process of extracting the multi-scale enhanced feature map based on the fractal-guided anatomically consistent attention submodule. The prediction unit is the process of predicting the position of Toldt's line inside the abdominal cavity based on the multi-scale feature decoder of Transformer. Specifically: Among them, the convolution unit is used to perform cross-temporal convolution on the feature maps of three consecutive frames in the multi-scale feature map at each scale using the spatiotemporal fusion module based on the 3D convolution layer to obtain the spatiotemporal fusion feature map F i Among them, F i The subscript i in [1] represents different feature map scales. Specifically, i={1,2,3,4} corresponds to feature map scales {1 / 4,1 / 8,1 / 16,1 / 32} of the original input, respectively. Furthermore, the spatiotemporal fusion module based on the 3D convolution layer aims to effectively utilize the temporal information between consecutive frames to improve the segmentation of Toldts lines. It is worth noting that 3D convolution, as a mature and widely used deep learning component, is often used to extract spatial features of 3D images and temporal information between consecutive video frames. The calculation unit is used to calculate the spatiotemporal fusion feature map F according to the fractal dimension estimation algorithm based on the "fractal dimension estimation module based on measure theory" in the Toldt line segmentation model. i The local fractal dimension of each pixel position in the pixel level is obtained to obtain the fractal dimension feature map D i Among them, the fractal dimension feature map D i {D1, D2, D3, D4}, whose size is the same as the "space-time fusion feature map F i "The same; fractal dimension can effectively characterize the texture complexity of different anatomical structures; using fractal dimension features as constraints helps the model better perceive different anatomical structures.

[0078] To apply this application example, please refer to Figure 4 , Figure 4 This is a flow chart of the fractal dimension estimation algorithm based on measure theory provided by this application, which shows the algorithm flow proposed in this implementation for estimating the fractal dimension of an image at the pixel level, aiming to more efficiently estimate the fractal dimension of a depth feature map; Reflect Padding represents a reflective padding operation, used to add edge pixels to the feature map. B(Fi[H,W],ε) represents the ε×ε neighborhood centered on Fi[H,W], and E[·] represents the mathematical expectation. q is a scaling exponent, which is set to 2 during fractal dimension estimation in this embodiment for ease of calculation. μ(x) represents the probability measure of the Fi pixel, and Dis(·) represents the Euclidean distance between two Fi pixels. The specific calculation process is as follows: ① Input feature map: Input a feature map Fi of size H×W.

[0079] ②Reflection filling: Perform reflection filling operation on the feature map Fi to increase edge pixels.

[0080] ③Define the neighborhood: For each pixel in the feature map Fi, define a neighborhood B(Fi[H,W],ε) with the pixel as the center and a size of ε×ε.

[0081] ④ Calculate the probability measure: For each pixel point, calculate its probability measure μ(x); and dμ(x) in the formula is used to represent the probability of each pixel point in the neighborhood B(Fi[H,W],ε) being sampled, and its size is a fixed value.

[0082] ⑤ Calculate distance and expectation: Calculate the Euclidean distance Dis(·) between a pixel x in the neighborhood B(Fi[H,W],ε) and any other pixel y in the neighborhood B(Fi[H,W],ε). The scaling factor q is not used in the calculation of the mathematical expectation, but is used in the calculation of (ε / E[·]). q-1 Calculation.

[0083] ⑥ Estimate fractal dimension: Use the measure-based fractal dimension estimation method, combined with the probability measure μ(x) and mathematical expectation E[·] calculated above, to estimate the fractal dimension of the feature map.

[0084] ⑦Output: Output the estimated fractal dimension.

[0085] In this embodiment, the convolution unit and the computation unit perform cross-temporal convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map, which can capture the temporal dependencies between video frames and effectively fuse spatial and temporal features. The local fractal dimension of each pixel position in the spatiotemporal fusion feature map is calculated using a fractal dimension estimation algorithm to obtain a fractal dimension feature map. Fractal dimension, as a geometric feature, can reflect the complexity and irregularity of anatomical structures. While box-counting dimension is effective in estimating the fractal dimension of various structures, the feature maps extracted by deep learning model backbones are often embedded in high dimensions, significantly increasing computational complexity. In contrast, the measure-based fractal dimension estimation methods used in this embodiment, such as the Rényi generalized dimension, demonstrate greater robustness when processing such high-dimensional data and can significantly alleviate the curse of dimensionality caused by direct geometric overlays.

[0086] Extraction subunit, used to fusion spatiotemporal feature map F i and fractal dimension feature map D i At the same time, the local texture-aware convolution (LTC) submodule is input; the local texture-aware convolution submodule belongs to the "fractal-guided feature aggregation module" in the Toldt line segmentation model, and the "fractal-guided feature aggregation module" also includes a fractal-guided anatomically consistent attention (FAA) submodule for subsequent extraction of multi-scale enhanced feature maps. In addition, the "fractal-guided feature aggregation module" combines the local texture-aware convolution module with the fractal-driven anatomically consistent attention, which helps FSA-Net collaboratively capture the contextual relationship between the internal anatomical structure and different anatomical structures, thereby improving the model's ability to distinguish different anatomical structures. The extraction subunit is also used to fusion the spatiotemporal feature map F along the channel dimension i and fractal dimension feature map D i Perform splicing to generate a joint feature tensor; The extraction subunit is also used to extract high-level semantics from the joint feature tensor to obtain the expansion coefficient E i and offset direction A i ; Extract subunits and also use them to calculate the expansion coefficient E i , offset direction A i and spatiotemporal fusion feature map F i The first output feature map output by the local texture perception convolution submodule is calculated.

[0087] To apply this application example, please refer to Figure 5 , Figure 5 This is a schematic diagram of the classification guidance feature aggregation module provided by this application, which shows the use of the classification guidance feature aggregation module to fusion feature map F i and fractal dimension feature map D i the process of data processing; The local texture-aware convolution submodule is designed to adaptively integrate features from different anatomical structures. Compared to standard deformable convolutional networks (DCNs), this implementation leverages anatomical knowledge embedded in fractal features to guide convolution kernel offsets, effectively preventing the model from learning unreasonable deformations.

[0088] Specific implementation is as follows Figure 5 As shown in the "(b) Local Texture Perceptual Convolution (LTC)" section, the spatiotemporal fusion feature map F i and fractal dimension feature map D i As input, they are used together to predict the offset of the DCN kernel. The offset generator consists of two 3×3 depth-separable convolution blocks, which are responsible for predicting the expansion coefficient E i and offset direction A i Then, using these predicted expansion coefficients and offset directions, the final offset can be calculated. i After obtaining the offset, combined with the original spatiotemporal fusion feature map F i and offset O i , the output features (i.e., the first output feature map) are calculated through the expansion and deformable convolution operation, that is, the "expansion and deformable convolution layer" used will accept the "spatiotemporal fusion feature map F i and offset O i ” as a parameter to calculate the output feature map of the local texture-aware convolution submodule.

[0089] The extraction subunit in this embodiment generates a joint feature tensor by concatenating the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension. The fractal dimension feature map provides detailed information about the anatomical texture, while the spatiotemporal fusion feature map contains high-level semantic information. This process achieves multi-scale feature fusion. High-level semantic extraction is performed on the joint feature tensor to obtain the expansion coefficient and offset direction, achieving the transformation from low-level features to high-level semantics. This high-level semantic information helps the model understand the overall layout and interrelationships of anatomical structures, providing strong support for subsequent segmentation tasks.

[0090] a filtering subunit, configured to input the first output feature map into a fractal-guided anatomically consistent attention submodule; The filter unit is also used to transform the fractal dimension feature map D using a linear transformation layer. i Convert to normalized distribution map M i , and from the distribution map M iThe index of the target foreground area and the index of the target background area are extracted to obtain an index set; the target foreground area refers to the anatomical structure or pathological area that needs to be focused on, which is usually directly related to the diagnosis and treatment goals; for example, the Toldt line-related area (as the anatomical boundary mark between the mesentery and retroperitoneal tissue in colon surgery, its accurate identification is crucial for surgical level separation and vascular protection); the target background area refers to the interfering area that is not the core diagnosis and treatment target, such as surgical instruments (which need to be shielded in intraoperative images to avoid affecting surgical field analysis) and distal tissues (such as adjacent organs or non-lesioned tissues, which usually need to be excluded from the analysis scope).

[0091] The filtering subunit is also used to flatten the first output feature map and generate a triple embedding containing Query, Key, and Value through matrix projection; The filtering subunit is also used to filter the key-value pairs corresponding to the target foreground area and the target background area according to the index set in the triple embedding, and obtain an embedding set containing only keys and values; The filtering subunit is also used to calculate the distribution map M based on the embedding set. i Select the key-value pairs belonging to the foreground area, use the global query and the foreground keys to calculate the attention weight, and then aggregate it with the foreground values ​​to generate the foreground attention result. Similarly, generate the background attention result; splice the foreground attention result and the background attention result to obtain the attention feature map; The filter subunit is also used to adjust the number of channels of the attention feature map to the spatiotemporal fusion feature map F by compressing the 1×1 convolution layer. i The number of channels is fused into the spatiotemporal fusion feature map F through skip-connection. i , thereby updating the spatiotemporal fusion feature map F i , and obtain the multi-scale enhanced feature map output by the fractal-guided anatomically consistent attention sub-module.

[0092] Specific implementation is as follows Figure 5 As shown in the “(c) Fractal-guided Anatomy-Agreeable Attention (FAA)” section in

[15] , the core of this module is a fractal-guided key-value pair filtering strategy, which aims to enhance the model's ability to capture inter-anatomical information in a divide-and-conquer manner.

[0093] The filtering subunit of this embodiment converts the fractal dimension feature map into a normalized distribution map, and extracts the indexes of the target foreground region and the target background region from it to obtain an index set. This process can accurately distinguish the foreground and background regions, providing accurate target regions for subsequent attention feature extraction. Furthermore, filtering the key-value pairs corresponding to the target foreground and background regions through the index set indicates the need to maintain the global nature of the query. Therefore, using the query and the filtered key-value pairs to calculate the foreground and background attention results enables the model to focus more on key local anatomical structures under the guidance of global information. Adjusting the number of channels in the attention feature map through convolutional compression to align it with the number of channels in the spatiotemporal fusion feature map not only preserves the key information of the attention feature map, but also ensures the consistency of subsequent feature fusion.

[0094] In general, the extraction subunit and the filtering subunit of this embodiment perform multiple feature extraction on the fractal dimension feature map to obtain a multi-scale enhanced feature map. This multi-path design can capture local detail features and global context information, and can construct a more comprehensive and rich multi-scale enhanced feature map.

[0095] Furthermore, the local texture-aware convolution submodule focuses on local texture features of anatomical structures, extracting high-level semantics through adaptive convolution kernels to produce the first output feature map. The fractal-guided anatomically consistent attention submodule, on the other hand, prioritizes global anatomical consistency and distinguishes foreground from background, filtering features through an attention mechanism to produce a multi-scale enhanced feature map. These two data processing approaches capture anatomical information at different scales, thereby constructing a more comprehensive and rich multi-scale enhanced feature map.

[0096] The prediction unit is used to input the multi-scale enhanced feature map into the Transformer-based multi-scale feature decoder of the Toldt's line segmentation model for prediction to obtain the position of the Toldt's line inside the abdominal cavity; The prediction unit is also used to automatically outline the predicted position of Toldt's line in the original surgical video image through an algorithm to achieve a segmentation visualization effect.

[0097] This visualization effect of the prediction unit in this embodiment provides surgical medical staff with intuitive and clear reference information, which can help them make more accurate decisions during the operation, thereby enhancing the safety of the operation and improving the overall effect of the operation.

[0098] To apply this application example, please refer to Figure 6-8 ; Figure 6 : is a schematic diagram of the FSA-Net workflow provided in this application, which shows the workflow of predicting the position of Toldt's line in the abdominal cavity based on the Toldt's line segmentation model (FSA-Net) in this embodiment 2.

[0099] Figure 7This is a visual comparison of the Toldt's line segmentation effects of different segmentation models provided in this application, showing the Toldt's line segmentation results of different segmentation models. Among them, the longitudinal image corresponding to "Ours" is the prediction result of the position of Toldt's line inside the abdominal cavity in Example 2; Figure 8 This is a comparison chart of the quantified results of Toldt's line segmentation performance of different segmentation models provided in this application, which shows the quantified results of Toldt's line segmentation energy of different segmentation models. Among them, the horizontal data corresponding to "FSA-Net" is the quantified results of Toldt's line segmentation performance inside the abdominal cavity in Example 2; Figure 7-8 As shown in FIG, the segmentation performance of FSA-Net in the second embodiment is significantly better than that of the existing endoscopic image segmentation model.

[0100] Overall, this embodiment has the following beneficial effects: This application first uses a Toldt's line segmentation model to model features in laparoscopic video frames. For the model, Toldt's lines, as part of the anatomical structure, have unique geometric properties. By accurately quantifying these geometric properties through a pre-set algorithm, the model learns the unique shape and orientation of Toldt's line boundaries. By guiding spatial attention through fractal dimension features, the model focuses on regions most relevant to Toldt's line boundaries, ignoring background noise and interference. This focus improves recognition accuracy and stability. Spatiotemporal feature fusion and aggregation are then performed on multi-scale feature maps. This process not only integrates spatial information at different scales but also captures dynamic changes in the temporal dimension, helping the model more accurately track and identify the location of Toldt's lines when processing dynamic abdominal videos. Finally, the fused features are fed into a multi-scale decoder for prediction. The multi-scale decoder leverages this rich feature information for refined segmentation prediction, accurately outputting the location of Toldt's lines and helping the model maintain high accuracy when dealing with complex anatomical structures and low-contrast areas.

[0101] Example 3: An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the fractal-driven laparoscopic Toldt's line positioning method; The fractal-driven laparoscopic Toldt's line localization method, if implemented as a software functional unit and used as a standalone product, can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0102] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A fractal-driven laparoscopic Toldt line positioning method, characterized in that: include: Acquire several laparoscopic video frames; performing feature modeling on the plurality of laparoscopic video frames according to a Toldt's line segmentation model to obtain a multi-scale feature map; wherein the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary through a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features; The multi-scale spatiotemporal feature fusion and aggregation processing is performed on the multi-scale feature map, and the processing result is input into the multi-scale decoder for prediction to obtain the position of Toldt's line inside the abdominal cavity.

2. The fractal-driven laparoscopic Toldt line positioning method according to claim 1, characterized in that: The multi-scale spatiotemporal feature fusion and aggregation processing is performed on the multi-scale feature map, and the processing result is input into the multi-scale decoder for prediction to obtain the position of Toldt's line inside the abdominal cavity, specifically: Performing cross-temporal convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map; Calculating the local fractal dimension of each pixel position in the spatiotemporal fusion feature map according to a fractal dimension estimation algorithm to obtain a fractal dimension feature map; Performing multiple feature extraction on the fractal dimension feature map to obtain a multi-scale enhanced feature map; The multi-scale enhanced feature map is input into the multi-scale decoder for prediction to obtain the position of the Toldt's line inside the abdominal cavity.

3. The fractal-driven laparoscopic Toldt line positioning method according to claim 2, characterized in that: Multiple feature extraction is performed on the fractal dimension feature map to obtain a multi-scale enhanced feature map, specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map; The first output feature map is input into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model for filtering to obtain the multi-scale enhanced feature map.

4. The fractal-driven laparoscopic Toldt line positioning method according to claim 3, characterized in that: The spatiotemporal fusion feature map and the fractal dimension feature map are input into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map, specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model; splicing the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension to generate a joint feature tensor; Performing high-level semantic extraction on the joint feature tensor to obtain an expansion coefficient and an offset direction; The first output feature map of the local texture perception convolution submodule is calculated according to the expansion coefficient, the offset direction and the spatiotemporal fusion feature map.

5. The fractal-driven laparoscopic Toldt line positioning method according to claim 3, characterized in that: The first output feature map is input into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model for filtering to obtain the multi-scale enhanced feature map, specifically: Inputting the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model; Converting the fractal dimension feature map into a normalized distribution map, and extracting the index of the target foreground area and the index of the target background area from the distribution map to obtain an index set; Flatten the first output feature map and generate a triple embedding consisting of Query, Key and Value through matrix projection; Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map.

6. The fractal-driven laparoscopic Toldt line positioning method according to claim 5, characterized in that: Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map, specifically: In the triple embedding, the key-value pairs corresponding to the target foreground area and the target background area are respectively filtered according to the index set to obtain an embedding set containing only keys and values; Calculating a foreground attention result and a background attention result according to the query and the embedding set, and concatenating the foreground attention result and the background attention result to obtain an attention feature map; The number of channels of the attention feature map is adjusted to the number of channels of the spatiotemporal fusion feature map through convolution compression, and the attention feature map after channel alignment is fused to the spatiotemporal fusion feature map through jump connection to obtain the multi-scale enhanced feature map.

7. The fractal-driven laparoscopic Toldt line positioning method according to claim 1, characterized in that: The Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary through a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features. Specifically: Acquire a surgical video frame containing a Toldt's line area; Marking the area where the Toldt's line is located in the surgical video frame to obtain a laparoscopic Toldt's line segmentation dataset; According to the fractal dimension estimation algorithm and the preset module, the deep learning model is trained using the laparoscopic Toldt's line segmentation dataset to obtain the Toldt's line segmentation model; wherein the fractal dimension estimation algorithm is used to calculate the fractal dimension, and the preset module is used to extract anatomical features by guiding spatial attention through fractal dimension features.

8. The fractal-driven laparoscopic Toldt line positioning method according to claim 1, characterized in that: The feature modeling of the laparoscopic video frames is performed according to the Toldt line segmentation model to obtain a multi-scale feature map, specifically: Inputting target continuous frames from the plurality of laparoscopic video frames into the Swin-Transformer backbone network encoder of the Toldt's line segmentation model; According to the Swin-Transformer backbone network encoder, the local features and global features of the target continuous frames are modeled through a sliding window attention mechanism to obtain the multi-scale feature map.

9. A fractal-driven laparoscopic Toldt line positioning device, characterized in that: Includes data module, modeling module and prediction module; Wherein, the data module is used to obtain a number of laparoscopic video frames; The modeling module is configured to perform feature modeling on the plurality of laparoscopic video frames according to a Toldt's line segmentation model to obtain a multi-scale feature map; wherein the Toldt's line segmentation model is established by quantifying the geometric characteristics of the Toldt's line boundary using a preset algorithm and using fractal dimension features to guide spatial attention to extract anatomical features; The prediction module is used to perform multi-scale spatiotemporal feature fusion and aggregation processing on the multi-scale feature map, input the processing results into the multi-scale decoder for prediction, and obtain the position of Toldt's line inside the abdominal cavity.

10. The fractal-driven laparoscopic Toldt's line positioning device according to claim 9, characterized in that: The prediction module includes a convolution unit, a calculation unit, an extraction unit and a prediction unit; The convolution unit is used to perform cross-time convolution on the multi-scale feature map to obtain a spatiotemporal fusion feature map; The calculation unit is used to calculate the local fractal dimension of each pixel position in the spatiotemporal fusion feature map according to the fractal dimension estimation algorithm to obtain a fractal dimension feature map; The extraction unit is used to perform multiple feature extraction on the fractal dimension feature map to obtain a multi-scale enhanced feature map; The prediction unit is configured to input the multi-scale enhanced feature map into the multi-scale decoder for prediction, thereby obtaining the position of the Toldt's line inside the abdominal cavity.

11. The fractal-driven laparoscopic Toldt's line positioning device according to claim 10, characterized in that: The extraction unit includes an extraction subunit and a filtering subunit; The extraction subunit is configured to input the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model for semantic extraction to obtain a first output feature map; The filtering subunit is used to input the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt line segmentation model for filtering to obtain the multi-scale enhanced feature map.

12. The fractal-driven laparoscopic Toldt's line positioning device according to claim 11, characterized in that: The extraction subunit is specifically: Inputting the spatiotemporal fusion feature map and the fractal dimension feature map into the local texture perception convolution submodule of the Toldt line segmentation model; splicing the spatiotemporal fusion feature map and the fractal dimension feature map along the channel dimension to generate a joint feature tensor; Performing high-level semantic extraction on the joint feature tensor to obtain an expansion coefficient and an offset direction; The first output feature map of the local texture perception convolution submodule is calculated according to the expansion coefficient, the offset direction and the spatiotemporal fusion feature map.

13. The fractal-driven laparoscopic Toldt's line positioning device according to claim 11, characterized in that: The filtering subunit is specifically: Inputting the first output feature map into the fractal-guided anatomically consistent attention submodule of the Toldt's line segmentation model; Converting the fractal dimension feature map into a normalized distribution map, and extracting the index of the target foreground area and the index of the target background area from the distribution map to obtain an index set; Flatten the first output feature map and generate a triple embedding consisting of Query, Key and Value through matrix projection; Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map.

14. The fractal-driven laparoscopic Toldt's line positioning device according to claim 13, characterized in that: Data screening and attention feature extraction are performed on the triple embedding according to the index set to obtain the multi-scale enhanced feature map, specifically: In the triple embedding, the key-value pairs corresponding to the target foreground area and the target background area are respectively filtered according to the index set to obtain an embedding set containing only keys and values; Calculating a foreground attention result and a background attention result according to the query and the embedding set, and concatenating the foreground attention result and the background attention result to obtain an attention feature map; The number of channels of the attention feature map is adjusted to the number of channels of the spatiotemporal fusion feature map through convolution compression, and the attention feature map after channel alignment is fused to the spatiotemporal fusion feature map through jump connection to obtain the multi-scale enhanced feature map.

15. The fractal-driven laparoscopic Toldt's line positioning device according to claim 9, characterized in that: The modeling module includes a video unit, a labeling unit and a training unit; Wherein, the video unit is used to obtain a surgical video frame containing a Toldt's line area; The marking unit is used to mark the area where the Toldt's line is located in the surgical video frame to obtain a laparoscopic Toldt's line segmentation dataset; The training unit is used to train the deep learning model using the laparoscopic Toldt's line segmentation dataset according to the fractal dimension estimation algorithm and the preset module to obtain the Toldt's line segmentation model; wherein the fractal dimension estimation algorithm is used to calculate the fractal dimension, and the preset module is used to extract anatomical features by guiding spatial attention through fractal dimension features.

16. The fractal-driven laparoscopic Toldt's line positioning device according to claim 9, characterized in that: The modeling module includes an input unit and a construction unit; The input unit is configured to input the target continuous frames from the plurality of laparoscopic video frames into the Swin-Transformer backbone network encoder of the Toldt's line segmentation model; The construction unit is used to model the local features and global features of the target continuous frames through a sliding window attention mechanism according to the Swin-Transformer backbone network encoder to obtain the multi-scale feature map.

17. A storage medium, characterized in that: The storage medium stores a computer program, which is called and executed by a computer to implement a fractal-driven laparoscopic Toldt's line positioning method as described in any one of claims 1 to 8.

Images