Intelligent evaluation method and system for excision cleanliness in breast tumor minimally invasive rotary excision

Abstract

The invention relates to the technical field of medical image analysis, and discloses an intelligent evaluation method and system for excision cleanliness in breast tumor minimally invasive rotary atherectomy, and the method comprises the steps: firstly obtaining preoperative and intraoperative ultrasonic images of a patient, matching a region of interest through a structural similarity index, and respectively intercepting a main image and a local patch; then, respectively extracting global features of the main image and texture features of local patches by using a deep neural network; performing spatial collaborative attention processing on the global features to establish spatial association between the preoperative image and the intraoperative image, and performing multi-instance pooling operation on the local features to aggregate key details; and the global and local features are adaptively fused through a gating network and then input into a classifier to output an evaluation result. The system further introduces a sample source weighting strategy and a man-machine cooperation closed-loop mechanism to optimize model training. According to the method, the problem of feature alignment caused by soft tissue deformation in an operation is effectively solved, and the objectivity and accuracy of excision cleanliness evaluation are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, specifically to an intelligent assessment method and system for the cleanliness of resection in minimally invasive breast tumor excision surgery. Background Technology

[0002] Minimally invasive breast biopsy is an important clinical method for diagnosing and treating breast lesions. It primarily uses an ultrasound-guided vacuum-assisted biopsy system to completely remove or biopsie breast tumors. Intraoperative assessment of resection cleanliness—accurately determining whether tumor tissue has been completely removed without residue—is a crucial step in determining the success of the surgery and reducing the risk of postoperative recurrence. During the procedure, real-time imaging monitoring of the resection extent and confirmation of the surgical margins is of great significance for ensuring patient prognosis.

[0003] In current techniques, the assessment of intraoperative tumor resection cleanliness primarily relies on high-frequency ultrasound imaging. Surgeons typically acquire ultrasound images of the tumor preoperatively to determine the lesion's location, size, and morphological characteristics, serving as a baseline for surgical planning. After the intraoperative resection is completed, the surgeon uses an ultrasound probe to scan the surgical cavity area, observing the ultrasound echogenicity of the cavity walls and surrounding tissues, and visually comparing the real-time intraoperative images with preoperative images. Clinicians mainly rely on personal experience, subjectively inferring the presence of residual tumor by identifying abnormal areas around the cavity that resemble the echogenicity of the original tumor.

[0004] However, a key challenge in using the above-mentioned assessment method lies in the difficulty of feature alignment caused by soft tissue deformation. Due to the high flexibility of breast tissue, the mechanical traction of the rotary excision blade, the negative pressure suction, and the collapse of the residual cavity after tumor removal during surgery can lead to significant non-rigid deformation of the tissue structure in the surgical wound area. This drastic change in spatial morphology causes the intraoperative ultrasound image to lose its direct geometrical correspondence with the preoperative baseline image. Consequently, the location of lesion features in the preoperative image cannot be accurately mapped to the corresponding coordinates in the intraoperative image, making it difficult to accurately identify tiny residual lesions in complex deformed wounds by relying solely on vision or simple image overlay techniques. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent assessment method and system for the cleanliness of resection during minimally invasive breast tumor excision, thus solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an intelligent assessment method and system for the cleanliness of resection during minimally invasive breast tumor resection. The first aspect of the present invention provides an intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor resection, comprising the following steps: Step S1: Medical Image Acquisition and Preprocessing Preoperative and intraoperative ultrasound images of the patient were acquired and standardized to a uniform size. During the determination of the region of interest (ROI), the preoperative ROI was determined based on the preoperative ultrasound image; based on the intraoperative ultrasound image, the ROI was automatically matched or manually specified by calculating the structural similarity index between each candidate region and the preoperative ROI. Then, using the center of the ROI as a reference, the main preoperative and intraoperative image regions containing global information, as well as multiple preoperative and intraoperative local patches containing local texture information, were extracted.

[0007] Step S2, Feature Extraction A deep neural network model was constructed and utilized to perform convolution operations on the preoperative and intraoperative main image regions to extract global feature maps. Simultaneously, convolution operations were performed on each preoperative and intraoperative local patch to extract local feature matrices. The deep neural network model integrates depthwise separable convolution modules and locally sparse convolution modules to adapt to ultrasound image features.

[0008] Step S3: Feature Enhancement Processing This step associates global features through spatial collaborative attention and aggregates local features through multi-instance pooling. In spatial collaborative attention, projection weight matrices are defined to convert the preoperative master image feature map into query features, and the intraoperative master image feature map into key and value features. The query and key features are flattened and then dot-producted to calculate their spatial correlation, generating an attention score matrix. A scaling factor is introduced to adjust the values, and the matrix is ​​normalized using the Softmax function to obtain the attention weight matrix. Finally, the attention weight matrix is ​​multiplied by the value features and fused with the original preoperative features to generate a collaborative attention fusion feature map. In the multi-instance pooling operation, a two-layer neural network is constructed to calculate the attention score for each patch feature vector. The score is then normalized using the Softmax function to obtain the attention weight. This weight is then used to weight and sum all patch feature vectors to obtain the preoperative patch aggregated features and the intraoperative patch aggregated features, respectively.

[0009] Step S4: Gated Adaptive Feature Fusion A gated network is used to achieve adaptive fusion of global and local features. First, global average pooling is performed on the collaborative attention fusion feature map, and the preoperative and intraoperative patch aggregated features are concatenated and projected. The pooled image features and the projected patch features are concatenated and input into the gated network. Parallel linear layers output normalized weights, and the weight range is constrained using the sigmoid function. Finally, the global and local features are weighted and summed based on these normalized weights to generate the final fused feature vector.

[0010] Step S5: Classification Evaluation and Model Optimization The final fused feature vector is input into the classifier, which outputs an evaluation result of the cleanliness of the excision. During the model training phase, a focus loss function is defined, and the attention given to hard-to-classify samples is adjusted by a class balancing factor and a focus parameter. Simultaneously, a sample source weighting strategy is introduced, assigning different sample weights to samples from manually delineated regions and samples from automatically selected regions. The focus loss function is then weighted based on these sample weights, and the model parameters are updated using a stochastic gradient descent optimizer.

[0011] In addition, the method also includes a visualization feedback step that generates a heatmap using the attention weight matrix in the feature processing process, and a human-computer collaborative closed-loop optimization step that is based on confidence threshold judgment and incorporates doctor verification.

[0012] A second aspect of the present invention provides an intelligent assessment system for the cleanliness of resection during minimally invasive excision of breast tumors, the system comprising: The data preprocessing module is configured to perform image standardization, region of interest matching, and cropping of the main image and local patches; The feature extraction module is configured to run a deep neural network model to extract global and local features. The feature enhancement module is configured to perform spatial collaborative attention processing to correlate preoperative and intraoperative features, and to perform multi-instance pooling operations to aggregate local features; The feature fusion module is configured to use a gated network and an adaptive weight allocation mechanism to fuse the enhanced global features and the aggregated local features. The classification and evaluation module is configured to output evaluation results and perform model training and parameter updates based on a weighted focus loss function.

[0013] This invention provides an intelligent assessment method and system for the cleanliness of breast tumor resection during minimally invasive excision surgery. It offers the following advantages: 1. This invention establishes a spatial mapping relationship between preoperative lesions and intraoperative wound features through a spatial collaborative attention mechanism, solving the feature alignment problem caused by the non-rigid deformation of soft tissue. Simultaneously, it utilizes multi-instance pooling to aggregate texture details from local image patches and combines a gating network to adaptively allocate weights between global morphology and local fine features. This dual fusion strategy of macroscopic structure and microscopic texture effectively overcomes the insufficient expressive power of single-scale features, significantly improving the accuracy of identifying minute residual lesions.

[0014] 2. This invention introduces a sample source weighting strategy and a focus loss function to differentiate training data from different sources. High-confidence samples selected manually are assigned higher weights, while low-confidence samples automatically selected by the algorithm are assigned lower weights, thereby reducing noise interference from automatic labeling errors. Combined with the focus loss function's ability to focus on difficult-to-classify samples, this solves the problems of imbalanced positive and negative samples and inconsistent training data quality, enhancing the model's generalization ability and robustness in complex clinical scenarios.

[0015] 3. This invention maps the attention weights within a deep neural network into an intuitive heatmap through visual feedback and a closed-loop human-machine collaboration mechanism, assisting doctors in understanding the model's focus areas. A confidence-based review process is established, filtering low-confidence samples for doctor verification, and then feeding back the verified correct data to the training set for model iteration. This mechanism not only improves the algorithm's interpretability but also establishes a data-driven continuous optimization path, enabling the evaluation system to continuously adapt to the dynamic changes in clinical data. Attached Figure Description

[0016] Figure 1 This is a flowchart of the model construction process for the present invention; Figure 2 This is a schematic diagram illustrating the model loss and accuracy changes of the present invention; Figure 3 This is a heat map of the present invention; Figure 4 This is a schematic diagram of the simulated clinical practice process of the present invention. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example: Please see the appendix Figure 1 -Appendix Figure 4 This invention provides an intelligent assessment method and system for the cleanliness of breast tumor resection during minimally invasive excision surgery, comprising the following steps: S1 Medical Image Analysis Data Preparation and Preprocessing This step is used to obtain standardized, high-quality image data, providing a reliable foundation for subsequent feature extraction and analysis. Specifically, it includes: S1.1 Medical Image Analysis: Acquisition of Original Images Preoperative and intraoperative ultrasound images of the patient's breast tumor were collected. The images were required to clearly show the texture and morphological information of the tumor area and surrounding tissues, without serious noise interference or artifacts, and to meet the image quality standards for subsequent algorithm analysis.

[0019] S1.2 Medical Image Analysis Image Standardization All acquired raw images undergo uniform standardization processing: First, a rectangular cropping algorithm is used to remove invalid border areas and redundant information containing instrument parameters, annotations, etc. from the image edges; then, bilinear interpolation is used to uniformly adjust the cropped images to a standard size of 224×224 pixels to ensure the size consistency of all input images and avoid the feature extraction effect being affected by size differences.

[0020] S1.3 Region of Interest Matching and Selection in Medical Image Analysis S1.3.1 Preoperative Region of Interest (ROI) Delineation in Medical Image Analysis: Experienced physicians manually delineate the region of interest (ROI) in the preoperative ultrasound image based on their medical expertise, ensuring that the delineated area accurately covers the tumor core and surrounding related tissues.

[0021] S1.3.2 Medical Image Analysis Similarity Calculation: The system uses the Structural Similarity Index (SSIM) algorithm to calculate the similarity between the manually delineated ROI before surgery and each candidate region in the intraoperative ultrasound image. This similarity comprehensively considers the brightness, contrast and structural information of the image to quantitatively evaluate the degree of matching between regions.

[0022] S1.3.3 Intraoperative Region of Interest (ROI) Determination in Medical Image Analysis: A similarity threshold of 0.7 is set. If the similarity between a candidate region during surgery and the preoperative ROI is greater than 0.7, the system automatically selects the candidate region as the intraoperative ROI. If the similarity between all candidate regions and the preoperative ROI is less than or equal to 0.7, the system returns to the doctor's interface, allowing the doctor to re-observe the intraoperative images and manually specify the intraoperative ROI.

[0023] S1.4 Medical Image Analysis: Image Region Cropping S1.4.1 Medical Image Analysis Main Image Region Extraction: Using the center of the selected preoperative region of interest and intraoperative region of interest as reference points, a square main image region of 224×224 pixels is extracted respectively. This region is used to extract the global morphological features of the tumor and wound.

[0024] S1.4.2 Local Patch Extraction for Medical Image Analysis: Using the center of the region of interest before and during surgery as a reference, eight local patches are evenly distributed in the horizontal, vertical and two diagonal directions. The size of each patch is set to 128×128 pixels, and there is no overlap between adjacent patches. This is used to capture subtle morphological and texture changes in the area surrounding the tumor, supplementing the deficiencies of global features.

[0025] S2 Medical Image Analysis Feature Extraction The LSNet deep neural network model was used to extract global and local features from preoperative and intraoperative images. After adaptation and optimization, this model can efficiently capture key information from ultrasound images. The specific steps are as follows: S2.1 Medical Image Analysis Model Configuration The network structure parameters of the LSNet model are set as follows: patch_size=16; the feature extraction stage is divided into 4 consecutive stages, with embeddance dimensions of 64, 128, 192, and 256 for each stage; the key dimension of each stage is 16; the number of convolutional blocks depth of each stage is 1, 2, 3, and 4 for each stage; and the number of attention heads of each stage is 4. The model integrates the RepVGGDW depthwise separable convolution module and the LSConv local sparse convolution module to adapt to the low contrast and strong noise characteristics of ultrasound images, thereby improving the targeting and effectiveness of feature extraction.

[0026] S2.2 Medical Image Analysis: Main Image Feature Extraction The preoperative and intraoperative master images obtained in S1.4.1 are input into the configured LSNet model. After convolution operations, normalization, activation function transformation, and attention calculation at each stage, the output is a preoperative master image feature map with dimensions H×W×256. Intraoperative master image feature map ,in For feature map height, The width of the feature map. and (Determined by the convolution operation rules within the model), and simultaneously outputs preoperative master image pooling feature vectors and intraoperative master image pooling feature vectors with a dimension of 256.

[0027] S2.3 Medical Image Analysis Patch Feature Extraction Each patch from the preoperative patch set and the intraoperative patch set obtained in S1.4.2 is sequentially input into the same LSNet model. After the same feature extraction process as the main image, a feature vector with a dimension of 256 is output for each patch. Finally, a preoperative patch feature matrix with a dimension of 8×256 (8 corresponds to the number of patches and 256 corresponds to the feature vector dimension) and an intraoperative patch feature matrix with a dimension of 8×256 are formed.

[0028] S3 Medical Image Analysis Feature Enhancement Processing Feature enhancement is achieved by capturing globally correlated features through spatial collaborative attention processing and aggregating local subtle features through multi-instance pooling. Specifically, this includes: S3.1 Spatial Co-attention Processing in Medical Image Analysis A spatial collaborative attention module was constructed to establish a bidirectional spatial association between preoperative and intraoperative main image features. The specific steps are as follows: S3.1.1 Medical Image Analysis Feature Projection and Dimension Alignment: Defining the Preoperative Feature Projection Weight Matrix Intraoperative feature key projection weight matrix Intraoperative eigenvalue projection weight matrix ,right and Perform a dimensionality transformation to ensure consistency between the two during attention calculation, resulting in query feature Q, key feature K, and value feature V. The projection calculation formula is as follows:

[0029]

[0030]

[0031] S3.1.2 Flattening the Feature Map of Medical Image Analysis: Flattening the two-dimensional query feature Q, key feature K, and value feature V according to... The sequence is expanded into a one-dimensional sequence, where Q, K, and V all have dimensions of 1. This ensures that the attention mechanism can act on each spatial location and preserve spatial structure information.

[0032] S3.1.3 Medical Image Analysis Attention Score Calculation: Using the expanded Q as the query vector and K as the key vector, the spatial correlation between the two is calculated through dot product operation to obtain the attention score matrix S. To alleviate the problem of excessively large values ​​caused by dot product operation, a scaling factor is introduced. The calculation formula is:

[0033] S3.1.4 Medical Image Analysis Attention Weight Normalization: The attention score matrix S is row-normalized using the Softmax function, ensuring that the sum of the attention weights corresponding to each query position is 1, resulting in the attention weight matrix A. The calculation formula is as follows:

[0034] in Let represent the element in the i-th row and j-th column of the attention score matrix S, where exp is the natural exponential function, and the summation range covers all spatial locations of the key features from k=1 to N.

[0035] S3.1.5 Application of Attention Weights in Medical Image Analysis: Perform matrix multiplication between the attention weight matrix A and the value feature V to obtain the weighted attention feature map. The calculation formula is:

[0036] S3.1.6 Medical Image Analysis, Reshaping, and Fusion: Using the Reshape function to reshape one-dimensional sequence forms Restore to Two-dimensional feature map Then Feature map of the original preoperative master image Element-level additive fusion is performed to preserve the basic information of preoperative features and incorporate the correlation information of intraoperative features, resulting in a collaborative attention fusion feature map. The calculation formula is:

[0037]

[0038] S3.2 Medical Image Analysis: Multi-Instance Pooling (Patch Feature Aggregation) The attention pooling mechanism learned through multiple instances aggregates subtle local features from preoperative and intraoperative patch sets. The specific steps are as follows: S3.2.1 Medical Image Analysis Patch Attention Score Calculation: For each patch feature vector in the preoperative patch feature matrix... (i=1,2,...,8, where 8 is the number of patches), input a two-layer neural network containing ReLU activation functions and calculate its attention score. The first layer of this two-layer neural network is a linear transformation layer, with a weight matrix... Bias vector The second layer is the output layer, with a weight matrix. Bias vector The calculation formula is:

[0039]

[0040] The expression for the ReLU function is: It is used to introduce nonlinear feature transformation to improve feature representation capabilities.

[0041] S3.2.2 Medical Image Analysis Attention Weight Normalization: The attention score of all preoperative patches is normalized using the Softmax function. Normalization is performed to obtain the attention weights for each preoperative patch. To ensure that the sum of all attention weights is 1, the calculation formula is:

[0042] The summation range covers indices j=1 to 8 of all preoperative patches.

[0043] S3.2.3 Local Feature Aggregation in Medical Image Analysis: This involves aggregating the feature vectors of each preoperative patch. With the corresponding attention weights Element-wise multiplication is performed, and then all product results are summed to obtain the preoperative patch aggregated features. Its dimension is 256, and the calculation formula is:

[0044] Following the same procedures as S3.2.1 to S3.2.3, attention scores, weight normalization, and feature aggregation operations are performed on the intraoperative patch feature matrix to obtain the intraoperative patch aggregated features. Its dimension is also 256.

[0045] S4 Medical Image Analysis Feature Fusion Adaptive fusion of global morphological features and local detailed features is achieved through gating networks to enhance the comprehensive expressive power of features. The specific steps are as follows: S4.1 Global Feature Pooling for Medical Image Analysis The collaborative attention fusion feature map obtained in S3.1.6 Global average pooling is performed to calculate the average value of all pixels in each channel of the feature map, extracting the core information of the global morphological features, and obtaining a pooled and fused image feature with a dimension of 256. The calculation formula is:

[0046] Where c is the feature channel index (c=1,2,...,256). This represents the pixel value in the i-th row and j-th column of the c-th channel of the fused feature map.

[0047] S4.2 Medical Image Analysis: Local Feature Mosaic Projection Aggregate the preoperative patch features obtained in S3.2.3 Intraoperative patch aggregation features Concatenate along the channel dimension to obtain the dimension as follows: spliced ​​feature vectors The concatenated feature vector is input into a linear projection layer, and the weight matrix of this linear projection layer is... Bias vector The spliced ​​features are projected back into the fused patch features of dimension 256 through a linear transformation. The calculation formula is:

[0048] S4.3 Medical Image Analysis Gated Adaptive Fusion Will and Concatenate along the channel dimensions to obtain the dimension as follows: spliced ​​feature vectors The concat_all_feat is input into a gating network, which contains two parallel linear layers that output normalized weights. and Both weights are constrained to between 0 and 1 using the Sigmoid function to ensure reasonable weight allocation. Finally, the final fused feature vector, `final_fused_vector`, is obtained through weighted summation, achieving adaptive fusion of global and local features. The calculation formula is as follows:

[0049]

[0050]

[0051] in , These are the weights and biases of the first linear layer, respectively. , Let these be the weights and biases of the second linear layer, respectively. The expression for the Sigmoid function is: .

[0052] S5 Medical Image Analysis, Classification, and Model Training The evaluation results are output through a classifier, and the parameters are optimized through model training to improve the evaluation accuracy and generalization ability. The specific steps are as follows: S5.1 Medical Image Analysis and Classification Calculation The final fused feature vector obtained in S4.3 The input classifier is constructed using fully connected layers combined with a softmax function. The weight matrix of the fully connected layers is also shown. Bias vector Output dimension 2 corresponds to two categories in medical image analysis: clean medical image analysis and unclean medical image analysis. The calculation formula is as follows:

[0053]

[0054] Where logits is the raw output of the fully connected layer. is the output value for the k-th class (k=1 corresponds to the clean medical image analysis category, and k=2 corresponds to the unclean medical image analysis category), and prob is the predicted probability for the two classes.

[0055] S5.2 Determination of Medical Image Analysis Results The category with the highest predicted probability is selected as the final evaluation result: when When the medical image analysis is clean, it indicates that there is no residual tumor in the surgical wound; when If the output medical image analysis is unclean, it indicates that there may be residual tumor in the surgical wound.

[0056] S5.3 Definition of Loss Function for Medical Image Analysis To address potential class imbalance in the dataset and encourage the model to focus on harder-to-classify samples, a focus loss (for medical image analysis) is used as the training loss function. The calculation formula is as follows:

[0057] in This is a category balancing factor, with a value ranging from 0 to 1, used to balance the difference in sample size between the clean and unclean medical image categories in medical image analysis. Let be the model's predicted probability for the true class t. When the true class of the sample is medical image analysis (clean medical image analysis), t=1. When the true category of the sample is medical image analysis (not clean medical image analysis), t=2. ; This is a focusing parameter, ranging from 0 to 5, used to adjust the degree of attention given to difficult-to-classify samples.

[0058] S5.4 Medical Image Analysis Sample Source Weighting To differentiate the contributions of data from different sources to model training, a sample source weighting strategy is introduced: weights are set for image data manually selected by doctors. Image data weights generated by the algorithm for automatic segmentation The weight w of each sample is determined according to its source, that is, the weight w of manually delineated samples is 1.0, and the weight w of automatically segmented samples is 0.5.

[0059] S5.5 Medical Image Analysis Batch Loss Calculation The final training batch loss is the average of the weighted focus losses, calculated using the following formula:

[0060] Where N is the number of samples in the training batch. The source weight of the k-th sample. Let be the focus loss for the k-th sample.

[0061] S5.6 Medical Image Analysis Optimizer Configuration The model parameters are updated using a stochastic gradient descent optimizer with the following configuration: the initial learning rate is set to 0.001, the learning rate is dynamically adjusted using a cosine annealing strategy, the weight decay coefficient is set to 0.0001, the momentum parameter is set to 0.9, the number of training iterations is set to 40 epochs, and the batch size is set to 32.

[0062] S5.7 Medical Image Analysis Model Iterative Update The training dataset is randomly divided into a training set and a validation set in an 8:2 ratio. The training set is used for model parameter training, and the validation set is used for model performance evaluation. During each iteration, the batch loss is calculated using the training set. The model updates all learnable parameters through backpropagation; the model's classification accuracy is evaluated using a validation set; training stops when the validation set accuracy does not improve for 5 consecutive epochs, and the current optimal model parameters are saved to avoid model overfitting.

[0063] S6 Medical Image Analysis, Visualization, and Human-Computer Collaboration The heatmap provides visual feedback and establishes a closed loop of human-machine collaboration to continuously optimize model performance. The specific steps are as follows: S6.1 Medical Image Analysis Heatmap Generation S6.1.1 Medical Image Analysis Attention Weight Extraction: Extract the attention weight matrix A of the spatial collaborative attention module and the attention weights of the multi-instance pooling module from the trained model. .

[0064] S6.1.2 Medical Image Analysis Heatmap Mapping: Reshaping the Attention Weight Matrix A into The two-dimensional matrix is ​​normalized by min-max to map the weight values ​​to a grayscale range of 0 to 255. Then, a pseudo-color mapping algorithm is used to convert the grayscale values ​​into a spatial collaborative attention heatmap. The larger the weight value, the closer the color is to red, which intuitively shows the correlation between the preoperative lesion area and the intraoperative wound area. The attention weights of multi-instance pooling are used. Mapping to the corresponding patch location, the color intensity indicates the importance of the local area. The larger the weight value, the darker the color of the patch, indicating that the local area has a greater impact on the cleanliness assessment.

[0065] S6.1.3 Visual Overlay for Medical Image Analysis: Spatial collaborative attention heatmap and multi-instance pooling heatmap are overlaid on the preoperative original image and intraoperative original image respectively to form a visual feedback result, enabling doctors to intuitively observe the focus of the model.

[0066] S6.2 Human-Machine Collaboration Closed Loop in Medical Image Analysis S6.2.1 Medical Image Analysis Model Prediction Output: Input the patient's preoperative and intraoperative ultrasound images into the trained optimal model. The model outputs the classification results of clean or unclean medical image analysis, the corresponding prediction probabilities, and the superimposed heatmap.

[0067] S6.2.2 Medical Image Analysis Doctor Review: Doctors review the model's output classification results, predicted probabilities, and heatmaps, paying particular attention to low-confidence cases with predicted probabilities below 0.8. By comparing the original ultrasound images with the heatmaps, doctors verify whether the model's identified regions of interest are consistent with medical common sense and clinical experience, judging the reasonableness of the model's predictions and providing a result indicating whether the medical image analysis is correct or incorrect.

[0068] S6.2.3 Medical Image Analysis Sample Labeling and Storage: Link and store the model prediction results, doctor's test results and corresponding ultrasound image data of all patient samples to establish a sample database.

[0069] S6.2.4 Iterative Optimization of Medical Image Analysis Model: Periodically select samples from the sample database that doctors have marked as medical image analysis errors, add them to the training dataset, retrain the model according to the model training process in S5, update the model parameters, and form a continuous improvement closed loop of medical image analysis prediction-validation-labeling-retraining, thereby continuously improving the model's evaluation accuracy and clinical applicability.

[0070] II. Intelligent Assessment System for Cleanliness of Breast Tumor Resection During Minimally Invasive Excision This system provides hardware and software support for the implementation of the above-mentioned evaluation methods. Through the collaborative work of various functional modules, it realizes the automated and standardized execution of the evaluation process, and mainly includes the following functional modules: Data input module: Using ultrasound image acquisition equipment or data interface, it acquires preoperative and intraoperative ultrasound images of patients with breast tumors, supports common medical image formats such as DICOM and JPG, and provides raw data support for evaluation.

[0071] Preprocessing module: Equipped with algorithms such as image cropping, resizing, and similarity calculation, it performs image standardization, region of interest matching and selection, and image region cropping operations in S1, outputting standardized main image and patch image to ensure that the data quality meets the requirements of subsequent processing.

[0072] Feature processing module: Built-in configured LSNet deep neural network, integrating spatial collaborative attention submodule and multi-instance pooling submodule, performs feature extraction in S2, spatial collaborative attention processing and multi-instance pooling operation in S3, and outputs collaborative attention fusion feature map and aggregated local features.

[0073] Feature fusion module: Deploys a global average pooling algorithm, a linear projection layer, and a gated network to perform global feature pooling, local feature concatenation and projection, and gated adaptive fusion operations using S4, outputting the final fused feature vector. .

[0074] Classification Training Module: Contains a classifier and model training engine, performs S5 classification calculation, loss function calculation, optimizer parameter update and model iteration update operations, outputs evaluation results of clean or unclean medical image analysis, and continuously optimizes model parameters.

[0075] Visualization and Interaction Module: Integrates heatmap generation algorithm and human-computer interaction interface, performs S6 heatmap generation, overlay and display operations, provides visualization of model prediction results, prediction probabilities and heatmaps; supports doctors to input and store test results, realizes human-computer collaborative feedback and model iterative optimization, and builds a closed-loop system for continuous improvement.

[0076] Each module is seamlessly connected through a data interface and automatically executes operations according to the steps of the assessment method without human intervention. This provides clinicians with an objective, fast, and accurate tool for assessing intraoperative wound cleanliness, while continuously improving the reliability and clinical suitability of the assessment through human-machine collaboration.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent assessment method for the cleanliness of breast tumor resection during minimally invasive excision surgery, characterized in that, Includes the following steps: Step S1: Acquire preoperative and intraoperative ultrasound images of the patient's breast tumor and standardize the image size through normalization processing; determine the preoperative region of interest based on the preoperative ultrasound image and the intraoperative region of interest based on the intraoperative ultrasound image; crop the preoperative main image region and extract multiple preoperative local patches based on the center of the preoperative region of interest, and crop the intraoperative main image region and extract multiple intraoperative local patches based on the center of the intraoperative region of interest. Step S2: Construct a deep neural network model, and use the deep neural network model to extract features from the preoperative main image region and the intraoperative main image region to obtain the preoperative main image feature map and the intraoperative main image feature map; use the deep neural network model to extract features from each preoperative local patch and each intraoperative local patch to obtain the preoperative patch feature matrix and the intraoperative patch feature matrix. Step S3: Perform spatial collaborative attention processing on the preoperative main image feature map and the intraoperative main image feature map to obtain a collaborative attention fusion feature map; perform multi-instance pooling operation on the preoperative patch feature matrix and the intraoperative patch feature matrix to aggregate the preoperative patch aggregated features and the intraoperative patch aggregated features. Step S4: Adaptively fuse the collaborative attention fusion feature map, the preoperative patch aggregated feature, and the intraoperative patch aggregated feature through a gating network to obtain the final fused feature vector; Step S5: Input the final fused feature vector into the classifier and output the evaluation result of the cleanliness of the resection.

2. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, The specific steps for determining the intraoperative region of interest based on intraoperative ultrasound images include: Calculate the structural similarity index between the preoperative region of interest and each candidate region in the intraoperative ultrasound image; A similarity threshold is set. If the structural similarity index between the candidate region and the preoperative region of interest is greater than the similarity threshold, the candidate region with the largest structural similarity index is automatically selected as the intraoperative region of interest. If the structural similarity index of all candidate regions and the preoperative region of interest is less than or equal to the similarity threshold, then the intraoperative region of interest is manually specified by receiving a manual instruction.

3. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, The deep neural network model integrates a RepVGGDW depthwise separable convolution module and an LSConv local sparse convolution module. The feature extraction stage of the deep neural network model is divided into multiple consecutive stages, each with different embedding dimensions and the number of convolutional blocks.

4. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, The spatial collaborative attention processing procedure for the preoperative master image feature map and the intraoperative master image feature map specifically includes the following steps: Define the preoperative feature projection weight matrix, the intraoperative feature key projection weight matrix, and the intraoperative eigenvalue projection weight matrix; The preoperative feature projection weight matrix is ​​used to perform dimensional transformation on the preoperative main image feature map to obtain query features. The intraoperative feature key projection weight matrix is ​​used to perform dimensional transformation on the intraoperative main image feature map to obtain key features. The intraoperative feature value projection weight matrix is ​​used to perform dimensional transformation on the intraoperative main image feature map to obtain value features. Flatten the query features and the key features into a one-dimensional sequence, and calculate the spatial correlation between the query features and the key features by dot product operation to obtain the attention score matrix; A scaling factor is introduced to adjust the numerical values ​​of the attention score matrix, and the attention score matrix is ​​normalized using the Softmax function to obtain the attention weight matrix. Perform matrix multiplication between the attention weight matrix and the value features to obtain a weighted attention feature map; The weighted attention feature map is reshaped into a two-dimensional feature map, and the reshaped two-dimensional feature map is fused with the original preoperative master image feature map element-wise to obtain the collaborative attention fusion feature map.

5. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, The multi-instance pooling operation performed on the preoperative patch feature matrix and the intraoperative patch feature matrix specifically includes the following steps: Construct a two-layer neural network that includes a linear transformation layer and an output layer; Each preoperative patch feature vector in the preoperative patch feature matrix is ​​input into the two-layer neural network to calculate the attention score of each preoperative patch feature vector. The attention scores of all the preoperative patch feature vectors are normalized using the Softmax function to obtain the attention weight of each preoperative patch feature vector. Each of the preoperative patch feature vectors is element-wise multiplied with the corresponding attention weight, and all product results are summed to obtain the preoperative patch aggregated feature. The intraoperative patch feature matrix is ​​calculated using the same procedure to obtain the intraoperative patch aggregated features.

6. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, The adaptive fusion of the collaborative attention fusion feature map, the preoperative patch aggregated feature, and the intraoperative patch aggregated feature through a gating network specifically includes the following steps: The collaborative attention fusion feature map is subjected to global average pooling to obtain pooled fused image features; The preoperative patch aggregated features and the intraoperative patch aggregated features are concatenated along the channel dimension, and the concatenated vector is projected back to the preset dimension through a linear projection layer to obtain the fused patch features. The pooled fused image features and the fused patch features are concatenated along the channel dimension and then input into the gating network. The gated network comprises two parallel linear layers, which output normalized weights respectively, and the Sigmoid function is used to constrain the normalized weights to between zero and one. Using the normalized weights output by the gating network, the pooled fused image features and the fused patch features are weighted and summed to obtain the final fused feature vector.

7. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, Step S5 also includes model training, specifically comprising the following steps: Define a focus loss function, which includes a class balance factor for balancing differences in sample numbers and a focus parameter for adjusting the degree of attention to hard-to-classify samples; A sample source weighting strategy is introduced, setting different sample weights for image data from different sources; For samples whose intraoperative region of interest was manually delineated, a first sample weight was assigned; For samples whose regions of interest are automatically selected by the algorithm during the operation, a second sample weight is set, and the value of the first sample weight is greater than the value of the second sample weight. The focus loss function is weighted based on the sample weights to obtain the final training batch loss, and the parameters of the deep neural network model are updated using a stochastic gradient descent optimizer.

8. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, Step S3 also includes visual feedback processing, specifically including the following steps: Extract the attention weight matrix in the spatial collaborative attention processing process, and the attention weights in the multi-instance pooling operation process; The attention weight matrix is ​​converted into a spatial co-attention heatmap using a pseudo-color mapping algorithm, and the spatial co-attention heatmap is superimposed on the preoperative ultrasound image and the intraoperative ultrasound image. Based on the attention weights of the multi-instance pooling operation, markers with different color depths are generated at the corresponding local patch positions to form a multi-instance pooling heatmap, which is then overlaid and displayed.

9. The intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in claim 1, characterized in that, Step S5 further includes human-machine collaboration closed-loop optimization processing, specifically including the following steps: Obtain the predicted probability output by the classifier. If the predicted probability is lower than a preset confidence threshold, prompt the doctor to conduct a verification test. The system receives test results input by the doctor. If the test results indicate that the classifier's evaluation result is incorrect, the corresponding preoperative ultrasound image, the intraoperative ultrasound image, and the correct test results are associated and stored in the sample database. Samples labeled as incorrect are periodically extracted from the sample database and added to the training dataset to retrain the deep neural network model.

10. An intelligent assessment system for the cleanliness of resection during minimally invasive breast tumor excision, used to implement the intelligent assessment method for the cleanliness of resection during minimally invasive breast tumor excision as described in any one of claims 1-9, characterized in that, Includes the following modules: The data preprocessing module is used to acquire the patient's preoperative and intraoperative ultrasound images, perform image standardization processing, determine the preoperative and intraoperative regions of interest, and extract the preoperative main image region, the intraoperative main image region, as well as multiple preoperative local patches and multiple intraoperative local patches. The feature extraction module is used to run a deep neural network model to extract global features of the preoperative main image region and the intraoperative main image region, as well as local features of the preoperative local patch and the intraoperative local patch, respectively. The feature enhancement module performs spatial collaborative attention processing to correlate preoperative and intraoperative master image features and performs multi-instance pooling operations to aggregate local patch features. The feature fusion module is used to fuse the enhanced global features and the aggregated local features through a gated network and an adaptive weight allocation mechanism to generate the final fused feature vector. The classification and evaluation module is used to map the final fused feature vector to the resection cleanliness category, and to perform model training and parameter updates based on the weighted focus loss function.

Images