Medical image segmentation method and system based on error-aware adaptive loss

Abstract

The application discloses a medical image segmentation method and system based on error-aware adaptive loss, which firstly acquires a multi-modal medical image and carries out pretreatment.Secondly, the pre-trained SAM network model weight is loaded and fine-tuned, and the feature extraction is carried out on the pretreated multi-modal medical image.Then, a structure consistency enhancement mechanism is introduced, and the extracted features are optimized: a redundancy perception unit is proposed to perform region-level overlapping perception division on the feature map, and the space coverage scale and the promotion granularity are adjusted.Finally, the error type in the image feature learning process is dynamically weighted combined with the error perception mechanism, and the loss function weight is adjusted according to the false positive and false negative distribution of the image feature in the training process.The application not only significantly improves the continuity modeling capability of the small lesion boundary in the image feature, but also effectively suppresses the missed detection problem of the false negative area through dynamic adjustment of the error weight.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and artificial intelligence assisted diagnosis, and particularly relates to a medical image segmentation method and system based on error perception adaptive loss. BACKGROUND

[0002] With the development of artificial intelligence technology, medical image segmentation has become a key technology in the key links of auxiliary diagnosis, lesion identification and preoperative evaluation. Deep convolutional neural network (CNN) is widely used in medical image segmentation tasks, typical representatives include U-Net and its derivative architecture, ResUNet based on residual mechanism and V-Net for volume data. Although the structure of the segmentation model is constantly evolving, such networks usually rely on fine-tuning on specific tasks, and model optimization is heavily dependent on specific data characteristics and prior knowledge. Under these model frameworks, the traditional loss function design generally lacks the ability to systematically model the actual characteristics of medical images such as "foreground sparsity", "structural continuity" and "error sensitivity". The performance bottleneck of the segmentation model is often not due to the lack of network expression ability, but the underlying loss function fails to accurately capture the key segmentation challenges in medical images.

[0003] In typical medical image data, there are obvious problems such as class imbalance (foreground pixel ratio is extremely low), boundary blur and fracture, cross-modal gray inconsistency, and significant differences in clinical cost of different error types. Existing loss functions can be roughly divided into global dependence type (such as Dice Loss) and point-by-point independence type (such as cross-entropy). The former alleviates the class imbalance problem through global pixel statistics, but the gradient of a single pixel is easily dominated by the overall prediction distribution in training, leading to insensitive response to small targets or edge structures, and easy "structural fracture" or overfitting of background areas; while the latter is more stable, but ignores spatial relationships, often outputting fragmented and isolated prediction masks in complex structures.

[0004] To alleviate these problems, various improvement strategies have been proposed, such as: loss function Tversky Loss, focal loss Focal Loss, which enhances the recognition of minority classes by adjusting the hyperparameters; regional block-level loss attempts to improve the discrimination of local features; attention mechanism improves the model's response to key areas. These methods have the following limitations: the error penalty weight is statically set and cannot dynamically adapt to the difference in clinical importance of false positives (FP) and false negatives (FN); regional segmentation methods are mostly based on hard segmentation, which destroys the structural continuity of the image; most are customized for specific models, lack of unified and easily integrated loss mechanism, and difficult to generalize to multi-modal and multi-task medical scenarios.

[0005] Therefore, in the current development context of medical image segmentation, there is an urgent need for a loss function modeling method that is oriented towards the lesion structure and error characteristics in real medical scenarios. This method should be able to effectively enhance structural continuity and improve the ability to perceive small lesions without relying on changes to the model architecture, and should be able to adaptively adjust the penalty strategies for FN and FP according to different training stages. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a medical image segmentation method and system based on error-aware adaptive loss. It aims to solve common problems in existing methods, such as class imbalance, boundary fragmentation, structural blurring, and asymmetric handling of false positives (FP) and false negatives (FN). This method is suitable for accurate boundary extraction and structure preservation of minute lesions in multimodal medical images, and has significant practical value, particularly in segmentation applications facing high class imbalance, blurred lesion boundaries, high risk of structural fragmentation, and significant differences in clinical costs due to false positives and false negatives. Combining the rich boundary-aware prior knowledge obtained by pre-training the SAM image segmentation model on a large number of natural image datasets, it possesses the ability to quickly capture key structural edge features in images. Furthermore, it requires only minimal parameter fine-tuning to transfer and adapt to the multimodal inputs and complex lesion structure recognition requirements of medical image segmentation tasks, aiming to solve the core bottlenecks of current mainstream segmentation methods in terms of structure preservation and dynamic error control. This method, while maintaining a lightweight model, fully leverages the feature extraction capabilities of the pre-trained model. Through synergy with the enhanced structural continuity and error-aware adaptive loss function constructed in this invention, it achieves more precise localization and correction of minute lesions, blurred boundaries, and structural fracture regions. This effectively alleviates the gradient shift problem caused by class imbalance and enhances the model's sensitivity to high-risk clinical errors. This method is particularly suitable for boundary extraction tasks of lesions with complex structures or unclear boundaries in multimodal medical images, ensuring the consistency and integrity of anatomical structures while improving segmentation accuracy.

[0007] The medical image segmentation method based on error-aware adaptive loss includes the following steps:

[0008] 1. Acquire and preprocess multimodal medical images: Collect target medical image data, including but not limited to multimodal medical images such as magnetic resonance imaging (MRI), computed tomography (CT), or ultrasound. Perform standardization, size normalization, denoising, and enhancement operations on the images to provide a unified input basis for subsequent feature extraction and training.

[0009] 2. Load and fine-tune the pre-trained SAM network model weights to extract features from the pre-processed multimodal medical images: Based on the SAM framework, load its pre-trained weights and fine-tune them on a small scale to adapt them to the feature distribution of the medical images.

[0010] 3. Introducing a structural consistency enhancement mechanism in the optimization feature learning process, optimizing the extracted features: For the difficulty of structural discontinuity in micro lesions and boundary fuzzy areas, the redundant perception unit proposed in the application is used for region-level overlapping perception division of the feature map. By adjusting the spatial coverage scale and the promotion granularity, the segmentation network retains the structure information across the region in local learning, effectively prevents boundary fracture and morphological distortion, and finally obtains the segmented image.

[0011] 4. Combining error perception mechanism to dynamically weight the error types in image feature learning process: According to the false positive and false negative distribution of image features in the training process, the loss function weight is adjusted adaptively to enhance the response ability of the model to key error types.

[0012] 5. Comparison of different segmentation methods for image feature learning effect: The SAM model after optimization feature learning is compared with different segmentation models for segmentation performance, verifying the advantages of the proposed method in boundary preservation, error control and generalization ability.

[0013] The medical image segmentation system based on error perception adaptive loss includes the following modules:

[0014] An image data module for obtaining multi-modal medical images and pre-processing.

[0015] A feature extraction module for loading pre-trained SAM network model weights and fine-tuning, and extracting features from pre-processed multi-modal medical images.

[0016] A consistency enhancement module for introducing a structural consistency enhancement mechanism to optimize the extracted features, and finally obtaining a segmented image.

[0017] An error perception module for combining error perception mechanism to dynamically weight the error types in image feature learning process, and adjusting the loss function weight according to the false positive and false negative distribution of image features in the training process.

[0018] Compared with the existing mainstream medical image segmentation method, the SAM segmentation framework is optimized in the application, the boundary perception advantage of the pre-training model in natural images is transferred to the medical image scene, and the structure continuity enhancement mechanism and the error perception adaptive loss function proposed in the application are combined to realize end-to-end collaborative optimization. The method not only significantly improves the model's ability to model the continuity of the small lesion boundary in the image features, but also effectively suppresses the missed detection problem of false negative areas by dynamically adjusting the error weight. The experimental results on multiple public medical image datasets show that the segmentation system based on the SAM model and the loss function of the application is superior to the traditional loss function method in terms of Dice coefficient, structural consistency and false negative control, and has higher precision, robustness and clinical application potential. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Figure 1 is a flow chart of fine-tuning the SAM model combined with the ITSRS loss function;

[0020] Figure 2 Figure 2 is an effect diagram of the interaction of red features in a complete image (the right side is a local enlargement);

[0021] Figure 3 Figure 3 is an effect diagram of the interaction of red features in a local area (the right side is a local enlargement). DETAILED DESCRIPTION

[0022] The method described in the application mainly solves the core problems of existing medical image segmentation technology in the following aspects: first, the existing loss function is easily affected by the background pixel dominant gradient when dealing with small volume, irregular structure or boundary feathering lesions, resulting in missed detection or boundary fracture of the segmentation model; second, the existing method generally uses fixed error weight, which cannot dynamically respond to the actual influence of false positives (FP) and false negatives (FN) in different diseases and different stages, and is difficult to meet the clinical difference tolerance strategy such as "preferably misdiagnosed, not missed"; third, the structure continuity is weak, especially when using non-overlapping local loss, the cross-region connection relationship is easy to lose, affecting the anatomical consistency and usability.

[0023] To this end, the application proposes a medical image segmentation method and (ITSRS) based on error-aware adaptive loss. Among them, by dividing the training image into multiple partially overlapping region perception units SRS, and adjusting the spatial coverage scale and the advancing granularity, the model receives information feedback from different structure granularities during loss calculation, realizing the implicit multi-scale structure modeling capability. This mechanism is particularly suitable for the identification and positioning of fuzzy boundaries, complex structures or small lesion areas, and can effectively avoid structure breakage and missed detection problems; at the same time, combined with the IT adaptive method, the internal weight of the loss function is automatically adjusted according to the real-time error distribution (FN and FP ratio), enhancing the model's response capability to the higher risk error type, thereby improving the overall segmentation robustness and clinical applicability. The loss function is combined with the SAM segmentation framework with strong generalization ability and good semantic perception characteristics, and the robust representation ability of SAM to structure boundaries in multi-modal input and complex scenes is used to significantly improve the expression quality of initial feature extraction. Combined with the error-aware and structure continuity adaptive loss method proposed in the application, the image features extracted by SAM can be further differentially reweighted and locally structured to guide the model to pay more attention to high-error-risk areas and structure-broken locations during training, realizing segmentation optimization for clinical concerns. In this way, the segmentation model not only improves the detection capability of small lesions, fuzzy boundaries and low-contrast structures, but also shows higher accuracy and robustness in actual clinical applications.

[0024] The technical solutions of the application will be further described in detail below in combination with the drawings and embodiments:

[0025] Step one: Obtain multi-modal medical images and perform preprocessing.

[0026] First, typical case images are collected from publicly available multi-modal medical image datasets, including dermoscopy images, endoscopy images, CT, MRI, etc. imaging modalities, covering different image feature expression structures from two-dimensional color images to three-dimensional medical volume data, ensuring that the training data is representative in terms of spatial scale, tissue structure, blur degree, etc. The specific description is as follows:

[0027] 1. PH2 dataset: This dataset is composed of high-resolution dermoscopy images. bmp, the image modality is dermoscopy. The segmentation object is mainly the skin lesion area, especially the melanoma, benign pigmented nevus and other different morphological epidermal lesions. In this kind of image, the lesion boundary and the surrounding skin color difference are not obvious, and the structure texture is similar, which leads to the blurring of image features, and the structure continuity modeling of small areas is challenged.

[0028] 2. Kvasir-SEG dataset: This dataset contains color images in.jpg format, with the modality of endoscopy. The segmentation object is the gastrointestinal polyp and related lesion area. Due to the strong light variation, mucus reflection, occlusion and organ folds in endoscopic images, the lesion edge often presents irregular shape or local disappearance, and the continuity of image feature structure is weak.

[0029] 3. BraTS2021 dataset: composed of four MRI sequence images, with the image suffix.nii.gz, and the modality of multi-modal MRI (T1, T1Gd, T2, FLAIR). The main segmentation object is the brain tumor area, including enhanced tumor (ET), tumor core (TC) and whole tumor area (WT) three structure sub-areas. The lesion in this dataset often presents as a mixed area, with fuzzy boundary transition or exudative diffusion. The intensity distribution difference between different modalities in the image feature is large, and it is easy to appear fracture or misclassification on the anatomical structure

[0030] 4. PDDCA dataset: composed of head and neck CT images, with the image suffix.nrrd, and the modality of CT. The main segmentation object is the important organs of head and neck (such as parotid gland, mandible, spinal cord, etc.). Due to the limited tissue contrast of CT images, the edges of multiple organs are close to the background gray scale, the boundary expression ability of image features is weak, and the anatomical structures are closely intertwined, which is easy to appear false connection or structure fracture in segmentation.

[0031] After obtaining the above image data, in order to ensure the uniformity and stability of the subsequent image feature extraction process, all data are preprocessed according to the following steps:

[0032] Firstly, the voxel spacing standardization processing is performed on all three-dimensional medical images. The image is adjusted to a uniform spatial resolution through resampling operation, usually to ensure that the data from different sources have comparability in spatial scale, and prevent the distortion or shift of image features in space due to the inconsistency of original voxel size.

[0033] Secondly, intensity normalization processing is performed on all images to unify the range of different image modalities in gray scale or intensity distribution. For MRI images, only the non-zero voxels in the foreground area are retained, and the intensity values are standardized, so that the model can obtain consistent contrast image feature information at the input stage, effectively suppressing the influence of modal difference on the training process. For CT images, the truncation strategy is used and mapped to a uniform range.

[0034] Then, the region of interest extraction and size normalization processing is performed. The image is subjected to foreground detection and ROI cropping to remove invalid background area, and all two-dimensional images are uniformly adjusted to The dimensions and 3D images are adjusted to a fixed volume size. .

[0035] Finally, data augmentation strategies are applied to enhance the diversity and generalization ability of image features during training. Various combinations of random rotation (angle range ±15°), mirror flipping (axial randomness), elastic deformation, brightness and contrast perturbations are used to further expand the feature space accessed by the model.

[0036] Through the above standardization and enhancement process, all input images have a unified and standardized image feature representation foundation before entering the network training stage, providing stable and reliable data support for subsequent high-quality feature extraction, structural modeling and error adjustment.

[0037] Step 2: Load and fine-tune the weights of the pre-trained SAM network model, and extract features from the pre-processed multimodal medical images.

[0038] This invention selects SAM as the basic segmentation network architecture, leveraging its prompt-based segmentation mechanism to achieve excellent cross-task transfer capabilities in structural boundary localization tasks. This mechanism guides the model to generate structured segmentation results through external cues (such as points, boxes, and masks), offering advantages such as high-precision boundary recovery, strong generalization ability, and high parameter utilization. However, SAM is primarily pre-trained on large-scale natural image datasets, whose image feature distribution differs significantly from medical images. Especially in scenarios with blurred tissue boundaries and low contrast in lesion areas, the original model's response to image features lacks the structural sensitivity required for medical tasks. Furthermore, the inherent global attention mechanism of the Transformer architecture suffers from local detail loss during low-level feature aggregation, easily weakening the representation of minute structures or boundary information in the image, leading to decreased prediction accuracy for small lesion areas.

[0039] To address the aforementioned issues, this invention introduces a structure-guided fine-tuning mechanism based on the SAM framework. Embedded features from the outputs of several key layers are selected and extracted from the SAM encoder. And by fusing these image features from different levels across layers, an enhanced representation is constructed. ,in Indicates a specific number of floors. , These are the height and width of the feature map, respectively. This represents the channel dimension. Because these mid-level features achieve a good balance between semantic abstraction and spatial structure, this invention utilizes... Constructing structurally enhanced feature representations through cross-layer fusion ,in It is the ReLU activation function, used to enhance the nonlinear response; is a batch normalization function for keeping the inter-channel statistics consistent; are the convolution kernel weights corresponding to the input features, respectively, and the convolution operation extracts the local spatial response. The fused image features have both cross-layer structural information and local boundary clarity, which can significantly improve the segmentation accuracy and stability of the fine-tuned SAM model in medical images. At the same time, since the above adjustment only affects the image feature path and does not interfere with the original prompt guidance mechanism, SAM has both semantic generalization ability and more targeted expression ability for medical image structures. This feature output will serve as the basis for modeling subsequent structural consistency loss and error weighting mechanisms.

[0040] Step three: Introduce a structural consistency enhancement mechanism to optimize the extracted features.

[0041] In medical image segmentation, DSC (Dice Similarity Coefficient) is widely used to evaluate model performance as an indicator of the overlap between predicted results and true labels. Based on DSC, Dice Loss is proposed as a loss function to optimize the segmentation model, aiming to maximize DSC to improve segmentation accuracy. By taking the partial derivative of , it is easy to see that the gradient of is not only affected by its own predicted probability and true label, but also by the predicted probabilities and true labels of all pixels in the entire image. This means that even if the prediction of some pixels is very accurate, its gradient will still be disturbed by the uncertainty of the prediction of other pixels. This global dependence leads to the prediction of accurate pixels receiving additional gradients during backpropagation, increasing the risk of model overfitting.

[0042] To solve the inherent defects of the global dependence loss, a feasible approach is to divide the image into K non-overlapping small regions , ,…, , and calculate the loss in each region independently. Compared with the traditional Dice Loss, the region-specific loss method (RS, Region-Specific Loss) calculates the loss locally, so that each pixel is only affected by the pixels in its own region when calculating the gradient, significantly reducing the interference of global information on local gradients and improving the model's attention to the target region, thereby improving the segmentation performance.

[0043] However, this hard split brings a more headache problem: each non-overlapping region only calculates loss based on its own prediction results, which leads to the lack of context understanding of surrounding areas in the back propagation of the loss function, and is easy to cause boundary ambiguity or misjudgment. For example, in Figure 1 the partition loss method for colon polyps, the location information such as the mucosal muscle layer may not be included in the back propagation of the loss (in the mucosal layer of the colon and rectum, polyps mainly originate from the crypt opening of the epithelial layer and protrude towards the intestinal cavity). Therefore, how to strengthen the connection between these hard-split regions, especially the information utilization at the edge of each small region, is the key to further improve these loss functions.

[0044] To solve the structural rupture problem in the loss back propagation of the model boundary feature learning process, a perception unit is introduced to enhance the structural consistency, and the key point correlation modeling idea in the pose estimation OpenPifPaf is used to introduce the following form to assist in explaining the necessity of the structure perception unit. The positioning body part technology PIFs (Part Intensity Fields) in the OpenPifPaf model is used to determine which pixels in the image are key points (similar to determining the ROI region in the segmentation model). Through a set of two-dimensional heat maps, each heat map corresponds to a specific key point category. For the i-th object of each category of key points k in the image , the corresponding PIF ground truth is generated as the confidence of whether each pixel point is a key point k:

[0045]

[0046]

[0047]

[0048] where σ is the standard deviation of the Gaussian kernel, which determines the extension range of the heat map, N is the total number of objects of the key point category k, is the pixel coordinate in the image. For multiple key points of the same category, multiple Gaussian distributions should be superimposed to obtain the final PIF ground truth heat map of the category.

[0049] However, under the method of partition loss calculation, the confidence calculation formula of each key point becomes:

[0050]

[0051] It can be seen from the formula that the ground truth label calculation of each region is only affected by the superposition of the Gaussian distribution of key points within the region, and other key points outside the region cannot participate in the label calculation within the region, thereby destroying the continuity and structural integrity of the image. Figure 2 and Figure 3 The comparison of the two shows the process.

[0052] To solve the above problems, a loss function SRS Loss is proposed. By deploying multiple perception units on the enhanced representation boundary, and by the local overlap of perception units, a balance is achieved between global information and local features, maintaining the structural integrity of the image. Its loss calculation formula is:

[0053]

[0054] wherein represents the mthperception unit, M is the total number of perception units existing around the target region, is the predicted value of the ithpixel point, is the label corresponding to the ithpixel point, is a coefficient to prevent zero. SRS Loss covers a larger area through the multiple overlapping region perception unit coupling mechanism, ensuring that the structural relationship between pixel points is preserved, and the model can utilize more contextual information when calculating loss, improving the recognition ability of the target region. At the same time, due to the existence of overlapping areas, the prediction results of multiple perception units for the same pixel will be integrated, making the gradient update smoother and more stable compared to the hard segmentation method, which helps the model to optimize more effectively in the global range.

[0055] To further verify the effectiveness of SRS Loss in boundary feature learning, comparative experiments were conducted on two datasets, Kvasir-SEG and PH², which have significant boundary segmentation challenges. The specific results are shown in Table 1 and Table 2. Table 1 shows the comparison of ITSRS loss and other traditional methods on Kvasir-SEG, where Baseline is the combination of loss functions Dice Loss and CELoss; Table 2 shows the comparison of different losses on PH², where BL is the boundary loss, FTL is the focal Tversky loss, and UFL is the unified focal loss. In the experimental design, the fine-tuned SAM model with SRS loss is compared with the following three groups of baselines: (1) Baseline, which is the original SAM combined with Dice Loss and cross-entropy (CE Loss); (2) Baseline+RS, which introduces a fixed partition segmentation mechanism to the fine-tuned SAM model, with partition block sizes set to 16x16 and 64x64.

[0056] Table 1. Comparison of ITSRS loss and other traditional methods on Kvasir-SEG

[0057]

[0058] Table 2. Comparison of different losses on PH²

[0059]

[0060] In the colon polyp segmentation task, due to the interference factors such as crypt structure, mucosal muscle layer texture, and mirror reflection, the model faces great challenges in boundary feature learning. From Table 1, it can be observed that the Baseline+RS method only brings a slight improvement in DSC (from 83.03 to 83.20), and even performance degradation in the case of 16x16 partition block, indicating that in complex scenarios, the hard split strategy interferes and damages the expression of image boundary features. In contrast, the fine-tuned SAM model with SRS mechanism achieves significant improvement in DSC (from 83.03 to 87.00), and the HD95 index also shows a significant decrease, indicating that it has better preservation and fitting ability for boundary structure.

[0061] Further experimental results on the PH² skin lesion dataset (see Table 2) show that even under the premise of already high segmentation performance, the SRS mechanism still shows potential for further optimization (from 87.00 to 87.15, and achieves the highest accuracy of 90.06). Guided by structural consistency, the model can continuously optimize the boundary expression of image features at fine-grained structures, thereby further improving segmentation accuracy and stability.

[0062] Step four: combine the error-aware mechanism to dynamically weight the error types in the image feature learning process.

[0063] Most traditional losses fail to distinguish the different importance of false positives (FP) and false negatives (FN) in clinical applications when punishing them, resulting in insufficient sensitivity to these two types of errors. In tumor detection, missing a malignant region (FN) may delay treatment and seriously endanger patients' lives, while false positives (FP) may lead to unnecessary follow-up examinations, but their consequences are relatively controllable. To address this issue, Tversky Loss was proposed, whose formula is as follows:

[0064]

[0065] Wherein, alpha and beta are the penalty coefficients of controlling FP and FN, respectively, for adjusting the relative importance of FP and FN. However, the hyperparameters in it depend on the experience judgment of clinicians, and cannot be dynamically adjusted in the actual training process, showing obvious limitations for different pathological characteristics and application scenarios. Therefore, designing an adaptive method that can dynamically change the penalty weight according to the actual situation in training is another key point to enhance the stability and sensitivity of model edge feature learning. For this purpose, a weight calculation strategy with error response dynamic adjustment capability is introduced. In terms of error control, instead of using fixed penalty coefficients, a responsive penalty factor function is constructed according to the false positive (FP) and false negative (FN) ratio generated by the model in the training process, so that the loss function can pay real-time attention to the change of the model error type, and adaptively adjust the sensitivity to different errors, especially in the scene where the FN risk is higher, give stronger response weight, prevent the SAM model after fine-tuning from deviating from the clinical core target in optimization.

[0066] In order to overcome the problems such as single response to error type and inability to dynamically adapt to clinical risk preference of traditional Tversky Loss in medical image segmentation, the present application proposes an adaptive adjustment strategy based on error perception, and the overall process is as shown in Figure 1 The strategy is integrated into the fine-tuned SAM segmentation model, which can perceive the dynamic distribution change of false positive (FP) and false negative (FN) in real time during image feature learning, and adjust the penalty weight of each type of error in the loss function accordingly, optimize the training direction and feature expression ability of the model. The specific implementation formula is as follows:

[0067]

[0068] Wherein And is the total number of pixels correctly predicted in the training process of the model, is the total number of pixels correctly predicted in the training process of the model, The design ensures that even if a certain type of error accounts for a low proportion, the model still retains basic structural attention to it, thereby avoiding the omission or imbalance of class information in the image feature training process. The specific implementation is as follows:

[0069]

[0070]

[0071] Wherein A and B are artificial parameters, is the coefficient of the first order term. This design makes And The loss function can be self-adjusted according to the proportion of the current training stage FP and FN, which enhances the adaptability of the loss function to different error types. In the actual model training process, only the values of A and B need to be set to automatically adjust the penalty weights of FP and FN classes, without increasing additional training parameters.

[0072] However, in the training stage where the image feature distribution changes dramatically or the error accumulation is nonlinearly enhanced, it is difficult to capture the complex dynamics by relying only on the first-order coefficient. Therefore, the present application further introduces a high-order adaptive adjustment mechanism, which is extended to a Taylor series form:

[0073]

[0074]

[0075] The coefficient of each high-order term According to decreasing design (where ) to avoid the influence of the over-amplified error proportion on the gradient, ensure the smoothness of the loss function surface, and prevent unstable fluctuations in the training process. In the experiment, the is set as an empirical parameter. Further, the above learning parameters can be changed according to the Taylor expansion:

[0076]

[0077]

[0078]

[0079]

[0080] From the above formula derivation, controlled by FN, and controlled by FP, forming a cross-dependent adjustment mechanism. When the FN proportion is high, it means that the model has a big problem in missing detection, at this time, adjusting alpha increases the weight of FP, so that the model suppresses the redundant positive class region in image feature extraction, and focuses more attention on identifying the ignored lesion structure. This adaptive mechanism is embedded in the SAM model loss function after fine-tuning, so that the model can adjust the strategy according to the error distribution in the image feature learning process. By dynamically controlling the weight distribution of different types of errors, the sensitivity of the model to the structure boundary and the adaptability to the clinical risk in the actual medical image segmentation task are improved, especially in the scene where the FN and FP clinical costs are asymmetric, which has more practical value.

[0081] Table 3 Comparison of different losses on BraTS2021,

[0082]

[0083] Table 4 Comparison of different losses on PDDCA.

[0084]

[0085] In Table 3 and Table 4, Table 3 is the comparison of different losses on BraTS2021, B is baseline, DC is dice, and ARS is adaptive region-specific loss. Table 4 is the comparison of different losses on PDDCA. The adaptive loss method is embedded into the fine-tuned SAM model, and is compared with the common loss on the MRI dataset and the CT dataset. The results show that the fine-tuned SAM model combined with the adaptive method has achieved good improvement in the comparison experiment of multiple segmentation objects. In the brain tumor segmentation scene, due to the significant gray overlap between tumor tissue and surrounding brain tissue, the traditional loss function lacks sufficient response mechanism when dealing with false negative and false positive errors, which easily leads the model to ignore the edge fuzzy area in the image feature learning process. The error perception mechanism introduced in the present application dynamically adjusts the penalty weight of different error types, guides the model to focus on learning the high uncertainty area of the image edge, so that the fine-tuned SAM can retain the original global perception ability while obtaining more detailed structural analysis ability, and the ITSRS obtains the optimal DSC value (91.16, 83.58, 76.93 respectively) in each type of segmentation task. The traditional segmentation model is easily disturbed by the high similar background and small organ target in the PDDCA scene, which leads to structure fracture or discontinuity of organ contour. The fine-tuned SAM model in the present application effectively enhances the attention of the model to the key boundary area by extracting multi-level image features and combining the structure consistency driven adaptive loss function. For example, in the parotid gland and vertebral body junction area, the adaptive loss adjusts the high-order error response to improve the punishment of the model to the false negative area, so that it tends to retain fine-grained boundary clues in the image feature learning process, thereby reducing the missed detection phenomenon, and the average DSC obtains the optimal value 74.33.

[0086] Step five: comparison of different segmentation methods for image feature learning effect.

[0087] In order to verify the improvement of the embedded ITSRS Loss method on the model performance in the multi-modal data segmentation scene, a systematic comparison experiment is carried out on the BraTS2021 brain tumor segmentation dataset. The ITSRS Loss is embedded in multiple mainstream segmentation frameworks (including ResUNet, TransBTS, MTSeg, BAT, nnUNet, TransUNet, Swin-UNetr, CoTr), and is compared with the baseline version of the original standard loss of these models.

[0088] Table 5 presents the generalization experiment of ITSRS under different models.

[0089]

[0090] Table 5 presents the comparative quantitative results, which show that ITSRS Loss is significantly better than other methods in mainstream evaluation indicators (for example, DSC from 78.4 to 89.8, and HD95 is significantly smaller than other baseline losses). Further analysis found that traditional CNN structures such as ResUNet and nnUNet have a certain depth in feature extraction, but there are deficiencies in cross-modal structure alignment and boundary continuity expression; Transformer-based models such as TransUNet and CoTr have global modeling capabilities, but due to the lack of local structure contrast reinforcement, they are prone to structure rupture in fuzzy areas. In contrast, the structure redundancy perception mechanism and error weight adaptive adjustment strategy in ITSRS Loss further promote the model to accurately align the anatomical boundaries and error types in image feature learning, and show superiority and promotion potential in complex medical image segmentation tasks.

[0091] The application also provides a medical image segmentation system based on error perception adaptive loss, comprising the following modules:

[0092] An image data module is used to acquire multi-modal medical images and perform preprocessing.

[0093] A feature extraction module is used to load pre-trained SAM network model weights and fine-tune, and extract features from the preprocessed multi-modal medical images.

[0094] A consistency enhancement module is used to introduce a structure consistency enhancement mechanism to optimize the extracted features, and finally obtain a segmentation image.

[0095] An error perception module is used to combine an error perception mechanism to dynamically weight the error types in the image feature learning process, and adjust the loss function weight according to the false positive and false negative distribution of the image features in the training process.

[0096] The above is a further detailed description of the application in combination with specific / preferred embodiments, and cannot be considered as limiting the specific implementation of the application to these descriptions. For ordinary skilled persons in the technical field to which the application belongs, without departing from the concept of the application, they can make several alternatives or modifications to the described embodiments, and these alternatives or modifications should be considered as belonging to the protection scope of the application.

[0097] The parts of the application not described in detail are known to those skilled in the art.

Claims

1. A medical image segmentation method based on error-aware adaptive loss, characterized in that, Includes the following steps: Step 1. Acquire multimodal medical images and perform preprocessing; Step 2. Load and fine-tune the weights of the pre-trained SAM network model, and extract features from the pre-processed multimodal medical images; the specific implementation of the SAM network model is as follows: Based on the SAM framework, a structure-guided fine-tuning mechanism is introduced to select and extract embedding features from the outputs of several key layers of the SAM encoder. Perform cross-layer fusion to build enhanced representation ,in Indicates the number of floors. , These are the height and width of the feature map, respectively. ; Constructing structurally enhanced feature representations through cross-layer fusion ; Step 3. Introduce a structural consistency enhancement mechanism to optimize the extracted features, and finally obtain the segmented image; The structural consistency enhancement mechanism is as follows: a redundant sensing unit is proposed to perform regional overlapping sensing division of the feature map. By adjusting the spatial coverage scale and the advancing granularity, the segmentation network retains cross-regional structural information during local learning. The specific implementation process of step 3 is as follows: SRS Loss is proposed to enhance the representation Multiple sensing units are deployed along the boundary, and a balance is achieved between global information and local features through the local overlap of sensing units, thus maintaining the structural integrity of the image. The formula for calculating SRS Loss is: ； in This represents the m-th sensing unit. This represents the total number of sensing units present around the target area. For the first The predicted value of each pixel. For the first The label corresponding to each pixel To prevent coefficients from being zero.

2. The medical image segmentation method based on error-aware adaptive loss according to claim 1, characterized in that, The method further includes step 4, which dynamically weights the error types in the image feature learning process using an error-aware mechanism. The specific implementation process is as follows: A weight calculation strategy with dynamic error response capability is introduced. In terms of error control, a responsive penalty factor function is constructed based on the ratio of false positives (FP) to false negatives (FN) generated by the model during training. This allows the loss function to monitor changes in the model's error type in real time and adaptively adjust its sensitivity to different errors.

3. The medical image segmentation method based on error-aware adaptive loss according to claim 2, characterized in that, The specific penalty factor function for the response is as follows: An error-aware adaptive adjustment strategy is proposed, which senses the dynamic distribution changes of false positives (FP) and false negatives (FN) in real time during image feature learning and automatically corrects the error based on the proportion of FP and FN in the total number of errors. The parameters are accumulated in order, and the penalty weights for various errors in the loss function are adjusted accordingly to optimize the training direction and feature representation ability of the model.

4. The medical image segmentation method based on error-aware adaptive loss according to claim 3, characterized in that, It also includes dynamically weighting the error types in the image feature learning process by combining an error perception mechanism, and adjusting the weights of the loss function based on the distribution of false positives and false negatives of image features during the training process.

5. A medical image segmentation system based on error-aware adaptive loss, used to implement the medical image segmentation method according to any one of claims 1 to 4, characterized in that, Includes the following modules: The image data module is used to acquire multimodal medical images and perform preprocessing. The feature extraction module is used to load and fine-tune the weights of the pre-trained SAM network model to extract features from the pre-processed multimodal medical images. The consistency enhancement module is used to introduce a structural consistency enhancement mechanism to optimize the extracted features and finally obtain the segmented image; The error awareness module is used to dynamically weight the error types in the image feature learning process by combining the error awareness mechanism, and adjust the weight of the loss function according to the distribution of false positives and false negatives of image features in the training process.

Images