An ultrasound sequence image segmentation model training method

Abstract

The application provides an ultrasonic sequence image segmentation model training method, which comprises the following steps: constructing an artificial labeling data set and an unlabeled data set based on a clinically collected original ultrasonic sequence; preheating training a teacher network and a double-branch student network by using the artificial labeling data set, wherein the double-branch student network comprises a model A and a model B; then performing multi-round iterative training, in each round, predicting an unlabeled image by using the model A and the model B obtained in the last round of training, combining a reference segmentation result of the unlabeled image by the teacher network, generating a pseudo-label data set according to the consistency of the prediction of the model A and the model B and the effectiveness of the reference segmentation result, jointly training the model A and the model B by using the artificial labeling data set and the pseudo-label data set, and taking the model A and the model B as an ultrasonic sequence image segmentation model after the training is completed. The application can reduce the dependence of the ultrasonic image segmentation process on large-scale artificial labeling, and improve the stability and boundary recognition ability of the model in a speckle noise background.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a training method for an ultrasound sequence image segmentation model based on boundary awareness and its adaptive evolution. Background Technology

[0002] Ultrasound-guided vascular puncture is widely used in various clinical scenarios such as intravenous infusion and arterial blood sampling. Compared with traditional blind puncture, this technology can display the depth of blood vessels and surrounding anatomical tissues in real time, greatly improving the success rate of clinical puncture. However, it requires a high level of experience and hand-eye coordination from the operator, which has spurred extensive research into ultrasound-guided autonomous puncture robotic systems. High-precision anatomical segmentation of tiny blood vessels and target areas from complex ultrasound images is the core foundation for the positioning accuracy of this type of robotic system.

[0003] At present, research on blood vessel segmentation based on ultrasound image features mainly includes traditional methods and deep network models. Due to the large amount of high-frequency speckle noise inherent in the physical imaging of ultrasound systems, the sound shadow and tissue boundary are incomplete and blurred. Early traditional image processing methods that relied on specific manually constructed features (such as watershed, threshold and edge gradient) often had poor robustness. Later, deep learning based on convolutional networks (such as the U-Net series architecture [1]) gradually dominated various medical vision tasks and showed great potential in ultrasound pathological image feature learning. For example, the Attention U-Net [2] proposed by Oktay O et al. in 2018 enhanced the focus on the target region by introducing an attention gating mechanism. However, traditional supervised deep networks generally require a massive amount of pixel-level and extremely accurate clinical expert gold standard annotation atlases. Considering the extremely high time cost of ultrasound image expert annotation, industry research has gradually turned to introducing weakly supervised learning, self-supervised or semi-supervised learning paradigms. For example, Jin FQ et al. based on threshold weak labeling [3], and many recent semi-supervised segmentation schemes that combine the zero-sample capability of large models [4]. Unfortunately, when dealing with real ultrasound images with low contrast and complex artifact interference, existing low-annotation-dependent methods still face many core performance and generalization bottlenecks.

[0004] Based on clinical trials and recent literature research, the current technical solutions have the following significant drawbacks:

[0005] First, the high temporal redundancy of ultrasound video sequences is not adequately addressed. Clinical DICOM ultrasound sequences contain various similar redundant frames derived from probe dwell and motion-blurred frames caused by violent probe movement. Feeding the entire unpurified sequence into the training loop leads to extremely uneven distribution, causing the network to fit ineffective stagnant information or be interfered with by blurred features. Most common single-frame spatial domain evaluation methods (such as the Laplacian variance check mentioned in reference [5]) are limited to static sharpness and fail to combine the real optical flow motion of temporal slices for double screening.

[0006] Secondly, severe confirmation bias leads to catastrophic degradation of iterative networks. Current weakly supervised or semi-supervised learning schemes, when generating a large proportion of automatically supplemented pseudo-labels for complex speckle target areas, lack strong external high-order visual logic supervision with high stability. The student model can easily identify severely distorted and distorted mislabels as truth through internal reasoning loops (such as the semi-supervised self-training risk discussed in reference [6]), and finally fit to a completely off-track direction. Although existing large models (such as MedSAM [4]) have zero-sample segmentation capabilities, they are extremely sensitive to speckle ultrasound artifacts and lack multi-level cross-arbitration protection specifically designed to resist medical speckle.

[0007] Third, current common data augmentation methods are prone to causing the disintegration of micro-target structures. Common enhancement techniques based on block permutation shearing, polarization color, and random truncation can easily tear apart the extremely small and fragile pathological lumen structures and their spatial connectivity topology in ultrasound. Abandoning macroscopic structural constraints in diversified amplification is highly likely to destroy the understanding of connectivity rules. Although traditional superpixel technology (such as SLIC[7]) can aggregate similar pixels, it has not been fully developed for background adversarial permutation under the protection of target connectivity topology.

[0008] Fourth, the tightly coupled architecture of two-dimensional tasks is insufficient for handling extremely blurred edges. Most conventional mainstream methods (such as traditional U-Net and its variants) adopt a single-stream mechanism that not only has to undertake the huge basic long-range semantic qualitative and macroscopic discrimination, but also has to implement fine-grained capture and localization for extremely discontinuous and low-difference boundaries. The mixed response of a single module and ordinary loss such as Dice[8] are extremely prone to causing phenomena such as blurred defocused boundaries, jagged artifacts, and even serious missing bullseyes at the incomplete parts. Although the specialized edge networks in recent years (such as PIDNet[9] for urban scene segmentation) have strong high-frequency boundary capture capabilities, they have not yet been successfully decoupled and introduced into the ultrasound medical multi-stream heterogeneous system with extreme adversarial interference.

[0009] Based on the above analysis, there is an urgent need for a robust solution that is noise-resistant, accurate, and relies minimally on the cost of manual annotation.

[0010] References:

[0011] [1] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net:Convolutional networks for biomedical image segmentation. In MICCAI . Springer.

[0012] [2] Oktay, O., Schlemper, J., Folgoc, L. L., et al. (2018). Attentionu-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.

[0013] [3] Jin, F. Q., Knight, A. E., Cardones, A. R., et al. (2021). Semi-automated weak annotation for deep neural network skin thickness measurement.Ultrasonic Imaging, 43(4), 167-174.

[0014] [4] Ma, J., He, Y., Li, F., et al. (2024). Segment anything inmedical images. Nature Communications, 15(1), 654.

[0015] [5] Pech-Pacheco, J. L., Cristóbal, G., Chamorro-Martinez, J., & Fernández-Valdivia, J. (2000). Diatom autofocusing in brightfield microscopy: acomparative study. In ICPR . IEEE.

[0016] [6] Arazo, E., Ortego, D., Albert, P., et al. (2020). Pseudo-labelingand confirmation bias in deep semi-supervised learning. In IJCNN IEEE.

[0017] [7] Achanta, R., Shaji, A., Smith, K., et al. (2012). SLICsuperpixels compared to state-of-the-art superpixel methods. IEEETransactions on Pattern Analysis and Machine Intelligence, 34(11), 2274-2282.

[0018] [8] Milletari, F., Navab, N., & Ahmadi, SA (2016). V-net: Fullyconvolutional neural networks for volumetric medical image segmentation. In3DV IEEE.

[0019] [9] Xu, J., Xiong, Z., Bhattacharyya, SP (2023). PIDNet: A real-time semantic segmentation network inspired by PID controllers. In CVPR . Summary of the Invention

[0020] This invention at least partially solves one of the technical problems existing in the prior art.

[0021] Therefore, the purpose of this invention is to provide a training method for an ultrasound sequence image segmentation model, so as to reduce the dependence of the ultrasound image segmentation process on large-scale manual annotation and improve the training stability and boundary recognition ability of the model in the context of speckle noise.

[0022] To achieve the above objectives, the present invention adopts the following technical solution:

[0023] The first aspect of this invention provides a method for training an ultrasound sequence image segmentation model, comprising:

[0024] A keyframe dataset is constructed based on the original ultrasound sequences acquired clinically. An artificially labeled dataset is constructed based on a preset proportion of keyframe images in the keyframe dataset. An unlabeled dataset is constructed using the remaining data in the keyframe dataset.

[0025] The teacher network and the two-branch student network are pre-trained using the manually labeled dataset. The teacher network is used to generate reference segmentation results for unlabeled images and to verify, correct, and filter pseudo-labels for the predictions of the two-branch student network. The two-branch student network includes model A for extracting global semantic information of the image and model B for extracting local boundary detail information of the image.

[0026] Multiple rounds of iterative training are conducted. In each round, Model A and Model B obtained from the previous round of training are used to predict unlabeled images. The reference segmentation results of the unlabeled images are combined with the teacher model. A pseudo-labeled dataset is generated based on the prediction consistency of Model A and Model B and the validity of the reference segmentation results. Model A and Model B are jointly trained using the manually labeled dataset and the pseudo-labeled dataset. The trained Model A and Model B are used as ultrasound sequence image models.

[0027] In some embodiments, constructing the keyframe dataset based on the raw ultrasound sequences acquired clinically includes:

[0028] Using optical flow and the Laplacian operator, a two-layer quality screening was performed on the raw DICOM ultrasound sequences acquired clinically to obtain the keyframe dataset that simultaneously meets the requirements of inter-frame information diversity and spatial clarity.

[0029] In some embodiments, the optical flow method uses the Farneback dense optical flow algorithm. If the average amplitude of the optical flow field of the current frame is greater than a preset motion intensity threshold and the Laplacian variance of the current frame is greater than a preset sharpness threshold, then the frame is included in the keyframe dataset.

[0030] In some embodiments, the teacher network uses a medical vision-based model;

[0031] Model A uses an improved EfficientNet-UNet structure, including an encoder and a decoder. The encoder uses the EfficientNet-B0 backbone network, where the first convolutional layer of EfficientNet-B0 is configured as a single-channel input. The decoder uses a UNet-style progressive upsampling decoder, which has multiple upsampling and convolutional fusion modules. Each convolutional fusion module uses a combination of convolution, batch normalization, ReLU activation, and Dropout regularization. By extracting multi-scale features from different stages of EfficientNet-B0 and concatenating and fusing them with upsampled features in the decoding stage, deep semantic information and shallow spatial details can be utilized simultaneously.

[0032] Model B uses an improved PIDNet-S architecture, including an input layer, a three-branch structure, a fusion module, and dual output heads. The three-branch structure includes a context branch, a high-resolution branch, and a boundary branch, with the boundary branch constrained by an independent boundary supervision loss during training. The first convolutional layer of the input layer is configured as a single-channel input. The fusion module is used to fuse the context information, high-resolution detail information, and boundary information extracted by the three-branch structure. The dual output heads include a segmentation output head and a boundary output head, with the boundary output head outputting a boundary prediction map during training.

[0033] In some embodiments, pseudo-labels for unlabeled images are generated according to the following formula:

[0034]

[0035]

[0036]

[0037]

[0038] in:

[0039] For any unlabeled image in the unlabeled dataset during the t-th training epoch. Generated pseudo-tags; , , These represent the consensus region, divergence region, and prompt box for the t-th training cycle, respectively. For the t-th training cycle, the teacher network is based on the prompt box. The output reference segmentation result, This is a symmetric difference operator; and The first The prediction results output by Model A and Model B after one training cycle;

[0040] Let be the validity index of the reference segmentation result in the t-th training period, and the value is determined according to the following rules: when the reference segmentation result area Furthermore, the teacher network's response to the reference segmentation results Predicted overlap quality index At that time, I recorded Otherwise remember , These are the lower and upper threshold values ​​for the area ratio of the target region, respectively. The first threshold;

[0041] The intersection-union ratio (IUU) is a consistency index between the prediction results of model A and model B, and its value is determined by the following rules: When the IUU of the prediction results of model A and model B is equal... At that time, I recorded Otherwise remember , This is the second threshold.

[0042] In some embodiments, the process of constructing the training data further includes enhancing the input image, which is divided into strong enhancement processing and weak enhancement processing.

[0043] The enhanced processing involves performing structure-preserving superpixel blending enhancement on the input image to obtain a superpixel image. :

[0044]

[0045] in, For the input image, For the input image Background image in the same domain, For superpixel masks, This is an element-wise multiplication operation;

[0046] The weak enhancement process includes: performing a geometric transformation on the superpixel image and then inputting it into model A; performing a geometric transformation on the superpixel image, applying noise and blurring, and then inputting it into model B.

[0047] In some embodiments, the total loss used when training models A and B is... for:

[0048]

[0049]

[0050]

[0051] in,

[0052] and The supervision losses for models A and B are respectively. and The cross-entropy loss for model A and model B are respectively. and These are the Dice losses for model A and model B, respectively.

[0053] Boundary augmentation loss for model A:

[0054]

[0055] In the formula, This represents the hyperparameter polarization multiplication parameter, used to amplify boundary pixel loss; This represents the set of image pixels in a single frame. For model A at pixel position The prediction results are given here; It is a pixel-level boundary prior weight map extracted based on a truth mask; Pixel-level segmentation of truth labels;

[0056] This is the boundary branch loss for model B;

[0057]

[0058] In the formula, This represents the boundary header output of model B;

[0059] For cross-supervision loss, a one-way constraint method is adopted, that is, only model A provides supervision signals to model B, and model B does not provide reverse constraints to model A. The expression is:

[0060]

[0061] In the formula, This indicates an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. The confidence threshold; and These are the prediction results for model A and model B, respectively. This represents the structural validity determination function, which considers both area and shape constraints: if the prediction result of model A... The proportion of the target region area to the total area of ​​the input image is located at Within the range, and the prediction results of model A After morphological smoothing, the smoothed intersection-union ratio is greater than or equal to the third threshold. At that time, it was considered Otherwise, it is considered , These are the lower and upper threshold values ​​for the area ratio of the target region, respectively.

[0062] , and They are respectively , and Weights, weights The training period t and training round e are set in segments.

[0063] In some embodiments, the weight The expression is:

[0064]

[0065] In the formula, This indicates the preheating stage. , and These are the first, second, and third training rounds, respectively. , and These are the first setting value, the second setting value, and the third setting value.

[0066] A second aspect of the present invention provides an ultrasound sequence image segmentation model training device, comprising:

[0067] The dataset construction module is configured to construct a keyframe dataset based on the original ultrasound sequences acquired clinically, construct an artificially labeled dataset based on a preset proportion of keyframe images in the keyframe dataset, and construct an unlabeled dataset using the remaining data in the keyframe dataset.

[0068] The preheating training module is configured to preheat the teacher network and the two-branch student network using the manually labeled dataset. The teacher network is used to generate reference segmentation results for unlabeled images and to verify, correct, and filter pseudo-labels for the predictions of the two-branch student network. The two-branch student network includes a model A for extracting global semantic information of the image and a model B for extracting local boundary detail information of the image.

[0069] The iterative joint training module is configured to perform multiple rounds of iterative training. In each round, the model A and model B obtained from the previous round of training are used to predict the unlabeled image, and the reference segmentation result of the teacher model is combined with the prediction consistency of model A and model B and the validity of the reference segmentation result to generate a pseudo-label dataset. The artificially labeled dataset and the pseudo-label dataset are used to jointly train model A and model B, and the trained model A and model B are used as ultrasound sequence image models.

[0070] A third aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the ultrasound sequence image segmentation model training method according to any embodiment of the first aspect of the present invention.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] 1. This invention combines optical flow motion intensity detection with image sharpness detection to screen key frames from DICOM ultrasound sequences, thereby reducing the impact of temporally redundant and blurred frames on subsequent training.

[0073] 2. This invention introduces a teacher model and a two-branch student network to jointly generate pseudo-labels, which can filter out low-quality pseudo-labels and reduce error accumulation during iterative training.

[0074] 3. By employing a dual-branch structure of Model A and Model B, structure-preserving superpixel blending enhancement, structure validity determination, and a confidence-guided cross-pseudo-supervision mechanism, this invention can improve the ability of the ultrasound image segmentation model to identify boundary regions and its training stability. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the overall process of a training method for an ultrasound sequence image segmentation model provided in the first aspect embodiment of the present invention;

[0076] Figure 2 yes Figure 1 The flowchart shown illustrates the process of extracting keyframe datasets using an optical flow filtering module in the method described.

[0077] Figure 3 These are schematic diagrams of the structures of Model A and Model B in this embodiment of the invention;

[0078] Figure 4 This is a schematic diagram of the HEAL-Net training framework used in this embodiment of the invention;

[0079] Figure 5 This is a schematic diagram of the structure of an electronic device provided in a third aspect embodiment of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application.

[0081] Conversely, this application covers any alternatives, modifications, equivalent methods, and schemes made within the spirit and scope of this application as defined by the claims. Furthermore, to provide the public with a better understanding of this application, certain specific details are described in detail below. However, this application can be fully understood by those skilled in the art even without these detailed descriptions.

[0082] See Figure 1 The first aspect of the present invention provides a method for training an ultrasound sequence image segmentation model, comprising the following steps:

[0083] Step S100: Perform a two-layer quality screening on the raw DICOM ultrasound sequences acquired clinically using optical flow and the Laplacian operator to obtain a keyframe dataset that simultaneously meets the requirements of inter-frame information diversity and spatial clarity. Manually annotate keyframe images in the keyframe dataset at a predetermined ratio and construct a manually annotated dataset. An unlabeled dataset is constructed using the remaining data in the keyframe dataset. ;

[0084] Step S200: Use manually labeled datasets. The teacher network and the two-branch student network were pre-trained. The teacher network was used to generate reference segmentation results for unlabeled images, and the predictions of the two-branch student network were verified, corrected and pseudo-labels were filtered. The two-branch student network includes model A for extracting global semantic information of the image and model B for extracting image boundary detail information.

[0085] Step S300: Perform iterative joint training on the pre-trained two-branch student network for T training cycles, with each training cycle containing E training epochs, to obtain the trained two-branch student network. Use this network as an ultrasound sequence image model. The dataset was manually labeled during each training cycle. and pseudo-label dataset The pseudo-label dataset serves as training data. The construction process includes:

[0086] For unlabeled datasets Any unlabeled image in Based on the first The prediction results output by Model A and Model B after one training cycle Compared with the prediction results The common and different parts are used to construct the consensus region (i.e. the part where the prediction results of the two intersect) for the t-th training cycle. and areas of divergence Based on the prediction results and prediction results The pop-up message generated by the merged result The results obtained after inputting the teacher network are used as the first Reference segmentation results for each training cycle Regarding the reference segmentation result Validity and prediction results Compared with the prediction results The consistency is determined based on the determination result and the consensus area. Disagreement Areas and reference segmentation results Generate the first Unlabeled images corresponding to each training epoch pseudo-tags ;

[0087] Using the first The pseudo-labels for all unlabeled images corresponding to the training epoch are used to construct the first training epoch. Pseudo-label dataset for training epochs .

[0088] In some embodiments, step S100 proposes an adaptive optical flow temporal and spatial layer joint filtering mechanism to perform a two-layer quality screening of the original clinical temporal acquisition DICOM ultrasound sequences: the first layer calculates the motion intensity of each ultrasound image frame based on the inter-frame temporal difference characteristics using optical flow to filter out redundant frames with high-frequency stagnation; the second layer further calculates the spatial domain sharpness of the ultrasound image based on Laplacian variance to block and filter blurred frames, thereby obtaining a high-quality keyframe dataset. A small portion of the keyframe dataset (e.g., 10% of the data) is manually annotated to obtain a manually annotated dataset. , for The first in Frames of manually annotated images, For Corresponding manual annotation information for The number of images contained in the dataset determines the number of unlabeled images to be constructed from the remaining data (e.g., 90% of the data) in the keyframe dataset. , for The first in Unlabeled image frame for The number of images contained in it.

[0089] like Figure 2 As shown, in a specific embodiment of this application, step S100 performs keyframe extraction and PNG normalization output on the raw DICOM ultrasound sequence acquired clinically. Specifically, it includes:

[0090] Use functions in Python programs Read the DICOM file, where: The path parameter for the DICOM file to be read specifies the storage location of the target file; To force parameter reading, this method skips standard DICOM header verification and forces the parsing of non-standard DICOM data, data without a DICOM identifier, or data with a corrupted header. When the DICOM file is a single-frame image, the frame is directly extracted and output; when the DICOM file is a multi-frame sequence, each frame undergoes preprocessing, sharpness assessment, motion detection, and keyframe filtering sequentially.

[0091] For each keyframe image, a preprocessing module is first used to perform grayscale conversion and bit depth normalization. If the input frame is not an 8-bit image, when the maximum pixel value is greater than 255, it is converted into an 8-bit grayscale image using a minimum-maximum linear normalization method; if the input frame is a three-channel image, grayscale transformation is used to obtain a single-channel grayscale image. Then, a post-processing block is used to perform region cropping and size normalization on the grayscale image: the coordinates of the top-left corner of the cropped region are set to... The coordinates of the lower right corner are set to The cropped image is scaled using nearest neighbor interpolation. Then center it and fill it into a black canvas with a side length of 256 to obtain a standardized result. PNG image.

[0092] Subsequently, the sharpness of each preprocessed frame was evaluated, using Laplacian variance as the evaluation metric.

[0093]

[0094] In the formula, Represents the Laplace operator. This represents variance calculation. Indicates the first Frame Image The sharpness index. In this embodiment, the sharpness threshold is set to... .when When, it indicates that clarity filtering is not enabled; when At that time, the sharpness condition needs to be met (i.e.) Only frames that meet certain criteria are allowed to be retained.

[0095] For multi-frame DICOM sequences, first determine the starting reference frame: when When, directly retain the first frame as the starting reference frame; when At that time, starting from frame 1, traverse sequentially and select the first frame that satisfies the condition. The first frame is used as the starting reference frame and saved as the first frame in the output sequence. If no frame that meets the resolution condition is found after the traversal, it is determined that no valid keyframe has been extracted from the DICOM sequence.

[0096] For subsequent candidate frames, the Farneback dense optical flow algorithm is used to calculate the optical flow field between the current reference frame and the candidate frame. Its parameters are set as follows: pyramid scale. Number of pyramid levels Window size Number of iterations Polynomial neighborhood size Gaussian smoothing coefficient Flag position Let the reference frame be... Candidate frames are The corresponding optical flow vector field is The optical flow amplitude is then converted to polar coordinates using a function in the OpenCV library. Calculate and take the average amplitude as the exercise intensity index:

[0097]

[0098] In the formula, It represents the set of image pixels of a single frame, that is, the entire pixel range of the input single image; Indicates reference frame With candidate frames The average optical flow amplitude between [a certain value]. In this embodiment, the motion intensity threshold is set to [a certain value]. .

[0099] Candidate frames are filtered according to the following steps: if and only if candidate frames Simultaneously satisfy and If a candidate frame is not satisfied, it is retained and saved as the next numbered PNG image in the output sequence. Simultaneously, the candidate frame is updated as a new reference frame for optical flow calculation in subsequent frames. If a candidate frame does not meet any of the above conditions, it is discarded, and the reference frame is not updated. Thus, the output images are renumbered in the retention order as 1.png, 2.png, 3.png, up to n.png, and saved in the images subfolder under the corresponding DICOM name directory, which is the final keyframe dataset obtained in this embodiment.

[0100] It is understandable that, through the above processing, this embodiment achieves the transformation from the original DICOM ultrasound sequence to a standardized one. Automatic conversion of PNG keyframe sequences can filter out stagnant and blurry frames while preserving significant motion information, thereby reducing redundancy in subsequent training data and improving the quality of input images.

[0101] In some embodiments, the medical vision foundation model MedSAM (Segment Anything in Medical Images) is selected as the teacher network. A two-branch student network is used as the ultrasound sequence image segmentation model to be trained. This two-branch student network includes model A and model B, as follows: Figure 3 As shown, Model A uses an improved EfficientNet-UNet structure, and Model B uses an improved PIDNet-S structure. Both models use single-channel ultrasound images as input, and the input size is uniform. The number of categories is set to 2.

[0102] The structural improvements of EfficientNet-UNet in this embodiment are reflected in the following aspects:

[0103] Single-channel input adaptation: The first layer convolution of the EfficientNet-B0 encoder was changed from three-channel input to single-channel input to adapt it to grayscale ultrasound images;

[0104] Improved pre-training weight transfer method: When using pre-trained EfficientNet-B0 parameters, the original three-channel convolutional kernels are fused along the channel dimension and then assigned to the new single-channel convolutional kernels to avoid completely random initialization of the first layer parameters;

[0105] Encoder-decoder combination improvement: The backbone network of EfficientNet-B0 is used as the encoder and combined with the UNet-style stepwise upsampling decoder to form the EfficientNet-UNet structure, instead of directly using the EfficientNet classification network;

[0106] Multi-scale skip connection improvement: Multi-scale features are extracted from different stages of EfficientNet-B0 and concatenated and fused with upsampled features in the decoding stage to simultaneously utilize deep semantic information and shallow spatial details;

[0107] Five-level decoding structure improvement: The decoder is equipped with multi-level upsampling and convolutional fusion modules, which gradually restore deep low-resolution features to the original image resolution. Each level of convolutional fusion module adopts a combination of convolution, batch normalization, ReLU activation and Dropout regularization to improve feature fusion capability and reduce the risk of overfitting.

[0108] The improvements to the PIDNet-S structure in this embodiment are reflected in the following aspects:

[0109] Single-channel input adaptation: The first convolutional layer of the PIDNet-S input layer is changed from three-channel input to single-channel input to adapt to ultrasound grayscale images;

[0110] Lightweight parameter configuration: The model adopts the miniaturized configuration of PIDNet-S, such as a smaller number of channels and module repetitions, making it more suitable for lightweight applications in medical image segmentation;

[0111] The three-branch structure is preserved and used for medical segmentation: global semantics, local details, and edge information are extracted using contextual branches, high-resolution branches, and boundary branches, respectively. Among them, the boundary branch is preserved and strengthened, enabling the network to pay more attention to the blurred or discontinuous boundaries of organs or lesions in ultrasound images. The strengthening of the boundary branch is reflected in both structural strengthening and supervision strengthening. Specifically, the boundary branch in PIDNet is preserved, and a differential enhancement convolution module and an independent boundary output head are set after the branch to output a single-channel boundary prediction map. During training, the boundary ground truth map is generated based on the manually labeled mask through morphological dilation and erosion operations, and the boundary prediction map is supervised separately using binary cross-entropy loss, thereby strengthening the boundary branch's ability to perceive the blurred or discontinuous boundaries of organs or lesions in ultrasound images.

[0112] Dual-output structure: A separate boundary output head is set up in addition to the existing segmentation output head. This boundary output head is used to output the boundary prediction map during the training phase. During the training phase, PIDNet-S outputs both the segmentation result and the boundary result, so that it can learn not only the region classification, but also the target contour.

[0113] Feature fusion structure improvement: The segmentation result is output after fusing contextual information, high-resolution detail information and boundary information, which improves the recognition ability of boundary regions and small structures.

[0114] In one specific embodiment of this application:

[0115] Model A is used to extract global semantic information from ultrasound images to be segmented. Its encoder uses the EfficientNet-B0 backbone network, and the first convolutional layer (a two-dimensional convolutional layer Conv2d) is changed from a three-channel input to a single-channel input, that is, the parameters of the first convolutional layer are changed from... Adjusted to , The parameters are, in order: number of input channels (representing the number of channels in the input image), number of output channels (representing the number of convolutional kernels), kernel size, stride (representing the stride by which the convolutional kernel slides across the input feature map), and padding (representing the number of zero-value pixels padded around the input feature map). When using pre-trained parameters, the original three-channel convolutional kernels are summed along the channel dimension and then transferred to a single-channel convolutional layer. The multi-scale features output by the encoder of Model A are as follows: The resolution is the same as the original image (i.e., the image of input model A). The number of channels is 16; The resolution is the same as the original image. The number of channels is 24; The resolution is the same as the original image. The number of channels is 40; The resolution is the same as the original image. The number of channels is 112; The resolution is the same as the original image. The number of channels is 320; the central feature layer has 1280 channels. This central feature layer is the global high-level semantic feature output of the last convolutional layer at the end of the encoder of Model A, corresponding to EfficientNet-B0 in... The feature map is then upsampled using a 1×1 convolution. Model A's decoder employs a five-level upsampling structure, with parameters for each level as follows: The first level maps the central features from 1280 channels to 320 channels and upsamples them to the original image. Resolution; the second stage maps the 432-channel features (i.e., concatenation) obtained by concatenating the current-level features of the decoder with the corresponding-resolution features of the encoder to 112 channels; the third stage maps the 152-channel features (since the input of the third stage is the 112-channel output of the second-level decoder, concatenated with the 40-channel features of the encoder at the corresponding scale, the input of the third stage is 152-channel features) to 40 channels; the fourth stage maps the 64-channel features to 24 channels; the fifth stage maps the 40-channel features to 16 channels; finally, through a... The convolutional layer outputs a 2-channel segmentation result. Each convolutional block in Model A's decoder includes a... The system consists of a convolutional layer, a batch normalization layer, a ReLU activation layer, and a two-dimensional random deactivation layer (Dropout2d) with a deactivation rate of 0.1. The upsampling method is bilinear interpolation.

[0116] Model B is used to extract boundary detail information from the ultrasound image to be segmented. Its backbone is an improved PIDNet-S structure, and the network parameters are set as follows: the number of repetitions of the residual block in the medium-resolution branch. Number of repetitions of residual blocks in low-resolution branches Basic channel count Number of channels in the pyramid pooling module The number of intermediate channels in the prediction head Whether to enable auxiliary enhancement branches ( This indicates that the auxiliary enhancement branch is not enabled. (This indicates that the auxiliary enhancement branch is enabled). The input layer of Model B consists of two consecutive... Convolutional layer, the parameters of the first convolutional layer are The parameters of the second convolutional layer are Model B also includes a high-resolution branch P-Branch, a context branch I-Branch, and a boundary branch D-Branch. The context branch I-Branch outputs 32, 64, 128, and 256 channel features sequentially, and compresses and fuses the 256-channel features into 128-channel features through the PAPPM (Parallel Aggregation Pyramid Pooling Module) module. The convolutional transformation is converted to 64 channels; the high-resolution branch P-Branch outputs 32, 64, and 128 channel features sequentially, and then... Compression to 64 channels; the boundary branch D-Branch outputs 32, 32, and 32-channel features sequentially, then passes through two... The convolutional layer transforms the boundary branch features into 64 channels. After the three branches are fused, Model B sets up two output heads: the segmentation head uses... and Output 2-channel segmentation results; boundary header uses and Output single-channel boundary results; finally, upsample both the segmentation results and the single-channel boundary results to the original input resolution.

[0117] In some embodiments, such as Figure 4 As shown, the training framework in this embodiment is HEAL-Ne (Heterogeneous Evolutionary Arbitration Learning Network). This training framework includes a warm-up training phase (…). ) and iterative training phase ( The following sections will describe the process in detail, in stages.

[0118] In some embodiments, during the warm-up training phase, in step S200, manually labeled datasets are used. The constructed teacher network and two-branch student network are analyzed. Specifically, a manually labeled dataset is used. Fine-tuning of the decoder part of MedSAM was performed using a manually labeled dataset. This enables model A and model B to have initial segmentation capabilities.

[0119] In some embodiments, in step S300, during the iterative training phase... Before the start of the training cycle, the following reasoning steps are first used to construct the first training cycle. The pseudo-label dataset required for each training cycle :

[0120] For any unlabeled image Model A and Model B respectively output prediction results and ,when hour, and Using pre-trained models A and B on unlabeled images, respectively. The output predicted labels are used to construct a consensus region based on the two prediction results. Disagreement Areas and prompt boxes :

[0121]

[0122]

[0123]

[0124] in, This is a symmetric difference operator;

[0125] Unlabeled images The corresponding prompt box Input the teacher network MedSAM to obtain the reference segmentation results. .

[0126] Subsequently, the reference segmentation results were analyzed. Perform validity determination and... and Perform a consistency check, specifically: when and At that time, record the validity index of the reference segmentation results. Otherwise, remember .when Record the consistency index of the prediction results of model A and model B. Otherwise, remember .in, Indicates the reference segmentation result area, and These represent the lower and upper threshold values ​​for the area ratio of the target region, respectively, and are set based on the target size range in the actual annotation. This indicates the teacher network's response to the reference segmentation results. Predictive overlap quality index, This represents the crossover ratio (CROR) of the prediction results from model A and model B. and These represent the first threshold and the second threshold, respectively. In this embodiment, we take... , .when and When the current data is considered valid, it is considered valid. and When, the current data is considered salvageable data; when and At that time, the current data is considered invalid.

[0127] Generate pseudo-tags according to the following formula. :

[0128]

[0129] Among them, the empty set This indicates that the generated pseudo-labels are invalid and will not be used as samples for iterative training.

[0130] Form the first set of pseudo-labels of the non-empty set. Pseudo-label dataset for training epochs and compared with manually labeled datasets Used together for the first Training samples for each training cycle.

[0131] Furthermore, in step S300, the training data construction process also includes enhancing the input image, which is divided into strong enhancement processing and weak enhancement processing.

[0132] The enhanced processing involves performing structure-preserving superpixel blending enhancement on the input image, specifically:

[0133] Given an input image (see...) Figure 3 Image A in the image is The input image The background image in the same domain is ( To match the original image Ultrasound images from the same task and the same ultrasound data distribution are used to provide the target structure to be segmented, such as vascular structures, and the corresponding mask; superpixel masks (see [link to documentation]). Figure 3 The mask of image B is In superpixel mask Use the same background image within the selected area. Content replacement in superpixel mask Keep the input image in the unselected areas. The content is then stitched together to form a new superpixel image. That is, the superpixel image Represented as:

[0134]

[0135] in, This indicates element-wise multiplication.

[0136] In this embodiment, the parameters set during superpixel enhancement include: the number of superpixel segments. Tightness Smoothing parameters Background area exchange ratio And the number of pixels extending outward from the target mask .

[0137] Weak enhancement method 1 and weak enhancement method 2 were applied to the input images of model A and model B, respectively. The weak enhancement parameters used in weak enhancement method 1 included: enabling horizontal and vertical flipping, and a rotation angle range of [value missing]. The scaling range is [0.9, 1.1], and the brightness perturbation range is... The contrast perturbation range is The overall enhancement probability is 0.5. The weak enhancement parameters used in weak enhancement method 2 include: enabling horizontal and vertical flip, and a rotation angle range of [missing information]. The scaling range is [0.8, 1.2], and the brightness perturbation range is... The contrast perturbation range is The overall enhancement probability is 0.8, the Gaussian noise application probability is 0.5, and the blur application probability is 0.3.

[0138] In some embodiments, after obtaining the enhanced image through the above enhancement operations, joint training of model A and model B can be performed. A joint loss function is used during training, and the training losses for model A and model B include supervision loss, boundary loss, and confidence-guided cross-pseudo-supervision loss, letting... This represents the current training epoch, and the total loss is... Represented as:

[0139]

[0140]

[0141]

[0142] in, and Let A and B represent the supervision losses respectively, and let B be the total loss. In one embodiment of this application, the main loss is a hybrid loss combining cross-entropy and Dice loss as the supervision loss. and Let the cross-entropy loss of model A and model B be represented respectively. and The weights of Dice loss, cross-entropy loss, and Dice loss in the supervision loss of model A and model B are both 0.5. This represents the boundary augmentation loss of model A, with its loss weights set to... ; This represents the boundary branch loss of model B, with fixed loss weights. ; This represents the confidence-guided cross-monitoring loss. This indicates the current training epoch. This represents the weights of the cross-pseudo-supervision loss in the current training epoch. ( The segmented setting method is adopted:

[0143]

[0144] In the preheating stage ( (before) Cross-supervision is not enabled within the first epoch, and then in the second epoch... To the It increases linearly within each epoch, reaching the first set value. After that, it remains unchanged; during the iterative training phase, i.e. At that time, before The second set value is taken for each epoch. Then take the third set value. It is understandable that segmenting the weights of the cross-supervision loss is to control the intensity of the introduction of pseudo-supervision signals based on the changes in the model's prediction reliability at different training stages, thereby reducing the risk of early noise propagation and enhancing the cooperative constraint effect between the two branches in the mid-to-late stages.

[0145] Boundary augmentation loss of model A Represented as:

[0146]

[0147] In the formula, This represents the hyperparameter polarization multiplication parameter, used to amplify boundary pixel loss. When boundary positions are incorrectly segmented, the penalty is increased, improving model A's boundary learning performance. In this embodiment, it is set as follows: ; This represents the set of image pixels in a single frame. For model A at pixel position The prediction results are given here; It is a pixel-level boundary prior weight map extracted based on the truth mask. Specifically, based on the manually labeled segmentation truth value corresponding to the input ultrasound image, the truth mask is contoured by mathematical morphology edge detection operator, the target refined edge region is extracted and binarized, and a boundary mask corresponding to the original image pixels is obtained. For pixel-level segmentation of truth labels, and They have the same spatial dimensions, but their semantic functions are independent of each other.

[0148] Boundary branch loss of model B Represented as:

[0149]

[0150] in, This represents the boundary head output of model B.

[0151] Cross-supervision loss A one-way constraint approach is adopted, meaning that only model A provides supervision signals to model B, and model B does not provide reverse constraints to model A. Let the prediction result output by model A in the current round be... The prediction result output by model B in the current round is Then the cross-monitoring loss It can be represented as:

[0152]

[0153] in, This indicates an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. This represents the confidence threshold, which is set to a value in this embodiment. ; This indicates the prediction result of model A. A validity determination function that considers both area and shape constraints: If The proportion of the target region area to the total area of ​​the input image is located at Within the range, and the prediction results of model A After two The smoothed crossover ratio after morphological smoothing is greater than or equal to the third threshold. At that time, it was considered Otherwise, it is considered Among them, the area threshold and The area ratio of the real-labeled mask (i.e., the manually labeled mask) is adaptively determined statistically: when there are sufficient real-labeled samples, i.e., when the number of non-empty labeled samples available for statistical analysis is no less than 3, the quartile method is used. , When the number of actually labeled samples is insufficient, the default range [0.05, 0.5] is used. and These represent the 25th and 75th percentiles of the set of actual annotated mask area proportions, respectively. This represents the interquartile range.

[0154] Regarding training parameters, both Model A and Model B used the AdamW optimizer, with a learning rate set to 0.005 and a weight decay coefficient set to... The number of epochs E per training cycle is set to 50, and the batch size is set to 8. The learning rate is set using a linear warm-up strategy for the first 5 epochs followed by a polynomial decay strategy. Specifically, when the current epoch is less than 5, the learning rate scaling factor is (e+1) / 5; when the current epoch is not less than 5, the learning rate scaling factor is:

[0155]

[0156] Regarding sample weight settings, the weight of the real labeled samples is set to 1.0, and the weight of the non-empty pseudo-label samples is set to 0.5. During the iterative training phase, a pseudo-label annealing strategy is adopted, in which all pseudo-label sample weights are reset to 0 in the last 5 epochs of each training cycle, and only the real labeled samples are retained for loss calculation.

[0157] The final ultrasound sequence image segmentation model consists of Model A and Model B. When performing ultrasound sequence image segmentation using the trained ultrasound sequence image segmentation model, the ultrasound sequence image to be segmented is input into Model A and Model B respectively. Model A outputs the extracted global semantic information of the image, and Model B outputs the extracted image boundary detail information. The outputs of the two are then fused to obtain the final segmentation result.

[0158] To verify the segmentation performance of the model trained by this invention, a comparative experiment was conducted with existing mainstream medical image segmentation methods. Among them, nnU-Net-v2 (2024) is a fully supervised model requiring complete annotation, while Mean Teacher (2017), CAT (2023), ABD (2024), DiffRect (2024), FDCL (2025), and this invention are all semi-supervised segmentation methods requiring only a small amount of annotation. Experiments were conducted on three public datasets (CAMUS heart, Carotid carotid artery, and Ultrasound Nerve) and a private dataset (RA radial artery), using Precision, Recall, F1, IoU, and HD95 as evaluation metrics.

[0159] Referring to Table 1, the experimental results show that the method of this invention achieves leading comprehensive segmentation performance on all datasets, significantly outperforming existing semi-supervised methods overall, with many key indicators approaching or even matching the level of fully supervised models. Using only a small amount of labeled data, this invention achieves excellent F1, IoU, and HD95 performance on the Carotid, RA, CAMUS, and Ultrasound Nerve datasets, demonstrating outstanding boundary accuracy and object detection capabilities. While significantly reducing annotation costs, it also significantly improves the model's segmentation performance, robustness, and generalization ability.

[0160] Table 1. Comparison of Segmentation Results

[0161]

[0162] A second aspect of the present invention provides an ultrasound sequence image segmentation model training device, comprising:

[0163] The dataset construction module is configured to construct a keyframe dataset based on the original ultrasound sequences acquired clinically, construct an artificially labeled dataset based on a preset proportion of keyframe images in the keyframe dataset, and construct an unlabeled dataset using the remaining data in the keyframe dataset.

[0164] The preheating training module is configured to preheat the teacher network and the two-branch student network using the manually labeled dataset. The teacher network is used to generate reference segmentation results for unlabeled images and to verify, correct, and filter pseudo-labels for the predictions of the two-branch student network. The two-branch student network includes a model A for extracting global semantic information of the image and a model B for extracting local boundary detail information of the image.

[0165] The iterative joint training module is configured to perform multiple rounds of iterative training. In each round, the model A and model B obtained from the previous round of training are used to predict the unlabeled image, and the reference segmentation result of the teacher model is combined with the prediction consistency of model A and model B and the validity of the reference segmentation result to generate a pseudo-label dataset. The artificially labeled dataset and the pseudo-label dataset are used to jointly train model A and model B, and the trained model A and model B are used as ultrasound sequence image models.

[0166] It should be noted that the foregoing explanation of the embodiment of the ultrasound sequence image segmentation model training method also applies to the ultrasound sequence image segmentation model training device of this embodiment, and will not be repeated here.

[0167] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing a computer program thereon, which is executed by a processor to perform the domain-adaptive endoscopic image restoration method of the above embodiments.

[0168] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present invention. It should be noted that the electronic device in the embodiments of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs, desktop computers, and servers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0169] like Figure 5 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device. The processing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0170] Typically, the following devices can be connected to I / O interface 105: input devices 106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, etc.; output devices 107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 109. Communication device 109 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0171] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 109, or installed from a storage device 108, or installed from a ROM 102. When the computer program is executed by the processing device 101, it performs the functions defined in the methods of the embodiments of the present invention.

[0172] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0173] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0174] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned ultrasound sequence image segmentation model training method.

[0175] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and Python, as well as conventional procedural programming languages ​​such as the "C-" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0178] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0179] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0180] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0181] Those skilled in the art will understand that implementing all or part of the steps of the methods in the above embodiments can be accomplished by instructing related hardware through a program. The developed program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0183] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for training an ultrasound sequence image segmentation model, characterized in that, include: A keyframe dataset is constructed based on the original ultrasound sequences acquired clinically. An artificially labeled dataset is constructed based on a preset proportion of keyframe images in the keyframe dataset. An unlabeled dataset is constructed using the remaining data in the keyframe dataset. The teacher network and the two-branch student network are pre-trained using the manually labeled dataset. The teacher network is used to generate reference segmentation results for unlabeled images and to verify, correct, and filter pseudo-labels for the predictions of the two-branch student network. The two-branch student network includes model A for extracting global semantic information of the image and model B for extracting local boundary detail information of the image. Multiple rounds of iterative training are conducted. In each round, Model A and Model B obtained from the previous round of training are used to predict unlabeled images. The reference segmentation results of the unlabeled images are combined with the teacher model. A pseudo-labeled dataset is generated based on the prediction consistency of Model A and Model B and the validity of the reference segmentation results. Model A and Model B are jointly trained using the manually labeled dataset and the pseudo-labeled dataset. The trained Model A and Model B are used as ultrasound sequence image models.

2. The training method according to claim 1, characterized in that, The keyframe dataset constructed based on the raw ultrasound sequences acquired clinically includes: Using optical flow and the Laplacian operator, a two-layer quality screening was performed on the raw DICOM ultrasound sequences acquired clinically to obtain the keyframe dataset that simultaneously meets the requirements of inter-frame information diversity and spatial clarity.

3. The training method according to claim 2, characterized in that, The optical flow method uses the Farneback dense optical flow algorithm. If the average amplitude of the optical flow field of the current frame is greater than the preset motion intensity threshold and the Laplacian variance of the current frame is greater than the preset sharpness threshold, then the frame is included in the keyframe dataset.

4. The training method according to claim 1, characterized in that, The teacher network uses a medical vision-based model. Model A uses an improved EfficientNet-UNet structure, including an encoder and a decoder. The encoder uses the EfficientNet-B0 backbone network, where the first convolutional layer of EfficientNet-B0 is configured as a single-channel input. The decoder uses a UNet-style progressive upsampling decoder, which has multiple upsampling and convolutional fusion modules. Each convolutional fusion module uses a combination of convolution, batch normalization, ReLU activation, and Dropout regularization. By extracting multi-scale features from different stages of EfficientNet-B0 and concatenating and fusing them with upsampled features in the decoding stage, deep semantic information and shallow spatial details can be utilized simultaneously. Model B uses an improved PIDNet-S architecture, including an input layer, a three-branch structure, a fusion module, and dual output heads. The three-branch structure includes a context branch, a high-resolution branch, and a boundary branch, with the boundary branch constrained by an independent boundary supervision loss during training. The first convolutional layer of the input layer is configured as a single-channel input. The fusion module is used to fuse the context information, high-resolution detail information, and boundary information extracted by the three-branch structure. The dual output heads include a segmentation output head and a boundary output head, with the boundary output head outputting a boundary prediction map during training.

5. The training method according to claim 1, characterized in that, Generate pseudo-labels for unlabeled images using the following formula: in: For any unlabeled image in the unlabeled dataset during the t-th training epoch. Generated pseudo-tags; , , These represent the consensus region, divergence region, and prompt box for the t-th training cycle, respectively. For the t-th training cycle, the teacher network is based on the prompt box. The output reference segmentation result, This is a symmetric difference operator; and The first The prediction results output by Model A and Model B after one training cycle; Let be the validity index of the reference segmentation result in the t-th training period, and the value is determined according to the following rules: when the reference segmentation result area Furthermore, the teacher network's response to the reference segmentation results Predicted overlap quality index At that time, I recorded Otherwise remember , These are the lower and upper threshold values ​​for the area ratio of the target region, respectively. The first threshold; The intersection-union ratio (IUU) is a consistency index between the prediction results of model A and model B, and its value is determined by the following rules: When the IUU of the prediction results of model A and model B is equal... At that time, I recorded Otherwise remember , This is the second threshold.

6. The training method according to claim 1, characterized in that, The process of constructing the training data also includes enhancing the input image, which is divided into strong enhancement processing and weak enhancement processing. The enhanced processing involves performing structure-preserving superpixel blending enhancement on the input image to obtain a superpixel image. : in, For the input image, For the input image Background image in the same domain, For superpixel masks, This is an element-wise multiplication operation; The weak enhancement process includes: performing a geometric transformation on the superpixel image and then inputting it into model A; performing a geometric transformation on the superpixel image, applying noise and blurring, and then inputting it into model B.

7. The training method according to claim 1, characterized in that, The total loss used when training models A and B for: in, and The supervision losses for models A and B are respectively. and The cross-entropy loss for model A and model B are respectively. and These are the Dice losses for model A and model B, respectively. Boundary augmentation loss for model A: In the formula, This represents the hyperparameter polarization multiplication parameter, used to amplify boundary pixel loss; This represents the set of image pixels in a single frame. For model A at pixel position The prediction results are given here; It is a pixel-level boundary prior weight map extracted based on a truth mask; Pixel-level segmentation of truth labels; This is the boundary branch loss for model B; In the formula, This represents the boundary header output of model B; For cross-supervision loss, a one-way constraint method is adopted, that is, only model A provides supervision signals to model B, and model B does not provide reverse constraints to model A. The expression is: In the formula, This indicates an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. The confidence threshold; and These are the prediction results for model A and model B, respectively. This represents the structural validity determination function, which considers both area and shape constraints: if the prediction result of model A... The proportion of the target region area to the total area of ​​the input image is located at Within the range, and the prediction results of model A After morphological smoothing, the smoothed intersection-union ratio is greater than or equal to the third threshold. At that time, it was considered Otherwise, it is considered , These are the lower and upper threshold values ​​for the area ratio of the target region, respectively. , and They are respectively , and Weights, weights The training period t and training round e are set in segments.

8. The training method according to claim 7, characterized in that, The weight The expression is: In the formula, This indicates the preheating stage. , and These are the first, second, and third training rounds, respectively. , and These are the first setting value, the second setting value, and the third setting value.

9. A training device for an ultrasound sequence image segmentation model, characterized in that, include: The dataset construction module is configured to construct a keyframe dataset based on the original ultrasound sequences acquired clinically, construct an artificially labeled dataset based on a preset proportion of keyframe images in the keyframe dataset, and construct an unlabeled dataset using the remaining data in the keyframe dataset. The preheating training module is configured to preheat the teacher network and the two-branch student network using the manually labeled dataset. The teacher network is used to generate reference segmentation results for unlabeled images and to verify, correct, and filter pseudo-labels for the predictions of the two-branch student network. The two-branch student network includes a model A for extracting global semantic information of the image and a model B for extracting local boundary detail information of the image. The iterative joint training module is configured to perform multiple rounds of iterative training. In each round, the model A and model B obtained from the previous round of training are used to predict the unlabeled image, and the reference segmentation result of the teacher model is combined with the prediction consistency of model A and model B and the validity of the reference segmentation result to generate a pseudo-label dataset. The artificially labeled dataset and the pseudo-label dataset are used to jointly train model A and model B, and the trained model A and model B are used as ultrasound sequence image models.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the ultrasound sequence image segmentation model training method according to any one of claims 1 to 8.

Images