Medical image segmentation method, device and equipment based on bounding box labeling and medium

Abstract

The application relates to a medical image segmentation method, device, equipment and medium based on a bounding box label, wherein the method comprises the following steps: acquiring a sample medical image, preprocessing the sample medical image to generate a training data set; inputting a target medical image and a bounding box mask into a secondary network to initialize the secondary network, and obtaining a basic secondary network; constructing pseudo-labeling of unlabeled voxels in the bounding box, training the basic secondary network in an alternating optimization mode to obtain a target secondary network; inputting the target medical image into a primary network to initialize the primary network to obtain a basic primary network; obtaining target pseudo-labeling output by the target secondary network, training the basic primary network in an alternating optimization mode to obtain a target primary network; acquiring a medical image to be segmented, and segmenting the medical image to be segmented based on the target primary network to obtain a target segmentation result. The application realizes accurate segmentation of a medical image only by using a bounding box label without complete voxel-level segmentation labeling.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a medical image segmentation method, apparatus, device, and medium based on bounding box annotation. Background Technology

[0002] Medical image segmentation plays a crucial role in computer-aided diagnosis. Deep learning models have been widely applied in this field. However, data-driven deep learning models require extensive voxel-level annotations, which consumes a significant amount of time and effort for doctors, limiting the widespread clinical application of deep learning models. Weakly supervised learning is an effective method to alleviate the annotation burden on doctors, utilizing less supervisory information to train the model. In medical image segmentation, weak annotation methods mainly include image-level annotation, doodle annotation, boundary point annotation, and bounding box annotation.

[0003] Image-level annotation refers to image category information. The mainstream approach is to train a classification network using image-level annotations to obtain CAM or Grad-CAM, and then generate voxel-level pseudo-annotations to supervise the segmentation model training. The drawback of this form of supervision is that CAM is not accurate enough in locating the segmented target. Doodle annotation refers to training the segmentation model using very few voxel-level annotations. The disadvantage is that the number of voxels in doodle annotations is small, which can easily lead to network overfitting. Boundary point annotation refers to the left, right, front, back, top, and bottom boundary points of the segmented target. These six boundary points simultaneously determine a bounding box. The mainstream approach is to search for the shortest integral path between the boundary points based on the image gradient map, generate initial doodle pseudo-annotations, and then proceed with subsequent training. The disadvantage is that for segmented targets with non-convex hull shapes, doodle pseudo-annotations may introduce noise. Conversely, bounding box annotation can provide coarse localization information for the segmented target. However, the unlabeled voxels inside the bounding box contain a mixture of positive and negative samples. The mainstream approach is to introduce volume priors of the segmented target to constrain the output of the segmentation model, but this is difficult to optimize for segmented targets with varying shapes and volumes. Therefore, there is an urgent need for a medical image segmentation method that can optimize segmentation targets with varying shapes and volumes, reduce noise introduced by false annotations, and improve image segmentation accuracy. Summary of the Invention

[0004] The purpose of this application is to propose a medical image segmentation method, apparatus, device, and medium based on bounding box annotation, which can optimize segmentation targets with varying shapes and volumes, reduce noise introduced by false annotations, and improve image segmentation accuracy.

[0005] To address the aforementioned technical problems, embodiments of this application provide a medical image segmentation method based on bounding box annotation, comprising:

[0006] Acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks;

[0007] The target medical image and the bounding box mask are input into the sub-network, and the sub-network is initialized based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network;

[0008] Based on the basic sub-network, pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images are constructed, and the basic sub-network is trained by alternating optimization of the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0009] The target medical image is input into the main network, and the main network is initialized based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network;

[0010] Obtain the target pseudo-labels output by the target sub-network, and train the base main network based on the method of alternating optimization of the relative entropy function and the second CRF regularized loss function to obtain the target main network;

[0011] A medical image to be segmented is acquired and preprocessed to obtain a target medical image to be segmented. The target medical image to be segmented is then segmented based on the target master network to obtain the target segmentation result.

[0012] To address the aforementioned technical problems, embodiments of this application provide a medical image segmentation device based on bounding box annotation, comprising:

[0013] A medical image acquisition unit is used to acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks;

[0014] The sub-network initialization unit is used to input the target medical image and the bounding box mask into the sub-network, and initialize the sub-network based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network;

[0015] The sub-network training unit is used to construct pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images based on the basic sub-network, and to train the basic sub-network by alternating optimization of the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0016] The main network initialization unit is used to input the target medical image into the main network and initialize the main network based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network.

[0017] The main network training unit is used to obtain the target pseudo-labels output by the target sub-network, and to train the basic main network based on the alternating optimization of the relative entropy function and the second CRF regularized loss function to obtain the target main network.

[0018] The segmentation result generation unit is used to acquire the medical image to be segmented, preprocess the medical image to be segmented to obtain the target medical image to be segmented, and perform segmentation processing on the target medical image to be segmented based on the target master network to obtain the target segmentation result.

[0019] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a computer device, including one or more processors; and a memory for storing one or more programs, such that the one or more processors implement the medical image segmentation method based on bounding box annotation as described in any one of the above-mentioned methods.

[0020] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the medical image segmentation method based on bounding box annotation as described above.

[0021] This invention provides a method, apparatus, device, and medium for medical image segmentation based on bounding box annotation. The method includes: acquiring sample medical images and preprocessing them to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks; inputting the target medical images and the bounding box masks into a sub-network, and initializing the sub-network based on a first cross-entropy loss function and a first projection loss function to obtain a basic sub-network; constructing pseudo-annotations for unlabeled voxels within the bounding boxes of the sample medical images based on the basic sub-network, and training the basic sub-network using an alternating optimization method based on a second cross-entropy loss function and a first CRF regularization loss function. The process involves several steps: first, obtaining a target sub-network; second, inputting the target medical image into the main network and initializing the main network using a third cross-entropy loss function and a second projection loss function to obtain a basic main network; third, acquiring the target pseudo-labels output by the target sub-network and training the basic main network using an alternating optimization method based on the relative entropy function and the second CRF regularization loss function to obtain the target main network; fourth, acquiring the medical image to be segmented and preprocessing it to obtain the target medical image to be segmented; and fifth, segmenting the target medical image based on the target main network to obtain the target segmentation result. This invention optimizes segmentation targets with varying shapes and volumes by initializing and training the sub-network and the main network separately, and correcting the network output during training to reduce errors caused by pseudo-label noise. This reduces noise introduced by pseudo-labels and improves image segmentation accuracy. Attached Figure Description

[0022] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the implementation of the medical image segmentation method based on bounding box annotation provided in this application embodiment;

[0024] Figure 2 This is a schematic diagram of the image segmentation framework provided in an embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating the implementation of a sub-process in the medical image segmentation method based on bounding box annotation provided in this application embodiment;

[0026] Figure 4 This is a flowchart illustrating the implementation of a sub-process in the medical image segmentation method based on bounding box annotation provided in this application embodiment;

[0027] Figure 5 This is a flowchart illustrating the implementation of a sub-process in the medical image segmentation method based on bounding box annotation provided in this application embodiment;

[0028] Figure 6 This is a flowchart illustrating the implementation of a sub-process in the medical image segmentation method based on bounding box annotation provided in this application embodiment;

[0029] Figure 7 This is a schematic diagram of a medical image segmentation device based on bounding box annotation provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the medical image segmentation method based on bounding box annotation provided in this application embodiment is generally executed by a server, and correspondingly, the medical image segmentation device based on bounding box annotation is generally configured in the server.

[0036] Please see Figure 1 and Figure 2 , Figure 1This paper illustrates a specific implementation of a medical image segmentation method based on bounding box annotation. Figure 2 This is a schematic diagram of the image segmentation framework provided in an embodiment of this application.

[0037] Specifically, both the main and sub-networks use 3D U-net as the segmentation network. The segmentation network includes an encoding module and a decoding module, each of which comprises two convolutional blocks. Each convolutional block contains a convolution, an activation function, and a normalization operation. Initially, the number of convolutional channels is 16. After the input medical image passes through the encoding module, the number of channels increases to twice the original, and the size is reduced to half the original. Before passing through the decoding module, the feature map needs to be upsampled to twice its original size using a transposed convolution, and a skip connection is used to concatenate the encoded feature map of the corresponding size with it.

[0038] Furthermore, the sub-network embeds a boundary encoder, which includes convolution operations and activation function operations. The boundary encoder is used to output bounding box feature maps, and performs a dot product between the bounding box feature maps and the output feature maps of the first two encoding modules of the sub-network. The dot-multiplied feature maps are then incorporated into the decoding module of the sub-network using skip connections.

[0039] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 1 Limited to the sequence shown, this method includes the following steps:

[0040] S1: Acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks.

[0041] Specifically, in this embodiment of the application, it is necessary to obtain sample medical images with bounding boxes, preprocess the sample medical images, and generate a training dataset, wherein the training dataset includes target medical images and bounding box masks.

[0042] Further, preprocessing includes: resampling at an appropriate target resolution based on the dataset to scale the image size. Then, the data distribution of the resampled images is standardized to obtain training data. During subsequent network training, fixed-size patches are randomly sampled from the sample images and input into the network. The target resolution selection strategy is as follows: the resolutions of each dimension of the training data are sorted in ascending order, and the median is taken as the target resolution; if the target resolution is anisotropic (the maximum value divided by the minimum value of the target resolution is greater than 3), the dimension with the lowest resolution is selected as the 10th percentile of the data in that dimension's resolution ranking as the target resolution for that dimension.

[0043] S2: Input the target medical image and the bounding box mask into the sub-network, and initialize the sub-network based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network.

[0044] Specifically, to avoid introducing noisy annotations during network initialization, bounding box annotations are converted into projected annotations, and these projected annotations are combined with the background voxel annotations outside the bounding box to initialize the sub-network. This allows the sub-network to output probabilistic pseudo-annotations for segmented targets with varying shapes and volumes. In the application embodiment, the target medical image and bounding box mask are input into the sub-network, and the sub-network is initialized based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network.

[0045] Please see Figure 3 , Figure 3 A specific implementation of step S2 is shown below:

[0046] S21: Input the target medical image and the bounding box mask into the sub-network, and convert the bounding box mask into projection annotations.

[0047] S22: Obtain the set of voxels outside the bounding box corresponding to the target medical image.

[0048] S23: The sub-network is initialized based on the voxel set, the projection annotation, and the target medical image using the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network.

[0049] Specifically, since the projection labels obtained by projecting the bounding box mask onto the three-dimensional coordinate axis are the same as the projection labels obtained by projecting the voxel-level labels onto the three-dimensional coordinate axis, in order to avoid introducing noisy labels during the initialization stage of the sub-network, this embodiment uses the background cross-entropy loss function and the projection loss function to jointly optimize the sub-network.

[0050] Furthermore, the first cross-entropy loss function and the first projection loss function are respectively:

[0051]

[0052]

[0053] in, Let X be the first cross-entropy loss value, B be the target medical image, and Ω be the bounding box mask. O Let be the set of voxels, φ be the parameters of the sub-network, and Q(X, B; φ) be the foreground probability. This is the first projection loss value. Let λ be the logarithmic barrier spread function. projLet S be the weight, v be the voxel, and S be the weight. t S represents the set of all cross-sections that intersect with the bounding box. c Let S represent the set of all coronal planes that intersect the bounding box. s |S| represents the set of all sagittal planes that intersect with the bounding box. t |、|S c |and|S s | represents the size of the corresponding set, with values ​​equal to the length, width, and height of the bounding box. and Let represent the foreground probabilities output by the sub-network along the projection path of the bounding box, respectively. First projection loss function. The sum of the probabilities for each foreground must be greater than the length, width, and height of the bounding box, meaning that the sub-network must predict at least one positive sample in each cross section, coronal plane, and sagittal plane within the bounding box.

[0054] Furthermore, the logarithmic barrier extension function for:

[0055]

[0056] Where b is the slope of the control obstacle curve.

[0057] In the application embodiment, a stochastic gradient descent optimization algorithm is required, using a first cross-entropy loss function and a first projection loss function to iteratively update the parameters of the sub-network. In a specific embodiment, all training data needs to be trained 20 times with a learning rate of 1e-4 and weights λ. proj =0.01, b=5.

[0058] S3: Based on the basic sub-network, construct pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images, and train the basic sub-network using an alternating optimization method based on the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0059] Specifically, in this embodiment, the parameters of the basic sub-network are alternately optimized using the second cross-entropy loss function and the first CRF regularization loss function, thereby training the basic sub-network to obtain the target sub-network.

[0060] Please see Figure 4 , Figure 4 A specific implementation of step S3 is shown below:

[0061] S31: Obtain the foreground probability of unlabeled voxels within each bounding box output by the basic sub-network to obtain the target foreground probability.

[0062] S32: Assign each foreground probability to a preset threshold to construct a pseudo-label for the unlabeled voxels within the bounding box of the sample medical image.

[0063] Specifically, to increase the number of voxel annotations, this embodiment assigns pseudo-annotations to unannotated voxels within the bounding box using a hard threshold; if the foreground probability Q(X,B;φ)(v) corresponding to a voxel is greater than the threshold t fg Then the voxel is considered as the foreground; if the probability Q(X,B;φ)(v) corresponding to a voxel is less than the threshold t bg If the voxel is considered as background, then when the comparison of unlabeled voxels within all bounding boxes is completed, pseudo-labels for all unlabeled voxels within the bounding boxes of the sample medical image are obtained.

[0064] It should be noted that the preset threshold is set according to the actual situation and is not limited here. In one specific embodiment, the threshold t is... fg Set the threshold t to 0.95. bg Set it to 0.05.

[0065] S33: The base sub-network is trained based on the pseudo-labels by alternately optimizing the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0066] Please see Figure 5 , Figure 5 A specific implementation of step S33 is shown below:

[0067] S331: The pseudo-labels are sampled using a random sampling method to form a pseudo-label sample set.

[0068] Specifically, the pseudo-labels have been divided into background and foreground as described above. In this embodiment of the application, pseudo-labels are sampled by random sampling in the two types of pseudo-labels to form a pseudo-label sample set, which facilitates the optimization of their cross-entropy loss function.

[0069] S332: The second cross-entropy loss function is used to calculate the loss based on the pseudo-labeled sample set to obtain the second cross-entropy loss value.

[0070] The second cross-entropy loss function is:

[0071]

[0072] Among them, Ω S This is a pseudo-labeled sample set. The pseudo-label represents voxel v, with 0 for background and 1 for foreground. This represents the second cross-entropy loss value.

[0073] S333: Update the parameters of the basic sub-network using the second cross-entropy loss value to obtain the first updated sub-network.

[0074] Specifically, calculation The gradient of the parameter φ of the basic secondary network is used to update φ, thus obtaining the first updated secondary network.

[0075] S334: Update the parameters of the first update sub-network using the first CRF regular loss function to obtain the second update sub-network, wherein the first CRF regular loss function is used to optimize the edges of the network segmentation results through the spatial and gray-level consistency of homogeneous regions of the image.

[0076] Specifically, when the first updated sub-network is obtained, the first CRF regularized loss function is optimized through the second forward propagation. The edges of the network segmentation result are optimized through the spatial and gray-level consistency of homogeneous regions in the image. The first CRF regularized loss function is as follows:

[0077]

[0078] Where, λ crf σ represents the weight, d represents the Euclidean distance between voxels, and σ represents the weight. α and σ β Control spatial and grayscale consistency separately.

[0079] S335: The second updated sub-network is re-optimized using the stochastic gradient descent optimization algorithm until the first preset number of iterations is reached, thereby obtaining the target sub-network.

[0080] Specifically, before the first preset number of iterations is reached, the second update sub-network is alternately optimized again using the stochastic gradient descent optimization algorithm until the first preset number of iterations is reached to obtain the target sub-network.

[0081] In one specific embodiment, the basic sub-network is trained iteratively 200 times on all training data using a stochastic gradient descent optimization algorithm. The learning rate is 0.0001, and the learning rate adjustment strategy is 0.0001 × (1 - (epoch / 200)^0.9). For the first CRF regularized loss function, this embodiment sets λ... crf =0.001, σ α =15, σ β =0.05. Finally, save the sub-network weights to obtain the target sub-network.

[0082] S4: Input the target medical image into the main network, and initialize the main network based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network.

[0083] Specifically, in the main network, the input data is the target medical image, and the main network is initialized using the third cross-entropy loss function and the second projection loss function to obtain the basic main network.

[0084] Specifically, the third cross-entropy loss function and the second projection loss function are respectively:

[0085]

[0086]

[0087] Where X represents the target medical image, Ω O Let P be the set of voxels outside the bounding box, θ be the parameters of the main network, and P(X;θ) be the foreground probabilities output by the main network.

[0088] In the application embodiment, a stochastic gradient descent optimization algorithm is required, using the third cross-entropy loss function and the second projection loss function to iteratively update the parameters of the sub-network. In a specific embodiment, all training data needs to be trained 20 times with a learning rate of 1e-4 and weights λ. proj =0.01, b=5.

[0089] S5: Obtain the target pseudo-labels output by the target sub-network, and train the base main network based on the alternating optimization of the relative entropy function and the second CRF regularized loss function to obtain the target main network.

[0090] Specifically, in this embodiment, it is necessary to import and freeze the weights of the sub-network, and use the output of the sub-network as pseudo-labels to guide the training of the main network through knowledge distillation; when the main network is trained, the parameters of the basic sub-network are alternately optimized using the relative entropy function and the second CRF regular loss function to obtain the target sub-network.

[0091] Please see Figure 6 , Figure 6 A specific implementation of step S3 is shown below:

[0092] S51: Import and freeze the weights of the target sub-network, and obtain the target pseudo-labels output by the target sub-network.

[0093] S52: Minimize the relative entropy between the output distribution of the main network and the output distribution of the target sub-network using the relative entropy function to update the parameters of the basic main network and obtain the first updated main network.

[0094] Specifically, the relative entropy function is:

[0095]

[0096] Where, λkd As the weight, Ω I To represent the set of voxels within the bounding box, Let the relative entropy be quantified. The gradient of the parameter θ of the base main network is used to update θ, thus obtaining the first updated main network.

[0097] It should be noted that, in the embodiments of this application, CRF refers to Conditional Random Field.

[0098] S53: Update the parameters of the first updated main network using the second CRF regularized loss function to obtain the second updated main network.

[0099] Specifically, the second CRF regularization loss function is:

[0100]

[0101] calculate The gradient of the parameters θ of the first updated main network is used to update θ, thus obtaining the second updated main network.

[0102] S54: The second updated main network is re-optimized using the stochastic gradient descent optimization algorithm until the second preset number of iterations is reached, and the target main network is obtained.

[0103] Specifically, before the second preset number of iterations is reached, the stochastic gradient descent optimization algorithm is used to alternately optimize the second update main network until the second preset number of iterations is reached, thus obtaining the target sub-network.

[0104] In one specific embodiment, all training data are trained iteratively 200 times using a stochastic gradient descent optimization algorithm. The learning rate is 0.0001, and the learning rate adjustment strategy is 0.0001 × (1 - (epoch / 200)^0.9). For weight λ kd The embodiments of this application use the following formula for setting:

[0105]

[0106] Among them, for This application's embodiment sets λ crf =0.001,σ α =15,σ β =0.05. Finally, save the main network weights to obtain the target main network.

[0107] S6: Obtain the medical image to be segmented, preprocess the medical image to be segmented to obtain the target medical image to be segmented, and perform segmentation processing on the target medical image to be segmented based on the target master network to obtain the target segmentation result.

[0108] Specifically, the image segmentation model has already been trained using the above steps. Therefore, when medical image segmentation is required, the medical image to be segmented is acquired and preprocessed, including resampling and data distribution standardization. After preprocessing, the target medical image to be segmented is obtained. This target medical image is then input into the target main network, and sliding window inference is performed on the target medical image in patch units. Finally, the reconstructed segmentation result is obtained and used as the target segmentation result.

[0109] In this embodiment, sample medical images are acquired and preprocessed to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks; the target medical images and the bounding box masks are input into a sub-network, and the sub-network is initialized based on a first cross-entropy loss function and a first projection loss function to obtain a basic sub-network; pseudo-labels of unlabeled voxels within the bounding boxes corresponding to the sample medical images are constructed based on the basic sub-network, and the basic sub-network is trained by alternately optimizing based on a second cross-entropy loss function and a first CRF regularization loss function. The process involves several steps: first, obtaining a target sub-network; second, inputting the target medical image into the main network and initializing the main network using a third cross-entropy loss function and a second projection loss function to obtain a basic main network; third, acquiring the target pseudo-labels output by the target sub-network and training the basic main network using an alternating optimization method based on the relative entropy function and the second CRF regularization loss function to obtain the target main network; fourth, acquiring the medical image to be segmented and preprocessing it to obtain the target medical image to be segmented, and then segmenting the target medical image based on the target main network to obtain the target segmentation result. This invention optimizes segmentation targets with varying shapes and volumes by initializing and training the sub-network and the main network separately, and correcting the network output during training to reduce errors caused by pseudo-label noise. This reduces noise introduced by pseudo-labels and improves image segmentation accuracy. Furthermore, this embodiment achieves accurate segmentation of medical images using only bounding box annotations without relying on complete voxel-level segmentation annotations.

[0110] Please refer to Figure 7 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a medical image segmentation device based on bounding box annotation. This device embodiment is similar to... Figure 1Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0111] like Figure 7 As shown, the medical image segmentation device based on bounding box annotation in this embodiment includes: a medical image acquisition unit 71, a sub-network initialization unit 72, a sub-network training unit 73, a main network initialization unit 74, a main network training unit 75, and a segmentation result generation unit 76, wherein:

[0112] The medical image acquisition unit 71 is used to acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks.

[0113] The sub-network initialization unit 72 is used to input the target medical image and the bounding box mask into the sub-network, and initialize the sub-network based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network;

[0114] The sub-network training unit 73 is used to construct pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical image based on the basic sub-network, and to train the basic sub-network by alternately optimizing the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0115] The main network initialization unit 74 is used to input the target medical image into the main network and initialize the main network based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network;

[0116] The main network training unit 75 is used to obtain the target pseudo-labels output by the target sub-network, and to train the basic main network based on the alternating optimization of the relative entropy function and the second CRF regular loss function to obtain the target main network.

[0117] The segmentation result generation unit 76 is used to acquire the medical image to be segmented, preprocess the medical image to be segmented to obtain the target medical image to be segmented, and perform segmentation processing on the target medical image to be segmented based on the target main network to obtain the target segmentation result.

[0118] Furthermore, the sub-network initialization unit 72 includes:

[0119] A bounding box mask conversion unit is used to input the target medical image and the bounding box mask into the sub-network, and convert the bounding box mask into projection annotations;

[0120] A voxel set acquisition unit is used to acquire the voxel set outside the bounding box corresponding to the target medical image;

[0121] The basic sub-network generation unit is used to initialize the sub-network based on the voxel set, the projection annotation, and the target medical image using the first cross-entropy loss function and the first projection loss function, thereby obtaining the basic sub-network.

[0122] Furthermore, the first cross-entropy loss function and the first projection loss function are respectively:

[0123]

[0124]

[0125] in, Let X be the first cross-entropy loss value, B be the target medical image, and Ω be the bounding box mask. O Let be the set of voxels, φ be the parameters of the sub-network, and Q(X, B; φ) be the foreground probability. This is the first projection loss value. Let λ be the logarithmic barrier spread function. proj Let S be the weight, v be the voxel, and S be the weight. t S represents the set of all cross-sections that intersect with the bounding box. c Let S represent the set of all coronal planes that intersect the bounding box. s |S| represents the set of all sagittal planes that intersect with the bounding box. t |、|S c |and|S s | represents the size of the corresponding set, with values ​​equal to the length, width, and height of the bounding box. and These represent the foreground probabilities and sums of the sub-network outputs on the projection path of the bounding box, respectively.

[0126] Furthermore, the sub-network training unit 73 includes:

[0127] The target foreground probability generation unit is used to obtain the foreground probability of unlabeled voxels within each bounding box output by the basic sub-network, and thus obtain the target foreground probability.

[0128] The pseudo-annotation construction unit is used to map each foreground probability to a preset threshold in order to construct pseudo-annotations for unannotated voxels within the bounding box of the sample medical image.

[0129] The target sub-network generation unit is used to train the base sub-network based on the pseudo-labels by alternately optimizing the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network.

[0130] Furthermore, the target sub-network generation unit includes:

[0131] The pseudo-label sample set composition unit is used to sample the pseudo-labels using a random sampling method to form a pseudo-label sample set;

[0132] The second cross-entropy loss value generation unit is used to perform loss calculation based on the pseudo-labeled sample set using the second cross-entropy loss function to obtain the second cross-entropy loss value.

[0133] The first updated sub-network generation unit is used to update the parameters of the basic sub-network using the second cross-entropy loss value to obtain the first updated sub-network.

[0134] The second update sub-network generation unit is used to update the parameters of the first update sub-network using the first CRF regular loss function to obtain the second update sub-network. The first CRF regular loss function is used to optimize the edges of the network segmentation results through the spatial and gray-level consistency of homogeneous regions in the image.

[0135] The first training completion unit is used to alternately optimize the second update sub-network using the stochastic gradient descent optimization algorithm until the first preset number of iterations is reached, thereby obtaining the target sub-network.

[0136] Furthermore, the main network training unit 75 includes:

[0137] The target pseudo-label acquisition unit is used to import and freeze the weights of the target sub-network, and acquire the target pseudo-labels output by the target sub-network.

[0138] The first updated main network generation unit is used to minimize the relative entropy between the output distribution of the main network and the output distribution of the target sub-network through the relative entropy function, so as to update the parameters of the basic main network and obtain the first updated main network.

[0139] The second updated master network generation unit is used to update the parameters of the first updated master network using the second CRF regular loss function to obtain the second updated master network;

[0140] The second training completion unit is used to perform alternating optimization on the second updated main network using the stochastic gradient descent optimization algorithm until the second preset number of iterations is reached to obtain the target main network.

[0141] Furthermore, both the main network and the sub-network use 3D U-net as the segmentation network. The segmentation network includes an encoding module and a decoding module, and each encoding module and decoding module includes two convolutional blocks. The sub-network embeds a boundary encoder, which includes convolution operations and activation function operations. The boundary encoder is used to output bounding box feature maps, and performs a dot product between the bounding box feature maps and the feature maps output by the first two encoding modules of the sub-network. The feature map after the dot product is then incorporated into the decoding module of the sub-network using skip connections.

[0142] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0143] Computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that only a computer device 8 with three components—memory 81, processor 82, and network interface 83—is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0144] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0145] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the memory 81 may also include both internal storage units and external storage devices of the computer device 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the computer device 8, such as the program code of a medical image segmentation method based on bounding box annotation. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.

[0146] In some embodiments, processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, processor 82 is used to run program code stored in memory 81 or process data, for example, to run the program code of the above-described medical image segmentation method based on bounding box annotation, to implement various embodiments of the medical image segmentation method based on bounding box annotation.

[0147] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.

[0148] This application also provides another embodiment, namely, a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the medical image segmentation method based on bounding box annotation as described above.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0150] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A medical image segmentation method based on bounding box annotation, characterized in that, include: Acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks; The target medical image and the bounding box mask are input into the sub-network, and the sub-network is initialized based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network; Based on the basic sub-network, pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images are constructed, and the basic sub-network is trained by alternating optimization of the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network. The target medical image is input into the main network, and the main network is initialized based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network; Obtain the target pseudo-labels output by the target sub-network, and train the base main network based on the method of alternating optimization of the relative entropy function and the second CRF regularized loss function to obtain the target main network; A medical image to be segmented is acquired, and the medical image to be segmented is preprocessed to obtain a target medical image to be segmented. The target medical image to be segmented is then segmented based on the target master network to obtain a target segmentation result. The first cross-entropy loss function and the first projection loss function are respectively: ； in, This is the first cross-entropy loss value. For the target medical image, The bounding box mask, It is a set of voxels. The parameters of the sub-network are... Foreground probability, This is the first projection loss value. For the logarithmic barrier expansion function, As weight, For voxels, This represents the set of all cross-sections that intersect with the bounding box. This represents the set of all coronal planes that intersect with the bounding box. This represents the set of all sagittal planes that intersect with the bounding box. , and These represent the size of the corresponding set, and their values ​​are equal to the length, width, and height of the bounding box. , and These represent the foreground probabilities output by the sub-network along the projection path of the bounding box, respectively; Both the main network and the sub-network use 3D U-net as the segmentation network. The segmentation network includes an encoding module and a decoding module, and each encoding module and decoding module includes two convolutional blocks. The sub-network embeds a boundary encoder, which includes convolution operations and activation function operations. The boundary encoder is used to output bounding box feature maps, and performs a dot product between the bounding box feature maps and the feature maps output by the first two encoding modules of the sub-network. The feature map after the dot product is then incorporated into the decoding module of the sub-network using skip connections.

2. The medical image segmentation method based on bounding box annotation according to claim 1, characterized in that, The process of inputting the target medical image and the bounding box mask into the sub-network, and initializing the sub-network based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network includes: The target medical image and the bounding box mask are input into the sub-network, and the bounding box mask is converted into projection annotations; Obtain the set of voxels outside the bounding box corresponding to the target medical image; The sub-network is initialized based on the voxel set, the projection annotation, and the target medical image using the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network.

3. The medical image segmentation method based on bounding box annotation according to claim 1, characterized in that, The process involves constructing pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images based on the basic sub-network, and training the basic sub-network using an alternating optimization method based on the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network, including: The foreground probability of the target foreground is obtained by acquiring the foreground probability of the unlabeled voxels within each bounding box output by the basic sub-network. Each foreground probability is mapped to a preset threshold to construct a pseudo-label for the unlabeled voxels within the bounding box of the sample medical image. The target sub-network is obtained by training the basic sub-network based on the pseudo-label using an alternating optimization method of the second cross-entropy loss function and the first CRF regularization loss function.

4. The medical image segmentation method based on bounding box annotation according to claim 3, characterized in that, The method of alternating optimization using the second cross-entropy loss function and the first CRF regularization loss function to train the base sub-network based on the pseudo-labels to obtain the target sub-network includes: The pseudo-labels are sampled using a random sampling method to form a pseudo-label sample set; The second cross-entropy loss function is used to calculate the loss based on the pseudo-labeled sample set to obtain the second cross-entropy loss value; The parameters of the basic sub-network are updated using the second cross-entropy loss value to obtain the first updated sub-network; The parameters of the first update sub-network are updated by the first CRF regular loss function to obtain the second update sub-network. The first CRF regular loss function is used to optimize the edge of the network segmentation result through the spatial and gray-level consistency of homogeneous regions of the image. The second updated subnetwork is re-optimized using the stochastic gradient descent optimization algorithm until the first preset number of iterations is reached, thus obtaining the target subnetwork.

5. The medical image segmentation method based on bounding box annotation according to any one of claims 1 to 4, characterized in that, The process of obtaining the target pseudo-labels output by the target sub-network and training the base main network using an alternating optimization method based on the relative entropy function and the second CRF regularization loss function to obtain the target main network includes: Import and freeze the weights of the target sub-network, and obtain the target pseudo-labels output by the target sub-network; The relative entropy of the output distribution of the main network and the output distribution of the target sub-network is minimized by the relative entropy function to update the parameters of the basic main network, thereby obtaining the first updated main network; The parameters of the first updated main network are updated using the second CRF regularized loss function to obtain the second updated main network; The second updated main network is re-optimized using the stochastic gradient descent optimization algorithm until the second preset number of iterations is reached, thus obtaining the target main network.

6. A medical image segmentation device based on bounding box annotation, characterized in that, include: A medical image acquisition unit is used to acquire sample medical images and preprocess the sample medical images to generate a training dataset, wherein the training dataset includes target medical images and bounding box masks; The sub-network initialization unit is used to input the target medical image and the bounding box mask into the sub-network, and initialize the sub-network based on the first cross-entropy loss function and the first projection loss function to obtain the basic sub-network; The sub-network training unit is used to construct pseudo-labels for unlabeled voxels within the bounding boxes of the sample medical images based on the basic sub-network, and to train the basic sub-network by alternating optimization of the second cross-entropy loss function and the first CRF regularization loss function to obtain the target sub-network. The main network initialization unit is used to input the target medical image into the main network and initialize the main network based on the third cross-entropy loss function and the second projection loss function to obtain the basic main network. The main network training unit is used to obtain the target pseudo-labels output by the target sub-network, and to train the basic main network based on the alternating optimization of the relative entropy function and the second CRF regularized loss function to obtain the target main network. The segmentation result generation unit is used to acquire the medical image to be segmented, preprocess the medical image to be segmented to obtain the target medical image to be segmented, and perform segmentation processing on the target medical image to be segmented based on the target master network to obtain the target segmentation result; The first cross-entropy loss function and the first projection loss function are respectively: ； in, This is the first cross-entropy loss value. For the target medical image, The bounding box mask, It is a set of voxels. The parameters of the sub-network are... Foreground probability, This is the first projection loss value. For the logarithmic barrier expansion function, As weight, For voxels, This represents the set of all cross-sections that intersect with the bounding box. This represents the set of all coronal planes that intersect with the bounding box. This represents the set of all sagittal planes that intersect with the bounding box. , and These represent the size of the corresponding set, and their values ​​are equal to the length, width, and height of the bounding box. , and These represent the foreground probabilities output by the sub-network along the projection path of the bounding box, respectively; Both the main network and the sub-network use 3D U-net as the segmentation network. The segmentation network includes an encoding module and a decoding module, and each encoding module and decoding module includes two convolutional blocks. The sub-network embeds a boundary encoder, which includes convolution operations and activation function operations. The boundary encoder is used to output bounding box feature maps, and performs a dot product between the bounding box feature maps and the feature maps output by the first two encoding modules of the sub-network. The feature map after the dot product is then incorporated into the decoding module of the sub-network using skip connections.

7. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the medical image segmentation method based on bounding box annotation as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the medical image segmentation method based on bounding box annotation as described in any one of claims 1 to 5.

Images