An iterative weakly supervised maxillary sinus segmentation method and system for bounding box annotation

Abstract

An iterative weakly supervised maxillary sinus segmentation method for bounding box annotation includes: generating superpixels that closely fit the edges of oral CBCT images using a superpixel algorithm, extracting superpixel features using a convolutional neural network, and constructing a superpixel graph based on the adjacency of the superpixels; to improve the performance of bounding box weakly supervised segmentation, in addition to using a superpixel-level CRF model to generate initial pseudo-labels for training an improved graph convolutional network for superpixel semantic classification, the prediction results of the improved graph convolutional neural network are periodically input into the superpixel-level CRF model to update and refine the pseudo-labels for iterative learning and training; applying the model obtained by weakly supervised iterative training to oral CBCT images can produce excellent maxillary sinus segmentation results. The present invention also includes an iterative weakly supervised maxillary sinus segmentation system for bounding box annotation. The present invention solves the problem of time-consuming and labor-intensive manual labeling of medical images.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an iterative weakly supervised maxillary sinus segmentation method for bounding box annotation. Background Art

[0002] When dentists perform dental implant surgery, oral CBCT images serve as a crucial reference. The maxillary sinus, the largest sinus in the human body, is a conical cavity within the maxillary bone. Its segmentation and volume measurement can help assess the diagnosis of sinusitis and verify the effectiveness of radiotherapy for malignant tumors. Furthermore, segmentation can determine the location and boundaries of the maxillary sinus, aiding preoperative planning for dental implants. Therefore, accurate maxillary sinus segmentation is crucial in clinical applications.

[0003] Traditional methods such as fuzzy C-means have been used to automatically segment the maxillary sinus. While these methods can quickly produce segmentation results, they often struggle with details and boundaries. The recent introduction of networks such as Unet and Vnet has provided excellent models for maxillary sinus segmentation on oral CBCT. However, the performance of these models often relies on the precise labeling of the training data, which is time-consuming and expensive. Furthermore, accurate labeling of medical images is typically performed manually by physicians, requiring considerable effort. Consequently, weakly supervised learning has attracted considerable attention in recent years. Weakly supervised learning requires only rough labels of the target region, such as points, bounding boxes, and simple strokes, to train a relatively good segmentation model. These roughly labeled labels can save significant time and money.

[0004] Weakly supervised methods can reduce the need for pixel-level annotation and can be broadly categorized into two types: coarse supervision and sparse supervision. Bounding boxes are a type of coarse supervision, marking the rectangular area within the box as the object and the background outside the box. Points and scribbles are sparse supervision, labeling only a small portion of the object's pixels. While bounding box annotation is more complex than sparse annotation methods like points and scribbles, it contains more object information and produces more satisfactory segmentation results, making it a popular method for weakly supervised learning. For example, the Grabcut method uses a Gaussian mixture model to model the background and foreground after the user defines one or more bounding boxes containing the object. Based on texture and boundary information in the image, only minimal user interaction is required to achieve object-background separation / segmentation. While Grabcut achieved some breakthroughs, its results were still less than ideal. In 2016, Deepcut further optimized Grabcut and achieved better results. Deepcut is a widely cited weakly supervised bounding box segmentation method. Inspired by the Grabcut method, it employs the idea of ​​generating and refining pseudo-label masks during training, simulating full supervision with iteratively generated pseudo-labels and updating them using conditional random fields in post-processing. However, due to the limited and coarse information provided by weak supervision, there is still a significant gap between coarse bounding box supervision and full supervision in medical image semantic segmentation. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides an iterative weakly supervised maxillary sinus segmentation method for bounding box annotation. It only requires the limited information provided by the bounding box, introduces superpixels, and uses a convolutional neural network to extract superpixel features. Then, through an iterative optimization strategy, it utilizes superpixel-level CRF and an improved graph convolutional neural network to ultimately achieve a segmentation accuracy close to that of full supervision.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] An iterative weakly supervised maxillary sinus segmentation method and system for bounding box annotation, comprising the following steps:

[0008] Step 1: After preprocessing several oral cone beam CT (CBCT) images based on the optimal window width and window position of the maxillary sinus cavity, the maxillary sinus region of the coronal images was selected and annotated using a bounding box. The annotated coronal image data was then divided into a training set D1 and a test set D2 according to a preset ratio.

[0009] Step 2: Input each single-channel image in D1 and D2 into linear spectral clustering (LSC) to obtain n superpixels of the corresponding image, where n is the preset number of superpixels, and convert D1 and D2 into superpixel sets S1 and S2 respectively.

[0010] Step 3: Use the superpixel set S1 to train the feature extraction network Unet, including:

[0011] 3.1 Use pixel voting strategy to determine the label of each superpixel. For the ground truth value G of each bounding box annotation in the training set D1, count the superpixels s in G i Corresponding area The number of pixels belonging to each category in , the category with the highest frequency is selected as s i The category label y i :

[0012]

[0013] Among them, sgn(.) is a function indicating the category, sgn(true) = 1, sgn(false) = 0, lp represents the category of the p-th pixel, k and K are the category number and the total number of categories in the training set D1, respectively. express The total number of pixels in the

[0014] 3.2 Place each superpixel in S1 into a rectangular image patch of a preset size. The centroid of the superpixel is the center of the patch. Pixels in the patch that do not belong to the superpixel area are set to 0 to ensure that each patch has only one superpixel.

[0015] 3.3 Each patch is fed into the feature extraction network to obtain a d-dimensional feature vector, which is then fed into a SoftMax layer for classification and recognition training until the maximum number of iterations T1 is reached. The label of each patch is the label of its corresponding superpixel. The feature extraction network is a Unet with an encoder-decoder structure. The encoder part contains 4 convolutional layers with a unified 3×3 convolution kernel and a padding of 1. Each convolution layer contains a batch normalization (BN) operation. The decoder part contains 4 deconvolution layers with a convolution kernel size of 2×2 and a stride of 2. The last layer is a fully connected layer with d neurons.

[0016] Step 4: For each image in the training set D1, use the trained feature extraction network Unet to extract its superpixels s i The characteristic f i ∈R d , d is the eigenvector f i The length of the feature matrix F∈R n×d , construct the superpixel graph G = (V, E, F) of each image based on the characteristics of the superpixel blocks and the spatial relationship between the superpixel blocks, where V = {s i|i=1,2,..,n} and E={e ij |i=1,2,..,n,j=1,2,…,n} represent the set of all vertices and edges respectively. Each superpixel s i Corresponding to a node on the graph, the edge e ij Represents the spatial superpixel s i and s j The neighbor relationship between nodes, the edge relationship of all nodes constitutes an adjacency matrix A∈R n×n ,

[0017]

[0018] Step 5: Based on the superpixel map of each image, a conditional random field (CRF) is constructed with the superpixel category as a random variable. The bounding box annotation information is used as the unary potential of the CRF. SSVM (Structured SVM solver) is used to train a model with parameters θ = {W u , W p} superpixel-level CRF model, the maximum a posteriori (MAP) label sequence calculated using the AD3 algorithm is the optimal label y * :

[0019]

[0020]

[0021]

[0022]

[0023] in, is a unary potential function, which means that the superpixel s i Divided into label categories y i The energy of the super pixel feature f i and coefficient W u ∈R K×d The linear combination of p(y i |f i ) means that given the eigenvector f i In the case of superpixel x i Assigned to label y i The probability of represents the pairwise potential function, j∈δ i represents superpixel s i All neighboring superpixels of , sgn(.) is a 0-1 indicator function, is the transition matrix W p ∈R K×K An element that represents the jThe neighboring superpixels of the superpixel are marked as y i The possibility of |D ij | represents two superpixels s i and s j The eigenvector f i With f j The distance between ik is the eigenvector f i The kth element of ;

[0024] Step 6: The optimal label y of each superpixel image * As the initial pseudo label LB, the connection matrix A∈R of each superpixel graph n×n and the feature matrix F∈R n×d Input the superpixel classification network, namely the improved graph convolutional network (IGCN), and use the initial pseudo label LB to train and update the IGCN model parameters W (l) , W1 and W2 are used to minimize the cross entropy loss function L of the node until the iterative step number epoch is reached, where the propagation between IGCN layers is:

[0025]

[0026] Z (l+1) =Relu(H (l+1) W1)W2, (8)

[0027] Where Re Lu(.) is the activation function, and Represent the output of the l+1th layer and the lth layer respectively, H (0) =F,d l and d l+1 are the feature vector dimensions of the lth layer and the l+1th layer respectively, Add the self-connected adjacency matrix A, I∈R n×n is the identity matrix, yes The degree matrix, W (l) is the trainable weight matrix of layer l, Z (l +1) is the updated l+1th layer output, and are the weights of the two fully connected layers. The last layer of IGCN uses SoftMax to calculate the classification probability and combines the pseudo-label LB to calculate the cross entropy loss function L:

[0028]

[0029] where y i is a superpixel s i Tags, is a superpixel s i Take y i =1 probability;

[0030] Step 7: Based on the superpixel-level CRF and superpixel classification network IGCN, the pseudo labels of superpixels are gradually improved in an iterative learning manner until the maximum number of iterations T2 is reached, including:

[0031] 7.1 When the number of iterations reaches an integer multiple of epoch, the prediction result of IGCN is used as the unary potential of CRF, and SSVM training is used to update the superpixel-level CRF model parameters θ = {W u , W p}, use the AD3 algorithm to infer the superpixel-level CRF model to obtain the updated pseudo-label LB;

[0032] 7.2 Update IGCN model parameter matrix W based on pseudo-label LB training (l) ,W1 and W2 are used to minimize the cross entropy loss function L of the training node;

[0033] Step 8: Extract features from each superpixel in the superpixel set S2 using the trained feature extraction network Unet;

[0034] Step 9: For each image in the test set D2, a superpixel graph is constructed based on the spatial relationship between each superpixel feature and superpixels to obtain the connection matrix A and feature matrix F;

[0035] Step 10: Input the connection matrix A and feature matrix F of the superpixel image into the trained superpixel classification network IGCN to obtain the segmentation result of the maxillary sinus.

[0036] The technical concept of this invention is to generate superpixels that closely align with the edges of oral CBCT images using a superpixel algorithm. A convolutional neural network is then used to extract superpixel features and construct a superpixel graph based on the adjacency of the superpixels. To improve the performance of weakly supervised bounding box segmentation, in addition to using a superpixel-level CRF model to generate initial pseudo-labels for training an improved graph convolutional network for superpixel semantic classification, the predictions of the improved graph convolutional network are periodically fed into the superpixel-level CRF model to update and refine the pseudo-labels for iterative learning and training. For oral CBCT images, the model obtained through weakly supervised iterative training can achieve excellent maxillary sinus segmentation results.

[0037] An iterative weakly supervised maxillary sinus segmentation method for bounding box annotation consists of three parts: a superpixel segmentation unit, a superpixel map construction unit, and an iterative training unit. The superpixel segmentation unit inputs the oral CBCT image into linear spectral clustering (LSC) to obtain n superpixel blocks of the corresponding image. The superpixel map construction unit uses Unet to extract the features of each superpixel block, and constructs a superpixel map of each image based on the features of the superpixel blocks and the spatial relationship between the superpixel blocks. In addition to using the superpixel-level CRF model to generate initial pseudo labels for training the improved graph convolutional network for superpixel semantic classification, the iterative training unit also periodically inputs the prediction results of the improved graph convolutional neural network into the superpixel-level CRF model to update and refine the pseudo labels for iterative learning and training, and finally completes the segmentation of medical image organs. The present invention uses superpixels to provide good edge information for the target object, while also reducing the complexity of subsequent tasks. The conditional random field (CRF) is introduced to optimize the superpixel segmentation results according to the superpixel features. The pseudo labels are made more refined through iterative updates. Finally, an improved graph convolutional neural network (IGCN) is added to classify the nodes of the superpixel graph, thereby solving the problem of poor segmentation performance due to the lack of information provided by weak supervision.

[0038] The present invention also includes a system for implementing the iterative weakly supervised maxillary sinus segmentation method for bounding box annotation of the present invention, comprising: a preprocessing module, a superpixel segmentation module, a superpixel feature extraction and training module, a training superpixel map construction module, a coarse label refinement module, a superpixel classification training module, an iterative training module, a superpixel feature extraction module, a superpixel map construction module, and a superpixel classification module. Each of these modules corresponds, in order, to steps 1 to 10 of the method of the present invention.

[0039] The beneficial effects of the present invention are as follows: the present invention uses weak supervision to solve the time-consuming and labor-intensive problem of medical professionals manually labeling medical images with accurate labels. It adds a superpixel mechanism to process the image into multiple superpixel blocks and constructs a graph based on the spatial relationship of the superpixel blocks, which relieves the pressure of subsequent calculations and provides good edge information for the target object. It uses conditional random fields (CRFs) to generate initial pseudo labels based on superpixel features to improve graph convolutional network training, introduces iterative learning, and periodically replaces the bounding box label with the probability map output by IGCN during training as the unary potential of the superpixel CRF. The updated pseudo-label information is then used as the label for subsequent training. This alleviates the problem of low accuracy in medical image segmentation caused by the use of coarse labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the algorithm block diagram of the present invention.

[0041] Figure 2 Flowchart of the present invention.

[0042] Figure 3 a- Figure 3 d is the original image of the maxillary sinus and the weakly supervised label illustration of the present invention, wherein, Figure 3 a is the original image, Figure 3 b is the bounding box label, Figure 3 c is the superpixel segmentation result, Figure 3 d is the pseudo label based on superpixels.

[0043] Figure 4 a- Figure 4 f The original image of the maxillary sinus, the real label and the dynamic pseudo label illustration of the present invention, wherein, Figure 4 a is the original image, Figure 4 b is the pseudo label of the 40th iteration. Figure 4 c is the pseudo label at the 80th iteration. Figure 4 d is the pseudo label of the 120th iteration. Figure 4 e is the pseudo label at the 160th iteration. Figure 4 f is the fully supervised true label. DETAILED DESCRIPTION

[0044] The present invention will be further described below with reference to the accompanying drawings.

[0045] Reference Figure 1 Taking clinical real oral CBCT images as an example, the present invention provides an iterative weakly supervised maxillary sinus segmentation method for bounding box annotation, which includes the following steps:

[0046] Step 1: After pre-processing several oral CBCT (Cone beam CT) images based on the optimal window width and position of the maxillary sinus cavity (e.g., window position WL = -300 and window width WW = 800), the maxillary sinus region of the coronal image is selected and marked in the form of a bounding box, see Figure 3 (b) The labeled coronal image data are divided into a training set D1 and a test set D2 according to a preset ratio (e.g., 4:1);

[0047] Step 2: After obtaining the RGB images by taking the grayscale values ​​of each single-channel image in D1 and D2 as the values ​​of the three RGB channels, they are input into the linear spectral clustering (LSC) to obtain n (e.g. 1500) superpixels of the corresponding images, see Figure 3 (c), where n is the preset number of superpixels, D1 and D2 are converted into superpixel sets S1 and S2, respectively.

[0048] Step 3: Use the superpixel set S1 to train the feature extraction network Unet, including:

[0049] 3.1 Use pixel voting strategy to determine the label of each superpixel. For the ground truth value G of each bounding box annotation in the training set D1, count the superpixels s in G i Corresponding area The number of pixels belonging to each category in , the category with the highest frequency is selected as s i The category label y i :

[0050]

[0051] Among them, sgn(.) is a function indicating the category, sgn(true)=1,sgn(false)=0,l p represents the category of the p-th pixel, k and K are the category number and the total number of categories in the training set D1, respectively. express The total number of pixels in the

[0052] 3.2 Place each superpixel in S1 into a rectangular image patch of a preset size (e.g., 32×32). The centroid of the superpixel is the center of the patch, and pixels in the patch that do not belong to the superpixel area are set to 0 to ensure that each patch has only one superpixel.

[0053] 3.3 Each patch is input into the feature extraction network to obtain a d-dimensional feature vector (for example, d = 100), and then sent to a SoftMax layer for classification and recognition training until the maximum number of iterations T1 (for example, T1 = 200) is reached. The label of each patch is the label of its corresponding superpixel. The feature extraction network is a Unet with an encoder-decoder structure. The encoder part contains 4 convolutional layers with a unified 3×3 convolution kernel and a padding of 1. Each convolution layer contains a batch normalization (BN) operation. The decoder part contains 4 deconvolution layers with a convolution kernel size of 2×2 and a stride of 2. The last layer is a fully connected layer with d neurons.

[0054] Step 4: For each image in the training set D1, use the trained feature extraction network Unet to extract its superpixels s i The characteristic f i ∈R d , d is the eigenvector f i The length of the feature matrix F∈R n×d , construct the superpixel graph G = (V, E, F) of each image based on the characteristics of the superpixel blocks and the spatial relationship between the superpixel blocks, where V = {s i |i=1,2,..,n} and E={e ij|i=1,2,..,n,j=1,2,…,n} represent the set of all vertices and edges respectively. Each superpixel s i Corresponding to a node on the graph, the edge e ij Represents the spatial superpixel s i and s j The neighbor relationship between nodes, the edge relationship of all nodes constitutes an adjacency matrix A∈R n×n ,

[0055]

[0056] Step 5: Based on the superpixel map of each image, a conditional random field (CRF) is constructed with the superpixel category as a random variable. The bounding box annotation information is used as the unary potential of the CRF. SSVM (Structured SVM solver) is used to train a model with parameters θ = {W u , W p} superpixel-level CRF model, the maximum a posteriori (MAP) label sequence calculated using the AD3 algorithm is the optimal label y * :

[0057]

[0058]

[0059]

[0060]

[0061] in, is a unary potential function, which means that the superpixel s i Divided into label categories y i The energy of the super pixel feature f i and coefficient W u ∈R K×d The linear combination of p(y i |f i ) means that given the eigenvector f i In the case of superpixel x i Assigned to label y i The probability of represents the pairwise potential function, j∈δ i represents superpixel s i All neighboring superpixels of , sgn(.) is a 0-1 indicator function, is the transition matrix W p ∈R K×K An element that represents the j The neighboring superpixels of the superpixel are marked as y i The possibility of |Dij | represents two superpixels s i and s j The eigenvector f i With f j The distance between ik is the eigenvector f i The kth element of ;

[0062] Step 6: The optimal label y of each superpixel image * As the initial pseudo label LB, the connection matrix A∈R of each superpixel graph n×n and the feature matrix F∈R n×d Input the superpixel classification network, namely the improved graph convolutional network (IGCN), and use the initial pseudo label LB to train and update the IGCN model parameters W (l) , W1 and W2 are used to minimize the cross entropy loss function L of the node until the number of iteration steps epoch (for example, epoch = 40) is reached, where the propagation between IGCN layers is:

[0063]

[0064] Z (l+1) =Relu(H (l+1) W1)W2, (8)

[0065] Where Re Lu(.) is the activation function, and Represent the output of the l+1th layer and the lth layer respectively, H (0) =F,d l and d l+1 are the feature vector dimensions of the lth layer and the l+1th layer respectively, Add the self-connected adjacency matrix A, I∈R n×n is the identity matrix, yes The degree matrix, W (l) is the trainable weight matrix of layer l, Z (l +1) is the updated l+1th layer output, and are the weights of the two fully connected layers. The last layer of IGCN uses SoftMax to calculate the classification probability and combines the pseudo-label LB to calculate the cross entropy loss function L:

[0066]

[0067] where y i is a superpixel s i Tags, is a superpixel s iTake y i =1 probability;

[0068] Step 7: Based on the superpixel-level CRF and the superpixel classification network IGCN, the pseudo labels of the superpixels are gradually improved in an iterative learning manner until the maximum number of iteration steps T2 (for example, T2 = 200) is reached, including:

[0069] 7.1 When the number of iterations reaches an integer multiple of epoch, the prediction result of IGCN is used as the unary potential of CRF, and SSVM training is used to update the superpixel-level CRF model parameters θ = {W u , W p}, use the AD3 algorithm to infer the superpixel-level CRF model to obtain the updated pseudo-label LB;

[0070] 7.2 Update IGCN model parameter matrix W based on pseudo-label LB training (l) ,W1 and W2 are used to minimize the cross entropy loss function L of the training node;

[0071] Step 8: Extract features from each superpixel in the superpixel set S2 using the trained feature extraction network Unet;

[0072] Step 9: For each image in the test set D2, a superpixel graph is constructed based on the spatial relationship between each superpixel feature and superpixels to obtain the connection matrix A and feature matrix F;

[0073] Step 10: Input the connection matrix A and feature matrix F of the superpixel image into the trained superpixel classification network IGCN to obtain the segmentation result of the maxillary sinus.

[0074] The present invention also includes a system for implementing the iterative weakly supervised maxillary sinus segmentation method for bounding box annotation of the present invention, comprising: a preprocessing module, a superpixel segmentation module, a superpixel feature extraction and training module, a training superpixel map construction module, a coarse label refinement module, a superpixel classification training module, an iterative training module, a superpixel feature extraction module, a superpixel map construction module, and a superpixel classification module. Each of these modules corresponds, in order, to steps 1 to 10 of the method of the present invention.

[0075] The present invention uses weak supervision to solve the time-consuming and labor-intensive problem of medical professionals manually labeling medical images with precise labels. The addition of a superpixel mechanism processes the image into multiple superpixel blocks, alleviating the pressure of subsequent calculations and providing good edge information for the target object. Ultimately, through iterative updates, the pseudo-labels are refined, thereby solving the problem of poor segmentation performance due to the lack of information provided by weak supervision.

[0076] As described above, the specific implementation steps of the present invention make the present invention clearer. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims fall within the protection scope of the present invention.

Claims

1. An iterative weakly supervised maxillary sinus segmentation method for bounding box annotation, characterized by: The steps include: Step 1: After preprocessing several oral CBCT images (i.e., ConebeamCT) based on the optimal window width and position of the maxillary sinus cavity, the maxillary sinus region of their coronal images is selected and annotated using a bounding box. The annotated coronal image data is then divided into a training set D1 and a test set D2 according to a preset ratio. Step 2: Input each single-channel image in D1 and D2 into linear spectral clustering (LSC) to obtain n superpixels of the corresponding image, where n is the preset number of superpixels, and convert D1 and D2 into superpixel sets S1 and S2 respectively; Step 3: Use the superpixel set S1 to train the feature extraction network Unet; Step 4: For each image in the training set D1, use the trained feature extraction network Unet to extract its superpixels s i The characteristic f i ∈R d , d is the eigenvector f i The length of the feature matrix F∈R n×d , construct the superpixel graph G = (V, E, F) of each image based on the characteristics of the superpixel blocks and the spatial relationship between the superpixel blocks, where V = {s i |i=1,2,..,n} and E={e ij |i=1,2,..,n,j=1,2,...,n} represent the set of all vertices and edges respectively. Each superpixel s i Corresponding to a node on the graph, the edge e ij Represents the spatial superpixel s i and s j The neighbor relationship between nodes, the edge relationship of all nodes constitutes an adjacency matrix A∈R n×n , Step 5: Based on the superpixel map of each image, a conditional random field (CRF) is constructed with the superpixel category as a random variable. The bounding box annotation information is used as the unary potential of the CRF. The structured SVM solver SSVM is used to train the CRF with parameters θ = {W u , W p } superpixel-level CRF model, the label sequence of the maximum a posteriori MAP calculated using the AD3 algorithm is the optimal label y * : in, is a unary potential function, which means that the superpixel s i Divided into label categories y i The energy of the super pixel feature f i and coefficient W u ∈R K×d The linear combination of p(y i |f i ) means that given the eigenvector f i In the case of superpixel x i Assigned to label y i The probability of represents the pairwise potential function, j∈δ i represents superpixel s i All neighboring superpixels of , sgn(.) is a 0-1 indicator function, is the transition matrix W p ∈R K×K An element that represents the j The neighboring superpixels of the superpixel are marked as y i The possibility of |D ij | represents two superpixels s i and s j The eigenvector f i With f j The distance between ik is the eigenvector f i The kth element of ; Step 6: Take the optimal label y* of each superpixel image as the initial pseudo label LB, and transform the connection matrix A∈R of each superpixel image into n×n and the feature matrix F∈R n×d Input the superpixel classification network, i.e. the improved graph convolutional network IGCN, respectively, and use the initial pseudo-label LB to train and update the IGCN model parameters W (l) , W1 and W2 are used to minimize the cross entropy loss function L of the node until the iterative step number epoch is reached, where the propagation between IGCN layers is: WITH (l+1) =Relu(H (l+1) W1)W2,(8) Where Re Lu(.) is the activation function, and Represent the output of the l+1th layer and the lth layer respectively, H (0) =F,d l and d l+1 are the feature vector dimensions of the lth layer and the / +1th layer respectively, Add the self-connected adjacency matrix A, I∈R n×n is the identity matrix, yes The degree matrix, W (l) is the trainable weight matrix of layer l, Z (l+1) is the updated l+1th layer output, and are the weights of the two fully connected layers. The last layer of IGCN uses SoftMax to calculate the classification probability and combines the pseudo-label LB to calculate the cross entropy loss function L: where y i is a superpixel s i Tags, is a superpixel s i Take y i =1 probability; Step 7: Based on the superpixel-level CRF and superpixel classification network IGCN, the pseudo labels of superpixels are gradually improved in an iterative learning manner until the maximum number of iterations T2 is reached; Step 8: Extract features from each superpixel in the superpixel set S2 using the trained feature extraction network Unet; Step 9: For each image in the test set D2, a superpixel graph is constructed based on the spatial relationship between each superpixel feature and superpixels to obtain the connection matrix A and feature matrix F; Step 10: Input the connection matrix A and feature matrix F of the superpixel image into the trained superpixel classification network IGCN to obtain the segmentation result of the maxillary sinus.

2. The iterative weakly supervised maxillary sinus segmentation method for bounding box annotation according to claim 1, characterized in that: Step three specifically includes: 3.1 Use pixel voting strategy to determine the label of each superpixel. For the ground truth value G of each bounding box annotation in the training set D, count the superpixels s in G, and the corresponding area The number of pixels belonging to each category in , the category with the highest frequency is selected as s i The category label y i : Among them, sgn(.) is a function indicating the category, sgn(true) = 1, sgn(false) = 0, l p represents the category of the p-th pixel, k and K are the category number and the total number of categories in the training set D1, respectively. express The total number of pixels in the 3.2 Place each superpixel in S1 into a rectangular image patch of a preset size. The centroid of the superpixel is the center of the patch. Pixels in the patch that do not belong to the superpixel area are set to 0 to ensure that each patch has only one superpixel. 3.3 Each patch is input into the feature extraction network to obtain a d-dimensional feature vector, and then sent to a SoftMax layer for classification and recognition training until the maximum number of iterations T1 is reached. The label of each patch is the label of its corresponding superpixel. The feature extraction network is a Unet with an encoder-decoder structure. The encoder part contains 4 convolutional layers, unified as 3×3 convolution kernels, with padding of 1. Each convolution layer contains a batch normalization (BN) operation. The decoder part contains 4 deconvolution layers, with a convolution kernel size of 2×2 and a stride of 2. The last layer is a fully connected layer with d neurons.

3. The iterative weakly supervised maxillary sinus segmentation method for bounding box annotation according to claim 1, characterized in that: Step seven specifically includes: 7.1 When the number of iterations reaches an integer multiple of epoch, the prediction result of IGCN is used as the unary potential of CRF, and SSVM training is used to update the superpixel-level CRF model parameters θ = {W u , W p }, use the AD3 algorithm to infer the superpixel-level CRF model to obtain the updated pseudo-label LB; 7.2 Update IGCN model parameter matrix W based on pseudo-label LB training (l) , W1 and W2 to minimize the cross entropy loss function L of the training node.

4. A system for implementing the iterative weakly supervised maxillary sinus segmentation method for bounding box annotation according to claim 1, characterized in that: include: Preprocessing module, superpixel segmentation module, superpixel feature extraction training module, superpixel map construction and training module, coarse label refinement module, superpixel classification training module, iterative training module, superpixel feature extraction module, superpixel map construction module, superpixel classification module.

Images