A condyle CBCT image segmentation method based on edge and texture features

Abstract

The application discloses a condyle CBCT image segmentation method based on edge and texture features, first, the original condyle CBCT image is input into a condyle segmentation model to obtain the region of interest of the whole condyle; then the region of interest of the condyle is input into a condyle cortex and cancellous bone segmentation model, corresponding texture feature data is obtained through a texture feature extraction module, the texture feature data and a feature coding network are input into a feature decoding network containing texture attention, texture fusion features are further obtained, and low-level features and high-level features in the feature coding network are input into an edge extraction module to obtain edge information of the condyle cortex and cancellous bone; the edge information and the texture fusion features are input into a fusion module to finally obtain a visual result graph of the condyle cortex and cancellous bone. The application enables the segmentation network to obtain more features of the cortex and cancellous bone, and improves the segmentation accuracy of the condyle cortex and cancellous bone.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and in particular relates to a method for segmenting condylar CBCT images based on edge and texture features. Background Technology

[0002] Temporomandibular joint disorder (TMD) is the most common disease of the oral and maxillofacial region. It is a group of diseases whose pathogenesis is not fully understood, and in which the temporomandibular joint (TMJ) cannot function normally. It has a high incidence rate, ranking fourth among common oral diseases. Temporomandibular arthritis (TMJ-OA) is a subtype of TMD, which can lead to severe joint pain, functional impairment, malocclusion, and a decline in health-related quality of life. On imaging, TMJ-OA can be diagnosed through changes in condylar microstructure. This is because the affected condyle can show changes in bone microstructure, including bone sclerosis and cortical erosion and blurring. Therefore, early identification of condylar bone microstructure changes is crucial for the diagnosis and treatment of temporomandibular arthritis.

[0003] Cone-beam computed tomography (CBCT) is now considered one of the most reliable methods for diagnosing changes in the mandibular condyle because it allows observation of the integrity of the condylar cortex and underlying cancellous bone. However, the mandibular condyle remains one of the most difficult structures to segment on CBCT images. Its diverse and complex morphology, relatively low bone density, proximity to the articular disc, and blurred glenoid fossa make it anatomically challenging to describe.

[0004] However, there is currently almost no research using deep learning to identify fine segmentation of condyles in CBCT images. Existing techniques for segmenting condyles in images using models can only perform overall semantic segmentation of the condyle, failing to further refine the segmentation of the cortex and underlying cancellous bone, thus not helping to improve the efficiency of doctors' diagnosis and treatment. Although some techniques have achieved instance segmentation, they have not used targeted deep learning methods to improve the segmentation accuracy of the condylar cortex and cancellous bone, have not fully considered the various feature information between the cortex and cancellous bone, and have failed to solve the problem of the blurred boundary between the cortex and cancellous bone after a lesion. This leads to inaccurate segmentation of the cortex and cancellous bone when further segmenting the condyle, and situations such as the cortex of abnormal condyles being incorrectly predicted as cancellous bone. Summary of the Invention

[0005] To overcome the deficiencies in the prior art, this application provides a condylar CBCT image segmentation method based on edge and texture features. On the basis of improving the prior art, it further realizes fine segmentation of the condylar cortex and cancellous bone, and solves the problem of segmentation difficulties caused by the blurred boundary between the condylar cortex and cancellous bone.

[0006] To achieve the above objectives, the technical solution of this application is as follows:

[0007] A method for segmenting condylar CBCT images based on edge and texture features, comprising:

[0008] The original condylar CBCT image is input into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, the original condylar CBCT image is cropped to obtain the region of interest (ROI) of the condyle. ROI image;

[0009] Region of interest I ROI The image is input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps F at different scales. en ;

[0010] Region of interest I ROI The image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features F. texture ;

[0011] Feature map F en The input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features F. de ;

[0012] Feature map F en Low-level features F e_low and advanced features F e_high The edge features F are input into the edge feature extraction module of the cortical and cancellous segmentation model to obtain the edge features. edge ;

[0013] edge features F edge and regional features F at different scales de The data is input into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of condylar cortex and cancellous bone.

[0014] Furthermore, the original condylar CBCT image is input into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, the original condylar CBCT image is cropped to obtain the region of interest (ROI) of the condyle. ROI Images, including:

[0015] Step 1.1: Input the original condylar CBCT image into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F1, whose dimension is...

[0016] Step 1.2: Perform max pooling on feature map F1 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F2, whose dimension is...

[0017] Step 1.3: Perform max pooling on feature map F2 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F3, whose dimension is...

[0018] Step 1.4: Perform max pooling on feature map F3 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F4, whose dimension is...

[0019] Step 1.5: Perform max pooling on feature map F4 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F5, whose dimension is...

[0020] Step 1.6: Upsample feature map F5, then concatenate it with feature map F4 at the channel level, and input it into a convolution block with a kernel size of 3×3×3 to obtain feature map F. up1 Its dimension size is the same as that of F4;

[0021] Step 1.7: Transfer the feature map F up1 Upsampling is performed, followed by channel-level concatenation with feature map F3, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up2 Its dimension size is the same as that of F3;

[0022] Step 1.8: Transfer the feature map F up2 Upsampling is performed, followed by channel-level concatenation with feature map F2, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up3 Its dimension is the same as that of F2;

[0023] Step 1.9: Transfer the feature map F up3 Upsampling is performed, followed by channel-level concatenation with feature map F1, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up4 Its dimension size is the same as that of F1;

[0024] Step 1.10: Transfer the feature map F up4 The input is fed into a convolutional block with a kernel size of 1×1×1, and the detection result R is obtained. condyle ;

[0025] Step 1.11, based on the test result R condyle The original condylar CBCT image was cropped to obtain the region of interest (ROI) of the condyle. ROI Its dimensions are D0×H0×W0.

[0026] Furthermore, the region of interest I ROI The image is input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps F at different scales. en ,include:

[0027] Step 2.1: Select region of interest I ROI The image is input into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en1 Its dimensions are C1×D1×H1×W1;

[0028] Step 2.2: Transfer the feature map F en1 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en2 Its dimensions are C2×D2×H2×W2;

[0029] Step 2.4: Transfer the feature map F en2 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en3 Its dimensions are C3×D3×H3×W3;

[0030] Step 2.5: Transfer the feature map F en3 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en4 Its dimensions are C4×D4×H4×W4;

[0031] Step 2.6: Transfer the feature map F en4 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en5 Its dimensions are C5×D5×H5×W5. The feature maps obtained from each convolutional block form feature maps F of different scales. en .

[0032] Furthermore, the region of interest I ROI The image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features F. texture ,include:

[0033] Region of interest I ROI The image is input to the texture feature extraction module, where features are extracted using the HOG method to obtain the texture feature map F. texture Its dimensions are D0×H0×W0.

[0034] Furthermore, the HOG feature extraction includes:

[0035] Step 3.1: Select region of interest IROI The image yields D0 condylar image slices of size D0×H0×W0 along the Z-axis, and each slice is normalized for pixel values, resulting in a slice set S. 1 ;

[0036] Step 3.2: Slice each condylar image into pieces. i s i ∈S 1 , i∈[0,D0-1] are divided into multiple identical windows of a certain size, and each adjacent window does not overlap. Then each window is divided into multiple identical blocks of a certain size, and each adjacent block does not overlap. Finally, each window is divided into multiple identical cells of a certain size, and each adjacent cell does not overlap.

[0037] Step 3.3, for cell c i c i For each pixel in cell (i = 1, 2, ..., n), calculate the horizontal and vertical gradient values ​​G. x and G y For pixels The formulas for calculating the gradient values ​​in the horizontal and vertical directions are as follows:

[0038] G x (x,y)=h(x+1,y)-h(x-1,y);

[0039] G y (x,y)=h(x,y+1)-h(x,y-1);

[0040] Where h(x,y) represents the pixel value of pixel (x,y) after image normalization;

[0041] Therefore, the formulas for calculating the gradient magnitude G(x,y) and gradient direction θ(x,y) of pixel (x,y) are as follows:

[0042]

[0043]

[0044] Step 3.4: Calculate the gradient direction histogram in each cell. Divide the plane from 0 to 180 degrees into n intervals, and the interval from 181 to 360 degrees corresponds to the n intervals. The cumulative gradient value corresponding to the angle range within the interval is used as the vertical axis of the histogram, and the gradient value corresponding to the boundary of the interval is used as the horizontal axis of the histogram, thereby obtaining the HOG feature of each cell.

[0045] Step 3.5: Normalize the HOG features obtained from the cells in each block to obtain the fastest HOG features;

[0046] Step 3.6: Combine the HOG features of the blocks within each window to finally obtain slice s. i HOG characteristics;

[0047] Step 3.7: Transfer the slice set S 1 Repeat steps 3.1 to 3.6 for all slices to obtain the HOG features corresponding to each slice;

[0048] Step 3.8: Transfer the slice set S 1 The HOG features corresponding to all slices are arranged in slice order to obtain the texture feature map F. texture .

[0049] Furthermore, the feature map F en The input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features F. de ,include:

[0050] Step 4.1: Transfer the feature map F en5 Feature map F en4 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td4 Its dimensions are C4×D4×H4×W4;

[0051] Step 4.2: Transfer the feature map F td4 Feature map F en3 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td3 Its dimensions are C3×D3×H3×W3;

[0052] Step 4.3: Transfer the feature map F td3 Feature map F en2 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td2 Its dimensions are C2×D2×H2×W2;

[0053] Step 4.4: Transfer the feature map F td2 Feature map F en1 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td1 Its dimensions are C1×D1×H1×W1, ultimately yielding region features F at different scales. de.

[0054] Furthermore, the texture feature attention module performs the following operations:

[0055] Step 5.1: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. en5 Consistent with feature map F en5 The data is concatenated by channel and upsampled to match the feature map F. en4 With the same size, it is compared with feature map F en4 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td4 Its dimensions are C4×D4×H4×W4;

[0056] Step 5.2: Transfer the texture feature F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td4 Consistent with feature map F td4 The data is concatenated by channel and upsampled to match the feature map F. en3 With the same size, it is compared with feature map F en3 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td3 Its dimensions are C3×D3×H3×W3;

[0057] Step 5.3: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td3 Consistent with feature map F td3 The data is concatenated by channel and upsampled to match the feature map F. en2 With the same size, it is compared with feature map F en2 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td2 Its dimensions are C2×D2×H2×W2;

[0058] Step 5.4: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td2 Consistent with feature map F td2 The data is concatenated by channel and upsampled to match the feature map F. en1 With the same size, it is compared with feature map Fen4 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td1 Its dimensions are C1×D1×H1×W1.

[0059] Furthermore, the feature map F en Low-level features F e_low and advanced features F e_high The edge features F are input into the edge feature extraction module of the cortical and cancellous segmentation model to obtain the edge features. edge ,include:

[0060] F en5 Perform upsampling to make its size the same as the feature map F. en2 Consistent with each other and concatenated according to the number of channels, the data is input into the edge extraction module to obtain the edge feature map F. edge Its size is similar to the feature map F en2 Consistent.

[0061] Furthermore, the edge feature F edge and regional features F at different scales de The input is fed into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of the condylar cortex and cancellous bone, including:

[0062] Step 6.1: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td4 Consistent with and blended with texture features F td4 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse4 Its dimensions are C1×D1×H1×W1;

[0063] Step 6.2: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td3 Consistent with and blended with texture features F td3 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse3 Its dimensions are C1×D1×H1×W1;

[0064] Step 6.3: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td2 Consistent with and blended with texture features F td2 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F.fuse2 Its dimensions are C1×D1×H1×W1;

[0065] Step 6.4: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td1 Consistent with and blended with texture features F td1 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse1 Its dimensions are C1×D1×H1×W1;

[0066] Step 6.5: Merge feature F fuse4 Fusion feature F fuse3 Fusion feature F fuse2 and fusion feature F fuse1 The features are concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 1×1×1 to obtain the final edge enhancement region features. These edge enhancement region features are then activated using the following formula to obtain the output feature map:

[0067]

[0068] Here, sigmoid(x) represents the sigmoid activation operation on x, and x represents the edge enhancement region feature.

[0069] This application proposes a condylar CBCT image segmentation method based on edge and texture features. First, a condylar segmentation model is used to obtain the region of interest (ROI) of the entire condyle, reducing interference from other bone tissues in the CBCT image. Then, the condylar segmentation result is used as location information to further guide the attention of the fine-tuning network, enhancing its learning of cortical and cancellous bone features. Furthermore, by incorporating texture features from the image, the segmentation network acquires more features of the cortex and cancellous bone. Additionally, the addition of edge feature guidance allows the segmentation network to better distinguish the boundaries between the cortex and cancellous bone, ultimately resulting in more accurate segmentation of the condylar cortex and cancellous bone, and yielding a visualization of the condylar cortex and cancellous bone. Attached Figure Description

[0070] Figure 1 This is a flowchart of the condylar CBCT image segmentation method based on edge and texture features proposed in this application;

[0071] Figure 2 This is a technical framework diagram of the condylar CBCT image segmentation method based on edge and texture features proposed in this application;

[0072] Figure 3 This is a schematic diagram of the texture feature extraction module structure of this application;

[0073] Figure 4 This is a schematic diagram of the feature decoding network structure with texture attention in this application;

[0074] Figure 5 This is a schematic diagram of the edge feature extraction module structure of this application;

[0075] Figure 6 This is a schematic diagram of the feature fusion module structure of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0077] In one embodiment, such as Figure 1 As shown, a method for condylar CBCT image segmentation based on edge and texture features is provided, including:

[0078] Step S1: Input the original condylar CBCT image into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, crop the original condylar CBCT image to obtain the region of interest I of the condyle. ROI image.

[0079] The condylar segmentation model described in this application includes a feature encoding network and a feature decoding network. The feature encoding network contains five convolutional blocks and a downsampling operation, while the feature decoding network contains four convolutional blocks, an upsampling operation, and skip connections. The original condylar CBCT image is input into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, the original condylar CBCT image is cropped to obtain the region of interest (ROI) of the condyle. ROI Images, including:

[0080] Step 1.1: Input the original condylar CBCT image into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F1, whose dimension is...

[0081] Step 1.2: Perform max pooling on feature map F1 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F2, whose dimension is...

[0082] Step 1.3: Perform max pooling on feature map F2 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F3, whose dimension is...

[0083] Step 1.4: Perform max pooling on feature map F3 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F4, whose dimension is...

[0084] Step 1.5: Perform max pooling on feature map F4 and input it into a convolutional block with a kernel size of 3×3×3 to obtain output feature map F5, whose dimension is...

[0085] Step 1.6: Upsample feature map F5, then concatenate it with feature map F4 at the channel level, and input it into a convolution block with a kernel size of 3×3×3 to obtain feature map F. up1 Its dimension size is the same as that of F4;

[0086] Step 1.7: Transfer the feature map F up1 Upsampling is performed, followed by channel-level concatenation with feature map F3, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up2 Its dimension size is the same as that of F3;

[0087] Step 1.8: Transfer the feature map F up2 Upsampling is performed, followed by channel-level concatenation with feature map F2, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up3 Its dimension is the same as that of F2;

[0088] Step 1.9: Transfer the feature map F up3 Upsampling is performed, followed by channel-level concatenation with feature map F1, and then input into a convolutional block with a kernel size of 3×3×3 to obtain feature map F. up4 Its dimension size is the same as that of F1;

[0089] Step 1.10: Transfer the feature map F up4 The input is fed into a convolutional block with a kernel size of 1×1×1, and the detection result R is obtained. condyle ;

[0090] Step 1.11, based on the test result R condyle The original condylar CBCT image was cropped to obtain the region of interest (ROI) of the condyle. ROI Its dimensions are D0×H0×W0.

[0091] In addition to the convolution of the kernel size mentioned above, the convolution block also includes a normalization function and a ReLU activation function.

[0092] Step S2: Select region of interest I ROIThe image is input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps F at different scales. en .

[0093] The cortical and cancellous segmentation model described in this application is as follows: Figure 2 As shown, it includes a feature encoding network, a texture feature extraction module, a feature decoding network, an edge feature extraction module, and a feature fusion module.

[0094] The feature encoding network comprises five convolutional blocks and a downsampling method. Each convolutional block consists of two concatenated convolutional layers, and each convolutional layer consists of a convolution, a normalization layer, and a ReLU activation function. The region of interest I... ROI The image is input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps F at different scales. en ,include:

[0095] Step 2.1: Select region of interest I ROI The image is input into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en1 Its dimensions are C1×D1×H1×W1;

[0096] Step 2.2: Transfer the feature map F en1 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en2 Its dimensions are C2×D2×H2×W2;

[0097] Step 2.4: Transfer the feature map F en2 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en3 Its dimensions are C3×D3×H3×W3;

[0098] Step 2.5: Transfer the feature map F en3 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en4 Its dimensions are C4×D4×H4×W4;

[0099] Step 2.6: Transfer the feature map F en4 Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map F. en5 Its dimensions are C5×D5×H5×W5. The feature maps obtained from each convolutional block form feature maps F of different scales. en .

[0100] Step S3: Select region of interest I ROIThe image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features F. texture .

[0101] The texture feature extraction module described in this application is as follows: Figure 3 As shown, the region of interest I ROI The image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features F. texture ,include:

[0102] Region of interest I ROI The image is input to the texture feature extraction module, where features are extracted using the HOG method to obtain the texture feature map F. texture Its dimensions are D0×H0×W0.

[0103] The HOG feature extraction includes:

[0104] Step 3.1: Select region of interest I ROI The image yields D0 condylar image slices of size D0×H0×W0 along the Z-axis, and the pixel values ​​of each slice are normalized using the following formula:

[0105]

[0106] Where x and y represent the horizontal and vertical coordinates of a pixel, and minPixel and maxPixel represent the minimum and maximum pixel values ​​in the slice, ultimately yielding the slice set S. 1 ;

[0107] Step 3.2: Slice each condylar image into pieces. i s i ∈S 1 i∈[0,D0-1] is divided into multiple identical windows of a certain size, and each adjacent window does not overlap. Then each window is divided into multiple identical blocks of a certain size, and each adjacent block does not overlap. Finally, each window is divided into multiple identical cells of a certain size, and each adjacent cell does not overlap.

[0108] Step 3.3, for cell c i c i For each pixel in cell (i = 1, 2, ..., n), calculate the horizontal and vertical gradient values ​​G. x and G y For pixel (x, y), The formulas for calculating the gradient values ​​in the horizontal and vertical directions are as follows:

[0109] G x(x,y)=h(x+1,y)-h(x-1,y);

[0110] G y (x,y)=h(x,y+1)-h(x,y-1);

[0111] Where h(x,y) represents the pixel value of pixel (x,y) after image normalization;

[0112] Therefore, the formulas for calculating the gradient magnitude G(x,y) and gradient direction θ(x,y) of pixel (x,y) are as follows:

[0113]

[0114]

[0115] Step 3.4: Calculate the gradient direction histogram in each cell. Divide the plane from 0 to 180 degrees into n intervals, and the interval from 181 to 360 degrees corresponds to the n intervals. The cumulative gradient value corresponding to the angle range within the interval is used as the vertical axis of the histogram, and the gradient value corresponding to the boundary of the interval is used as the horizontal axis of the histogram, thereby obtaining the HOG feature of each cell.

[0116] Step 3.5: Normalize the HOG features obtained from the cells in each block to obtain the fastest HOG features.

[0117] Step 3.6: Combine the HOG features of the blocks within each window to finally obtain slice s. i HOG features.

[0118] Step 3.7: Transfer the slice set S 1 Repeat steps 3.1 to 3.6 for all slices to obtain the HOG features corresponding to each slice;

[0119] Step 3.8: Transfer the slice set S 1 The HOG features corresponding to all slices are arranged in slice order to obtain the texture feature map F. texture .

[0120] Step S4: Transfer feature map F en The input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features F. de .

[0121] The feature decoding network described in this application is as follows: Figure 4 As shown, the feature map F is combined with texture feature attention, including four convolutional blocks, skip connections, an upsampling method, and a texture feature attention module. enThe input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features F. de ,include:

[0122] Step 4.1: Transfer the feature map F en5 Feature map F en4 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td4 Its dimensions are C4×D4×H4×W4;

[0123] Step 4.2: Transfer the feature map F td4 Feature map F en3 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td3 Its dimensions are C3×D3×H3×W3;

[0124] Step 4.3: Transfer the feature map F td3 Feature map F en2 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td2 Its dimensions are C2×D2×H2×W2;

[0125] Step 4.4: Transfer the feature map F td2 Feature map F en1 and texture features F texture The input is fed into the texture feature attention module to obtain the texture fusion feature F. td1 Its dimensions are C1×D1×H1×W1, ultimately yielding region features F at different scales. de ;

[0126] The texture feature attention module performs the following operations:

[0127] Step 5.1: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. en5 Consistent with feature map F en5 The data is concatenated by channel and upsampled to match the feature map F. en4 With the same size, it is compared with feature map F en4 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td4 Its dimensions are C4×D4×H4×W4;

[0128] Step 5.2: Transfer the texture feature Ftexture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td4 Consistent with feature map F td4 The data is concatenated by channel and upsampled to match the feature map F. en3 With the same size, it is compared with feature map F en3 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td3 Its dimensions are C3×D3×H3×W3;

[0129] Step 5.3: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td3 Consistent with feature map F td3 The data is concatenated by channel and upsampled to match the feature map F. en2 With the same size, it is compared with feature map F en2 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td2 Its dimensions are C2×D2×H2×W2;

[0130] Step 5.4: Transfer texture features F texture The input is fed into a fully connected layer with a 1×1×1 convolutional kernel, the size of which is equal to that of the feature map F. td2 Consistent with feature map F td2 The data is concatenated by channel and upsampled to match the feature map F. en1 With the same size, it is compared with feature map F en4 The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion feature F is obtained. td1 Its dimensions are C1×D1×H1×W1.

[0131] Step S5: Transfer feature map F en Low-level features F e_low and advanced features F e_high The edge features F are input into the edge feature extraction module of the cortical and cancellous segmentation model to obtain the edge features. edge .

[0132] The edge extraction module described in this application is as follows: Figure 5 As shown, it includes a convolutional layer with a kernel size of 3×3×3, a normalization layer, and a ReLU activation function. The feature map F... en Low-level features F e_lowand advanced features F e_high The input is fed into the edge feature extraction module to obtain the edge features F. edge ,include:

[0133] F en5 Perform upsampling to make its size the same as the feature map F. en2 Consistent with each other and concatenated according to the number of channels, the data is input into the edge extraction module to obtain the edge feature map F. edge Its size is similar to the feature map F en2 Consistent.

[0134] Step S6: Transfer edge features F edge and regional features F at different scales de The data is input into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of condylar cortex and cancellous bone.

[0135] The feature fusion module described in this application is as follows: Figure 6 As shown, it includes four convolutional blocks and an upsampling operation, which modifies the edge features F edge and regional features F at different scales de The input is fed into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of the condylar cortex and cancellous bone, including:

[0136] Step 6.1: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td4 Consistent with and blended with texture features F td4 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse4 Its dimensions are C1×D1×H1×W1;

[0137] Step 6.2: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td3 Consistent with and blended with texture features F td3 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse3 Its dimensions are C1×D1×H1×W1;

[0138] Step 6.3: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td2 Consistent with and blended with texture features F td2 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse2Its dimensions are C1×D1×H1×W1;

[0139] Step 6.4: Transfer edge features F edge Perform max pooling to make its size match the texture fusion feature F td1 Consistent with and blended with texture features F td1 The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain a new fused feature F. fuse1 Its dimensions are C1×D1×H1×W1;

[0140] Step 6.5: Merge feature F fuse4 Fusion feature F fuse3 Fusion feature F fuse2 and fusion feature F fuse1 The features are concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 1×1×1 to obtain the final edge enhancement region features. These edge enhancement region features are then activated using the following formula to obtain the output feature map:

[0141]

[0142] Here, sigmoid(x) represents the sigmoid activation operation on x, where x represents the edge enhancement region feature. The output feature result map is a visualization result prediction map of the segmentation of condylar cortex and cancellous bone.

[0143] It should be noted that in this application, C represents the number of channels, D represents the depth of the image, H represents the height of the image, W represents the width of the image, and the subscripts of the letters represent the serial numbers used to distinguish the dimensions of different feature maps.

[0144] This application utilizes a condylar segmentation model to obtain the entire region of interest (ROI) of the condyle as input to a fine segmentation model, significantly reducing interference from other soft tissues in CBCT images. The diagnosis of TMJ-OA relies on observing changes in the condylar microstructure, with the condylar cortex and cancellous bone being the primary areas of change. Therefore, the segmentation results of the condyle are used as location information to guide the attention of the fine segmentation network, enhancing its learning of the regional features of the condylar cortex and cancellous bone. Simultaneously, texture and edge features from the image are incorporated to guide the segmentation network, enabling it to better distinguish the boundaries between the cortex and cancellous bone. This not only improves the segmentation accuracy of the condylar cortex and cancellous bone but also yields a visual representation of these areas.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for segmenting condylar CBCT images based on edge and texture features, characterized in that, The condylar CBCT image segmentation method based on edge and texture features includes: The original condylar CBCT image is input into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, the original condylar CBCT image is cropped to obtain the region of interest of the condyle. image; Area of ​​interest Images are input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps at different scales. ; Area of ​​interest The image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features. ; feature map The input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features. ; feature map Low-level features and advanced features The edge features are input into the edge feature extraction module of the cortical and cancellous segmentation model to obtain edge features. ; edge features and regional features at different scales The data is input into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of condylar cortex and cancellous bone; Among them, the region of interest Images are input into the feature encoding network of the cortical and cancellous segmentation model to obtain feature maps at different scales. ,include: Step 2.1: Select the region of interest. The image is input into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 2.2: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 2.4: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 2.5: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 2.6: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is The feature maps obtained from each convolutional block form feature maps of different scales. ; The feature map Low-level features and advanced features The edge features are input into the edge feature extraction module of the cortical and cancellous segmentation model to obtain edge features. ,include: Will Upsample it to make its size the same as the feature map. Consistent with each other and concatenated according to the number of channels, the data is input into the edge extraction module to obtain the edge feature map. Its size and feature map Consistent.

2. The condylar CBCT image segmentation method based on edge and texture features according to claim 1, characterized in that, The original condylar CBCT image is input into the condylar segmentation model to obtain the detection result of the entire condyle. Based on the detection result, the original condylar CBCT image is cropped to obtain the region of interest of the condyle. Images, including: Step 1.1: Input the original condylar CBCT image into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 1.2: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 1.3: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 1.4: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 1.5: Transfer the feature map Max pooling is performed, and the result is fed into a convolutional block with a kernel size of 3×3×3 to obtain the output feature map. Its dimension size is ; Step 1.6: Transfer the feature map Upsampling is performed, followed by comparison with the feature map. Channel-level concatenation is performed and input into a convolutional block with a kernel size of 3×3×3 to obtain the feature map. Its dimensional size is Consistent; Step 1.7: Transfer the feature map Upsampling is performed, followed by comparison with the feature map. Channel-level concatenation is performed and input into a convolutional block with a kernel size of 3×3×3 to obtain the feature map. Its dimensional size is Consistent; Step 1.8: Transfer the feature map Upsampling is performed, followed by comparison with the feature map. Channel-level concatenation is performed and input into a convolutional block with a kernel size of 3×3×3 to obtain the feature map. Its dimensional size is Consistent; Step 1.9: Transfer the feature map Upsampling is performed, followed by comparison with the feature map. Channel-level concatenation is performed and input into a convolutional block with a kernel size of 3×3×3 to obtain the feature map. Its dimensional size is Consistent; Step 1.10: Transfer the feature map The input is fed into a convolutional block with a kernel size of 1×1×1 to obtain the detection results. ; Step 1.11: Based on the test results The region of interest of the condyle was obtained by cropping the original CBCT image of the condyle. Its dimension size is .

3. The condylar CBCT image segmentation method based on edge and texture features according to claim 1, characterized in that, The region of interest The image is input into the texture feature extraction module of the cortical and cancellous segmentation model to extract texture features. ,include: Area of ​​interest The image is input to the texture feature extraction module, where features are extracted using the HOG method to obtain a texture feature map. Its dimension size is .

4. The condylar CBCT image segmentation method based on edge and texture features according to claim 3, characterized in that, The feature extraction using the HOG method includes: Step 3.1: Select the region of interest. The image is obtained along the Z-axis direction. Zhang Dawei The condylar images were sliced, and the pixel values ​​of each image were normalized to obtain the final slice set. ; Step 3.2: Slice each condylar image. Divide the window into multiple identical windows of a certain size, ensuring that adjacent windows do not overlap. Then, divide each window into multiple identical blocks of a certain size, ensuring that adjacent blocks do not overlap. Finally, divide each window into multiple identical cells of a certain size. Each adjacent Non-overlapping; Step 3.3, on cells Calculate the horizontal and vertical gradient values ​​for each pixel. and For pixels The formulas for calculating the gradient values ​​in the horizontal and vertical directions are as follows: ； ； in Represents the pixel after image normalization Pixel values; Thus, pixels can be obtained. gradient magnitude and gradient direction The calculation formula is as follows: ； ； Step 3.4, in each cell The internal statistical gradient direction histogram divides the plane from 0 to 180 degrees into... There are several intervals, and the interval between 181 and 360 degrees is... The histogram is constructed by summing the gradient values ​​corresponding to the angle range within each interval, using the vertical axis as the histogram and the horizontal axis as the gradient values ​​corresponding to the boundaries of the intervals. This allows us to obtain the gradient values ​​for each cell. HOG characteristics; Step 3.5: Remove the cells from each block. The obtained HOG features are normalized to obtain fast HOG features; Step 3.6: Combine the HOG features of the blocks within each window to obtain the final slice. HOG characteristics; Step 3.7: Set the slices Repeat steps 3.1 to 3.6 for all slices to obtain the HOG features corresponding to each slice; Step 3.8: Set the slices The HOG features corresponding to all slices are arranged in slice order to obtain the texture feature map. .

5. The condylar CBCT image segmentation method based on edge and texture features according to claim 1, characterized in that, The feature map The input is fed into the feature decoding network of the cortical and cancellous segmentation model to decode regional features at different scales and obtain regional features. ,include: Step 4.1: Transfer the feature map Feature map and texture features The input is fed into the texture feature attention module to obtain the texture fusion features. Its dimension size is ; Step 4.2: Transfer the feature map Feature map and texture features The input is fed into the texture feature attention module to obtain the texture fusion features. Its dimension size is ; Step 4.3: Transfer the feature map Feature map and texture features The input is fed into the texture feature attention module to obtain the texture fusion features. Its dimension size is ; Step 4.4: Transfer the feature map Feature map and texture features The input is fed into the texture feature attention module to obtain the texture fusion features. Its dimension size is Ultimately, regional features at different scales are obtained. .

6. The condylar CBCT image segmentation method based on edge and texture features according to claim 5, characterized in that, The texture feature attention module performs the following operations: Step 5.1: Transfer texture features The input is fed into a fully connected layer with a 1×1×1 kernel, the size of which is equal to the feature map. Consistent with the feature map The data is concatenated by channel and then upsampled to match the feature map. With the same size, compare it with the feature map The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion features are obtained. Its dimension size is ; Step 5.2: Transfer texture features The input is fed into a fully connected layer with a 1×1×1 kernel, the size of which is equal to the feature map. Consistent with the feature map The data is concatenated by channel and then upsampled to match the feature map. With the same size, compare it with the feature map The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion features are obtained. Its dimension size is ; Step 5.3: Transfer texture features The input is fed into a fully connected layer with a 1×1×1 kernel, the size of which is equal to the feature map. Consistent with the feature map The data is concatenated by channel and then upsampled to match the feature map. With the same size, compare it with the feature map The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion features are obtained. Its dimension size is ; Step 5.4: Transfer texture features The input is fed into a fully connected layer with a 1×1×1 kernel, the size of which is equal to the feature map. Consistent with the feature map The data is concatenated by channel and then upsampled to match the feature map. With the same size, compare it with the feature map The data is concatenated along channels and fed into a 3×3×3 convolutional block. After normalization and ReLU activation, the texture fusion features are obtained. Its dimension size is .

7. The condylar CBCT image segmentation method based on edge and texture features according to claim 5, characterized in that, The edge features and regional features at different scales The input is fed into the feature fusion module of the cortical and cancellous bone segmentation model to obtain a visualization prediction map of the segmentation results of the condylar cortex and cancellous bone, including: Step 6.1: Extract edge features Perform max pooling to make its size blend with the texture features. Consistent and blended with texture features The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain new fused features. Its dimension size is ; Step 6.2: Extract edge features Perform max pooling to make its size blend with the texture features. Consistent and blended with texture features The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain new fused features. Its dimension size is ; Step 6.3: Extract edge features Perform max pooling to make its size blend with the texture features. Consistent and blended with texture features The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain new fused features. Its dimension size is ; Step 6.4: Extract edge features Perform max pooling to make its size blend with the texture features. Consistent and blended with texture features The data is concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 3×3×3 to obtain new fused features. Its dimension size is ; Step 6.5: Merge features Fusion characteristics Fusion characteristics and fusion features The features are concatenated according to the number of channels and then fed into a convolutional block with a kernel size of 1×1×1 to obtain the final edge enhancement region features. These edge enhancement region features are then activated using the following formula to obtain the output feature map: ； in, Indicates performing an operation on x. The activation operation, where x represents the edge enhancement region feature.

Images