Model image throat segmentation method combining attention mechanism and multi-scale fusion
By introducing joint attention mechanism and multi-scale fusion module in UCtransNet network, the problems of high time complexity and low recognition accuracy in complex pore structures are solved, and more efficient and accurate throat segmentation results are achieved .
Patent Information
- Application Number
- CN202310506859.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional throat segmentation algorithms have problems such as high time complexity, undersegment or oversegment when dealing with complex pore structures, making it difficult to accurately identify throat targets.
Using a joint attention mechanism and multi-scale fusion model, feature extraction and segmentation are performed through the improved UCtransNet network, combining the CBAM attention mechanism, AFFN multi-scale feature fusion module and multi-head cross-attention mechanism to improve the throat segmentation performance.
It significantly improves the accuracy and computing efficiency of throat segmentation, reduces processing time, and can more accurately identify complex throat structures.
Smart Images

Figure BDA0004215820650000031 
Figure BDA0004215820650000032 
Figure HDA0004215820660000011
Abstract
Description
Technical Field
[0001] The present invention designs a model image laryngeal tract segmentation method combining a joint attention mechanism and multi-scale fusion, and relates to the fields of image processing and deep learning. Background Art
[0002] In underground reservoirs, pores are often divided into two parts: cavities and throats. A closed area usually surrounded by multiple rock particles is called a cavity, which can store fluids. The throat is a narrow part between rocks, which is usually small in area and can affect the seepage of fluids. The ability to correctly divide the position range of cavities and throats is the basis for the subsequent calculation of pore characteristic parameters (such as shape factor, throat width, equivalent circle radius, etc.) of the microscopic displacement model image. Since the traditional throat segmentation algorithm has a high time complexity in extracting pore networks and segmentation processing, the program processing time is slow. In addition, for some throats with complex pore structures, the traditional segmentation algorithm may not be able to identify the throat target, resulting in under-segmentation, or misidentify the throat, resulting in over-segmentation. The image segmentation method based on deep learning has a more powerful ability to extract semantic feature information, has better expressiveness when facing the task of segmenting complex throats, and has certain advantages in processing time compared with traditional algorithms. Summary of the invention
[0003] The present invention provides a method for segmenting the laryngeal tract of a model image by combining an attention mechanism and multi-scale fusion to solve the above segmentation problem. The present invention obtains more advanced and more detailed features by improving the UCtransNet network to enhance the segmentation performance of the model for the laryngeal tract.
[0004] The present invention achieves the above-mentioned purpose through the following technical solutions:
[0005] (1) During the training process, the image I after preprocessing is input, and features are extracted from the input image through maximum pooling to obtain a feature map A;
[0006] (2) Downsample the feature map A obtained in step (1) to obtain feature map B, and add the CBAM attention mechanism module in the downsampling process;
[0007] (3) The feature map B obtained in step (2) and the feature map A in step (1) are concatenated through a multi-scale feature fusion module, i.e., an AFFN module, to obtain a feature map C;
[0008] (4) Processing the feature map C obtained in step (3) through a multi-head cross attention mechanism module, namely, the Ctrans module;
[0009] (5) The image A input in step (1) is divided into blocks, an adaptive image overlap area is set, and the image I is divided into blocks to obtain a set of divided images I Set ;
[0010] (6) Processing each image in the image set obtained in step (5) in accordance with steps (1), (2), (3), and (4) in sequence to obtain a corresponding laryngeal segmentation result image;
[0011] (7) All the throat segmentation images obtained in step (6) are spliced together to obtain the final segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 It is a principle block diagram of the improved and enhanced UCtransNet network structure of the present invention.
[0013] Figure 2 It is a multi-scale fusion module AFFN module.
[0014] Figure 3 It is the image block processing process.
[0015] Figure 4 It is the throat segmentation result of the present invention. DETAILED DESCRIPTION
[0016] The present invention will be further described below in conjunction with the accompanying drawings:
[0017] Figure 1 The overall schematic diagram of the network model includes the following steps:
[0018] During the training process, the original image is subjected to the maximum pooling operation to obtain the feature map; then, the CBAM attention mechanism is used to guide the model to obtain more accurate detail features, including the channel attention mechanism and the spatial attention mechanism; then, the obtained feature map is put into the AFFN multi-scale fusion module as input data for the first fusion of features of different scales; secondly, the feature map processed by the AFFN module is used as the input of the Ctrans module for the second feature fusion; finally, the decoding constructs the final laryngeal segmentation result according to the output of the Ctrans module.
[0019] Figure 2It is a structural diagram of the multi-scale feature fusion module AFFN designed by the present invention. Among them, the L1 layer feature map in the feature map C is used as the first-level input of the Ctrans module. After the L1 layer feature map is subjected to the maximum pooling operation, the size of the feature map is reduced, and the number of feature maps is increased to obtain the L2 layer feature map; at this time, the feature map of the L1 layer is downsampled to obtain a new feature map f1. The size of the feature map of the L2 layer needs to be consistent during the downsampling process; the downsampling in this module is achieved through convolution, and the downsampling described in the subsequent modules is a convolution process; at this time, the feature maps L2 and f1 are spliced, and a 1×1 convolution operation is performed on them to reduce the number of channels. The processed feature map is f2, and the result is used as the second-level input of the Ctrans module; similarly, the maximum pooling operation is continued on the feature map of the L2 layer to obtain the feature map of the L3 layer; At this time, the feature map f2 is downsampled and kept to have the same size as the feature map of the L3 layer. The downsampled feature map f3 is concatenated with the feature map L3, and a 1×1 convolution is performed on the concatenated feature map to reduce the dimension, thereby obtaining the feature map f4 as the third-level input of the Ctrans module. Similarly, the maximum pooling operation is continued on the feature map of the L3 layer to obtain the feature map L4, and then the feature map f4 is downsampled to obtain the feature map f5, while ensuring that the size of f5 is consistent with that of the feature map L4. At this time, the feature maps L4 and f5 are concatenated to obtain a new feature map, and a 1×1 convolution operation is performed on the feature map to reduce the dimension to obtain the feature map f6 as the fourth-level input of the Ctrans module to obtain higher-level and more precise features.
[0020] Figure 3 The image block processing technology designed by the present invention comprises the following specific steps: first, scanning the pixel area at the right edge of the image, and if the area is a rock particle, it is considered that there is no throat target here; if the scanned area belongs to the pore part, it is considered that there is a throat target, and scanning is performed from the current position toward the left edge of the image. When the first rock particle, i.e., the first connected area, is scanned, the connected area of the entire rock particle is extracted, and the distance between the leftmost pixel position of the connected area and the right edge of the image is recorded; returning to the right edge position and continuing to scan toward the lower edge of the image, the connected area of the rock and the pore is continuously extracted, and the distance between the leftmost coordinate of the connected area and the right edge of the image is recorded; finally, the maximum value of the recorded distance is multiplied by 2, which is the width of the overlapping cropping.
[0021] Table 1 shows the comparison results of the improved and enhanced UCtransNet training network and the basic network, which shows that the improved network is better than the basic network. Table 2 shows the time taken by the network of the present invention and the current mainstream traditional laryngeal segmentation network to process model images of the same size. The results show that the network of the present invention is significantly better than the traditional algorithm in terms of computational efficiency.
[0022] Table 1 Comparison between the improved and enhanced UCtransNet training network and the basic network
[0023]
[0024] Table 2 Calculation time statistics
[0025]
Claims
1. A method for laryngeal segmentation based on a joint attention mechanism and multi-scale fusion model image, characterized in that The following steps are involved: (1) During the training process, the image I after preprocessing is input, and features are extracted from the input image through maximum pooling to obtain a feature map A; (2) Downsample the feature map A obtained in step (1) to obtain feature map B, and add the CBAM attention mechanism module in the downsampling process; (3) The feature map B obtained in step (2) is concatenated with the feature map A in step (1) through a multi-scale feature fusion module, i.e., an AFFN module, to obtain a feature map C; (4) Processing the feature map C obtained in step (3) through a multi-head cross attention mechanism module, namely, the Ctrans module; (5) The image A input in step (1) is divided into blocks, an adaptive image overlap area is set, and the image I is divided into blocks to obtain a set of divided images I Set ; (6) The image set I obtained in step (5) Set Each image in is processed in accordance with steps (1), (2), (3), and (4) in sequence to obtain the corresponding laryngeal segmentation result image; (7) All the throat segmentation images obtained in step (6) are spliced together to obtain the final segmentation result.
2. According to the method of claim 1, the multi-scale feature fusion operation is in step (3), and the specific operation is as follows: The L1 layer feature map in the feature map C is used as the first-level input of the Ctrans module. After the maximum pooling operation, the size of the L1 layer feature map is reduced and the number of feature maps is increased to obtain the L2 layer feature map. At this time, the feature map of the L1 layer is downsampled to obtain a new feature map f1. The size of the feature map of the L2 layer needs to be consistent with the size of the feature map of the L2 layer during the downsampling process. The downsampling in this module is achieved through convolution, and the downsampling described in the subsequent modules is a convolution process. At this time, the feature maps L2 and f1 are concatenated, and a 1×1 convolution operation is performed on them to reduce the number of channels. The processed feature map is f2, and the result is used as the second-level input of the Ctrans module. Similarly, the maximum pooling operation is continued on the feature map of the L2 layer to obtain L 3 layers of feature maps; at this time, the feature map f2 is downsampled and kept to have the same size as the L3 layer feature map, the downsampled feature map f3 is concatenated with the feature map L3, and the concatenated feature map is convolved with 1×1 to reduce the dimension to obtain the feature map f4, which is used as the third-level input of the Ctrans module; similarly, the L3 layer feature map is continued to be max-pooled to obtain the feature map L4, and then the feature map f4 is downsampled to obtain the feature map f5, while ensuring that the size of f5 is consistent with that of the feature map L4; at this time, the feature maps L4 and f5 are concatenated to obtain a new feature map, and a 1×1 convolution operation is performed on the feature map to reduce the dimension to obtain the feature map f6, which is used as the fourth-level input of the Ctrans module.
3. According to the method of claim 1, the specific operation of the block processing in step (5) is as follows: First, scan the pixel area on the right edge of image I. If the area is a rock particle, it is considered that there is no throat target here; if the scanned area belongs to the pore part, it is considered that there is a throat target. At this time, when the first rock particle, that is, the first connected area, is scanned from the current position to the image, the connected area of the entire rock particle is extracted, and the distance between the leftmost pixel position of the connected area and the right edge of the image is recorded; return to the right edge position and continue scanning to the lower edge of the image, continue to extract the connected area of the rock and the pore, and record the distance between the leftmost coordinate of the connected area and the right edge of the image; finally, multiply the maximum value of the recorded distance by 2, which is the width of the overlapping cropping.
Citation Information
Patent Citations
Skin disease image segmentation method and system based on joint attention convolutional neural network
CN115457021A
Method for identifying cultivated land area by using remote sensing vector
CN115497008A
