Adenoid recognition method and device based on image instance segmentation

Abstract

The invention discloses an adenoid body recognition method and device based on image instance segmentation, and the method comprises the steps: selecting a skull side position film image in a medical image database, and carrying out the marking of the skull side position film image to clearly know the position of a key structure; preprocessing each image to generate a training sample; and training an instance segmentation model by using the samples, wherein the model comprises a backbone network, a feature fusion module and a detection head. The backbone network is responsible for preliminary feature extraction, and the C2f module further performs deep processing on feature maps, generates multi-scale feature maps and fuses the multi-scale feature maps into a comprehensive feature map. And the feature fusion module adopts a path aggregation network to integrate all scale feature maps. And the detection head receives the fusion feature map, generates a candidate frame and performs classification prediction. And adjusting the candidate frame according to the prediction result to ensure accurate identification of the adenoid body, and outputting the adjusted candidate frame and the category label as a final identification result. The problem of how to improve the accuracy and efficiency of diagnosis by accurately segmenting and identifying adenoid bodies and related structures thereof is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for adenoids recognition based on image instance segmentation. Background Art

[0002] Adenoids are collections of lymphoid tissue located in the posterior airway of the nasopharynx. Adenoids hypertrophy is a common pathological enlargement of the nasopharyngeal tonsils in children and is associated with a variety of facial features and symptoms. The recognition of adenoids hypertrophy is particularly important in pediatrics.

[0003] In terms of the identification of adenoid hypertrophy, most existing technologies rely on professional X-rays or CT images and require the experience of specialists to judge. Although these methods are effective, they have certain limitations and inconveniences, such as equipment availability, radiation hazards, and discomfort during the examination. Although these methods can provide high-quality image data, radiation exposure is harmful to children, especially during multiple or frequent examinations.

[0004] Therefore, how to improve the accuracy and efficiency of diagnosis, reduce radiation hazards, and increase the popularity of diagnosis by accurately segmenting and identifying adenoids and their related structures has become a problem to be solved. Summary of the invention

[0005] In an embodiment of the present application, by providing an adenoids recognition method based on image instance segmentation, the problem of how to improve the accuracy and efficiency of diagnosis, reduce radiation hazards, and increase the popularity of diagnosis by accurately segmenting and identifying adenoids and their related structures is solved.

[0006] In a first aspect, an embodiment of the present application provides an adenoids recognition method based on image instance segmentation, the method comprising: selecting a lateral head radiograph image from a medical image database, and annotating the image to obtain a category label to clarify the location of key structures; wherein the key structures include adenoids and upper airway; preprocessing each selected lateral head radiograph image to obtain a training sample; using the training sample to train an instance segmentation model; wherein the instance segmentation model comprises a backbone network, a feature fusion module and a detection head; the backbone network is responsible for preliminary feature extraction, and the spatial size of the feature map is gradually reduced and the number of channels is increased through multiple processing stages; using C2f The module performs deep processing on the feature map, extracts features through residual connection and Bottleneck module, and generates multi-scale feature maps. The multi-scale feature maps are fused into comprehensive feature maps as the input of the feature fusion module; the feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated, and outputs the fused feature map; the detection head receives the fused feature map as input to generate candidate boxes and perform classification prediction; the position and size of the candidate boxes are adjusted according to the classification prediction results to ensure accurate adenoids recognition, and the adjusted candidate boxes and their corresponding category labels are output as the final adenoids recognition results.

[0007] In a possible implementation, the method further includes: introducing a semantic segmentation head, generating a segmentation map having the same size as the lateral skull radiograph image through upsampling and convolution operations; wherein each pixel on the segmentation map corresponds to a category label.

[0008] In a possible implementation, the preprocessing of each selected lateral skull image includes: randomly cropping the lateral skull image to obtain image blocks of different areas; randomly scaling the image blocks to generate images of different sizes; normalizing the randomly cropped and randomly scaled image blocks to a uniform size; and using a mirror flipping technique to generate a mirror version of the image.

[0009] In a possible implementation, the backbone network is responsible for preliminary feature extraction, and gradually reduces the spatial size of the feature map and increases the number of channels through multiple processing stages, including: the backbone network includes multiple processing stages, each processing stage includes multiple convolutional layers, batch normalization layers and activation functions; the backbone network performs preliminary feature extraction on the training samples, and each time a processing stage is passed, the spatial size of the feature map gradually decreases, and the number of channels gradually increases.

[0010] In a possible implementation, the C2f module is used to perform deep processing on the feature map, features are extracted through residual connection and Bottleneck module, and a multi-scale feature map is generated. The multi-scale feature map is fused into a comprehensive feature map as the input of the feature fusion module, including: the C2f module divides each input feature map into two parts, one part of the feature map is transmitted to the output end of the C2f module through the residual connection, and the other part of the feature map is subjected to feature extraction through two layers of Bottleneck modules; the feature map output by the Bottleneck module is spliced ​​with the feature map of the residual connection part in the channel dimension to obtain the spliced ​​feature map; the spliced ​​feature map is compressed and the features are fused through the convolution layer to generate the output feature map; the spatial pyramid pooling fast module is introduced, and the output feature map is downsampled by applying pooling kernels of different sizes in parallel to generate a multi-scale feature map; the multi-scale feature maps are fused to generate a comprehensive feature map.

[0011] In a possible implementation, the feature fusion module uses a path aggregation network to perform feature extraction steps, including: upsampling a high-level feature map to match the feature map of its previous layer in the spatial dimension; concatenating the upsampled high-level feature map with the feature map of its previous layer in the channel dimension to form a new feature map; the new feature map is passed through a C2f module to extract features and enhance the expressiveness of the features; the low-level feature map extracted by the C2f module is convoluted to reduce its size and then concatenated with the adjacent high-level feature map, and features are extracted again through the C2f module.

[0012] In one possible implementation, the detection head receives the fused feature map as input for generating candidate boxes and performing classification prediction, including: the detection head receives the fused feature map as input, and uses the input fused feature map to generate candidate boxes; extracts features through a convolutional layer and / or a fully connected layer, and uses the extracted features to perform classification prediction on the candidate boxes.

[0013] In a second aspect, an embodiment of the present application provides an adenoids recognition device based on image instance segmentation, the device comprising: a labeling module, used to select a lateral head image in a medical image database, and label it to obtain a category label to clarify the location of key structures; wherein the key structures include adenoids and upper airway; a preprocessing module, used to preprocess each selected lateral head image to obtain a training sample; a training module, used to train an instance segmentation model using the training samples; wherein the instance segmentation model comprises a backbone network, a feature fusion module and a detection head; the backbone network is responsible for preliminary feature extraction, and gradually reduces the spatial size of the feature map and increases the number of channels through multiple processing stages; The C2f module is used to perform deep processing on the feature map, and features are extracted through residual connection and Bottleneck module to generate multi-scale feature maps. The multi-scale feature maps are fused into comprehensive feature maps as the input of the feature fusion module; the feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated, and outputs the fused feature map; the detection head receives the fused feature map as input to generate candidate boxes and perform classification prediction; the recognition result acquisition module is used to adjust the position and size of the candidate box according to the classification prediction result to ensure accurate adenoids recognition, and outputs the adjusted candidate box and its corresponding category label as the final adenoids recognition result.

[0014] In a third aspect, an embodiment of the present application provides an adenoids recognition server based on image instance segmentation, comprising a memory and a processor; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions to implement the method described in the first aspect or any possible implementation method of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores executable instructions, and when a computer executes the executable instructions, the method described in the first aspect or any possible implementation method of the first aspect can be implemented.

[0016] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects:

[0017] The embodiment of the present application provides an adenoids recognition method based on image instance segmentation. By annotating and preprocessing the lateral head radiograph images in the medical image database, high-quality training samples are provided for the instance segmentation model, ensuring that the model can learn the precise features of the adenoids and their related structures. Using an instance segmentation model including a backbone network, a feature fusion module and a detection head, deep feature extraction and precise segmentation of the adenoids and their related structures are achieved. The detection head can generate candidate frames and perform classification predictions, further improving the accuracy of adenoids recognition. The C2f module is used to perform deep processing on the feature map, and multi-scale features are extracted through residual connections and Bottleneck modules, and fused into a comprehensive feature map. This multi-scale feature fusion strategy enables the model to capture adenoids features at different scales and improves the model's adaptability to complex scenes. The feature fusion module uses a path aggregation network to achieve effective integration of feature maps of different scales. This network structure can make full use of feature information of different scales and improve the model's recognition ability of adenoids and their related structures. According to the classification prediction results, the position and size of the candidate frame are adjusted to ensure accurate adenoids recognition. This flexible adjustment mechanism enables the model to adapt to adenoids of different sizes and shapes, improving the robustness of recognition. This application is not only suitable for the diagnosis of adenoids hypertrophy, but can also be used to evaluate the size, shape and position of adenoids, providing an important basis for clinical decision-making. It solves the problem of how to improve the accuracy and efficiency of diagnosis, reduce radiation hazards, and increase the popularity of diagnosis by accurately segmenting and identifying adenoids and their related structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of an adenoids recognition method based on image instance segmentation provided in an embodiment of the present application;

[0020] Figure 2 A specific flow chart for preprocessing each selected lateral head radiograph image provided in an embodiment of the present application;

[0021] Figure 3 A specific flow chart of using a C2f module to perform deep processing on a feature map, extracting features through a residual connection and a Bottleneck module, and generating a multi-scale feature map, which is fused into a comprehensive feature map as an input of a feature fusion module, provided in an embodiment of the present application;

[0022] Figure 4 A specific flow chart of the feature fusion module provided in the embodiment of the present application using a path aggregation network to perform feature extraction steps;

[0023] Figure 5 A schematic diagram of instance segmentation results after the instance segmentation model provided in an embodiment of the present application completes training;

[0024] Figure 6 A schematic diagram of the change in loss values ​​of the two major tasks of instance segmentation and classification during the instance segmentation model training process provided in the embodiment of the present application;

[0025] Figure 7 A histogram of instance segmentation evaluation indicators provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram showing the classification performance of an instance predicted by an instance segmentation model through a confusion matrix provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of the detection status of indicators in different categories provided in the embodiments of the present application;

[0028] Fig.10 A schematic diagram of an adenoids recognition device based on image instance segmentation provided in an embodiment of the present application;

[0029] Fig.11 A schematic diagram of an adenoids recognition server based on image instance segmentation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0032] The present application embodiment provides a method for identifying adenoids based on image instance segmentation. Figure 1As shown, the method includes steps S101 to S108. Among them, Figure 1 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of a method for adenoids recognition based on image instance segmentation. If the final result can be achieved, Figure 1 The steps shown may be performed in parallel or reversed.

[0033] S101: Select a lateral head radiograph image from a medical image database, and annotate it to obtain a category label to clarify the location of key structures, wherein the key structures include adenoids and upper airway.

[0034] Specifically, lateral cephalogram images in medical image databases usually come from hospitals, clinics or research institutions, and have been digitized for computer analysis and processing. Ensure that the selected lateral cephalogram images have sufficient resolution and clarity to accurately identify key structures. Select appropriate annotation tools, such as semi-automatic or fully automatic annotation software, and professional medical image annotation platforms. Clearly mark the locations of key structures such as adenoids and upper airway on the image. This usually involves drawing bounding boxes, contours, or using point markers on the image to indicate the boundaries of the structure. Assign a unique category label to each annotated key structure. For example, adenoids can be labeled "adenoids" and the upper airway can be labeled "upper airway".

[0035] S102: Preprocess each selected lateral skull image to obtain a training sample.

[0036] Figure 2 The specific flow chart of preprocessing each selected lateral head image provided in the embodiment of the present application is as follows: Figure 2 As shown, it includes steps S201 to S204.

[0037] S201: Randomly crop the lateral skull radiograph image to obtain image blocks in different regions.

[0038] Specifically, random cropping helps the instance segmentation model learn the joint structure features at different locations in the image and improve its ability to capture local information.

[0039] S202: Randomly scale the image blocks to generate images of different sizes.

[0040] Specifically, random scaling helps the instance segmentation model learn features at different scales, thereby enhancing adaptability to scale changes.

[0041] S203: Normalize the randomly cropped and randomly scaled image blocks to a uniform size.

[0042] Specifically, the present application normalizes the randomly cropped and randomly scaled image blocks to 640×640 pixels, which can ensure stable performance during instance segmentation model training and reduce the computational burden caused by image size differences.

[0043] S204: Generate a mirror version of the image using a mirror flipping technique.

[0044] Specifically, this step not only doubles the number of training samples, but also helps the instance segmentation model learn the symmetry and direction invariance features in the image, thereby improving its generalization ability.

[0045] S103: Use the training samples to train the instance segmentation model, wherein the instance segmentation model includes a backbone network, a feature fusion module and a detection head.

[0046] Specifically, the instance segmentation model used in this application is the YOLOv8 model.

[0047] S104: The backbone network is responsible for preliminary feature extraction, gradually reducing the spatial size of the feature map and increasing the number of channels through multiple processing stages.

[0048] Specifically, in order to balance training efficiency and memory usage, the batch size is set to 16. This means that each iteration will process 16 images of training samples at the same time. The learning rate is a key hyperparameter that affects the speed and effect of model training. The initial learning rate is set to 0.01, which will be used to adjust the update amplitude of model parameters during training. In order to prevent the model from overfitting, the L2 regularization term is introduced and its weight is set to 0.005. L2 regularization helps the model learn more generalized features by penalizing the size of model parameters. SGD (stochastic gradient descent) is selected as the optimizer of the model, and the number of iterations is set to 100. The SGD optimizer calculates the gradient and updates the model parameters in the direction of gradient descent. It is a commonly used optimization method in deep learning training. In order to further improve the training stability and effect of the model, a hot start strategy is adopted to gradually adjust the learning rate and the momentum parameter of SGD. In the early stage of training, start with a small learning rate and gradually increase it to the set learning rate (0.01) within the first 3 epochs. This gradual increase in learning rate helps the model adapt to the training data more smoothly in the initial stage and avoids training instability caused by too large a learning rate. Similarly, a hot start strategy is also used to gradually increase the momentum parameter of SGD. The momentum parameter is used to accelerate the convergence of SGD in the relevant direction and suppress oscillations. Starting from the initial 0.8, it gradually increases to 0.937. This gradual increase in momentum parameter helps the model update parameters more smoothly in the early stage of training and avoids training instability caused by large parameter updates.

[0049] The backbone network is responsible for preliminary feature extraction, and gradually reduces the spatial size of the feature map and increases the number of channels through multiple processing stages, including: The backbone network includes multiple processing stages, each of which includes multiple convolutional layers, batch normalization layers, and activation functions. The backbone network performs preliminary feature extraction on the training samples, and each time through a processing stage, the spatial size of the feature map gradually decreases and the number of channels gradually increases.

[0050] Specifically, the convolution layer is the core part of the backbone network, which captures the local features of the input data through convolution operations. Each convolution layer contains multiple convolution kernels, which slide on the input data and calculate the corresponding feature maps. As the convolution layer goes deeper, the model can capture more and more complex feature information. The batch normalization layer is used to normalize the output of the convolution layer to ensure that the distribution of the data is more stable. This step helps to speed up the training process of the model and improve the generalization ability of the model. The activation function is used to introduce nonlinear factors so that the model can learn more complex feature representations. Common activation functions include ReLU, Sigmoid, etc. In the backbone network, the activation function is selected to enhance the nonlinear expression ability of the model.

[0051] When a training sample is input to the backbone network, it goes through multiple processing stages in sequence. In each processing stage, the spatial size of the feature map will gradually decrease, while the number of channels will gradually increase. This is because the convolution layer usually uses convolution operations with a step size greater than 1 or pooling operations to reduce the spatial resolution of the feature map, thereby extracting more abstract feature representations. At the same time, as the convolution layer goes deeper, the number of channels in the feature map will gradually increase to accommodate more feature information. Through this series of processing stages, the backbone network can gradually extract deep feature information from the training samples. This feature information will be further utilized by the subsequent feature fusion module and detection head to achieve accurate object detection and segmentation.

[0052] In this application, there are 5 processing stages in the backbone network, and the P1 stage is the first processing stage. It includes two convolutional layers and one pooling layer. Both convolutional layers use 3×3 filters, marked as Conv (k=3). After the two convolutional layers, there is a pooling layer, which uses pooling operations (such as maximum pooling or average pooling) to reduce the spatial size of the feature map. The size of the output image in the P1 stage is That is, the size of the output image in the P1 stage is 320 pixels in height and width, and the number of channels is 64. The P2 stage is the second processing stage, and its structure is similar to the P1 stage, also containing two convolutional layers and one pooling layer, but using the C2f module. The size of the output image in the P2 stage is The output image is smaller than that of the P1 stage because it has undergone more convolution and pooling operations, that is, the spatial size is halved and the number of channels is doubled. The P3 stage is the third processing stage, and its structure is consistent with the P2 stage. The size of the output image of the P3 stage is The P4 stage is the fourth processing stage. Its structure is consistent with the P3 stage. The size of the output image in the P4 stage is The P5 stage is the last processing stage of the backbone network. It also contains two convolutional layers and one pooling layer, and uses the C2f module. The size of the output image of the P5 stage is r is an additional parameter used to further adjust the number of channels. SPPF is a fast module for spatial pyramid pooling, which accepts the feature map of the previous processing stage as input.

[0053] S105: Use the C2f module to perform deep processing on the feature map, extract features through residual connection and Bottleneck module, and generate multi-scale feature maps. The multi-scale feature maps are fused into a comprehensive feature map as the input of the feature fusion module.

[0054] Figure 3 The specific flow chart of using the C2f module to perform deep processing on the feature map provided in the embodiment of the present application, extracting features through residual connection and Bottleneck module, and generating multi-scale feature maps, which are fused into a comprehensive feature map as the input of the feature fusion module, is as follows Figure 3 As shown, it includes steps S301 to S305.

[0055] S301: The C2f module divides each input feature map into two parts. One part of the feature map is transmitted to the output end of the C2f module through a residual connection, and the other part of the feature map is subjected to feature extraction through a two-layer Bottleneck module.

[0056] Specifically, the C2f module receives feature maps from the previous processing stage, which contain the basic information and preliminary feature representation of the image. The C2f module first divides each input feature map into two parts in the channel dimension. This segmentation operation helps the model capture more contextual information and thus identify the target more accurately. The two parts of the feature map after segmentation are recorded as A and B respectively. Feature map A is directly passed to the output of the C2f module through residual connection. Residual connection is an effective technique that can help the model maintain the integrity of information during training and avoid the problem of gradient disappearance or explosion. The feature map A of the residual connection part will be retained in subsequent processing and spliced ​​with the feature map processed by the Bottleneck module. Feature map B enters the two-layer Bottleneck module for further feature extraction. The Bottleneck module is a convolutional neural network module with a special structure, which contains two 3×3 convolutional layers and a Batch Normalization (BN) layer. The main function of the BN layer is to accelerate the training process, improve the generalization ability of the model, and reduce internal covariate shift. By normalizing the distribution of each small batch of data, the BN layer helps stabilize the gradient during the learning process, thereby optimizing the convergence of the model. The BN layer is followed by the nonlinear activation function SiLU (Sigmoid LinearUnit). The SiLU function introduces nonlinear characteristics to the model, enabling it to learn complex nonlinear relationships. In the first layer of Bottleneck, the feature map B is first transformed by the first convolutional layer (cv1), and the output feature map is divided into two parts. These two parts are processed by different convolutional layers and then merged. The merged feature map is then processed by the second convolutional layer (cv2), and the final output is the result of the first layer of Bottleneck. The second layer of Bottleneck takes the output of the first layer of Bottleneck as input and repeats the above process to extract deeper features.

[0057] S302: The feature map output by the Bottleneck module is concatenated with the feature map of the residual connection part in the channel dimension to obtain a concatenated feature map.

[0058] Specifically, the output feature map of the Bottleneck module has undergone deep feature extraction and transformation, and contains rich high-level feature information. The feature map of the residual connection part maintains the basic information of the input feature map, which helps prevent the network from experiencing gradient vanishing or gradient exploding problems during training. The splicing operation is performed on the channel dimension, which means that the width and height of the two feature maps need to be consistent, while the number of channels can be different. When splicing, the number of channels of the two feature maps is added together to obtain a new feature map with more channels, namely the spliced ​​feature map.

[0059] S303: compressing the number of channels and fusing the features of the concatenated feature map through a convolutional layer, thereby generating an output feature map.

[0060] Specifically, the Bottleneck module compresses the number of channels and fuses features of the concatenated feature maps through a 1×1 convolution layer. The 1×1 convolution layer adjusts the dimension of the output feature map by changing the number of channels of the feature map, while achieving fusion between different features. This design not only maintains the compactness of the feature map, but also enhances the expressiveness of the features.

[0061] S304: Introduce a spatial pyramid pooling fast module, apply pooling kernels of different sizes in parallel to downsample the output feature map, and generate a multi-scale feature map.

[0062] Specifically, after the P5 processing stage, the model further enhances its feature extraction capability by introducing the Spatial Pyramid Pooling Fast (SPPF). The introduction of this module aims to capture multi-scale contextual information, thereby improving the performance of the model when dealing with complex scenes and objects of different sizes. The Spatial Pyramid Pooling Fast module can extract features of adenoids at different scales by applying pooling kernels of different sizes in parallel. This multi-scale feature extraction method helps to capture the detailed information of adenoids at different resolutions, thereby improving the accuracy of recognition. In addition to extracting multi-scale features, the SPPF module can also fuse these features to generate a feature map that integrates information at multiple scales, namely a multi-scale feature map. This fusion process helps the model better understand the relationship between adenoids and their surrounding tissues, thereby more accurately identifying adenoids. Specifically, it uses three sizes of pooling kernels: 3×3, 5×5, and 7×7. These pooling kernels perform spatial division and pooling operations on the feature map in parallel, thereby generating feature maps with receptive fields of different scales. Each pooling kernel is responsible for extracting contextual information of a specific scale in the feature map. For example, a 3×3 pooling kernel can capture local features in a smaller range, while a 5×5 and 7×7 pooling kernel can capture contextual information in a larger range. Compared with the traditional spatial pyramid pooling (SPP) method, the fast module of spatial pyramid pooling has higher computational efficiency while maintaining the same performance. This helps to reduce the inference time of the model and improve real-time performance, which is particularly important in medical image analysis scenarios that require fast response.

[0063] S305: Fusing the multi-scale feature maps to generate a comprehensive feature map.

[0064] Specifically, in the process of adenoids recognition, since the size, shape and position of adenoids vary significantly in different individuals and images, a single-scale feature map is often difficult to fully capture the features of adenoids. Multi-scale feature maps can more comprehensively reflect the details and contextual information of adenoids by extracting features at different scales. Methods for fusing multi-scale feature maps include splicing, weighted summation, etc. Feature maps of different scales can be spliced ​​in the channel dimension to form a comprehensive feature map containing richer information. This method can retain the independence of feature maps of different scales, which helps the model to better utilize this information in subsequent processing. It is also possible to assign different weights to feature maps of different scales according to their importance and perform weighted summation. This method can more flexibly control the contribution of feature maps of different scales to the final comprehensive feature map, which helps the model make more accurate judgments in complex scenarios. The generated comprehensive feature map integrates the information of multi-scale feature maps and has stronger representation ability and robustness. In the scenario of adenoids recognition, the comprehensive feature map can more accurately reflect the characteristics of adenoids, which helps the model achieve better performance in subsequent classification, positioning or segmentation tasks.

[0065] S106: The feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated, and outputs a fused feature map.

[0066] Figure 4 The feature fusion module provided in the embodiment of the present application uses a path aggregation network to perform a specific flow chart of the feature extraction step, such as Figure 4 As shown, it includes steps S401 to S404.

[0067] S401: The high-level feature map is upsampled to match the feature map of the previous layer in the spatial dimension.

[0068] S402: Concatenate the upsampled high-level feature map with the previous layer feature map in the channel dimension to form a new feature map.

[0069] S403: The new feature map passes through the C2f module to extract features and enhance the expressiveness of the features.

[0070] S404: The low-level feature map extracted by the C2f module is concatenated with the adjacent high-level feature map after being reduced in size by a convolution operation, and features are extracted again by the C2f module.

[0071] Specifically, the feature fusion module of YOLOv8 plays a crucial role in the model. It uses the path aggregation network (PAN) technology to significantly improve the detection ability and robustness of the model by integrating feature maps from different scales. In the YOLOv8 model, the feature maps output from the backbone network (P3, P4 and P5 stages) are fed into the feature fusion module. These feature maps represent information at different scales, where the P3 stage contains more detailed information, while the P5 stage contains more global and abstract information. First, the P5 stage feature map is upsampled to double its size, but the number of channels remains unchanged to ensure that it can match the P4 stage feature map in the spatial dimension. Subsequently, the upsampled P5 stage feature map is concatenated with the P4 stage feature map in the channel dimension to form a new feature map. This new feature map is then processed by the C2f module to further extract features and enhance the expressiveness of the features. Next, the processed P4 stage feature map is also upsampled to match its size with the P3 stage feature map. Then, the upsampled P4 feature map is concatenated with the P3 stage feature map in the channel dimension and processed by the C2f module. This process ensures that the model can capture detailed information and global information from low-level to high-level, thereby improving the model's detection ability for objects of different scales. In order to further enhance feature fusion, the downsampling and concatenation operations of the path aggregation network are adopted. The P3 stage feature map extracted by the C2f module undergoes a convolution operation, and its size is reduced by half, and then it is concatenated with the P4 stage feature map extracted by the C2f module. This new feature map is processed again by the C2f module to retain the detailed information of the low-level features and combine it with the semantic information of the high-level features. Similarly, the processed P4 stage feature map also undergoes a convolution operation, and its size is further reduced and concatenated with the P5 stage feature map. This new feature map is processed again by the C2f module, thereby further enhancing the expressiveness of the features.

[0072] S107: The detection head receives the fused feature map as input to generate candidate boxes and perform classification prediction.

[0073] The detection head receives the fused feature map as input, and is used to generate candidate boxes and perform classification prediction, including: the detection head receives the fused feature map as input, and uses the input fused feature map to generate candidate boxes. Features are extracted through convolutional layers and / or fully connected layers, and the extracted features are used to perform classification prediction on the candidate boxes.

[0074] Specifically, the primary task of the detection head is to receive the output from the feature fusion module, i.e., the fused feature maps. These feature maps contain rich information extracted from multiple scales, which are crucial for subsequent object detection. Using the input fused feature maps, the detection head generates a series of candidate boxes. These candidate boxes are preset rectangular boxes with different sizes and aspect ratios, which cover various positions of the input image. The method for generating candidate boxes can be an anchor-based method (such as RPN in Faster R-CNN) or a region proposal network (RPN)-based method. In order to make classification predictions for each candidate box, the detection head needs to extract features from the feature map corresponding to the candidate box through convolutional layers and / or fully connected layers. These features are usually representations of local features of the region where the candidate box is located, which can reflect the category information of whether the candidate box contains the object. The extracted features are then fed into a classifier (usually one or more fully connected layers or convolutional layers followed by a softmax layer or sigmoid layer) for classification prediction. The classifier outputs a probability distribution for each candidate box, indicating the probability that the candidate box contains each target category.

[0075] S108: adjusting the position and size of the candidate box according to the classification prediction result to ensure accurate adenoids recognition, and outputting the adjusted candidate box and its corresponding category label as the final adenoids recognition result.

[0076] Specifically, the detection head has generated a series of candidate boxes based on the fused feature map, and has classified and predicted each candidate box, and obtained the probability that each candidate box contains adenoids. According to the classification prediction results, those candidate boxes with a high probability of containing adenoids are screened out. These candidate boxes will be used as the objects of subsequent adjustment. For each candidate box that needs to be adjusted, the adjustment parameters of its position and size are calculated. These parameters may include translation and scaling factors. Among them, the translation is used to adjust the center position of the candidate box, and the scaling factor is used to adjust the width and height of the candidate box. The calculation of the adjustment parameters is usually based on the classification prediction results and the position relationship between the candidate box and the real glandular sample. For example, a regression algorithm can be used to fit the position deviation between the candidate box and the real glandular sample to obtain the adjustment parameters. The calculated adjustment parameters are applied to the candidate boxes to obtain the adjusted candidate boxes. These adjusted candidate boxes will locate the adenoids more accurately. For each adjusted candidate box, its position, size and corresponding category label are output. These adjusted candidate boxes and their category labels will be used as the final adenoids recognition results, which can be used for subsequent decision-making, visualization or further analysis.

[0077] It also includes: introducing a semantic segmentation head, generating a segmentation map with the same size as the lateral head radiograph image through upsampling and convolution operations, wherein each pixel on the segmentation map corresponds to a category label.

[0078] Specifically, the semantic segmentation head is a neural network module for generating segmentation maps. The feature maps received by the semantic segmentation head are usually extracted from the feature extraction module, which contain rich image information and features of different scales. In order to make the generated segmentation map consistent with the size of the original lateral head radiograph image, the semantic segmentation head first performs an upsampling operation on the feature map. The upsampling operation can be achieved by methods such as transposed convolution (deconvolution) or bilinear interpolation, aiming to enlarge the size of the feature map to the same size as the original image. After upsampling, the semantic segmentation head applies a series of convolution operations to extract finer features. These convolution operations usually include multiple convolution layers, each of which uses a different convolution kernel to extract features. Through multiple convolution operations, the semantic segmentation head can gradually capture more complex image features and generate more accurate segmentation maps. After multiple convolution operations, the output of the semantic segmentation head is a segmentation map consistent with the size of the original lateral head radiograph image. Each pixel on the segmentation map corresponds to a category label, indicating which category the pixel belongs to, such as adenoids or background. By observing the segmentation map, you can intuitively see the position and shape of the adenoids in the lateral cranial radiograph. This helps to more accurately identify and locate the adenoids. The segmentation map can also serve as a basis for auxiliary decision-making. For example, in the diagnosis of adenoid hypertrophy, the area or volume of the adenoids on the segmentation map can be used to judge the degree of adenoid hypertrophy, so as to formulate a more appropriate treatment plan.

[0079] Figure 5 This is a schematic diagram of the instance segmentation results after the instance segmentation model provided in the embodiment of the present application is trained. Figure 5 As shown in the figure, the two main target instances are accurately segmented and highlighted. These green rectangular boxes accurately mark the location of the adenoids. As the key target of this segmentation task, the boundaries of the adenoids are accurately captured and outlined by the model. The close fit of the green box intuitively reflects the high-precision segmentation capability of the model. At the same time, the red rectangular box accurately identifies the location of the upper airway. As another segmentation target, the upper airway is also segmented and highlighted with high confidence by the model. The precise location of the red box further verifies the excellent performance of the model in recognizing complex anatomical structures. Through this picture, you can intuitively feel the excellent performance of the model in instance segmentation tasks. Whether it is adenoids or upper airway, the model can achieve accurate segmentation, providing strong support for subsequent medical diagnosis, treatment planning, etc.

[0080] Figure 6 This is a schematic diagram of the change in loss values ​​of the two major tasks of instance segmentation and classification during the instance segmentation model training process provided in the embodiment of the present application. Figure 6As shown in the figure, the horizontal axis represents the time process (in seconds), while the vertical axis indicates the size of various loss values. Two sets of key data changes can be observed: one set is the training classification loss and the validation classification loss, and the other set is the training segmentation loss and the validation segmentation loss. In the first 10 epochs, the performance of the model has begun to show its edge, showing a very excellent starting performance. Specifically, both the training classification loss and the validation classification loss have dropped rapidly, and the gap between the two has remained at a low level, showing that the model not only performs well in learning efficiency, but also has a good initial state in generalization ability. Similarly, the training segmentation loss and the validation segmentation loss also show a similar downward trend, further confirming the excellent performance of the model in the instance segmentation task. As the training continues to deepen, in order to further improve the generalization ability of the model, a strategy of gradually reducing the learning rate is adopted for fine-tuning. During this process, the loss values ​​continue to decline steadily, but the decline gradually slows down, reflecting that the model is gradually approaching its performance ceiling. After multiple iterations of training, the model finally converged and reached its optimal performance state. At this point, both the training classification loss, validation classification loss, training segmentation loss, and validation segmentation loss have dropped to extremely low levels and remain stable. This achievement fully demonstrates the model's excellent performance in instance segmentation and classification tasks, as well as the effectiveness of improving the model's generalization ability by reasonably adjusting the learning rate strategy.

[0081] Figure 7 This is a histogram of instance segmentation evaluation indicators provided in the embodiment of the present application. Figure 7 As shown in the figure, in the instance segmentation task, the performance of the model on the sample set was comprehensively evaluated, focusing on key indicators such as precision, recall, and the area under the mean precision-recall curve (mAP). The precision of the model reached more than 95%, indicating that the proportion of instances predicted as positive samples by the model is very high. The recall rate is as high as more than 98%, which means that the model successfully identified the vast majority of true positive sample instances. When the IOU (intersection over union) threshold is set to 0.5, the mAP50 of the model reaches 99%. This extremely high score shows that under relatively loose matching criteria, the model performs well in both precision and recall, and can accurately and comprehensively identify target instances. We further studied the changes in mAP values ​​when the IOU threshold changes from 0.5 to 0.9, and calculated the mean of mAP50-95. mAP50-95 represents the average of the mAP values ​​of the model at various thresholds when the IOU threshold ranges from 0.5 to 0.95. The results show that as the IOU threshold increases, the mAP value gradually decreases. This is because a higher IOU threshold requires the model to more strictly match the standard area when predicting, thereby increasing the requirements for positioning accuracy. Therefore, in practical applications, it is necessary to set an appropriate IOU threshold according to specific needs to balance the performance requirements of the model.

[0082] Figure 8 The schematic diagram of the present application embodiment shows the classification performance of the instance segmentation model predicted by the confusion matrix. Figure 8 As shown in the figure, the horizontal axis of the confusion matrix represents the manually annotated labels (true labels), and the vertical axis represents the output predicted by the model (predicted labels). Each element in the matrix represents the number of true labels of a certain class predicted by the model as a certain class of predicted labels. The main diagonal elements: represent the number of correctly detected samples (TP), that is, the model correctly classifies the samples as their true labels. The lower left triangular area belongs to the number of samples missed by the model (FN), also known as false negatives. The true labels of these samples exist, but the model fails to detect them correctly, causing them to be misclassified into other categories or not recognized. The upper right triangular area belongs to the number of samples misdetected by the model (FP), also known as false positives. The true labels of these samples do not exist in the current category, but the model mistakenly classifies them as the current category. In a specific task, the background category means that the model mistakenly detects the background as the labeled class (i.e., false positive, belonging to FP) or fails to detect the labeled class (i.e., false negative, belonging to FN).

[0083] Through the confusion matrix, we can calculate the accuracy (Accuracy), precision (Precision), recall (Recall), F1 value (F1 Score) and specificity (FPR) of the instance segmentation model. Among them, TN means that the model correctly classifies the unlabeled samples as unlabeled. The accuracy measures the overall classification performance of the model on all samples. Precision measures the proportion of instances that the model predicts to be positive that are actually positive. Recall measures the proportion of true positive instances that the model successfully identified out of all true positive instances. Specificity measures the proportion of unlabeled class examples that the model correctly classifies as unlabeled.

[0084] Fig. 9 This is a schematic diagram of the detection of indicators in different categories (adenoids and upper airway) provided in the embodiment of the present application. Each category of indicators is intuitively presented through two bar graphs to show the performance differences of different categories. Fig. 9As shown in the figure, precision reflects the proportion of instances predicted as positive by the model that are truly positive. As can be seen from the figure, the precision of adenoids is higher than that of the upper airway, which means that the model is more accurate in predicting adenoids. Recall reflects the proportion of all instances that are truly positive that are correctly predicted as positive by the model. As shown in the figure, the recall of adenoids is also relatively high, indicating that the model can cover all true adenoid instances well when detecting adenoids. mAP50 is one of the important indicators for evaluating the performance of target detection models. It considers the balance between precision and recall and is calculated when the IOU threshold is 0.5. As can be seen from the figure, the mAP50 value of adenoids is higher than that of the upper airway, which further confirms the superiority of the model in detecting adenoids. However, when the IOU threshold range is expanded to 0.5 to 0.9 (i.e., mAP50-90), the performance of the upper airway exceeds that of adenoids. This shows that under more stringent judgment criteria, the detection effect of the upper airway is better. This may be because the morphology of the upper airway is relatively stable, while the adenoids show greater morphological variation due to individual differences, which makes the model face greater challenges in detecting adenoids. The morphology of adenoids varies greatly in medical imaging, which may make it difficult for the model to learn stable feature representations during training. In contrast, the morphology of the upper airway varies less, and it is easier for the model to capture its stable features.

[0085] The embodiment of the present application also provides an adenoids recognition device 1000 based on image instance segmentation, such as Fig.10 As shown, the device includes: a labeling module 1001, a preprocessing module 1002, a training module 1003 and a recognition result obtaining module 1004.

[0086] The labeling module 1001 is used to select lateral head radiographs in the medical image database and label them to obtain category labels to clarify the locations of key structures, including adenoids and upper airway.

[0087] The preprocessing module 1002 is used to preprocess each selected lateral skull image to obtain a training sample.

[0088] The training module 1003 is used to train the instance segmentation model using training samples. The instance segmentation model includes a backbone network, a feature fusion module and a detection head. The backbone network is responsible for preliminary feature extraction, and gradually reduces the spatial size of the feature map and increases the number of channels through multiple processing stages. The C2f module is used to perform deep processing on the feature map, extract features through residual connections and Bottleneck modules, and generate multi-scale feature maps. The multi-scale feature maps are fused into a comprehensive feature map as the input of the feature fusion module. The feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated and outputs the fused feature map. The detection head receives the fused feature map as input to generate candidate boxes and perform classification predictions.

[0089] The recognition result obtaining module 1004 is used to adjust the position and size of the candidate box according to the classification prediction result to ensure accurate adenoids recognition, and output the adjusted candidate box and its corresponding category label as the final adenoids recognition result.

[0090] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0091] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0092] The methods, devices or modules described in this application can be implemented in a computer-readable program code mode. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor, a logic gate, a switch, an application-specific integrated circuit (English: Application Specific Integrated Circuit, referred to as: ASIC), a programmable logic controller and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a pure computer-readable program code mode, the controller can be completely implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0093] like Fig.11 As shown, an embodiment of the present application also provides an adenoids recognition server based on image instance segmentation, including a memory 1101 and a processor 1102; the memory 1101 is used to store computer executable instructions; the processor 1102 is used to execute computer executable instructions to implement an adenoids recognition method based on image instance segmentation described above in an embodiment of the present application.

[0094] An embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When a computer executes the executable instructions, it can implement the adenoids recognition method based on image instance segmentation described above in the embodiment of the present application.

[0095] It can be known from the description of the above implementation mode that a person skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiment of the present application.

[0096] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations.

[0097] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A method for adenoids recognition based on image instance segmentation, characterized in that: include: Select lateral head radiographs from the medical imaging database and annotate them to obtain category labels to clarify the locations of key structures; the key structures include adenoids and upper airway; Preprocess each selected lateral head radiograph image to obtain a training sample; Using training samples to train an instance segmentation model; wherein the instance segmentation model includes a backbone network, a feature fusion module and a detection head; The backbone network is responsible for preliminary feature extraction, gradually reducing the spatial size of the feature map and increasing the number of channels through multiple processing stages; The C2f module is used to perform deep processing on the feature map, and features are extracted through residual connection and Bottleneck module to generate multi-scale feature maps. The multi-scale feature maps are fused into comprehensive feature maps as the input of the feature fusion module. The feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated and outputs the fused feature maps; The detection head receives the fused feature map as input to generate candidate boxes and perform classification prediction; The position and size of the candidate box are adjusted according to the classification prediction results to ensure accurate adenoids recognition, and the adjusted candidate box and its corresponding category label are output as the final adenoids recognition result.

2. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: Also includes: A semantic segmentation head is introduced to generate a segmentation map with the same size as the lateral skull radiograph through upsampling and convolution operations; each pixel on the segmentation map corresponds to a category label.

3. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: The preprocessing of each selected lateral head image comprises: Randomly crop the lateral skull radiographs to obtain image blocks in different areas; Randomly scale image patches to generate images of different sizes; Normalize the randomly cropped and randomly scaled image patches to a uniform size; Generates a mirrored version of an image using the mirror flip technique.

4. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: The backbone network is responsible for preliminary feature extraction, gradually reducing the spatial size of the feature map and increasing the number of channels through multiple processing stages, including: The backbone network consists of multiple processing stages, each of which includes multiple convolutional layers, batch normalization layers, and activation functions; The backbone network is used to perform preliminary feature extraction on the training samples. With each processing stage, the spatial size of the feature map gradually decreases and the number of channels gradually increases.

5. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: The C2f module is used to perform deep processing on the feature map, extract features through residual connection and Bottleneck module, and generate multi-scale feature maps. The multi-scale feature maps are fused into a comprehensive feature map as the input of the feature fusion module, including: The C2f module divides each input feature map into two parts. One part of the feature map is transmitted to the output end of the C2f module through the residual connection, and the other part of the feature map is extracted through two layers of Bottleneck modules. The feature map output by the Bottleneck module is concatenated with the feature map of the residual connection part in the channel dimension to obtain the concatenated feature map. The concatenated feature map is compressed and features are fused through the convolution layer to generate the output feature map. A fast spatial pyramid pooling module is introduced to apply pooling kernels of different sizes in parallel to downsample the output feature maps and generate multi-scale feature maps. The multi-scale feature maps are fused to generate a comprehensive feature map.

6. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: The feature fusion module uses a path aggregation network to perform feature extraction steps, including: The feature maps of the higher layers are upsampled to match the feature maps of the previous layer in spatial dimensions; The upsampled high-level feature map is concatenated with the previous layer feature map in the channel dimension to form a new feature map; The new feature map is passed through the C2f module to extract features and enhance the expressiveness of features; The low-level feature map extracted by the C2f module is reduced in size through a convolution operation and then concatenated with the adjacent high-level feature map, and features are extracted again through the C2f module.

7. The adenoids recognition method based on image instance segmentation according to claim 1, characterized in that: The detection head receives the fused feature map as input to generate candidate boxes and perform classification prediction, including: The detection head receives the fused feature map as input, and generates a candidate frame using the input fused feature map; Features are extracted through convolutional layers and / or fully connected layers, and the extracted features are used to make classification predictions for candidate boxes.

8. An adenoids recognition device based on image instance segmentation, characterized in that: include: The annotation module is used to select lateral head radiographs from the medical image database and annotate them to obtain category labels to clarify the locations of key structures; wherein the key structures include adenoids and upper airway; A preprocessing module, used for preprocessing each selected lateral head radiograph image to obtain a training sample; A training module is used to train an instance segmentation model using training samples; wherein the instance segmentation model includes a backbone network, a feature fusion module and a detection head; the backbone network is responsible for preliminary feature extraction, and gradually reduces the spatial size of the feature map and increases the number of channels through multiple processing stages; the feature map is deeply processed using a C2f module, features are extracted through residual connections and a Bottleneck module, and a multi-scale feature map is generated, and the multi-scale feature map is fused into a comprehensive feature map as an input of the feature fusion module; the feature fusion module uses a path aggregation network to perform feature extraction steps until feature maps of all scales are integrated, and outputs the fused feature map; the detection head receives the fused feature map as input, and is used to generate candidate boxes and perform classification prediction; The recognition result obtaining module is used to adjust the position and size of the candidate box according to the classification prediction result to ensure accurate adenoids recognition, and output the adjusted candidate box and its corresponding category label as the final adenoids recognition result.

9. An adenoids recognition server based on image instance segmentation, characterized in that: including memory and processor; The memory is used to store computer executable instructions; The processor is configured to execute the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores executable instructions, and when a computer executes the executable instructions, the method according to any one of claims 1 to 7 can be implemented.

Images