Adenoids detection and segmentation method and system for home endoscope video

Abstract

The present invention relates to a method and system for detecting and segmenting adenoids in a home endoscope video, which belongs to the field of image segmentation technology, and includes: intercepting adenoids images from a home endoscope video, screening out images containing adenoids, and enhancing them; marking the adenoids region and the posterior nasal region of the enhanced adenoids image to form training samples; inputting the training samples into a deep segmentation network for training to obtain an adenoids segmentation model; and using the adenoids segmentation model to complete the segmentation of the target adenoids image to obtain a segmentation result. The present invention intercepts adenoids images from a video obtained by a home endoscope and enhances the images, thereby improving the quality of image samples, helping machine learning models to capture richer features, reducing the risk of overfitting, and improving the segmentation performance of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of household endoscope image segmentation, and in particular to a method and system for detecting and segmenting adenoids in household endoscope videos. Background Art

[0002] Adenoids are a type of lymphoid tissue located in the nasopharynx that plays an important immune function in childhood. However, adenoid hypertrophy (AH), caused by excessive adenoid proliferation, is one of the most common diseases in childhood, with a prevalence of more than 1 / 3 in school-age and preschool children. Adenoid hypertrophy can not only lead to nocturnal hypoxia, hearing loss, intellectual and growth retardation, but also cause facial deformities, seriously affecting children's physical and mental health, and significantly affecting the quality of life of patients.

[0003] Traditionally, the diagnosis and identification of adenoid hypertrophy mainly relies on the manual segmentation of nasopharyngeal endoscopy images by doctors through naked eye observation during clinical examinations. However, since each doctor has different years of work and clinical experience, there are large differences in the identification of adenoid boundaries, which can easily lead to missed diagnosis or misdiagnosis. In addition, as a large medical device, nasopharyngeal endoscope has replaced skull X-ray examination as the gold standard for the diagnosis of adenoid hypertrophy. However, its procurement cost is high, and not all levels of medical institutions can be equipped with it. In addition, the cost of nasopharyngeal endoscopy is also relatively high in clinical practice, which makes many families unable to afford the examination costs due to economic reasons, which in turn leads to AH patients failing to receive timely diagnosis and treatment. Summary of the invention

[0004] To solve the above problems, an object of the embodiments of the present invention is to provide a method and system for detecting and segmenting adenoids in home endoscope videos.

[0005] A method for detecting and segmenting adenoids in home endoscope videos, comprising:

[0006] Step 1: Get the original home endoscope video;

[0007] Step 2: downsampling the home endoscope video to form a home endoscope image sequence;

[0008] Step 3: Use the home endoscope image sequence as a sample and input it into the deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first 4 stages of the ResNet-50 network are frozen, and only the parameters of the 3 residual blocks in stage 5 and the parameters of the output linear layer are optimized;

[0009] Step 4: Use the adenoids image recognition model to detect the target home endoscope video and extract the image containing the adenoids;

[0010] Step 5: enhancing the image containing the adenoids to obtain an enhanced adenoids image;

[0011] Step 6: Annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form training samples;

[0012] Step 7: Input the training samples into the deep segmentation network for training to obtain an adenoid segmentation model;

[0013] Step 8: Use the adenoids segmentation model to complete the segmentation of the target adenoids image and obtain a segmentation result.

[0014] Preferably, in step 2, downsampling the home endoscope video to form a home endoscope image sequence includes:

[0015] The original home endoscope video was downsampled at a frame rate of 1 fps using ffmpeg software to obtain a home endoscope image sequence.

[0016] Preferably, the step 3: taking the home endoscope image sequence as a sample and inputting it into the deep convolutional neural network Backbone for transfer learning to obtain the adenoids image recognition model comprises:

[0017] Step 3.1: The images containing adenoids in the home endoscope image sequence are taken as positive samples, and the images not containing adenoids are taken as negative samples, and the ratio of the number of positive and negative samples is set to 1:2;

[0018] Step 3.2: Input the positive and negative samples into the ResNet-50 network for training to obtain the adenoid image recognition model; when training the ResNet-50 network, the stochastic gradient descent method is used to optimize the ResNet-50 network, and the momentum coefficient during the training process is 0.99, and the learning rate is The learning rate decay uses a strategy that combines cosine annealing decay and ReduceLROnPlateau.

[0019] Preferably, the step 5: enhancing the image containing the adenoids to obtain an enhanced adenoids image comprises:

[0020] Step 5.1: Calculate the global average brightness of the adenoid image;

[0021] Step 5.2: Performing color inversion on the image corresponding to the global average brightness greater than the preset threshold to obtain a negative image, performing gamma correction on the negative image to obtain a corrected image, and performing negative image conversion on the corrected image again to obtain a preprocessed image;

[0022] Step 5.3: The image corresponding to the global average brightness being less than the preset threshold is directly used as the preprocessed image;

[0023] Step 5.4: Enhance the preprocessed image based on the histogram of the preprocessed image to obtain an enhanced adenoid image.

[0024] Preferably, in step 5.1, the formula is used:

[0025]

[0026] Calculate the global average brightness of the image containing the adenoids; where, represents the global average brightness, Indicates the width of the image. Indicates the height of the image. Indicates that the input image is The pixel value at location.

[0027] Preferably, the step 5.4: enhancing the preprocessed image based on the histogram of the preprocessed image to obtain an enhanced adenoids image comprises:

[0028] Step 5.4.1: Calculate the gray level histogram of the preprocessed image;

[0029] Step 5.4.2: Smoothing the gray level histogram to obtain a smoothed histogram;

[0030] Step 5.4.3: Calculate the cumulative distribution function based on the smoothed histogram;

[0031] Step 5.4.4: Determine the adaptive gamma function using the cumulative distribution function;

[0032] Step 5.4.5: Use the adaptive gamma function to enhance the corresponding image to form an enhanced adenoids image.

[0033] Preferably, the step 5.4.3: calculating the cumulative distribution function based on the smoothed histogram comprises:

[0034] Using the formula:

[0035]

[0036]

[0037]

[0038] The cumulative distribution function is calculated using the smoothed histogram; where is the cumulative distribution function, is the normalized histogram, is the original grayscale value, is the smoothed histogram, is the gray level histogram, is the maximum value in the gray level histogram, is the minimum value in the gray level histogram.

[0039] Preferably, the step 5.4.5: using an adaptive gamma function to enhance the corresponding image to form an enhanced adenoids image comprises:

[0040] Using the formula:

[0041]

[0042]

[0043] The corresponding image is enhanced to form an enhanced adenoid image; wherein, , is the adaptive gamma function, For the enhanced image, is the input image Zhongzai The pixel intensity at position, Indicates integer.

[0044] Preferably, the step 7: inputting the training samples into a deep segmentation network for training to obtain an adenoid segmentation model comprises:

[0045] The training samples are input into the nnUNet model for training, and the nnUNet model is optimized using the stochastic gradient descent method with momentum to obtain the adenoid segmentation model; wherein the formula of the stochastic gradient descent method is:

[0046]

[0047]

[0048] In the formula, is the current moment based on the sample The calculated loss function gradient is yes t Momentum at time +1, yes t Momentum of moment, is the learning rate, is the momentum coefficient, which indicates the degree of attenuation of past gradients. yes t The model parameters at time t, yes t+ Model parameters at time 1;

[0049] The loss function during training uses the hybrid cross entropy DICE loss:

[0050]

[0051] in, is the total loss function, is the weighted ratio of the loss function, is the cross entropy loss, It's a DICE loss.

[0052] The present invention also provides an adenoid detection and segmentation system for home endoscope video, comprising:

[0053] A home endoscope video acquisition module, used to acquire original home endoscope videos;

[0054] A video downsampling module, used for downsampling the home endoscope video to form a home endoscope image sequence;

[0055] A training module is used to input a home endoscope image sequence as a sample into a deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first four stages of the ResNet-50 network are frozen, and only the parameters of the three residual blocks in stage 5 and the parameters of the output linear layer are optimized;

[0056] An adenoids image extraction module is used to detect the target home endoscope video using an adenoids image recognition model to extract an image containing adenoids;

[0057] an adenoids image enhancement module, used for enhancing an image containing adenoids to obtain an enhanced adenoids image;

[0058] A sample annotation module is used to annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form a training sample;

[0059] A segmentation model training module, used for inputting the training samples into a deep segmentation network for training to obtain an adenoid segmentation model;

[0060] The adenoids segmentation module is used to segment the target adenoids image using the adenoids segmentation model to obtain a segmentation result.

[0061] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0062] The present invention relates to a method and system for adenoids detection and segmentation in home endoscope videos. Compared with the prior art, the present invention can significantly improve the quality of samples by preprocessing and enhancing home endoscope videos, help machine learning models capture richer features, reduce the risk of overfitting, and improve the segmentation performance of the model.

[0063] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 A flow chart of a method for detecting and segmenting adenoids in home endoscope videos provided by the present invention;

[0066] Figure 2 The improved ResNet-50 network structure diagram provided by the present invention;

[0067] Figure 3 A comparison chart of the image enhancement effect provided by the present invention;

[0068] Figure 4 This is the adenoid segmentation result diagram provided by the present invention. DETAILED DESCRIPTION

[0069] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0070] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0071] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] See also Figure 1 , a method for detecting and segmenting adenoids in home endoscopy videos, comprising:

[0073] Step 1: Get the original home endoscope video;

[0074] The average length of the anterior and posterior nostrils of children is 3.5 cm. Although parents or primary care doctors can easily reach the adenoid area of ​​the posterior nasal cavity using a probe, there is still a problem of how to capture accurate and appropriate adenoid images. Based on this, the present invention obtains a video of a parent or primary care doctor operating a home nasal endoscope and uses it as a sample to provide a standardized basis for subsequent data analysis and model establishment.

[0075] Step 2: downsampling the home endoscope video to form a home endoscope image sequence;

[0076] Specifically, the present invention can use ffmpeg to downsample the video to extract the image to be screened through the following steps: first, import the original video into the ffmpeg tool, set the frame rate to 1fps (1 frame per second) to reduce the sampling rate; then, use ffmpeg to process the original video frame by frame and output it as an image sequence to form an image set with a lower frame rate, which is convenient for subsequent screening and analysis.

[0077] Step 3: Use the home endoscope image sequence as a sample and input it into the deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first 4 stages of the ResNet-50 network are frozen, and only the parameters of the 3 residual blocks in stage 5 and the parameters of the output linear layer are optimized;

[0078] Furthermore, the step 3 comprises:

[0079] Step 3.1: The images containing adenoids in the home endoscope image sequence are taken as positive samples, and the images not containing adenoids are taken as negative samples, and the ratio of the number of positive and negative samples is set to 1:2;

[0080] Step 3.2: Input the positive and negative samples into the ResNet-50 network for training to obtain the adenoid image recognition model; when training the ResNet-50 network, the stochastic gradient descent method is used to optimize the ResNet-50 network, and the momentum coefficient during the training process is 0.99, and the learning rate is The learning rate decay uses a strategy that combines cosine annealing decay and ReduceLROnPlateau.

[0081] It should be noted that the original ResNet-50 is a typical deep convolutional neural network consisting of 50 layers. Its core is to solve the gradient vanishing problem in deep networks through residual connections (skip connections) and support deeper network training. The network structure of ResNet-50 includes an input convolution layer, five stages (Stage 1 to Stage 5) consisting of multiple residual blocks (Residual Blocks), and finally the classification task is completed through the global average pooling layer and the fully connected layer. Each residual block is usually composed of three convolutional layers (1x1, 3x3, 1x1), and the input is directly added to the output through residual connections, which alleviates the training difficulties of deep networks. The original ResNet-50 is designed for multi-classification tasks. The output of the fully connected layer is 1000 neurons, which is used for classification tasks of datasets such as ImageNet.

[0082] See also Figure 2 This method modifies the fully connected layer of the original ResNet-50 network. The traditional ResNet-50 network is designed to handle 1000-class classification tasks, and its fully connected layer outputs 1000 neurons. In combination with specific application scenarios, this method changes the number of output neurons of the fully connected layer to 2, and the output is processed by the Softmax function. , we get the probability of whether the image contains adenoids, which is suitable for the binary classification task of judging whether there are adenoids or not.

[0083] During the network training process, the present invention adopts an optimization strategy of transfer learning, including the following specific steps:

[0084] 1. Pre-trained model loading: First, load the ResNet-50 model pre-trained on the ImageNet dataset to take advantage of the pre-trained model's advantages in feature extraction.

[0085] 2. Parameter freezing: Next, freeze all parameters of the first four stages of the network (Stage 1 to Stage 4), that is, the weights of these layers remain unchanged during the training process. This operation is intended to retain the feature extraction capability of the shallow layer of the network to adapt to the adenoids detection task.

[0086] 3. Fine-tuning optimization: Only the parameters of the three residual blocks and the fully connected layer in the last stage (Stage 5) of the network are adjusted. In this way, the network can better adapt to the characteristics of the new task while reducing training time and dependence on large datasets.

[0087] 4. Loss function selection and optimization: Cross-Entropy Loss is used in combination with stochastic gradient descent (SGD) for parameter optimization.

[0088] 5. Learning rate adjustment: The learning rate is gradually adjusted during the training process through the learning rate scheduler to ensure rapid convergence in the early stage of training and fine-tune in the later stage to obtain higher model accuracy. Specifically, at the beginning of training, a higher initial learning rate is set to quickly update the weights so that the model is close to the optimal area in a short time. When the verification loss stops decreasing within several epochs, the learning rate is automatically reduced. This ensures that the model has enough learning rate for small-scale parameter adjustments during the fine-tuning stage. In the present invention, the attenuation of the learning rate uses a strategy that combines cosine annealing attenuation and ReduceLROnPlateau to meet the needs of the model at different training stages.

[0089] 6. Cosine Annealing: In the main stage of training, a cosine annealing attenuator is used to gradually reduce the learning rate according to the cosine curve. This method allows the learning rate to gradually decrease during the training process while avoiding large fluctuations. The formula for cosine annealing is:

[0090] in is the initial learning rate, is the minimum learning rate, is the current epoch, is the decay cycle.

[0091] 7.ReduceLROnPlateau: ReduceLROnPlateau is used to adaptively adjust the learning rate during the fine-tuning phase of the model. When the validation loss no longer decreases over a period of time (for example, it has not increased for 5 consecutive epochs), ReduceLROnPlateau will reduce the learning rate to a smaller value (such as 10% of the current learning rate) to ensure that the model can perform detailed parameter optimization with a smaller learning step size. Using this strategy can further reduce the risk of overfitting while improving the final model accuracy.

[0092] The training data was manually screened by an otolaryngologist with 3 years of experience. Images containing adenoids in the examination videos were selected as positive samples, and images without adenoids were randomly sampled as negative samples for training. The ratio of positive and negative samples was 1:2. This ratio was selected because the number of clips without adenoids in the original video was much higher than the number of clips containing adenoids. Completely random sampling would lead to an imbalance in the number of samples, making it difficult for the model to correctly classify images containing adenoids. Selecting this ratio can ensure that the data set contains a sufficient number of images containing adenoids while increasing the diversity of images without adenoids, thereby improving the robustness of the model.

[0093] The model was optimized using stochastic gradient descent (SGD) with a momentum coefficient of 0.99 and a learning rate of , the loss function is the cross entropy loss:

[0094]

[0095] in, are the predicted labels and the true labels respectively.

[0096] Step 4: Use the adenoids image recognition model to detect the target home endoscope video and extract the image containing the adenoids;

[0097] Step 5: enhancing the image containing the adenoids to obtain an enhanced adenoids image;

[0098] Furthermore, step 5 includes:

[0099] Step 5.1: Calculate the global average brightness of the adenoid image;

[0100] Step 5.2: Performing color inversion on the image corresponding to the global average brightness greater than the preset threshold to obtain a negative image, performing gamma correction on the negative image to obtain a corrected image, and performing negative image conversion on the corrected image again to obtain a preprocessed image;

[0101] Step 5.3: The image corresponding to the global average brightness being less than the preset threshold is directly used as the preprocessed image;

[0102] Step 5.4: Enhance the preprocessed image based on the histogram of the preprocessed image to obtain an enhanced adenoids image.

[0103] Among them, step 5.4 includes:

[0104] Step 5.4.1: Calculate the gray level histogram of the preprocessed image;

[0105] Step 5.4.2: Smoothing the gray level histogram to obtain a smoothed histogram;

[0106] Step 5.4.3: Calculate the cumulative distribution function based on the smoothed histogram;

[0107] Furthermore, in step 5.4.3, the formula is used:

[0108]

[0109]

[0110]

[0111] The cumulative distribution function is calculated using the smoothed histogram; where is the cumulative distribution function, is the normalized histogram, is the original grayscale value, is the smoothed histogram, is the gray level histogram, is the maximum value in the gray level histogram, is the minimum value in the gray level histogram.

[0112] Step 5.4.4: Determine the adaptive gamma function using the cumulative distribution function;

[0113] Step 5.4.5: Use the adaptive gamma function to enhance the corresponding image to form an enhanced adenoids image.

[0114] In step 5.4.5, the formula is used:

[0115]

[0116]

[0117] The corresponding image is enhanced to form an enhanced adenoid image; wherein, , is the adaptive gamma function, For the enhanced image, is the input image Zhongzai The pixel intensity at position, Indicates integer.

[0118] The image enhancement process of the present invention is further described below in conjunction with specific embodiments:

[0119] First, the input image is preprocessed. ,in , , calculate the global average brightness As a criterion for judging image type:

[0120]

[0121] in, and are the width and height of the image respectively. Threshold constant It is usually set to half the maximum pixel intensity (e.g., 128 for 8-bit images) to represent the expected global average brightness of normal natural images. By comparison, it can be determined whether the image is too dark, too bright, or normally illuminated. For images that are too dark, they are directly corrected; for images that are too bright, the input image is converted to its negative image. Then perform Gamma correction to get Then perform negative image transformation to obtain the corrected original image .

[0122] Next, we calculate the gray level histogram of the input image (or its negative, depending on the type of image) , and use the weighted distribution function to smooth the initial histogram:

[0123]

[0124] in, and They are The maximum and minimum values ​​of .

[0125] Then calculate the weighted cumulative distribution function CDF. Normalized to get: , from the normalized histogram Calculate the cumulative distribution function (CDF): , then calculate the cumulative distribution function CDF to determine the Gamma value. The CDF is truncated to obtain the adaptive gamma function:

[0126]

[0127] in, Avoid Gamma values ​​that are too small.

[0128] Finally, use the adaptive gamma function Transform the input image Pixel values ​​and keep integers:

[0129]

[0130] in, is the input image Middle position The pixel intensity at .

[0131] See also Figure 3 Compared with the original image, the enhancement method of the present application has significantly enhanced the contrast and dark details. Although the traditional CLAHE can also enhance the contrast and dark details, the color is obviously distorted compared with AGCWD, which is not conducive to observation.

[0132] Step 6: Annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form training samples;

[0133] Step 7: Input the training samples into the deep segmentation network for training to obtain an adenoid segmentation model;

[0134] Furthermore, step 7 includes:

[0135] The training samples are input into the nnUNet model for training, and the nnUNet model is optimized using the stochastic gradient descent method with momentum to obtain the adenoid segmentation model; wherein the formula of the stochastic gradient descent method is:

[0136]

[0137]

[0138] In the formula, is the current moment based on the sample The calculated loss function gradient is yes t Momentum at time +1, yes t Momentum of time, is the learning rate, is the momentum coefficient, which indicates the degree of attenuation of past gradients. yes t The model parameters at time t, yes t+ Model parameters at time 1;

[0139] The loss function during training uses the hybrid cross entropy DICE loss:

[0140]

[0141] in, is the total loss function, is the weighted ratio of the loss function, is the cross entropy loss, It's a DICE loss.

[0142] In practical applications, the deep segmentation network is the nnUNet model, which is an automated framework for medical image segmentation, mainly based on the U-Net architecture. U-Net is a convolutional neural network designed for biomedical image segmentation. Its basic structure includes an encoder and a decoder, and the feature map in the encoder is connected to the feature map in the decoder through a jump connection.

[0143] For the training data construction of the deep segmentation network, the sample photos of the present invention are annotated by a doctor with 3 years of experience in the adenoid area and the posterior nasal area, and then the data set is divided into a training set and a test set at a ratio of 4:1. The present invention adopts a five-fold cross-validation method, randomly dividing the training set into five parts, and each time four of them are used as training sets, and the remaining one is used as a validation set to train and test the model.

[0144] For the model construction of the deep segmentation network, the non-zero part of the image in the training set was cropped according to the preprocessing method of nnUnet and the minimum, maximum, median, average, upper 0.05 quantile and lower 0.05 quantile of each channel were analyzed. The mean, median, standard deviation, lower 0.05 quantile and upper 0.05 quantile of the R channel were 122.78, 122, 50.09, 13, 255 respectively; the mean, median, standard deviation, lower 0.05 quantile and upper 0.05 quantile of the G channel were 77.01, 77, 39.57, 0, 237 respectively; the mean, median, standard deviation, lower 0.05 quantile and upper 0.05 quantile of the B channel were 54.89, 55, 34.61, 0, 211 respectively. According to the above values, the network order is 7, the feature dimensions of each stage are 32, 64, 128, 256, 512, 512, 512, the convolution kernel sizes of each stage are 3, 3, 3, 3, 3, 3, and the batch size is 13.

[0145] For the model training of the deep segmentation network, the stochastic gradient descent method (SGD) with momentum is used to optimize the model, with an initial learning rate of 0.01, a momentum coefficient of 0.99, and a weight decay of , using a polynomial learning rate decay.

[0146] The stochastic gradient descent method SGD with momentum can be expressed as:

[0147]

[0148]

[0149] in, is the current moment based on the sample The calculated loss function gradient is is the momentum at different times, is the learning rate, The momentum coefficient represents the degree of attenuation of past gradients. are the model parameters at different times. SGD with momentum can effectively solve the common problems of slow convergence and parameter update oscillation in standard SGD. By accumulating historical gradients, the momentum term helps accelerate convergence and reduce the instability of parameter updates.

[0150] The loss function uses the hybrid cross entropy DICE loss:

[0151]

[0152] The significance of the hybrid loss is that the DICE loss directly measures the overlap between the prediction and the true label, focusing on the region overlap rather than the pixel-level accuracy, which can effectively reduce the impact of class imbalance and improve the segmentation effect of small target regions. The cross entropy loss focuses on the classification accuracy of each pixel and evaluates whether the model is correctly classified at each pixel. It can calculate the loss for each pixel, which helps to improve the segmentation accuracy of large areas.

[0153] For the segmentation of adenoids, it is necessary to pay attention to the segmentation of the relatively small adenoids area and the relatively large posterior nasal area at the same time. The mixed loss helps to improve the overall segmentation performance.

[0154] Table 1 below shows the DICE score comparison with the ordinary 5-layer UNet, which shows that the performance of the deep segmentation network of the present invention is significantly improved:

[0155] Table 1

[0156]

[0157] By enhancing home endoscope images, the present invention can significantly improve the quality of image samples, help machine learning models capture richer features, reduce the risk of overfitting, and improve the segmentation performance of the model.

[0158] Step 8: Use the adenoids segmentation model to complete the segmentation of the target adenoids image and obtain a segmentation result.

[0159] like Figure 4 As shown, after obtaining the segmentation result, the present invention can calculate the adenoids hypertrophy rating according to the segmentation result of the adenoids and the posterior nasal region.

[0160] Assume the pixel area of ​​the adenoid region is ,in Represents the pixels in the predicted mask of the adenoids region The pixel area of ​​the posterior nasal cavity is ,in Represents the pixels in the predicted mask of the posterior nasal region The value of . Let the hypertrophy score be ,according to According to the different values, the present invention can classify adenoids hypertrophy into normal: 0.01; I degree: 0.25; II degree: 0.5; III degree: 0.75; IV degree: 1.

[0161] The present invention also provides an adenoid detection and segmentation system for home endoscope video, comprising:

[0162] A home endoscope video acquisition module, used to acquire original home endoscope videos;

[0163] A video downsampling module, used for downsampling the home endoscope video to form a home endoscope image sequence;

[0164] A training module is used to input a home endoscope image sequence as a sample into a deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first four stages of the ResNet-50 network are frozen, and only the parameters of the three residual blocks in stage 5 and the parameters of the output linear layer are optimized;

[0165] An adenoids image extraction module is used to detect the target home endoscope video using an adenoids image recognition model to extract an image containing adenoids;

[0166] an adenoids image enhancement module, used for enhancing an image containing adenoids to obtain an enhanced adenoids image;

[0167] A sample annotation module is used to annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form a training sample;

[0168] A segmentation model training module, used for inputting the training samples into a deep segmentation network for training to obtain an adenoid segmentation model;

[0169] The adenoid segmentation module is used to segment the target adenoid image using the adenoid segmentation model to obtain a segmentation result. Compared with the prior art, the beneficial effects of the adenoid detection and segmentation system for home endoscope images provided by the present invention are the same as the beneficial effects of the adenoid detection and segmentation method for home endoscope videos described in the above technical solution, which will not be repeated here.

[0170] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technical solution that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting and segmenting adenoids in home endoscope videos, characterized in that: include: Step 1: Get the original home endoscope video; Step 2: downsampling the home endoscope video to form a home endoscope image sequence; Step 3: Use the home endoscope image sequence as a sample and input it into the deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first 4 stages of the ResNet-50 network are frozen, and only the parameters of the 3 residual blocks in stage 5 and the parameters of the output linear layer are optimized; Step 4: Use the adenoids image recognition model to detect the target home endoscope video and extract the image containing the adenoids; Step 5: enhancing the image containing the adenoids to obtain an enhanced adenoids image; Step 6: Annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form training samples; Step 7: Input the training samples into the deep segmentation network for training to obtain an adenoid segmentation model; Step 8: Use the adenoids segmentation model to complete the segmentation of the target adenoids image and obtain a segmentation result; In step 2, downsampling the home endoscope video to form a home endoscope image sequence includes: The original home endoscope video was downsampled to a frame rate of 1 fps using ffmpeg software to obtain a home endoscope image sequence; The step 3: using the home endoscope image sequence as a sample, inputting it into the deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model, including: Step 3.1: The images containing adenoids in the home endoscope image sequence are taken as positive samples, and the images not containing adenoids are taken as negative samples, and the ratio of the number of positive and negative samples is set to 1:2; Step 3.2: Input the positive and negative samples into the ResNet-50 network for training to obtain the adenoid image recognition model; when training the ResNet-50 network, the stochastic gradient descent method is used to optimize the ResNet-50 network, and the momentum coefficient during the training process is 0.99 and the learning rate is 1×10 -5 The learning rate decay uses a strategy that combines cosine annealing decay and ReduceLROnPlateau.

2. The adenoid detection and segmentation method for home endoscope video according to claim 1, characterized in that: The step 5: enhancing the image containing the adenoids to obtain an enhanced adenoids image, comprises: Step 5.1: Calculate the global average brightness of the adenoid image; Step 5.2: Performing color inversion on the image corresponding to the global average brightness greater than the preset threshold to obtain a negative image, performing gamma correction on the negative image to obtain a corrected image, and performing negative image conversion on the corrected image again to obtain a preprocessed image; Step 5.3: The image corresponding to the global average brightness being less than the preset threshold is directly used as the preprocessed image; Step 5.4: Enhance the preprocessed image based on the histogram of the preprocessed image to obtain an enhanced adenoids image.

3. The adenoid detection and segmentation method for home endoscope video according to claim 2, characterized in that: In step 5.1, use the formula: Calculate the global average brightness of the adenoids image; where t represents the global average brightness, M represents the width of the image, N represents the height of the image, and I(x,y) represents the pixel value of the input image at the (x,y) position.

4. According to claim 3, a method for detecting and segmenting adenoids in home endoscope videos, characterized in that: The step 5.4: enhancing the preprocessed image based on the histogram of the preprocessed image to obtain an enhanced adenoids image, comprises: Step 5.4.1: Calculate the gray level histogram of the preprocessed image; Step 5.4.2: Smoothing the gray level histogram to obtain a smoothed histogram; Step 5.4.3: Calculate the cumulative distribution function based on the smoothed histogram; Step 5.4.4: Determine the adaptive gamma function using the cumulative distribution function; Step 5.4.5: Use an adaptive gamma function to enhance the corresponding image to form an enhanced adenoids image.

5. The adenoid detection and segmentation method for home endoscope video according to claim 4, characterized in that: The step 5.4.3: calculating the cumulative distribution function based on the smoothed histogram, comprises: Using the formula: The cumulative distribution function is calculated using the smoothed histogram; where c w (l) is the cumulative distribution function, p w ′(l) is the normalized histogram, l is the original grayscale value, p w (l) is the smoothed histogram, p(l) is the gray level histogram, p max is the maximum value in the gray level histogram, p min is the minimum value in the gray level histogram.

6. The adenoid detection and segmentation method for home endoscope video according to claim 5, characterized in that: The step 5.4.5: using an adaptive gamma function to enhance the corresponding image to form an enhanced adenoids image, comprises: Using the formula: c w ′(l)=max(τ,1-c w (l)) The corresponding image is enhanced to form an enhanced adenoid image; wherein, τ = 0.3, γ w ′(l) is the adaptive gamma function, I e (x, y) is the enhanced image, I(x, y) is the pixel intensity at position (x, y) in the input image I, and round means taking an integer.

7. The adenoid detection and segmentation method for home endoscope video according to claim 1, characterized in that: The step 7: inputting the training samples into a deep segmentation network for training to obtain an adenoid segmentation model, including: The training samples are input into the nnUNet model for training, and the nnUNet model is optimized using the stochastic gradient descent method with momentum to obtain the adenoid segmentation model; wherein the formula of the stochastic gradient descent method is: i t+1 =θ t -v t+1 In the formula, is the current moment based on sample x (i) The calculated loss function gradient, degree, v t+1 is the momentum at time t+1, v t is the momentum at time t, η is the learning rate, γ is the momentum coefficient indicating the degree of attenuation of past gradients, and θ t is the model parameter at time t, θ t+1 is the model parameter at time t+1; The loss function during training uses the hybrid cross entropy DICE loss: in, is the total loss function, α is the weighted ratio of the loss function, is the cross entropy loss, It's a DICE loss.

8. An adenoids detection and segmentation system for home endoscope video, characterized in that: include: A home endoscope video acquisition module, used to acquire original home endoscope videos; A video downsampling module, used for downsampling the home endoscope video to form a home endoscope image sequence; A training module is used to input a home endoscope image sequence as a sample into a deep convolutional neural network Backbone for transfer learning to obtain an adenoid image recognition model; wherein the deep convolutional neural network Backbone is a ResNet-50 network, and the number of output neurons in the fully connected layer of the ResNet-50 network is 2. During the transfer learning process, the parameters of the first four stages of the ResNet-50 network are frozen, and only the parameters of the three residual blocks in stage 5 and the parameters of the output linear layer are optimized; An adenoids image extraction module is used to detect the target home endoscope video using an adenoids image recognition model to extract an image containing adenoids; an adenoids image enhancement module, used for enhancing an image containing adenoids to obtain an enhanced adenoids image; A sample annotation module is used to annotate the adenoids region and the posterior nasal region of the enhanced adenoids image to form a training sample; A segmentation model training module, used for inputting the training samples into a deep segmentation network for training to obtain an adenoid segmentation model; An adenoids segmentation module, used to segment the target adenoids image using the adenoids segmentation model to obtain a segmentation result; The home endoscope video is downsampled to form a home endoscope image sequence, including: The original home endoscope video was downsampled to a frame rate of 1 fps using ffmpeg software to obtain a home endoscope image sequence; The home endoscope image sequence is used as a sample and input into the deep convolutional neural network Backbone for transfer learning to obtain the adenoid image recognition model, including: The images containing adenoids in the home endoscope image sequence were taken as positive samples, and the images without adenoids were taken as negative samples. The ratio of positive and negative samples was set to 1:

2. The positive and negative samples were input into the ResNet-50 network for training to obtain the adenoids image recognition model. When training the ResNet-50 network, the stochastic gradient descent method was used to optimize the ResNet-50 network. The momentum coefficient during the training process was 0.99, and the learning rate was 1×10 -5 The learning rate decay uses a strategy that combines cosine annealing decay and ReduceLROnPlateau.

Images