An intelligent classification and identification method for ear-nose-throat lesion images

Abstract

The application discloses an intelligent classification and identification method for ear-nose-throat lesion images, and relates to the technical field of image recognition, comprising the following steps: synchronously collecting a nasopharynx white light image and an i-Scan image, screening images meeting quality standards, associating pathology and grading annotation, and constructing a data set containing multi-center cases; carrying out denoising processing on the white light image and the i-Scan image respectively, extracting key landmark points and completing image registration, dividing mucosa surface area, submucosal blood vessel area and lymph tissue enrichment area through semantic segmentation, and carrying out feature enhancement on different areas; extracting morphological features and texture features from the enhanced images, calculating feature importance weights and fusing into low-dimensional feature vectors; constructing a benign and malignant binary classification model and a pathology type multi-classification model respectively, combining a physician diagnosis rule base and a verification index to output a classification result, and significantly improving the accuracy and clinical practicability of ear-nose-throat lesion diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to an intelligent classification and recognition method for ear-nose-throat lesion images. BACKGROUND

[0002] Early diagnosis of ear-nose-throat lesions is of great significance for clinical treatment. Traditional diagnosis methods mainly rely on endoscopic physicians' visual assessment of white light or i-Scan images, combined with pathological biopsy results for judgment. In the prior art, there have been attempts to use image processing techniques to extract lesion features, such as texture analysis based on gray level co-occurrence matrix, calculation of vascular morphological parameters, etc., and to combine machine learning algorithms for classification. In addition, some methods improve the contrast of the lesion area by multi-modal image registration, or use semantic segmentation technology to divide the mucosa and blood vessel area to assist in diagnosis.

[0003] However, the prior art still has obvious deficiencies. Most methods rely on a single modality image, such as only white light or i-Scan images, and do not fully utilize the complementary information of multi-modal images, resulting in insufficient sensitivity and specificity for subtle lesions. Secondly, the standardization of image preprocessing and feature extraction is low, and the image differences caused by different devices or acquisition conditions may affect the generalization ability of the model. Existing classification models usually use a single algorithm, such as pure CNN or traditional machine learning, which lacks the synergistic optimization of benign and malignant binary classification and pathological multi-classification tasks. In addition, the model lacks interpretability, making it difficult to effectively combine with physicians' clinical experience, resulting in limited credibility and practicality of the diagnosis results. These problems restrict the practical application effect of intelligent classification technology in ear-nose-throat lesion diagnosis. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies of the prior art, the present application provides an intelligent classification and recognition method for ear-nose-throat lesion images, which solves the problems of insufficient sensitivity and specificity, poor model generalization ability, and low classification result interpretability in the prior art by synchronously collecting multi-modal images, optimizing the image preprocessing and feature extraction process, constructing a double-layer classification model, and combining a physician diagnosis rule base. The accuracy and clinical practicality of ear-nose-throat lesion diagnosis are significantly improved.

[0006] (II) Technical solutions

[0007] To achieve the above purpose, the present application is implemented by the following technical solutions: an intelligent classification and recognition method for ear-nose-throat lesion images, comprising:

[0008] Synchronous acquisition of white light images and i-Scan images of the nasopharynx, screening of images meeting the quality standards, and correlation of pathology, grading labeling to construct a multi-center case data set;

[0009] Denoising of white light images and i-Scan images, extraction of key landmark points and completion of image registration, division of mucosal surface area, submucosal blood vessel area and lymph tissue enrichment area by semantic segmentation, and feature enhancement of different areas;

[0010] Extraction of morphological features and texture features from the enhanced images, calculation of feature importance weights and fusion into low-dimensional feature vectors;

[0011] Support vector machine and CNN-LSTM hybrid network are used to construct benign and malignant binary classification model and pathological type multi-classification model respectively, and the classification results are output combined with the physician diagnosis rule base and verification index.

[0012] Further, the endoscopic device is used to synchronously acquire white light images and i-Scan images of the nasopharynx, and the white light group has no enhancement mode; the i-Scan group image parameters include surface enhancement, contrast enhancement and color tone enhancement; at least 4 non-overlapping images are taken in at least 3 anatomical regions of the nasopharyngeal roof, eustachian tube opening and pharyngeal recess, the i-Scan image acquisition adopts white light positioning, shortcut key switching i-Scan mode, white light and i-Scan alternating confirmation process; the images are screened based on the mean value of edge gradient, motion artifact proportion and reflection artifact proportion, and each image must contain at least one key anatomical landmark point.

[0013] Further, the pathological results are divided into 5 levels: 1 level for chronic inflammation of mucosa and low-grade lymph tissue hyperplasia, 2 level for chronic inflammation of mucosa and high-grade lymph tissue hyperplasia, 3 level for low-grade squamous epithelial metaplasia and atypical hyperplasia, 4 level for non-keratinizing squamous cell carcinoma or undifferentiated carcinoma, and 5 level for lymphoma.

[0014] Further, the adaptive median filter algorithm is used for denoising of white light images, and the wavelet transform denoising algorithm is used for denoising of i-Scan images, the images are decomposed into low-frequency approximation components and high-frequency detail components, the soft threshold value processing is adopted for the high-frequency detail components, the soft threshold value = 0.02 x image gray maximum value, and the Harris corner point detection algorithm is used to extract key landmark points in each white light image and corresponding i-Scan image;

[0015] The improved SIFT algorithm is used, combined with the random sample consensus algorithm to remove false matching points, the points with a matching error greater than 2 pixels are considered as false matching, the i-Scan image is mapped to the white light image coordinate system through affine transformation, and the gray correlation of the registered images is calculated, and the Pearson correlation coefficient ≥ 0.85 is considered as qualified registration.

[0016] Further, the U-Net semantic segmentation model is used to automatically divide the registered image into three regions, including a mucosa surface region, a submucosal blood vessel region and a lymph tissue enrichment region, the limited contrast self-adaptive histogram equalization is used for the mucosa surface region, the gamma correction and Frangi blood vessel filtering are used for the submucosal blood vessel region, and the Gaussian blur and Laplace operator are used for the lymph tissue enrichment region; the bilinear interpolation algorithm is used to scale all the enhanced images, and the pixel value is normalized from [0, 255] to [0, 1], and the batch normalization algorithm is used to correct the gray distribution of the images collected by different endoscopic devices.

[0017] Further, for the i-Scan image, the Zhang-Suen thinning algorithm is used to extract the blood vessel center axis of the submucosal blood vessel region, the number of nodes connected to ≥3 adjacent pixels in the blood vessel skeleton is counted as the number of blood vessel branches, the blood vessel skeleton is segmented, the curvature of each segment is calculated after fitting a straight line, the standard deviation of the curvatures of all segments is taken as the blood vessel tortuosity, and the number of breaks of the continuous pixel spacing ≥2 pixels in the blood vessel skeleton is counted as the number of interruptions; the Canny edge detection and Snakes active contour model are used to extract the lesion boundary, the ratio of the boundary length to the perimeter of the minimum circumscribed rectangle of the lesion is taken as the boundary irregularity, and the average gray gradient of the boundary pixels is taken as the boundary definition.

[0018] Further, for the white light image, the gray level co-occurrence matrix is constructed in the mucosa surface region and the lymph tissue enrichment region, the contrast, correlation and entropy values of the gray level co-occurrence matrix are calculated, and the local binary pattern operator is used to count the bin features of the local binary pattern histogram.

[0019] Further, based on the training set data, the Spearman correlation coefficient of each feature and the pathological grade is calculated, each feature is multiplied by the corresponding attention weight, and then spliced into a fusion feature vector, the minimum redundancy maximum correlation algorithm and principal component analysis are used for dimension reduction to obtain a low-dimensional feature vector, the morphological features include the number of branches, the tortuosity, the number of interruptions, the boundary irregularity and the boundary definition, and the texture features include the contrast, the correlation, the entropy value and the bin features of the local binary pattern.

[0020] Further, the support vector machine is used to build a first layer benign and malignant binary classification model, the low-dimensional feature vector is input, and benign or malignant and classification confidence is output; the CNN-LSTM hybrid network is used to build a second layer pathological type multi-classification model, the input is the low-dimensional feature vector, the white light image and the i-Scan image, and the output is the probability of 5 types of pathological grades.

[0021] Further, a physician diagnosis rule library is constructed: rule 1: if the vascular tortuosity is greater than or equal to 0.8 and the number of interruptions is greater than or equal to 3, it is determined that the nasopharyngeal carcinoma is grade 4; rule 2: if the boundary irregularity is greater than or equal to 0.7 and the gray level co-occurrence matrix entropy value is greater than or equal to 1.5, it is determined that the atypical hyperplasia is grade 3; rule 3: if the number of vascular branches is less than or equal to 5 and the boundary definition is greater than or equal to 50, it is determined that the benign inflammation is grade 1-2; if the binary classification confidence is greater than or equal to a confidence threshold and the multi-classification result is consistent with the rule library determination, the classification result is directly output, otherwise, it is pushed for review.

[0022] (Three) beneficial effects

[0023] The application provides an intelligent classification and identification method for ear-nose-throat lesion images, which has the following beneficial effects:

[0024] (1) By synchronously collecting nasopharyngeal white light images and i-Scan images and strictly screening images meeting the quality standard, a multi-center and multi-modal data set is constructed, the comprehensiveness and representativeness of the data are ensured, the pathological grading annotation is associated, accurate supervision signals are provided for subsequent model training, the reliability of image analysis is improved, human error is reduced through a standardized acquisition process, a high-quality data foundation is laid for intelligent classification and identification, and the generalization ability and diagnostic accuracy of the model are significantly improved.

[0025] (2) The white light and i-Scan images are denoised by adaptive median filtering and wavelet transform respectively, key details are effectively retained and artifacts are suppressed; precise registration is realized by using Harris corner points and an improved SIFT algorithm, and the spatial consistency of multi-modal images is ensured; the images are divided into mucosal surface area, blood vessel area and lymph tissue enrichment area based on U-Net semantic segmentation, and enhancement technologies such as CLAHE and Frangi filtering are used for different areas, which significantly improves the contrast and distinguishability of lesion features, lays a high-quality image foundation for subsequent feature extraction and classification, and enhances the sensitivity and robustness of the model to subtle lesions.

[0026] (3) By extracting morphological features such as the number of vascular branches, tortuosity and interruption times, and texture features such as contrast and entropy, the importance weight of the features is calculated by using the Spearman correlation coefficient, the key indicators strongly related to pathological grading are effectively selected, the least redundancy and maximum correlation algorithm and principal component analysis are used for dimension reduction, redundant features are removed and more than 95% of the information is retained, a low-dimensional feature vector is generated, the representativeness and classification efficiency of the features are significantly improved, high-distinguishability input data is provided for subsequent model construction, the ability of the model to capture subtle differences in lesions is enhanced, and finally the accuracy and robustness of classification are improved.

[0027] (4) A double-layer classification model is constructed by a support vector machine (SVM) and a CNN-LSTM hybrid network, and a physician diagnosis rule base is combined to realize high-precision benign and malignant binary classification and pathological type multi-classification. The SVM rapidly screens benign and malignant based on a low-dimensional feature vector, the CNN-LSTM fuses image and feature data to enhance the fine-grained classification ability, the rule base provides an explainable auxiliary decision, manual review is triggered when the model confidence is insufficient or conflicts with the rule, and the result reliability is ensured. Multi-center verification and cross-validation show that the model stability is improved, and the diagnostic efficiency and consistency are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of an intelligent classification and recognition method for ear-nose-throat lesion images according to the present application;

[0029] Figure 2 A flowchart of an intelligent classification and recognition method for ear-nose-throat lesion images according to the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0031] Please refer to Figure 1 and Figure 2 The present application provides an intelligent classification and recognition method for ear-nose-throat lesion images, comprising the following steps:

[0032] Step one: Synchronously collecting nasopharyngeal white light images and i-Scan images, screening images meeting the quality standard, and associating pathological and grading labels to construct a data set containing multi-center cases;

[0033] The step one comprises the following contents:

[0034] Step 101: Using a Pentax EPK-3000 series i-Scan endoscope device to synchronously collect nasopharyngeal white light images and i-Scan images. The white light image parameters are: light source intensity 800 lux, resolution 1920x1080 pixels, and storage format DICOM3.0. The i-Scan image parameters are: surface enhancement +3, contrast enhancement +2, and color tone enhancement r, g and v. The r channel: red component enhancement; the g channel: green component enhancement; and the v channel: brightness component enhancement. The resolution and storage format are the same as those of the white light image.

[0035] Step 102: After inserting the endoscope into the nasopharynx through the nasal cavity, take four non-overlapping images of each of the three key anatomical regions: the roof of the nasopharynx, the opening of the Eustachian tube, and the pharyngeal recess, to avoid angular deviations greater than 15°; i-Scan image acquisition adopts the process of "white light positioning → shortcut key to switch i-Scan mode → alternating white light and i-Scan confirmation" to ensure that no lesion area is missed;

[0036] Step 103: Calculate the mean edge gradient of the image using Python's OpenCV library. The formula is: mean edge gradient = Σ|pixel gray level difference| / total number of edge pixels. Calculate the proportion of artifact regions using an image segmentation algorithm, including the proportion of motion artifacts and reflection artifacts. Filter the images, retaining those with a mean edge gradient greater than 20, a motion artifact proportion less than 5%, and a reflection artifact proportion less than 3%. Each image must contain at least one key anatomical landmark, such as the opening of the Eustachian tube or the nasopharyngeal roof ridge.

[0037] Step 104: Classify the pathological results into 5 grades: Grade 1 is low-grade lymphoid hyperplasia of chronic mucosal inflammation, Grade 2 is high-grade lymphoid hyperplasia of chronic mucosal inflammation, Grade 3 is low-grade squamous metaplasia and dysplasia, Grade 4 is non-keratinizing squamous cell carcinoma or undifferentiated carcinoma, Grade 5 is lymphoma, and other rare lesions not included can be included through new grades.

[0038] Step 105: For each patient's endoscopic images, associate them with the corresponding biopsy pathology results and use image-pathology correspondence annotation. Each image is annotated with the corresponding pathological grade, lesion location, and visual features. The lesion location is such as the roof of the nasopharynx, and the visual features are such as whether there is submucosal vascular SV tortuosity or dendritic vascular BV disorder.

[0039] Step 106: Collect case data from at least 3 tertiary hospitals, with a total sample size of ≥1000 cases. Each case contains 4 white light images and 3 i-Scan images. Divide the dataset into training set, validation set, and test set in a 7:2:1 ratio to ensure that the sample proportion of each pathological grade is consistent in the three groups, such as 20% for grade 1 in the training set and 20% for grade 1 in the validation set.

[0040] When using this method, please refer to steps 101 to 106:

[0041] By simultaneously acquiring white light images and i-Scan images of the nasopharynx and rigorously screening images that meet quality standards, a multi-center, multi-modal dataset was constructed, ensuring the comprehensiveness and representativeness of the data. Corresponding pathological grading annotations provided accurate supervision signals for subsequent model training. This not only improved the reliability of image analysis but also reduced human error through standardized acquisition procedures, laying a high-quality data foundation for intelligent classification and recognition, and significantly enhancing the model's generalization ability and diagnostic accuracy.

[0042] Step 2: Denoise the white light image and i-Scan image respectively, extract key landmarks and complete image registration, divide the mucosal surface area, submucosal vascular area and lymphoid tissue rich area by semantic segmentation, and perform feature enhancement for different areas;

[0043] Step two includes the following:

[0044] Step 201: Use an adaptive median filtering algorithm to denoise the white light image and dynamically adjust the window size. When the noise density is ≤10%, the window is 3×3; when the noise density is 10%~20%, the window is 5×5. Noise density = the percentage of pixels whose gray values ​​deviate from the mean by more than 2 standard deviations.

[0045] Step 202: The i-Scan image is denoised using a wavelet transform denoising algorithm. The wavelet basis function is selected as db4, and the decomposition layer is 3. The image is decomposed into low-frequency approximation components and high-frequency detail components. Soft thresholding is applied to the high-frequency detail components. Soft threshold = 0.02 × maximum image gray value to suppress motion artifacts while preserving microvascular details.

[0046] Step 203: Using the Harris corner detection algorithm, with a corner response function threshold of 0.04 and a non-maximum suppression window of 3×3, extract 8 to 12 key landmarks in each white light image and the corresponding i-Scan image, such as the edge of the Eustachian tube opening and the apex of the nasopharyngeal roof.

[0047] Step 204: Use the improved SIFT algorithm with a feature point matching distance threshold of 200. Combine this with the Random Sampling Consensus (RANSAC) algorithm to remove mismatched points. Points with a matching error greater than 2 pixels are considered mismatched. Map the i-Scan image to the white light image coordinate system through affine transformation. The registration error must be ≤1 pixel. For the registered image, calculate the gray-level correlation, i.e., the Pearson correlation coefficient. A Pearson correlation coefficient ≥0.85 is considered a successful registration. For unsuccessful images, repeat step 203.

[0048] Step 205: Using the U-Net semantic segmentation model, the encoder is a VGG16 pre-trained weight, the decoder is a deconvolutional layer, and the loss function is Dice loss, the registered image is automatically divided into three regions, including the mucosal surface region, the submucosal vascular region, and the lymphoid tissue enrichment region. The mucosal surface region mainly reflects surface roughness and ulceration features; the submucosal vascular region mainly includes SV and BV; and the lymphoid tissue enrichment region mainly reflects lymphoid hyperplasia.

[0049] Step 206: Apply contrast-limited adaptive histogram equalization (CLAHE) to the mucosal surface area, with a contrast limit parameter of 2.0 and a grid size of 8×8, to enhance surface texture differences; apply gamma correction + Frangi vascular filtering to the submucosal vascular area, with a gamma value of 0.8, σ range of 1~3, β1=0.5, and β2=20, to highlight the grayscale difference between blood vessels and surrounding tissues; apply Gaussian blur + Laplacian operator to the lymphatic tissue-rich area, with Gaussian blur: standard deviation = 1.0, and Laplacian operator: 3×3 kernel, to suppress noise while enhancing tissue boundaries;

[0050] It should be noted that gamma correction is an algorithm that adjusts image brightness through nonlinear grayscale mapping. When the gamma value is less than 1, it can brighten dark areas of the image. Submucosal blood vessels in the nasopharynx, because they are covered by the mucosa, often appear as dark details in i-Scan images. A gamma value of 1... 0.8 can specifically brighten vascular areas; σ is a scale parameter, corresponding to the size of the filter kernel, directly matching the microvessels of the nasopharynx, ensuring that microvessels of different thicknesses can be detected in a 512×512 pixel image; β1 is the flat structure suppression weight, used to distinguish between blood vessels (tubular) and background (flat); the smaller the value, the stronger the suppression of flat structures. β1=0.5 can effectively suppress flat backgrounds such as mucosal tissue; β2 is the spherical structure suppression weight, used to distinguish between blood vessels (tubular) and noise points (spherical); the larger the value, the stronger the suppression of spherical noise. β2=20 can specifically filter point noise generated by lymphatic tissue, highlighting the slender tubular features of blood vessels; Gaussian blur is an algorithm that smooths the image using a Gaussian function, determining the degree of blur; the Laplacian operator is an algorithm that detects edges by calculating the second derivative of pixel grayscale, and the kernel size determines the precision of edge detection; the 3×3 kernel is the smallest effective edge detection kernel, which can accurately capture the subtle boundaries between lymphatic tissue and surrounding mucosa;

[0051] Step 207: Scale all enhanced images to 512×512 pixels, and use bilinear interpolation to avoid distortion, normalizing pixel values ​​from [0, 255] to [0, 1], using the following formula: ,in, I represents the normalized pixel value, and I represents the original pixel value. , These are the minimum and maximum gray values ​​of a single image, respectively. A batch normalization algorithm is used to correct the gray distribution of images acquired by different endoscopic devices to ensure that the gray distribution deviation of images of the same pathological type is ≤5%. The parameters are set as follows: moving average coefficient = 0.99, epsilon = 1e-5.

[0052] When using this method, refer to steps 201 to 207:

[0053] Noise reduction was achieved for white light and i-Scan images using adaptive median filtering and wavelet transform, respectively, effectively preserving key details and suppressing artifacts. Precise registration was achieved using Harris corner detection and an improved SIFT algorithm, ensuring spatial consistency across multimodal images. Semantic segmentation based on U-Net divided the images into mucosal surface, vascular, and lymphoid tissue enrichment regions. Enhancement techniques such as CLAHE and Frangi filtering were applied to different regions, significantly improving the contrast and recognizability of lesion features. This laid a high-quality image foundation for subsequent feature extraction and classification, while also enhancing the model's sensitivity and robustness to subtle lesions.

[0054] Step 3: Extract morphological and texture features from the enhanced image, calculate feature importance weights, and fuse them into a low-dimensional feature vector;

[0055] Step three includes the following:

[0056] Step 301: For the i-Scan image, the Zhang-Suen thinning algorithm is used to extract the central axis of the blood vessels in the submucosal vascular region. The iteration termination condition is: no pixel meets the deletion rule. The number of nodes connecting ≥3 adjacent pixels in the vascular skeleton is counted as the number of vascular branches. The vascular skeleton is segmented, and the length of each segment is ≥5 pixels. After fitting a straight line for each segment, the curvature is calculated. The standard deviation of the curvature of all segments is taken as the vascular tortuosity. A standard deviation ≥0.8 indicates a possible malignancy. Curvature = 1 / radius, and the radius is the radius of the circumcircle of the fitted straight line. The number of breaks in the vascular skeleton with a continuous pixel spacing ≥2 pixels is counted as the number of interruptions. A number ≥3 indicates a possible malignancy.

[0057] Step 302: Extract lesion boundaries using Canny edge detection + Snakes active contour model. Canny edge detection: high threshold = 0.2, low threshold = 0.1. Snakes active contour model: elastic coefficient = 0.1, tension coefficient = 0.1, iteration count 50. Calculate the ratio of boundary length to the perimeter of the minimum bounding rectangle of the lesion as boundary irregularity. Boundary irregularity ≥ 0.7 suggests malignancy. Calculate the mean gray-level gradient of boundary pixels as boundary clarity. Gradient mean ≥ 50 suggests clear boundaries, mostly benign.

[0058] Step 303: For white light images, construct gray-level co-occurrence matrices in the mucosal surface area and the lymphoid tissue enrichment area, respectively, with distance = 1 and angles = 0°, 45°, 90°, and 135°. Calculate the contrast, correlation, and entropy of the gray-level co-occurrence matrices at the four angles. Use the Local Binary Pattern (LBP) operator with radius = 1 and neighborhood pixel count = 8 to statistically analyze the bin features of the LBP histogram. A bin value ratio ≥ 70% in the low gray-level range indicates benignity, while dispersed bin values ​​in the high gray-level range indicate malignancy.

[0059] Step 304: Based on the training set data, calculate the Spearman correlation coefficient between each feature and the pathological grade. The larger the absolute value, the higher the importance of the feature. The attention weight = absolute value of the correlation coefficient / sum of absolute values ​​of the correlation coefficients of all features. The sum of weights = 1. After multiplying each feature by the corresponding attention weight, they are concatenated into a fusion feature vector. Morphological features include the number of branches, distortion, number of interruptions, boundary irregularity, and boundary clarity. Texture features include contrast, correlation, entropy value, and local binary pattern features.

[0060] Step 305: Using the minimum redundancy maximum correlation algorithm, calculate the redundancy and correlation of each feature in the fused feature vector, and remove features with redundancy ≥ redundancy threshold (e.g., 0.8) and correlation ≤ correlation threshold (e.g., 0.3). Redundancy: Pearson correlation coefficient with other features, correlation: Spearman correlation coefficient with pathological grade. Using the principal component analysis algorithm, retain the principal components with a cumulative variance contribution rate ≥ 95% to obtain the low-dimensional feature vector.

[0061] When using this method, refer to steps 301 to 305:

[0062] By extracting morphological features such as the number of vascular branches, tortuosity, and number of interruptions, as well as texture features such as contrast and entropy, and combining Spearman correlation coefficient to calculate feature importance weights, key indicators strongly correlated with pathological grading are effectively screened out. The minimum redundancy maximum correlation algorithm and principal component analysis are used for dimensionality reduction to remove redundant features while retaining more than 95% of the information, generating low-dimensional feature vectors. This significantly improves the representativeness of the features and classification efficiency, providing highly discriminative input data for subsequent model construction. At the same time, it enhances the model's ability to capture subtle differences in lesions, ultimately improving the accuracy and robustness of classification.

[0063] Step 4: Construct a binary classification model for benign and malignant diseases and a multi-classification model for pathological types using a hybrid network of support vector machine and CNN-LSTM, respectively. Combine the physician's diagnostic rule base and validation indicators to output the classification results.

[0064] Step four includes the following:

[0065] Step 401: A first-layer binary classification model for benign and malignant diseases is constructed using a support vector machine (SVM). The kernel function is a radial basis function (RBF). The parameters are determined by grid search on the validation set: gamma=0.1, C=10. The input is a low-dimensional feature vector, and the output is benign or malignant and the classification confidence score. Benign diseases correspond to pathological grades 1-2, and malignant diseases correspond to pathological grades 3-5. The confidence score is 0-1, with ≥0.8 indicating high confidence and <0.8 indicating low confidence. A second-layer multi-class classification model for pathological types is constructed using a CNN-LSTM hybrid network. The CNN is ResNet18, and the input is a white light image and an i-Scan image, outputting 256-dimensional spatial features. The LSTM is a two-layer bidirectional LSTM with 128 hidden units, outputting 128-dimensional temporal features. The fully connected layer outputs the probability of 5 pathological grades. The input is a low-dimensional feature vector, a white light image, and an i-Scan image. The pathological type with the highest probability is output as the initial prediction result.

[0066] Step 402: Use the Adam optimizer with an initial learning rate of 0.001. Halve the learning rate if the validation set accuracy does not improve for three consecutive rounds. Use the cross-entropy loss function for the binary classification model and the weighted cross-entropy loss function for the multi-class model. The weights are equal to the total number of samples / (number of pathological type samples × 5). Use Dropout + L2 regularization to avoid overfitting. The Dropout probability is 0.3 and it is only applied to the fully connected layers. The L2 regularization coefficient is 0.001. Train the binary classification model until the validation set accuracy is ≥88%. Use the output of the binary classification model as the auxiliary input of the multi-class model with a weight of 0.2. Jointly train the multi-class model until the validation set multi-class accuracy is ≥85%.

[0067] Step 403: Construct a physician diagnostic rule base: Rule 1: If the tortuosity of the blood vessel is ≥0.8 and the number of interruptions is ≥3, it is judged as grade 4 nasopharyngeal carcinoma; Rule 2: If the boundary irregularity is ≥0.7 and the gray-level co-occurrence matrix entropy value is ≥1.5, it is judged as grade 3 dysplasia; Rule 3: If the number of blood vessel branches is ≤5 and the boundary clarity is ≥50, it is judged as grade 1~2 benign inflammation, with 0 interruptions for grade 1, otherwise grade 2; If the binary classification confidence is ≥confidence threshold (e.g., 0.8) and the multi-classification result is consistent with the rule base judgment, the classification result is directly output; If the binary classification confidence is <0.8 or the multi-classification result is inconsistent with the rule base judgment, manual review is initiated, and the model prediction result + rule base judgment result is output;

[0068] Step 404: Use the paired four-fold table method to evaluate the sensitivity (≥88%), specificity (≥91%), accuracy (≥91%), and Kappa value (≥0.62) of the binary classification model, and evaluate the accuracy (≥85%) and Kappa value (≥0.7) of the multi-class classification model; use data from 3 hospitals that did not participate in the training for multi-center validation to ensure that the accuracy deviation is ≤3%; use 5-fold cross-validation to ensure that the standard deviation of the accuracy of 5 validations is ≤3% to verify the stability of the model.

[0069] When using this method, please refer to the content of steps 401 to 403:

[0070] A two-layer classification model was constructed using a hybrid Support Vector Machine (SVM) and CNN-LSTM network. Combined with a physician diagnostic rule base, this model achieved high-precision binary classification of benign and malignant tumors and multi-class classification of pathological types. SVM rapidly screens for benign and malignant tumors based on low-dimensional feature vectors, while CNN-LSTM fuses image and feature data to enhance fine-grained classification capabilities. The rule base provides interpretable decision support. Manual review is triggered when the model's confidence is insufficient or conflicts with rules, ensuring the reliability of the results. Multi-center validation and cross-validation demonstrate the model's stability and significantly improve diagnostic efficiency and consistency.

[0071] In the application, the various formulas mentioned are all calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The coefficients in the formulas are set by those skilled in the art according to the actual situation.

[0072] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, and combinations thereof. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0074] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent classification and recognition of images of ear, nose, and throat lesions, characterized in that: include: Simultaneously acquire nasopharyngeal white light images and i-Scan images, filter images that meet quality standards, and associate them with pathological and grading annotations to construct a dataset containing multi-center cases; Denoising was performed on the white light image and the i-Scan image respectively, key landmarks were extracted and image registration was completed, and the mucosal surface area, submucosal vascular area and lymphoid tissue-rich area were divided by semantic segmentation, and feature enhancement was performed on different areas. Morphological and texture features are extracted from the enhanced image, and feature importance weights are calculated and fused into a low-dimensional feature vector. Specifically, for i-Scan images, the Zhang-Suen thinning algorithm is used to extract the central axis of the submucosal vascular region. The number of nodes connecting ≥3 adjacent pixels in the vascular skeleton is counted as the number of vascular branches. The vascular skeleton is segmented, and the curvature of each segment is calculated after fitting a straight line. The standard deviation of the curvature of all segments is taken as the vascular tortuosity. The number of breaks in the vascular skeleton with a continuous pixel spacing ≥2 pixels is counted as the number of interruptions. The Canny edge detection + Snakes active contour model is used to extract the lesion boundary. The ratio of the boundary length to the perimeter of the minimum bounding rectangle of the lesion is calculated as the boundary irregularity. The boundary pixel is calculated. The mean of the gray-level gradient is used as the boundary sharpness. For white light images, gray-level co-occurrence matrices are constructed in the mucosal surface area and the lymphoid tissue enrichment area, respectively. The contrast, correlation, and entropy of the gray-level co-occurrence matrices are calculated. The bin features of the local binary pattern histogram are statistically analyzed using the local binary pattern operator. Based on the training set data, the Spearman correlation coefficient between each feature and the pathological grade is calculated. Each feature is multiplied by the corresponding attention weight and then concatenated into a fused feature vector. The dimensionality is reduced by the minimum redundancy maximum correlation algorithm and principal component analysis to obtain a low-dimensional feature vector. Morphological features include the number of branches, distortion, number of interruptions, boundary irregularity, and boundary sharpness. Texture features include contrast, correlation, entropy, and bin features of the local binary pattern. A binary classification model for benign and malignant diseases and a multi-classification model for pathological types were constructed using a support vector machine and a CNN-LSTM hybrid network, respectively. The classification results were output by combining the physician's diagnostic rule base and validation indicators.

2. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 1, characterized in that: Endoscopic equipment was used to simultaneously acquire white light images and i-Scan images of the nasopharynx. The white light group had no enhancement mode. The i-Scan group image parameters included surface enhancement, contrast enhancement, and hue enhancement. At least four non-overlapping images were taken from each of the three anatomical regions: the roof of the nasopharynx, the opening of the Eustachian tube, and the pharyngeal recess. The i-Scan image acquisition process used white light positioning, shortcut key switching of i-Scan mode, and alternating confirmation of white light and i-Scan images. Images were selected based on the mean edge gradient, the proportion of motion artifacts, and the proportion of reflection artifacts, and each image had to contain at least one key anatomical landmark.

3. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 2, characterized in that: The pathological results were classified into 5 grades: Grade 1 was low-grade lymphoid hyperplasia of chronic mucosa, Grade 2 was high-grade lymphoid hyperplasia of chronic mucosa, Grade 3 was low-grade squamous metaplasia and dysplasia, Grade 4 was non-keratinizing squamous cell carcinoma or undifferentiated carcinoma, and Grade 5 was lymphoma.

4. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 1, characterized in that: An adaptive median filtering algorithm is used to denoise the white light image, and a wavelet transform denoising algorithm is used to denoise the i-Scan image. The image is decomposed into low-frequency approximation components and high-frequency detail components. Soft thresholding is applied to the high-frequency detail components. The soft threshold is 0.02 × the maximum gray value of the image. The Harris corner detection algorithm is used to extract key marker points in each white light image and the corresponding i-Scan image. An improved SIFT algorithm is adopted, combined with a random sampling consistency algorithm to remove mismatched points. Points with a matching error greater than 2 pixels are considered mismatches. The i-Scan image is mapped to the white light image coordinate system through affine transformation. For the registered image, the gray-level correlation is calculated. A Pearson correlation coefficient ≥ 0.85 is considered as qualified registration.

5. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 4, characterized in that: The U-Net semantic segmentation model was used to automatically divide the registered image into three regions: the mucosal surface region, the submucosal vascular region, and the lymphoid tissue enrichment region. Contrast-limited adaptive histogram equalization was applied to the mucosal surface region, gamma correction and Frangi vascular filtering were applied to the submucosal vascular region, and Gaussian blur and Laplacian operator were applied to the lymphoid tissue enrichment region. Bilinear interpolation algorithm was used to scale all enhanced images, normalizing pixel values ​​from [0, 255] to [0, 1]. Batch normalization algorithm was used to correct the grayscale distribution of images acquired by different endoscopic devices.

6. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 1, characterized in that: A first-layer binary classification model for benign and malignant diseases is constructed using a support vector machine. The input is a low-dimensional feature vector, and the output is benign or malignant and the classification confidence. A second-layer multi-class classification model for pathological types is constructed using a CNN-LSTM hybrid network. The input is a low-dimensional feature vector, a white light image, and an i-Scan image, and the output is the probability of five pathological grades.

7. The intelligent classification and recognition method for images of ear, nose, and throat lesions according to claim 6, characterized in that: Construct a physician diagnostic rule base: Rule 1: If the tortuosity of blood vessels is ≥0.8 and the number of interruptions is ≥3, it is judged as grade 4 nasopharyngeal carcinoma; Rule 2: If the boundary irregularity is ≥0.7 and the gray-level co-occurrence matrix entropy value is ≥1.5, it is judged as grade 3 dysplasia; Rule 3: If the number of blood vessel branches is ≤5 and the boundary clarity is ≥50, it is judged as grade 1~2 benign inflammation; If the binary classification confidence is ≥confidence threshold and the multi-classification result is consistent with the rule base judgment, the classification result is directly output. Conversely, if the result is not satisfactory, the request will be sent for review.

Images