Adenoid surgery indication evaluation method based on multi-modal deep learning
By integrating nasal endoscopic images, respiratory audio, and clinical data through multimodal deep learning, the shortcomings of single-modal assessment in existing technologies are overcome, enabling objective and quantifiable assessment of indications for adenoidectomy in children and improving the adaptability and accuracy of the model.
Patent Information
- Application Number
- CN202611334085.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
AI Technical Summary
Existing methods for assessing indications for pediatric adenoid surgery rely excessively on a single image modality or the physician’s subjective experience, making it difficult to simultaneously utilize endoscopic images, respiratory audio, and structured clinical data. This results in insufficient adaptability to variations in imaging, patient cooperation, and sample imbalance.
A multimodal deep learning approach was adopted, which integrates six-view nasal endoscopy images, respiratory audio data and clinical structured data. The perspective association was established by pre-setting anatomical relationships, and the importance of anatomical association branches was dynamically adjusted by using image quality parameters and audio features to construct a multimodal risk prediction model.
It provides an objective and quantifiable auxiliary assessment tool that can reduce the interference of low-quality images on the results, improve the recognition ability under small sample and class imbalance conditions, output the probability assessment of the indications for pediatric adenoid surgery, and support doctors' case review and risk management.
Smart Images

Figure CN123120310A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical image processing, audio signal processing, clinical structured data processing, and deep learning technology, and in particular to a method for assessing indications for adenoidectomy in children based on multimodal deep learning. Background Technology
[0002] The adenoids are lymphatic tissue located at the junction of the roof and posterior wall of the nasopharynx, playing an important role in the child's immune system. Enlarged adenoids in children can cause nasal congestion, mouth breathing, snoring during sleep, shortness of breath, hearing loss, and difficulty concentrating. In severe cases, it can affect sleep quality and maxillofacial development. For some children, adenoidectomy or adenoidectomy and tonsillectomy is an important treatment; however, whether surgery is indicated usually requires a comprehensive assessment of multiple factors, including imaging findings, symptom severity, sleep apnea, and underlying physical signs.
[0003] Current assessment methods primarily rely on nasal endoscopy, lateral X-ray films, multiplanar CT reconstruction, or physician experience. Nasal endoscopy provides direct visualization of the nasopharyngeal structures, but its acquisition is easily affected by the child's cooperation, the shooting angle, local reflections, obstruction by secretions and air bubbles, and the physician's subjective interpretation experience. Lateral X-ray films suffer from two-dimensional overlapping imaging, making it difficult to fully present the spatial structure of the nasopharynx; CT scans provide rich spatial information, but radiation exposure and cost issues generally make them unsuitable as routine screening methods for children.
[0004] In recent years, deep learning techniques such as convolutional neural networks, spatial transformation networks, and attention mechanisms have been applied to medical image analysis. However, existing intelligent recognition schemes for adenoid hypertrophy in children often focus on a single image modality, failing to fully utilize respiratory audio and clinical structured features. A single image modality is easily limited by image quality, shooting angle, and sample size, and it is also difficult to fully reflect changes in respiratory sounds caused by airway obstruction, as well as differences in symptoms and signs in children. In addition, the clarity, reflectivity, exposure, and effective visible area of nasal endoscopy images from different angles may vary. If fixed weights are applied to image features from different angles or if images are directly stitched together, low-quality images may interfere with the fusion results. Existing multimodal fusion methods typically struggle to simultaneously utilize the anatomical correspondences between different nasal endoscopy angles, and also find it difficult to adaptively determine the importance of image features in different anatomical regions based on airflow changes reflected in respiratory audio.
[0005] Therefore, a method for assessing indications for adenoidectomy in children is needed, which integrates six-view nasal endoscopic images, respiratory audio data, and clinical structured data. This method can establish the correlation between different nasal endoscopic views based on preset anatomical relationships, reduce the impact of low-quality images on the fusion results using image quality parameters, and dynamically adjust the importance of different anatomical correlation branches based on audio feature vectors, thereby providing objective and quantifiable auxiliary assessment information for indications for adenoidectomy in children. Summary of the Invention
[0006] The purpose of this invention is to address the problem that existing methods for assessing indications for pediatric adenoidectomy rely excessively on a single image modality or the physician's subjective experience, making it difficult to simultaneously utilize nasal endoscopic images, respiratory audio, and structured clinical data. This results in insufficient adaptability to variations in imaging, patient cooperation, and imbalanced sample types. This invention proposes a multimodal deep learning-based method for assessing indications for pediatric adenoidectomy. The method outputs a risk probability, which can serve as auxiliary information for physicians in case review, risk stratification, and follow-up management, avoiding the direct use of model outputs as diagnostic or treatment conclusions.
[0007] The specific technical solution for achieving the objective of this invention is as follows:
[0008] A method for assessing indications for adenoidectomy in children based on multimodal deep learning includes the following steps:
[0009] Step 1: Multimodal data acquisition and preprocessing
[0010] During the model training phase, multiple historical cases were acquired. Each historical case included six-view nasal endoscopy images, respiratory audio data, clinical structured data, and surgical or non-surgical category labels for the same subject. The six-view nasal endoscopy images were preprocessed, and corresponding image quality parameters were calculated based on the sharpness, reflectivity, exposure, and effective visible area of each image. Spatial correction was performed on the preprocessed nasal endoscopy images, and image coding features corresponding to each viewpoint were obtained through an image feature extraction network. Sound pressure level, low-frequency energy ratio, and Mel-frequency cepstral coefficients were extracted from the respiratory audio data to obtain audio feature vectors. The clinical structured data underwent category coding, missing value imputation, and standardization to obtain clinical feature vectors.
[0011] Step 2: Construction of Anatomical Related Branches and Extraction of Multimodal Features
[0012] Pre-defined anatomical relationships: The image coding features of the right nasal cavity and the right adenoid are combined to obtain the right airway branch features; the image coding features of the left nasal cavity and the left adenoid are combined to obtain the left airway branch features; the right airway branch features, the left airway branch features, and their differential features are combined to obtain the bilateral airway associated branch features; the image coding features of the larynx and the pharynx are combined to obtain the pharyngeal branch features; these four branch features are collectively referred to as anatomically associated branch features; the branch quality parameters are determined based on the image quality parameters of the images corresponding to each branch; the audio feature vectors are input into the audio encoder to obtain the audio coding features; the clinical feature vectors are input into the clinical encoder to obtain the clinical coding features;
[0013] Step 3: Multimodal model training and parameter optimization
[0014] An audio-guided dynamic fusion approach is adopted. Based on anatomical association branch features, branch quality parameters, and audio coding features, dynamic fusion weights for anatomical association branches are calculated. These weights are then used to weight and fuse the anatomical association branch features to obtain structural-acoustic association features. The structural-acoustic association features, audio coding features, and clinical coding features are combined to obtain a comprehensive feature vector. The comprehensive feature vector is used to calculate the prediction results for whether historical cases belong to surgical or non-surgical categories. A loss function is used to calculate the difference between the predicted category and the true category label of the historical case, and the model parameters are adjusted to obtain a trained multimodal risk prediction model.
[0015] Step 4: Surgical indication assessment and output
[0016] Acquire six-view nasal endoscopy images, respiratory audio data, and clinical structured data of the test subjects, and preprocess them according to the processing method described in step one; input the preprocessed data into the trained multimodal risk prediction model to obtain the assessment score of whether the test subject belongs to the surgical or non-surgical category; normalize the assessment score to obtain the probability that the child's adenoids need surgery.
[0017] Furthermore, the six-view nasal endoscopy images include laryngeal view images, pharyngeal view images, right nasal cavity view images, right adenoid view images, left nasal cavity view images, and left adenoid view images. The view images are input in a preset view order to establish the association between the nasal endoscopy images and the corresponding anatomical locations.
[0018] Furthermore, in step one, the six-view nasal endoscope images are preprocessed, including scaling each view image to a preset size and normalizing it according to a preset mean and a preset standard deviation; determining the sharpness based on the brightness variation of the image, determining the reflection status based on the brightness and color saturation, determining the exposure based on the average brightness, and determining the effective visible area based on the pixel ratio within the effective brightness range, thereby obtaining the corresponding image quality parameters.
[0019] Further, in step one, the extraction process of the image coding features for each viewpoint specifically includes: using a spatial transformation module to predict the affine transformation parameters of the nasal endoscope images for each viewpoint, and performing spatial correction on the nasal endoscope images for each viewpoint according to the affine transformation parameters; inputting the spatially corrected nasal endoscope images for each viewpoint into a convolutional feature extraction module with shared network parameters, and obtaining the deep image features corresponding to each viewpoint through convolution operations and global average pooling; performing linear transformation, layer normalization, and activation processing on the deep image features to obtain the image coding features corresponding to each viewpoint; combining the image coding features for each viewpoint according to a preset anatomical relationship to obtain the right airway branch features, left airway branch features, bilateral airway associated branch features, and pharyngeal branch features.
[0020] Furthermore, the audio feature vector includes sound pressure level features, low-frequency energy ratio features, and Mel frequency cepstral coefficient features; the audio feature vector is input into the audio encoder to obtain audio coding features; the audio encoder includes a fully connected layer, a batch normalization layer, an activation layer, and a random deactivation layer.
[0021] Furthermore, the clinical structured data includes gender, age, height, weight, body mass index, method of conception, method of delivery, whether premature birth occurred, and symptom information; the clinical structured data is subjected to categorical variable encoding, missing value imputation, and standardization to obtain a clinical feature vector; the clinical feature vector is input into a clinical encoder to obtain clinical coding features; the clinical encoder includes a fully connected layer, a batch normalization layer, an activation layer, and a random deactivation layer.
[0022] Furthermore, the training process of the multimodal risk prediction model described in step three specifically includes: dividing historical cases with surgical-related or non-surgical-related category labels into training data and validation data; in the first training phase, freezing the backbone network parameters of the image feature extraction module, and training the spatial transformation module, anatomical association branch module, audio encoder, clinical encoder, dynamic weight calculation module, and risk probability calculation network; in the second training phase, unfreezing the backbone network parameters and fine-tuning the multimodal risk prediction model as a whole; setting category loss weights according to the number of samples in each category in the training data, adjusting the model parameters using the category-weighted cross-entropy loss function, and saving the model parameters based on the evaluation results of the validation data.
[0023] Beneficial effects
[0024] 1) Multimodal fusion: This invention unifies six-view nasal endoscopy images, audio feature vectors and clinical feature vectors into the same deep learning framework, which can comprehensively form auxiliary assessment results from three dimensions: morphology, acoustics and clinical symptoms.
[0025] 2) Image processing: This invention performs spatial correction on nasal endoscope images through a spatial transformation module to reduce the impact of differences in shooting angle, scaling, and framing; it utilizes the constructed preset anatomical relationships: right airway branch, left airway branch, bilateral airway associated branches, and pharyngeal branches to take advantage of the anatomical relationships between different nasal endoscope perspectives; it calculates the dynamic fusion weight of each anatomical associated branch based on image quality parameters and audio coding features, which helps to reduce the interference of low-quality images on the fusion results, and dynamically adjusts the importance of different anatomical associated branches according to the airflow changes reflected by breathing sounds.
[0026] 3) Improving training stability under conditions of small sample size and class imbalance: This invention employs a phased training strategy. First, the backbone parameters of the image feature extraction network are fixed, and the anatomical association branch, audio encoder, clinical encoder, and dynamic weight calculation module are trained. Then, the backbone parameters are unfrozen, and the overall model is jointly fine-tuned. Simultaneously, class loss weights are set according to the number of surgical and non-surgical cases, and the learning rate is dynamically adjusted, thereby improving the recognition ability of minority class samples and reducing the model's prediction bias towards the majority class.
[0027] 4) Quantitative presentation of results: This invention outputs assessment scores indicating whether the subjects to be evaluated belong to the surgery-related category or the non-surgery-related category, and converts these assessment scores into probabilities of indications for adenoidectomy in children. These probabilities provide quantitative auxiliary information for physicians in case review, risk stratification, and follow-up management. Attached Figure Description
[0028] Figure 1 This is a flowchart of the present invention;
[0029] Figure 2 This is an anatomical branch diagram of the present invention;
[0030] Figure 3 This is the audio-guided dynamic fusion diagram of the present invention. Detailed Implementation
[0031] See Figure 1 A method for assessing indications for adenoidectomy in children based on multimodal deep learning, specifically including:
[0032] 1) Multimodal data acquisition
[0033] Using subjects as the basic data unit, six-view nasal endoscopy images, respiratory audio data, and clinical structured data were acquired. During the model training phase, surgery-related category labels for each historical case were also obtained; these category labels included both surgery-related and non-surgery-related categories.
[0034] The six-view nasal endoscopy images include images from the larynx, pharynx, right nasal cavity, right adenoid, left nasal cavity, and left adenoid. These images are input into the model in the aforementioned preset order to establish a correlation between the images and their corresponding anatomical locations.
[0035] Sound pressure level, low-frequency energy ratio of 0–200 Hz, low-frequency energy ratio of 0–250 Hz, low-frequency energy ratio of 0–400 Hz, and cepstral coefficients of Mel frequencies from the first to the thirteenth order were extracted from the respiratory audio data of the subjects during calm breathing or sleep to form a 17-dimensional audio feature vector.
[0036] The structured clinical data includes gender, age, height, weight, body mass index, method of conception, method of delivery, whether premature birth occurred, and symptom information such as snoring and mouth breathing, nasal congestion and runny nose, sneezing and rubbing the nose, nasal obstruction, ear fullness and hearing loss, and poor concentration. In one embodiment, the above information constitutes a 14-dimensional clinical feature vector.
[0037] 2) Data preprocessing and image quality assessment
[0038] The endoscopic images of the nose from each viewpoint are scaled to 224×224 pixels, and the three RGB channels are normalized according to preset mean and preset standard deviation. In one embodiment, the preset mean values for the three RGB channels are 0.485, 0.456, and 0.406, and the preset standard deviations are 0.229, 0.224, and 0.225.
[0039] During the training phase, random horizontal flipping and color perturbation can be performed, and random augmentation does not change the anatomical viewpoint to which the image belongs. Random augmentation is not performed during the validation and inference phases.
[0040] Image quality parameters are calculated based on the sharpness, reflectivity, exposure, and effective visible area of endoscopic images from various viewpoints:
[0041]
[0042] in, Indicates the first Image quality parameters for images from each viewpoint; This represents the sharpness parameter obtained based on the brightness variation of adjacent pixels; This indicates the proportion of pixels with high brightness and low saturation. This represents the exposure parameters obtained based on the average brightness of the image; This indicates the percentage of pixels within the preset effective brightness range. A higher image quality parameter indicates higher image quality.
[0043] In an alternative embodiment, local reflections, secretions, or bubble-occluded areas can be detected based on the HSV color space, and the impact of these areas on image feature extraction can be reduced through morphological processing, area filtering, and local blurring.
[0044] Missing values were imputed and numerical formats were standardized for audio feature vectors. Clinical structured data underwent categorical coding, missing value imputation, and standardization. Standardization parameters were determined only from training data and applied to validation, test, and evaluation data to avoid data leakage.
[0045] 3) Construction of multimodal risk prediction model
[0046] The multimodal risk prediction model includes a spatial transformation module, an image feature extraction module, an anatomical association branch module, an audio encoder, a clinical encoder, a dynamic weight calculation module, and a risk probability calculation network.
[0047] 3-1 Image Feature Extraction and Anatomical Association Branch Construction
[0048] The spatial transformation module is used to predict the affine transformation parameters of images from each viewpoint, and to perform spatial correction on the nasal endoscope images from each viewpoint based on the affine transformation parameters, so as to reduce the impact of differences in shooting angle, scaling and framing.
[0049] The spatially corrected six-view nasal endoscopy images are input into the image feature extraction module with shared network parameters. In one embodiment, the image feature extraction module uses the EfficientNet-B0 network. After convolution and global average pooling, 1280-dimensional deep image features are obtained from each view image, and then 128-dimensional image coding features are obtained through linear mapping, layer normalization, and activation processing.
[0050] See Figure 2 Four anatomically related branches are constructed according to the preset anatomical relationships.
[0051] The characteristics of the right airway branch are:
[0052]
[0053] The characteristics of the left airway branch are:
[0054]
[0055] The characteristics of bilateral airway associated branches are:
[0056]
[0057] The characteristics of the pharyngeal branches are:
[0058]
[0059] in, , , and These represent the image coding features corresponding to the right nasal cavity, right adenoid, left nasal cavity, and left adenoid, respectively. and These represent the image coding features corresponding to the larynx and pharynx, respectively; the symbol ";" indicates feature concatenation. , and This represents a feature mapping network.
[0060] The quality parameters of the right airway branch are the average of the image quality parameters of the right nasal cavity and the right adenoid; the quality parameters of the left airway branch are the average of the image quality parameters of the left nasal cavity and the left adenoid; the quality parameters of the bilateral airway associated branches are the average of the image quality parameters of the left and right airway branches; and the quality parameters of the pharyngeal branches are the average of the image quality parameters of the larynx and pharynx.
[0061] 3-2 Audio Coding and Clinical Coding
[0062] The 17-dimensional audio feature vector is input into the audio encoder to obtain 64-dimensional audio coding features. The processed 14-dimensional clinical feature vector is input into the clinical encoder to obtain 32-dimensional clinical coding features. Both the audio encoder and the clinical encoder include fully connected layers, batch normalization layers, activation layers, and random deactivation layers.
[0063] 3-3 Audio-guided dynamic fusion
[0064] See Figure 3 For the first Each anatomically related branch, its branch characteristics Audio coding features and branch quality parameters Input the branch scoring network to obtain the branch scores:
[0065]
[0066] in, This represents a branch scoring network consisting of a fully connected layer, an activation layer, a random deactivation layer, and a scoring output layer.
[0067] Using branch quality parameters as quality priors, the dynamic fusion weights of each anatomically related branch are calculated:
[0068]
[0069] in, Indicates the first The dynamic fusion weights of the four anatomical association branches are 1. A positive number set to prevent invalid values from occurring in logarithmic operations.
[0070] The features of the four anatomical association branches are weighted and fused according to dynamic fusion weights:
[0071]
[0072] in, This represents the structural-acoustic correlation characteristics.
[0073] By concatenating the structure-acoustic correlation features, audio coding features, and clinical coding features, a comprehensive feature vector is obtained:
[0074]
[0075] In one embodiment, the structure-acoustic correlation features are 128-dimensional, the audio coding features are 64-dimensional, the clinical coding features are 32-dimensional, and the comprehensive feature vector is 224-dimensional.
[0076] 3-4 Risk Probability Calculation
[0077] The comprehensive feature vector is input into the risk probability calculation network to obtain assessment scores for non-surgical related categories and surgical related categories. The two assessment scores are then normalized using the Softmax function.
[0078]
[0079] in, and These represent the assessment scores for the non-surgical related category and the surgical related category, respectively. and These represent the assessed probabilities for the non-surgical related category and the surgical related category, respectively. Assess the probability of indications for adenoidectomy in children.
[0080] 4) Model Training
[0081] A multimodal risk prediction model is trained using historical cases labeled with either surgical-related or non-surgical-related categories. In one embodiment, stratified five-fold cross-validation is performed on a per-subject basis to keep the proportion of cases in each compromise relatively stable.
[0082] The model employs a two-stage training strategy. In the first training stage, the backbone network parameters of the image feature extraction module are frozen, while the spatial transformation module, anatomical association branch module, audio encoder, clinical encoder, dynamic weight calculation module, and risk probability calculation network are trained. In the second training stage, the backbone network parameters are unfrozen, and the model is fine-tuned overall using a smaller learning rate.
[0083] The class loss weights are set according to the number of samples in each class, and the class-weighted cross-entropy loss function is used:
[0084]
[0085] in, This indicates the number of training samples in a batch. Indicates the first The true category of each training sample This represents the loss weight for the corresponding category. This represents the model's predicted probability of the true class.
[0086] The AdamW optimizer and adaptive learning rate adjustment strategy are used during training. The learning rate is reduced when the validation AUC does not improve over multiple consecutive training epochs; training is stopped when the validation AUC continues to fail to improve, and the model parameters corresponding to the highest validation AUC are saved.
[0087] 5) Surgical indication assessment and output
[0088] Six-view nasal endoscopy images, respiratory audio data, and clinical structured data of the subjects to be evaluated were acquired and preprocessed according to the same rules as in the model training phase.
[0089] The preprocessed data is input into the trained multimodal risk prediction model. The model sequentially performs image quality assessment, image spatial correction, image feature extraction, anatomical association branch construction, audio encoding, clinical encoding, audio-guided dynamic fusion, and risk probability calculation, outputting the probability of indication for pediatric adenoidectomy.
[0090] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.
Claims
1. A method for assessing indications for adenoidectomy in children based on multimodal deep learning, characterized in that, Includes the following steps: Step 1: Multimodal data acquisition and preprocessing During the model training phase, multiple historical cases were acquired. Each historical case included six-view nasal endoscopy images, respiratory audio data, clinical structured data, and surgical or non-surgical category labels for the same subject. The six-view nasal endoscopy images were preprocessed, and corresponding image quality parameters were calculated based on the sharpness, reflectivity, exposure, and effective visible area of each image. Spatial correction was performed on the preprocessed nasal endoscopy images, and image coding features corresponding to each view were obtained through an image feature extraction network. Sound pressure level, low-frequency energy ratio, and Mel frequency cepstral coefficients are extracted from the respiratory audio data to obtain an audio feature vector; the clinical structured data is subjected to category coding, missing value imputation, and standardization to obtain a clinical feature vector; Step 2: Construction of Anatomical Related Branches and Extraction of Multimodal Features Preset anatomical relationships: Combining the image coding features of the right nasal cavity with those of the right adenoid to obtain the right airway branch features; combining the image coding features of the left nasal cavity with those of the left adenoid to obtain the left airway branch features; combining the right airway branch features, the left airway branch features, and their differences to obtain the bilateral airway association branch features; combining the image coding features of the larynx with those of the pharynx to obtain the pharyngeal branch features; these four branch features are collectively referred to as anatomical association branch features. The branch quality parameters are determined based on the image quality parameters of the images corresponding to each branch; the audio feature vector is input into the audio encoder to obtain the audio coding features; the clinical feature vector is input into the clinical encoder to obtain the clinical coding features; Step 3: Multimodal model training and parameter optimization An audio-guided dynamic fusion approach is adopted. Based on anatomical association branch features, branch quality parameters, and audio coding features, dynamic fusion weights for anatomical association branches are calculated. These weights are then used to weight and fuse the anatomical association branch features to obtain structural-acoustic association features. The structural-acoustic association features, audio coding features, and clinical coding features are combined to obtain a comprehensive feature vector. The comprehensive feature vector is used to calculate the prediction results for whether historical cases belong to surgical or non-surgical categories. A loss function is used to calculate the difference between the predicted category and the true category label of the historical case, and the model parameters are adjusted to obtain a trained multimodal risk prediction model. Step 4: Surgical indication assessment and output Acquire six-view nasal endoscopy images, respiratory audio data, and clinical structured data of the test subject, and preprocess them according to the processing method described in step one; The preprocessed data is input into the trained multimodal risk prediction model to obtain the assessment score of whether the test subject belongs to the surgical or non-surgical category; the assessment score is normalized to obtain the probability that the child's adenoids need surgery.
2. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, The six-view nasal endoscopy images include laryngeal view images, pharyngeal view images, right nasal cavity view images, right adenoid view images, left nasal cavity view images, and left adenoid view images. The view images are input in a preset view order to establish the association between the nasal endoscopy images and the corresponding anatomical locations.
3. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, Step one involves preprocessing the six-view nasal endoscope images, including scaling each view image to a preset size and normalizing it according to a preset mean and a preset standard deviation; determining the sharpness based on the brightness variation of the image, determining the reflection status based on the brightness and color saturation, determining the exposure based on the average brightness, and determining the effective visible area based on the pixel ratio within the effective brightness range, thereby obtaining the corresponding image quality parameters.
4. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, In step one, the extraction process of the image coding features for each viewpoint specifically includes: using a spatial transformation module to predict the affine transformation parameters of the nasal endoscope images for each viewpoint, and performing spatial correction on the nasal endoscope images for each viewpoint according to the affine transformation parameters; inputting the spatially corrected nasal endoscope images for each viewpoint into a convolutional feature extraction module with shared network parameters, and obtaining the deep image features corresponding to each viewpoint through convolution operations and global average pooling; and performing linear transformation, layer normalization, and activation processing on the deep image features to obtain the image coding features corresponding to each viewpoint.
5. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, The audio encoder includes a fully connected layer, a batch normalization layer, an activation layer, and a random deactivation layer.
6. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, The clinical structured data includes gender, age, height, weight, body mass index, method of conception, method of delivery, whether premature birth occurred, and symptom information; the clinical encoder includes a fully connected layer, a batch normalization layer, an activation layer, and a random inactivation layer.
7. The method for assessing indications for adenoidectomy in children according to claim 1, characterized in that, The training process of the multimodal risk prediction model described in step three specifically includes: acquiring historical cases where surgery was known to have been performed; dividing surgical cases and non-surgical cases into training data and validation data; in the first training phase, freezing the backbone network parameters of the image feature extraction module, and training the spatial transformation module, anatomical association branch module, audio encoder, clinical encoder, dynamic weight calculation module, and risk probability calculation network; in the second training phase, unfreezing the backbone network parameters, setting the category loss weights according to the number of samples in each category in the training data, adjusting the model parameters using the category-weighted cross-entropy loss function, and saving the model parameters based on the evaluation results of the validation data.
