Infant ear deformity intervention prediction method, apparatus, device, medium, and product

Abstract

This invention relates to the field of deep learning technology and discloses a method, device, equipment, medium, and product for predicting interventions for infantile auricular deformities. The method includes: acquiring target auricular image data and target structured information of the target object; inputting the target auricular image data into a pre-built detection model to obtain the target auricular region; inputting the target auricular region into a pre-built classification model to obtain an auricular deformity recognition result; and inputting the target auricular region, the auricular deformity recognition result, and the target structured information into a pre-built intervention scoring model to obtain an intervention score. This invention employs a two-stage recognition framework combining a detection model and a classification model. It first automatically detects and locates the auricular region, and then identifies the type of auricular deformity. This effectively reduces background interference, improves the accuracy and robustness of recognition, and also introduces preoperative multimodal efficacy prediction, which can be effectively used for preoperative expectation management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a baby auricle deformity intervention prediction method, device, equipment, medium and product. BACKGROUND

[0002] Baby auricle deformity is a common congenital external ear development abnormality, and the clinical types include cup ear, wind ear, and ape ear. These deformities not only affect the appearance of the baby, but also may affect hearing, and even in some cases, accompanied by other physiological or psychological problems.

[0003] The current diagnosis and intervention technology mainly depends on the clinical observation and experience of doctors, but there are some obvious shortcomings. For example, the traditional visual inspection and manual measurement method may be affected by subjective factors, and different doctors may give different evaluation results. Secondly, although the existing imaging technology (such as ultrasonic wave or CT scan) can provide structural information, it is often difficult for doctors with insufficient experience to predict the effect after intervention. SUMMARY

[0004] The present application provides a baby auricle deformity intervention prediction method, device, equipment, medium and product to solve the problem that different doctors may give different evaluation results when discriminating and predicting intervention of baby auricle deformity in the prior art.

[0005] In a first aspect, the present application provides a baby auricle deformity intervention prediction method, which comprises: obtaining target auricle image data and target structured information of a target object; inputting the target auricle image data into a pre-constructed detection model to obtain a target auricle region; inputting the target auricle region into a pre-constructed classification model to obtain an auricle deformity recognition result; in the case that the auricle deformity recognition result is a preset auricle deformity type, inputting the target auricle region, the auricle deformity recognition result and the target structured information into a pre-constructed intervention scoring model to obtain an intervention score; the intervention score is an aesthetic score predicted before intervention for the target object after auricle deformity intervention.

[0006] In an optional implementation, the detection model is constructed based on a YOLO model; and the classification model is constructed based on a SwinV2Transformer model.

[0007] In an optional implementation, the intervention scoring model is established by the following steps: obtaining a first sample data set, the first sample data set comprising auricle region sample images, auricle deformity sample types, structured sample information and post-intervention aesthetic scores of different sample objects; The first sample dataset is preprocessed to obtain the second sample dataset. The preprocessing includes: image standardization of the auricle region sample images, encoding of auricle deformity sample types, and text standardization of structured sample information. The second sample dataset is divided into a training set, a validation set, and a test set according to the sample objects. The preprocessed auricle region sample image is fed into the image feature encoding branch to obtain the image feature vector; The preprocessed auricular deformity sample type is fed into the type encoding branch to obtain the deformity feature vector; The preprocessed structured sample information is fed into the structured information encoding branch to obtain the structured feature vector; The image feature vector, structured feature vector, and deformed feature vector are fused to obtain the fused feature. The fused features are fed into the heteroscedastic regression prediction head to obtain the mean and variance of the predicted scores for the aesthetic scores after intervention. The loss value is calculated based on the mean of the predicted scores, the variance of the predicted scores, and the actual aesthetic scores after intervention. The parameters of all branches of the model are updated based on backpropagation of the loss value, and the update stops when the loss on the validation set increases for a preset number of consecutive rounds. When the test set meets the preset performance indicators, the intervention scoring model is obtained.

[0008] In one alternative implementation, the detection model is established through the following steps: Obtain the second sample dataset, which includes several auricle sample images; Auricular region annotation is performed on auricular sample images to obtain auricular annotation data; The initial YOLO model was trained based on the second sample dataset and auricle annotation data to obtain the detection model.

[0009] In one alternative implementation, the classification model is established through the following steps: Obtain the third sample dataset, which includes several auricular region sample images obtained through detection model and the corresponding auricular deformity sample types; The sample image of the auricle region is divided into blocks to obtain several sub-images of the auricle region; Project each sample sub-image of the auricle region onto the image to obtain the projection matrix; The projection matrix is ​​divided into preset windows, and attention calculation is performed. For the feature matrix that has completed attention calculation, it is re-divided according to the preset translation window, and the translation window multi-head self-attention calculation is performed. The feature matrix that has completed the multi-head self-attention calculation of the translation window is downsampled layer by layer and the features are merged to obtain a pyramid-shaped hierarchical feature structure. Based on a pyramid-shaped hierarchical feature structure, image-level feature vectors are obtained through global average pooling. Image-level feature vectors are fed into a linear classification layer, and after Softmax activation, the probability distribution of each type of auricular deformity is output. The loss value is calculated based on the probability distribution and the actual auricular deformity sample types. The parameters of all levels of the model are updated by backpropagation based on the loss value, thus obtaining the classification model.

[0010] In one alternative implementation, the sample dataset is collected using pre-defined standardized collection requirements.

[0011] In a second aspect, the present invention provides an intervention and prediction device for infant auricular deformity, the device comprising: The acquisition module is used to acquire the target auricle image data and target structure information of the target object; The detection module is used to input the target auricle image data into a pre-built detection model to obtain the target auricle region; The classification module is used to input the target auricle region into a pre-built classification model to obtain auricle deformity recognition results; The prediction module is used to input the target auricle region, the auricle deformity recognition result, and the target structured information into a pre-built intervention scoring model to obtain an intervention score when the auricle deformity recognition result is a preset auricle deformity type. The intervention score is the aesthetic score predicted before intervention for the target object after auricle deformity intervention.

[0012] Thirdly, the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the infant auricular deformity intervention and prediction method described in the first aspect or any corresponding embodiment thereof.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the infant auricle deformity intervention and prediction method of the first aspect or any corresponding embodiment described above.

[0014] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the infant auricle deformity intervention and prediction method described in the first aspect or any corresponding embodiment thereof.

[0015] The beneficial effects of this invention are as follows: This invention employs a two-stage recognition framework combining a detection model and a classification model. It first automatically detects and locates the auricle region, then identifies the type of auricle deformity. Compared to existing methods that directly classify the entire image, this invention effectively reduces background interference and improves the accuracy and robustness of recognition. Building upon the detection and classification of auricle deformities, it also introduces preoperative multimodal efficacy prediction. Based solely on preoperative auricle images and the infant's structured features, it can output continuous efficacy scores and uncertainties consistent with doctor ratings, effectively enabling preoperative expectation management. This method represents a substantial advancement in data acquisition standardization, recognition robustness, quantifiable preoperative prediction, and interpretability. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the first process of an intervention and prediction method for infant auricular deformity according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the two-stage identification process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the third process for predicting intervention scores according to an embodiment of the present invention; Figure 4 This is a structural block diagram of an infant auricular deformity intervention and prediction device according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] Current technologies primarily focus on malformation identification itself, lacking unified data collection standards and objective, quantifiable tools for predicting treatment efficacy. Clinical efficacy assessments often rely on physicians' subjective observations, making it difficult to quantify expected outcomes preoperatively, thus limiting patient enrollment and treatment options. With the development of deep learning, Transformer architectures (such as SwinV2) demonstrate excellent performance in medical image processing. When combined with structured clinical information and supplemented by standardized data collection and governance, they hold promise for achieving integrated identification and assessment, as well as preoperatively interpretable efficacy predictions.

[0022] In view of this, according to an embodiment of the present invention, an embodiment of an intervention and prediction method for infant auricular deformity is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment provides a method for predicting and intervening in infant ear deformities, which can be used on servers, terminals, and mobile terminals such as mobile phones and tablets. Figure 1 This is a flowchart of an infant auricle deformity intervention and prediction method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the target auricle image data and target structured information of the target object.

[0024] In this embodiment, the target object can be the infant itself or the infant's ear, etc. Target ear image data refers to an image containing the ear. The target structured information in this embodiment includes: starting age, sex, whether twins, birth weight, gestational age, delivery method, mother's age, parity / partum, etc., and these are combined into a structured vector. .

[0025] Furthermore, the target auricle image data selected in this invention employs a pre-defined standard image acquisition method, consistent with the acquisition method used for the sample datasets used to train the detection model, classification model, and intervention scoring model. Specifically, auricle images can be acquired under conditions of fixed lighting, uniform angle, and specified distance and resolution, avoiding image quality fluctuations caused by differences in acquisition conditions in existing technologies, and ensuring data stability and comparability.

[0026] Step S102: Input the target auricle image data into the pre-built detection model to obtain the target auricle region.

[0027] In some optional implementations, the detection model is preferentially built based on the YOLO model. In addition to the YOLO series models, other mainstream target detection algorithms can also be used for automatic detection and localization of the auricle region, such as Faster R-CNN, SSD, DETR (Detection Transformer), or anchor-free models like CenterNet and YOLOX. These detection models can also achieve automatic localization of the infant's auricle region. However, inputting the target auricle image data into a pre-built YOLO series detection model can achieve faster and more accurate auricle localization and recognition, thereby improving diagnostic efficiency and accuracy.

[0028] Step S103: Input the target auricle region into the pre-built classification model to obtain the auricle deformity recognition result.

[0029] In some optional implementations, the classification model is built based on the SwinV2 Transformer model. Additionally, in the identification of auricular deformity types and postoperative efficacy analysis, besides SwinV2 Transformer, other convolutional neural networks (such as ResNet, EfficientNet, DenseNet) or other Transformer architectures (such as ViT, ConvNeXt, Swin Transformer V1 / V2 / V3) can be used for feature extraction and classification / regression tasks. These models can also complete the identification of auricular deformity types and efficacy scoring. However, inputting the target auricular region into a pre-built SwinTransformer V2 classification model can leverage its powerful feature extraction capabilities to achieve high-precision identification of auricular deformities, thereby providing more reliable diagnostic evidence for clinical practice.

[0030] Step S104: If the auricular deformity identification result is a preset auricular deformity type, the target auricular region, the auricular deformity identification result, and the target structured information are input into a pre-built intervention scoring model to obtain an intervention score. The intervention score is the aesthetic score predicted before intervention for the target object after auricular deformity intervention. The preset auricular deformity type can be ape ears, cup ears, auricular defects, etc.

[0031] Before constructing the intervention scoring model, two or more physicians from the organization will conduct aesthetic scoring on the postoperative images of the infant's auricle using a 5-point Likert scale. For example, the score ranges from 1 to 5, with higher scores indicating that the auricle's appearance better meets aesthetic standards.

[0032] In intervention scoring model prediction, regression tasks can directly output continuous scores. Besides this, a hierarchical classification task can be used, discretizing the 1–5 Likert scale into 5 categories to indirectly reflect the efficacy level through multi-classification output. Furthermore, ranking learning methods can be employed, transforming efficacy assessment into a ranking problem to rank the relative effectiveness of treatment among different patients. In addition to the Likert scale, other objective indicators can be used as efficacy labels, such as auricular symmetry and contour smoothness indicators calculated based on 3D scan data, or multi-dimensional comprehensive scoring systems provided by clinicians. These indicators can also serve as supervisory signals for model training, guiding the model to complete efficacy analysis.

[0033] This invention employs a two-stage recognition framework combining the YOLO model and the SwinV2 Transformer model. It first automatically detects and locates the auricle region, then identifies the type of auricle deformity. Compared to related technologies that directly classify the entire image, this invention effectively reduces background interference and improves the accuracy and robustness of recognition. Building upon the classification of auricle deformities, it also introduces preoperative multimodal efficacy prediction. Based solely on preoperative auricle images and the infant's structured features, it can output continuous efficacy scores and uncertainties (with a 95% confidence interval) consistent with the doctor's rating, effectively enabling preoperative expectation management and treatment plan selection. This method represents a substantial advancement in data acquisition standardization, recognition robustness, quantifiable preoperative prediction, and interpretability.

[0034] This invention proposes a multimodal fusion model based solely on preoperative auricular images and the infant's structured information (such as initial age, gestational week, delivery method, and complications). This model outputs continuous efficacy scores before treatment, better meeting clinical decision-making needs (enrollment selection, intervention protocol communication), and overcoming the limitations of existing methods that only perform postoperative assessments or output discrete categories. It automates the entire process from image acquisition, preprocessing, and malformation identification to efficacy scoring, significantly reducing the workload of physicians and improving the efficiency and consistency of diagnosis and efficacy analysis.

[0035] In some alternative implementations, the intervention scoring model is established through the following steps: Step a1: Obtain the first sample dataset, which includes sample images of the auricle region of different sample objects, auricle deformity sample types, structured sample information, and aesthetic scores after intervention; and preprocess the first sample dataset to obtain the second sample dataset; the preprocessing includes: image standardization of the auricle region sample images, encoding of auricle deformity sample types, and text standardization of the structured sample information.

[0036] Specifically, after image acquisition, multiple qualified physicians can independently diagnose the infant's pre-treatment auricular images and classify and number the auricular morphology according to existing plastic surgery textbooks and literature on auricular deformity classification methods, as follows: Normal ear: 0; Lop ear: 1; Stahl's ear: 2; Helix deformity: 3; Cup ear: 4; Microtia / auricular hypoplasia: 5; Prominent ears: 6.

[0037] After the treatment, two or more physicians can assess the aesthetic appearance of the infant's auricle using a 5-point Likert scale to obtain a post-intervention aesthetic score. This scale is a widely used subjective evaluation tool, with scores ranging from 1 to 5. Higher scores indicate that the auricle's appearance better conforms to aesthetic standards. The specific definitions are as follows: 1 point (Very Poor): The shape of the auricle is severely abnormal, the overall proportions are unbalanced, and there is a significant difference from normal anatomy. There is almost no improvement after surgery. 2 points (Poor): The shape of the auricle is obviously abnormal, with local deformities or contour defects. Although there is some improvement, the appearance is not ideal. 3 points (moderate / Fair): The shape of the auricle is close to normal, the overall structure is clear, there are still some differences or asymmetries, but they are basically acceptable; 4 points (Good): The shape of the auricle is close to normal, the structure is coordinated, the symmetry is good, and the aesthetic effect is significantly improved. 5 points (Excellent): The shape of the auricle is completely normal or nearly normal, with clear outline, natural symmetry, and good aesthetic appearance; Scoring method: Each post-treatment auricle image is scored by two independent physicians; if the scores are inconsistent, the average value is taken as the final aesthetic score.

[0038] Before scoring, physicians should read the standardized scoring guidelines together to reduce subjective differences; if necessary, a consistency coefficient (such as the Kappa value) should be calculated to evaluate the reliability of the scoring and ensure consistency.

[0039] This embodiment introduces a senior physician to score the postoperative effect of auricle surgery based on a 5-point Likert scale, and uses it as a supervision signal for model training. This makes the model's prediction results consistent with commonly used clinical evaluation standards, enhancing the interpretability and clinical generalizability of the model in medical scenarios.

[0040] Furthermore, storing aesthetic scores alongside electronic medical records serves as both an indicator for evaluating treatment effectiveness and a label or validation standard for subsequent intelligent model training. Finally, an electronic medical record is established to record the infant's basic demographic information, intervention time, pre- and post-treatment images, and relevant clinical data, achieving a correspondence between image information and structured information storage, providing data support for subsequent model training and clinical applications.

[0041] After completing the acquisition and archiving of infant auricle images, in order to ensure that the subsequent model can accurately identify the auricle region, the original auricle region sample images are preprocessed, specifically including the following steps: 1. Data cleaning: Remove blurry, overexposed, severely obscured, or non-compliant images to ensure stable input data quality; 2. Auricle Region Annotation: Annotators accurately delineate the target region of the auricle in the image according to pre-defined annotation specifications. The annotation method primarily uses rectangular bounding boxes, which are used for subsequent training of the object detection model. 3. Each image corresponds to a labeling result, which is a training format that the model can directly use (the labeling file is in TXT format and contains bounding box coordinate information). 4. Data verification: The research team randomly selected a portion of the annotation results for manual review. The auricle area must be completely annotated without omitting any major anatomical structures (such as the helix and auricle), and the proportion of non-ear areas in the annotation box should not exceed 20%. The above preprocessing steps generate standardized sample images of the auricle region, providing high-quality data support for subsequent operations.

[0042] In the data preprocessing stage, this invention can also introduce steps such as image cleaning, auricle integrity checking, non-ear region proportion control, and manual verification to ensure that the data used for training has high quality and consistency, and avoid the model performance degradation problem caused by data noise or incorrect labeling in related technologies.

[0043] Step a2: Divide the second sample dataset into a training set, a validation set, and a test set according to the sample objects.

[0044] Step a3: The preprocessed auricle region sample image is fed into the image feature encoding branch to obtain the image feature vector.

[0045] Step a4: The preprocessed auricular deformity sample type is sent to the type encoding branch to obtain the deformity feature vector.

[0046] Step a5: The preprocessed structured sample information is fed into the structured information encoding branch to obtain the structured feature vector.

[0047] Step a6 involves fusing the image feature vector, structured feature vector, and deformed feature vector to obtain fused features.

[0048] In this embodiment, feature fusion can replace early splicing / FiLM modulation with Cross-Attention (phenotype is Query, image is KV), gated fusion (GMU), TabTransformer and visual feature splicing, etc.

[0049] Step a7: Input the fused features into the heteroscedastic regression prediction head to obtain the mean and variance of the predicted scores for the aesthetic scores after intervention.

[0050] In this embodiment, the heteroscedastic Gaussian regression can also be replaced by Huber / MSE (simple and robust), quantile regression (directly outputting the upper and lower quantile intervals), ordinal / CORAL regression (closer to the orderliness of 1–5 levels), and pairwise / list-based learning (only requiring relative consistency in quality).

[0051] Step a8: Calculate the loss value based on the predicted score mean, predicted score variance, and the actual aesthetic score after intervention.

[0052] Step a9: Update the parameters of all branches of the model based on backpropagation of the loss value, and stop updating when the loss on the validation set increases for a preset number of consecutive rounds.

[0053] Step a10: When the test set meets the preset performance indicators, the intervention scoring model is obtained.

[0054] In this embodiment, the intervention scoring model takes preoperative auricular images and infant structured information as input, and outputs continuous scores and their uncertainties through a cross-modal fusion network to assist in intervention decision-making and communication of expectations. The intervention scoring model in this embodiment includes: an image feature encoding branch, a structured information encoding branch, a cross-modal feature fusion layer, a heteroscedasticity regression prediction head, and a joint training mechanism module. (Refer to...) Figure 2 As shown, the details are as follows: Image input: Preoperative standardized image of the auricle The auricle bounding box output by the Stage 1 detection model (YOLO series). Perform 5%–8% outward trimming and uniform size (default). ).

[0055] Structured input: Preoperative malformation type Starting age ,gender Are they twins? Birth weight, gestational age Delivery method Mother's age Parity / partum , ), and combine them into a structured vector. Enter the input.

[0056] Output: Postoperative aesthetic score was and standard deviation (used to give a 95% confidence interval) 1.96 , +1.96 ]. Figure 2 The pre-trained multimodal preoperative assessment module for postoperative efficacy is the intervention scoring model in this embodiment.

[0057] Image Feature Encoding Branch (based on the SwinV2 model): Image feature extraction employs a hierarchical visual backbone based on SwinV2. Internally, it includes patch embedding, window attention, and downsampling mechanisms. The model output is then subjected to global average pooling to obtain the image representation vector. ; Structured information coding branch (Tab-MLP + Embedding): After standardization, the input is a two-layer multilayer perceptron (Tab-MLP): ; in, for activation, for Function, output dimension .

[0058] Cross-modal fusion and high-dimensional representation: stitching images with structured features: ; Mapped to the projection space via the fusion sensor: This vector comprehensively represents the preoperative auricular morphology and the patient's basic information.

[0059] Furthermore, heteroscedastic Gaussian regression is employed to output both the mean and variance of the scores: Heteroscedastic regression predictor: ; in: The vector representation after fusing the two modalities; These are the learnable parameters for the heteroscedastic regression predictor head; This is a point estimate (mean) of the postoperative score for the model. The prediction variance associated with the data (reflecting uncertainty) is used to output confidence intervals and improve robustness.

[0060] Loss function (Gaussian negative log-likelihood): ; in: The doctor's actual score for the postoperative images based on a 5-point Likert scale (average of scores from multiple doctors); For data fitting terms, the larger the residual, the better. The smaller the child, the heavier the punishment; As a penalty for complexity / uncertainty, The larger the variance, the greater the penalty, preventing the network from "escaping error with infinite variance".

[0061] Finally, loss and optimization are performed: minimize... The optimizer is AdamW (learning rate). Weight decay 0.05, cosine annealing, Warmup 5 epochs), epoch=200, batch size=32–64.

[0062] Uncertainty estimation divided by σ 2 In addition, MC-Dropout, Deep Ensemble, Evidential regression, or Laplace approximation can be used to obtain the interval and calibration.

[0063] In this embodiment, stratified sampling (segmented by malformation type and score) can effectively alleviate the problem of uneven distribution.

[0064] This invention uses heteroscedastic regression instead of classification in the efficacy analysis module, directly outputting a continuous numerical score (1.0–5.0) consistent with physician ratings. This can delicately reflect the differences in efficacy after auricular surgery, and is closer to the actual evaluation methods of clinicians, making up for the shortcomings of related technologies that can only output discrete categories.

[0065] This invention achieves automation, quantification, and clinical interpretability of auricular deformity identification and efficacy assessment through standardized data collection, rigorous preprocessing, a two-stage deep learning framework, and regression-based efficacy analysis.

[0066] In some alternative implementations, the detection model is established through the following steps: Step b1: Obtain the second sample dataset, which includes several auricle sample images.

[0067] Step b2: Label the auricle region on the auricle sample image to obtain auricle labeling data.

[0068] Step b3: Train the initial YOLO model based on the second sample dataset and auricle annotation data to obtain the detection model.

[0069] The specific steps for implementing step b3 above are as follows: Input: Original image of an infant's auricle sample and corresponding auricle region annotation data.

[0070] Tags: Rectangle coordinates (x, y, w, h).

[0071] The model structure uses the YOLOv8 series detectors; the model consists of a backbone (feature extraction), a neck (feature fusion), and a head (detection head); the output is the candidate bounding box position and confidence score.

[0072] Training: The training objective of the YOLO object detection model is to minimize the following loss function: ; in, For bounding box regression loss, the CIoU loss function is used: ; Parameter definition: The number of images in a batch; : The set of predicted indices (generated by the matching rules) that are assigned as positive samples in the nth image; : The center coordinates and width and height of the i-th prediction box in the n-th image (in pixels or normalized to [0,1]); : The actual bounding box that matches it; : A measure of consistency between predicted bounding boxes and ground truth bounding boxes.

[0073] Bounding box regression loss can constrain the position, scale, and aspect ratio of the predicted box to approximate the true box, thereby improving localization accuracy.

[0074] Confidence loss Calculate the binary cross-entropy (BCE) for each predicted location: ; Parameter definition: : The set of indices of all detection locations (anchor boxes or no anchor points) on the nth image; The raw logit value of the confidence score at this position; The Sigmoid function; The confidence level target labels are as follows: For positive sample locations, take ; For negative sample locations, take ; For "negligible negative sample" locations where the IoU with any ground truth bounding box exceeds the neglect threshold, the loss can be ignored.

[0075] In this embodiment, learning whether an arbitrary target (in this task, "auricle") exists at the location can suppress false background detections and improve the balance between recall and precision.

[0076] Classification loss: In this scenario, only the "ear" class is included. It can be ignored.

[0077] Specifically, the training process is as follows: Data preparation: Load the quality-controlled labeled data; Forward propagation: For each batch, obtain , ; Sample allocation: Based on IoU matching of ground truth bounding boxes, the following results are obtained. and ; Loss calculation: Calculated according to the above formula. , Total loss ; Reverse and Update: To Backpropagation updates parameters using an optimizer (SGD / AdamW) and learning rate scheduling (such as cosine annealing).

[0078] After the model is trained, input any image of an infant's ear, and the YOLO model can output the coordinates of a rectangular bounding box of the ear region; this bounding box is used for subsequent cropping and serves as input for the second-stage SwinV2 model.

[0079] In this embodiment, training the initial YOLO model based on the sample dataset and auricle annotation data can significantly improve the model's recognition ability, making it more accurate and efficient in handling auricle deformity recognition tasks. This not only enhances the model's generalization ability but also provides a more reliable diagnostic tool for clinicians, helping to improve the early screening and intervention effects of auricle deformities.

[0080] In some alternative implementations, the classification model is established through the following steps: Step c1: Obtain the third sample dataset, which includes several auricular region sample images obtained by the detection model and the corresponding auricular deformity sample types.

[0081] Step c2 involves dividing the auricular region sample image into blocks to obtain several auricular region sample sub-images.

[0082] Step c3: Project the sample sub-image of each auricle region to obtain the projection matrix.

[0083] Step c4: Divide the projection matrix into preset windows and perform attention calculation.

[0084] Step c5: For the feature matrix that has completed attention calculation, redivide it according to the preset translation window and perform translation window multi-head self-attention calculation.

[0085] Step c6 involves downsampling the feature matrix that has completed the multi-head self-attention calculation for the translation window layer by layer and merging the features to obtain a pyramid-shaped hierarchical feature structure.

[0086] Step c7: Based on the pyramid-shaped hierarchical feature structure, image-level feature vectors are obtained through global average pooling.

[0087] Step c8: The image-level feature vector is fed into the linear classification layer, and after Softmax activation, the probability distribution of each type of auricular deformity is output.

[0088] Step c9: Calculate the loss value based on the probability distribution and the actual auricular deformity sample type.

[0089] Step c10: Update the parameters of all levels of the model based on backpropagation of the loss value to obtain the classification model.

[0090] Reference Figure 3 As shown, the following is a detailed explanation: Input: An image of the auricle region cropped by the detection bounding box of the Stage 1 detection model; Tags: Types of auricular deformities marked by doctors (such as normal ear, lop ear, Stahl's ear, cup ear, microtia, protruding ear, etc.); The input auricular region image is denoted as: ; Divide it into sides of length The non-overlapping small block partitioning yielded a total of There are several patches. Each patch, when flattened, looks like this: ; Each flattened vector is mapped by a linear mapping matrix E: ; The final result is a D-dimensional patch embedding: ; in, It is a learnable location code used to preserve local location information.

[0091] Furthermore, window multi-head self-attention (W-MSA) is performed: Embedded sequence By window size Divided into several windows, each window contains Each token. Within each window, perform multi-head self-attention computation: ; in, These are query, key, and value matrices, respectively. For single-head feature dimensions, This is a learnable position offset matrix used to compensate for relative position information within the window.

[0092] Furthermore, perform translation window attention (SW-MSA): To enable cross-window information interaction, the window in the adjacent layer is shifted by half the window size (i.e., ( / 2 patches), then calculate attention as described above. By alternately stacking W-MSA and SW-MSA, a wider range of global relationships can be captured while maintaining local modeling.

[0093] Further, downsampling and hierarchical feature construction are performed: At the end of each stage, for adjacent Small blocks are combined with features and linearly mapped to reduce spatial resolution and increase the number of channels: ; Among them, default and Setting it to 2, the final formula can be obtained through merging: ; This forms a pyramid-shaped hierarchical feature structure: ; Features of the final stage output Image-level feature vectors are obtained after global average pooling: ; Then, after passing through a linear classifier and Softmax, the output K-class probability distribution is obtained: ; The final prediction category is: ; Considering the severe imbalance in the categories of infantile auricular deformities, Focal Loss optimization is adopted, which can effectively alleviate the impact of positive and negative sample imbalance on model convergence.

[0094] In this embodiment, the classification and loss function can be as follows: ; in: Number of categories; Tag it as one-hot; This is the category balance coefficient; This is the adjustment factor (usually set to 2).

[0095] In this embodiment, the classification model uses the Swin V2 Transformer as the auricular deformity classifier. First, the image is segmented into small patches and projected into vectors. Attention calculation (W-MSA) is performed within a small window to capture local details. Then, translation windowing (SW-MSA) allows information from different windows to interact, thereby obtaining global features. Layer-by-layer downsampling forms a pyramid structure, simultaneously extracting features at different scales. Finally, a classification head outputs the category probability distribution. Through effective interaction of local and global features and multi-scale feature extraction, the accuracy and robustness of auricular deformity classification can be significantly improved, providing more precise support for clinical diagnosis.

[0096] In some alternative implementations, the sample dataset is collected using pre-defined standardized collection requirements.

[0097] After obtaining informed consent from the child's guardian, standardized images of the infant's auricle were captured before and after treatment to ensure the stability and comparability of the results. Specifically: Shooting environment: A constant light source was set up in a fixed laboratory environment, and the color temperature was controlled at 5000–5500K to obtain uniform and natural lighting conditions; Shooting posture and angle: Keep the camera lens parallel to the auricle, with the child's head positioned in a centered position, ensuring a clear and complete view of the auricle from the front. To comprehensively record the auricle's morphology, the shooting angles should include: a frontal view (showing the overall shape and symmetry of the auricle), 45° oblique views to the left and right (highlighting the three-dimensional structures such as the helix, antihelix, and concha), standard lateral views to the left and right (clearly showing the height, outline, and depth of the concha), and, if necessary, supplementary posterior oblique and postauricular views (used to observe the angle and fit between the auricle and the skull). Through these multi-angle photographs, the characteristics of the auricle deformity and changes before and after treatment can be comprehensively and objectively reflected.

[0098] Shooting method: The camera is mounted on a fixed bracket, with the distance between the lens and the auricle maintained at 30–50 cm. The distance is adjusted according to the lens focal length so that the auricle occupies about 60%–70% of the image.

[0099] Imaging requirements: The resolution of the acquired two-dimensional images shall not be less than 4000 × 3000 pixels to meet the needs of subsequent medical processing and analysis.

[0100] In the image acquisition stage, in addition to using two-dimensional high-definition photos, three-dimensional scanning, structured light imaging, or stereo cameras can also be used to acquire three-dimensional structural information of the infant's auricle, thereby providing richer features for model training and efficacy analysis.

[0101] In this embodiment, by establishing controlled imaging and annotation quality control standards, the high quality and consistency of auricular sample images can be ensured. This not only improves the standardization level of data collection but also reduces human error, ensuring the accuracy and reliability of the labeled data. Such quality control measures can effectively improve the subsequent model training effect, making the trained detection and classification models more accurate. This provides a more solid foundation for the identification and diagnosis of auricular deformities, ultimately promoting the effectiveness and credibility of clinical applications.

[0102] This embodiment also provides an infant auricular deformity intervention and prediction device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0103] This embodiment provides an intervention and prediction device for infant auricular deformity, such as... Figure 4 As shown, it includes: The acquisition module 401 is used to acquire the target auricle image data and target structure information of the target object; The detection module 402 is used to input the target auricle image data into a pre-built detection model to obtain the target auricle region; The classification module 403 is used to input the target auricle region into a pre-built classification model to obtain auricle deformity recognition results; The prediction module 404 is used to input the target auricle region, the auricle deformity recognition result, and the target structured information into a pre-built intervention scoring model to obtain an intervention score when the auricle deformity recognition result is a preset auricle deformity type. The intervention score is the aesthetic score predicted before intervention for the target object after auricle deformity intervention.

[0104] In some alternative implementations, the apparatus further includes an intervention scoring model construction module, specifically used for: Obtain the first sample dataset, which includes sample images of the auricle region of different sample objects, auricle deformity sample types, structured sample information, and aesthetic scores after intervention; The first sample dataset is preprocessed to obtain the second sample dataset. The preprocessing includes: image standardization of the auricle region sample images, encoding of auricle deformity sample types, and text standardization of structured sample information. The second sample dataset is divided into a training set, a validation set, and a test set according to the sample objects. The preprocessed auricle region sample image is fed into the image feature encoding branch to obtain the image feature vector; The preprocessed auricular deformity sample type is fed into the type encoding branch to obtain the deformity feature vector; The preprocessed structured sample information is fed into the structured information encoding branch to obtain the structured feature vector; The image feature vector, structured feature vector, and deformed feature vector are fused to obtain the fused feature. The fused features are fed into the heteroscedastic regression prediction head to obtain the mean and variance of the predicted scores for the aesthetic scores after intervention. The loss value is calculated based on the mean of the predicted scores, the variance of the predicted scores, and the actual aesthetic scores after intervention. The parameters of all branches of the model are updated based on backpropagation of the loss value, and the update stops when the loss on the validation set increases for a preset number of consecutive rounds. When the test set meets the preset performance indicators, the intervention scoring model is obtained.

[0105] In some optional implementations, the apparatus further includes a detection model building module, specifically used for: Obtain the second sample dataset, which includes several auricle sample images; Auricular region annotation is performed on auricular sample images to obtain auricular annotation data; The initial YOLO model was trained based on the second sample dataset and auricle annotation data to obtain the detection model.

[0106] In some alternative implementations, the apparatus further includes a classification model construction module, specifically used for: Obtain the third sample dataset, which includes several auricular region sample images obtained through detection model and the corresponding auricular deformity sample types; The sample image of the auricle region is divided into blocks to obtain several sub-images of the auricle region; Project each sample sub-image of the auricle region onto the image to obtain the projection matrix; The projection matrix is ​​divided into preset windows, and attention calculation is performed. For the feature matrix that has completed attention calculation, it is re-divided according to the preset translation window, and the translation window multi-head self-attention calculation is performed. The feature matrix that has completed the multi-head self-attention calculation of the translation window is downsampled layer by layer and the features are merged to obtain a pyramid-shaped hierarchical feature structure. Based on a pyramid-shaped hierarchical feature structure, image-level feature vectors are obtained through global average pooling. Image-level feature vectors are fed into a linear classification layer, and after Softmax activation, the probability distribution of each type of auricular deformity is output. The loss value is calculated based on the probability distribution and the actual auricular deformity sample types. The parameters of all levels of the model are updated by backpropagation based on the loss value, thus obtaining the classification model.

[0107] The infant auricular deformity intervention and prediction device provided in this embodiment of the invention can execute the infant auricular deformity intervention and prediction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0108] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0109] The following is a detailed reference. Figure 5The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0110] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0111] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the infant auricular deformity intervention prediction method of the embodiments of the present invention.

[0112] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0113] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the infant auricular deformity intervention and prediction method shown in the above embodiments is implemented.

[0114] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0115] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for predicting and intervening in infantile auricular deformities, characterized in that, The method includes: Acquire the target auricle image data and target structure information of the target object; The target auricle image data is input into a pre-built detection model to obtain the target auricle region; The target auricle region is input into a pre-built classification model to obtain auricle deformity recognition results; If the auricular deformity identification result is a preset auricular deformity type, the target auricular region, the auricular deformity identification result, and the target structured information are input into a pre-constructed intervention scoring model to obtain an intervention score; the intervention score is the aesthetic score predicted before intervention for the target object after auricular deformity intervention.

2. The method according to claim 1, characterized in that, The detection model is based on the YOLO model; the classification model is based on the SwinV2 Transformer model.

3. The method according to claim 1, characterized in that, The intervention scoring model is established through the following steps: Obtain the first sample dataset, which includes sample images of the auricle region of different sample objects, auricle deformity sample types, structured sample information, and aesthetic scores after intervention; The first sample dataset is preprocessed to obtain the second sample dataset; the preprocessing includes: image standardization of the auricular region sample images, encoding of the auricular deformity sample types, and text standardization of the structured sample information. The second sample dataset is divided into a training set, a validation set, and a test set according to the sample objects. The preprocessed auricular region sample image is fed into the image feature encoding branch to obtain the image feature vector; The preprocessed auricular deformity sample type is fed into the type encoding branch to obtain the deformity feature vector; The preprocessed structured sample information is fed into the structured information encoding branch to obtain the structured feature vector; The image feature vector, the structured feature vector, and the deformed feature vector are fused to obtain fused features; The fused features are fed into the heteroscedastic regression prediction head to obtain the mean and variance of the predicted aesthetic scores after intervention. The loss value is calculated based on the mean of the predicted scores, the variance of the predicted scores, and the actual aesthetic scores after intervention. The parameters of all branches of the model are updated by backpropagation based on the loss value, and the update stops when the loss on the validation set increases for a preset number of consecutive rounds. When the test set meets the preset performance indicators, the intervention scoring model is obtained.

4. The method according to claim 1, characterized in that, The detection model is established through the following steps: Obtain a second sample dataset, which includes several auricle sample images; The auricle sample image is annotated with auricle region to obtain auricle annotation data; The initial YOLO model is trained based on the second sample dataset and the auricle annotation data to obtain the detection model.

5. The method according to claim 1, characterized in that, The classification model is established through the following steps: Obtain a third sample dataset, which includes several auricular region sample images obtained by the detection model and the corresponding auricular deformity sample types. The sample image of the auricle region is divided into blocks to obtain several sub-images of the auricle region; Project each sample sub-image of the auricle region onto the image to obtain a projection matrix; The projection matrix is ​​divided into preset windows, and attention calculation is performed. For the feature matrix that has completed the attention calculation, it is re-divided according to the preset translation window, and the translation window multi-head self-attention calculation is performed. The feature matrix that has completed the multi-head self-attention calculation of the translation window is downsampled layer by layer and the features are merged to obtain a pyramid-shaped hierarchical feature structure. Based on the pyramid-shaped hierarchical feature structure, image-level feature vectors are obtained through global average pooling. The image-level feature vectors are fed into a linear classification layer, and after Softmax activation, the probability distribution of each type of auricular deformity is output. The loss value is calculated based on the probability distribution and the actual auricular deformity sample types; The parameters of all levels of the model are updated by backpropagation based on the loss value, thereby obtaining the classification model.

6. The method according to any one of claims 3 to 5, characterized in that, The sample dataset was collected using pre-defined standardized collection requirements.

7. A device for predicting and intervening in infant auricular deformities, characterized in that, The device includes: The acquisition module is used to acquire the target auricle image data and target structure information of the target object; The detection module is used to input the target auricle image data into a pre-built detection model to obtain the target auricle region; The classification module is used to input the target auricle region into a pre-built classification model to obtain auricle deformity recognition results; The prediction module is used to input the target auricle region, the auricle deformity recognition result, and the target structured information into a pre-built intervention scoring model to obtain an intervention score when the auricle deformity recognition result is a preset auricle deformity type; the intervention score is the aesthetic score predicted before intervention for the target object after auricle deformity intervention.

8. An electronic device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the infant auricular deformity intervention prediction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the infant auricular deformity intervention prediction method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the infant auricular deformity intervention prediction method according to any one of claims 1 to 6.

Images