Adenoids hypertrophy recognition system based on multi-angle face images

Abstract

The present invention discloses an adenoids hypertrophy recognition system based on multi-angle facial images, and the method of using the system includes the following steps: S1, collecting and preprocessing multi-angle facial images, collecting frontal and side face data sets of patient children and normal children, cropping, size standardization, and normalization of the data sets, and arranging the data sets according to the patient ID to obtain the patient dimension and single-viewing angle facial images for training; S2, constructing a lightweight recognition model. Due to the particularity of adenoids hypertrophy, generally only professional doctors can detect adenoids hypertrophy based on X-rays or CT images, and more accurate diagnosis depends on nasopharyngeal endoscopy. The present invention proposes an adenoids hypertrophy recognition system based on multi-angle facial images, which greatly reduces the difficulty of doctors' work and saves the cost of X-rays, CT images and nasopharyngeal endoscopy; at the same time, it allows patients to avoid the harm of radiation and the discomfort of invasive examinations.

Description

Technical Field

[0001] The invention relates to the technical field of face image processing, and in particular to an adenoids hypertrophy recognition system based on multi-angle face images. Background Art

[0002] The adenoids are collections of lymphoid tissue located in the posterior airway of the nasopharynx. Adenoids appear in early childhood between the ages of 6 and 10 years and usually disappear by the age of 16 years. Adenoid hypertrophy, a pathological enlargement of the nasopharyngeal tonsils, is the most common cause of nasal obstruction in children and is associated with the "adenoid face pattern," characterized by a narrow maxillary arch, posterior crossbite, mandibular retrusion, and elongated face. The prevalence of adenoid hypertrophy ranges from 34% in the general population of children to 42% to 70% in the pediatric population. Adenoids hypertrophy is very harmful, but patients often delay treatment due to factors such as the lack of obvious symptoms in the early stages of the disease, large differences between individuals, and insufficient experience of medical staff. The diagnosis of adenoids hypertrophy in children at home and abroad still relies on specialist nasopharyngeal endoscopy and sleep polygraphs or maxillofacial CBCT airway reconstruction, lateral cranial radiographs and other professional examination methods, and these professional instruments are only available in hospitals with higher qualifications. At the same time, the identification of adenoids hypertrophy requires specialists with rich clinical experience. For grassroots and relatively remote and backward areas, there is usually a lack of experienced specialists, so the screening misdiagnosis rate for children with adenoids hypertrophy is high.

[0003] In the current public patents, a relatively close method for identifying adenoids is: obtain a head side X-ray image; input the head side X-ray image into the preprocessing module to extract the adenoids area map and grayscale normalization processing; the adenoids area map is defined as two categories: adenoids pathological hypertrophy image and adenoids normal image, and the image of the adenoids area is input into the deep learning model for forward reasoning; obtain the prediction result of the category to which the image output by the deep learning model belongs, but the head side image used in the public patent is an X-ray film, and the acquisition of head side X-ray films must be in a hospital with certain qualifications. Not all hospitals can take head side X-ray films, so the acquisition of data is limited by the region and hospital qualifications. In addition, X-ray films have certain radiation, which is harmful to children.

[0004] Therefore, it is necessary to provide a system for identifying adenoids hypertrophy based on multi-angle facial images to solve the above technical problems. Summary of the invention

[0005] The purpose of the present invention is to provide an adenoids hypertrophy recognition system based on multi-angle facial images to solve the problems existing in the above-mentioned background technology. The technical solution of the present invention provides a solution that is significantly different from the prior art to address the technical problem that the prior art solution is too single.

[0006] To achieve the above object, the present invention provides the following technical solution: a system for identifying adenoids hypertrophy based on multi-angle facial images, the method for using the system comprising the following steps:

[0007] S1, collection and preprocessing of multi-angle facial images, collecting frontal and side face datasets of patient children and normal children, cropping, resizing, and normalizing the datasets, and arranging the datasets according to patient IDs to obtain patient dimension and single-angle dimension facial images for training;

[0008] S2, constructing a lightweight recognition model, using the face image obtained in S1 to construct a lightweight recognition model of adenoids hypertrophy in two different dimensions: patient dimension and single-view dimension, importing the corresponding face image into the model, and pre-training the model to learn the feature representation of the image;

[0009] S3, model optimization and evaluation, uses a five-fold cross-validation method to improve the performance and stability of the model built in S2, evaluates model performance in each cross-validation, and comprehensively evaluates the overall performance of the model by combining the performance of all models on the test set.

[0010] Preferably, step S1 comprises the following steps:

[0011] S1-1, the collected face images include frontal face images, 45° side face images, and 90° side face images to ensure that the dataset covers a variety of angles, expressions, and lighting conditions to improve the robustness of the model. All images are passed through the RetinaFace segmentation network to crop the images to only contain the face area;

[0012] S1-2, all images are sampled into RGB images of size 512*512*3 to ensure that the input images meet the standard input size and format required by the model, and the data set is quality checked and cleaned to exclude low-quality or noisy image data;

[0013] S1-3, each image is normalized individually using the mean and variance of the data set according to the following formula:

[0014]

[0015] Among them, mean represents the mean of ImageNet, std represents the variance of the ImageNet dataset, and x norm It is a face image that is normalized to the interval [0,1] using the maximum and minimum values. This operation can highlight the relative differences between images, allowing the model to converge to a higher accuracy faster.

[0016] Preferably, step S2 comprises the following steps:

[0017] S2-1, construction of adenoids hypertrophy model in single-view dimension;

[0018] Based on the EfficientNet-b4 model, three single-angle dimension networks of 0°, 45° and 90° for the frontal face were constructed to extract the deep features of faces at different angles.

[0019] S2-2, patient-dimensional adenoids hypertrophy model construction;

[0020] The three different angles of the face images of the same patient ID are respectively passed into the corresponding single-angle dimension network to obtain the deep features of the patient at different facial angles. Then, the three sets of features are fused in the direction of the feature layer number and input into the decision layer of EfficientNet-b4 to obtain the patient dimension model.

[0021] S2-3, adenoids hypertrophy model training;

[0022] Given an input image x at the kth angle with a class label vector y k , the optimization objectives of the single-angle dimension model are as follows:

[0023]

[0024] Among them, h θ represents the model and its parameters, p k To identify the model pair x k The prediction vector is , N is the number of samples, and the model parameters are updated with the above loss. The initial learning rate is selected as 0.01, and 120 rounds are iterated. The learning rate is reduced by 10 times at 75 and 90 rounds respectively. When training the patient dimension model, the pre-trained models of three different angles should be frozen, and only the decision layer of the patient dimension model should be updated.

[0025] Preferably, step S3 comprises the following steps:

[0026] S3-1, five-fold cross validation, includes the following steps:

[0027] Taking patient ID as the unit, the images were divided into training validation set and test set in a ratio of 4:1. The training validation set was randomly divided into 5 equal parts, and the training set and validation set were arranged and combined in a ratio of 4:1 to obtain 5 different training validation sets. The above model was trained using 5 different data sets, and 5 adenoids hypertrophy recognition models with excellent performance were obtained in the single angle dimension and patient dimension at different angles.

[0028] S3-2, model performance evaluation, includes the following steps:

[0029] After obtaining five adenoids hypertrophy recognition models in the single angle dimension and patient dimension at different angles, each model was evaluated using the test set. The evaluation indicators include AUC, average precision, accuracy, sensitivity, and specificity. These indicators are defined as follows:

[0030]

[0031] Among them, TP, FP, TN and FN represent true positive, false positive, true negative and false negative samples respectively, AUC is the area under the ROC characteristic curve, and AP is the area under the PR characteristic curve;

[0032] After obtaining the evaluation indicators of each model, the mean and variance of the models in the single angle dimension and patient dimension at different angles were calculated to evaluate the overall performance of the model.

[0033] In summary, the present invention has developed a system for recognizing adenoids hypertrophy based on multi-angle facial images. The system is based on multi-angle facial images, and trains adenoids hypertrophy recognition models by constructing single-angle dimension and patient dimension models at different angles, and combines five-fold cross validation to improve the stability and credibility of the model. Through the above method, a reliable adenoids hypertrophy recognition of multi-angle facial images is well achieved.

[0034] Preferably, the adenoids hypertrophy recognition system for multi-angle facial images includes a facial image preprocessing module, a recognition model building module, a recognition model pre-training module and a five-fold cross-validation module;

[0035] The face image preprocessing module is used to crop, size, normalize and organize the face data set to obtain face images for training;

[0036] The recognition model building module is used to build a single-angle dimension recognition model at different angles, and use the single-angle dimension to perform deep feature extraction on the face image, so as to build a patient dimension recognition model;

[0037] The recognition model pre-training module is used to import the face images corresponding to the single angle dimension and patient dimension recognition models at different angles into the recognition model, perform model pre-training, and obtain a trained adenoids hypertrophy recognition model;

[0038] The five-fold cross-validation module is used to improve the performance and stability of the model and evaluate the overall performance of the model by comprehensively analyzing the performance of all models on the test set.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. Due to the particularity of adenoids hypertrophy, generally only professional doctors can detect adenoids hypertrophy based on X-rays or CT images, and more accurate diagnosis depends on nasopharyngeal endoscopy. The present invention proposes an adenoids hypertrophy recognition system based on multi-angle facial images, which greatly reduces the difficulty of doctors' work and saves the cost of CT images; at the same time, it allows patients to avoid the harm of radiation and the discomfort of invasive examinations.

[0041] 2. Based on the lightweight EfficientNet-b4 network, the present invention proposes an adenoids hypertrophy recognition module that can identify adenoids hypertrophy for facial images of patients at different angles. The combination of multiple angle information makes the recognition result more stable and reliable;

[0042] 3. The present invention utilizes multi-angle face image recognition and combines it with a five-fold cross-validation method to propose an adenoids hypertrophy recognition system based on multi-angle face images, thereby improving the stability and credibility of the adenoids hypertrophy recognition system and achieving more accurate adenoids hypertrophy recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the adenoids hypertrophy recognition method based on multi-angle face images of the present invention;

[0044] Figure 2 The figure is a general structural diagram of the adenoids hypertrophy recognition method based on multi-angle face images of the present invention;

[0045] Figure 3 Face images at different angles. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 and Figure 2 , a system for identifying adenoids hypertrophy based on multi-angle face images, the method for using the system comprises the following steps:

[0048] S1, collection and preprocessing of multi-angle face images, collecting frontal and side face data sets of patient children and normal children, and sorting the data sets according to patient IDs to obtain patient dimension and single-angle dimension face images for training;

[0049] S2, constructing a lightweight recognition model, using the facial image obtained in S1 to construct a lightweight recognition model of adenoids hypertrophy in two different dimensions: patient dimension and single angle dimension, and importing the corresponding facial image into the model to pre-train the model;

[0050] S3, model optimization and evaluation, uses a five-fold cross-validation method to improve the performance and stability of the model built in S2, evaluates model performance in each cross-validation, and comprehensively evaluates the overall performance of the model by combining the performance of all models on the test set.

[0051] Wherein, step S1 comprises the following steps:

[0052] S1-1, the collected face images include frontal face images, 45° side face images, and 90° side face images to ensure that the dataset covers a variety of angles, expressions, and lighting conditions to improve the robustness of the model. All images are passed through the RetinaFace segmentation network to crop the images to only contain the face area;

[0053] S1-2, all images are sampled into RGB images of size 512*512*3 to ensure that the input images meet the standard input size and format required by the model, and the data set is quality checked and cleaned to exclude low-quality or noisy image data;

[0054] S1-3, each image is normalized individually using the mean and variance of the data set according to the following formula:

[0055]

[0056] Among them, mean represents the mean of ImageNet, std represents the variance of the ImageNet dataset, and x norm It is a face image that is normalized to the interval [0,1] using the maximum and minimum values. This operation can highlight the relative differences between images, allowing the model to converge to a higher accuracy faster.

[0057] Wherein, the step S2 comprises the following steps:

[0058] S2-1, construction of a single-angle dimension adenoid hypertrophy model;

[0059] Based on the EfficientNet-b4 model, three single-angle dimension networks of 0°, 45° and 90° for the frontal face were constructed to extract the deep features of faces at different angles.

[0060] S2-2, patient-dimensional adenoids hypertrophy model construction;

[0061] The three different angles of the face images of the same patient ID are respectively passed into the corresponding single-angle dimension network to obtain the deep features of the patient at different facial angles. Then, the three sets of features are fused in the direction of the feature layer number and input into the decision layer of EfficientNet-b4 to obtain the patient dimension model.

[0062] S2-3, adenoids hypertrophy model training;

[0063] Given an input image x at the kth angle with a class label vector y k , the optimization objectives of the single-angle dimension model are as follows:

[0064]

[0065] Among them, h θ represents the model and its parameters, p k To identify the model pair x k The prediction vector is , N is the number of samples, and the model parameters are updated with the above loss. The initial learning rate is selected as 0.01, and 120 rounds are iterated. The learning rate is reduced by 10 times at 75 and 90 rounds respectively. When training the patient dimension model, the pre-trained models of three different angles should be frozen, and only the decision layer of the patient dimension model should be updated.

[0066] Wherein, the step S3 comprises the following steps:

[0067] S3-1, five-fold cross validation, includes the following steps:

[0068] Taking patient ID as the unit, the images were divided into training validation set and test set in a ratio of 4:1. The training validation set was randomly divided into 5 equal parts, and the training set and validation set were arranged and combined in a ratio of 4:1 to obtain 5 different training validation sets. The above model was trained using 5 different data sets, and 5 adenoids hypertrophy recognition models with excellent performance were obtained in the single angle dimension and patient dimension at different angles.

[0069] S3-2, model performance evaluation, includes the following steps:

[0070] After obtaining five adenoids hypertrophy recognition models in the single angle dimension and patient dimension at different angles, each model was evaluated using the test set. The evaluation indicators include AUC, average precision, accuracy, sensitivity, and specificity. These indicators are defined as follows:

[0071]

[0072] Among them, TP, FP, TN and FN represent true positive, false positive, true negative and false negative samples respectively, AUC is the area under the ROC characteristic curve, and AP is the area under the PR characteristic curve;

[0073] After obtaining the evaluation indicators of each model, the mean and variance of the models in the single angle dimension and patient dimension at different angles were calculated to evaluate the overall performance of the model.

[0074] The adenoids hypertrophy recognition system for multi-angle facial images includes a facial image preprocessing module, a recognition model building module, a recognition model pretraining module and a five-fold cross-validation module;

[0075] The face image preprocessing module is used to crop, size, normalize and organize the face data set to obtain face images for training;

[0076] The recognition model building module is used to build recognition models of single angle dimensions at different angles, and use the single angle dimension to perform deep feature extraction on the face image, so as to build a patient dimension recognition model;

[0077] The recognition model pre-training module is used to import the face images corresponding to the single angle dimension and patient dimension recognition models at different angles into the recognition model, perform model pre-training, and obtain a trained adenoids hypertrophy recognition model;

[0078] The five-fold cross-validation module is used to improve the performance and stability of the model and evaluate the overall performance of the model by comprehensively analyzing the performance of all models on the test set.

[0079] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A system for identifying adenoid hypertrophy based on multi-angle facial images, characterized in that: The method of using the system includes the following steps: S1, collection and preprocessing of multi-angle facial images, collecting frontal and side face datasets of patient children and normal children, and organizing the datasets according to patient IDs to obtain facial images of patient dimensions and single-view dimensions for training; S2, constructing a lightweight recognition model, using the face image obtained in S1 to construct a lightweight recognition model of adenoids hypertrophy in two different dimensions: patient dimension and single-view dimension, and importing the corresponding face image into the model to pre-train the model; S3, model optimization and evaluation, uses a five-fold cross-validation method to improve the performance and stability of the model built in S2, evaluates the model performance in each cross-validation, and comprehensively evaluates the overall performance of the model by combining the performance of all models on the test set; The step S2 comprises the following steps: S2-1, construction of adenoids hypertrophy model in single-view dimension; Based on the EfficientNet-b4 model, three single-view dimensional networks of 0°, 45° and 90° for the frontal face were constructed to extract the deep features of faces at different angles. S2-2, patient-dimensional adenoids hypertrophy model construction; The three different angles of face images of the same patient ID are input into the corresponding single-view dimension network to obtain the deep features of the patient at different face angles. Then, the three sets of features are fused in the direction of feature layers and input into the decision layer of EfficientNet-b4 to obtain the patient dimension model. S2-3, adenoids hypertrophy model training; Given a vector with class labels No. Input image at angles , the optimization objectives of the single-view dimension model are as follows: ； in, represents the model and its parameters, To identify the model The prediction vector of is the number of samples, the model parameters are updated with the above loss, the initial learning rate is selected as 0.01, and 120 rounds are iterated. The learning rate is reduced by 10 times at the 75th and 90th rounds respectively. When training the patient dimension model, the pre-trained models of three different angles are frozen, and only the decision layer of the patient dimension model is updated.

2. According to claim 1, a system for identifying adenoids hypertrophy based on multi-angle facial images, characterized in that: The step S1 comprises the following steps: S1-1, the collected face images include 0° frontal face images, 45° side face images, and 90° side face images to ensure that the dataset covers a variety of angles, expressions, and lighting conditions to improve the robustness of the model. All images are passed through the RetinaFace segmentation network to crop the images to only contain the face area; S1-2, all images are sampled into RGB images of size 512*512*3 to ensure that the input images meet the standard input size and format required by the model, and the data set is quality checked and cleaned to exclude low-quality or noisy image data; S1-3, each image is normalized individually using the mean and variance of the data set according to the following formula: ； in, express The mean of express The variance of the dataset, It is a face image that is normalized to the interval [0,1] using the maximum and minimum values. This operation can highlight the relative differences between images, allowing the model to converge to a higher accuracy faster.

3. The adenoids hypertrophy recognition system based on multi-angle facial images according to claim 1, characterized in that: The step S3 comprises the following steps: S3-1, five-fold cross validation, includes the following steps: Taking patient ID as the unit, the images were divided into training validation set and test set in a ratio of 4:

1. The training validation set was randomly divided into 5 equal parts, and the training set and validation set were arranged and combined in a ratio of 4:1 to obtain 5 different training validation sets. The above model was trained using 5 different data sets, and 5 adenoids hypertrophy recognition models were obtained in the single-view dimension and patient dimension at different angles. S3-2, model performance evaluation, includes the following steps: After obtaining five adenoids hypertrophy recognition models in the single-view dimension and patient dimension at different angles, each model was evaluated using the test set. The evaluation indicators included AUC, average precision, accuracy, sensitivity, and specificity. The indicators were defined as follows: ； Among them, average precision AP, accuracy Acc., sensitivity Sens., specificity Spec.; TP, FP, TN and FN represent true positive, false positive, true negative and false negative samples respectively, AUC is the area under the ROC characteristic curve, and AP is the area under the PR characteristic curve; After obtaining the evaluation indicators of each model, the mean and variance of the models in the single-view dimension and patient dimension at different angles were calculated to evaluate the overall performance of the model.

4. The adenoids hypertrophy recognition system based on multi-angle facial images according to claim 1, characterized in that: The adenoids hypertrophy recognition system for multi-angle facial images includes a facial image preprocessing module, a recognition model building module, a recognition model pretraining module and a five-fold cross-validation module; The face image preprocessing module is used to crop, size, normalize and organize the face data set to obtain face images for training; The recognition model building module is used to build recognition models of different single-view dimensions, and use the single-view dimension to perform deep feature extraction on the face image, so as to build a patient dimension recognition model; The recognition model pre-training module is used to import the face images corresponding to the single-view dimension and patient dimension recognition models at different angles into the recognition model, perform model pre-training, and obtain a trained adenoids hypertrophy recognition model; The five-fold cross-validation module is used to improve the performance and stability of the model and evaluate the overall performance of the model by comprehensively analyzing the performance of all models on the test set.

Images