Cooperative age estimation and gender classification method and system based on saliency map constraint

Abstract

The invention discloses a collaborative age estimation and gender classification method and system based on saliency map constraint. The method comprises the following steps: acquiring a training sample set, a verification sample set and a test sample set; performing filling, zooming and normalization processing on the image; performing iterative training on the adaptive scaling convolutional neural network; constructing a saliency map constraint training sample set, a saliency map constraint verification sample set and a saliency map constraint test sample set; obtaining a self-copying sample set and a mixed training sample set; constructing an adaptive scaling convolutional neural network based on saliency map constraint; x-ray skull localization side position films and copies of the X-ray skull localization side position films are spliced to serve as input data, and age estimation and gender classification results are obtained based on a self-adaptive scaling convolutional neural network constrained by a trained saliency map; according to the method, the accuracy and the stability of age estimation by using the X-ray image are improved, and the difference of age-increasing changes generated by development and aging of male and female bones is fully distinguished.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and relates to a collaborative age estimation and gender classification method, and specifically to a collaborative age estimation and gender classification method and system based on saliency map constraints. Background Art

[0002] Age estimation and gender classification are of great significance in civil trials, growth and development abnormality detection, and identity determination. They are also one of the urgent problems to be solved in the current forensic judicial practice. The mean absolute error is usually used as the accuracy indicator of the age estimation task. The smaller the mean absolute error, the better the accuracy of the age estimation. The performance of gender classification is usually evaluated by accuracy, specificity, sensitivity, and F-1 score. The larger the value of these indicators, the better the performance of gender classification.

[0003] Due to age-related changes, many parts of the human body can be used for age estimation. During the maturation and degeneration of human bones, age-related morphological changes occur, so age can be estimated by evaluating the size, shape and degree of ossification of bones. Currently, many parts of the human body are used for age estimation, including teeth, hands and wrists, knees, feet and clavicles. However, these parts usually work better in the pre-adult development period and are less effective for age estimation after adulthood. Compared with bones, teeth are the hardest organs in the human body and can be preserved for a long time under various extreme conditions in vivo and in vitro. Teeth are also easier to observe and image. Therefore, many forensic odontologists infer age by studying the development and physiological characteristics of teeth. However, since the development of teeth is completed after adulthood, age can only be estimated by age-related changes, such as tooth wear, periodontal disease, root transparency, cementum ringing, root resorption, increased root roughness, color changes and secondary dentin deposition. However, these age-related changes are difficult to quantify, and experienced clinicians or forensic doctors are still needed to participate in age assessment.

[0004] Skeletal structures such as the pelvis, skull, and long bones often show sexual dimorphism, meaning that there are characteristic differences between males and females. Sex classification using skeletal structures in X-ray images is a common method in forensic anthropology and medical imaging. In X-ray images, features such as the shape, size, density, and proportion of bones can be analyzed to determine sex. For example, various features of the skull, including the size and shape of the mandible (lower jaw), mastoid process, brow ridge (supraorbital ridge), and angle of the forehead (frontal bone), can show sexual dimorphism; the shape and dimensions of the pelvis, including the pelvic inlet, pelvic outlet, and sciatic notch, can differ between males and females. These differences are associated with childbirth and are often used for sex determination in forensic anthropology. In addition, some features of the hand and wrist bones, such as the size and shape of the carpal and metacarpal bones, can also show sexual dimorphism and can be used for sex classification. Although male-female dimorphism in these bones exists, it is difficult to accurately characterize and objectively quantify. In addition, methods based on manual measurement comparison are time-consuming and subjective, which limits the efficiency and accuracy of skeletal sex classification.

[0005] In recent years, deep learning methods have been widely used in various tasks of medical image analysis and processing, including organ / lesion segmentation, target detection, regression analysis, and diagnostic classification. As a key model in deep learning image processing, the Convolutional Neural Network (CNN) imitates the perception mechanism of human vision and can automatically extract features from images. Its parameter sharing and local perception mechanism enable accurate feature extraction with fewer parameters. Currently, CNN has been applied to age estimation and gender classification tasks in medical images.

[0006] In the task of automatic age estimation in medical images, for example, Nicolás Vila-Blanco et al. proposed a method for age estimation based on deep CNN for dental fragments in the paper Deep Neural Networks for Chronological Age Estimation From OPG Images in IEEE Transactions on Medical Imaging Volume 39 in 2020. This method takes a fragment containing only the tooth area taken from the front as input. Its model contains two relatively independent CNNs composed of convolutional layers and pooling layers in series for age estimation. Some features of the CNN used for gender classification are copied to the features of the CNN used for age estimation, so that the gender label guides the age estimation. Finally, the two CNNs output age and gender prediction values ​​respectively. The losses of the two tasks are calculated by the predicted values ​​of age estimation and gender classification and their corresponding labels, and the weighted sum is obtained to obtain the final loss. For example, Jaeyoung Kim et al. used the DenseNet-121 network to estimate age on X-ray dental fragments in the paper Development and Validation of Deep Learning-based Algorithms for the Estimation of Chronological Age using Panoramic Dental X-ray Images in Proceedings of Machine Learning Research in 2019. They added an attention mechanism and a curriculum learning mechanism to the DenseNet-121 network to better extract the features of the input image and gradually learn the mapping relationship between the extracted features and the age label from easy to difficult.

[0007] In the task of automatic gender classification of skull X-ray images, MV Rajee et al. proposed a gender classification method based on deep learning in the paper Gender classification on digital dental x-ray images using deep convolutional neural network in Volume 69 of Biomedical Signal Processing and Control in 2019. The method includes the preprocessing process of image enhancement and image bone foreground segmentation. The preprocessed image is input into a ResNet convolutional neural network model to obtain the gender classification result. Denis et al. published a paper titled Estimating Biological Gender from Panoramic Dental X-Ray Images at the 11th IEEE International Symposium on Image and Signal Processing and Analysis in 2019, which used a CNN model to classify the gender of dental X-ray images collected from Europeans. For example, Wenchi Ke et al. proposed a multi-feature fusion CNN model for gender classification of panoramic dental X-ray images in the paper Biological Gender Estimation from Panoramic Dental X-ray Images Based on Multiple Feature Fusion Model in Volume 21 of the journal Sensing and Imaging in 2020.

[0008] In summary, the existing automated age estimation and gender classification methods directly use medical images as input and extract features with the local receptive field of the CNN model, but do not strengthen the global features. As an effective image processing method, the saliency map constraint technology is widely used in feature extraction and image analysis. Through the saliency map constraint, the key areas in the X-ray skull lateral image can be more accurately located, and the accuracy and stability of age estimation can be improved. Although the age estimation method based on saliency map constraint and X-ray skull positioning lateral image (CN 112950631A) has achieved good application results, since the development process of human males and females is not exactly the same and there are gender differences, these methods ignore the correlation between age and gender, thereby reducing the accuracy and stability of age estimation and gender classification. In view of this, the present invention proposes a collaborative age estimation and gender classification method based on saliency map constraint. This method further introduces the function of gender classification based on the age estimation method based on saliency map constraints and X-ray head positioning lateral images. By comprehensively considering gender factors and feature information under saliency map constraints, a more accurate estimation of individual age is achieved. At the same time, gender classification results are provided, providing more comprehensive information for medical image analysis and auxiliary diagnosis. Summary of the invention

[0009] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a collaborative age estimation and gender classification method based on saliency map constraints and X-ray head positioning lateral images, which is used to solve the technical problems of poor estimation accuracy and stability in the prior art and can be used for age estimation and gender classification tasks in judicial review of evidence confirmation, growth and development abnormality detection and identity determination.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is: a collaborative age estimation and gender classification method based on saliency map constraints, comprising the following steps:

[0011] Generate a sample set constrained by the saliency map through the trained EfficientNet-B0 model and Grad-CAM;

[0012] Construct an adaptive scaling convolutional neural network EfficientNet-S model based on saliency map constraints. The EfficientNet-S model supports dual-channel input. For samples under the age of 25, the X-ray head positioning lateral film and its copy are used as the input data of the EfficientNet-S model. For samples aged 25 and above, the samples aged 25 and above and their corresponding saliency maps are used as the input data of the EfficientNet-S model.

[0013] Based on the trained EfficientNet-S, the X-ray skull positioning lateral film and its copy are used as input data to obtain the first predicted age. If the predicted age is less than 25 years old, the age estimation and gender classification are used as the final prediction result. If the predicted age is greater than or equal to 25 years old, the X-ray skull positioning lateral film and its corresponding saliency map are input into EfficientNet-S for prediction to obtain the second age estimation and gender classification result, which is used as the final prediction result.

[0014] Furthermore, when training the EfficientNet-B0 model, the X-ray head positioning lateral film is used as the input data of the model EfficientNet-B0 to train EfficientNet-B0; specifically, the following steps are included:

[0015] Get the training sample set D train , validation sample set D val And the test sample set D test ;

[0016] Setting model training parameters and iteratively training the adaptive scaling convolutional neural network EfficientNet-B0 includes the following steps:

[0017] S11, the training sample set D train Divide into S1 training sample subsets Each training sample subset Include b1 training samples and As the input of EfficientNet-B0, the features of the age-related change area of ​​each training sample are extracted. The age corresponding to the training sample is regressed and predicted through this feature to obtain b1 age prediction values ​​and b1 gender classification probabilities. Then, the average error function is used to calculate the average absolute error value L between each age prediction value and its corresponding age label. 1 a , and use the Softmax function and cross entropy loss function to calculate the cross entropy loss value L of each pair of gender classification probability and its corresponding gender label 1 g , represents the s1th training sample subset;

[0018] S12 uses the adaptive learning rate optimization algorithm Adam and is the learning rate, and the loss function L 1 a and L 1 g Update the weight W1 and bias I1 parameters of EfficientNet-B0 to obtain the adaptive scaling convolutional neural network EfficientNet-B0-e1 after the e1th iteration;

[0019] S13, the verification sample set D val As the input of EfficientNet-B0-e1, the features of the age-related change regions of each validation sample are extracted one by one, and the age and gender corresponding to the validation sample are regressed and classified through the features to obtain D val Corresponding age prediction value and gender classification values

[0020] S14, calculated using the average error function The mean absolute error (MAE) between each age estimate and its corresponding age label in v_1 , and calculate The accuracy of each gender classification and its corresponding gender label in ACC v_1 ;

[0021] S15, determine MAE v_1 <MAE best_1 or ACC v_1 <ACC best_1 Is it true? If so, set C1 = 0 and execute S16; otherwise, set C1 = C1 + 1 and execute S17;

[0022] S16, determine MAE v_1 <MAE best_1 Is it true? If true, let MAEbest_1 =MAE v_1 , and judge ACC v_1 <ACC best_1 Is it true? If true, let ACC best_1 =ACC v_1 ;

[0023] S17, determine whether e1=E1 or C1=3 is true, if so, obtain the trained adaptive scaling convolutional neural network Effi best , otherwise let e1=e1+1, and execute S11.

[0024] Further, obtain the training sample set D train , validation sample set D val And the test sample set D test The following steps are involved:

[0025] S21, obtain N X-ray head positioning lateral images D = {D n |1≤n≤N} and its corresponding age label set A={A n |1≤n≤N} and gender label G={G n |1≤n≥N}, and locate the lateral position of each X-ray head image D n Perform histogram equalization to obtain a set of X-ray head positioning lateral images after data enhancement Where N ≥ 10000, D n A represents the nth X-ray head positioning lateral image with a rectangular shape and containing the head, teeth and spine with age-related changes. n Indicates D n The corresponding age label, Indicates D n Data augmentation results:

[0026] S22, for each data enhanced X-ray head positioning lateral image The short side of is padded with zeros, and the size of the square X-ray head positioning lateral image after zero padding is scaled to H×H to obtain a set of X-ray head positioning lateral images after size transformation Among them, 800≤H≤1200, express The result after size transformation;

[0027] S23, will Each size-transformed X-ray head positioning lateral image Perform normalization processing to obtain a normalized X-ray head positioning lateral image set And D is divided into 3 The images in are divided into training sample set D train , validation sample set D val And the test sample set D test , express The normalized result of .

[0028] Furthermore, the saliency maps corresponding to all samples are generated by the trained EfficientNet-B0 model and Grad-CAM, and a sample set constrained by the saliency map is obtained based on the splicing of the saliency maps, including:

[0029] Get the training sample set D train , validation sample set D val And the test sample set D test ;

[0030] Construct saliency map to constrain training sample set DG train , saliency map constraint verification sample set DG val And the saliency map constrained test sample set DG test ;

[0031] The training sample set D train , validation sample set D val And the test sample set D test As the trained adaptive scaling convolutional neural network Effi best The input is used, and the gradient back propagation is performed based on the Grad-CAM algorithm to obtain D train , D val and D test The corresponding salient atlas G train , G val and G test ;

[0032] For the training sample set D train Each training sample in the salient atlas G train The corresponding saliency maps in the validation sample set D are spliced ​​together. val Each validation sample in the salient atlas G val The corresponding saliency maps in the test sample set D are spliced. test Each test sample in the salient atlas G test The corresponding saliency maps in are spliced ​​to realize the saliency region constraint of each training sample, each verification sample and each test sample. The size of each saliency map constraint sample after the saliency region constraint is H×H×2, 2 represents dual channels, and the saliency map constraint training sample set DG is obtained. train , saliency map constraint verification sample set DGval And the saliency map constrained test sample set DG test .

[0033] Furthermore, the iterative training of the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints includes:

[0034] S31, setting model training parameters;

[0035] S32, the mixed training sample set M train Divide into S2 mixed training sample subsets Each subset Include b2 mixed training samples and subset the mixed training samples As the input of EfficientNet-S, the features of the aging change area of ​​each mixed training sample are extracted. The age is regressed and predicted based on the features of the aging change area to obtain b2 age prediction values ​​and b2 gender classification probabilities. The average error function is used to calculate the average absolute error between each age prediction value and the age marked with its corresponding age label. The Softmax function and cross entropy loss function are used to calculate the cross entropy loss value L of each pair of gender classification probability and its corresponding gender label 2 g , represents the s2th mixed training sample subset;

[0036] S33, using the adaptive learning rate optimization algorithm Adam, and is the learning rate, through L 2 a +L 2 g Update the weight and bias parameters of EfficientNet-S to obtain the adaptive scaling convolutional neural network EfficientNet-S-e2 after the e2th iteration;

[0037] S34, the X-film self-replication verification sample set DD val Each self-replication verification sample in is input into the adaptive scaling convolutional neural network EfficientNet-B0-e2 one by one, and the features of the aging change area of ​​each self-replication verification sample are extracted. Then, the age and are regressed and classified respectively through the features of the aging change area of ​​each self-replication verification sample to obtain DD val Corresponding age prediction value and gender classification values

[0038] S35, judgment Is each age prediction value in less than 25? If so, add the age prediction value less than 25 to the predefined age prediction value set and add the corresponding gender classification results to the predefined gender classification value set Otherwise, the DG corresponding to the remaining age prediction values val The saliency map constraint verification samples in the saliency map are input into EfficientNet-S-e2, and the features of the age-related change area of ​​each saliency map constraint verification sample and the attention constraint information in the saliency map are extracted. The age and gender of each sample are regressed and classified respectively based on the features and constraint information, and the age prediction results and gender estimation are added to and middle, and All age prediction values ​​and all gender classification values ​​added in are the validation sample set D val Age estimates and sex-disaggregated results;

[0039] S36, calculated using the mean absolute error function The mean absolute error (MAE) between each age estimate and its corresponding age label in v_2 , and calculate The accuracy of each gender classification value and its corresponding gender label in ACC v_2 ;

[0040] S37, judge MAE v_2 <MAE best_2 or ACC v_2 <ACC best_2 Is it true? If so, set C2=0 and execute S28; otherwise, set C2=C2+1 and execute S39;

[0041] S38, judge MAE v_2 <MAE best_2 Is it true? If true, let MAE best_2 =MAE v_2 , and judge ACC v_2 <ACC best_2 Is it true? If true, let ACC best_2 =ACC v_2 ;

[0042] S39, determine whether e2=E2 or C2=3 is true, if so, obtain the trained adaptive scaling convolutional neural network K based on saliency map constraints, otherwise let e2=e2+1, and execute S32.

[0043] Further, obtaining the self-replication sample set and the mixed training sample set includes the following steps:

[0044] For the training sample set D train Make a copy and set D train Each training sample in the replicated training sample set D t ' rain The corresponding training samples in the validation sample set D are concatenated. val D val Each validation sample in is compared with the replicated validation sample set D v ' al The corresponding verification samples in the test sample set D are spliced ​​together. test Copy, D test Each test sample in is compared with the replicated test sample set D t ' est The corresponding test samples in are spliced ​​to convert the samples of size H×H into dual-channel self-replicating samples of size H×H×2, and the self-replicating training sample set DD is obtained. train , self-replication verification sample set DD val and self-replication test sample set DD test ;

[0045] For the training sample set D train Make a copy and determine the training sample set after copying Is the age label value corresponding to each sample in less than 25? If so, train Less than 25 training samples pass DD train Replace the corresponding self-replicated training samples in D train Training samples greater than 25 are passed through DG train The corresponding saliency map constraint training samples are replaced to obtain the mixed training sample set M train .

[0046] Further, the test results for obtaining age estimation and gender classification include:

[0047] The self-replicating test sample set DD test Each self-replication verification sample in the training is input into the trained EfficientNet-S one by one, the features of the aging change region of each self-replication test sample are extracted, and the age and gender of each sample are regressed and classified respectively according to the features of the aging change region, and DD test Corresponding age prediction value and gender classification values

[0048] judge Whether the age prediction value of each sample in is less than 25, the age prediction value and gender classification value corresponding to all samples with predicted age less than 25 are added to the pre-defined age prediction value set respectively. and and the DG corresponding to the remaining samples test The saliency map constraint verification samples in the saliency map are input into EfficientNet-S-e2, and the features of the age-related change area of ​​each saliency map constraint verification sample and the attention constraint information in the saliency map are extracted. The age regression prediction and gender estimation are performed on each sample based on the features and constraint information, and the age prediction results and gender classification results are added to and middle, and All the age prediction values ​​and gender estimation values ​​added in are the test sample set D test Age estimates and sex classifications.

[0049] Furthermore, the adaptive scaling convolutional neural network EfficientNet-B0 comprises a Stem module, Block1, Block2, Block3, Block4, Block5, Block6, Block7 and a final layer connected in sequence; wherein the Stem module comprises a convolutional layer, a rescaling layer, a normalization layer, a zero padding layer, a batch normalization layer and an activation layer connected in sequence; Block1 comprises a Module1 structure; Block2 and Block3 both comprise a Module2 structure, a Module3 structure and a summation layer connected in sequence; Block4, Block5 and Block6 both comprise a Module2 structure, a Module3 structure, a summation layer, a Module3 structure and a summation layer connected in sequence; Block7 comprises a Module2 structure; the final layer comprises a convolutional layer, a batch normalization layer and an activation layer connected in sequence; the Module1 structure comprises a deep convolutional layer, a batch normalization layer and an activation layer connected in sequence; the Module2 structure comprises a The structure of Module3 includes a global average pooling layer, a rescaling layer, a convolution layer and a convolution layer l connected in sequence; the final layer includes two parallel fully connected layers, of which the number of neurons in the fully connected layer that outputs the age prediction result is 1, and the number of neurons in the fully connected layer that outputs the gender classification result is 2; the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints includes the Stem module, Block1, Block2, Block3, Block4, Block5, Block6, Block7 and the final layer connected in sequence; different from the adaptive scaling convolutional neural network EfficientNet-B0, the Stem module in the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints includes a convolutional neural network that receives dual-channel saliency map constraint training samples, a rescaling layer, a normalization layer, a zero padding layer, a batch normalization layer and an activation layer connected in sequence.

[0050] Based on the concept of the above method, the present invention also provides a collaborative age estimation and gender classification system based on saliency map constraints and X-ray head positioning lateral images, including a sample set acquisition module, a model construction module and an estimation and classification module;

[0051] The sample set acquisition module is used to generate a sample set constrained by the saliency map through the trained EfficientNet-B0 model and Grad-CAM;

[0052] The model building module is used to build the EfficientNet-S model, an adaptive scaling convolutional neural network based on saliency map constraints. The EfficientNet-S model supports dual-channel input. For samples under the age of 25, the X-ray head positioning lateral film and its copy are used as the input data of the EfficientNet-S model. For samples aged 25 and above, the samples aged 25 and above and their corresponding saliency maps are used as the input data of the EfficientNet-S model.

[0053] The estimation and classification module is based on the trained EfficientNet-S. It uses the X-ray head positioning lateral film and its copy as input data to obtain the first predicted age. If the predicted age is less than 25 years old, the age estimation and gender classification are used as the final prediction result. If the predicted age is greater than or equal to 25 years old, the X-ray head positioning lateral film and its corresponding saliency map are input into EfficientNet-S to predict the second age estimation and gender classification result, which is used as the final prediction result.

[0054] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the collaborative age estimation and gender classification method based on saliency map constraints described in the present invention.

[0055] Compared with the prior art, the present invention has at least the following beneficial effects: the present invention provides global information of X-ray head positioning lateral film with the saliency map generated by the trained network and Grad-CAM method and constrains the attention of the collaborative age estimation and gender classification model, thereby improving the accuracy and stability of age estimation and gender classification based on X-ray head positioning lateral film; the present invention can collaboratively perform age estimation and gender classification, fully distinguish the differences in age-related changes in male and female bones during development and aging, thereby completing age estimation and gender classification at the same time, and can simultaneously improve the accuracy of age estimation and gender classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the implementation of the present invention;

[0057] Figure 2 It is the X-ray head positioning lateral image adopted by the present invention;

[0058] Figure 3 It is a saliency map of the normalized X-ray head positioning lateral image obtained by the present invention based on the Grad-CAM algorithm;

[0059] Figure 4is an example diagram of saliency map constraint data of the present invention;

[0060] Figure 5 This is an example diagram of self-replicating data of the present invention. DETAILED DESCRIPTION

[0061] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0062] Reference Figure 1 The method of the present invention for age estimation and gender classification comprises the following steps:

[0063] Step 1) Obtain training sample set D train , validation sample set D val And the test sample set D test :

[0064] Step 1a) Obtain a set of X-ray head positioning lateral images D = {D n |1≤n≤N} and its corresponding age label set A={A n |1≤n≤N} and gender label G={G n |1≤n≥N}, Table 1 shows the age and gender distribution of samples in D;

[0065]

[0066]

[0067] Step 1b) For each X-ray head positioning lateral image D n Perform histogram equalization to obtain a data-enhanced X-ray head positioning lateral image set Among them, N = 20178, D n Indicates the nth Figure 2 The shape shown is a rectangle, and the X-ray head positioning lateral image includes the head, teeth and spine, which are the areas of aging changes. Compared with the curved fragment that only includes the teeth area, the X-ray head positioning lateral image of the head, teeth and spine provides richer aging change characteristics. Indicates D n The data enhancement result is as follows. The histogram equalization method used in this embodiment is a histogram equalization method with limited contrast;

[0068] Step 1c) For each data-enhanced X-ray head positioning lateral image The short side of the image is padded with zeros, and the size of the square X-ray head positioning lateral image after zero padding is scaled to 1000×1000 to obtain a set of X-ray head positioning lateral images after size transformation.

[0069] Step 1d) To eliminate the influence of the dimension of the X-ray head positioning lateral image on the network training, each size-transformed X-ray head positioning lateral image Each image in the image is normalized, that is, the value of each pixel in each image is divided by the difference between the maximum and minimum values ​​of all pixels in the image to obtain a normalized X-ray head positioning lateral image set And D 3 The images in the image are divided into training sample set D according to the ratio of 7:1.5:1.5. train , validation sample set D val And the test sample set D test .

[0070] Step 2) Iteratively train the adaptive scaling convolutional neural network EfficientNet-B0:

[0071] Step 2a) Initialize the number of iterations to e1, the maximum number of iterations to E1 = 150, the learning rate to η1 = 0.0001, and the learning rate for the e1th iteration to The number of stagnation times is C1, and the optimal mean absolute error is MAE best_1 , the pre-trained weights and bias parameters of EfficientNet-B0 on ​​the natural image dataset ImageNet are used as the initial weights and bias parameters of EfficientNet-B0, and e1=1, C1=0, MAE best_1 =9223372036854775807,ACC best_1 =0,

[0072] Step 2b) From the training sample set D train After randomly discarding 2 samples, the training sample subsets are divided into S1. Represents the s1th training sample subset, each training sample subset Include b1 training samples and As the input of EfficientNet-B0, the features of the age-related change area of ​​each training sample are extracted, and the age is regressed and predicted based on the features of the age-related change area of ​​each training sample to obtain b1 age prediction values ​​and b1 gender classification probabilities. In this example, S1=2828, b1=5.

[0073] Step 2c) Use the average error function to calculate the average absolute error value L between each age prediction value and the age marked with its corresponding age label 1 a , the calculation formula is:

[0074]

[0075] in, Represents a subset of training samples The age prediction value corresponding to the nth training sample, A n Indicates A n The corresponding age label indicates the age, ∑ indicates summation, and |·| indicates calculation of absolute value;

[0076] Step 2d) Use the Softmax function and the cross entropy loss function to calculate the cross entropy loss value L of each pair of gender classification probabilities and their corresponding gender labels 1 g , the calculation formula is:

[0077]

[0078] in, and Represents the training sample subsets The output values ​​of male and female corresponding to the nth training sample, and are the classification probabilities of male and female respectively. If the gender of the nth training sample is male, then g n = 0, if the gender of the nth training sample is female, then g n =1;

[0079] Step 2e) Optimize the model: Adopt the adaptive learning rate optimization algorithm Adam, and is the learning rate, through L 1 a and L 1 g The weight W1 and bias I1 parameters of EfficientNet-B0 are updated to obtain the adaptive scaling convolutional neural network EfficientNet-B0-e1 after the e1th iteration. The update formulas are:

[0080]

[0081]

[0082] Among them, W1 and I1 are the weight and bias parameters of EfficientNet-B0, W′ and I1′ are the update results of W1 and I1, respectively. represents the gradient of L1 with respect to W1, represents the gradient of L1 with respect to I1, · represents the dot product;

[0083] Step 2f) Verification: Verify the sample set D val As the input of EfficientNet-B0-e1, the features of the aging-related change regions of each validation sample are extracted one by one. Based on the features of the aging-related change regions of each validation sample, the age and gender corresponding to the validation sample are regressed and classified to obtain D val Corresponding age prediction value and gender classification values

[0084] (2e) Evaluate the performance of the model: use the average error function to calculate The mean absolute error (MAE) between each age estimate and its corresponding age label in v_1 , and calculate The accuracy of each gender classification and its corresponding gender label in ACC v_1 ;

[0085] Step 2f) Determine MAE v_1 <MAE best_1 or ACC v_1 <ACC best_1 Is it true? If so, set C1 = 0 and execute step 2g); otherwise, set C1 = C1 + 1 and execute step 2h);

[0086] Step 2g) Determine MAE v_1 <MAE best_1 Is it true? If true, let MAE best_1 =MAE v_1 , and judge ACC v_1 <ACC best_1 Is it true? If true, let ACC best_1 =ACC v_1 ;

[0087] Step 2h) Determine whether e1=E1 or C1=3 is true, if so, obtain the trained adaptive scaling convolutional neural network Effi best , otherwise let e1=e1+1, and execute step 2b);

[0088] Step 3) For the training sample set D train , validation sample set D val And the test sample set D test Perform salient region constraints:

[0089] Step 3a) The training sample set Dtrain , validation sample set D val And the test sample set D test As the trained adaptive scaling convolutional neural network Effi best The input is used, and the gradient back propagation is performed based on the Grad-CAM algorithm to characterize the contribution of each area in the image to the final result, and D train , D val and D test The corresponding salient atlas G train , G val and G test , the process of generating a saliency map in this embodiment is:

[0090] Step 3a1) Training sample set D train , validation sample set D val And the test sample set D test Each sample in is a trained adaptive scaling convolutional neural network Effi best The global average of the 320 feature maps of each sample is obtained, and the rth (1≤r≤320, r∈Z) feature map F is calculated. r The global average of the gradient α r The formula is:

[0091]

[0092] Among them, N p Represents the feature map M r The number of pixels, Indicates F r The pixel at position (i,j) is Effi best Output about input samples;

[0093] Step 3a2) Take α k Calculate the weighted sum of all feature maps for the weights and get the average feature map M:

[0094]

[0095] Step 3a3) To remove negative significance, use the ReLU function to process M, and obtain Figure 3 The saliency map M corresponding to the sample is shown * :

[0096] M * =ReLU(M)

[0097] like Figure 3As shown in the figure, the saliency map contains the contribution of each area of ​​the X-ray head positioning lateral image to the final age estimation result, and since the saliency map is based on the trained Effi best The generated image contains characteristic information of each age group, so it can be used as a constraint information to provide global attention to each X-ray head positioning lateral image, thereby improving the performance and stability of the network.

[0098] Step 3b) For the training sample set D train Each training sample in the salient atlas G train The corresponding saliency maps in the validation sample set D are spliced ​​together. val Each validation sample in the salient atlas G val The corresponding saliency maps in the test sample set D are spliced. test Each test sample in the salient atlas G test The corresponding saliency maps in are spliced ​​to realize the saliency region constraint of each training sample, each verification sample and each test sample. The size of each saliency map constraint sample after the saliency region constraint is H×H×2, 2 represents dual channels, and the saliency map constraint training sample set DG is obtained. train , saliency map constraint verification sample set DG val And the saliency map constrained test sample set DG test , the process of obtaining saliency map constraint samples in this embodiment is:

[0099] Step 3b1) for D train , D val and D test Each X-ray head positioning lateral film P with a size of 1000×1000 pixels d Select G train , G val and G test The corresponding salient map P with a size of 1000×1000 pixels g ;

[0100] Step 3b2) For each pair P d and P g Splice and get Figure 4 The corresponding saliency map constraint sample P is shown dg , the size is 1000×1000×2 pixels, where 2 represents dual channels, one channel is the original X-ray head positioning lateral film, and the other channel is the saliency map corresponding to the original X-ray head positioning lateral film;

[0101] Step 4) Obtain the self-replication sample set and the mixed training sample set:

[0102] Step 4a) For the training sample set D train Make a copy and set Dtrain Each training sample in the replicated training sample set D t ' rain The corresponding training samples in the validation sample set D are concatenated. val Make a copy and set D val Each validation sample in is compared with the replicated validation sample set D v ' al The corresponding verification samples in the test sample set D are spliced ​​together. test Make a copy and set D test Each test sample in is compared with the replicated test sample set D t ' est The corresponding test samples in are spliced ​​to convert the samples of size H×H into dual-channel self-replicating samples of size H×H×2, and the self-replicating training sample set DD is obtained. train , self-replication verification sample set DD val and self-replication test sample set DD test ,The process of obtaining self-replicating samples in this example is:

[0103] Step 4a1) For D train , D val and D test Each X-ray head positioning lateral film P with a size of H×H pixels d Select D t ' rain , D v ' al and D t ' est The corresponding salient map P of size H×H pixels d ';

[0104] Step 4a2) For each pair P d and P d 'Splice and get Figure 5 The corresponding saliency map constrained sample P dd′ , the size is H×H×2 pixels, where 2 represents two channels, and both channels are the same X-ray head positioning lateral film. The reason for copying and splicing the X-ray head positioning lateral film is to ensure the consistency of the network input size;

[0105] Step 4b) For the training sample set D train Make a copy and determine the training sample set after copying Is the age label value corresponding to each sample in less than 25? If so, train Less than 25 training samples pass DD train Replace the corresponding self-replicated training samples in D trainTraining samples greater than 25 are passed through DG train The corresponding saliency map constraint training samples are replaced to obtain the mixed training sample set M train ;

[0106] Step 5) Construct an adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints:

[0107] Construct an adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints, which includes the sequentially connected Stem module, Block1, Block2, Block3, Block4, Block5, Block6, Block7 and the final layer; the Stem module includes a sequentially connected convolutional neural network that receives dual-channel saliency map constraint training samples, a rescaling layer, a normalization layer, a zero padding layer, a batch normalization layer and an activation layer; Block1 contains a Module1 structure; Block2 and Block3 both contain sequentially connected Module2 structures, Module3 structures and summation layers; Block4, Block5 and Block6 all contain sequentially connected Mo dule2 structure, Module3 structure, sum layer, Module3 structure and sum layer; Block7 contains a Module2 structure; the final layer contains sequentially connected convolutional layers, batch normalization layers and activation layers; the Module1 structure contains sequentially connected deep convolutional layers, batch normalization layers and activation layers; the Module2 structure contains sequentially connected deep convolutional layers, batch normalization layers, activation layers, zero padding layers, deep convolutional layers, batch normalization layers and activation layers; the Module3 structure contains sequentially connected global average pooling layers, rescaling layers, convolutional layers and convolutional layers; the final layer contains two parallel fully connected layers, where the number of neurons in the fully connected layer that outputs the age prediction result is 1, and the number of neurons in the fully connected layer that outputs the gender classification result is 2;

[0108] The difference between the EfficientNet-S network and the basic EfficientNet-B0 structure is that the input layer is modified from a convolutional neural network to a convolutional neural network that accepts dual-channel saliency map-constrained training samples, thereby realizing the constraint of the saliency map on the saliency area of ​​the X-ray head positioning lateral film;

[0109] Step 6) Iteratively train the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints:

[0110] Step 6a) Initialize the number of iterations to e2, the maximum number of iterations to E2 = 150, the learning rate to η2 = 0.0001, and the learning rate for the e2th iteration to The number of stagnation times is C2, and the optimal mean absolute error is MAE best_2 , and let e2=1, C2=0, MAE best_2 =9223372036854775807, The pre-trained weights and bias parameters of EfficientNet-S on the natural image dataset ImageNet are used as the initial weights and bias parameters of EfficientNet-S;

[0111] Step 6b) Mix the training sample set M train After randomly discarding two samples, it is divided into S2 mixed training sample subsets represents the s2th mixed training sample subset, each subset Include b2 mixed training samples and The features of the age-related change region of each mixed training sample are extracted as the input of EfficientNet-B0, and the age is regressed and predicted based on the features of the age-related change region of each mixed training sample to obtain b2 age prediction values. In this embodiment, S2=2828, b2=5;

[0112] Step 6c) Use the average error function to calculate the average absolute error between each age prediction value and the age marked with its corresponding age label The calculation formula is:

[0113]

[0114] in, Represents a subset of training samples The age prediction value corresponding to the nth training sample, A n Indicates A n The corresponding age label indicates the age, ∑ indicates summation, and |·| indicates calculation of absolute value;

[0115] Step 6d) Use the Softmax function and the cross entropy loss function to calculate the cross entropy loss value of each pair of gender class probabilities and their corresponding gender label age The calculation formula is:

[0116]

[0117] in, and Represents the training sample subsets The output values ​​of male and female corresponding to the nth training sample, and are the classification probabilities of male and female respectively. If the gender of the nth training sample is male, then g n= 0, if the gender of the nth training sample is female, then g n =1.

[0118] Step 6e) Adopt the adaptive learning rate optimization algorithm Adam, and is the learning rate, through The weights and biases of EfficientNet-B0 are updated to obtain the adaptive scaling convolutional neural network EfficientNet-B0-e2 after the e2th iteration. The update formulas are:

[0119]

[0120]

[0121] Among them, W2 and I2 are the weight and bias parameters of EfficientNet-B0, respectively, and W2′ and I2′ represent the update results of W2 and I2, respectively. represents the gradient of L2 with respect to W2, represents the gradient of L2 with respect to I2, · represents the dot product;

[0122] Step 6f) The X-sheet self-replication verification sample set DD val Each self-replication verification sample in the DD is input into EfficientNet-B0-e2 one by one, the features of the aging change region of each self-replication verification sample are extracted, and the age and gender are regressed and classified respectively according to the features of the aging change region of each self-replication verification sample to obtain DD val Corresponding age prediction value and gender classification values

[0123] Step 6g) Firstly, a rough classification is performed to distinguish between samples under 25 years old and samples over 25 years old. Is each age prediction value in less than 25? If so, add the age prediction value less than 25 to the predefined age prediction value set and add the corresponding gender classification results to the predefined gender classification value set Otherwise, the age regression prediction is performed again with the samples constrained by the saliency map, that is, the DG corresponding to the remaining age prediction values test The saliency map constraint verification samples are input into EfficientNet-B0-e2, and the features of the age-related change area of ​​each saliency map constraint verification sample and the attention constraint information in the saliency map are extracted. The age and gender of each sample are regressed and classified based on the features and constraint information of the age-related change area of ​​each saliency map constraint verification sample, and then the age prediction results and gender estimation are added to and middle, and middle;

[0124] Step 6h) Calculate using mean absolute error function The mean absolute error (MAE) between each age estimate and its corresponding age label in v_2 , and calculate The accuracy of each gender classification value and its corresponding gender label in ACC v_2 ;

[0125] Step 6i) Determine MAE v_2 <MAE best_2 or ACC v_2 <ACC best_2 Is it true? If so, set C2 = 0 and execute step (5j); otherwise, set C2 = C2 + 1 and execute step (5k);

[0126] Step 5j) Determine MAE v_2 <MAE best_2 Is it true? If true, let MAE best_2 =MAE v_2 , and judge ACC v_2 <ACC best_2 Is it true? If true, let ACC best_2 =ACC v_2 ;

[0127] Step 6k) Determine whether e2=E2 or C2=3. If so, obtain the trained adaptive scaling convolutional neural network K. Otherwise, e2=e2+1, and execute step 6b);

[0128] Step 7) Get the age estimation result:

[0129] Step 7a) The self-replication test sample set DD test Each self-replication verification sample in is input into K one by one, the characteristics of the aging change region of each self-replication test sample are extracted, and the age and gender of each sample are regressed and classified according to the characteristics of the aging change region of each self-replication test sample to obtain DD test Corresponding age prediction value and gender classification values

[0130] Step 7b) First, perform a rough classification to distinguish between samples under 25 years old and samples over 25 years old. Whether the age prediction value of each sample in is less than 25, the age prediction value and gender classification value corresponding to all samples with predicted age less than 25 are added to the pre-defined age prediction value set respectively. and and the DG corresponding to the remaining samples test The saliency map constraint verification samples in the saliency map are input into EfficientNet-S-e2, and the features of the aging change area of ​​each saliency map constraint verification sample and the attention constraint information in the saliency map are extracted. The age regression prediction and gender estimation are performed on each sample based on the features and constraint information of the aging change area of ​​each saliency map constraint verification sample. The age prediction results and gender classification results are then added to and middle, and All the age prediction values ​​and gender estimation values ​​added in are the test sample set D test Age estimation and gender classification, in this embodiment The mean absolute error between the age estimates and their corresponding age annotation labels is 1.30.

[0131] Based on the trained EfficientNet-S, the X-ray skull positioning lateral film and its copy are used as input data to obtain the first predicted age. If the predicted age is less than 25 years old, the age estimation and gender classification are used as the final prediction result. If the predicted age is greater than or equal to 25 years old, the X-ray skull positioning lateral film and its corresponding saliency map are input into EfficientNet-S for prediction to obtain the second age estimation and gender classification result, and the classification result is the final prediction result.

[0132] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the collaborative age estimation and gender classification method based on saliency map constraints described in the present invention.

[0133] The computer device may be a laptop computer, a desktop computer or a workstation.

[0134] The processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).

[0135] The memory described in the present invention may be an internal storage unit of a notebook computer, a desktop computer or a workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0136] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, it is possible to make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A collaborative age estimation and gender classification method based on saliency map constraints, characterized in that: The following steps are involved: The sample set constrained by the saliency map generated by the trained EfficientNet-B0 model and Grad-CAM; Construct an adaptive scaling convolutional neural network EfficientNet-S model based on saliency map constraints. The EfficientNet-S model supports dual-channel input. For samples under the age of 25, the X-ray head positioning lateral film and its copy are used as the input data of the EfficientNet-S model. For samples aged 25 and above, the samples aged 25 and above and their corresponding saliency maps are used as the input data of the EfficientNet-S model. Based on the trained EfficientNet-S, the X-ray skull positioning lateral film and its copy are used as input data to obtain the first predicted age. If the predicted age is less than 25 years old, the age estimation and gender classification are used as the final prediction result. If the predicted age is greater than or equal to 25 years old, the X-ray skull positioning lateral film and its corresponding saliency map are input into EfficientNet-S for prediction to obtain the second age estimation and gender classification result, and the classification result is the final prediction result.

2. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 1, characterized in that: When training the EfficientNet-B0 model, the X-ray head positioning lateral film is used as the input data of the EfficientNet-B0 model to train EfficientNet-B0; specifically, the following steps are included: Get the training sample set D train , validation sample set D val And the test sample set D test ; Setting model training parameters and iteratively training the adaptive scaling convolutional neural network EfficientNet-B0 includes the following steps: S11, the training sample set D train Divide into S1 training sample subsets Each training sample subset Include b1 training samples and As the input of EfficientNet-B0, the features of the age-related change area of ​​each training sample are extracted. The age corresponding to the training sample is regressed and predicted through this feature to obtain b1 age prediction values ​​and b1 gender classification probabilities. Then, the average error function is used to calculate the average absolute error value L between each age prediction value and its corresponding age label. 1 a , and use the Softmax function and cross entropy loss function to calculate the cross entropy loss value L of each pair of gender classification probability and its corresponding gender label 1 g , represents the s1th training sample subset; S12 uses the adaptive learning rate optimization algorithm Adam and is the learning rate, and the loss function L 1 a and L 1 g Update the weight W1 and bias I1 parameters of EfficientNet-B0 to obtain the adaptive scaling convolutional neural network EfficientNet-B0-e1 after the e1th iteration; S13, the verification sample set D val As the input of EfficientNet-B0-e1, the features of the age-related change regions of each validation sample are extracted one by one, and the age and gender corresponding to the validation sample are regressed and classified through the features to obtain D val Corresponding age prediction value and gender classification values S14, calculated using the average error function The mean absolute error (MAE) between each age estimate and its corresponding age label in v_1 , and calculate The accuracy of each gender classification and its corresponding gender label in ACC v_1 ; S15, determine MAE v_1 <MAE best_1 or ACC v_1 <ACC best_1 Is it true? If so, set C1 = 0 and execute S16; otherwise, set C1 = C1 + 1 and execute S17; S16, determine MAE v_1 <MAE best_1 Is it true? If true, let MAE best_1 =MAE v_1 , and judge ACC v_1 <ACC best_1 Is it true? If true, let ACC best_1 =ACC v_1 ; S17, determine whether e1=E1 or C1=3 is true, if so, obtain the trained adaptive scaling convolutional neural network Effi best , otherwise let e1=e1+1, and execute S11.

3. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 1, characterized in that: Get the training sample set D train , validation sample set D val And the test sample set D test The following steps are involved: S21, obtain N X-ray head positioning lateral images D = {D n |1≤n≤N} and its corresponding age label set A={A n |1≤n≤N} and gender label G={G n |1≤n≥N}, and locate the lateral position of each X-ray head image D n Perform histogram equalization to obtain a set of X-ray head positioning lateral images after data enhancement Where N ≥ 10000, D n A represents the nth X-ray head positioning lateral image with a rectangular shape and containing the head, teeth and spine with age-related changes. n Indicates D n The corresponding age label, Indicates D n Data augmentation results: S22, for each data enhanced X-ray head positioning lateral image The short side of is padded with zeros, and the size of the square X-ray head positioning lateral image after zero padding is scaled to H×H to obtain a set of X-ray head positioning lateral images after size transformation Among them, 800≤H≤1200, express The result after size transformation; S23, will Each size-transformed X-ray head positioning lateral image Perform normalization processing to obtain a normalized X-ray head positioning lateral image set And D is divided into 3 The images in are divided into training sample set D train , validation sample set D val And the test sample set D test , express The normalized result of .

4. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 1, characterized in that: The saliency maps corresponding to all samples are generated by the trained EfficientNet-B0 model and Grad-CAM, and a sample set constrained by the saliency map is obtained based on the splicing of the saliency maps, including: Get the training sample set D train , validation sample set D val And the test sample set D test ; Construct saliency map to constrain training sample set DG train , saliency map constraint verification sample set DG val And the saliency map constrained test sample set DG test ; The training sample set D train , validation sample set D val And the test sample set D test As the trained adaptive scaling convolutional neural network Effi best The input is used, and the gradient back propagation is performed based on the Grad-CAM algorithm to obtain D train , D val and D test The corresponding salient atlas G train , G val and G test ; For the training sample set D train Each training sample in the salient atlas G train The corresponding saliency maps in the validation sample set D are spliced ​​together. val Each validation sample in the salient atlas G val The corresponding saliency maps in the test sample set D are spliced. test Each test sample in the salient atlas G test The corresponding saliency maps in are spliced ​​to realize the saliency region constraint of each training sample, each verification sample and each test sample. The size of each saliency map constraint sample after the saliency region constraint is H×H×2, 2 represents dual channels, and the saliency map constraint training sample set DG is obtained. train , saliency map constraint verification sample set DG val And the saliency map constrained test sample set DG test .

5. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 1, characterized in that: Iterative training of the saliency map-constrained adaptive scaling convolutional neural network EfficientNet-S includes: S31, setting model training parameters; S32, the mixed training sample set M train Divide into S2 mixed training sample subsets Each subset Include b2 mixed training samples and subset the mixed training samples As the input of EfficientNet-S, the features of the aging change area of ​​each mixed training sample are extracted. The age is regressed and predicted based on the features of the aging change area to obtain b2 age prediction values ​​and b2 gender classification probabilities. The average error function is used to calculate the average absolute error between each age prediction value and the age marked with its corresponding age label. The Softmax function and cross entropy loss function are used to calculate the cross entropy loss value L of each pair of gender classification probability and its corresponding gender label 2 g , represents the s2th mixed training sample subset; S33, using the adaptive learning rate optimization algorithm Adam, and is the learning rate, through L 2 a +L 2 g Update the weight and bias parameters of EfficientNet-S to obtain the adaptive scaling convolutional neural network EfficientNet-S-e2 after the e2th iteration; S34, the X-film self-replication verification sample set DD val Each self-replication verification sample in is input into the adaptive scaling convolutional neural network EfficientNet-B0-e2 one by one, and the features of the aging change area of ​​each self-replication verification sample are extracted. Then, the age and are regressed and classified respectively through the features of the aging change area of ​​each self-replication verification sample to obtain DD val Corresponding age prediction value and gender classification values S35, judgment Is each age prediction value in less than 25? If so, add the age prediction value less than 25 to the predefined age prediction value set and add the corresponding gender classification results to the predefined gender classification value set Otherwise, the DG corresponding to the remaining age prediction values val The saliency map constraint verification samples in the saliency map are input into EfficientNet-S-e2, and the features of the age-related change area of ​​each saliency map constraint verification sample and the attention constraint information in the saliency map are extracted. The age and gender of each sample are regressed and classified respectively based on the features and constraint information, and the age prediction results and gender estimation are added to and middle, and All age prediction values ​​and all gender classification values ​​added in are the validation sample set D val Age estimates and sex-disaggregated results; S36, calculated using the mean absolute error function The mean absolute error (MAE) between each age estimate and its corresponding age label in v_2 , and calculate The accuracy of each gender classification value and its corresponding gender label in ACC v_2 ; S37, judge MAE v_2 <MAE best_2 or ACC v_2 <ACC best_2 Is it true? If so, set C2=0 and execute S28; otherwise, set C2=C2+1 and execute S39; S38, judge MAE v_2 <MAE best_2 Is it true? If true, let MAE best_2 =MAE v_2 , and judge ACC v_2 <ACC best_2 Is it true? If true, let ACC best_2 =ACC v_2 ; S39, determine whether e2=E2 or C2=3 is true, if so, obtain the trained adaptive scaling convolutional neural network K based on saliency map constraints, otherwise let e2=e2+1, and execute S32.

6. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 4, characterized in that: Obtaining the self-replication sample set and the mixed training sample set includes the following steps: For the training sample set D train Make a copy and set D train Each training sample in and the replicated training sample set D′ train The corresponding training samples in the validation sample set D are concatenated. val D val Each validation sample in and the replicated validation sample set D′ val The corresponding verification samples in the test sample set D are spliced ​​together. test Copy, D test Each test sample in and the replicated test sample set D′ test The corresponding test samples in are spliced ​​to convert the samples of size H×H into dual-channel self-replicating samples of size H×H×2, and the self-replicating training sample set DD is obtained. train , self-replication verification sample set DD val and self-replication test sample set DD test ; For the training sample set D train Make a copy and determine the training sample set after copying Is the age label value corresponding to each sample in less than 25? If so, train Less than 25 training samples pass DD train Replace the corresponding self-replicated training samples in D train Training samples greater than 25 are passed through DG train The corresponding saliency map constraint training samples are replaced to obtain the mixed training sample set M train .

7. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 4, characterized in that: The average error function described in S32 is used to calculate the average absolute error between each age prediction value and the age marked with the corresponding age label The calculation formula is: in, Represents a subset of training samples The age prediction value corresponding to the nth training sample, A n Indicates A n The corresponding age label indicates the age, ∑ indicates summation, and |·| indicates calculation of absolute value; The Softmax function and cross entropy loss function are used to calculate the cross entropy loss value of each pair of gender class probabilities and their corresponding gender labels. The calculation formula is: in, and Represents the training sample subsets The output values ​​of male and female corresponding to the nth training sample, and are the classification probabilities of male and female respectively. If the gender of the nth training sample is male, then g n = 0, if the gender of the nth training sample is female, then g n =1; The adaptive learning rate optimization algorithm Adam is used as described in S33, and is the learning rate, through Update the weights and biases of EfficientNet-B0, and the update formulas are: Among them, W2 and I2 are the weight and bias parameters of EfficientNet-B0, respectively, and W2′ and I2′ represent the update results of W2 and I2, respectively. represents the gradient of L2 with respect to W2, represents the gradient of L2 with respect to I2, and · represents the dot product.

8. The collaborative age estimation and gender classification method based on saliency map constraints according to claim 1, characterized in that: The adaptive scaling convolutional neural network EfficientNet-B0 comprises a Stem module, Block1, Block2, Block3, Block4, Block5, Block6, Block7 and a final layer connected in sequence; wherein the Stem module comprises a convolutional layer, a rescaling layer, a normalization layer, a zero padding layer, a batch normalization layer and an activation layer connected in sequence; Block1 comprises a Module1 structure; Block2 and Block3 both comprise a Module2 structure, a Module3 structure and a summation layer connected in sequence; Block4, Block5 and Block6 both comprise a Module2 structure, a Module3 structure, a summation layer, a Module3 structure and a summation layer connected in sequence; Block7 comprises a Module2 structure; the final layer comprises a convolutional layer, a batch normalization layer and an activation layer connected in sequence; the Module1 structure comprises a deep convolutional layer, a batch normalization layer and an activation layer connected in sequence; The Module2 structure includes a deep convolutional layer, a batch normalization layer, an activation layer, a zero padding layer, a deep convolutional layer, a batch normalization layer and an activation layer connected in sequence; the Module3 structure includes a global average pooling layer, a rescaling layer, a convolutional layer and a convolutional layer l connected in sequence; the final layer includes two parallel fully connected layers, where the number of neurons in the fully connected layer that outputs the age prediction result is 1, and the number of neurons in the fully connected layer that outputs the gender classification result is 2; the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints includes a sequentially connected Stem module, Block1, Block2, Block3, Block4, Block5, Block6, Block7 and a final layer; different from the adaptive scaling convolutional neural network EfficientNet-B0, the Stem module in the adaptive scaling convolutional neural network EfficientNet-S based on saliency map constraints includes a sequentially connected convolutional neural network that receives dual-channel saliency map constraint training samples, a rescaling layer, a normalization layer, a zero padding layer, a batch normalization layer and an activation layer.

9. A collaborative age estimation and gender classification system based on saliency map constraints and X-ray head positioning lateral images, characterized in that: It includes a sample set acquisition module, a model building module and an estimation and classification module; The sample set acquisition module is used to generate a sample set constrained by a saliency map through the trained EfficientNet-B0 model and Grad-CAM; The model building module is used to build the EfficientNet-S model, an adaptive scaling convolutional neural network based on saliency map constraints. The EfficientNet-S model supports dual-channel input. For samples under the age of 25, the X-ray head positioning lateral film and its copy are used as the input data of the EfficientNet-S model. For samples aged 25 and above, the samples aged 25 and above and their corresponding saliency maps are used as the input data of the EfficientNet-S model. The estimation and classification module is based on the trained EfficientNet-S. It uses the X-ray head positioning lateral film and its copy as input data to obtain the first predicted age. If the predicted age is less than 25 years old, the age estimation and gender classification are used as the final prediction result. If the predicted age is greater than or equal to 25 years old, the X-ray head positioning lateral film and its corresponding saliency map are input into EfficientNet-S to predict the second age estimation and gender classification result, which is used as the final prediction result.

10. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the collaborative age estimation and gender classification method based on saliency map constraints as described in any one of claims 1 to 8.

Images