Head-side X-ray age and gender estimation method and system based on latent variable model

Abstract

The present invention discloses a method and system for estimating age and gender from cephalometric X-ray images based on a latent variable model. The method includes: obtaining a cephalometric X-ray image of a target to be detected; preprocessing the cephalometric X-ray image and inputting it into an age and gender recognition model to generate an age and gender estimate for the target to be detected. Among them, the age and gender recognition model includes a feature extraction unit, and the feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16. The specific process includes: obtaining the dependency relationships between the convolutional layers in the feature extractor of the pre-trained VGG16, and establishing a dependency graph; grouping the parameters in the dependency graph, and the parameters in each group have the same dependency relationship; for each group, calculating the importance of each parameter in the group; using the importance of each parameter, calculating the joint importance of each group; using the joint importance of each group to prune the group. The present invention greatly reduces the computational complexity and improves the processing speed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a multi-task estimation method and system for the age and gender of cephalometric X-ray films based on a latent variable model. Background Art

[0002] Cephalometric X-ray films have a wide range of applications in multiple disciplines, including medicine, anthropology, and forensic medicine. In the field of anthropology, age and gender estimation based on cephalometric X-ray films can reveal patterns of human evolution and population migration. In forensic medicine, experts analyze the characteristics of cephalometric X-ray films to help determine the identity and age of unidentified corpses. Orthodontics in dentistry also uses cephalometric X-ray films to analyze the characteristics of orthodontic patients in order to provide them with the most suitable treatment plan. However, the actual applications in these fields often rely on the personal judgment and interpretation of experts, which undoubtedly brings a certain degree of subjectivity and also requires a large amount of human resources. Facing the current situation of scarce medical resources and shortage of professional medical talents in China, the feasibility of relying on manual evaluation of cephalometric X-ray films is becoming lower and lower. Therefore, more and more demands are focused on a method and system that can automatically process and accurately estimate the age and gender of cephalometric X-ray film images.

[0003] However, the existing processing method models currently have a relatively complex structure, resulting in an increase in computational complexity and a significant impact on the processing speed. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a multi-task estimation method and system for the age and gender of cephalometric X-ray films based on a latent variable model, so as to lightweight the latent variable model and then cooperate with the Transformer module for multi-task deep learning, which not only greatly reduces the computational complexity but also improves the processing speed.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A multi-task estimation method for the age and gender of cephalometric X-ray films based on a latent variable model, comprising:

[0007] Obtain the cephalometric X-ray film of the target to be detected;

[0008] Preprocess the cephalometric X-ray film of the target to be detected;

[0009] Input the preprocessed image into the age and gender recognition model to generate an age and gender estimate for the target to be detected,

[0010] wherein, the age and gender recognition model includes a feature extraction unit, an age prediction unit, and a gender prediction unit.

[0011] The feature extraction unit is used to extract features from the input image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction.

[0012] The feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16, and its specific process includes:

[0013] Obtain the dependency relationship between each convolutional layer in the feature extractor of the pre-trained VGG16, and establish a dependency graph. Each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel;

[0014] Group the parameters in the dependency graph, and the parameters in each group have the same dependency relationship;

[0015] For each group, calculate the importance of each parameter in the group;

[0016] Using the importance of each parameter, calculate the joint importance of each group;

[0017] Using the joint importance of each group, prune the groups.

[0018] The present invention also provides a multi-task estimation system for the age and gender of the cephalic X-ray film based on the latent variable model, including:

[0019] An image acquisition module, configured to acquire the cephalic X-ray film of the target to be detected and a data set, and the data set contains cephalic X-ray images, and each image has a corresponding age label and gender label;

[0020] A preprocessing module, configured to preprocess the acquired image;

[0021] A model training module, configured to train the age and gender recognition model using the data set;

[0022] A prediction module, configured to predict the gender and age of the target to be detected in the cephalic X-ray film of the target to be detected using the age and gender recognition model,

[0023] Wherein, the age and gender recognition model includes a feature extraction unit, an age prediction unit and a gender prediction unit,

[0024] The feature extraction unit is used to extract features from the input image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction,

[0025] The feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16, and its specific process includes:

[0026] Obtain the dependencies between the convolutional layers in the feature extractor of the pre-trained VGG16, and establish a dependency graph. Each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel;

[0027] Group the parameters in the dependency graph, and the parameters in each group have the same dependency relationship;

[0028] For each group, calculate the importance of each parameter in the group;

[0029] Utilize the importance of each parameter to calculate the joint importance of each group;

[0030] Utilize the joint importance of each group to perform pruning on the groups.

[0031] Furthermore, the method further includes training an age and gender recognition model, wherein the training of the age and gender recognition model includes:

[0032] Obtain a dataset, which contains head-side X-ray images, and each image has a corresponding age label and gender label;

[0033] Preprocess each image in the dataset;

[0034] Utilize the preprocessed dataset to train the age and gender recognition model.

[0035] Furthermore, the preprocessing includes splitting the image into N sub-images and merging the N sub-images along the channel direction into a tensor, and the tensor serves as the input to the age and gender recognition model.

[0036] Furthermore, the specific grouping of the parameters in the dependency graph includes:

[0037] Assume there are L convolutional layers, then

[0038] In each convolutional layer l from the 1st to the L-1th convolutional layer, for each convolutional kernel therein, form a group with the parameters of the corresponding channels of all convolutional kernels in the next convolutional layer, which is expressed by the formula:

[0039] G l,i ={w l,i}∪{w l+1,j,i |j=1,...,K l+1},

[0040] where, G l,i represents the i-th group of the l-th layer, w l,iDenote the \(i\)-th convolutional kernel parameter of the \(l\)-th layer as \(w\). l+1,j,i Denote the \(i\)-th channel parameter of the \(j\)-th convolutional kernel of the \((l + 1)\)-th layer as \(K\). l+1 Denote the number of convolutional kernels of the \((l + 1)\)-th layer;

[0041] For the last convolutional layer, i.e., the \(L\)-th convolutional layer, use the convolutional kernel parameters in this convolutional layer as a group, which is expressed by the formula:

[0042] G L,i =\{w L,i \}, i = 1, 2, …, K L ,

[0043] where \(G L,i denotes the \(i\)-th group of the \(L\)-th layer, \(w L,i denotes the \(i\)-th convolutional kernel parameter of the \(L\)-th layer, and \(K L denotes the number of convolutional kernels of the \(L\)-th layer.

[0044] Furthermore, calculating the importance of each parameter in the group includes using the gradient of backpropagation to evaluate the importance of each parameter. For any parameter \(P t , its importance \(I t is expressed by the formula:

[0045]

[0046] where Loss represents the total loss function of age regression loss and gender classification loss.

[0047] Furthermore, calculating the joint importance of each group is expressed by the formula:

[0048]

[0049] where \(I total denotes the joint importance of each group, and \(n\) denotes the number of parameters in the group.

[0050] Furthermore, the age prediction unit uses a transformer model for age regression and adopts a mean square error loss function.

[0051] Furthermore, the gender prediction unit uses a transformer model for gender classification and adopts a cross-entropy loss function.

[0052] Furthermore, pruning the feature extractor of the pre-trained VGG16 involves multiple pruning processes. After each pruning, fine-tuning is performed. If the recognition performance before this pruning can be restored after fine-tuning, the parameters after this pruning are saved and the next pruning is carried out. If the recognition performance cannot be restored after fine-tuning after this pruning, the pruning is stopped and the parameters saved last time are used as the final pruned model.

[0053] The beneficial effects of the present invention are as follows:

[0054] The present invention designs a new method for lightweighting the feature extractor to lightweight the latent variable model. By constructing a dependency graph between layers, the dependency relationships between layers are recorded. Furthermore, by grouping the parameters in the dependency graph, the parameters in each group have the same dependency relationship. When pruning the feature extractor, the entire group will be removed. Then, combined with the Transformer module for multi-task deep learning, it not only greatly reduces the computational complexity and improves the processing speed, but also verifies its excellent performance in many practical applications.

[0055] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings, where:

[0057] Figure 1 is a flowchart of a method for lightweighting a feature extractor according to an embodiment of the present application;

[0058] Figure 2 is a schematic flowchart of a multi-task estimation method for age and gender according to an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of image preprocessing;

[0060] Figure 4 is a schematic diagram of a multi-task estimation method for age and gender of a cephalic X-ray based on a latent variable model;

[0061] Figure 5 is a schematic diagram of an age and gender recognition model;

[0062] Figure 6 is a block diagram of a multi-task estimation system for age and gender of a cephalic X-ray based on a latent variable model. Detailed implementation manners

[0063] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than limiting the protection scope of the present invention.

[0064] The present invention provides a multi-task estimation method for the age and gender of a cephalic X-ray image based on a latent variable model. As Figure 2 shown, the method includes:

[0065] Obtain a cephalic X-ray image of a target to be detected;

[0066] Preprocess the cephalic X-ray image of the target to be detected;

[0067] Input the preprocessed image into an age and gender recognition model to generate an estimation of the age and gender of the target to be detected.

[0068] Among them, after obtaining the original cephalic X-ray image of the target to be detected, the image can also be preprocessed. For example, the obtained original cephalic X-ray image of the target to be detected can be cropped to obtain more image information related to age and gender, and the image information irrelevant to age and gender prediction can be removed, so as to effectively reduce the calculation amount and speed up the prediction speed. For example, the center of the original image I can be determined first, and then, the image information I' of a preset size area (for example, a rectangular area) centered on the image center can be extracted.

[0069] Figure 3 is a schematic diagram of an exemplary image preprocessing. As Figure 3 shown, assuming that the size of the original cephalic X-ray image is 2896×2304, a maximum rectangular frame with an aspect ratio of 1.2:1 can be extracted from the center and then scaled to 1200×1000 according to the ratio.

[0070] In some embodiments, the preprocessing may further include splitting the image (for example, the obtained image I') into N (N can take any appropriate value) sub-images, and merging the N sub-images along the channel direction into a tensor, and the tensor is used as the input of the age and gender recognition model.

[0071] As an example, Figure 2 the image I' in Figure 4 is evenly split into 16 sub-images (as shown in the image content in

[0072] In some embodiments, the age and gender recognition model includes a feature extraction unit, an age prediction unit, and a gender prediction unit. Combining as Figure 4 andFigure 5 As shown, the feature extraction unit is used to extract features from the input image to obtain the feature representation of the image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction.

[0073] In some embodiments, the feature extraction unit can be obtained by pruning the feature extractor of the pre-trained VGG16. Figure 1 It is a schematic flowchart of a method for lightweighting the feature extractor. As Figure 1 shown, the method may include:

[0074] Obtain the dependency relationships between the convolutional layers in the feature extractor of the pre-trained VGG16, and establish a dependency graph. Among them, each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel;

[0075] Group the parameters in the dependency graph, and the parameters in each group have the same dependency relationship;

[0076] For each group, calculate the importance of each parameter in the group;

[0077] Utilize the importance of each parameter to calculate the joint importance of each group;

[0078] Utilize the joint importance of each group to prune the groups, that is, the groups with low joint importance are pruned.

[0079] The grouping of the parameters in the dependency graph may specifically include:

[0080] Assume that there are a total of L convolutional layers. Then, in each convolutional layer l among the first to L - 1 convolutional layers, for each convolutional kernel therein, form a group with the parameters of the corresponding channels of all convolutional kernels in the next convolutional layer. It is expressed by the formula:

[0081] G l,i ={w l,i}∪{w l+1,j,i |j=1,...,K l+1},

[0082] where, G l,i represents the i-th group of the l-th layer, w l,i represents the i-th convolutional kernel parameter of the l-th layer, w l+1,j,i represents the i-th channel parameter of the j-th convolutional kernel of the l + 1-th layer, K l+1 represents the number of convolutional kernels of the l + 1-th layer;

[0083] For the last convolutional layer, i.e., the L-th convolutional layer, the convolutional kernel parameters in this convolutional layer are used as a group, which is expressed by the formula:

[0084] G L,i ={w L,i}, i = 1, 2,..., K L ,

[0085] where G L,i represents the i-th group of the L-th layer, and w L,i represents the i-th convolutional kernel parameter of the L-th layer, and K L represents the number of convolutional kernels in the L-th layer.

[0086] In some embodiments, calculating the importance of each parameter in the group includes using the gradient of backpropagation to evaluate the importance of each parameter. For any parameter P t (i.e., any w l,i above, here for convenience, the parameter is re-represented as P t ), its importance I t is expressed by the formula:

[0087]

[0088] where Loss represents the total loss function of the age regression loss L age and the gender classification loss L sex , that is

[0089] Loss = L age + L sex .

[0090] In some embodiments, calculating the joint importance of each group is expressed by the formula:

[0091]

[0092] where I total represents the joint importance of each group, and n represents the number of parameters in the group.

[0093] After determining the combined importance of each group, the combined importance of each group can be sorted (e.g., in ascending order), and then a preset number of groups with lower combined importance (or groups with combined importance less than a preset value) can be directly removed from the feature extractor and fine-tuned. If the recognition performance before this pruning can be restored after fine-tuning, the parameters after this pruning are saved and the next pruning is performed. If the recognition performance before this pruning cannot be restored after fine-tuning this time, the pruning is stopped and the parameters saved last time are used as the final pruned model. Therefore, by repeating pruning multiple times, the lightweight of the feature extractor can be achieved, thereby reducing the amount of calculation and accelerating the recognition speed of the model.

[0094] After obtaining the feature extractor with a streamlined structure, all its internal parameters are reset, and the streamlined feature extractor is used to extract the features of the image. Then, in some embodiments, the feature expression is processed to a size of 64×63 and then input into two independent prediction units, namely the age prediction unit and the gender prediction unit. Among them, the age prediction unit uses a transformer model for age regression, and the gender prediction unit also uses a transformer model for gender classification. In some embodiments, the parameters of the two transformer modules are set as follows: the feature dimension is 63, the number of heads is 8, and the number of encoder layers is 2. The output feature expression is used for age and gender prediction through two fully connected layers.

[0095] In some embodiments, the mean squared error can be used as the loss function for the age prediction task, and the cross-entropy loss function can be used for the gender prediction task.

[0096] The following briefly describes the training process of the age and gender recognition model. The training of the age and gender recognition model can include:

[0097] S1: Obtain a data set, which contains head-side X-ray images, and each image has a corresponding age label y age and gender label y sex , where y age ∈{0,1}, 0 represents female, and 1 represents male.

[0098] S2: Preprocess each image in the data set. The preprocessing method is the same as that for the head-side X-ray film of the target to be detected, which will not be elaborated here.

[0099] S3: Use the preprocessed data set to train the age and gender recognition model.

[0100] When training the age and gender recognition model, the batch size is 16, the learning rate is 0.001, and the Adam optimizer is used. The goal of the model is to minimize the loss function Loss.

[0101] The present invention also provides a multi-task estimation system for the age and gender of a cephalic X-ray image based on a latent variable model. As Figure 6 shown, the system may include:

[0102] An image acquisition module, configured to acquire a cephalic X-ray image of a target to be detected and a data set, where the data set contains cephalic X-ray images, and each image has a corresponding age label and gender label;

[0103] A preprocessing module, configured to preprocess the acquired images;

[0104] A model training module, configured to train an age and gender recognition model using the data set;

[0105] A prediction module, configured to predict the gender and age of the target to be detected in the cephalic X-ray image of the target to be detected by using the age and gender recognition model.

[0106] Wherein, the age and gender recognition model includes a feature extraction unit, an age prediction unit, and a gender prediction unit.

[0107] The feature extraction unit is configured to extract features from the input image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction.

[0108] The feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16. The specific process includes:

[0109] Obtain the dependency relationship between each convolutional layer in the feature extractor of the pre-trained VGG16, and establish a dependency graph. Each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel;

[0110] Group the parameters in the dependency graph, and the parameters in each group have the same dependency relationship;

[0111] For each group, calculate the importance of each parameter in the group;

[0112] Use the importance of each parameter to calculate the joint importance of each group;

[0113] Use the joint importance of each group to prune the group.

[0114] The method and system provided by the present invention can effectively improve the accuracy of age and gender assessment, and at the same time improve the accuracy and efficiency of cephalometric X-ray film analysis. Moreover, the feature extractor lightweight method proposed by the present invention lightweightens the latent variable model and then cooperates with the transformer module for multi-task deep learning, which not only greatly reduces the computational complexity and improves the processing speed, but also verifies its superior performance in many practical applications. Therefore, this system has the potential to provide strong technical support for solving the problems currently faced by our country, thus promoting the progress of related technologies in the medical field of our country.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A multi-task estimation method for age and gender of cephalic X-ray images based on a latent variable model, characterized in that, Comprising: Obtain the cephalic X-ray image of the target to be detected; Preprocess the cephalic X-ray image of the target to be detected; Input the preprocessed image into the age and gender recognition model to generate the age and gender estimation of the target to be detected, wherein, the age and gender recognition model includes a feature extraction unit, an age prediction unit and a gender prediction unit, the feature extraction unit is used to extract features from the input image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction, the feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16, and its specific process includes: Obtain the dependency relationship between each convolutional layer in the feature extractor of the pre-trained VGG16, establish a dependency graph, each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel; Group the parameters in the dependency graph, and the parameters in each group have the same dependency relationship; For each group, calculate the importance of each parameter in the group; Use the importance of each parameter to calculate the joint importance of each group; Use the joint importance of each group to prune the group; The specific grouping of the parameters in the dependency graph includes: Assume there are L convolutional layers, then In each convolutional layer l in the 1st to L-1th convolutional layers, for each convolutional kernel therein, form a group with the parameters of the corresponding channels of all convolutional kernels in the next convolutional layer, which is expressed by the formula: G l,i = {w l,i} ∪ {w l+1,j,i | j = 1,..., K l+1}, Among them, G l,i represents the i-th group of the l-th layer, w l,i represents the i-th convolutional kernel parameter of the l-th layer, w l+1,j,i represents the i-th channel parameter of the j-th convolutional kernel of the (l + 1)-th layer, K l+1 represents the number of convolutional kernels of the (l + 1)-th layer; For the last convolutional layer, that is, the Lth convolutional layer, use the convolutional kernel parameters in this convolutional layer as a group, which is expressed by the formula: G L,i = {w L,i}, i = 1, 2, ..., K L , Among them, G L,i represents the i-th group of the L-th layer, w L,i represents the i-th convolutional kernel parameter of the L-th layer, K L represents the number of convolutional kernels in the L-th layer; Calculating the importance of each parameter in a group includes using the gradient of backpropagation to evaluate the importance of each parameter. For any parameter P t , its importance I t is expressed by the formula: wherein, Loss represents the total loss function of the age regression loss and the gender classification loss; The calculation of the joint importance of each group is expressed by the formula: Among them, I total represents the combined importance of each group, and n represents the number of parameters in the group.

2. The multi-task estimation method for head-side X-ray age and gender based on a latent variable model according to claim 1, wherein The method further includes training the age and gender recognition model, wherein the training of the age and gender recognition model includes: Obtain a data set, the data set contains cephalic X-ray images, and each image has a corresponding age label and gender label; Preprocess each image in the data set; Use the preprocessed data set to train the age and gender recognition model.

3. The multi-task estimation method for the age and gender of the cephalic X-ray film based on the latent variable model according to claim 1 or 2, characterized in that, The preprocessing includes splitting the image into N sub-images, and merging the N sub-images along the channel direction into a tensor, and the tensor is used as the input of the age and gender recognition model.

4. The multi-task estimation method for the age and gender of the cephalic X-ray film based on the latent variable model according to claim 1, wherein The age prediction unit uses the transformer model for age regression and adopts the mean square error loss function.

5. The multi-task estimation method for the age and gender of cephalic X-ray films based on the latent variable model according to claim 1, characterized in that The gender prediction unit uses the transformer model for gender classification and adopts the cross entropy loss function.

6. The multi-task estimation method for the age and gender of the cephalic X-ray based on the latent variable model according to claim 1, wherein Pruning the feature extractor of the pre-trained VGG16 includes multiple pruning processes. After each pruning, fine-tuning is performed. If the recognition performance before this pruning is restored after fine-tuning, save the parameters after this pruning and perform the next pruning. If the fine-tuning after this pruning cannot restore the recognition performance before this pruning, stop pruning and use the parameters saved last time as the final pruned model.

7. A multi-task estimation system for the age and gender of head-side X-ray images based on a latent variable model according to claim 1, characterized in that, Comprising: An image acquisition module for acquiring a cephalic X-ray film and a data set of the target to be detected, where the data set contains cephalic X-ray images, and each image has a corresponding age label and gender label; A preprocessing module for preprocessing the acquired images; A model training module for training an age and gender recognition model using the data set; A prediction module for predicting the gender and age of the target to be detected in the cephalic X-ray film of the target to be detected using the age and gender recognition model, wherein the age and gender recognition model includes a feature extraction unit, an age prediction unit, and a gender prediction unit, the feature extraction unit is configured to extract features from the input image, and output the extracted features to the age prediction unit and the gender prediction unit respectively for age and gender prediction, the feature extraction unit is obtained by pruning the feature extractor of the pre-trained VGG16, and the specific process includes: Obtaining the dependency relationships between the convolutional layers in the feature extractor of the pre-trained VGG16, and establishing a dependency graph. Each convolutional layer corresponds to a dependency graph, and the dependency graph records the correspondence between each convolutional kernel in each convolutional layer and all model parameters directly connected to the convolutional kernel; Grouping the parameters in the dependency graph, and the parameters in each group have the same dependency relationship; For each group, calculating the importance of each parameter in the group; Using the importance of each parameter, calculating the joint importance of each group; Using the joint importance of each group to prune the groups.

Images