Deep learning-based classification method and system for deducing age and gender in combination with cervical vertebra

Abstract

The invention discloses a deep learning-based combined cervical vertebra age and gender inference classification method and system, and the method comprises the steps: taking an X-ray head positioning side image as the input of a segmentation network based on a U-Net architecture, and obtaining a segmentation mask of a single cervical vertebra in the X-ray head positioning side image; designing various input working conditions according to the cervical vertebra segmentation mask; setting a single cervical vertebra and a plurality of cervical vertebra combination input combinations according to each input working condition; taking the upper left corner of the X-ray skull positioning side image as a coordinate origin setting range, and taking a cervical vertebra region guided by ARDA; using the trained Officentnet-B0 model to perform age deduction and classification on a single cervical vertebra and multiple cervical vertebra combination input combinations set for each input working condition and a cervical vertebra region guided by ARDA to obtain the probability and age of the gender corresponding to the X-ray head positioning side image; the age estimation accuracy and reliability are further improved through comparison of age and gender inference precision of different cervical vertebra combination modes and a single cervical vertebra.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and specifically relates to a method and system for joint cervical vertebrae inference age and gender classification based on deep learning. Background Art

[0002] Forensic medicine is usually used in criminal cases where the remains of a body are in the decomposition stage. Forensic anthropologists can help identify the deceased whose remains are burned, rotten, unrecognizable or otherwise dismembered. Therefore, the application of age inference and gender classification technology in forensic anthropology is crucial. The skeletal structure undergoes a series of morphological changes during development and aging, and these changes can be used as a basis for inferring age and gender classification. The bones commonly used to determine age are the wrists, while the bones used to determine gender are the pelvic bones. In the age inference of minors, the hands, wrists, medial clavicle, first rib, knees, and cervical spine are often used as observation objects. Wrist X-rays have become the most commonly used method to determine the maturity of children due to their intuitiveness and accuracy. However, the skeletal development of adults has matured, and traditional observation methods are difficult to accurately reflect the aging changes of bones and surrounding tissues. This also limits the application of bone age inference in adult age assessment to a certain extent.

[0003] The cervical spine is composed of seven closely connected vertebrae. By taking lateral shots of the cervical vertebrae, the aging changes during growth, development and aging can be clearly shown. Once the epiphysis is combined, the development of the vertebrae stops. According to the morphological changes of the cervical spine, its development process is usually divided into six different stages. By observing the stage of the cervical spine, the cervical vertebra maturity (CVM) can be evaluated. This evaluation method has been proven to be as reliable and effective as the wrist region in assessing bone age. Since bone maturation is a continuous process, the assessment of CVM is essentially a regression problem. The developmental grading of the cervical spine is to discretize the continuous CVM into 6 categories according to the level, converting the regression problem into a classification problem. Even for experienced doctors, discretization is difficult to obtain satisfactory performance, especially for marginal subjects, the six stages of CVM may be difficult to distinguish. The X-ray head positioning lateral image with high image noise level has poor image clarity, making staging difficult. In addition, the image distortion caused by the patient's angle and posture may also make the evaluation of the staging of the X-ray head positioning lateral image difficult. Many studies have reported poor repeatability and reliability of CVM assessment. This conclusion is mainly based on the identification criteria of the concave lower edge of the vertebral body and the subjective determination of the vertebral shape (trapezoidal, horizontal rectangle, square or vertical rectangle), which affects the reliability of cervical spine staging. It is believed that the CVM atlas cannot evaluate growth in an objective and detailed manner because the results may vary from operator to operator. Moreover, the atlas cannot be used to calculate age like the TW2 method. Due to the above problems, Perinetti et al. believe that clinicians need to undergo special training before they can use the CVM method to make a more satisfactory assessment.

[0004] Since the size and shape of the cervical vertebrae change with age, and the vertebrae are easy to measure by imaging technology, the age inference based on the cervical spine in the past mainly relies on the morphological changes of the cervical spine. Some scholars have used the ratio of vertebral height and (or) vertebral length, the concavity and angle of different vertebrae as variables to infer the age of adolescents. Mito et al. used the width / height ratio of C3 and C4 vertebrae to study the cervical spine images of 176 girls aged 7-15 years, and determined the regression formula for cervical skeletal age, with a mean absolute error (MAE) of 1.17; the correlation R between cervical skeletal age and wrist skeletal age was 0.869; the results showed that the cervical skeletal age inferred was as reliable as the skeletal age inferred by the Tanner-Whitehouse 2 (TW2) method, and could objectively evaluate the cervical bone maturity and calculate the skeletal age. Caldas et al. measured the width and anterior height of the cervical C3 and C4 vertebrae and used the aspect ratio of the two vertebrae (AH3 / AP3 and AH4 / AP4) as variables to derive a multiple regression equation for age inference. Later, Caldas et al. used the above formula to assist in skeletal age inference using a computer system. The method is simple to operate, less time-consuming, and highly objective. Beit et al. measured the morphological changes (concavity, anterior height, and angle) of the C2 to C4 vertebrae and found, by comparing with the age of the wrist bones, that the concavity of C2, C3, and C4, the anterior height of C3 and C4, and the C3 angle were significantly correlated with bone age, and a multiple regression equation for bone age inference was derived using these three indicators as variables. Cameriere et al. used the ratio of the anterior (a) and posterior (b) heights of the C4 vertebrae to infer age. The development of the cervical spine is completed at around 25 years old, and the spinous processes and upper and lower epiphyseal rings of C3-C7 fuse with the vertebral bodies. After that, the cervical spine begins to undergo degenerative changes, such as age-related changes in the intervertebral disc, cervical curvature, and bone density. However, since traditional measurement methods are difficult to clearly and accurately describe the characteristics of degenerative changes in the cervical spine, there are currently few studies that infer the age of adults through aging changes in the cervical spine. In the age-related dynamic attention to the development and aging analysis of the head positioning lateral film, we found that the cervical spine area is strongly correlated with age, especially in the age group of 25-40 years old, which also suggests that the age-related changes in the cervical spine can be used to infer the age of adults.

[0005] Due to methodological limitations, previous traditional studies only measured the lengths of various parts of the cervical vertebrae and used the ratio as a variable. This method cannot fully reflect the aging changes in the shape of the cervical spine. The use of artificial intelligence deep learning methods will help reduce the subjective influence of analyzing X-rays and the time required for diagnosis, achieve high efficiency and accuracy, and thus help clinicians make accurate assessments. In 2012, Baptista et al. used a Bayesian classifier to classify CVM, and the recognition rate of the correct CVM stage was about 70%. In 2015, Dzemidzic et al. used a decision tree to classify CVM after analyzing the rectangular features and concavity of the contours of C2, C3, and C4. Moraes et al. used a simple multilayer perceptron to learn the concavity of the bottom of the C3 vertebra to output bone age. In 2019, et al. defined 19 reference points on C2, C3, and C4 and performed 20 different linear measurements. Seven artificial intelligence algorithms commonly used in the classification field were selected for comparison. Amasya et al. also considered the landmark points of C5 (a total of 26 reference points needed to be defined and manually marked), and classified the features by extracting multiple ratios between the length and width of the vertebral body. Five commonly used artificial intelligence algorithms were selected for comparison. The results of both studies suggest that artificial neural networks are the preferred method for measuring CVM. The above studies used classic machine learning-based methods, which can more accurately identify CVM stages and achieve high accuracy. However, these methods still require professional knowledge to manually mark features on medical images, which is time-consuming and labor-intensive, and prone to errors.

[0006] Deep learning models can extract features of given data by learning. Therefore, compared with classical machine learning methods, it does not require experts to perform manual labeling. In 2019, Makaremi et al. first proposed a CNN-based deep learning method to determine the 6 stages of CVM, using classification entropy as the optimization criterion. Studies have shown that when data is evenly distributed across all CVM stages, the results perform better, reaching 90% accuracy. However, this method requires manual segmentation of the region of interest (ROI). In 2021, Seo et al. compared the results of CVM staging accuracy of 6 commonly used CNN models and found that the accuracy of all deep learning models exceeded 90%, among which Inception-ResNet-v2 performed best. Grad-CAM was used to visualize each deep learning model and determine the feature heat map of each deep learning model, but the system is semi-automatic because manual cropping of ROI is required as preprocessing. In 2022, Li Haizhen et al. selected four CNN models, VGG16, GoogLeNet, DenseNet161 and ResNet152, for comparison to determine the most suitable CNN model for CVM staging. The results showed that the average area under the curve (AUC) of the ResNet152 model was 0.933, and the total accuracy was 67.06%, making it the best model for CVM classification. However, compared with end-to-end fully automatic inference, the above studies still require manual participation, which is time-consuming and laborious, and all of them are CVM staging studies. No research has yet involved deep learning cervical age inference.

[0007] In addition, accurate gender assessment is crucial for identifying unknown bodies. Males and females usually show sexual dimorphism in skeletal structure, so skeletal contours usually use sex-specific standards. Among them, using skeletal structure in X-ray images for gender classification is a common method in forensic anthropology and medical imaging. The pelvis is a reliable bone for assessing gender. In order to adapt to the needs of childbirth, the female pelvis has a transverse diameter of the inlet that is larger than the anterior-posterior diameter and a wider pubic arch. These features facilitate gender classification. Metric analysis of skull gender assessment can also provide objective results. For example, there is sexual dimorphism in the size and shape of the mandible (mandible), mastoid process, brow ridge (supraorbital ridge) and forehead (frontal bone); in addition, some characteristics of the hand and wrist bones, such as the size and shape of the carpal bones and metacarpal bones, can also be used for gender classification. Studies on the sexual dimorphism of the cervical spine have shown that there are significant gender differences in vertebral size and morphology. The study by Stemper et al. showed that male vertebrae are larger in multiple measurements, including vertebral body width, intervertebral disc facet joint depth, and segmental support area; Madhavan et al. and Stan et al. found that the second cervical vertebra (C2) has been identified as particularly useful for sex estimation, with an accuracy of approximately 77.5%; Amores et al. showed that other cervical vertebrae, C3-C6, also exhibited significant sexual dimorphism. Therefore, sex classification through cervical vertebrae has potential in forensics. Although the cervical spine exhibits sexual dimorphism, its morphological differences are difficult to accurately characterize and objectively quantify. In recent years, studies have demonstrated the high accuracy and efficiency of sex classification using deep convolutional neural networks (CNNs) for dental X-ray images. Various CNN architectures, including ResNet50, DenseNet121, and custom models, have been used for this task, which have significant advantages over manual methods, including faster processing time and reduced reliance on expert knowledge (Ilic et al., 2019; et al., 2019). Moreover, the traditional manual measurement and comparison method is not only time-consuming and tedious, but also inevitably introduces the interference of subjective factors, thereby weakening the efficiency and accuracy of skeletal gender classification. Therefore, it is particularly important to develop an automated and intelligent system that can simultaneously achieve accurate inference of cervical age and accurate gender classification.

[0008] In a study based on deep learning technology, aging feature analysis and extraction of X-ray head positioning lateral images were conducted. It was found that the best results were obtained by using the cervical spine area for age inference in the 26-40 age group, which also shows that the cervical spine area has unique importance in the study of aging of adult subjects. In addition, since the reference atlas currently widely used for age inference and gender estimation is mainly based on the body measurement data of people in the UK and North America, this may lead to deviations when applied to other regions or different populations. Therefore, in order to achieve more accurate age inference, it is necessary to develop and verify standard atlases based on a wider range of regions and different group diversity to ensure that forensic anthropology technology can serve the global judicial system without bias. Summary of the invention

[0009] In order to address the limitations of traditional methods in characterizing age-related changes in bones and surrounding tissues, especially in the inference of adult bone age, the present invention proposes a joint cervical vertebrae inference age and gender classification method based on deep learning. This method is based on the age estimation method of X-ray head positioning lateral images, and also introduces the function of gender classification. It considers the characteristic information of gender factors, analyzes and quantifies the development and aging characteristics of the cervical spine area of ​​the northern Chinese population, achieves a more accurate estimation of individual age, and is suitable for adult age inference, effectively expanding the applicable age range.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is: a method for combining cervical vertebrae inference age and gender classification based on deep learning, comprising the following steps:

[0011] The X-ray head positioning lateral image is used as the input of the segmentation network based on the U-Net architecture to obtain an image with segmentation results, and obtain the segmentation mask of a single cervical vertebra in the X-ray head positioning lateral image;

[0012] Performing median filtering, corrosion operation, expansion operation, binarization and re-expansion processing on the segmentation mask to obtain an accurate cervical vertebra segmentation mask;

[0013] Design multiple input working conditions based on the obtained accurate cervical vertebra segmentation mask;

[0014] Set up single cervical vertebra and multiple cervical vertebra combination input combinations for each input condition;

[0015] The upper left corner of the X-ray head positioning lateral image was used as the coordinate origin to set the range and take the cervical spine area guided by ARDA;

[0016] The trained Efficentnet-B0 model is used to infer and classify the age of a single cervical spine and multiple cervical spine combination input combinations for each input condition, as well as the cervical spine area guided by ARDA, and the probability and age of the corresponding gender of the X-ray head positioning lateral image are obtained.

[0017] Furthermore, the training of the segmentation network of the U-Net architecture includes the following steps:

[0018] Divide the training dataset into several batches, each batch contains multiple images;

[0019] Input batch images into the U-Net network and calculate the prediction results for each pixel

[0020] Calculate the error between the predicted result and the true label according to the comprehensive loss function L;

[0021] The gradient of loss to model parameters is calculated by back-propagation algorithm:

[0022] Use the Adam optimizer to update the model parameters according to the gradient until the preset training rounds are reached.

[0023] Furthermore, when training the segmentation network of the U-Net architecture, a multi-classification cross entropy loss function is used:

[0024]

[0025] Among them, N is the total number of pixels, C is the number of categories, and y i,c is the one-hot encoding of the true label, For the predicted probability of category c, define the Dice loss:

[0026]

[0027] Get the weighted sum of cross entropy loss and Dice loss:

[0028] L=αL CE +βL Dice

[0029] Among them, α and β are weight parameters.

[0030] Furthermore, obtaining the training data set includes the following steps:

[0031] Obtain an image dataset with annotation information of key areas of the cervical spine, including original images and corresponding labeled images. In the labeled images, the target part of interest in the original image is annotated with a set pixel value, and the other parts are annotated as background;

[0032] Preprocess the images in the image dataset: for each X-ray head positioning lateral image D n Perform histogram equalization to obtain a set of X-ray head positioning lateral images after data enhancement Where, N = 20174, D nrepresents the nth X-ray head positioning lateral image with a rectangular shape and containing the head, teeth and spine with age-related changes. Indicates D n The data enhancement results are

[0033] For each enhanced X-ray head positioning lateral image The short side of is padded with zeros, and the size of the X-ray head positioning lateral image with a square shape after zero padding is scaled to obtain a set of X-ray head positioning lateral images after size transformation

[0034] Each size-transformed X-ray head positioning lateral image Each image in is normalized to obtain a normalized X-ray head positioning lateral image set

[0035] Extract the normalized X-ray head positioning lateral image set D 3 Some images are used as training sets to train the segmentation network based on the U-Net structure.

[0036] Furthermore, obtaining the label image includes:

[0037] For X-ray head positioning lateral images from different age groups, key areas of the cervical spine were manually annotated for image segmentation to generate labeled images: instance segmentation was performed on the C1-2, C3, and C4 parts of the X-ray head positioning lateral images. Since C1 and C2 overlapped and could not be separated, they were treated as one cervical vertebra C1-2.

[0038] Furthermore, the input conditions include: the cervical spine area defined by the minimum upper and lower boundaries of the segmentation template, the cervical spine with the surrounding tissue removed, the cervical spine segmentation contour template, and the cervical spine area divided by fixed coordinates; a single cervical spine under four input modes, including C1-2, C3, C4 and cervical spine combinations (C1-2+C3, C1-2+C4, C3+C4, C1-C4), a total of 7 cervical spine areas are used for age inference and gender classification.

[0039] Furthermore, the upper left corner of the X-ray head positioning lateral image is used as the coordinate origin to set the range of the cervical spine area guided by ARDA, including:

[0040] The upper left corner of the X-ray head positioning lateral image is taken as the coordinate origin, the right and bottom are the positive directions of the x-axis and y-axis respectively, the width and height are set as W and H respectively, and the coordinate system is established with the width of a single pixel as the unit length. On the coordinate system, (2 / 3W+100; 1 / 2H-100) is the coordinate of the upper left corner, and (W,H) is the coordinate of the lower right corner, and the cervical spine area guided by ARDA is cut.

[0041] Based on the concept of the method, the present invention also provides a joint cervical vertebra inference age and gender classification system based on deep learning, including a segmentation mask acquisition module, an input design module and an inference module;

[0042] The segmentation mask acquisition module is used to use the X-ray head positioning lateral image as the input of the segmentation network based on the U-Net architecture to obtain an image with segmentation results, and obtain a segmentation mask of a single cervical vertebra in the X-ray head positioning lateral image; perform median filtering, corrosion operation, expansion operation, binarization and re-expansion processing on the segmentation mask to obtain an accurate cervical vertebra segmentation mask;

[0043] The input design module is used to design multiple input working conditions based on the obtained accurate cervical segmentation mask; set a single cervical vertebra and multiple cervical vertebra combination input combinations for each input working condition; set the range to take the cervical vertebra area guided by ARDA with the upper left corner of the X-ray head positioning lateral image as the coordinate origin;

[0044] The inference module uses the trained Efficentnet-B0 model to perform age inference and classification on the cervical spine area guided by ARDA for a single cervical spine and multiple cervical spine combination input combinations for each input condition, and obtains the probability and age of the corresponding gender for the X-ray head positioning lateral image.

[0045] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the deep learning-based joint cervical vertebra inference age and gender classification method described in the present invention.

[0046] At the same time, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the method for combining cervical vertebrae inference age and gender classification based on deep learning described in the present invention can be implemented.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] Compared with traditional methods, deep learning can comprehensively study and reflect the correlation between the aging changes in the shape of the cervical spine and age, including both the shape changes and the density changes (image contrast) of the cervical spine. The present invention designs different input modes to verify the impact of manual annotation on age inference under different circumstances. The use of cervical spine masks can reflect the changes in cervical spine morphology, and the use of cervical spine contours as input modes can understand whether the minimum input information can have an impact on the inference results. The cervical spine images segmented using masks can both represent shape changes and reflect density changes, and the segmented cervical spine area images can reflect both changes in cervical spine hard tissue (shape and density) and changes in some soft tissues around the cervical spine (intervertebral discs, muscles and ligaments, etc.). By restricting the cervical spine area information entering the age inference and gender classification network to varying degrees, the contribution of bone tissue and soft tissue in different cervical spine areas to age inference and gender classification is verified, and the accuracy of age and gender inference is compared with that of a single cervical spine through different cervical spine combination modes, further improving the accuracy and reliability of age estimation.

[0049] Compared with the semi-automatic method of manually extracting features and then using a machine learning model to learn a fitting model, the method used in the present invention is fully automatic and does not require manual extraction of specific features. By extracting specific aging features, the automated model is not limited to the growth stage before the age of 18, so it can be applied to a wider age range. The end-to-end fully automated model based on deep learning not only has advantages in manpower and time in feature extraction, but also can achieve better performance in age inference tasks. In addition, considering the existence of sexual dimorphism in the cervical spine, in order to improve the accuracy and reliability of age estimation, age estimation and gender classification are performed in coordination to distinguish the differences in age-related changes in the development and aging of the male and female cervical spine. The function of gender classification is further introduced to provide gender classification results, and age estimation and gender classification are completed at the same time, and the accuracy of age estimation and gender classification tasks in the cervical spine area can be improved at the same time;

[0050] The present invention analyzes and quantifies the age-related change characteristics of the development and aging of the cervical spine region in the northern Chinese population. A segmentation mask (SM) was generated through an automatic segmentation method using deep learning. By constructing a cervical spine age inference model with different input modes, the contribution of manual annotation, soft tissue around the cervical spine, and bones to the cervical spine age inference was studied. In addition, since there are significant gender differences in the size and morphology of the cervical spine, the correlation between age and gender cannot be ignored. In order to improve the accuracy and stability of age estimation, a gender classification function was further introduced to provide gender classification results, providing more comprehensive information for medical image analysis and auxiliary diagnosis.

[0051] Previous studies on cervical morphology have mostly focused on the adolescent stage. However, for adults, since the cervical spine begins to undergo degenerative changes after the age of 18, its characteristics are difficult to describe clearly through conventional measurement methods. Research on degenerative changes in the cervical spine is relatively scarce, and no scholar has yet studied the age inference of adults through aging changes in the cervical spine. This paper applies deep learning technology to the inference of adult cervical age for the first time, filling the gap in this field and opening up new ideas for bone age inference research. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is an implementation flow chart of the present invention,

[0053] Figure 2 It is the X-ray head positioning lateral image adopted by the present invention,

[0054] Figure 3 This is the U-net architecture diagram: each blue box corresponds to a multi-channel feature map, the number of channels is indicated at the top of the box, the xy dimensions are provided at the lower left edge of the box, the white boxes represent copied feature maps, and the arrows represent different operations.

[0055] Figure 4 .Single cervical vertebrae (C1-2, C3, C4) and corresponding segmentation masks in the lateral X-ray head positioning image.

[0056] Figure 5 A lateral cephalogram showing the location of the dental, craniofacial, and cervical spine regions.

[0057] Figure 6 Represents different input modes for the design of cervical spine age inference and gender classification study (taking C1-2 as an example). DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Based on deep learning technology, age estimation and gender classification are performed. The Efficentnet-B0 model is used to quantify the aging significance of a large number of continuous age sample X-ray head positioning lateral images, and the correlation between the region and aging in the X-ray head positioning lateral images is expressed in an orderly manner. The average aging significance is calculated by merging all aging saliency maps of the same age, thereby obtaining the dynamic attention (ARDA) related to aging. The present invention will analyze in detail the influence of factors such as manual annotation, soft tissue and bones around the cervical spine on the accuracy of cervical age inference for specific cervical regions, and complete age estimation and gender classification at the same time, thereby improving the accuracy of cervical regional age estimation and gender classification tasks.

[0060] Step 1) Dataset collection: Collect X-ray head positioning lateral images from different age groups: Obtain 20174 X-ray head positioning lateral images collected from subjects aged 4-40 years old, such as Figure 2 , the male-female ratio was in the range of 2:5 to 5:2, with more than 7,000 images for males and more than 7,000 images for females. Inclusion criteria: 1) accurate age records; 2) based on ID card information (first three digits), the population included was from northern China. Exclusion criteria: 1) incorrect imaging position of X-ray head positioning lateral images (such as the orbital-auricular plane not parallel to the ground plane); 2) images with poor image quality (such as unclear bone structure and the presence of metal artifacts); 3) head, face, neck tumors, bone, and tooth integrity-destructive diseases. All subjects were divided into 7 groups at intervals of 5 years. The age and gender distribution of the subjects are shown in Table 1.

[0061] Table 1 Age and gender groups of X-ray head positioning lateral image dataset

[0062]

[0063] Step 2) Manually mark the key areas of the cervical spine for image segmentation to generate labeled images: Perform instance segmentation on the C1-2, C3 and C4 parts of the X-ray head positioning lateral image: Due to anatomical reasons, C1 and C2 overlap and cannot be separated, so they are treated as one cervical vertebra C1-2.

[0064] Step 3) Define the U-Net architecture, such as Figure 1As shown, it includes a downsampling path as an encoder (left side) and an upsampling path as a decoder (right side); the downsampling path follows the typical architecture of a convolutional network. Each downsampling in the encoder consists of two 3×3 convolution modules connected in series, each of which is followed by a rectified linear unit (ReLU) and a 2×2 max pooling operation for downsampling. After each downsampling module, the number of feature channels is doubled. Each upsampling module in the decoder contains a 2×2 deconvolution module convolution and two 3×3 convolution modules connected in series, each of which is followed by a ReLU module and reduces the number of feature channels by half. In addition, the feature map output by each deconvolution is connected to the corresponding cropped feature map in the downsampling path. In the last layer, a 1×1 convolution is used to map each 64-channel feature map to the required channel (i.e., the number of targets). Reference Figure 3 , each blue box corresponds to a multi-channel feature map, the number of channels is indicated at the top of the box, the xy dimensions are provided at the bottom left edge of the box, the white boxes represent the copied feature maps, and the arrows represent the different operations.

[0065] Step 4) Prepare the dataset:

[0066] Step 4a) collect an image dataset with annotated information of key areas of the cervical spine, which should include original images and corresponding labeled images. The labeled image annotates the target part of interest in the original image with a set pixel value, and the other parts are annotated as background.

[0067] Step 4b) Preprocess the image to ensure compatibility of the input with the model: For each X-ray head positioning lateral image D n Perform histogram equalization to obtain a set of X-ray head positioning lateral images after data enhancement Where, N = 20174, D n Indicates the nth Figure 2 The X-ray lateral cephalographic image shown is rectangular in shape and includes the head, teeth and spine, which are areas of age-related changes. Indicates D n The data enhancement result is that the histogram equalization method used in this embodiment is a histogram equalization method with limited contrast;

[0068] Step 4c) For each enhanced X-ray head positioning lateral image The short side of the image is padded with zeros, and the size of the square X-ray head positioning lateral image after zero padding is scaled to 1000×1000 to obtain a set of X-ray head positioning lateral images after size transformation.

[0069] Step 4d) To eliminate the influence of the dimension of the X-ray head positioning lateral image on the network training, each size-transformed X-ray head positioning lateral image Each image in the image is normalized, that is, the value of each pixel in each image is divided by the difference between the maximum and minimum values ​​of all pixels in the image to obtain a normalized X-ray head positioning lateral image set

[0070]

[0071] Step 4e) Extract the normalized X-ray head positioning lateral image set D 3 70% of the images are used as the training set D for the segmentation network based on the U-Net structure. train .

[0072] Step 5) Train the U-Net segmentation network

[0073] Step 5a) Define the loss function. For multi-target segmentation tasks, a multi-classification cross entropy loss function is used:

[0074]

[0075] Among them, N is the total number of pixels, C is the number of categories, and y i,c is the one-hot encoding of the true label, is the predicted probability of category c. And define Dice loss:

[0076]

[0077] Get the weighted sum of cross entropy loss and Dice loss:

[0078] L=αL CE +βL Dice

[0079] Among them, α and β are weight parameters, and are both set to 0.5 in this example.

[0080] Step 5b) Select Adam optimizer to update model parameters, set initial learning rate η = 0.001, β1 = 0.9, β2 = 0.999, and adjust the parameters θ t Update to get the updated θ t+1 :

[0081]

[0082] Among them, m t and v t are the momentum of the gradient and the momentum of the squared gradient, ∈ = 10 -6 A small constant to prevent division by zero errors.

[0083] Step 5c) Model training

[0084] Step 5c1) The training data set D train Divide into several batches, each batch contains B = 32 images.

[0085] Step 5c2) Input the batch of images into the U-Net network and calculate the prediction results for each pixel

[0086]

[0087] Step 5c3) Calculate the error between the predicted result and the true label according to the comprehensive loss function L:

[0088]

[0089] Step 5c4) Back propagation: Calculate the gradient of loss with respect to model parameters through the back propagation algorithm:

[0090]

[0091] Step 5c5) Use the Adam optimizer to update the model parameters according to the gradient:

[0092]

[0093] Step 5c6) executes 5c1) until the preset training round E=1000 is reached.

[0094] Step 6) The trained U-Net obtained in step 5c generates a normalized X-ray head positioning lateral image set D 3 The segmentation mask for each image in .

[0095] Step 7) Visualize or save it as an image with segmentation results to obtain the segmentation mask of a single cervical vertebra (C1-2, C3, C4) in the X-ray head positioning lateral image, such as Figure 4 As shown,

[0096] Step 8) Post-process the segmentation mask to improve the accuracy and visual effect of the segmentation result, remove noise, smooth the boundary, and restore the deformation that may be caused by the processing process to obtain an accurate cervical vertebrae segmentation mask:

[0097] Step 8a) Median filtering is used to eliminate speckle noise: the grayscale value of each pixel is set to the median of the grayscale values ​​of all pixels in a certain neighborhood window of the point, thereby maintaining edge information to a certain extent.

[0098] Step 8b) Erosion operation, to deal with slightly larger planar noise and make the target area smoother: perform a logical AND operation on each pixel in the image and its neighboring pixels to obtain a new pixel value. Through multiple erosion operations, larger noise areas can be gradually eliminated.

[0099] Step 8c) performs a dilation operation to restore the target area that has been reduced by the erosion operation and fill the holes. In contrast to the erosion operation, a logical OR operation is performed on each pixel in the image and its neighboring pixels. The dilation operation can fill the holes that may be generated by the erosion operation, making the target area more complete.

[0100] Step 8d) Binarization further removes noise and emphasizes contours: set a threshold, set pixels with values ​​greater than the threshold to white (or 1), and pixels with values ​​less than the threshold to black (or 0). This can filter out some darker pixels, reduce the impact of noise, and highlight the main contours.

[0101] Step 8e) In order to meet the requirement of making the segmentation result slightly larger than the true value and to further fill the possible holes, the expansion operation is applied again to obtain the processed segmentation masks of each cervical vertebra, as shown in Figure 6 As shown in a.

[0102] Step 9) Input conditions: To ensure maximum coverage of various cervical vertebrae information in forensic practice, based on the processed segmentation masks of each cervical vertebra obtained in step 8), taking C1-2 as an example, four input conditions are designed as follows:

[0103] Step 9a); Use OpenCV to draw a circumscribed rectangle to obtain the cervical spine area defined by the minimum upper and lower boundaries of the segmentation template. Figure 6 b, denoted as I reg .

[0104] Step 9b) In I reg On this basis, the pixels outside the segmentation mask are set to 0 to obtain the cervical vertebra without the surrounding tissue. Figure 6 c, denoted as I seg ;

[0105] Step 9c) Binarize the segmentation mask, set the foreground pixels to 1, and the background pixels to 0, to obtain a binary segmentation mask. Figure 6 d, denoted as I mask ;

[0106] Step 9d) Use OpenCV's findContours function to detect the binary segmentation mask I mask The contour of the cervical vertebrae is obtained by Figure 6 e, denoted as I con ;

[0107] Step 10) Take the upper left corner of the X-ray head positioning lateral image as the coordinate origin, the right and bottom are the positive directions of the x-axis and y-axis respectively, set its width and height respectively as W and H, and take the width of a single pixel as the unit length to establish Figure 5 The coordinate system shown in the figure, in which (2 / 3W+100; 1 / 2H-100) is the coordinate of the upper left corner and (W,H) is the coordinate of the lower right corner, the cervical spine area guided by ARDA is cut out. Figure 6 f, denoted as I ARDA ;

[0108] Step 11) Since different types of cervical vertebrae and different numbers of cervical vertebrae contain different age inference and gender classification information, and incomplete information may be encountered in forensic practice, such as dismemberment cases, skeletonized cases, etc., only a single cervical vertebra is obtained, or the tissue sample of a certain cervical vertebra is missing, considering the reality faced by the above-mentioned forensic identification practice, for each input condition obtained in step 9), a single cervical vertebra (C1-2, C3, C4) and cervical vertebra combination (C1-2+C3, C1-2+C4, C3+C4, C1-C4) are set, a total of 7 input combinations and the I obtained in step 10) ARDA For age inference and gender classification,

[0109] Step 12) The LCR data set is divided into a training set, a validation set and a test set in a ratio of 7:1.5:1.5 for training the age inference and gender classification model, and based on the classification model completed by training, validation and testing, a single cervical vertebra and multiple cervical vertebra combination input combinations are set according to each input condition and the cervical vertebra area guided by the ARDA is used for age inference and gender classification to obtain the probability and age of the gender corresponding to the X-ray head positioning lateral image.

[0110] Step 13) Cervical age inference model measurement: The aging changes of the cervical spine are continuous variables. The cervical age inference based on deep learning is a regression task. The difference between the estimated age and the true age is measured using the L1 loss function. The L1 loss function is defined as:

[0111]

[0112] in and are the age label and prediction result of the nth LCR sample respectively.

[0113] Step 14) Cervical vertebra gender classification model measurement: Cervical vertebra gender classification is a discrete variable. Cervical vertebra gender classification based on deep learning is a classification task. The difference between the estimated age and the true age is measured using the cross entropy loss function. The cross entropy loss function is defined as:

[0114]

[0115] in is the gender label of the nth LCR sample. If the LCR sample is male otherwise y′ n ∈(0,1) is the probability of the corresponding gender classification being female, (1-y′ n ) is the probability of being classified as male.

[0116] Based on the conception of the method, a deep learning-based joint cervical vertebra inference age and gender classification system includes a segmentation mask acquisition module, an input design module, and an inference module;

[0117] The segmentation mask acquisition module is used to use the X-ray head positioning lateral image as the input of the segmentation network based on the U-Net architecture to obtain an image with segmentation results, and obtain a segmentation mask of a single cervical vertebra in the X-ray head positioning lateral image; perform median filtering, corrosion operation, expansion operation, binarization and re-expansion processing on the segmentation mask to obtain an accurate cervical vertebra segmentation mask;

[0118] The input design module is used to design multiple input working conditions based on the obtained accurate cervical segmentation mask; set a single cervical vertebra and multiple cervical vertebra combination input combinations for each input working condition; set the range to take the cervical vertebra area guided by ARDA with the upper left corner of the X-ray head positioning lateral image as the coordinate origin;

[0119] The inference module uses the trained Efficentnet-B0 model to perform age inference and classification on the cervical spine area guided by ARDA for a single cervical spine and multiple cervical spine combination input combinations for each input condition, and obtains the probability and age of the corresponding gender for the X-ray head positioning lateral image.

[0120] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the deep learning-based joint cervical vertebra inference age and gender classification method described in the present invention.

[0121] On the other hand, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the deep learning-based joint cervical vertebra inference age and gender classification method described in the present invention can be implemented.

[0122] The computer device may be a laptop computer, a desktop computer or a workstation.

[0123] The processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).

[0124] The memory of the present invention may be an internal storage unit of a laptop computer, a desktop computer or a workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.

[0125] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drive (SSD) or optical disk, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).

[0126] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, it is possible to make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A method for joint cervical vertebrae inference age and gender classification based on deep learning, characterized in that: The following steps are involved: The X-ray head positioning lateral image is used as the input of the segmentation network based on the U-Net architecture to obtain the segmentation mask of a single cervical vertebra in the X-ray head positioning lateral image; Performing median filtering, corrosion operation, expansion operation, binarization and re-expansion processing on the segmentation mask to obtain an accurate cervical vertebra segmentation mask; Design multiple input working conditions based on the obtained accurate cervical vertebra segmentation mask; Set up single cervical vertebra and multiple cervical vertebra combination input combinations for each input condition; The upper left corner of the X-ray head positioning lateral image was used as the coordinate origin to set the range and take the cervical spine area guided by ARDA; The trained Efficentnet-B0 model is used to infer and classify the age of a single cervical spine and multiple cervical spine combination input combinations for each input condition, as well as the cervical spine area guided by ARDA, and the probability and age of the corresponding gender of the X-ray head positioning lateral image are obtained.

2. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 1, characterized in that: The training of the segmentation network of the U-Net architecture consists of the following steps: Divide the training dataset into several batches, each batch contains multiple images; Input batches of images into the U-Net network and calculate the prediction results for each pixel ; According to the comprehensive loss function Calculate the error between the predicted result and the true label; The gradient of loss to model parameters is calculated by back-propagation algorithm: Use the Adam optimizer to update the model parameters according to the gradient until the preset training rounds are reached.

3. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 2 is characterized in that: When training the segmentation network of the U-Net architecture, a multi-classification cross entropy loss function is used: in, is the total number of pixels, is the number of categories, is the one-hot encoding of the true label, For Category The predicted probability of , defines the Dice loss: Get the weighted sum of cross entropy loss and Dice loss: in, and is the weight parameter.

4. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 2, characterized in that: The acquisition of training data set includes the following steps: Obtain an image dataset with annotation information of key areas of the cervical spine, including original images and corresponding labeled images. In the labeled images, the target part of interest in the original image is annotated with a set pixel value, and the other parts are annotated as background; Preprocess the images in the image dataset: for each X-ray head positioning lateral image Perform histogram equalization to obtain a set of X-ray head positioning lateral images after data enhancement , where N=20174, Indicates A rectangular X-ray lateral head positioning image containing the head, teeth and spine with age-related changes. express The data enhancement results are For each enhanced X-ray head positioning lateral image The short side of is padded with zeros, and the size of the X-ray head positioning lateral image with a square shape after zero padding is scaled to obtain a set of X-ray head positioning lateral images after size transformation ; Each size-transformed X-ray head positioning lateral image Each image in is normalized to obtain a normalized X-ray head positioning lateral image set , Extract the normalized X-ray head positioning lateral image set Some images are used as training sets to train the segmentation network based on the U-Net structure.

5. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 4 is characterized in that: Acquiring the label image includes: For X-ray head positioning lateral images from different age groups, key areas of the cervical spine were manually annotated for image segmentation to generate labeled images: instance segmentation was performed on the C1-2, C3, and C4 parts of the X-ray head positioning lateral images. Since C1 and C2 overlapped and could not be separated, they were treated as one cervical vertebra C1-2.

6. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 1, characterized in that: The input conditions include: the cervical spine area defined by the minimum upper and lower boundaries of the segmentation template, the cervical spine without surrounding tissue, the cervical spine segmentation contour template, and the cervical spine area divided by fixed coordinates; a single cervical spine in four input modes, including C1-2, C3, C4 and cervical spine combinations (C1-2+C3, C1-2+C4, C3+C4, C1-C4), a total of 7 cervical spine areas were used for age inference and gender classification.

7. The method for combining cervical vertebrae inference age and gender classification based on deep learning according to claim 1, characterized in that: The cervical spine area guided by ARDA is set with the upper left corner of the X-ray head positioning lateral image as the coordinate origin, including: The upper left corner of the X-ray head positioning lateral image is taken as the coordinate origin, the right and bottom are the positive directions of the x-axis and y-axis respectively, the width and height are set as W and H respectively, and the coordinate system is established with the width of a single pixel as the unit length. On the coordinate system, (2 / 3W+100; 1 / 2H-100) is used as the coordinate of the upper left corner, and (W, H) is used as the coordinate of the lower right corner to cut out the cervical spine area guided by ARDA.

8. A joint cervical vertebra inference age and gender classification system based on deep learning, characterized in that: It includes a segmentation mask acquisition module, an input design module, and an inference module; The segmentation mask acquisition module is used to use the X-ray head positioning lateral image as the input of the segmentation network based on the U-Net architecture to obtain an image with segmentation results and obtain the segmentation mask of a single cervical vertebra in the X-ray head positioning lateral image; Performing median filtering, corrosion operation, expansion operation, binarization and re-expansion processing on the segmentation mask to obtain an accurate cervical vertebra segmentation mask; The input design module is used to design multiple input working conditions based on the obtained accurate cervical segmentation mask; set a single cervical vertebra and multiple cervical vertebra combination input combinations for each input working condition; set the range to take the cervical vertebra area guided by ARDA with the upper left corner of the X-ray head positioning lateral image as the coordinate origin; The inference module uses the trained Efficentnet-B0 model to perform age inference and classification on the cervical spine area guided by ARDA for a single cervical spine and multiple cervical spine combination input combinations for each input condition, and obtains the probability and age of the corresponding gender for the X-ray head positioning lateral image.

9. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it can implement the deep learning-based joint cervical vertebra inference age and gender classification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the deep learning-based joint cervical vertebrae inference age and gender classification method as described in any one of claims 1 to 7.

Images