A small sample skull image similarity measurement method based on deep learning
Patent Information
- Application Number
- CN202410108748.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-25
- Publication Date
- 2026-10-20
- Estimated Expiration
- 2044-01-25
AI Technical Summary
[0003]1982年中国专家兰玉文等根据解剖投影的二维标准，提出了颅像重合鉴定标准及鉴定方法，首次考虑了标志点和边缘轮廓，随后的研究者，大多都是基于标志点和边缘轮廓实现颅像重合，例如：Bajnoczky等人基于标准点的坐标指数构建数学模型进行度量；Gordon基于人体形态学和解剖学理论，将特征点分为三个等级，根据点间距离进行度量；Pesce等人基于高阶多项式方程和傅里叶谐波分析理论设计抛物线拟合法对颅骨和人像进行拟合度量；由于上述传统方法过分依赖于X线影像、解剖学专业知识以及专用设备，操作复杂，且耗时长，因此，不易推广利用
本发明省略了对颅像数据进行特征点和特征线的标注过程，采用结合CBAM的ResNet50网络特征提取模块，可以有效提取颅像的全局特征和局部特征，使得模型达到关注图像全局信息和局部细节信息的效果，再利用基于孪生神经网络的度量模块度量颅骨图像和人像数据对之间复杂的非线性关系，对同源、异源样本有较高的区分能力，具体地，本发明的度量颅像相似性的准确率达到了96.8%，且对同源样本对的相似性度量均值为0.833，异源样本对的相似性度量均值为0.358，均优于现有的网络结构。
Smart Images
Figure CN118470352B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition technology, specifically a method for measuring the similarity of small-sample skull images based on deep learning. Background Technology
[0002] Skull identification is an important means of tracing skulls in criminal cases. Skull image superimposition is an important technique for identifying the identity of unidentified skulls during criminal investigation. Based on the database of skull remains of unknown origin and photos of missing persons, skull image superimposition defines a series of indicators for comparison based on the anatomical and projection characteristics of the human face in relation to the skull. These indicators are used to evaluate the reliability of superimposition identification results, and overlap tests are conducted to achieve identity verification based on the overlap results.
[0003] In 1982, Chinese experts Lan Yuwen et al. proposed a standard and method for identifying cranial image coincidence based on the two-dimensional standard of anatomical projection. This was the first time that landmark points and edge contours were considered. Subsequent researchers have mostly achieved cranial image coincidence based on landmark points and edge contours. For example, Bajnoczky et al. constructed a mathematical model based on the coordinate index of standard points for measurement; Gordon divided feature points into three levels based on human morphology and anatomy theory and measured them according to the distance between points; Pesce et al. designed a parabolic fitting method based on higher-order polynomial equations and Fourier harmonic analysis theory to fit and measure the skull and human image. However, because the above traditional methods rely too much on X-ray images, anatomical expertise, and specialized equipment, they are complex to operate and time-consuming, and therefore not easy to promote and use.
[0004] In recent years, with the rapid development of computer technology, the accuracy and efficiency of computer-aided cranial image superposition technology have gradually improved. For example, Ma Zhentao designed an improved Harris algorithm to extract strong corner points, designed a Snake model to fit contour curves, and improved the accuracy of cranial image recognition based on fuzzy C-means clustering algorithm. Li Wenyong proposed a fast cranial image superposition model based on RBF (Radial Basis Function) neural network, labeled feature points based on Dlib library, and labeled feature lines based on template and knowledge combination method, which improved the accuracy of cranial image recognition. However, existing machine learning-based cranial image superposition methods are still greatly affected by the accuracy of feature point and feature line labeling, resulting in low efficiency of cranial image superposition methods.
[0005] Considering that the problem of cranial image overlap is essentially a problem of cranial image similarity measurement, it is hoped that an accurate and efficient method for measuring cranial image similarity can be proposed by leveraging image recognition technology and deep learning theory, so as to improve the accuracy and efficiency of cranial image similarity measurement. Summary of the Invention
[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a deep learning-based method for measuring small-sample cranial image similarity, thereby improving the accuracy and efficiency of cranial image similarity measurement.
[0007] To achieve the above objectives, the present invention employs the following technical solution: A deep learning-based method for measuring small-sample cranial image similarity includes the following steps: Step 1: First, preprocess the collected skull images and human images separately, then normalize the preprocessed skull images and human images, and then label the skull images and human images in pairs according to the same person's standard to obtain the skull image dataset. Step 2: Construct a deep learning-based small-sample cranial image similarity measurement network, including a data augmentation module, a feature extraction module, and a measurement module. Specifically: an online augmentation module based on a deep learning framework is used to perform random augmentation processing on the input cranial image data; a feature extraction module combining CBAM is formed by introducing CBAM into the ResNet50 network; and a Siamese neural network is used as the measurement module to output cranial image data similarity metrics. Step 3: Train a deep learning-based small-sample cranial image similarity measurement network; Step 4: Input the skull image data into the trained deep learning-based small sample skull image similarity measurement network model to obtain the skull image similarity.
[0008] Furthermore, the process of preprocessing the skull image in step 1 is as follows: Step 1.1.1: Using Trimble RealWorks and Houdini software, the collected multi-dimensional skull point cloud data is used to reconstruct a three-dimensional skull model to obtain a skull image. Step 1.1.2: Use Geomagic Studio software to repair noise and holes in the skull image; Step 1.1.3: Use the Frankfurt coordinate system to perform orientation correction on the skull image; Step 1.1.4: Use Geomagic Studio software to perform positive sampling on the skull image to obtain the preprocessed skull image.
[0009] Furthermore, the process of preprocessing the portrait in step 1 is as follows: Step 1.2.1: Remove the influence of light on the portrait during the shooting process by performing local adaptive histogram equalization. Step 1.2.2: Remove the scanning noise of the portrait by median filtering to obtain the preprocessed portrait.
[0010] Furthermore, the normalization process in step 1 involves cropping the skull image and the portrait separately to obtain a skull image and portrait with a resolution of 600×450.
[0011] Furthermore, step 2, which introduces CBAM into the ResNet50 network, is as follows: The original ResNet50 network's convolutional layer modules and parameters were retained, the mean pooling layer and fully connected layer after the flattening operation were removed, and CBAM modules were added before the first max pooling layer and the last mean pooling layer, respectively.
[0012] Furthermore, the specific process of training the deep learning-based few-sample cranial image similarity measurement network in step 3 is as follows: Step 3.1: Input the cranial image data into the enhancement module for online random enhancement; Step 3.2: Input the skull and human image from the enhanced cranial image data into two weight-shared feature extraction modules respectively to obtain the global and local features of the cranial image data; Step 3.3: Input the feature map output by the feature extraction module into the metric module. Calculate the distance between the skull and human image data pairs using the metric algorithm to form a distance matrix. Connect each neuron in the fully connected layer of the extraction module with all neurons in the previous layer to integrate the discriminative local information of the feature map obtained from the previous layer. Activate the feature map using an activation function and perform feature mapping on each element in the feature map to obtain the skull-image similarity. Step 3.4: Train the deep learning-based small-sample cranial image similarity measurement network using the binary logistic cross-entropy loss function, as shown below: In the formula, Let y be the model prediction value, w be the label value, w=1, and σ be the sigmoid function. During training, a batch-size sample pair of data is first generated, and the loss value of the network model is calculated according to the loss function. Then, the network model parameters are updated by backpropagation, resulting in a trained deep learning-based small sample head image similarity measurement network model.
[0013] Furthermore, the online random enhancement in step 3.1 includes magic enhancement, brightness adjustment, movement, noise reduction, and cropping.
[0014] Furthermore, the specific process of step 3.2 is as follows: Step 3.2.1: Use the channel domain attention module to assign different importance weights among different channels to improve the representation ability of the feature map. Obtain channel information by calculating the average and maximum values of each channel, and generate feature weights for weighting the feature map to emphasize useful channels and suppress channels that are not useful for the task. Step 3.2.2: Using the spatial domain attention mechanism, through max pooling, mean pooling, convolution operations and activation operations, calculate the corresponding feature weights for different positions of different feature maps to obtain the global and local features of the skull and human image.
[0015] Furthermore, the formula for calculating the distance between the skull and portrait data pairs in step 3.3 is expressed as follows: In the formula, i represents the feature points in the skull, Let i represent the feature value corresponding to i; j represent the feature point in the portrait. , This represents the eigenvalue corresponding to j.
[0016] Compared with the prior art, the present invention has the following technical effects: This invention omits the process of labeling feature points and feature lines in cranial image data. Instead, it employs a ResNet50 network feature extraction module combined with CBAM, which can effectively extract global and local features of cranial images. This allows the model to focus on both global and local details of the image. Furthermore, a measurement module based on Siamese neural networks is used to measure the complex nonlinear relationship between cranial images and human portrait data pairs, demonstrating a high ability to distinguish between homologous and heterologous samples. Specifically, the accuracy of this invention in measuring cranial image similarity reaches 96.8%, with an average similarity measurement of 0.833 for homologous sample pairs and an average similarity measurement of 0.358 for heterologous sample pairs, both of which are superior to existing network structures. Attached Figure Description
[0017] Figure 1 : Flowchart of the preprocessing of cranial image data in this invention; Figure 2 : A schematic diagram of the structure of the cranial image similarity measurement network of the present invention; Figure 3 : A schematic diagram of the data enhancement of the present invention; Figure 4 : A schematic diagram of the feature extraction module of the present invention; Figure 5 : A schematic diagram of the measurement module of the present invention. Detailed Implementation
[0018] The specific content of the present invention will be further explained in detail below with reference to the embodiments.
[0019] A deep learning-based method for measuring small-sample cranial image similarity includes the following steps: Step 1, as follows Figure 1 As shown, the acquired skull images and portraits are preprocessed, and then normalized to obtain skull images and portraits with a resolution of 600×450. Then, according to the standard of the same person, the skull images and portraits are paired and labeled to obtain a skull image dataset. The specific process is as follows: Step 1.1: Due to uncontrollable factors such as human error and environmental interference during CT scanning, the obtained scan point cloud data contains redundancy and voids. Therefore, effective preprocessing of the cranial image data is required. The process is as follows: Step 1.1.1: Using Trimble RealWorks and Houdini software, the multi-dimensional skull point cloud data acquired by CT scan is used to reconstruct a three-dimensional skull model to obtain skull images. Step 1.1.2: Use Geomagic Studio software to repair noise and holes in the skull image; Step 1.1.3: Use the Frankfurt coordinate system to perform orientation correction on the skull image; Step 1.1.4: Use Geomagic Studio software to perform orthogonal sampling on the skull image to obtain the preprocessed skull image; Step 1.2: Use the acquired photos that meet the criteria of "frontal view," "above the neck," and "no obstruction by clothing" as portrait data. Due to interference from lighting angles, fluctuations, and other external factors during the shooting process, the acquired portrait data contains noise. To ensure the accuracy and fidelity of the portrait, preprocessing of the portrait data is required. The specific process is as follows: Step 1.2.1: Remove the influence of light on the portrait during the shooting process by performing local adaptive histogram equalization. Step 1.2.2: Remove portrait scanning noise by median filtering to obtain the preprocessed portrait; Step 1.3: Cropping the preprocessed skull image and portrait using the same resolution to achieve normalization, resulting in skull images and portraits with a resolution of 600×450. According to the standard of whether they are "the same person", the skull images and portraits are paired and labeled to form sample pairs, thus obtaining the skull image dataset. Step 2, construct as follows Figure 2 The deep learning-based few-sample skull image similarity measurement network shown includes the following specific processes: Step 2.1: Use the online augmentation module based on the deep learning framework as a data augmentation module to perform random augmentation processing on the input cranial image data, such as... Figure 3 As shown, this includes blurring, brightness alteration, motion, noise reduction, and cropping to expand the feature space of the internet user model learning, enhance generalization ability, enrich the sample feature space of the cranial image data, and increase the diversity of training samples. Step 2.2: The original ResNet50 network's convolutional layers and parameters are retained. The mean pooling layer and fully connected layer following the flattening operation are removed. A CBAM attention mechanism module is added before the first max pooling layer and before the last mean pooling layer, resulting in a ResNet50 feature extraction network incorporating the CBAM attention mechanism module. This network serves as the feature extraction module, with the structure shown below. Figure 4 As shown, the ResNet50 network with the added skip connection module can cope with the degradation problem in the process of skull image feature extraction, and the CBAM module enables the model to focus on both global and local details of the image. Step 2.3: Use the Siamese neural network as the metric module, with the following structure: Figure 5 As shown, since Siamese neural networks are a training method based on pairwise data, feature mapping is performed on each input sample, and then the distance between sample pairs is calculated through a metric algorithm to finally obtain a similarity metric value for the input data. Therefore, the metric module uses absolute distance to measure the distance between points of cranial image features, calculates the degree of difference of cranial images based on the distance between points of cranial image features, and finally obtains cranial image similarity. Step 3: Train the parameters of a deep learning-based small-sample cranial image similarity measurement network. The specific process is as follows: Step 3.1: Input the cranial image data into the enhancement module for online random enhancement to enhance the feature space of the sample pairs used in network model training; Step 3.2: Divide the sample pairs obtained from the enhancement module into skull samples and human image samples, and input them into two ResNet50 feature extraction networks (i.e., feature extraction networks) with shared weights and combined with CBAM modules. CBAM is a lightweight network plugin that can adapt to most convolutional neural networks. It operates on the input features through two dimensions: channel and space. The obtained weighted features are multiplied with the original input features to obtain new output feature maps. Therefore, the channel domain attention module is first used to assign different importance weights to different channels to improve the representation ability of the feature maps. The average and maximum values of each channel are calculated to obtain channel information and generate feature weights for weighting the feature maps to emphasize useful channels and suppress channels that are not useful for the task. Then, the spatial domain attention mechanism is used to calculate the corresponding feature weights for weighting different positions of different feature maps through max pooling, mean pooling, convolution operations, and activation operations to obtain the global and local features of the sample pairs. Step 3.3: Input the feature map output by the feature extraction module into the metric module. Calculate the distance between the skull and human image data pairs using the metric algorithm. Use absolute distance to measure the distance between points of the skull image features, expressed as: In the formula, i represents the feature points in the skull, Let i represent the feature value corresponding to i; j represent the feature point in the portrait. , This represents the eigenvalue corresponding to j; The calculated distances between points form a distance matrix. The value at each position in the matrix represents the degree of difference between sample pairs at different positions. Each neuron in the fully connected layer of the extraction module is fully connected to all neurons in the previous layer, thereby integrating the discriminative local information of the feature map obtained from the previous layer. An activation function is used to activate the feature map, and each element in the feature map is feature-mapped to obtain the cranial image similarity. Step 3.4: Train the deep learning-based small-sample cranial image similarity measurement network using the binary logistic cross-entropy loss function, as shown below: In the formula, Let y be the model prediction value, w be the label value, w=1, and σ be the sigmoid function. The binary logistic cross-entropy loss function is based on the inter-class competition mechanism, which makes it more focused on learning the differences between different categories, so that the model can effectively distinguish between class differences and improve the measurement efficiency and performance for homologous and heterologous sample pairs. During training, a batch-size sample pair of data is first generated, the model loss value is calculated according to the loss function, and finally the network model parameters are updated according to the backpropagation of the loss to obtain a trained deep learning-based small sample skull image similarity measurement network model. Step 4: Input the skull image data into the trained deep learning-based small sample skull image similarity measurement network model to obtain the skull image similarity.
[0020] Based on the cranial image dataset constructed in this embodiment, the measurement method of this embodiment is compared with other measurement methods. At the same time, it is described whether each method is based on neural network design, whether feature annotation is required during the measurement process, and what measurement method is used. The experiment is shown in Table 1. Table 1. Comparison of Accuracy of Different Measurement Methods The results show that the accuracy of the measurement method in this embodiment reaches 96.80%, which is better than other traditional methods. Compared with other non-neural network-based measurement methods, it improves the accuracy by up to 19.90%, and compared with other neural network-based measurement methods, it improves the accuracy by up to 3.20%. Furthermore, no feature point and feature line calibration is required during the measurement process, making the experimental process simpler.
Claims
1. A method for measuring small-sample cranial image similarity based on deep learning, characterized in that, Includes the following steps: Step 1: First, preprocess the collected skull images and human images separately, then normalize the preprocessed skull images and human images, and then label the skull images and human images in pairs according to the same person's standard to obtain the skull image dataset. Step 2: Construct a deep learning-based small-sample cranial image similarity measurement network, including a data augmentation module, a feature extraction module, and a measurement module. Specifically: an online augmentation module based on a deep learning framework is used to perform random augmentation processing on the input cranial image data; a feature extraction module combining CBAM is formed by introducing CBAM into the ResNet50 network; and a Siamese neural network is used as the measurement module to output cranial image data similarity metrics. The process of introducing CBAM into the ResNet50 network is as follows: The original ResNet50 network's convolutional layer modules and convolutional parameters were retained, the mean pooling layer and fully connected layer after the flattening operation were removed, and CBAM modules were added before the first max pooling layer and the last mean pooling layer, respectively. Step 3: Train a deep learning-based small-sample cranial image similarity measurement network; The specific process is as follows: Step 3.1: Input the cranial image data into the enhancement module for online random enhancement; Step 3.2: Input the skull and human image from the enhanced cranial image data into two weight-shared feature extraction modules respectively to obtain the global and local features of the cranial image data; Step 3.3: Input the feature map output by the feature extraction module into the metric module, calculate the distance between the skull and human image data pairs using the metric algorithm, construct a distance matrix map, process the distance matrix map through two fully connected layers, then activate the feature map using an activation function, perform feature mapping on each element in the feature map, and obtain the skull-image similarity. Step 3.4: Train the deep learning-based small-sample cranial image similarity measurement network using the binary logistic cross-entropy loss function, as shown below: In the formula, Let y be the model prediction value, w be the label value, w=1, and σ be the sigmoid function. During training, a batch-size sample pair data is first generated, and the loss value of the network model is calculated according to the loss function. Then, the network model parameters are updated by backpropagation, resulting in a trained deep learning-based small sample head image similarity measurement network model. Step 4: Input the skull image data into the trained deep learning-based small sample skull image similarity measurement network model to obtain the skull image similarity.
2. The method for measuring small-sample cranial image similarity based on deep learning according to claim 1, characterized in that, The process of preprocessing the skull image in step 1 is as follows: Step 1.1.1: Using Trimble RealWorks and Houdini software, the collected multi-dimensional skull point cloud data is used to reconstruct a three-dimensional skull model to obtain a skull image. Step 1.1.2: Use Geomagic Studio software to repair noise and holes in the skull image; Step 1.1.3: Use the Frankfurt coordinate system to perform orientation correction on the skull image; Step 1.1.4: Use Geomagic Studio software to perform orthogonal sampling on the skull image to obtain the preprocessed skull image.
3. The method for measuring small-sample cranial image similarity based on deep learning according to claim 2, characterized in that, The process of preprocessing the portrait in step 1 is as follows: Step 1.2.1: Remove the influence of light on the portrait during the shooting process by performing local adaptive histogram equalization. Step 1.2.2: Remove noise from the portrait image by median filtering to obtain the preprocessed portrait.
4. The method for measuring small-sample cranial image similarity based on deep learning according to claim 1, characterized in that, The normalization process in step 1 involves cropping the skull image and the portrait separately to obtain a skull image and portrait with a resolution of 600×450.
5. The method for measuring small-sample cranial image similarity based on deep learning according to claim 1, characterized in that, The online random enhancements in step 3.1 include magic enhancement, brightness adjustment, movement, noise reduction, and cropping.
6. The method for measuring small-sample cranial image similarity based on deep learning according to claim 1, characterized in that, The specific process of step 3.2 is as follows: Step 3.2.1: Use the channel domain attention module to assign different importance weights among different channels to improve the representation ability of the feature map. Obtain channel information by calculating the average and maximum values of each channel, and generate feature weights for weighting the feature map to emphasize useful channels and suppress channels that are not useful for the task. Step 3.2.2: Using the spatial domain attention mechanism, through max pooling, mean pooling, convolution operations and activation operations, calculate the corresponding feature weights for different positions of different feature maps to obtain the global and local features of the skull and human image.
7. The method for measuring small-sample cranial image similarity based on deep learning according to claim 1, characterized in that, The formula for calculating the distance between the skull and portrait data pairs in step 3.3 is expressed as follows: In the formula, i represents the feature points in the skull, Let i represent the feature value corresponding to i; j represent the feature point in the portrait. , This represents the eigenvalue corresponding to j.