Method of personal identification based on dental images

Abstract

The embodiment of the present application discloses a kind of personal identification methods based on dental image, comprising: the RGB image and infrared image of tooth are acquired, the RGB image includes: three photographing collection angles, two teeth RGB images of each photographing angle, the infrared image includes: the same photographing angle with the three photographing angles, two teeth infrared images of each photographing angle;The RGB image and infrared image of tooth are input to the tooth feature extraction neural network model of training completion, and the output tooth feature is obtained;The output tooth feature is compared with the preexisting tooth feature template, and personal identification result is obtained.The network model can more fully extract the feature information of dental image.The accuracy of biometric recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a personal identification method based on tooth images. BACKGROUND

[0002] The field of biometric recognition has been booming in recent years. Currently, the three most popular ways in the field of biometric recognition are face, iris, and fingerprint recognition. However, any one modality has its shortcomings, for example: face recognition is not effective in recognizing the faces of people who have undergone plastic surgery, twins with small facial differences, and the same person in the process of aging; fingerprint recognition is difficult in cases where the fingerprint area is small, damaged or worn out, resulting in no fingerprint, or the fingerprint is blurred; the related collection equipment of iris recognition is expensive, the recognition distance is limited, and the iris colors of people in the East and West are different, making it difficult for the algorithm to be universal, or the interference of contact lenses. Compared with the above, teeth have the advantages of not being easily damaged, tampered with, or leaked, and can replace or be combined with other modalities such as face and fingerprint features in certain scenarios to explore the effectiveness of multi-modal feature fusion. Tooth features can include tooth size, shape, position, and surface wear degree, and tooth images can be used to resist image, video, and physical attacks.

[0003] In the process of implementing the present application, the inventors found the following technical problems: Since teeth are arranged in a circle, a single tooth image cannot fully reflect the characteristics of tooth arrangement, and tooth images taken only in a natural light environment are easily affected by light and cannot reflect features such as surface wear degree. Therefore, tooth infrared images need to be introduced, which makes the number of images used to extract features numerous, and the information between each image is also strongly associated, making it difficult to accurately extract the image features of the teeth, and thus reducing the accuracy of using tooth images to achieve person identification. SUMMARY

[0004] The embodiment of the present application provides a personal identification method based on tooth images to solve the technical problem of low accuracy of using tooth images to achieve person identification in the prior art.

[0005] The embodiment of the present application provides a personal identification method based on tooth images, comprising:

[0006] Obtaining an RGB image and an infrared image of the teeth, the RGB image comprising: three shooting angles, two tooth RGB images at each shooting angle, and the infrared image comprising: the same shooting angles as the three shooting angles, two tooth infrared images at each shooting angle;

[0007] Inputting the RGB image and the infrared image of the teeth into a tooth feature extraction neural network model trained to obtain output tooth features.

[0008] compare the outputted tooth feature with a pre-stored tooth feature template to obtain a personal identification result;

[0009] The tooth feature extraction neural network model comprises:

[0010] The authentication branch receives an input RGB image and an infrared image of a tooth and outputs an identity mapping image feature;

[0011] The interdependent relationship extraction branch receives the input RGB image and the infrared image of the tooth, performs dimensionality increasing, convolution, and dimensionality reducing feature extraction on the input RGB image and the infrared image of the tooth, performs global information embedding and adaptive recalibration operations on the extracted features, obtains the importance of each feature channel, and adjusts the features according to the importance to obtain adjusted features;

[0012] The global attention feature extraction branch receives the input RGB image and the infrared image of the tooth, performs channel-by-channel convolution and point-by-point convolution on the input RGB image and the infrared image of the tooth in sequence, performs dimensionality increasing and feature extraction on the initial features after convolution, and inputs the extracted features into a first global attention mechanism module for relationship perception, so as to realize relationship perception from two dimensions of channels and space and obtain global attention features;

[0013] The output module is configured to fuse the identity mapping image feature, the adjusted features, and the global attention features, perform pooling down-sampling operations, and obtain output features through a fully connected layer.

[0014] Further, the interdependent relationship extraction branch comprises:

[0015] The feature extraction unit sequentially performs 1*1 convolution dimensionality increasing, 3*3 convolution, and 1*1 convolution dimensionality reducing operations on the input RGB image and the infrared image of the tooth.

[0016] The importance calculation unit is configured to perform global pooling on the features after the dimensionality reducing operation, change each two-dimensional feature channel into a real number, multiply the real number with the features after the dimensionality reducing operation, assign a weight to each channel, and obtain the importance of each feature channel through a bottleneck of two fully connected layers and an activation function.

[0017] Further, the global attention feature extraction branch comprises:

[0018] The S0 module is used for sequentially performing channel-by-channel convolution and point-by-point convolution on an input image, the channel-by-channel convolution generates a feature map with the same number of channels as the input image, and is responsible for extracting image features; the point-by-point convolution uses a 1*1*M convolution kernel, M is the number of channels of the previous layer, is used for weighted combination of the feature map generated in the previous step, and the number of convolution kernels is used to control the number of new feature maps generated;

[0019] The S1 module and the S2 module have the same structure, are used for performing 1*1 point convolution on the feature map to increase the dimension, passing the feature after the dimension increase through a 3*3 deep convolution to extract the feature in depth, passing the feature after the depth extraction through an RGA attention block to infer attention by using a relationship perception global attention module, and performing dimension reduction on the feature by 1*1 point convolution to refine the feature; the refined feature is input into a block attention unit,

[0020] The relationship perception global attention mechanism is combined from the channel and spatial dimensions, the channel dimension is that each different channel is given different weights for differentiation, and the spatial dimension is that the importance of different position pixels is different, so that the data features are better extracted, and finally the feature is dimensionally reduced by 1*1 point convolution to refine the feature; the refined feature is used to perform local feature interaction by applying self-attention in a sub-window; then the feature block after the local feature interaction is applied with self-attention on the grid after the sub-window decomposition to calibrate the feature.

[0021] Further, the RGB image and the infrared image of the teeth are obtained, including:

[0022] The RGB image and the infrared image of two full teeth exposed in the front, two half teeth exposed in the front, one full tooth exposed in the left at an angle of 30°, and one full tooth exposed in the right at an angle of 30°.

[0023] Further, the RGB image and the infrared image of the teeth are obtained, including:

[0024] The image is preprocessed, and a tooth ROI region image is extracted as the RGB image and the infrared image of the teeth.

[0025] Further, the RGB image and the infrared image of the teeth are obtained, including:

[0026] The face image key points in the RGB image and the infrared image are extracted;

[0027] The face contour is constructed based on the face image key points;

[0028] The pixel points of the face contour are set to 0, and are multiplied by the RGB image and the infrared image to obtain the RGB image and the infrared image of the teeth.

[0029] Further, the self-attention in the sub-window is applied to the refined features for local feature interaction, including:

[0030] The refined features are set as HxWxC;

[0031] The features are first divided into H / pxW / p windows of p*p size, and the windows do not overlap each other, and then self-attention is applied in each sub-window for local feature interaction

[0032] The personal identification method based on tooth images provided by the embodiment of the application, by acquiring the RGB image and the infrared image of the teeth, the RGB image includes: three shooting and collecting angles, two tooth RGB images of each shooting angle, the infrared image includes: the same shooting angle as the three shooting angles, two tooth infrared images of each shooting angle; the RGB image and the infrared image of the teeth are input into the tooth feature extraction neural network model trained, and the output tooth feature is obtained; the output tooth feature is compared with the pre-stored tooth feature template, and a personal identification result is obtained; the tooth feature extraction neural network model comprises: an authentication branch, the authentication branch receives the input RGB image and infrared image of the teeth, and outputs an identity mapping image feature; a mutual dependence relationship extraction branch, the mutual dependence relationship extraction branch receives the input RGB image and infrared image of the teeth, and performs dimensionality reduction, convolution, and re-dimensionality reduction feature extraction on the input RGB image and infrared image of the teeth, embeds global information into the extracted features, and performs adaptive re-correction operation, obtains the importance of each feature channel, and adjusts the features according to the importance, and obtains the adjusted features; a global attention feature extraction branch, the global attention feature extraction branch receives the input RGB image and infrared image of the teeth, performs channel-by-channel convolution and point-by-point convolution on the input RGB image and infrared image in turn, performs dimensionality reduction and feature extraction on the initial features after convolution, and inputs the extracted features into a relationship-aware first global attention mechanism module to realize relationship awareness from two dimensions of channel and space, and obtain global attention features; an output module, configured to fuse the identity mapping image feature, the adjusted features and the global attention features, perform pooling down-sampling operation, and obtain output features through a full connection layer. By using the characteristics of high resolution and contrast but poor anti-interference ability of the RGB image, and the characteristics of low resolution and contrast but strong anti-interference ability of the thermal infrared image, using deep separable convolution, combining block and grid attention, spatial and channel attention, and relationship-aware global attention, the features extracted by the three branches are fused, so that the network model can more fully extract the feature information of the tooth image. The accuracy of biometric identification is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Other features, objects, and advantages of the application will become more apparent from the following detailed description when read in connection with the following drawings:

[0034] Figure 1 is a flowchart of a personal identification method based on a tooth image provided by an embodiment of the application;

[0035] Figure 2 is a structural diagram of a tooth feature extraction neural network model in a personal identification method based on a tooth image provided by an embodiment of the application;

[0036] Figure 3 is a structural diagram of an MRSA module in a tooth feature extraction neural network model in a personal identification method based on a tooth image provided by an embodiment of the application. DETAILED DESCRIPTION

[0037] The application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the application, but not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings, not all the structures.

[0038] Figure 1 is a flowchart of a personal identification method based on a tooth image provided by an embodiment of the application, which can be applied to the case of biological identification using multiple and multi-angle tooth images. The method can be executed by a personal identification device based on a tooth image, and can be integrated in a server. Specifically, the method comprises the following steps:

[0039] In step 110, an RGB image and an infrared image of the teeth are obtained. The RGB image comprises three photographing and collecting angles, and two tooth RGB images for each photographing angle. The infrared image comprises photographing angles same as the three photographing angles, and two tooth infrared images for each photographing angle.

[0040] Exemplarily, the infrared thermal imager can be used to collect the image data of the teeth of a person. Considering the richness of the data collection and the generalization ability of the model, 6 images of each person are collected, and are divided into 3 shooting angles, that is, 2 images of full teeth in the front, 2 images of half teeth in the front, 1 image of full teeth with a left tilt of 30°, and 1 image of full teeth with a right tilt of 30°. 6 thermal infrared tooth images are obtained, and the corresponding RGB images are derived through IRSoft. Each person has a total of 12 tooth images (6 RGB images and 6 thermal infrared images). Because the RGB image has high resolution and contrast, but the image anti-interference ability is poor and is easily affected by natural conditions such as bad weather, while the thermal infrared image data has low resolution and contrast, but has strong anti-interference ability, the complementary data of the two modalities can greatly improve the richness of the data.

[0041] Correspondingly, the method further includes: pre-processing the images, and extracting a tooth ROI region image as the RGB image and the infrared image of the tooth. Exemplarily, the extraction of the tooth ROI region image as the RGB image and the infrared image of the tooth can include: extracting a face image key point in the RGB image and the infrared image; constructing a face contour based on the face image key point; setting a pixel point of the face contour to 0, and multiplying the pixel point by the RGB image and the infrared image to obtain the RGB image and the infrared image of the tooth. The collected image data contains information of other parts except the teeth, which will interfere with the training model. Therefore, by means of the 68 key points of the face in the dlib library, appropriate feature key points around the teeth are selected, the pixel points in the polygon region surrounded by the key points are set to 1 as a mask of the region of interest, and other pixel points outside the region are set to 0. Then, the image data of the region of interest is obtained by multiplying the pixel points by the image to be processed.

[0042] In step 120, the RGB image and the infrared image of the tooth are input into the trained tooth feature extraction neural network model to obtain the output tooth feature.

[0043] In this embodiment, the tooth feature extraction neural network model can perform feature extraction on the input image.

[0044] Exemplarily, the tooth feature extraction neural network model includes:

[0045] The authentication branch receives the input RGB image and infrared image of the tooth, and outputs an identity mapping image feature.

[0046] The interdependence relationship extraction branch receives the input RGB image and infrared image of the tooth, and performs dimensionality elevation, convolution, and feature extraction again after dimensionality reduction on the input RGB image and infrared image of the tooth, performs global information embedding and adaptive recalibration operations on the extracted features, obtains the importance of each feature channel, and adjusts the features according to the importance to obtain adjusted features; the global attention feature extraction branch receives the input RGB image and infrared image of the tooth, performs channel-by-channel convolution and point-by-point convolution on the input RGB image and infrared image of the tooth in sequence, performs dimensionality elevation and feature extraction on the initial features after convolution, and inputs the extracted features into a first global attention mechanism module for relationship perception, so that relationship perception is realized from two dimensions of channels and space to obtain global attention features; and the output module is configured to perform pooling downsampling operation after fusing the identity mapping image features, the adjusted features, and the global attention features, and obtain output features through a fully connected layer.

[0047] Figure 2 is a structural schematic diagram of a tooth feature extraction neural network model in a personal identification method based on tooth images provided by an embodiment of the present application, referring to Figure 2 , which includes three branches:

[0048] The first branch can be a dentity structure, which is inspired by the identity mapping in the ResNet network, that is, the features of the original input image extracted are not processed at all.

[0049] The second branch is an IR-CA structure, which first elevates the dimension of the features through 1x1 convolution to improve the richness of the features, so as to facilitate subsequent operations to extract more comprehensive features, then extracts the features through 3x3 convolution, and then reduces the dimension of the features through 1x1 convolution to further refine the features; and adds a channel attention mechanism, which first performs feature compression along the spatial dimension to change each two-dimensional feature channel into a real number, which has a global receptive field to some extent, and the output dimension matches the number of input feature channels, and a mechanism similar to the gate in the recurrent neural network is used, which generates weights for each feature channel through different parameters to explicitly model the correlation between feature channels.

[0050] The third branch is an MRSA module, Figure 3 is a structural schematic diagram of an MRSA module in a tooth feature extraction neural network model in a personal identification method based on tooth images provided by an embodiment of the present application. Referring to Figure 3, specifically comprising S0, S1 and S2 three structural blocks. First is S0, which performs twice depth separable convolution on the input image. The depth separable convolution comprises channel-wise convolution and point-wise convolution in sequence. One convolution kernel of the channel-wise convolution is responsible for one channel. The feature map generated in this process has the same number of channels as the input, and only extracts image features. The point-wise convolution uses a 1x1xM convolution kernel, where M is the number of channels of the previous layer, that is, the feature map generated in the previous step is combined by weighting. The number of convolution kernels is used to control the number of new feature maps generated. Compared with ordinary convolution, the depth separable convolution can greatly save the number of parameters and reduce the operation cost, and can make the convolution deeper on the basis of consuming the same number of parameters, so that the feature extraction is more sufficient.

[0051] The S1 module and the S2 module have the same structure, which is used to perform 1x1 point convolution on the feature map to increase the dimension, and then perform 3x3 depth convolution on the feature map after dimension increase to extract the feature in depth. The feature after depth extraction is input into the RGA attention block to infer attention by using the relationship perception global attention module, and then the feature is refined by 1x1 point convolution. The refined feature is input into the block attention unit,

[0052] By combining the relationship perception global attention mechanism in the channel and spatial dimensions, the different channels are distinguished by giving different weights, and the importance of different position pixels is different, so that the data features can be better extracted. Finally, the feature is refined by 1x1 point convolution. The refined feature is used to perform local feature interaction by applying self-attention in the sub-window. Then, the feature block after local feature interaction is applied with self-attention on the grid after sub-window decomposition for feature calibration. The method for applying self-attention in the sub-window to perform local feature interaction on the refined feature comprises: setting the refined feature as HxWxC; first, the feature is divided into H / pxW / p p x p windows, and the windows do not overlap with each other, and then self-attention is applied in each sub-window to perform local feature interaction. Although local attention avoids cumbersome calculation, it is not suitable for large-scale data sets. The method further proposes a sparse global attention, called grid attention, which no longer divides the feature by a fixed-size window, but uses g x g grids with size H / g x W / g to globally refine the feature, and applies self-attention on the decomposed grid. In this way, the same color can be mixed in space through self-attention operation.

[0053] Specifically, the specific operation of each block of the S1 module and the S2 module is described as follows: first, 1*1 point convolution is performed to upgrade the features and improve the richness of the features, thereby creating conditions for subsequent feature extraction; second, 3*3 deep convolution is performed on the features for deep feature extraction, which is the key to feature extraction; then, an RGA attention block is added, and the RGA module refers to a relationship-aware global attention mechanism. The relationship-aware global attention mechanism is used to infer attention. Specifically, when calculating the attention of a certain feature position, in order to grasp the information in the global range, the feature itself and the pair-wise correlation with all feature positions are stacked together so as to learn the attention through convolution operation. In this way, both local features and global features are considered. Finally, the relationship-aware global attention mechanism is combined from the channel and spatial dimensions. The channel dimension refers to that each different channel is given different weights for differentiation, and the spatial dimension refers to that the importance of different position pixels is different, so that the data features are better extracted. Finally, 1*1 point convolution is performed on the features for dimension reduction to refine the features; the block attention is first to divide the image features into a plurality of non-overlapping sub-windows of P*P, and then self-attention is applied in each sub-window to calibrate the features; the grid attention is to use a fixed G*G uniform grid to grid the tensor into a shape, instead of using a fixed window size to divide the feature map. The global attention covers the pixels in the sparse uniform grid on the entire 2D space. All the above operations are a block.

[0054] Then, the features extracted by the above three branches are fused to obtain more comprehensive feature representations extracted in multiple dimensions, which is expressed by the following formula:

[0055] F Teeth =F identity +F IR-CA +F MRSA .

[0056] Wherein, F identity is the original image feature represented by the identity in the ResNet, F IR-CA is the image feature extracted by the IR-CA branch, F MRSA is the image feature extracted by the MRSA module, and F Teeth is the final fused tooth image feature. After the features extracted by the branches are fused, a pooling down-sampling operation is performed, and an output feature is obtained through a fully connected layer.

[0057] In step 130, the output tooth feature is compared with a pre-stored tooth feature template to obtain a personal identification result.

[0058] The existing template in the matching database is matched, similarity is calculated, and a comparison threshold is used for matching. A personal identification result is obtained according to the matching result.

[0059] It should be noted that the above only describes the preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A personal identification method based on dental images, characterized by, The method comprises the following steps: acquiring an RGB image and an infrared image of teeth, wherein the RGB image comprises: three shooting angles, two RGB images of teeth at each shooting angle, and the infrared image comprises: the same shooting angles as the three shooting angles, two infrared images of teeth at each shooting angle; inputting the RGB image and the infrared image of the teeth into a trained tooth feature extraction neural network model to obtain output tooth features; comparing the output tooth features with a pre-stored tooth feature template to obtain a personal identification result; the tooth feature extraction neural network model comprises: an authentication branch, which receives the input RGB image and infrared image of teeth and outputs an identity mapping image feature; a mutual dependence relationship extraction branch, which receives the input RGB image and infrared image of teeth, performs dimensionality elevation, convolution, and dimensionality reduction feature extraction on the input RGB image and infrared image of teeth, performs global information embedding and adaptive recalibration operations on the extracted features, obtains the importance of each feature channel, and adjusts the features according to the importance to obtain adjusted features; a global attention feature extraction branch, which receives the input RGB image and infrared image of teeth, performs channel-by-channel convolution and point-by-point convolution on the input RGB image and infrared image of teeth in sequence, performs dimensionality elevation and feature extraction on the initial features after convolution, and inputs the extracted features into a first global attention mechanism module with relationship perception to realize relationship perception from two dimensions of channels and space, and obtain global attention features; an output module, which is configured to perform pooling and down-sampling operations on the identity mapping image feature, the adjusted features, and the global attention features, and obtain output features through a fully connected layer.

2. The method of claim 1, wherein, the mutual dependence relationship extraction branch comprises: a feature extraction unit, which sequentially performs 1*1 convolution dimensionality elevation, 3*3 convolution, and 1*1 convolution dimensionality reduction operations on the input RGB image and infrared image of teeth; an importance calculation unit, which is configured to perform global pooling on the features after the dimensionality reduction operation, change each two-dimensional feature channel into a real number, multiply the real number with the features after the dimensionality reduction operation, assign a weight to each channel, and obtain the importance of each feature channel through a bottleneck of two fully connected layers and an activation function.

3. The method of claim 2, wherein, the global attention feature extraction branch comprises: S0, S1, and S2 modules, the S0 module is configured to sequentially perform channel-by-channel convolution and point-by-point convolution on an input image, the channel-by-channel convolution generates a feature map with a number of channels consistent with the number of channels of the input image, and is responsible for extracting image features; the point-by-point convolution adopts a 1×1×M convolution kernel, M is the number of channels of the previous layer, is configured to perform weighted combination on the feature map generated in the previous step, and controls the number of new feature maps generated through the number of convolution kernels. The S1 module and the S2 module are structurally identical, are used for dimension increasing of 1*1 point convolution on the feature map, depth extraction of the feature after dimension increasing through 3*3 depth convolution, local feature interaction of the feature after depth extraction through RGA attention block, inferring attention by using a relationship perception global attention module, dimension reduction of the feature through 1*1 point convolution, and refining the feature; the refined feature is input into a block attention unit, By combining the relationship perception global attention mechanism in two dimensions of channels and space, the channel dimension is that each different channel is given different weights for differentiation, and the space dimension is that the importance of different position pixels is different, so that the data features are better extracted, and finally the feature is refined through 1*1 point convolution; the refined feature is used to apply self-attention in a sub-window for local feature interaction; then the feature block of the local feature interaction is applied to self-attention on the grid after sub-window decomposition for feature calibration.

4. The method of claim 1, wherein, The RGB image and the infrared image of the teeth are obtained, including: 2 RGB images and infrared images of full teeth, 2 RGB images and infrared images of half teeth, 1 RGB image and infrared image of left inclined 30° full teeth, and 1 RGB image and infrared image of right inclined 30° full teeth.

5. The method of claim 4, wherein, The RGB image and the infrared image of the teeth are obtained, including: The image is preprocessed, and a tooth ROI region image is extracted as the RGB image and the infrared image of the tooth.

6. The method of claim 5, wherein, The image is preprocessed, and a tooth ROI region image is extracted as the RGB image and the infrared image of the tooth. The image is preprocessed, and a tooth ROI region image is extracted as the RGB image and the infrared image of the tooth. The face image key points in the RGB image and the infrared image are extracted; The face contour is constructed based on the face image key points; 7. The method of claim 3, wherein, The pixel points of the face contour are set to 0, and are multiplied by the RGB image and the infrared image to obtain the RGB image and the infrared image of the tooth. The refined feature is used to apply self-attention in a sub-window for local feature interaction, including: The refined feature is set as H*W*C; First, the feature is divided into H / p*W / p windows of p*p size, and the windows do not overlap, and then self-attention is applied in each sub-window for local feature interaction.

Images