Soft-label based intra- and extra-fingerprint extraction method for OCT
By using the U-net network and the Gaussian weight descent soft-label method, the robustness and accuracy issues of OCT fingerprint extraction methods are solved, generating high-quality internal and external fingerprint images that are adaptable to different skin thicknesses and complex and varied conditions at the active epidermal junction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
- Filing Date
- 2023-05-23
- Publication Date
- 2026-08-04
AI Technical Summary
Existing OCT methods for extracting internal and external fingerprints are not robust enough to adapt to varying skin thicknesses and complex and variable ridge lines at the junction of active epidermis, making it difficult to extract high-quality internal and external fingerprints.
We employ the U-net semantic segmentation network from deep learning combined with the soft-label method of Gaussian weight descent. By preprocessing and segmenting the OCT volume data, we use Gaussian filters to assign different weights to the stratum corneum and dermis regions to extract internal and external fingerprints.
It achieves higher accuracy and robustness in internal and external fingerprint extraction, generating high-quality internal and external fingerprint images that can be adapted to different populations.
Smart Images
Figure CN116758592B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of optical coherence tomography and image processing technology, and in particular to a soft-label-based OCT internal and external fingerprint extraction method. Background Technology
[0002] In recent years, with the rapid development of biometric recognition technology in the field of security authentication, various biometric-based identity authentication technologies have also emerged, such as fingerprint recognition, facial recognition, and iris recognition. Currently, fingerprint recognition technology is the most common and widely used technology in daily life, employed in applications such as smartphone unlocking, mobile payment authentication, and smart door locks. The widespread use of fingerprint recognition technology stems from the universality, uniqueness, convenience, and ease of collection of fingerprints.
[0003] According to biological knowledge, the skin consists of three layers from the inside out: the epidermis, the dermis, and the subcutaneous tissue. The stratum corneum, the top layer of the epidermis, has depressions and protrusions that form the external fingerprint pattern. The junction between the epidermis and dermis is called the active epidermal junction, and the depressions and protrusions of this junction form the internal fingerprint pattern. Long-term observation has shown that the patterns of internal and external fingerprints are consistent with those of the external fingerprint. External fingerprints are easily affected by scars, stains, etc., while internal fingerprints are less susceptible to these external factors. Furthermore, external fingerprints are easily forged using materials such as silicone, while internal fingerprints effectively prevent deception. Optical coherence tomography (OCT) is a non-destructive, high-resolution, and non-invasive imaging technique that can effectively acquire information from 1-3 mm of subcutaneous tissue, such as sweat glands, the papillary layer, the stratum corneum, and the dermis. This makes it possible to extract internal and external fingerprints from 3D fingerprint data.
[0004] Existing OCT fingerprint extraction methods mainly fall into two categories: contour-based methods and en-face slicing methods. The former locates the contour at the junction of the stratum corneum and the active epidermis, and then generates a fingerprint image based on grayscale information. The latter generates a fingerprint image based on the depth of the active epidermal junction, selecting the average grayscale information of one or more cross-sections. However, existing methods are difficult to fully adapt to different skin thicknesses and the complex and varied ridge-valley lines at the active epidermal junction. They lack robustness, cannot adapt to different populations, and struggle to extract high-quality internal and external fingerprints. Summary of the Invention
[0005] To overcome the complex and variable situation of skin thickness and ridge lines at the junction of active epidermis, this invention provides a more robust and accurate OCT internal and external fingerprint extraction method. It uses the U-net semantic segmentation network in deep learning and a soft-label method with Gaussian weight descent to extract internal and external fingerprints.
[0006] To achieve the above objectives, the technical solution adopted by this invention is: a soft-label-based OCT internal and external fingerprint extraction method, comprising the following steps:
[0007] 1) Select B-scan images from the OCT volume data, manually annotate the stratum corneum and dermal regions of the images, and perform data preprocessing on the annotated images.
[0008] 2) Construct a U-net network model, set the hyperparameters, and train it using the training set obtained in step 1.
[0009] 3) Use the optimal network model obtained in step 2 to predict the B-scan image of the OCT fingerprint and obtain the segmentation result.
[0010] 4) Based on the segmentation results obtained in step 3, calculate the gray values of the fingerprint images according to the weights, and stitch them together in sequence. Finally, extract the external fingerprint from the stratum corneum region and the internal fingerprint from the dermis region.
[0011] Furthermore, step 1) includes the following steps:
[0012] (11) Manually annotate the prepared OCT fingerprint slice image with a resolution of 1800*500, marking the areas where the stratum corneum and dermis are located, and marking the position of the outline of the stratum corneum and the boundary of the active epidermis.
[0013] (12) Based on the positions of the stratum corneum contour and the active epidermal junction contour, a Gaussian filter is used to assign different weights to each pixel in the stratum corneum and dermal regions. The calculation formula is as follows:
[0014]
[0015] Where l(x,y) is the weight of pixel (x,y), y′(x) is the depth of the pixel with x-coordinate on the contour at the junction of the stratum corneum or active epidermis, z(x,y) is the depth of pixel (x,y), and the value of σ is related to the selected range.
[0016] (13) Data augmentation is performed on the labeled images, mainly by rotating them clockwise by 90 degrees, 180 degrees, 270 degrees and flipping them horizontally, so as to obtain more training data.
[0017] Furthermore, step 2) includes the following steps:
[0018] (21) Since the U-net network has a good effect on segmentation of biomedical images, the U-net network is selected as the basic network of this invention. The size of an OCT fingerprint B-scan image is 1800×500. The image needs to be cropped. The size of the cropped image is 256×256. The entire network model contains eleven parts.
[0019] Parts one through four each consist of two convolutional layers and one max pooling layer. For the i-th part, each convolutional layer first undergoes a 32*2... i The output size is 256*2 after three 3×3 convolution operations and ReLU activation. -i ×256*2 -i ×32*2 i Finally, the output size of the fourth part is 16×16×512.
[0020] The fifth part consists of a Line Projection module and two Transformer modules. After linear projection, Patch Embedding is performed to serialize the feature map and add position encoding. The output is a one-dimensional vector with position information. This vector is then fed into two Transformer encoders to extract global features and outputs a one-dimensional vector of the same size as the original.
[0021] The sixth part first reshapes the feature map to a size of 16×16×512, then a convolutional layer processes it through 1024 3×3 convolutional kernels and the ReLU activation function, outputting a feature of 16×16×1024.
[0022] Parts 7 through 10 each consist of one deconvolutional layer and two convolutional layers. For the i-th part, the feature map size output by each deconvolutional layer is (1024*2). -(i-6) )×(16*2 (i-6) )×(16*2 (i-6) The dimension here needs to be concatenated with the feature map of the left side, specifically the 11-ith part on the left. The result after concatenation is (1024*2). -(i-6) *2)×(16*2 (i-6) )×(16*2 (i-6) Finally, the feature map output in Part 10 has a size of 256×256×64.
[0023] The eleventh part is the last part, consisting of a convolutional layer and a softmax function. The convolutional layer consists of a 3×3 convolutional kernel. After passing through the softmax function, a probability prediction map is obtained, with an output size of 3×256×256. Three probability maps of size 256×256 are generated, representing the probability of a pixel being the stratum corneum, dermis, or background, respectively.
[0024] (22) Determine the hyperparameters of the network, set the batch size to 2, load 2 batches each time to train the network model, and iterate 100 times to obtain the trained network model.
[0025] (23) The parameter updates of each network layer are calculated using the mini-batch-SGD algorithm with a momentum term, where the value of the momentum term is 0.2.
[0026] Furthermore, step 3) includes the following steps:
[0027] (31) In order to conform to the input image size of the neural network, the vertical cross-sectional image (B-scan) of the input OCT fingerprint needs to be segmented to obtain an image with a size of 256×256.
[0028] (32) Input the segmented sub-images from the above steps into the trained neural network to obtain the corresponding probabilities of the stratum corneum, dermis, and background layer. The pixel prediction is the category with the higher probability. Then restore the segmented image to the original OCT image size.
[0029] Furthermore, step 4) includes the following steps:
[0030] (41) For the obtained stratum corneum region or dermal region, first multiply the gray value of each pixel in each region with its weight, and then accumulate the gray value information of the pixels in each column. The calculation formula is as follows:
[0031]
[0032]
[0033] Where f(x) is the grayscale information of the fingerprint pattern generated in a certain B-scan, a and b are the starting and ending depths of the stratum corneum region or the dermis region, l is the weight of the pixel (x,y), and I is the grayscale information of the pixel (x,y).
[0034] (42) The fingerprint pattern information obtained from each B-scan image in the 3D fingerprint data is spliced together in order to finally generate internal and external fingerprint images.
[0035] The working principle of this invention is as follows: By combining information from an OCT fingerprint image, two distinct effective regions can be segmented, termed the stratum corneum and the dermis. Information from either the stratum corneum or the entire dermis contributes to the generation of the external or internal fingerprint image. Therefore, this method utilizes soft-labeling to segment the image, and then assigns different weights to the pixels in the labeled regions using a Gaussian function with a kernel of 1. Based on the segmentation results, high-quality internal and external fingerprint images are extracted.
[0036] The advantages of this invention are: it can effectively utilize information from both the stratum corneum and the dermis, assign different weights to each pixel in the entire region, generate higher quality internal and external fingerprints, and has strong robustness. Attached Figure Description
[0037] Figure 1 This is a flowchart of the internal and external fingerprint extraction process in this invention.
[0038] Figure 2 This is a schematic diagram of the longitudinal cross-section of the internal and external fingerprints from an OCT scanner (B-Scan image).
[0039] Figure 3 This is a diagram of the U-net network structure in this invention.
[0040] Figure 4a It is the generated external fingerprint image. Figure 4b It is the generated internal fingerprint image. Detailed implementation method:
[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0042] like Figure 1 As shown, the OCT internal and external fingerprint extraction method based on soft-label includes the following steps:
[0043] 1) Select B-scan images from OCT volume data, such as... Figure 2 As shown, the stratum corneum and dermis regions of the image are manually labeled, and the labeled image is then preprocessed.
[0044] (11) Manually annotate the prepared OCT fingerprint slice image with a resolution of 1800*500, marking the areas where the stratum corneum and dermis are located.
[0045] (12) Based on the positions of the stratum corneum contour and the active epidermal junction contour, a Gaussian filter is used to assign different weights to each pixel in the stratum corneum and dermal regions. The calculation formula is as follows:
[0046]
[0047] Where l(x,y) is the weight of pixel (x,y), y′(x) is the depth of the pixel with x coordinate on the contour at the junction of the stratum corneum or active epidermis, z(x,y) is the depth of the pixel with coordinate (x,y), and the value of σ is related to the selected range.
[0048] 2) Construct a U-net network model, such as Figure 3 As shown, after setting the hyperparameters, the U-net neural network model is trained using the training set obtained in step 1.
[0049] (21) Since the U-net network has a good effect on segmentation of biomedical images, the U-net network is selected as the basic network of this invention. The size of an OCT fingerprint B-scan image is 1800×500. The image needs to be cropped. The size of the cropped image is 256×256. The entire network model contains eleven parts.
[0050] Parts one through four each consist of two convolutional layers and one max pooling layer. For the i-th part, each convolutional layer first undergoes a 32*2... i The output size is 256*2 after three 3×3 convolution operations and ReLU activation. -i ×256*2 -i ×32*2 i Finally, the output size of the fourth part is 16×16×512.
[0051] The fifth part consists of a Line Projection module and two Transformer modules. After linear projection, Patch Embedding is performed to serialize the feature map and add position encoding. The output is a one-dimensional vector with position information. This vector is then fed into two Transformer encoders to extract global features and outputs a one-dimensional vector of the same size as the original.
[0052] The sixth part first reshapes the feature map to a size of 16×16×512, then a convolutional layer processes it through 1024 3×3 convolutional kernels and the ReLU activation function, outputting a feature of 16×16×1024.
[0053] Parts 7 through 10 each consist of one deconvolutional layer and two convolutional layers. For the i-th part, the feature map size output by each deconvolutional layer is (1024*2). -(i-6) )×(16*2 (i-6) )×(16*2 (i-6)The dimension here needs to be concatenated with the feature map of the left side, specifically the 11-i part on the left. The result after concatenation is (1024*2). -(i-6) *2)×(16*2 (i-6) )×(16*2 (i-6) Finally, the feature map output in Part 10 has a size of 64×256×256.
[0054] The eleventh part is the last part, consisting of a convolutional layer and a softmax function. The convolutional layer consists of a 3×3 convolutional kernel. After passing through the softmax function, a probability prediction map with a size of 3×256×256 is obtained, generating three probability maps with a size of 256×256, representing the probability of a pixel being the stratum corneum, dermis, or background, respectively.
[0055] (22) Determine the hyperparameters of the network, set the batch size to 2, load 2 batches each time to train the network model, and iterate 100 times to obtain the trained network model.
[0056] (23) The parameter updates of each network layer are calculated using the mini-batch-SGD algorithm with a momentum term, where the value of the momentum term is 0.2.
[0057] 3) Use the optimal network model obtained in step 2 to predict the B-scan image of the OCT fingerprint and obtain the segmentation result.
[0058] (31) In order to conform to the input image size of the neural network, the vertical cross-sectional image (B-scan) of the input OCT fingerprint needs to be segmented to obtain an image with a size of 256×256.
[0059] (32) Input the segmented sub-images from the above steps into the trained neural network to obtain the corresponding probabilities of the stratum corneum, dermis, and background layer. The pixel prediction is the category with the higher probability. Then restore the segmented image to the original OCT image size.
[0060] 4) Based on the segmentation results obtained in step 3, calculate the gray values of the fingerprint images according to the weights, and stitch them together in sequence. Finally, extract the external fingerprint from the stratum corneum region and the internal fingerprint from the dermis region.
[0061] (41) For the obtained stratum corneum region or dermal region, first multiply the gray value of each pixel in each region with its weight, and then accumulate the gray value information of the pixels in each column. The calculation formula is as follows:
[0062]
[0063]
[0064] Where f(x) represents the grayscale information of the fingerprint pattern generated in a certain B-scan, a and b are the starting and ending depths of the stratum corneum region, l is the weight of pixel (x,y), and I is the grayscale information of pixel (x,y).
[0065] (42) The results obtained from each B-scan image in the 3D fingerprint data are stitched together in order to finally generate the internal and external fingerprint images, as shown in Figure 4. Figure 4a It is an external fingerprint image. Figure 4b It is an internal fingerprint image.
[0066] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.
Claims
1. A method for OCT internal and external fingerprint extraction based on soft-label, characterized in that, Includes the following steps: 1) Select B-scan images from the OCT volumetric data, manually label the stratum corneum and dermis regions of the images, and perform data preprocessing on the labeled images; specifically including: (11) Manually annotate the prepared OCT fingerprint slice image with a resolution of 1800*500, marking the areas where the stratum corneum and dermis are located, and marking the position of the outline of the stratum corneum and the boundary of the active epidermis. (12) Based on the position of the stratum corneum contour and the active epidermal junction contour, a Gaussian filter is used to assign different weights to each pixel in the stratum corneum and dermal regions. The calculation formula is as follows: in It is the weight of the pixel (x, y). This represents the depth of the pixel with a horizontal coordinate of x on the contour of the junction between the stratum corneum and the active epidermis. Let (x, y) be the depth of the pixel. The value depends on the range selected; (13) Perform data augmentation on the labeled images, including clockwise rotation of 90 degrees, 180 degrees, 270 degrees and horizontal flipping, to obtain more training data; 2) Construct a U-net network model, set the hyperparameters, and train it using the training set obtained in step 1); 3) Use the optimal network model obtained in step 2) to predict the B-scan image of the OCT fingerprint and obtain the segmentation result; 4) Based on the segmentation results obtained in step 3), calculate the grayscale values of the fingerprint image according to the weights, and stitch them together sequentially. Finally, extract the external fingerprint from the stratum corneum region and the internal fingerprint from the dermis region; including the following steps: (41) For the obtained stratum corneum region or dermal region, first multiply the gray value of each pixel in each region with its weight, and then accumulate the gray value information of the pixels in each column. The calculation formula is as follows: Where f(x) is the grayscale information of the fingerprint pattern generated in a certain B-scan, a and b are the starting and ending depths of the stratum corneum region or the dermis region, l is the weight of the pixel (x,y), and I is the grayscale information of the pixel (x,y). (42) The fingerprint pattern information obtained from each B-scan image in the 3D fingerprint data is spliced together in order to finally generate internal and external fingerprint images.
2. The OCT internal and external fingerprint extraction method based on soft-label according to claim 1, characterized in that, Step 2) includes the following steps: (21) The U-net network is selected as the basic network. The size of an OCT fingerprint B-scan image is 1800×500. The image is cropped and the size of the cropped image is 256×256. The entire network model contains eleven parts. Parts one through four each consist of two convolutional layers and one max pooling layer. For the i-th part, each convolutional layer first undergoes a 32*2... i The output size is 256*2 after three 3×3 convolution operations and ReLU activation. -i ×256*2 -i ×32*2 i Finally, the output size of the fourth part is 16×16×512. The fifth part consists of a Line Projection module and two Transformer modules. After linear projection, Patch Embedding is performed to serialize the feature map and add position encoding. The output is a one-dimensional vector with position information. This vector is then fed into two Transformer encoders to extract global features and outputs a one-dimensional vector of the same size as the original. The sixth part first reshapes the feature map to a size of 16×16×512, then a convolutional layer processes it through 1024 3×3 convolutional kernels and the ReLU activation function, outputting a feature of 16×16×1024. Parts 7 through 10 each consist of one deconvolutional layer and two convolutional layers. For the i-th part, the feature map size output by each deconvolutional layer is (1024*2). -(i-6) )×(16*2 (i-6) )×(16*2 (i-6) The dimension here needs to be concatenated with the feature map of the left side, specifically the 11-ith part on the left. The result after concatenation is (1024*2). -(i-6) *2)×(16*2 (i-6) )×(16*2 (i-6) Finally, the feature map output in Part 10 has a size of 256×256×64. The eleventh part is the last part, consisting of a convolutional layer and a softmax function. The convolutional layer consists of a 3×3 convolutional kernel. After passing through the softmax function, a probability prediction map is obtained, with an output size of 3×256×256. Three probability maps of size 256×256 are generated, representing the probability of a pixel being the stratum corneum, dermis, or background, respectively. (22) Determine the hyperparameters of the network, set the batch size to 2, load 2 batches each time to train the network model, and iterate 100 times to obtain the trained network model. (23) The parameter updates of each network layer are calculated using the mini-batch-SGD algorithm with a momentum term, where the value of the momentum term is 0.
2.
3. The OCT internal and external fingerprint extraction method based on soft-label according to claim 1 or 2, characterized in that, Step 3) includes the following steps: (31) In order to conform to the input image size of the neural network, the input OCT fingerprint longitudinal cross-sectional image B-scan needs to be segmented to obtain an image with a size of 256×256; (32) Input the segmented sub-images from the above steps into the trained neural network to obtain the corresponding probabilities of the stratum corneum, dermis, and background layer. The pixel prediction is the category with the higher probability of the category. Then restore the segmented image to the original OCT image size.