A new method based on minnet algorithm for matching crime scene images having a scale differences

Abstract

The present invention relates to a method for matching latent fingerprint images with different resolutions to sensor fingerprint images. With the method according to the invention, patches corresponding to the minutia detected on the crime scene fingerprint images are generated, feature extraction is performed for these patches, and the patch feature vectors are combined and compared with the sensor fingerprint image feature vectors. The patches are sized according to the most frequent value of the average of the neighbouring minutia distances for minutiae. The method according to the invention provides high matching performance with limited system resource utilization and high processing speed.

Description

[0001] A NEW METHOD BASED ON MINNET ALGORITHM FOR MATCHING CRIME SCENE IMAGES HAVING A SCALE DIFFERENCES

[0002] Technical Field

[0003] The present invention relates to a method for matching latent fingerprint images with different resolutions to sensor fingerprint images.

[0004] Prior Art

[0005] In order to match crime scene fingerprints with records, they need to be compared with sensor fingerprints. However, if crime scene fingerprint images have different spatial resolutions than sensor fingerprint images, the matching performance is degraded. Sensor fingerprint images are collected at standard sizes, such as 500x800 dpi or 900x950 dpi. In contrast, collecting crime scene fingerprint images requires a camera to capture the image and the image size varies depending on the distance between the camera and the fingerprint.

[0006] In the document numbered CN112733670A, a method for feature extraction (detection of minutiae in fingerprints) from crime scene fingerprint images is described. For this purpose, the images are first passed through a convolutional network for basic feature extraction. Then, for specific feature extraction, multiple atrous filters with different scales are applied in parallel and the outputs are combined. The merged data is again used to obtain X- directional translation, Y-directional translation, orientation and minutia score maps using convolutional networks.

[0007] In the document numbered CN105528591A, a method for detecting forged fingerprints is described. It also discusses how to obtain features by combining histograms obtained at different scales.

[0008] In the document numbered CN115690855 A, a method is described to determine the distance between the lines forming the fingerprint using images taken at different scales. For this purpose, a training set is created from images taken at different scales and a prediction model is trained. “MinNet: Minutia Patch Embedding Network for Automated Latent Fingerprint Recognition,” (H. I. Oztiirk, B. Selbes and Y. Artan, 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA, 2022, pp. 1626-1634, doi: 10.1109 / CVPRW56347.2022.00169.) describes a MinNet (minutia patch embedding network) model for matching fingerprint images. A 128x 128 patches are created around the minutiae detected on the images, and these are coded to preserve the line flow and minutiae distribution, generating 1 x256 feature vectors.

[0009] Objects and Brief Description of the Invention

[0010] It is an object of the present invention to provide a method for matching crime scene fingerprint images with sensor fingerprint images regardless of their spatial resolution.

[0011] It is a further object of the present invention to provide a method for achieving high matching performance.

[0012] It is a further object of the present invention to provide a method that allows for limited system resource utilization and high processing speed.

[0013] According to the invention, patches corresponding to the minutiae detected on crime scene fingerprint images are created, features are extracted for these patches, and the patch feature vectors are combined and compared with the sensor fingerprint image feature vectors. The patches are sized according to the most frequent value of the average of neighbouring minutia distances for minutiae.

[0014] The method of the invention allows for a reliable comparison of crime scene fingerprint images collected at different scales with sensor fingerprint images at standard scales by using patches of different sizes. The average distance between neighbouring minutiae is used as an indicator of image scale when determining patch sizes. In this way, patch sizes that are compatible with the expected scales of crime scene fingerprint images can be obtained.

[0015] Detailed Description of the Invention

[0016] The method for achieving the objects of the present invention is described in the attached figures.

[0017] Figure 1 Flowchart of a method according to the invention. Figure 2 Histogram showing the average neighbouring minutia distances for a set of crime scene fingerprint images.

[0018] Figure 3 Histogram showing the average neighbouring minutia distances for a set of sensor fingerprint images.

[0019] The method according to the invention, which enables crime scene fingerprint images to be matched with sensor fingerprint images regardless of their spatial resolution, essentially comprises the process steps

[0020] 101. extracting up to a predefined number of minutiae for each fingerprint image,

[0021] 102. calculating the average of the predefined number of neighbouring minutia distances for the minutiae in pixels,

[0022] 103. generating a plurality of first patches for each crime scene fingerprint image,

[0023] 104. generating a second patch for each sensor fingerprint image,

[0024] 105. generating feature vectors for each first patch and second patch,

[0025] 106. merging the relevant first patch feature vectors for each minutia of the crime scene fingerprint images,

[0026] 107. calculating a matching score by comparing the merged first patch feature vectors for selected minutiae of the crime scene fingerprint images with the second patch feature vectors for minutiae of the sensor fingerprint images to identify matching minutiae,

[0027] 108. comparing the matching score with a threshold value to determine whether there is a match.

[0028] Here, patches are portions of the fingerprint image that fall within frames whose size is defined in pixels and within a portion of the image that surrounds the minutia of interest. Patches are created such that their edges are greater than the average of the distances to neighbouring minutiae whose lengths are calculated. A second patch is created for each minutia in sensor fingerprint images and at least two first patches are created for each minutia in crime scene fingerprint images.

[0029] Fingerprint images and their patches are in the form of a pixel map.

[0030] Prior to minutia extraction, fingerprint images with different sizes can be resized to a predefined size suitable for comparison. Fingerprint images that are smaller than the predefined size can be resized by interpolation. After the first patches are created, the first patches are brought to the same size as the second patch, preferably by interpolation.

[0031] The merged first patch feature vectors can be downsampled to limit their size on the RAM and database and their processing time. Techniques such as downsampling can be used for this.

[0032] The merged first patch feature vectors and second patch feature vectors can be normalized for comparison.

[0033] Multiple scoring methods can be used in combination to determine that a match exists between the merged first patch feature vectors and the second patch feature vectors. For example, a first match score obtained using cosine distance and an LSA (local similarity assignment) algorithm can be used to determine that there is a match if the first match score is above the first threshold and no match if it is below a second threshold, while an average score obtained by averaging the first match score with a second match score can be used if it is between the first and second thresholds. It is determined that there is a match if this average score is above a third threshold and no match if it is below it. The first, second and third thresholds can be predefined.

[0034] The predefined number of neighbours used in the calculation of the mean of neighbouring minutia distances is between 3 and 7, preferably 5. This number can be determined by empirical methods.

[0035] In an exemplary embodiment of the invention, the 25 minutiae with the highest quality value in the crime scene fingerprint images are identified. If the total number of minutiae is less than 25, all minutiae values with a quality value above a predefined threshold are selected. Here, the quality value refers to the pixel quality in the local neighbourhood of the minutia. The number of minutiae selected was determined empirically based on performance. For each of these minutiae, first patches are produced in 96x96, 128x 128 and 172x 172 pixel sizes. In crime scene fingerprint images, the 25 minuscule images with the highest quality value are determined. For each of these minutiae images, second patches are produced in 128x 128 pixel sizes.

[0036] The first and second patches are fed into a feature extraction model, preferably a MinNet model, to obtain the relevant feature vectors. With the MinNet model, first patch feature vectors and second patch feature vectors are obtained in 1 ^256 dimensions and normalized to have L2 norms of 1. The first patch feature vectors and the second patch feature vectors are downsampled at a rate of 1 in 4 and reduced to 1 x64 (with L2 norms of 1 for each patch). Then, the first patch feature vectors are merged to obtain merged first patch feature vectors with dimensions of 1 x 192.

[0037] For each minutia pair obtained using the minutiae set of the crime scene fingerprint images and the minutiae set of the sensor fingerprint images, a similarity value is calculated by comparing the first patch feature vectors of the corresponding crime scene minutiae with the second patch feature vector of the corresponding sensor minutiae. The similarity value is preferably calculated using cosine distance. The similarity value can be calculated by selecting the largest of the cosine distances obtained for each pair of patches of the minuscule pair in question, or as a weighted sum of these cosine distances obtained using normalized weights.

[0038] A similarity matrix is created using the similarity values obtained for each minusha pair. The generated similarity matrix is fed to an LSA algorithm. Thus, a one-to-one sensor minusha and crime scene minusha matching is performed and a minusha score is obtained for each matched minusha pair. The highest 8 of the minusha scores are summed to obtain the first matching score. If the first matching score is greater than 0.25, it is determined that the related crime scene and sensor fingerprint images are matched, and if it is less than 0.18, it is determined that the images in question are not matched and the next image pair is started. If the first match score is between the mentioned thresholds, a second match score is calculated based on a minutia cylinder code (MCC). If the average of the first match score and the second match score is greater than 0.2, the images are matched, and if the average of the first match score and the second match score is less than 0.2, the images are not matched. The average of the first match score and the second match score can be weighted.

[0039] In a preferred embodiment of the invention, the method for determining the second patch size and the first patch sizes further comprises the process steps

[0040] 201. setting a predefined base patch size edge length as the second patch size edge length, 202. determining the most frequent values (mode) of the averages of neighboring minutiae distances calculated for each minutia in the crime scene fingerprint image set and the sensor fingerprint image set,

[0041] 203. calculating the ratio of the difference of the determined most frequent values to the most frequent values for sensor fingerprint images,

[0042] 204. calculating p(l — X) and p(l + X), where X is the calculated ratio and p is the predefined basic patch size edge length,

[0043] 205. using / ?, determining the values of the form 2"' / / / closest to p(l — X) and p(l + X), where m is an integer and n is a prime number, as the first patch size edge length. The predefined basic patch size edge length is preferably 128 pixels to ensure high performance with the MinNet model. A first patch size edge length of 2mn also ensures high performance with the MinNet model.

[0044] In an example according to this embodiment of the invention, the average neighbouring minutia distances are calculated using 5 neighbouring minutiae for each minutia. Accordingly, the most frequent value of the mean neighbouring minutia distances is found to be 47 for crime scene fingerprint images and 37 for sensor fingerprint images, and the ratio X is calculated as (47 — 37) / 37 = 0.27. According to this ratio, first patches with 96x96, 128x 128 and 160x 160 pixel sizes are generated.

Claims

CLAIMS1. A method for enabling crime scene fingerprint images to be matched with sensor fingerprint images regardless of their spatial resolution, characterized by comprising the process steps extracting up to a predefined number of minutiae for each fingerprint image, calculating the average of the predefined number of neighbouring minutia distances for the minutiae in pixels, generating a plurality of first patches for each crime scene fingerprint image, generating a second patch for each sensor fingerprint image, generating feature vectors for each first patch and second patch, merging the relevant first patch feature vectors for each minutia of the crime scene fingerprint images, calculating a matching score by comparing the merged first patch feature vectors for selected minutiae of the crime scene fingerprint images with the second patch feature vectors for minutiae of the sensor fingerprint images to identify matching minutiae, comparing the matching score with a threshold value to determine whether there is a match.

2. A method according to claim 1, characterized by having a predefined number of between 3 and 7 neighbours used in the calculation of the average of the neighbouring minutiae distances.

3. A method according to claim 2, characterized by having a predefined number of 5 neighbours used in the calculation of the average of the neighbouring minutiae distances.

4. A method according to claim 1, characterized by generating the patches such that the lengths of the edges are greater than the calculated average of the distances to the neighbouring minutiae lengths.

5. A method according to claim 1 , characterized by the first patches being resized to the same size as the second patches by interpolation.

6. A method according to claim 1, characterized by feeding the first patches and second patches into a feature extraction model, which is a MinNet model, to generate the first patch and second patch feature vectors.

7. A method according to claim 1 , wherein the fingerprint images having different sizes are set to a default size prior to minutia extraction, characterized in by comprising, for calculating the second patch size and the first patch sizes, the process steps setting a predefined base patch size edge length as the second patch size edge length, determining the most frequent values (mode) of the averages of neighboring minutiae distances calculated for each minutia in the crime scene fingerprint image set and the sensor fingerprint image set, calculating the ratio of the difference of the determined most frequent values to the most frequent values for sensor fingerprint images, calculating p(l — X) and p(l + X), where X is the calculated ratio and p is the predefined basic patch size edge length,- using p, determining the values of the form 2m^n closest to p(l — X) and p(l + X), where m is an integer and n is a prime number, as the first patch size edge length.

8. A method according to claim 1 , characterized by a predefined basic patch size having an edge length of 128 pixels.

Images