Morphed face detection system and method against identity theft

Abstract

The invention relates to a morphed face detection system and method against identity theft that allows detecting whether an image is a morphed image or a bona fide image when given, and also detecting by which method the morphed image was morphed. The invention particularly relates to a morphed face detection system and method against identity theft that uses machine learning-based detection methods to determine whether a face image is a morphed version, captures traces left by morphing methods on images, detects whether a given test image has undergone morphing, and comprises features such as co-occurrence matrix.

Description

[0001] MORPHED FACE DETECTION SYSTEM AND METHOD AGAINST IDENTITY THEFT

[0002] Technical Field of the Invention

[0003] The invention relates to a morphed face detection system and method against identity theft that allows detecting whether an image is a morphed image or a bona fide image when given, and also detecting by which method the morphed image was morphed (generated).

[0004] The invention particularly relates to a morphed face detection system and method against identity theft that uses machine learning-based detection methods to determine whether a face image is a morphed version, captures traces left by morphing methods on images, detects whether a given test image has undergone morphing, and comprises features such as co-occurrence matrix (CM).

[0005] State of the Art

[0006] Identity theft is one of the most common cybercrimes today, posing a serious threat to individuals’ security. This situation leads to individuals’ identity information being compromised by malicious individuals, resulting in serious financial losses. In this context, photo ID documents are frequently used to verify identity, but studies have shown that matching unrecognised face pairs is a difficult task even for trained identity checkers. This difficulty makes identity verification processes vulnerable to fraudsters.

[0007] Recently, fraudsters have developed new fraud methods using altered passport photos. Previously, fraudsters would produce fake passports or create fake IDs by changing the photo on a real passport. However, by means of the measures taken against fake passports, such attempts have now become easier to detect. For example, patterns that can be seen under artificial lighting are one of these measures. Fraudsters have found ways to obtain Real Passports by Fraud (RPF). Since these RPF passports are real documents issued by an authorised party, they can bypass counterfeiting measures. Morphed face detection systems offer an important protection mechanism in this context. These systems analyse biometric data to detect facial features and compare this data with records in the database. Supported by advanced technologies such as artificial intelligence and machine learning, these systems make face recognition processes faster and more reliable, thus accelerating the process of verifying users' identities.

[0008] Face recognition algorithms can detect users' faces with high accuracy using deep learning methods and can achieve effective results even in images taken from different angles and under variable lighting conditions. This feature increases the probability of detecting fraud attempts and provides a significant advantage against identity theft.

[0009] Morphed face detection systems can be integrated with multi-factor authentication methods in order to increase the level of security. This integration creates an additional layer of security in the process of verifying users' identities, significantly reducing fraud cases. Thus, it becomes an important security tool for financial institutions and government bodies.

[0010] However, the use of these systems also brings some ethical issues with it. Concerns about how users' private data is stored and used make the effectiveness of these systems questionable. For this reason, morphed face detection systems need to be operated transparently. Otherwise, users' trust is damaged and the usage rate of these systems decreases.

[0011] Developed morphed face detection systems are constantly updated and become more resistant to new generation identity theft methods. The evolution of technology contributes to the development of these systems and as a result, their ability to resist identity theft increases. Encryption and anonymisation methods are also used to ensure the security of users' data, and these approaches aim to protect user privacy. At the same time, the effectiveness of these systems is increased by minimising false positive and negative rates.

[0012] As a result, morphed face detection systems stand out as an effective defence tool against identity theft. The correct implementation of these systems increases the security of individuals and institutions and prevents fraud cases. However, these systems must be operated transparently in order not to damage users' trust. In this way, the reliability of identity verification processes is increased and users' faith in these systems is reinforced.

[0013] Although various suggestions and applications have been developed for morphed face detection against identity theft in the state of the art, these developments are not sufficient. Some applications belonging to the inventions developed for this purpose are given below.

[0014] The patent application numbered “CN117593779A” in the state of the art discloses a face-fused colluder face traceability method based on enhanced generation. The traceability model comprises a coding and decoding feature separation module and an enhancement generation module. Firstly, potential features and multi-resolution spatial features of a colluder identity are separated from a fused face image by means of a criminal image through a coding and decoding feature separation module; and then, an enhancement generation module is utilized to convert the potential features mapped into the intermediate potential codes into intermediate potential convolution features, and the multi-resolution spatial features modulate the intermediate potential convolution features through a channel segmentation spatial feature transform (CS-SFT) layer, so that fusion of the two groups of features is carried out, and enhancement of fusion of the face traceability of the face colluders is realized. According to the method, double-feature separation is adopted, and a StyleGAN2 generator is utilized to realize fusion of two groups of features so as to realize enhanced generation.

[0015] In the patent application numbered "WO2024025621A1" in the state of the art, a process and system for detecting face morphing by one-to-many face recognition is mentioned. The invention comprises the process steps of obtaining, by at least one processor of a computing device a probe image; performing a probe one-to-many search for the probe image among a gallery; producing a probe candidate list including a plurality of probe similarity scores; comparing the highest probe similarity scores of the probe candidate list to a morph decision boundary; and determining whether the probe image is a bona fide face image or a morph face image as a result of comparing the highest probe similarity scores to detect face morphing.

[0016] In the patent application numbered "CA3040971 A1" in the state of the art, it is mentioned that the embodiments, in at least one aspect, enable methods and systems to authenticate at least one face in at least one digital image using techniques to mitigate spoofing. The methods and systems trigger an image capture device to capture a sequence images of the user performing the sequence of one or more position requests based on the pitch and yaw movements. The methods and systems generate a series of face signatures for the sequence of images of the user performing the sequence of one or more position requests. The methods and systems compare the generated series of face signatures to stored face signatures corresponding to the requested sequence of the one or more position requests. As another example, pulse data, light data or eye tracking data can be used, or a combination thereof.

[0017] In the state of the art, there is a need for a system and method that provides the ability to obtain wavelet coefficients and prediction errors by applying wavelet transform on images, the creation of a new CM matrix using wavelet coefficients and prediction errors, the approach of creating CM from wavelet coefficients and prediction errors and using these matrices as features and the proposed features, the detection of whether the images have been morphed and with which method, and the determination of both morphed image detection and the specific morphing method with high success rates.

[0018] As a result, due to the negativities described above and the inadequacy of existing solutions on the subject, it has become necessary to make a development in the relevant technical field.

[0019] The Aim of the Invention

[0020] The most important aim of the invention is to determine whether any image is a morphed image or a bona fide image when given, and also to determine by which method the morphed image was generated.

[0021] Another aim of the invention is to create CM with wavelet coefficients and prediction errors and to provide high performance with the effectiveness of the proposed features by using it in the detection system.

[0022] Another aim of the invention is to provide information on how the coefficients of various colours, scales and orientations interact with each other. In this way, it represents how often certain wavelet coefficients and prediction error values occur together. Another aim of the invention is to analyse CMs and to determine the combinations of common and rare values. In this way, it helps in the detection of anomalies as it can indicate the existence of unusual or rare value combinations.

[0023] Another aim of the invention is to provide the classifier system designed with the proposed features to determine whether the given test image has undergone the morphing process and also to classify successfully with which morphing method it was obtained.

[0024] Description of Drawings

[0025] FIGURE-1 is the drawing that gives the flow chart of the method of morphed face detection against identity theft, which is the subject of the invention.

[0026] FIGURE-2 is the drawing showing the morphed face detection system against identity theft, which is the subject of the invention.

[0027] Reference Numbers

[0028] 1 : Computer or mobile device

[0029] 2: Interface

[0030] 3: Server

[0031] 100: Users uploading photos to a computer or mobile device.

[0032] 200: Computers or mobile devices saving photos to the server via the interface

[0033] 300: Determining the morphed face and the method by which it was morphed by creating and analysing CM

[0034] 310: Acquiring a three-channel colour image

[0035] 320: Calculating three level Wavelet Transform of each channel

[0036] 330: Calculating the sub-band coefficients H, V and D and the prediction errors H_E, V_E and D_E for each level 340: Quantising coefficients and prediction errors with four levels to keep the volume of the Co-Occurrence Matrix to be created small,

[0037] 341 : Dividing the given range of wavelet coefficients by four,

[0038] 342: Determining the midpoints of four ranges

[0039] 343: Reducing the wavelet coefficient to four values

[0040] 344: Completing the process of reducing the size of the matrix by quantisation 350: Constructing six-dimensional Co-occurrence Matrix using wavelet coefficients and prediction errors

[0041] 360: Training phase

[0042] 360.1 : Creating nine Co-occurrence Matrices by three-level wavelet transform of each channel of three-channel colour image.

[0043] 360.2: Converting each matrix into a vector

[0044] 360.3: Creating the feature vector corresponding to the image by adding nine vectors together sequentially

[0045] 360.4: Creating a new Training Matrix by removing all zero columns in the training matrix

[0046] 360.5: Designing special detectors (detection systems) with machine learning using training matrix

[0047] 370: Performing the testing phase

[0048] 370.1 : Creating the feature vector of the given test image

[0049] 370.1.1 : Calculating the feature vector of a colour image in the training phase 370.2: Rearranging the feature vector 370.2.1 : Removing the values in the indices corresponding to the column numbers recorded in the process of creating the feature vector corresponding to the image by adding nine vectors together sequentially

[0050] 370.3: Giving the feature vector as input to the detection system

[0051] 370.4: The detection system evaluating the image

[0052] 370.4.1 : The detection system deciding whether the image is a bona fide image or a morphed image

[0053] 370.4.1.1 : If it is a morphed image, determining the method by which it was morphed

[0054] 370.4.1.2: If it is not a morphed image, determining this fact.

[0055] 400: Displaying the detection result on the interface

[0056] Description of the Invention

[0057] The invention allows to detect whether an image is a morphed image or a bona fide image when given, and also to detect by which method the morphed image was generated.

[0058] The invention is, in general terms, a morphed face detection system and method against identity theft, comprising a system comprising a computer or mobile device (1 ), an interface (2) and a server (3), and a method that detects the morphed face and by which method it was generated.

[0059] The invention creates CM with wavelet coefficients and prediction errors, and provides high performance with the effectiveness of the proposed features by using this in the detection system.

[0060] The invention uses a multidimensional co-occurrence matrix (CM) to capture statistical relationships and patterns in multidimensional data. This matrix represents the frequency of co-occurrence of values in multiple dimensions. Each entry in the matrix indicates how many times a certain combination of values occurs together in the data. To create CMs, the values of wavelet coefficients and prediction errors in different colour channels, scales and orientations are used. This approach represents how often certain wavelet coefficients and prediction error values occur together. Thus, it provides information on how coefficients of various colours, scales and orientations interact with each other.

[0061] The invention determines common and rare value combinations using CMs. This information helps to detect the anomalies as it can indicate the existence of unusual or rare value combinations. The values of these matrices are used as features in the classification.

[0062] The computer or mobile device (1) is the element that saves the photos to the server via the interface (2).

[0063] The interface (2) is the element where the photos and the detection result are displayed.

[0064] The server (3) is the element where the photos are saved by the computer or mobile device and in which the detection system is located.

[0065] The morphed face detection method against identity theft comprises the process steps of:

[0066] • users uploading images to a computer or mobile device. (100),

[0067] • computers or mobile devices saving images to the server via the interface(200), • determining the morphed face and the method by which it was morphed by creating and analysing CM (300), and

[0068] • Displaying the detection result on the interface (400).

[0069] The morphed face detection method against identity theft consists of two phases. In the first phase, a Training Matrix is created by extracting feature vectors from each of the images in the T raining Set. Face detection systems with machine learning methods are designed using the Training Matrix. The second phase is the testing phase. In the testing phase, the feature vector of any given colour image is calculated. This feature vector is given as input to the designed detection system. The output of the detection system is a value indicating whether the image is bona fide or morphed, and if morphed, by which method it was morphed. A morphed face detection method against identity theft, wherein the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process steps of:

[0070] • acquiring a three-channel colour image (310),

[0071] • calculating three level Wavelet Transform of each channel (320),

[0072] • calculating the sub-band coefficients H, V and D and the prediction errors H_E, V_E and D_E for each level (330),

[0073] • quantising coefficients and prediction errors with four levels to keep the volume of the Co-Occurrence Matrix to be created small (340),

[0074] • constructing six-dimensional Co-occurrence Matrix using wavelet coefficients and prediction errors (350),

[0075] • the training phase (360), and

[0076] • performing the testing phase (370).

[0077] A morphed face detection method against identity theft, wherein the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process step of quantising coefficients and prediction errors with four levels to keep the volume of the Co-Occurrence Matrix to be created small (340), which in turn comprises the process steps of:

[0078] • dividing the given range of wavelet coefficients by four (341 ),

[0079] • determining the midpoints of four ranges (342),

[0080] • deducing the wavelet coefficient to four values (343), and

[0081] • completing the process of reducing the size of the matrix by quantisation (344). Let's assume that the wavelet coefficients take 80 different integer values between 1 and 80. Quantising the range of 1 to 80 to 4 levels means dividing this range by four and now the wavelet coefficients take only one of the 4 values. These four values are simply chosen as the midpoints of the four ranges. For example, if the value of the wavelet coefficient falls between 1-20, it is quantised to the value of 10, and if it falls between 41-60, it is quantised to the value of 50. In other words, the wavelet coefficient no longer takes one of the 80 different values, but only one of the 4 values. These values are the midpoints of the ranges, 10, 30, 50, 70. 1 -20-> 10, 21 -40-> 30, 41 -60-> 50, 61 -80-> 70. The reason we quantise is to reduce the volume of the matrix. If we did not quantise, the number of elements in the matrix would be 806= 262,144,000,000. When we quantise, 46= 4096. Here, 4 corresponds to the quantisation level and 6 corresponds to the size of the matrix.

[0082] Support Vector Machine is used as a machine learning method and special detectors (detection systems) are designed.

[0083] A morphed face detection method against identity theft, wherein the Training phase (360) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process steps of:

[0084] • creating nine Co-occurrence Matrices by three-level wavelet transform of each channel of three-channel colour image (360.1 ),

[0085] • converting each matrix into a vector (360.2),

[0086] • creating the feature vector corresponding to the image by concatenating nine vectors together sequentially (360.3),

[0087] • creating a new Training Matrix by removing all zero columns in the training matrix (360.4), and

[0088] designing machine learning special detectors (detection systems) using training matrix (360.5).

[0089] A morphed face detection method against identity theft, wherein performing the testing phase (370) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process steps of:

[0090] • creating the feature vector of the given test image (370.1 )

[0091] • rearranging the feature vector (370.2)

[0092] • giving the feature vector as input to the detection system (370.3), and

[0093] • the detection system evaluating the image (370.4).

[0094] A morphed face detection method against identity theft, wherein the process step of creating the feature vector of the given test image (370.1) of performing the testing phase (370) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process steps of:

[0095] • users uploading photos to a computer or mobile device (100),

[0096] • computers or mobile devices saving photos to the server via the interface(200), and

[0097] • determining the morphed face and the method by which it was morphed by creating and analysing CM (300).

[0098] A morphed face detection method against identity theft, wherein in the process step of rearranging the feature vector (370.2) of performing the testing phase (370) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300), the process of removing the values in the indices corresponding to the column numbers recorded in the process of creating the feature vector (360.3) corresponding to the image by concatenating nine vectors together sequentially (370.2.1) is performed.

[0099] A morphed face detection method against identity theft, wherein the process step of the detection system evaluating the image (370.4) of performing the testing phase (370) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process step of:

[0100] • the detection system deciding whether the image is a bona fide image or a morphed image (270.4.1).

[0101] A morphed face detection method against identity theft, wherein in the process step of the detection system deciding whether the image is a bona fide image or a morphed image (270.4.1 ) of the detection system evaluating the image (370.4) of performing the testing phase (370) of the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300), If it is a morphed image, determining the method by which it was morphed (370.4.1.1) and If it is not a morphed image, determining this fact (370.4.1.2) is performed. For the morphing process, a starting (source) and ending (target) image or object is selected; the key points of the source and target objects are matched; intermediate images are created between the source and target objects that have the properties of both; the created intermediate images are combined.

Claims

1. CLAIMS1. The morphed face detection system against identity theft, comprising:3.• Computer or mobile device (1) that saves photos to the server via interface (2),4.• Interface where photos and detection results are displayed (2), and • Server where photos are saved and detection system is located via computer or mobile device (3).

2. The morphed face detection method against identity theft, comprising the process steps of:6.• users uploading photos to a computer or mobile device (100),7.• computers or mobile devices saving photos to the server via the interface(200), and8.• determining the morphed face and the method by which it was morphed by creating CM and extracting features (300), and9.• displaying the detection result on the interface (400).

3. A morphed face detection method against identity theft according to Claim 2, wherein the process step of detecting the morphed face and the method by which it was morphed with the creation and analysis of CM (300) comprises the process steps of:11.• acquiring a three-channel colour image (310),12.• calculating three level Wavelet Transform of each channel (320),13.• calculating the sub-band coefficients H, V and D and the prediction errors H E, V_E and D E for each level (330),14.• quantising coefficients and prediction errors with four levels to keep the volume of the Co-Occurrence Matrix to be created small (340), • constructing six-dimensional Co-occurrence Matrix using wavelet coefficients and prediction errors (350),15.• the training phase (360), and16.• performing the testing phase (370).

4. A morphed face detection method against identity theft according to Claim 2 or Claim 3, wherein the process step of quantising coefficients and predictionerrors with four levels to keep the volume of the Co-Occurrence Matrix to be created small (340) comprising the process steps of:18.• dividing the given range of wavelet coefficients by four (341 ),19.• determining the midpoints of four ranges (342),20.• deducing the wavelet coefficient to four values (343), and21.• completing the process of reducing the size of the matrix by quantisation (344).

5. A morphed face detection method against identity theft according to Claim 2 or Claim 3, wherein the process step of Training phase (360) comprises the process steps of:23.• creating nine Co-occurrence Matrices by three-level wavelet transform of each channel of three-channel colour image (360.1),24.• converting each matrix into a vector (360.2),25.• creating the feature vector corresponding to the image by concatenating nine vectors together sequentially (360.3),26.• creating a new Training Matrix by removing all zero columns in the training matrix (360.4), and27.• designing special detectors (detection systems) machine learning (Support Vector Machine is used as a machine learning method) using training matrix (360.5).

6. A morphed face detection method against identity theft according to Claim 2 or Claim 3, wherein the process step performing the testing phase (370) comprises the process steps of:29.• creating the feature vector of the given test image (370.1 ),30.• rearranging the feature vector: removing the values in the indices corresponding to the column numbers recorded in the process of creating the feature vector corresponding to the image by concatenating nine vectors together sequentially31.• giving the feature vector as input to the detection system (370.3), and, • the detection system evaluating the image (370.4).

7. A morphed face detection method against identity theft according to Claims 2, 3 or 6, wherein the process step creating the feature vector of the given test image (370.1) comprises the process steps of:• users uploading photos to a computer or mobile device (100),33.• computers or mobile devices saving photos to the server via the interface(200), and34.• determining the morphed face and the method by which it was morphed by creating CM and extracting features (300).

8. A morphed face detection method against identity theft according to Claims 2, 3 or 6, wherein the process steps of rearranging the feature vector (370.2) comprises the process step of removing the values in the indices corresponding to the column numbers recorded in the process of creating the feature vector (360.3) corresponding to the image by concatenating nine vectors together sequentially (370.2.1).

9. A morphed face detection method against identity theft according to Claims 2, 3 or 6, wherein the process step the detection system evaluating the image (370.4) comprises the process step of:37.• the detection system deciding whether the image is a bona fide image or a morphed image (270.4.1).

10. A morphed face detection method against identity theft according to Claims 2, 3, 6 or 8, wherein the process the detection system deciding whether the image is a bona fide image or a morphed image (270.4.1 ) comprises the process steps of:39.If it is a morphed image, determining the method by which it was morphed (370.

4. 1.1), and40.If it is not a morphed image, determining this fact (370.4.1.2).

Images