System and method for a smart antifraud interaction system
The system uses device cameras and neural networks to authenticate users and secure messaging, addressing identity verification and onboarding issues, enhancing security and preventing fraud.
Patent Information
- Application Number
- PCT/GB2025/050687
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-04-22
- Filing Date
- 2025-03-31
- Publication Date
- 2025-10-30
AI Technical Summary
Existing antifraud systems are inadequate in verifying the identity of account holders, allowing fraudsters to impersonate users through advanced AI technologies, and require cumbersome onboarding processes for passbook users, with vulnerabilities in device security and reactive fraud detection.
A system utilizing built-in cameras on devices to extract and process digital data, employing neural networks for facial and text recognition, and integrating unique message IDs to authenticate users and secure messaging, ensuring only legitimate access to digital assets.
Enhances security by verifying the actual account holder's identity, simplifies onboarding for passbook users, and prevents unauthorized access, reducing fraud and improving overall system responsiveness.
Smart Images

Figure GB2025050687_30102025_PF_FP_ABST
Abstract
Description
[0001] SYSTEM AND METHOD FOR A SMART ANTIFRAUD INTERACTION
[0002] SYSTEM
[0003] BACKGROUND OF THE INVENTION
[0004] 1. Field of the Invention
[0005] The field of the invention relates to antifraud systems, and to related computer- implemented methods.
[0006] 2. Technical Background
[0007] Traditional methods of managing potential fraud on users of certain services, or any type of secure or financial transactions, in existing systems and methods are evolving at an ultra-fast pace from physical authentication and execution methods to so called digital authentication and remote execution through a server and an App (application) on a smartphone or through a portable computer or desktop computer.
[0008] The evolution of these services technology has accelerated the transformation of remote execution of transactions, thanks to the internet and its shift from in person towards digitization (but not limited to the financial sector and online gaming & gambling, online purchasing, insurance, and mortgage providers), which have helped to create the right framework for remote transactions from the comfort of the home. Consequently, the methods with which to commit fraud have also evolved, resulting in an unacceptable high number of people attempting to use the available online platforms for illicit purposes, such as fraud or money laundering.
[0009] To counteract this wave of fraud, money laundering or potential terrorist financing, a multitude of contingency methods have been developed to combat the illicit use of the prior art in online systems and methods currently widely used.
[0010] The current most advanced methods to combat anti-fraud and anti-money laundering or anti -terrorist financing, have added in their systems many methods to mitigate it partially, ranging from verifying patterns in transactions or checking destination country or destination person’s name against blacklists shared between & with multiple sources, and so forth, adding a strong focus on stopping potential fraudsters or money launderers resulting in unacceptable high levels of governments, companies and individuals being de-frauded by other dishonest individuals or organized crime.
[0011] In some extreme cases, illicit financial transactions by unsuspecting willing participants, for example, to fund terrorist activities or to provide their account credentials to organized gangs who then commit widespread fraud, are prevented. Even blocking such accounts, it represents a drop in the ocean, as fraudsters or money launderers can use many accounts created by genuine users and obtain free or paid access to these accounts in the name of third party willing (paid user to open accounts with their personal data and ID) or un-willing participants (identity fraud) for their illicit means of collecting money. A typical example of fraud in certain countries is the illegal sale of government or private health / benefits services, thus obtaining such health or government benefits illicitly.
[0012] In general, these fraudulent financial movements are made possible by certain participants who are willingly or un-willingly providing their account credentials or simply making themselves payments to criminals or gangs who organize such frauds, as fraudsters can be very good in convincing certain individuals of parting with their money or providing certain personal data that allows those fraudsters to then somehow impersonate that unsuspecting individual or access an individual account (be it his / her bank account, card details, online purchase account, etc.
[0013] That said, it is not sufficient to combat fraud and / or money laundering and / or terrorist financing with the existing prior art, which is why examples of our invention solve certain of these previous mentioned shortcomings.
[0014] Some of the shortcomings of the prior art are as follows:
[0015] 1. A very worrying shortcoming in the prior art occurs when a user or many users give their credentials (willingly or un-willingly) to an individual or an organized gang or illicit organization with the intent to commit fraud or to launder money or for terrorist financing purposes, and they execute transactions without proper user identification of the actual account holder person, which are not identified by any of the current known methods or systems. In the prior art users are identified by several complex means such as login credentials and Two-factor authentication (2FA), 3- F actor Authentication (3FA) and biometric-ID of the device user, which still leaves those users who have willingly or un-willingly provided these access credentials or simply been hacked vulnerable to fraudsters. Which is why examples of the present invention focus on authenticating the actual account holder person yet being robust against current artificial intelligence technologies capable of impersonating an individual’s face or voice.
[0016] 2. Yet another shortcoming, when using the prior art is the onboarding of a Passbook user, basically a user with a printed booklet with the user’s account details and full name and listing all the -in and -outgoing transactions of such account. This requires the user, typically older or vulnerable individuals, to go in person in-branch (at a bank building society, health provider etc) with the accompanying inconveniences of displacement and in some cases with branches being closed at an alarming rate or not available at a reasonable short distance it becomes a serious issue to those users.
[0017] Onboarding those users in-branch to a digital platform for remote access by the user, that have already an existing account with all their personal details and KYC (know your customer) info accessible digitally in-branch, is a stressful undertaking for most. Having those vulnerable users, who are typically older and with little knowledge of electronic devices, do a full onboarding from scratch (despite having all that info already available at the service provider) is an even higher threshold for those users with the current prior art and thus requires an easier / simpler, more user-friendly solution with the examples of the present invention.
[0018] 3. Yet another shortcoming of the prior art, is when a user receives a communication (for example in writing) that supposedly comes from one of their service providers. The user has no reliable way to verify that the communication is legitimate, and its content is truthful, so they receive said communications with suspicion, and may lose something important due to distrust, or worse still, they may follow the instructions of a fraudulent message with terrible consequences.
[0019] It is particularly serious in the event when the message is malicious and contains a link, which can compromise the security of the device or the user's data and his or her digital assets.
[0020] 4. Another shortcoming in the prior art, in the opposite case to that described above, when a service provider receives a communication from an alleged user, often a customer service request that may compromise the service provider’s sensitive user’s information or even their financial position. Usually, the service provider does not have a reliable way to verify the identity of the user making the request, and usually resorts to identification methods based on questions and answers about the user's personal data, which can be found or obtained very easily in many cases.
[0021] 5. Yet another shortcoming, is where the current prior art systems and methods are reactive in nature, that is, they work correctly by obtaining data from frauds that have already occurred and avoid strictly those only if auto detectable. This allows fraudsters a window of time from when they implement a new form of scam or fraud until they are detected and resolved by these systems where they can operate on a large scale.
[0022] 6. Another shortcoming of the prior art, is the concept in which every message is considered suspicious and must be examined, and in which a single error allows a fraudulent message to be passed to the recipient, contrary to what is described in the examples of this invention, where every message sent through the system is authenticated and identified and thus legitimate.
[0023] The prior art design concepts in theory allow legitimate messages to pass through and eliminates malicious ones, but as seen in the aforementioned shortcomings, there is a high probability of passing malicious messages to the recipient (service provider or end user). Introducing an error in the continuously changing new confirmed scams and their post event detection solutions as updates is also very common and has serious consequences, whether allowing a malicious message to pass through, or deleting a legitimate one due to a false positive, has very negative effects on the recipient, putting them at risk or depriving them of legitimate communications.
[0024] 7. Yet another shortcoming, is where the current state of the prior art bases its security on access to the receiving device. However, the device's security itself can be compromised in various ways, as it largely depends on the user's technical knowledge. In most cases, a simple 4-digit PIN guarantees access not only to the use of the device but also to its configuration and security, allowing a hypothetical scammer / fraudster to guarantee future access to it by adding their biometric data to the authorized ones, this action being invisible to the legitimate user of the device and allowing indefinite and undetectable access to the scammer / fraudster to that device, its configuration and all of its content, including applications that, as a security measure, use again only the same biometric access control of the already compromised device to enter into applications such as bank accounts etc.
[0025] Referring to the prior art as described herein, all prior art shortcomings described herein may have been overcome with the examples and claims of the present invention.
[0026] 3. Discussion of Related Art
[0027] Some fraud detection processes described in the prior art are grouped and / or concentrated in the construction of a fraud detector according to a set of defined parameters and / or rules, creating a firewall that deploys a multi-faceted approach to assess the legitimacy of text messages received by end-users. Where, in addition to examining the presence and nature of URLs in messages, the text content is analysed in detail for patterns and characteristics associated with fraudulent messages. This involves comparing the message content with pre-defined, company-specific templates, which allows for the identification of discrepancies or irregularities that could indicate potential fraud attempts. Another crucial aspect of the prior art process is the consultation of blacklists containing information on senders previously reported for fraudulent activity. These lists provide an up-to-date database of suspicious senders, which allows the firewall to identify and block text messages from untrusted sources. In addition, the reputation of the domain or sender of the message is assessed, using specific metrics and algorithms to determine the likelihood of the message being legitimate or fraudulent.
[0028] Taken together, this comprehensive, multi -criteria approach seemingly provides a robust defence against fraudulent activity in text messages. By combining advanced content analysis with real-time blacklist queries and sender reputation assessments, the firewall can identify and block fraudulent text messages, protecting end users against potential scams and fraud, yet fraud statistics as can be seen throughout many press articles, keep creeping up even with all the prior art measures. WO2023139367A1 relates to a system and method for an antifraud scoring system.
[0029] WO2022219351A1 relates to systems and methods for fraud prevention.
[0030] SUMMARY OF THE INVENTION
[0031] The invention relates to a system and method for operating fraud prevention in the reception of messages, wherein the verification of the identity of the sender of the message is mandatory to prevent fraudulent activities such as phishing, spoofing or smishing among others, and wherein the verification of the identity of the account holder person is done using the methods described herein. ‘Phishing’ may refer to sending fraudulent communications that appear to come from a legitimate and reputable source, usually through email and text messaging. ‘ Spoofing’ may refer to where someone or something forges the sender's information and pretends to be a legitimate source, business, colleague, or other trusted contact for the purpose of gaining access to personal information, acquiring money, spreading malware, or stealing data. ‘Smishing’ may refer to phishing delivered by text, to mobile phones and messaging applications.
[0032] Additionally, the system and method herein are adapted for operating onboarding printed passbook users to a digital platform, avoiding its fraudulent use or account access, wherein the passbook user does not have to repeat the existing valid know- your-customer (KYC) process, and wherein the verification of the identity of the account holder person is done using the methods described herein for identity fraud prevention.
[0033] In addition, this invention increases the security of existing accounts access to the digital assets by adapting the existing systems and methods to those described herein for this invention, avoiding access for potentially fraudulent individuals or organizations who are not the actual account holder, regardless if those fraudulent individuals are the actual device holder as is the case in most if not all fraud cases or unauthorised account access cases.
[0034] The present invention aims to resolve those issues encountered when individuals are targeted by a potential fraudster, such as (i) enhance the security of accessing individuals digital assets / accounts, (ii) allow the onboarding of passbook users with an existing account onto a digital platform without the need to displace themselves yet without having to repeat their know-your-customer (KYC) process and by protecting the security of access to the digital assets strictly to the actual individual account holder(s) of the account assets, (iii) and reduce the unintended willing or un-willing provisioning of any personal data or account(s) access to malicious individuals or organizations contacting the unsuspecting user(s).
[0035] The present invention aims to overcome the shortcomings of the prior art and to provide in some cases a manual, semi-automated or in some cases fully automated way of resolving the shortcomings of the prior art specifically in the prevention and detection of potential identity fraud when accessing a given account, passbook user onboarding to a digital platform, or in the event of personal details spoofing by potential fraudsters.
[0036] The systems and methods applicable to devices included herein for the invention apply to fixed- or wireless- devices, smartphones, tablets, laptops- or desktops- and any other different device that have an integrated camera and can download an application software or open a web-browser that includes the method(s) of examples of this invention, or has the application integrated by the manufacturer, and is adapted to communicate with the cloud hardware and cloud application software of the system of an example of this invention.
[0037] According to a first aspect of the invention, there is provided a system including at least one or more internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system including at least one or more internet enabled servers (SI to Sn), wherein: each respective internet enabled device (IEMD1 to lEMDn) includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with a server (SI) of the one or more internet enabled servers, each server of the one or more internet enabled servers including a respective second non-transitory storage medium (NTSM_1 to NTSM_n), and a respective second computer program product embodied on the respective second non-transitory storage medium, the respective second computer program product executable on the respective server to communicate with a first internet enabled device (IEMD1) of the one or more internet enabled devices, and wherein a respective first computer program product is executable on the first internet enabled device to use the built-in camera (BIDCI) to extract digital data from an image recorded and stored using the built-in camera (DC1) of the first internet enabled device, and to transmit such digital data through communication channels (CCH1 and CCH S1) between the first internet enabled device and the internet and between the internet and the server, wherein the server is configured to receive the digital data, and the respective second computer program product is executable on the server to process the received digital data, wherein the respective second computer program product is executable on the server to detect new data being received from the one or more internet enabled devices (IEMD1 to lEMDn), and to store the detected new data in the respective second non-transitory storage medium and to index the stored detected new data such that each such stored detected new data is associated to an originating user account of the first internet enabled device of the one or more internet enabled devices (IEMD1 to lEMDn), and wherein
[0038] (i) each server includes a respective third non-transitory storage medium including a respective third computer program product adapted to include a plurality of sub-program products, each respective sub-program product adapted to a respective different imaging processing task of different imaging processing tasks, the different imaging processing tasks including: (nl) separating any text or writing; (n2) separating any human face; (n3) separating shapes and objects; (n4) separating colours; the sub-program products respectively executable to process the new data (BIDCI to BIDCn) in parallel for each new input data, resulting in a respective plurality of outputs referred to as “encoding data 1 to n” (EDI to n), and
[0039] (ii) each server includes a respective fourth non-transitory storage medium including a respective fourth computer program product, wherein each such respective plurality of outputs “encoding data 1 to n” is used as input to a respective fourth computer program product executable on a respective server (SI to Sn) to compare all the “encoding data 1 to n” and to ignore all such outputs that are inconclusive, and to generate a new modified output of conclusive outputs, referred to as “auto modified encoding data” (AMED), and to store such “auto modified encoding data” in the respective second non-transitory storage medium indexed to a user account corresponding a respective internet enabled device (IEMD1 to lEMDn) of the system.
[0040] An advantage is efficient collection and storage of data which may be used to prevent fraud.
[0041] The system may be one wherein the digital data from the camera (BIDCI) of the first internet enabled device (IEMD1), comprises single images, or images extracted from video recorded by the camera, for example which include an image of a passbook or an account holder identification item (in example, live front camera video of account holder, or a bank card and card holder live front camera video / selfie) and which include a (e.g. selfie) image of an account holder of the passbook or the account holder identification item (in example, but not limited to, a bank account holder) and wherein,
[0042] (i) a sub-program product (n5) is executable on a server to process output data corresponding to the (nl) task that represents text or writing data, to identify an account holder’s full name (userl), and account number(s) and to compare the account holder’s full name, and the account number(s) against a users’ bank account database (DB1) hosted by or accessible from the server, and to request the KYC (know your customer) face identification digital data (FIDDD userl) of the account holder from the database and to store data received from the database in a non- transitory storage medium of the server (TNT SMI), and wherein,
[0043] (ii) a sub-program product (n6) is executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl), to identify a passbook holder or account holder (userl) and to compare the identified passbook holder or account holder against the face identification digital data (FIDDD userl) of the passbook account holder or account holder previously stored in the non-transitory storage medium of the server (TNT SMI) and wherein,
[0044] (iii) in the event that the human face (e.g. face biometric) data (FBD userl) and the face identification digital data (FIDDD userl) match above a threshold of X%, where X is a threshold predefined in the sub-program product (n6) executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl), then the sub-program product (n6) is executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl) to automatically allow the identified passbook holder or account holder full access, through the first internet enabled device (IEMD1) (for example a smartphone) including the first respective non- transitory storage medium (NTSM IEMDl) to the identified passbook holder’s account data or account holder account data (for example banking assets) stored on the server.
[0045] An advantage is improved security of access to a passbook holder’s account. An advantage is efficient collection and storage of data which may be used to prevent fraud.
[0046] The system may be one wherein for tasks nl, n2, n3 and n4, each sub-program product is adapted to a respective task, and the sub-program products are pretrained neural network program products (PTNNPP1 to PTNNPP4), wherein each such pretrained neural network program product is pretrained for a specific task or sub-task to generate respectively a digital representation of: (i) writing or text in a detected language of the passbook or of the account holder identification item (nl), (ii) the face of a person for a single or multiple frame (n2), (iii) the objects and shapes (n3), (iv) the predominant or colour pallet of each area or sub-area (n4), resulting in the plurality of outputs referred to as the “encoding data 1 to n” (EDI to n), and wherein the nl task image data is (i) auto rotated such that the detected writing or text is put always in the same order of top to bottom and left to right, regardless of if the nl task data was rotated in any angle between more than 0 degrees and less than 360 degrees; (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file size preprogramed in the third computer program product of the server (SI), and wherein the n2 task image data is (i) auto rotated such that the detected face is put always in the same arrangement of eyes up to nose down and mouth below nose, regardless of if the nl task data was rotated in any angle between more than 0 degrees and less than 360 degrees; (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file size preprogramed in the third computer program product of the server (SI), (iii) the third computer program product of the server verifies if one or both eyes are present and open.
[0047] An advantage is efficient collection and storage of data which may be used to prevent fraud.
[0048] According to a second aspect of the invention, there is provided a system including a plurality of internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system further including a plurality of internet enabled servers (SI to Sn), wherein, the plurality of internet enabled devices includes a first plurality of internet enabled devices associated with a first server of the plurality of internet enabled servers, and wherein the plurality of internet enabled devices includes a second plurality of internet enabled devices associated with a second server of the plurality of internet enabled servers, wherein each respective internet enabled device (IEMD1 to lEMDn) of the first plurality of internet enabled devices includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with the first server (SI) of the plurality of internet enabled servers, wherein a first internet enabled device of the first plurality of internet enabled devices includes a first respective computer program product executable on the first internet enabled device to communicate with the first server (SI); wherein each respective internet enabled device (IEMD2 to IEMD2n) of the second plurality of internet enabled devices includes a first respective non-transitory storage medium (NTSM_IEMD2 to NTSM_IEMD2n), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with the second server (S2) of the plurality of internet enabled servers, wherein a second internet enabled device of the second plurality of internet enabled devices includes a first respective computer program product executable on the second internet enabled device to communicate with the second server (S2), wherein the first server includes a third non-transitory storage medium (NTSM 1), and a third computer program product embodied on the third non-transitory storage medium, the third computer program product executable on the first server to communicate with first internet enabled device and to communicate with the second server, and wherein the second server includes a fourth non-transitory storage medium (NTSM 2), and a fourth computer program product embodied on fourth non- transitory storage medium, the fourth computer program product executable on the second server to communicate with the second internet enabled device and to communicate with the first server, and where in response to the third computer program product of the first server detecting new data being received from a device of the first plurality of internet enabled devices, or from the second server, such new input data is stored in the third non-transitory storage medium of the first server and is indexed such that each such new input data is associated to an originating user account of the device of the first plurality of internet enabled devices of the first server of the system, and where in response to the fourth computer program product of the second server detecting new data being received from a device of the second plurality of internet enabled devices of second server of the system or from the first server, such new input data is stored in the fourth non- transitory storage medium of the second server and is indexed such that each such new input data is associated to the originating user account of a device of the second plurality of devices of the second server of the system, and wherein;
[0049] (i) a respective fifth computer program product is embodied on a respective non- transitory storage medium of each server of the plurality of servers, and is adapted to include sub-program products, each sub-program product adapted to a plurality of different unique message ID (UMID1 to UMIDn) processing groups, including respectively: inserting a unique message ID (nl); extracting the inserted unique message ID(s) (n2), comparing the inserted unique message ID(s) with the extracted unique message ID (n3), and
[0050] (ii) wherein each such unique message ID (UMID1 to UMIDln) is inserted in a message at an originating message sending server (in an example the first server) before forwarding the message to a receiving messaging server (in an example the second server) before forwarding the message to an end destination internet enabled device (e.g. IEMD1 to lEMDln if S2 was origination server and SI receiving server, or IEMD2 to IEMD2n if SI was origination server and S2 receiving server) and to ignore or delete or replace all such messages that do not have a unique message ID before forwarding any such message to its end destination, such end destination corresponding to a respective user account of an internet enabled device of the system.
[0051] An advantage is improved control of security of messaging in a system including a plurality of internet enabled devices and including a plurality of internet enabled servers.
[0052] According to a third aspect of the invention, there is provided a system including at least one or more internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system including at least one or more internet enabled servers (SI to Sn), wherein: each respective internet enabled device (IEMD1 to lEMDn) includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with a server (SI) of the one or more internet enabled servers, each server of the one or more internet enabled servers including a respective second non-transitory storage medium (NTSM_1 to NTSM_n), and a respective second computer program product embodied on the respective second non-transitory storage medium, the respective second computer program product executable on the respective server to communicate with a first internet enabled device (IEMD1) of the one or more internet enabled devices, and wherein a respective first computer program product is executable on the first internet enabled device to use the built-in camera (BIDCI) to extract digital data from an image recorded and stored using the built-in camera (DC1) of the first internet enabled device, and to transmit such digital data through communication channels (CCH1 and CCH S1) between the first internet enabled device and the internet and between the internet and the server, wherein the server is configured to receive the digital data, and the respective second computer program product is executable on the server to process the received digital data, wherein the respective second computer program product is executable on the server to detect new data being received from the one or more internet enabled devices (IEMD1 to lEMDn), and to store the detected new data in the respective second non-transitory storage medium and to index the stored detected new data such that each such stored detected new data is associated to an originating user account of an internet enabled device of the one or more internet enabled devices (IEMD1 to lEMDn), and wherein
[0053] (i) each server includes a respective third non-transitory storage medium including a respective third computer program product adapted to include a plurality of sub-program products, each respective sub-program product adapted to a respective different ethnical imaging processing task of different ethnical imaging processing tasks, the different ethnical imaging processing tasks including:
[0054] (nl) separating each input image per its corresponding ethnical group; (n2) separating the human facial biometric of the input data from the rest of the image; (n3) separating shapes and objects from the rest of the image; (n4) separating colours in colour pallets of areas of the image; the sub-program products respectively executable to process new data (BIDCI to BIDCn) in parallel for each new input data, resulting in a respective plurality of outputs referred to as “encoding data 1 to n” (EDI to n), and
[0055] (ii) each server includes a respective fourth non-transitory storage medium including a respective fourth computer program product, wherein each such respective plurality of outputs “encoding data 1 to n” is used as input to a respective fourth computer program product executable on a respective server (SI to Sn) to compare all the “encoding data 1 to n” and to ignore all such outputs that are inconclusive, and to generate a new modified output of conclusive outputs, referred to as “auto modified encoding data” (AMED), and to store such “auto modified encoding data” in the respective second non-transitory storage medium indexed to a user account corresponding to a respective internet enabled device (IEMD1 to lEMDn) of the system, and
[0056] (iii) wherein the digital data from the camera (DC1) of the first internet enabled device corresponds to the a user frame(s) of the user showing his face and part of shoulders obtained through the built-in camera (BIDCI) when performing a login through the first computer program product (Application) of the first internet enabled device, such login being performed independently of the first internet enabled device access using biometrics or PIN number, meaning the login herein separates the plurality of internet devices access from the plurality of internet devices access (login) to a user account in a server of the plurality of servers, wherein the “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) at each login of an internet enabled device user into an account (in example a bank account) of the plurality of servers is verified and compared against the second non-transitory storage medium (NTSM Sl) facial biometrical data of the corresponding first internet enabled device login user account when it was created and optionally against any past successful login “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) of that same user account, therefore not allowing a login when a user can access (login) the first internet enabled device but the user does not pass the verification of the actual facial biometrics of the account holder of the account of the first server that the user wants to access (login) which are verified independently of the device login access for the plurality of internet enabled devices.
[0057] An advantage is improved security of access to a user’s account.
[0058] The system may be one wherein the respective third computer program products (CPP S1 to CPP Sn) of the respective servers, are pretrained neural network program products (PTNNPP1 to PTNNPP4), wherein each such pretrained neural network program product is pretrained for a specific task or sub-task to generate respectively a digital representation of (i) (nl) each input image per its corresponding ethnical group; (n2) the human facial biometric of the input data; (n3) the shapes and objects on the image; (n4) colours bundled in colour range of predefined areas of the image, resulting in a plurality of outputs referred to as “auto modified encoding data” (AMED1 to AEMDn); wherein the nl image data is (i) auto rotated such that the detected human face is put always in the same order of eyes up to nose down and mouth below nose, and such rotation left or right degrees is stored for use in next n2 to n4 rotation, (ii) the digital image file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file size preprogramed in the respective third computer program product, (iii) the respective third computer program product is executable to verify if one or both eyes are present and open, and wherein the respective third computer program product (i) is executable to rotate the n2 to n4 data in the same rotation direction and angle as was applied to the nl image data, and wherein a respective computer program product of a respective internet enabled devices is executed automatically when, (i) in one case, right after a successful biometrical device login by the respective internet enabled device and / or (ii) the biometrical device login / access by the respective internet enabled device is skipped; in both cases, for example when user of the respective internet enabled device opens a bank application the respective computer program product is executable to auto initiate a real time streaming from the respective internet enabled device to the first server, wherein at least one or more frames are extracted and processed by the pretrained neural network program products (PTNNPP1) into nl to n4 and compared against that a user account of the respective internet enabled device, in particular a user account stored date when that account was first opened, and against recent successful logins and allowed access to the first server user account of the respective internet enabled device, if the predefined threshold levels for nl to n4 are reached.
[0059] An advantage is improved security of access to a user’s account.
[0060] According to a fourth aspect of the invention, there is provided a computer- implemented method of processing image data recorded by a first internet enabled wireless mobile device including a display, the method including using a built-in camera facing in the same direction as the display, and using at least one internet enabled server device, wherein, the first internet enabled wireless mobile device includes a first non-transitory storage medium, and a first computer program product embodied on the first non-transitory storage medium, the first computer program product executing on the first internet enabled wireless mobile device to communicate with the internet enabled server device, and the internet enabled server device includes a second non-transitory storage medium, and a second computer program product embodied on the second non-transitory storage medium, the second computer program product executing on the internet enabled server device to communicate with at least the first internet enabled wireless mobile device and optionally with more internet enabled wireless mobile device(s), and wherein the first computer program product executes on the first internet enabled wireless mobile device to operate a data communication with the server, and, wherein, the first computer program product executes on the first internet enabled wireless mobile device, during a remote login by the wireless device userl into a remote account in the server, uses the built-in camera to take a live video stream of X frames per second (FPS) (wherein X is a predefined parameter, in example 30 FPS) of a subject person in near proximity in front of the built-in camera, and wherein the first computer program product restricts / prohibits the use of any external camera interfacing with the first mobile device, and streams the live video stream to the server wherein, in the event the server second computer program executing on the server detects data received from the first internet enabled wireless mobile device or from a second internet enabled wireless mobile device, the data is stored in the server non-transitory storage medium indexed such that each data is associated to an originating user account (in example userl) of the first internet enabled wireless mobile device user or of a second internet enabled wireless mobile device user (user2), for further processing and wherein, the second computer program product executes on the internet enabled server, during a remote login by the wireless device user into a remote account in the server, to extract the following frames from the live video stream,
[0061] - a first frame (F0) at time TO and then stores the first frame (F0) in the second non- transitory storage medium of userl account, and
[0062] - a second frame (Fl) which is a next first frame after TO where a face is detected in the second frame defined as time Tl, and the time between TO and T1 is stored together with Fl in the second non-transitory storage medium of userl account, and
[0063] - optionally a third frame (F2) Y sec after Tl (such parameter Y predefined in the second computer program product, in example Y=0.2 sec) defined as time T3, and wherein in the first frame (F0) and in the second frame (Fl) and in the third frame (F2) several predefined areas (al to an) in the background and several points on the body (not face, but for example the shoulders) of the user are checked against each other and, (i) if the match of the respective areas (FO al to an and Fl al to an and F2_al_to_an) is equal to or above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), it’s a pass to proceed to the next step and if less than the threshold Zl%, then the login is denied, and (ii) if the predominant colours are within the same colour range for the respective areas (FO al to an and Fl al to an and F2_al_to_an) with a match equal to or above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), it’s a pass to proceed to the next step and if less than the threshold Z2%, then the login is denied, and wherein the detected person’s face first frame (F0) and second frame (Fl) and third frame (F2) are checked against each other and, if the match of respective areas (FO facel and Fl_face2 and F2_face3) is equal to or above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), it’s a pass to proceed to the next step but if less than the threshold Z3%, then the next step is denied; where in the event of a pass, then the method continues to the final step (STn), wherein the final step includes extracting from the server database the accountl KYC (know your customer) picture PO accountl corresponding with the picture of the face of the user of accountl when the accountl was first opened, and optionally of more recent accountl user face picture(s) Pn accountl of the corresponding account of the first internet enabled wireless mobile device user and to compare pictures PO accountl and Pn accountl with the first frame (F0) and the second frame (Fl) and the third frame (F2), and if the match in each comparison is equal to or above a threshold Z4% (wherein Z4 is a predefined parameter, in example Z4=95 percent), it’s a pass to allow login but if a match in a comparison is less than the threshold Z4%, then the login is denied, wherein in the event of a pass, the method continued to an actual remote login, wherein the second computer program product on the internet enabled server executes to allow the userl a remote login (in example login on a remote banking service provider) by the wireless device user into the remote userl account in the server and informs and passes the userl account data to the first computer program product on the first internet enabled wireless mobile device.
[0064] An advantage is improved security of access to a user’s account.
[0065] The method may be one wherein, in the event of a pass, then before going to the final step STn, the first computer program product on the first internet enabled wireless mobile device stops the first live video streaming and restarts a second video streaming with different setting of the camera and repeats the complete process of the fourth aspect of the invention, and wherein the following additional steps are done, - (i) if the match of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming is above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), if equal or above Z1 it’s a pass to the next step and if less, then the login is denied, and (ii) if the predominant colours are within the same colour range of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are with a match above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), if equal or above Z2 it’s a pass to the next step and if less, then the login is denied, and, wherein the detected person’s face frame FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are checked against each other and, if the match of respective areas FO facel of first live video streaming and FO facel of second live video streaming and Fl_face2 of first live video streaming and Fl_face2 of second live video streaming and F2_face3 of first live video streaming and F2_face3 of second live video streaming is above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), if equal or above Z3 it’s a pass and it continues to the final step STn of the fourth aspect of the invention, but if less than Z3, then the final step is denied. An advantage is improved security of access to a user’s account.
[0066] The method may be one wherein, prior to the final step STn of the fourth aspect of the invention, the second computer program product when executed retrieves all the pictures of the frames of each live stream video and stores them in the second non- transitory storage medium of the server, and executes the following steps,
[0067] (i) detects in all those frame pictures with a face the presence of eyes (FPE l to n), and then
[0068] (ii) detects which of those FPE l to n have both pupils present, and then
[0069] (iii) compares in the order of first to last in time if the eyes close and open at least one time and if no eyes closing and opening sequence is detected the next step is denied, and if at least one eyes closing and opening sequence is found then it’s a pass (person present liveness) and proceeds to the final step STn.
[0070] An advantage is improved security of access to a user’s account.
[0071] The method may be one including using the first internet enabled wireless mobile device including an additional built-in camera2, on the reverse side of the device to the side with the display, the additional built-in camera2 facing away from the device user, and using at least one server, wherein, prior to the steps of the fourth aspect of the invention, the first computer program product when executing on the first internet enabled wireless mobile device, executes the following steps,
[0072] (i) the device takes a picture Pl with camera2 of a passbook or of an account holder identification item, wherein a passbook is defined as a standard bank or building society banking transactions printed booklet or any such other account holder identification item (such as a live video or image (e.g. selfie) of the account holder) of a client with an account in the server and wherein at least the account holder’s full name and bank account details are printed, and
[0073] (ii) such picture Pl is sent by the first computer program product executing on the first internet enabled wireless mobile device to the second computer program product and is stored in the second non- transitory storage medium for processing of the next steps, (ii) extracting the full account holder’s name and bank account details “accountl” from picture Pl, and
[0074] (iii) then proceed to the fourth aspect of the invention full method execution to allow or deny a remote login by the userl of the first internet enabled wireless mobile device into the corresponding account of the server corresponding to that passbook or account holder identification item.
[0075] An advantage is improved security of access to a user’s account corresponding to a passbook.
[0076] Aspects of the invention may be combined.
[0077] BRIEF DESCRIPTION OF THE FIGURES
[0078] Aspects of the invention will now be described, by way of example(s), with reference to the following Figures, in which:
[0079] Figure 1 represents a top-level diagram of a typical example of the present invention encompassing examples at a system level.
[0080] Figure 2 represent a typical diagram of the prior art on how the data of a Passbook is processed and updated, e.g. how the passbook is onboarded.
[0081] Figure 3 represents a diagram of a typical example of the present invention, wherein the in-branch prior art Passbook method, described in Figure 2 is shown as branch (Bl) and Passbook (PPB 2) in such branch, whereas the novelty comprises the adapted device IEMD 1 and the adapted server AS1.
[0082] Figure 4 represents a flow-chart of a typical example of the present invention for onboarding a passbook of a user, with an existing account, onto a digital account of a digital platform accessible from the user’s adapted mobile device through an Application (App) and through an adapted server.
[0083] Figure 5 represent a typical flow-chart of the prior art on how messages originating from a service provider to a user are filtered with previously known fraud patterns.
[0084] Figure 6 represents a flow-chart of a typical example of a messaging platform relating to the present invention on how incoming messages to a user-A that originated from a trusted service provider 1 are filtered with a unique message ID per message.
[0085] Figure 7 represents a different flow-chart of another typical example of a messaging platform relating to the present invention, on how messages in the opposite direction as in Figure 6, namely incoming messages to a service provider 1 that originated from a trusted user-A of that service provider 1 are filtered with a unique message ID per message introduced at user-A message origination.
[0086] Figure 8 represents a typical flow-chart of the prior art on how a user login is performed in wireless device and in some cases on PCs, where the device login biometrics are the same as the login into sensitive applications or user accounts through wireless device and in some cases on PCs. Figure 9 represents a typical flow-chart of an example of the present invention relating to the novelties for a user login into the sensitive applications or user accounts through wireless device and in some cases on PCs, wherein the device login biometrics are NOT the same, as the login into sensitive applications or user accounts through wireless device and in some cases on PCs.
[0087] Figures 10A, 10B and IOC represent a typical example of a method of detecting malicious interference within a user liveness and present recognition process during a remote account login.
[0088] Figure 11 represents a typical example of a method of detecting malicious interference within a facial- biometrical recognition process during a remote account login, unrelated to the device biometrical access which is outside of the scope of this invention.
[0089] Figure 12 represents a typical example of a method of detecting malicious interference within a liveness- and facial- recognition video process.
[0090] DETAILED DESCRIPTION
[0091] The invention relates to a system and method for operating fraud prevention in the reception of messages, wherein the verification of the identity of the sender of the message is mandatory to prevent fraudulent activities such as phishing, spoofing or smishing among others, and wherein the verification of the identity of the person using an application is identified. This system and method are applicable to mobile devices, such as smartphones, tablets or other wireless devices, as well as desktop computers, laptops or other computing devices, connected to a mobile network or to the internet in general, that allow the reception of messages or the use of applications relating to the methods of examples of this invention.
[0092] Such fixed or wireless devices, connected to a mobile network or the internet, (e.g. smart phones, tablets or any other electronic device) are adapted according to examples of this invention and include as part of the system a local application unit to process the incoming messages and a remote unit or server for receiving and processing the data sent by the application, an operator (e.g. a company) operating the system and method herein, and / or 3rd party potentially fraudulent -companies or - individuals sending potentially fraudulent message(s). Aspects of the disclosures relate to a system and methods adapted to obtaining information from a received message and displaying the body of said message in some text and / or image and / or audio compatible format.
[0093] Aspects of the disclosures also relate to a system and methods adapted to obtaining information from fixed or wireless devices, connected to a mobile network, allowing to identify the person using a given service through the internet, e.g. through a mobile application or a website and verifying if such person is authorised to use such service, wherein such person may be different than the person authorised to access and use the fixed or wireless device(s).
[0094] Various aspects of the disclosures relate to a system and method that enables verification of the legitimacy of a message using a set of methods including a proprietary method and / or system of generating proprietary verification codes or multi-factor authentication. This security method enables users‘ devices adapted with the methods of examples of this invention to provide valuable information to the company issuing a message, on the one hand, by adding extra methods or submethods for user identification, because the process prompts the user (person using a given online service) to identify him / herself on the relevant platform and / or validate the veracity of the message(s) received before accessing any -content with sensitive information, -an online account or -a service. And, on the other hand, it provides a control system on the acceptance verification of its own sent messages (meaning those sent by the service provider of the services the person is using on his device), while at the same time warning about potential fraudulent messages sent under the impersonating name of that company or service provider, notifying of possible corporate impersonation which typically results in fraud if the user interacts or engages with the content of such message.
[0095] In this method of verifying the authenticity of the message, the user receives a text and / or image message and / or email corresponding to the notification or communication that the legitimate issuing company wants to send to its user, in which each message includes a “Unique Message Identification Code” (UMID), generated by the legitimate company sending the message on whose platform the user is registered for a given service.
[0096] To confirm that the message received is legitimate, the user can enter the platform and / or application of the corresponding legitimate company where it has a registered account and enter the code included in the content of the message. The services company will then verify that the UMID entered by the user and the UMID registered as the user's unique message code of the recent message(s) to that user matches or not, and if the result is clear match, then the user will receive a confirmation that the message corresponding to that code is a legitimate message from such service provider.
[0097] An additional aspect related to this disclosure is the method of alerting the legitimate company (corporation) of a potential corporate impersonation through messages sent to any of its user(s), with the potential of fraud to its user(s), such as phishing, spoofing or smishing, among others. The validation method defined in an example of this invention provides necessary and sufficient information to trigger alarms and inform the legitimate service provider(s) and / or its user(s) about this potential fraudulent event and thus be able to undertake any such protective action(s) with the aim, firstly, of protecting users from possible fraud and secondly, of eradicating it and strengthening the communication channels between any legitimate company and its users.
[0098] In examples of this invention, unlike in the prior art, the user of a service has the ability to validate the legitimacy of any message for a service provider adapted with the methods and system herein, before taking any further action that would put his or her personal data or online possessions in any compromised position and avoid engaging with any non-verified messages that could lead to potentially paying nonlegitimate websites (that look the same or similar as the legitimate website) by clicking on a link and provide account access details by simply logging in and lose all assets of that account as a consequence of such scam or fraud initiated by a simple incoming message.
[0099] However, there is a basic aspect that the prior art methods do not offer, a confirmation of the veracity of the message between the receiving user and the legitimate service provider to such user, which is considered of vital importance, since it is one of the reasons why fraud in sending messages is still prevalent, and this is precisely one of the SHORTCOMINGS of the current state of the art, where basic security methods apply, for example, to messages received by the user asking him to send sensitive information after registering on a platform, notifications of updates of the platform where the user has registered, etc.
[0100] Another SHORTCOMING of the approach described in the prior art is the lack of verification of the identity of the person who is writing the message, in the case of an entity sending a message to a company. When a user sends a message to a company claiming to be a customer, the identity of the user needs to be verified. The current state of the art offers non reliable methods like asking questions to the user and comparing the answers with the previously collected data, like the ID doc number and so forth.
[0101] Another SHORTCOMING related to the approach described in the prior art lies in its reactive rather than preventive nature. The fraud detection method relies on the identification of previous fraud cases in order to anticipate and prevent future fraudulent activities. This means that the system needs to be exposed to instances of fraud before it can learn and adapt to recognize and block similar attempts in the future. However, this reactive nature provides a window of opportunity for fraudsters, as they can take advantage of the system's limitations to adjust and modify their tactics, thus avoiding detection. Essentially, by acting only after a fraud has occurred, the prior art does not provide proactive protection that can deter potential criminals, scammers or fraudsters and prevent potential losses or harm to individuals or endusers. This lack of preventative capability may expose individuals and companies and their custom ers / end-users to an increased risk of falling victim to fraudulent activity, highlighting the need to develop more proactive, preventive and adaptive approaches to combating message fraud.
[0102] Another SHORTCOMING of the approach described in the prior art is the lack of verification of the identity of the person who has read the private message. Since the user is not required to access any specific platform or to confirm his or her identity in any additional way other than standard login methods (e.g. user / password and 2FA or 3FA code verification), this means that, once the unique message code is entered correctly, access to the private message is granted. This means that, once the message verification code is entered correctly, access to the private message or user’s account is available without any additional verification of the individual's or user’s accountholder actual identity. As a result, there is no effective guarantee that the person reading the message is actually the authorized recipient. This lack of identity verification may facilitate unauthorized access to private messages or user’s accounts, compromising the confidentiality and security of the communication.
[0103] This last point is a serious additional SHORTCOMING that the prior art does not address satisfactory. Specifically, the identity of a user of a given service is verified as follows, when logging into a mobile application the user can enter the mobile phone by choice, through PEST verification, finger- print scan or face-ID. Once inside the mobile phone or smartphone, the sensitive applications of service providers (typically which contain sensitive information or assets, such as banking applications etc.) allow for the service provider application again to enter / login the application by a dedicated PEST, username / password with 2FA / 3FA verification codes to the user’s email or mobile phone Short Message Service (SMS), local finger-print scan, or Face-ID of the mobile device / Smartphone. This means that the actual account holder identity when using the device fingerprint or Face-ID is not verified by the service provider but only means that it’s a user authorized to use that device. When accessing service providers accounts through websites, the prior art is even more limited. Thus separating the user’s biometric verification of the authorized person allowed to use a given device and the user authorized to use the service provider services (such as accessing a bank account) is crucial to reduce scams, frauds and loss of assets of end-users of services provided by legitimate service providers and which an example of this invention addresses and resolves when adapting a device and system with the methods described herein. This means that once the identity of the account holder (end-user) of a service provider is verified, by the service provider adapted to the system and methods of examples of this invention, meaning even if a scammer, fraudster or malicious individual or impersonating company manages to get through the prior art shortcomings, they will still not be able to access any of user accounts to steal his assets (e.g. accessing a user’s bank account) even if they managed to get a hold of the actual user’s device (e.g. it was stolen) and manage to enter the device or obtained the user’s login credentials to login on a mobile application or website because an example of the invention herein will add an additional methods to verify not the device authorized user but the account holder authorized user with the method and system of an example of this invention.
[0104] Another SHORTCOMING of the prior art is the need to maintain regular updates to their methods thresholds or their past reported fraudsters databases, in which such prior arts give a weight that determines if the message is defined as fraud or not, resulting in the presence of false positives limiting its effectiveness, meaning a valid message can pass as a fraud message, removing or blocking legal messages to the end-user and also notifying the legitimate company or entity that a legal message has been detected as fraud, resulting in a penalty score to the legitimate company domain. This is caused because the message is not verified by the two parts, the end-user and the legitimate company or entity providing the service the user. This means that once the message code has been verified, manually by the end-user or automated by the system and methods of examples of this invention in the end-user device, this message has been verified by the two parts knowing if the message is fraud or not, this grants the ability to automate the blocking or erasing of the messages without any false-positive use case.
[0105] In examples of the methods and system of this invention, the veracity of the message can be understood better when explaining some of the several use cases. In a first use case, the user receives a message from the company where he / she is registered, notifying him / her of the requirement to update certain sensitive and personal data, after which the user, before providing any information, goes to the corresponding platform and validates the legitimacy of the message. Once the validation is confirmed, the user makes the data change securely and calmly.
[0106] In a second use case, the user receives a message from the company with which he / she is registered, notifying him / her of the requirement to update certain sensitive and personal data, after which the user, before providing any information, goes to the corresponding platform and validates the legitimacy of the message. However, the validation is not confirmed, which informs the user of possible fraud and alerts the company to the case of attempted fraud.
[0107] In one of the EXAMPLES of this invention, the various SHORTCOMINGS mentioned above are addressed by a proactive or preventive approach that (i) requires the user to validate the legitimacy of the message and / or (ii) when accessing his service providers accounts it requires the user to confirm the identity of the account holder (rather than the identity of the device holder as is the case in the prior art). This is achieved through (i) a multi-factor authentication process that may include entering a message identification code (UMID) included in the message body itself, and in on the other hand (ii) the service provider actual user’s identity verification methods. As a result, these EXAMPLES of this invention offer a more effective and proactive / preventive solution to address the challenges associated with verifying the legitimacy of messages and protecting users' security & privacy, specifically protecting user’s digital assets.
[0108] Another SHORTCOMING addressed by EXAMPLES of this invention is the uncertainty about the identity of the person receiving and reading the sent message. By proactively validating the identity of the user, before performing the validation of the private message, the need to wait for cases of fraud before taking corrective action is eliminated, as a higher level of security is ensured and the risk of compromising the confidentiality of the communication is significantly reduced.
[0109] Consequently, these EXAMPLES offer a more effective and proactive / preventive solution to address the challenges associated with verifying the legitimacy of private messages and protecting the user’s digital assets.
[0110] A different aspect of disclosures, also related to fraud, is found in cases where any type of transaction or payment involving the use of a physical card is made. In this context, prior art fraud detection solutions have been evolving, incorporating advanced data analysis techniques and artificial intelligence to identify suspicious patterns in the data obtained from transactions, one of the remaining challenges in the prior art being the verification that the person using the card is actually the legitimate owner of such payment card. Although various security measures have been implemented, such as security codes, digital signatures, or multi-factor authentication, these do not always guarantee sufficiently the actual identity of the person absolutely, meaning there is no sufficient certainty the person using the card is the actual card holder. That is why it is one of the SHORTCOMINGS improved in its preventive nature that is improved with the methods and system of examples of this invention.
[0111] A key EXAMPLE in this invention and one that directly addresses that previous SHORTCOMING focuses on the more accurate identification of the legitimate account holder associated with a physical card during a transaction. This process begins when the user presents the card to make the purchase or transaction. At this point, in the system process and method of an example of this invention, the identity verification method of an example of this invention is activated, where the merchant and / or person in charge of executing the transaction (in the event of a physical presence) must confirm manually the identity of the person who wants to make the purchase or transaction, by a visual facial comparison between the person present and the biometric and / or selfie record of the user at the card provider, who is the legitimate owner (card holder) of the card used for the transaction.
[0112] Similarly, an alternative EXAMPLE of this invention, resolving a SHORTCOMING of the prior art, is in the event of a user making an online payment (card not present) where the user enters the card data manually or stored on the device, in that case the payments processor or card provider as part of its standard checks will adapt its system and method to those of examples of this invention to add an automated verification of the real biometric identity and / or selfie between the person not present and the biometric and / or selfie record of the user at the card provider, who is the legitimate owner (card holder) of the card used for the transaction.
[0113] The previous two EXAMPLES provide an additional layer of security to the physical card transaction process by ensuring that only the legitimate account holder or card holder can complete the transaction whilst the authorisation in person or card present cases can be done manually by the merchant or automated and in card not present (online) automated.
[0114] In another EXAMPLE of the present invention, the use case wherein end-user of savings account, building society accounts or mortgage accounts (service providers) using the decades old used Passbook, which is in effect a printed booklet with the users account info and users full name as it appears on the digital account of the service provider except on paper. These users must go physically in branch in person with their Passbook to do any transactions and / or update the transactions by printing them in their Passbook, yet typically these are the more vulnerable and older individuals, which is a major SHORTCOMING of the prior art. By adapting the current systems and online access applications with the EXAMPLES of the present invention, those user with Passbook can be onboarded to the existing digital platforms of those service providers of those Passbooks, by the owner of the Passbook, present in branch or at the comfort of their home by themselves or by their authorised family member, thus giving them access to perform any transactions and get access to any up to date transactions history digitally online through the service provider -website or - mobile application. Such onboarding in examples of the present invention would be done by taking a picture of the Passbook account and user date page and auto extracting the data digitally and verifying it against the digital records of the user account at the service provider (e.g. a bank or building society, etc) followed by a selfie with a high accuracy threshold and compare it with the photo of the account holder or photo of the ID document stored in the existing digital account of that user at the service provider as the identity check of the account holder (and not the identity check of the device holder). In yet another EXAMPLE of the present invention the identity check will allow for multiple authorised account holders, in which case the identity of the photo taken during onboarding of the Passbook or future logins, will allow for the authorised people who previously also have passed a KYC of the existing digital account to be onboarded through the Passbook of the account holder by verifying such authorised person’s identity in the same way as the Passbook holder (regardless of the device holder authorised identity check).
[0115] In another EXAMPLE of the present invention, the identity verification process will perform an open-eyes check, to verify if the person showed in the video frames is live and conscious of the process. As shown for example in Figures 10A, 10B and 10C, both eyes are checked to detect if one or both are open, and the pupil(s) is / are present. In case of one or both closed, or missing pupils, we can determine that the picture could have been manipulated in any form with deep fake artificial intelligence-based methods / systems to commit identity fraud.
[0116] In yet another EXAMPLE of the present invention, once the video streaming is auto started to verify the identity of the user, only certain selected frames are used to extract the necessary info (human face biometrics). In order to avoid deep fake Al based systems interference, the example of the invention takes the first frame or a frame at time TO containing the face and another frame after a certain period at time Tl. As shown in the example of Figure 11, wherein certain areas (see 1.0 and 1.1 in Fig. 11) in the background and / or certain areas on the body of the user (see 1.2 in Fig. 11) of TO are compared with the frame of T1 (see 2.0, 2.1 and 2.2 in Fig. 11), to detect anomalies that can indicate the presence of a deep fake Al based system or moving the device camera between TO and T1 towards a printed image or an image / video on a screen of another device instead of the face of the device user, and to make this more difficult to control by malicious device users this process is restricted by this example of the invention to the front camera of a smartphone to facilitate well intentioned users to see their image on the screen in front of them.
[0117] In another EXAMPLE of the present invention, the video streaming frame requests are done multiple times in different video requests, at video request 1 frames at TO, T1 and T3 plus another video request 2 frames at T4, T5 and T6, as showed in the example of Figure 12. Every time the system requests a new video streaming in the fastest possible way limited strictly and only by the device hardware response time, keeping for the user the illusion that the video request is continuous, the user’s device restarts a 2nd video request with different settings, allowing this example of the invention to capture the first frame of video request 2 at T4 (first frame of every separate video stream with its own hardware settings) and a frame after a certain period T5 and optionally another at T6, comparing certain areas in the background and / or certain areas on the body of the user, comparing additionally between the corresponding frames of video request 1 and video request 2 (meaning comparing frame of TO with that of T4, frame of T1 with that of T5, and frame of T3 with that of T6) to detect anomalies that can indicate the presence of a deep fake Al based system. This is in addition to the comparison by predefined thresholds of the frames of the same video request, comparing frame of TO with the frame of T1 and T2 and comparing frame of T2 with frame of T3.
[0118] In yet another EXAMPLE of the present invention, the invention forces the user’s device to use the front camera during login, avoiding the use of the rear camera and thus reducing the possibilities of framing a different source of video (in example from a screen of another device).
[0119] In the same way, in another EXAMPLE of the present invention, the system disables the external devices, including but not limited to Universal Serial Bus (USB), Wi-Fi or Bluetooth devices that could act as an external video source, thus avoiding potential interface with the device internal camera video streaming output.
[0120] In yet another EXAMPLE of the present invention, the system captures at each device remote account login start the timestamp and metadata, including but not limited to uniquelD, modellD, localized name, manufacturer, Position, camera settings, etc., of the video source in different video streams in real time and compares them with the metadata of the video streams captured by the server, to find possible differences that can indicate the presence of deep fake Al based systems.
[0121] In the same way, in another EXAMPLE of the present invention, the system of this invention captures at each device remote account login start the timestamp and the device and its hardware and microprocessor metadata, including but not limited to interop index, processing software, subfile type, width, height, bits per sample, compression, photometric interpretation, thresholding, cell width, cell length, strip offsets, orientation, rows per strip, resolution, etc., of the photo source in different pictures in real time and compares them with the metadata of the pictures captured by the server, to find possible differences that can indicate the presence of deep fake Al based systems.
[0122] In yet another EXAMPLE of the present invention, the systems requests changes in the parameters of the device video source device, including but not limited to the settings / parameter values of the zoom, frame rate, focus, brightness, the embedded camera in case of multiple cameras, etc, wherein this example can detect discrepancies in those settings / parameter values between consecutive different video request to detect the potential presence of deep fake Al based systems or methods during the video requests frames processes under the system and methods of examples of this invention.
[0123] Figure 1 represents a diagram of a typical example of the present invention. An example of the present invention (e.g. Fig. 1) comprises internet enabled adapted wireless devices (IEMD1, IEPC2, IEBD3) with a built-in camera with, the cloud (Internet), and Server (SI to Sn) with computer programs (Si l to S 1 3 and Sn_l to Sn_3) stored in the non-transitory memory (NTSM_1 to NTSM_n), wherein each internet enabled wireless device with a built-in digital camera (BIDCI to BIDC3) respectively connected through (DC1 to DC3) is connected to the cloud, through the computer programs (CPP1 to CPP3) stored in the non-transitory memory (NTSM IEMDl, NTSM IEPC2 and NTSM IBD3) in communication with the internet with servers SI to Sn through communications channel (CCH1 to CCH3 and CCH_S1 to CCH Sn).
[0124] Figure 2 represents a typical diagram of the prior art on how the data of a Passbook is processed and updated. A user goes physically in-branch, and a branch attendant scans the user’s Passbook (PPB l), from a dedicated Passbook-scanner-printer (PBSP) reader interface (C3_l) and the PPB l connected through communications channel (C3_2) to an in-branch computer (IBC). The internet enabled in-branch computer IBC connected through a communications channel (CC1) through the internet with a server (SI) providing service to such branch to access that scanned user’s Passbook account details in the database (DB1) to confirm the Passbook user’s identity manually and visually and if confirmed, to extract the latest account data and print it through PBSP on the user’s Passbook (PPB l).
[0125] Figure 3 represents a diagram of a typical example of the present invention, wherein the in-branch prior art Passbook method elements, described in Figure 2, are shown as branch (Bl) and Passbook (PPB 2) in such branch. The server of Figure 1 is adapted with the examples of the present invention and shown as adapted server (AS1). The computer programs (Sn_l to Sn_3) are the computer programs of Figure 1 adapted to process digital data extracted from the adapted internet enabled mobile device (IEMD1). In this example the mobile device (IEMD1) user takes a photo, specifically restricted to be taken with the built-in back camera (BICI back), of the Passbook (PPB l) and is processed by computer program product (CPP1) of the non- transitory memory program (NTSM IEMDl) of mobile device IEMD1. This data is then transferred through a wireless communications channel (Cl) through the internet to server (AS1) for data extraction and processing, if the data corresponds to an existing account the server (AS1) instructs by response to NTSM IEMDl to proceed with the next step for digital onboarding of Passbook user (Userl) by restricting the next step to the use of mobile device (IEMD1) front camera (BIC2_front) and record a live stream video from IEMD1 to server AS1 for further analysis and identify verification if user Userl digital data corresponds to the KYC (know your customer data) in AS1 matching the Passbook data of PPB l sent in the prior step to server AS1. If it’s a match, the server AS1 will respond to mobile device IEMD1 with instant access to the passbook holder’s full account information and transactions history held in the user’s database of AS1. This last step ends the digital onboarding of an existing user with an account in adapted server AS1, onboarded through his physical Passbook using an adapted mobile device IEMD1.
[0126] Figure 4 represents a flow-chart of a typical example of the present invention for onboarding a Passbook user, with an existing account (in example at a Bank, building society etc), onto a digital platform accessible from the user’s adapted mobile device through an Application (App).
[0127] The in-branch prior art Passbook method is shown as flow-chart step S1PA where the user would go in branch and follow the process as described in detail in Figure 2.
[0128] Whereas step SI in the flow-chart is the first step of the example herein (e.g. Fig. 4), where the Passbook user wireless device is adapted with the computer program product of this flow chart (to include step SI and S2) and stored in the non-transitory memory of the Passbook user wireless device. In this step, the user takes a photo of the Passbook page where his account details and his account holder name are shown, restricted by the computer program product to be taken strictly by the wireless device front camera such that the user can see on his screen the areas it is taking a photo from. This photo is then sent to the adapted, Passbook service provider, server with the computer program product of this flow chart (to include step S2 to S12) and stored in the non-transitory memory of the Passbook service provider server. In step S2 the photo of the previous mentioned Passbook page is received in digital form and processed, wherein the photo is checked if it’s not in the correct position and if not, then its auto rotated, checked if the file size is more than the maximum allowed and if so then it's auto reduced in file size, then it is checked for the content integrity and if it passes those checks, then the OCR (optical character recognition) data is extracted AS STEP S3. In step S4 the OCR data is used to extract from the KYC (know your customer) database the user selfie photo or photo from the user ID that corresponds with the account of the OCR data, and such data is stored temporarily as S5 pending receipt of the data of S8.
[0129] The process continues with step S6 where the user shows his face on screen in the marked area, restricted by the computer program product to be a video live streaming taken strictly by the wireless device front camera such that the user can see himself on his screen during the short period of this step S6. This video recording is streamed live to the adapted Passbook service provider. In step S7 the one or more photos are extracted from frames of the live steam and represent the Selfie photo of step S6 and is further processed, wherein the photo is checked if it is not in the correct position and if not, then its auto rotated in the correct position, checked if the file size is more than the maximum allowed and if so then it is auto reduced in file size, then it is checked for the content integrity and if it passes those checks, then the photo is checked if one or both eyes are detected and open and if so then checks are performed to verify for liveness to avoid static live stream of a photo from another screen or a paper photo to avoid identity fraud. Any fail in any of the S7 verifications are rejected and sent back to step S6, and if they all pass the Selfie photo will be presented as S8. In the next step S9 both photos of S8 and S5 are compared if their data points match with a high threshold (this threshold level is programmable depending on how high the level is required by the Passbook service provider). If below the programmed threshold, then it is rejected and the process ends as step SI 0.2 if the number of attempts allowed = X in the event of X = 1, whereas if X = 2 it’s sent back to step S6 on the first attempt and on the second attempt to SI. Assuming it passes the threshold in step S9 it continues to step SI 0.1, and the data of that account is extracted from the account database (DB2) in step Si l and sent back to the wireless device that started the process in step S12.
[0130] Figure 5 represents a typical flow-chart of the prior art on how messages are filtered with past fraud patterns or potentially harmful content to users.
[0131] Step SI is the start and triggering step S2 wherein a service provider 1 wants to send a message to a user-A and creates its content. In a further step S3 the service provider 1 sends that message to user-A. The message is then received by the service provider-A of user-A in step S4. User-A is then notified of the message received in step S5. At this point the message received by user-A in Step S6 it verifies if an anti-scam program is available and if NOT then it proceeds to step S10 allowing user-A to access to message but if YES then it proceeds to step S7 and the anti-scam program scans for past pre-programmed known scam or potentially fraudulent content. In a further step S8 any collision / match with a previously known scam means the message is removed from the in-tray.
[0132] Figure 6 represents a flow-chart of a typical example of the present invention on how incoming messages are filtered with a unique message ID per message to avoid scams, fraud, or potentially harmful content to users as a preventive measure rather than as an after the fact filtering based on past patterns or past known scam content.
[0133] In the example of the present invention (e.g. Fig. 6), step SI is the start and triggering step S2 wherein a service provider 1 wants to send a message to a user-A and creates its content. In contrast to the prior art of Figure 5, in this example of the present invention, instead of proceeding to step S3 it proceeds to steps S2_l to S2_4 before proceeding to step S3. In step S2_l the service provider 1 requests a unique message ID (UMID) from the anti-scam messaging platform S2_2 which is stored in the unique message ID database DB-M in step S2_3 to avoid the same UMID to be used in other messages. The service provider 1 then inserts the UMID within the message content or metadata in step S2_4 and proceeds with steps S3 to S5. Once the user-A device receives the message, the rest of the steps are also novel versus the prior art. In a further step once the message is received by the device of user-A, step S6 then extracts the UMID from the message content or metadata and proceeds to step S7 where it requests to the anti-scam messaging platform of step S2_2 for the UMID corresponding to that specific message content extracted from database DB-M in step S2_3. At this point both UMID received from DB-M and the UMID extracted from the message in step S7 are verified for its validity in step S8, and if valid YES then the User-A is given access to such message. If, however in Step S8 the validity is a NO, then it proceeds to step S10 where the original message is deleted or replaced by a non-harmful message and proceeds to step Si l wherein the user-A is alerted of a deleted or replaced message thus avoiding User-A to be scammed or defrauded by incoming not authenticated messages with the system and methods of the examples of the present invention.
[0134] Figure 7 represents a different flow-chart of another typical example of the present invention on how incoming messages from potential users to service providers (the opposite of Figure 6 where the messages filtered were sent from a service provider to its users) are filtered with a unique message ID per message to avoid scams, fraud, or potentially harmful content to companies as a preventive measure rather than as an after the fact filtering based on past patterns or past known scam content.
[0135] In an example of the present invention (e.g. Fig. 7), step SI is the start and triggering step S2 wherein a User-A wants to send a message to a service provider 1 and creates its content. In contrast to the prior art of Figure 5, in this example of the present invention, instead of proceeding to step S3 it proceeds to steps S2_l to S2_4 before proceeding to step S3. In step S2_l the user-A requests a unique message ID (UMID) from the anti-scam messaging platform in step S2_2 which is stored in the unique message ID database DB-M in step S2_3 to avoid the same UMID to be used in other messages. The user-A then inserts the UMID within the message content or auto inserted in the message metadata in step S2_4 and proceeds with steps S3 to S5. Once a company device of service provider 1 receives the message, the rest of the steps are also novel versus the prior art. In a further step once the message is received by a device of the service provider 1, step S6 then extracts the UMID from the message content or metadata and proceeds to step S7 where it requests to the anti-scam messaging platform of step S2_2 for the UMID corresponding to that specific message content extracted from database DB-M of step S2_3. At this point both UMID received from DB-M and the UMID extracted from the message in step S7 are compared and verified for its validity in step S8, and if valid YES then the service provider 1 device person is given access to such message. If, however in step S8 the validity is a NO, then it proceeds to step S10 where the original message is deleted or replaced by a non-harmful message and proceeds to step Si l wherein the service provider 1 authorised person is alerted of a deleted or replaced message thus avoiding service provider 1 to be scammed or defrauded by incoming not authenticated messages with the system and methods of the examples of the present invention. Figure 8 represents a typical flow-chart of the prior art on how a user login is performed in wireless device and in some cases on PCs.
[0136] Step SI is the start by a user with an interaction with a device seeking access to that device, triggering the step S2 wherein the operating system of the device performs the biometric identification (e.g. fingerprint or facial identification) of the user with a pass match requirement of threshold = X. If the threshold in step S3 is not passed (NO) it reverts to step S2 until the limit of attempts = Y is reached and after that if still not passed it reverts for a manual login with PIN and ignoring the biometrics of the user. This last is a massive shortcoming of the prior art as access to a device can be given even if the biometrics fail by simply guessing the PIN.
[0137] Assuming the step S3 threshold is passed (YES) the user gets access to the content of the device account and when a user wishes to access a specific application (App), then when opening the App it proceeds to steps S4 and S5, which is in fact exactly a duplication or a repeat of steps S2 and S3, meaning it performs all the same functions except that when step S6 threshold is passed (YES) it will allow access to all the digital assets of the App account holder (user-A) which may not be the same person who initiated the process in step SI. Meaning if the App accessed is a banking app, that would mean that a different person than the actual bank account holder could access his banking app and commit identity -fraud and / or money laundering, this is typical with so called “mules” where a person (mule) pays a legitimate 3rd party person to buy access credentials to a bank account or pays a 3rd party person to use his personal data and ID documents or obtains illicitly his personal data and ID document photo and creates one or many bank accounts on his own device (mule). In this way the ‘mule” has access to his device, through the prior art, but also has access to those bank accounts on that same device allowing him to commit freely fraud to other unsuspecting honest people by convincing them to transferring money, for so called fraudulent and illicit services, to these bank accounts that he has access to (which do not use the mule personal details nor his ID document). This last mentioned shortcoming of the prior art is allowing still way too many cases of identity fraud, where access to sensitive applications of service providers who do in fact have the personal data and the KYC (know your customer) and ID documents of their users (e.g. bank, building societies, insurance companies, government services for benefits, etc) are provided access to the assets (mainly money / cards etc) without verifying the identity of the account holder but rather verifying not the identity but the biometrics of a non KYC user of a device.
[0138] Figure 9 represents a typical flow-chart of an example of the present invention on the novelties for a user login into the sensitive applications or user accounts through wireless device and in some cases on PCs.
[0139] In an example of the present invention (e.g. Fig. 9), steps SI to S5 are the same as the prior art. In contrast to the prior art of Figure 8, in this example of the present invention, instead of proceeding to step S6 it proceeds from step S5 to step S7 to Si l before imitating step S6. After step S6 is passed (YES) a live video streaming is started from the APP-1 device to the APP-1 service provider server, strictly from the front camera of the device and for a limited time (T1 to T2), where steps SI to S7 are performed in the adapted device. The adapted server of the APP-1 service provider performs the steps S8 to Si l. Continuing from step S7 to step S8 where one or more frames of the live video stream are extracted as photos and requests from the “Accounts KYC Database” (DB1), as step S9, the selfie photo and / or the ID document photo of the corresponding actual account holder of the account being attempted to be accessed through APP-1 and returns such digital information back to step S8, which then passes on the data (selfie / ID photo AND the frame photo) to SI 1 to verify and authenticate if the user attempting the APP-1 login is in fact the same biometrics as the account holder of the APP-1 account holder as stored at the APP-1 service provider. If checks pass (YES) then it triggers step 6 to allow access to all functions and assets of APP-1 user-A account holder which is accessed from S10 to SI 1, wherein S10 service provider holds all the user assets of user-A of APP-1.
[0140] In a different example of the present invention, S4 and S5 are bypassed whereby S3 when a pass (YES) continues to S7 and in this case the output YES of S8 would go to the input of S2.
[0141] In yet another different example of the present invention, S4 and S5 are bypassed whereby S3 when a pass (YES) continues to S7 and in this case the output YES of S8 would lead to the automatic blocking of the access of User-A account at the service provider of APP-1 and S10 would disconnect from S6 thus prohibiting access to any assets or functions of APP-1 User-A.
[0142] These examples resolve the shortcomings mentioned with respect to the prior art Figure 8 as well as those other shortcomings mentioned in any of the texts herein.
[0143] Figures 10A, 10B, 10C represent a typical example of eyes checking process as part of a facial recognition process. When a face is detected, the eyes are inspected to determine if both eyes are closed, both are open or one eye is open and the other one is closed. This is done by extracting different frames at different time frames of a live video recording restricted in this example to the front camera of a device (same side as the device screen). In case of detecting open eye(s), the presence of pupil(s) is determined, which is a mandatory detection presence when eyes are determined as open and present, or when one is present; and rejected when one or both are closed or open but no pupil present.
[0144] In a typical example of the present invention, only if both eyes are open and both pupils present is this considered as successful liveness pass considering the person present.
[0145] Figure 11 represents a typical example of a video streaming verification process, checking for malicious interference on frames sequences.
[0146] In a typical example of this invention (e.g. Fig. 11), two or more frames are selected and analysed, taking always a first frame where a user face is detected (time stamped as TO), and, at least, another frame after this first frame (time stamped as Tl), giving time enough between both frames to let a potential malicious present video alteration system or malicious user camera movement towards a 3rd party screen modify the video content.
[0147] In the frame TO several points or areas in the background are checked (1.0 and 1.1), and several points in the body (not face, but for example the shoulders) of the user as well (1.2). This last allows to verify if the areas (e.g. background areas or clothes areas) of frame TO and those of Tl match above a predefined threshold X or if the predominant colours are within the same colour range. Figure 12 represents a typical example of a video login session with the user, comprising of one or multiple different video requests with same or different camera settings.
[0148] In an example (e.g. Fig. 12), the system requests a video session 1 to the user’s device built-in camera. If the user’s device contains a deep fake Al based system (3.2) or the user moves and points the device camera away from him or herself to modify the user’s device video stream output and show an alternative reality, this will be detected by an example of the present invention to avoid identity fraud, specifically during a remote login to a user account (e.g. login to a bank account through a banking application on a device or through a website on a computer).
[0149] In this example of the invention (e.g. Fig. 12), the system makes a video request 1 to the user’s device. The deep fake Al based system would take some frames to create a response (after period A), so the first response from the device is a small group of frames of the real user. After a period, the deep fake Al based system responds with modified frames. The same result will be when the user moves the device camera away from itself. In this example, the system makes a new video request 2 after a predefined time period so the deep fake Al based system is forced to stop and sends again a small group of frames of the real user. And again, after a period B wherein period B is different than period A just in case the Al was too slow to respond in period A (short real time period as current Al systems are perceived as fast), the deep fake Al based system responds again or for 1st time, if it was not fast enough during period A, with modified frames. This comparing frames allows this example of this invention to detect interference and deny login access to a remote account login.
[0150] CONCEPTS
[0151] 1. A system including at least one or more internet enabled devices (wireless devices, smartphones, portable / desktop computer), with a build-in camera and at least one or more internet enabled server (SI to Sn), wherein,
[0152] The first internet enabled device (IEMD1), a first non-transitory storage medium (NTSM IEMDl), and a first computer program product embodied on the first non- transitory storage medium, the first computer program product executable on the IEMD1 when executed communicates with the server (SI), and with a second non- transitory storage medium (NTSM 1), and a second computer program product in the second non-transitory storage medium, the second computer program product executable on SI when executed communicates with IEMD1, and wherein the first computer program product when executed on IEMD1 uses a built-in camera (BIDCI) to extract the “digital data from the camera” (DC1) of IEMD1, and transmit such (BIDCl)data through the communication channels (CCH1 and CCH S1) between IEMD1 and the internet and between the internet and SI, and is processed by SI computer program product and where in the event SI computer program product detects new data being received from IEMD1 to lEMDn, such new input data is stored in the S 1 non-transitory storage medium and indexed such that each such new input data is associated to the originating user account of IEMD1 to lEMDn of the system, and wherein;
[0153] (i) a third computer program product in SI to Sn adapted to include “n” subprogram products, each sub-program product adapted to “n” different imaging processing groups, separating any writing (nl), another separating any human face (n2), another separating shapes and objects (n3), another separating colours (n4) to process new data (BIDCI to BIDCn) in parallel for each new input data, resulting in “n” outputs referred to as “encoding data 1 to n” (EDI to n), and
[0154] (ii) wherein each such output “encoding data 1 to n” is used as input to a fourth computer program product in SI to Sn which compares all the “encoding data 1 to n” and ignores all such outputs that are inconclusive and generates a new modified output of the conclusive outputs, referred to as “auto modified encoding data” (AMED), and stores such “auto modified encoding data” in the second non-transitory storage medium indexed to the corresponding user account of IEMD1 to lEMDn of the system.
[0155] 2. The system of any preceding concept 1 wherein the “digital data from the camera” (BIDCI) of IEMD1, consists of a single image, or extracts an image from the camera video, for example as the “image of a passbook” or a “selfie image of the account holder” (in example but not limited to a bank account holder) and wherein,
[0156] (i) “sub-program product” (n5) processes the (nl) data that represents the writing data only, identifying that account holder’s full name (userl), account number(s) and compares it against the adapted server SI users’ bank account database (DB1) and requests the KYC (know your customer) “face identification digital data” (FIDDD userl) of the matching account holder and stores in a temporary non- transitory storage medium of SI (TNT SMI), and wherein,
[0157] (ii) “sub-program product” (n6) processes the (n2) data that represents the face biometric data (FBD userl), identifying that passbook or account holder (userl) and compares it against the “face identification digital data” (FIDDD userl) of the passbook or account account holder of (n5) previously stored in the temporary non- transitory storage medium of SI (TNT SMI) and wherein,
[0158] (iii)in the event that the face biometric data (FBD userl)and the “face identification digital data” (FZDDD userl)match with above a threshold of X%, where X is a threshold predefined in the adapted computer program n6, then the SI computer program n6 automatically allows the userl full access, through his device IEMD1 (in example smartphone) and NTSM IEMDl to all his account data (in example banking assets) in SI.
[0159] 3. The system of any preceding concept 1 wherein the “sub-program products”, each sub-program product adapted to 4 different functional groups, are “pretrained neural network program products” (PTNNPP1 to PTNNPP4), wherein each such PTNNPP is pretrained for a specific task or sub-task generating each respectively a digital representation of; (i) the writing in the detected language of the passbook or any such account holder identification item nl, (ii) the face of a person for a single or multiple frame n2, (iii) the objects and shapes n3, (iv) the predominant or colour pallet of each area or sub-area n4, resulting in “n” outputs referred to as “encoding data 1 to n” (EDI to n) in Concept 1, and wherein the nl data is (i) auto rotated such that the detected writing is put always in the same order of top to bottom and left to right, regardless of if the nl data was rotated in any angle between more than 0 and less than 360 degrees, (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y, where y is the maximum file size preprogramed in the computer program of SI, and wherein the n2 data is (i) auto rotated such that the detected face is put always in the same order of eyes up to nose down and mouth below nose, regardless of if the nl data was rotated in any angle between more than 0 and less than 360 degrees, (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y, where y is the maximum file size preprogramed in the computer program of SI, (iii) verifies if one or both eyes are present and open.
[0160] 4. A system including at least one or more internet enabled devices (wireless devices, smartphones, portable / desktop computer), with a build-in camera and at least one or more internet enabled server (SI to Sn), wherein,
[0161] The first internet enabled device (IEMD1), a first non-transitory storage medium (NTSM IEMDl), and a first computer program product embodied on the first non- transitory storage medium, the first computer program product executable on the IEMD1 when executed communicates with the server (SI), and a second internet enabled device (IEMD2), a second non-transitory storage medium (NTSM IEMD2), and a second computer program product embodied on the second non-transitory storage medium, the second computer program product executable on the IEMD2 when executed communicates with the server (S2), and a third non-transitory storage medium (NTSM 1), and a third computer program product in the third non-transitory storage medium, the third computer program product executable on SI when executed communicates with IEMD1 and S2, and a fourth non-transitory storage medium (NTSM 2), and a fourth computer program product in the fourth non-transitory storage medium, the fourth computer program product executable on S2 when executed communicates with IEMD2 and SI, and where in the event SI computer program product detects new data being received from IEMD1 to lEMDln or S2, such new input data is stored in the SI non-transitory storage medium and indexed such that each such new input data is associated to the originating user account of IEMD1 to lEMDln of SI of the system or associated to the originating user account of IEMD2 to IEMD2n of S2 of the system, and where in the event S2 computer program product detects new data being received from IEMD2 to IEMD2n or SI, such new input data is stored in the S2 non-transitory storage medium and indexed such that each such new input data is associated to the originating user account of IEMD2 to IEMD2n of S2 of the system or associated to the originating user account of IEMD1 to lEMDln of SI of the system, and wherein;
[0162] (i) a fifth computer program product in SI, S2 to Sn adapted to include “n” sub- program products, each sub-program product adapted to “n” different unique message ID (UMID1 to UMIDn) processing groups, inserting a unique message ID (nl), another extracting the inserted unique message ID(s) (n2), another comparing the inserting unique message ID(s) with the extracted unique message ID (n3, and
[0163] (ii) wherein each such “unique message ID” UMID1 to UMIDln is inserted in the originating message sending server (in example SI) before forwarding it to receiving messaging server (in example S2) before forwarding the message to the end destination (IEMD1 to lEMDln if S2 was origination server and SI receiving server or IEMD2 to IEMD2n if SI was origination server and S2 receiving server) and ignores or deletes or replaces all such messages that do not have a unique message ID before forwarding any such message to its end destination, such end destination corresponding to a user account of IEMD1 to IEMD2n or IEMD2 to IEMD2n of the system.
[0164] 5. A system including at least one or more internet enabled devices (wireless devices, smartphones, portable / desktop computer), with a build-in camera and at least one or more internet enabled server (SI to Sn), wherein,
[0165] The first internet enabled device (IEMD1), a first non-transitory storage medium (NTSM IEMDl), and a first computer program product embodied on the first non- transitory storage medium, the first computer program product executable on the IEMD1 when executed communicates with the server (SI), and with a second non- transitory storage medium (NTSM Sl), and a second computer program product in the second non-transitory storage medium, the second computer program product executable on SI when executed communicates with IEMD1, and wherein the first computer program product when executed on IEMD1 uses a built-in camera (BIDCI) to extract the “digital data from the camera” (DC1) of IEMD1, and transmit such (DC1) data through the communication channels (CCH1 and CCH S1) respectively between IEMD1 and the internet and between the internet and SI, and is processed by the SI computer program product and where in the event SI computer program product detects new data being received from IEMD1 to lEMDn, such new input data is stored in the SI non-transitory storage medium and indexed such that each such new input data is associated to the originating user account of IEMD1 to lEMDn of the system, and wherein; (i) a third computer program product in SI to Sn adapted to include “n” subprogram products, each sub-program product adapted to “n” different ethnical imaging processing groups, separating each input image per its corresponding ethnical group (nl), another separating the facial biometric of the input data from the rest of the image (n2), another separating shapes and objects from the rest of the image (n3), another separating colours in colour pallets of areas of the image (n4) to process new data (BIDCI to BIDCn) in parallel for each new input data, resulting in “n” outputs referred to as “encoding data 1 to n” (EDI to n), and
[0166] (ii) wherein each such output “encoding data 1 to n” is used as input to a fourth computer program product in SI to Sn which compares all the “encoding data 1 to n” and ignores all such outputs that are inconclusive and generates a new modified output of the conclusive outputs, referred to as “auto modified encoding data” (AMED), and stores such “auto modified encoding data” in the second non-transitory storage medium in SI indexed to the corresponding user account of IEMD1 to lEMDn of the system, and
[0167] (iii) wherein the “digital data from the camera” (DC1) of IEMD1 corresponds to the IEMD1 user frame(s)of himself showing his face and part of shoulders obtained through IEMD1 built-in camera (BIDCI) when performing a login through the first computer program product (Application) of IEMD1, such login being performed independently of the IEMD1 device access with biometrics or PIN number, meaning the login herein separates the IEMD1 to lEMDn devices access to the IEMD1 to lEMDn access (login) to a user account in SI to Sn, wherein the “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) at each login of an IEMD1 to lEMDn user into an account (in example a bank account) of SI to Sn is verified and compared against the second non-transitory storage medium (NTSM Sl) facial biometrical data of the corresponding IEMD1 login user account when it was created and optionally against any past successful login “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) of that same user account. Thus, not allowing a login when a user can access (login) IEMD1 device but does not pass the verification of the actual facial biometrics of the account holder of the account of SI that it wants to access (login) which are verified independently of the IEMD1 to lEMDn device login access. 6. The system of any preceding Concept 5 wherein the “computer program products” CPP S1 to CPP Sn of SI to Sn, are “pretrained neural network program products” (PTNNPP1 to PTNNPP4), wherein each such PTNNPP is pretrained for a specific task or sub-task generating each respectively a digital representation of; (i) each input image per its corresponding ethnical group (nl), the human facial biometric of the input data (n2), the shapes and objects on the image (n3), colours bundled in colour range of predefined areas of the image (n4), resulting in “n” outputs referred to as “auto modified encoding data” (AMED1 to AEMDn) in Concept 3 and the “computer program product” CPP1 to CPPn of IEMD1 to lEMDn, wherein the nl data is (i) auto rotated such that the detected human face is put always in the same order of eyes up to nose down and mouth below nose (such rotation left or right degrees is stored for use in next n2 to n4 rotation), (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y, where Y is the maximum file size preprogramed in the computer program of SI, (iii) verifies if one or both eyes are present and open, and wherein the n2 to n4 data is (i) auto rotated in the same rotation direction and angle as was applied in previous nl, and wherein the “computer program products” CPP IEMDl to CPP IEMDn of IEMD1 to lEMDn, is / are invoked automatically when, (i) in one case, right after a successful biometrical device login by the device IEMD1 to lEMDn (device login / access is outside of the scope of this invention) and / or (ii) skipping the biometrical device login / access by the device IEMD1 to lEMDn. In both cases, in example when user of IEMD1 opens a bank application the “computer program products” CPP IEMDl will auto initiate a real time streaming from IEMD1 to SI “computer program product” CPP S1 wherein at least one or more frames are extracted and processed by the “pretrained neural network program products” (PTNNPPl)into nl to n4 and compared against that same IEMD1 user account stored date when that account was opened first ever and against recent successful logins and allow access to the SI user account of IEMD1 if passes the predefined threshold levels for nl to n4.
[0168] 7. A method including a first internet enabled wireless mobile device with a build-in camera (facing the display), and at least one server, wherein, the first internet enabled wireless mobile device, a first non-transitory storage medium, and a first computer program product embodied on the first non- transitory storage medium, the first computer program product executable on the first internet enabled wireless mobile device when executed communicates with the server, and the internet enabled server device, with a second non-transitory storage medium, and a second computer program product embodied on the second non-transitory storage medium, the second computer program product executable on the internet enabled server device when executed communicates with at least with the first and / or more internet enabled wireless mobile device(s), and wherein when the first computer program product is executable on the first internet enabled wireless mobile device to operate said first data communication with the server, and, wherein, the first computer program product when executed on the first internet enabled wireless mobile device, during a remote login by the wireless device userl into a remote account in the server, uses a bult-in camera to take a live video stream of X frames per second (FPS) (wherein X is a predefined parameter, in example 30 FPS) from the subject person in near proximity in front of the built-in camera, and wherein the first computer program product restricts / prohibits the use of any external camera interfacing with the first mobile device and streams it live to the server wherein, in the event the server computer program detects data received from the first or a second internet enabled wireless mobile device, the data is stored in the server transitory storage medium indexed such that each data is associated to the originating user account (in example userl) of the first or second internet enabled wireless mobile device user (user2), for further processing and wherein, the second computer program product when executed on the second internet enabled server, during a remote login by the wireless device user into a remote account in the server, extracts the following frames from the live video stream,
[0169] - a first frame (F0) at time TO and stores F0 in the second non-transitory storage medium of userl account, and
[0170] - a second frame (Fl) on the next first frame after TO where a face is detected on the frame defined as time Tl, such time between TO and T1 is stored together with Fl in the second non-transitory storage medium of userl account, and
[0171] - optionally a third frame Y sec after Tl (such parameter Y predefined, in example Y=0.2 sec) defined as time T3, and wherein the frame F0 and Fl and F2 several predefined areas (al to an) in the background and several points on the body (not face, but for example the shoulders) of the user are checked against each other and, (i) if the match of respective areas FO al to an and Fl al to an and F2_al_to_an is above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), if equal or above Z1 it’s a pass to the next step and if less, then the login is denied, and (ii) if the predominant colours are within the same colour range of respective areas FO al to an and Fl al to an and F2_al_to_an with a match above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), if equal or above Z2 it’s a pass to the next step and if less, then the login is denied, and wherein the detected person’s face frame F0 and Fl and F2 are checked against each other and, if the match of respective areas FO facel and Fl_face2 and F2_face3 is above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), if equal or above Z3 it’s a pass to the next step but if less than Z3, then the next step is denied, in the event of a pass, then it continuous to the final step (STn), wherein the final step “STn” consists of extracting from the server database the accountl KYC (know your customer) picture PO accountl corresponding with the picture of the face of the user of accountl when the accountl was first opened, and optionally of more recent accountl user face picture(s) Pn accountl of the corresponding account of the first internet enabled wireless mobile device user and compare pictures PO accountl and Pn accountl with pictures F0, Fl and F2, and if the match between each combination is above a threshold Z4% (wherein Z4 is a predefined parameter, in example Z4=95 percent), if equal or above Z4 it’s a pass to allow login but if less than Z4, then the login is denied, in the event of a pass, then it continuous to the actual remote login, wherein the second computer program product on the second internet enabled server, allows the userl a remote login (in example login on a remote banking service provider) by the wireless device user into the remote userl account in the server and informs and passes the usrl account data to the first computer program product on the first internet enabled wireless mobile device.
[0172] 8. A method of Concept 7 wherein, in the event of a pass, then before going to the final step STn, the first computer program product on the first internet enabled wireless mobile device stops the first live video streaming and restarts a second video streaming with different setting of the camera and repeats the complete process of Concept 7, and wherein the following additional steps are done,
[0173] - (i) if the match of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming is above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), if equal or above Z1 it’s a pass to the next step and if less, then the login is denied, and (ii) if the predominant colours are within the same colour range of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are with a match above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), if equal or above Z2 it’s a pass to the next step and if less, then the login is denied, and, wherein the detected person’s face frame FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are checked against each other and, if the match of respective areas FO facel of first live video streaming and FO facel of second live video streaming and Fl_face2 of first live video streaming and Fl_face2 of second live video streaming and F2_face3 of first live video streaming and F2_face3 of second live video streaming is above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), if equal or above Z3 it’s a pass and it continues to the final step STn of Concept 7 but if less than Z3, then the final step is denied.
[0174] 9. A method of Concept 7, wherein, prior to the final step STn of Concept 7, the second computer program product when executed retrieves all the pictures of the frames of each live stream video and stores them in the second non-transitory storage medium of the server, and executes the following steps,
[0175] (i) detects in all those frame pictures with a face the presence of eyes (FPE l to n), and then
[0176] (ii) detects which of those FPE l to n have both pupils present, and then
[0177] (i) compares in the order of first to last in time if the eyes close and open at least one time and if no eyes closing and opening sequence is detected the next step is denied, and if at least one eyes closing and opening sequence is found then it’s a pass (person present liveness) and proceeds to the final step STn.
[0178] 10. A method of Concept 7 including a first internet enabled wireless mobile device with an additional build-in camera2 (on the back side of the display facing away from the device user), and at least one server, wherein, prior to the steps of Concept 7, the first computer program product when executed on the first internet enabled wireless mobile device, executes the following steps,
[0179] (i) userl takes a picture Pl with camera2 of a passbook or any such other account holder identification item, wherein a passbook is defined as a standard bank or building society banking transactions printed booklet or other account holder identification item such as a live video or image (selfie) of the account holder of a client with an account in the server and wherein at least the account holder’s full name and bank account details are printed, and
[0180] (ii) such picture Pl is send by the first computer program product when executed on the first internet enabled wireless mobile device to the second computer program product and stored in the second non- transitory storage medium for processing of the next steps,
[0181] (ii) extracting the full account holder’s name and bank account details “accountl”, and
[0182] (iii) then proceed to Concept
[0183] 1 full method execution to allow or deny a remote login by the userl of the first internet enabled wireless mobile device into the corresponding account of the server corresponding to that passbook or such other account holder identification item. Note
[0184] It is to be understood that the above-referenced arrangements are only illustrative of the application for the principles of the present invention. Numerous modifications and alternative arrangements can be devised without departing from the spirit and scope of the present invention. While the present invention has been shown in the drawings and fully described above with particularity and detail in connection with what is presently deemed to be the most practical and preferred example(s) of the invention, it will be apparent to those of ordinary skill in the art that numerous modifications can be made without departing from the principles and concepts of the invention as set forth herein.
[0185] Several modifications and variations of the different examples, concepts and claims of this present invention are possible in view of the disclosures herein including the text, figures, drawings, flow-charts, explanations, concepts and claims. It is therefore to be understood that, within the scope of the appended claims, the invention can be practiced other than as specifically described in the claims of this invention and different claims can be extracted as new claims or as claims of a divisional patent. The inventions herein, which are intended to be protected should not, however, be construed as limited to the different forms or examples disclosed in the concepts, claims, or implementation examples outlined, as these are to be regarded as illustrative rather than restrictive. Variations and changes could be made by those skilled in the art without deviating from the novelty of the invention. Accordingly, the detailed descriptions and figures of this invention should be considered exemplary in nature and not limited to the novelties of the invention as set forth in the claims.
Claims
CLAIMS1. A system including at least one or more internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system including at least one or more internet enabled servers (SI to Sn), wherein: each respective internet enabled device (IEMD1 to lEMDn) includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with a server (SI) of the one or more internet enabled servers, each server of the one or more internet enabled servers including a respective second non-transitory storage medium (NTSM_1 to NTSM_n), and a respective second computer program product embodied on the respective second non-transitory storage medium, the respective second computer program product executable on the respective server to communicate with a first internet enabled device (IEMD1) of the one or more internet enabled devices, and wherein a respective first computer program product is executable on the first internet enabled device to use the built-in camera (BIDCI) to extract digital data from an image recorded and stored using the built-in camera (DC1) of the first internet enabled device, and to transmit such digital data through communication channels (CCH1 and CCH S1) between the first internet enabled device and the internet and between the internet and the server, wherein the server is configured to receive the digital data, and the respective second computer program product is executable on the server to process the received digital data, wherein the respective second computer program product is executable on the server to detect new data being received from the one or more internet enabled devices (IEMD1 to lEMDn), and to store the detected new data in the respective second non-transitory storage medium and to index the stored detected new data such that each such stored detected new data is associated to an originating user account of the first internet enabled device of the one or more internet enabled devices (IEMD1 to lEMDn), and wherein(i) each server includes a respective third non-transitory storage medium including a respective third computer program product adapted to include a pluralityof sub-program products, each respective sub-program product adapted to a respective different imaging processing task of different imaging processing tasks, the different imaging processing tasks including: (nl) separating any text or writing; (n2) separating any human face; (n3) separating shapes and objects; (n4) separating colours; the sub-program products respectively executable to process the new data (BIDCI to BIDCn) in parallel for each new input data, resulting in a respective plurality of outputs referred to as “encoding data 1 to n” (EDI to n), and(ii) each server includes a respective fourth non-transitory storage medium including a respective fourth computer program product, wherein each such respective plurality of outputs “encoding data 1 to n” is used as input to a respective fourth computer program product executable on a respective server (SI to Sn) to compare all the “encoding data 1 to n” and to ignore all such outputs that are inconclusive, and to generate a new modified output of conclusive outputs, referred to as “auto modified encoding data” (AMED), and to store such “auto modified encoding data” in the respective second non-transitory storage medium indexed to a user account corresponding a respective internet enabled device (IEMD1 to lEMDn) of the system.
2. The system of Claim 1 wherein the digital data from the camera (BIDCI) of the first internet enabled device (IEMD1), comprises single images, or images extracted from video recorded by the camera, for example which include an image of a passbook or an account holder identification item (in example, live front camera video of account holder, or a bank card and card holder live front camera video / selfie) and which include a (e.g. selfie) image of an account holder of the passbook or the account holder identification item (in example, but not limited to, a bank account holder) and wherein,(i) a sub-program product (n5) is executable on a server to process output data corresponding to the (nl) task that represents text or writing data, to identify an account holder’s full name (userl), and account number(s) and to compare the account holder’s full name, and the account number(s) against a users’ bank account database (DB1) hosted by or accessible from the server, and to request the KYC (know your customer) face identification digital data (FIDDD userl) of the account holder from the database and to store data received from the database in a non- transitory storage medium of the server (TNT SMI), and wherein,(ii) a sub-program product (n6) is executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl), to identify a passbook holder or account holder (userl) and to compare the identified passbook holder or account holder against the face identification digital data (FIDDD userl) of the passbook account holder or account holder previously stored in the non-transitory storage medium of the server (TNT SMI) and wherein,(iii) in the event that the human face (e.g. face biometric) data (FBD userl) and the face identification digital data (FIDDD userl) match above a threshold of X%, where X is a threshold predefined in the sub-program product (n6) executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl), then the sub-program product (n6) is executable on the server to process output data corresponding to the (n2) task that represents human face (e.g. face biometric) data (FBD userl) to automatically allow the identified passbook holder or account holder full access, through the first internet enabled device (IEMD1) (for example a smartphone) including the first respective non- transitory storage medium (NTSM IEMDl) to the identified passbook holder’s account data or account holder account data (for example banking assets) stored on the server.
3. The system of any preceding claim wherein for tasks nl, n2, n3 and n4, each sub-program product is adapted to a respective task, and the sub-program products are pretrained neural network program products (PTNNPP1 to PTNNPP4), wherein each such pretrained neural network program product is pretrained for a specific task or sub-task to generate respectively a digital representation of: (i) writing or text in a detected language of the passbook or of the account holder identification item (nl), (ii) the face of a person for a single or multiple frame (n2), (iii) the objects and shapes (n3), (iv) the predominant or colour pallet of each area or sub-area (n4), resulting in the plurality of outputs referred to as the “encoding data 1 to n” (EDI to n), and wherein the nl task image data is (i) auto rotated such that the detected writing or text is put always in the same order of top to bottom and left to right, regardless of if the nl task data was rotated in any angle between more than 0 degrees and less than 360 degrees; (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file sizepreprogramed in the third computer program product of the server (SI), and wherein the n2 task image data is (i) auto rotated such that the detected face is put always in the same arrangement of eyes up to nose down and mouth below nose, regardless of if the nl task data was rotated in any angle between more than 0 degrees and less than 360 degrees; (ii) the digital file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file size preprogramed in the third computer program product of the server (SI), (iii) the third computer program product of the server verifies if one or both eyes are present and open.
4. A system including a plurality of internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system further including a plurality of internet enabled servers (SI to Sn), wherein, the plurality of internet enabled devices includes a first plurality of internet enabled devices associated with a first server of the plurality of internet enabled servers, and wherein the plurality of internet enabled devices includes a second plurality of internet enabled devices associated with a second server of the plurality of internet enabled servers, wherein each respective internet enabled device (IEMD1 to lEMDn) of the first plurality of internet enabled devices includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with the first server (SI) of the plurality of internet enabled servers, wherein a first internet enabled device of the first plurality of internet enabled devices includes a first respective computer program product executable on the first internet enabled device to communicate with the first server (SI); wherein each respective internet enabled device (IEMD2 to IEMD2n) of the second plurality of internet enabled devices includes a first respective non-transitory storage medium (NTSM_IEMD2 to NTSM_IEMD2n), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internetenabled device to communicate with the second server (S2) of the plurality of internet enabled servers, wherein a second internet enabled device of the second plurality of internet enabled devices includes a first respective computer program product executable on the second internet enabled device to communicate with the second server (S2), wherein the first server includes a third non-transitory storage medium (NTSM 1), and a third computer program product embodied on the third non-transitory storage medium, the third computer program product executable on the first server to communicate with first internet enabled device and to communicate with the second server, and wherein the second server includes a fourth non-transitory storage medium (NTSM 2), and a fourth computer program product embodied on fourth non- transitory storage medium, the fourth computer program product executable on the second server to communicate with the second internet enabled device and to communicate with the first server, and where in response to the third computer program product of the first server detecting new data being received from a device of the first plurality of internet enabled devices, or from the second server, such new input data is stored in the third non-transitory storage medium of the first server and is indexed such that each such new input data is associated to an originating user account of the device of the first plurality of internet enabled devices of the first server of the system, and where in response to the fourth computer program product of the second server detecting new data being received from a device of the second plurality of internet enabled devices of second server of the system or from the first server, such new input data is stored in the fourth non- transitory storage medium of the second server and is indexed such that each such new input data is associated to the originating user account of a device of the second plurality of devices of the second server of the system, and wherein;(i) a respective fifth computer program product is embodied on a respective non- transitory storage medium of each server of the plurality of servers, and is adapted to include sub-program products, each sub-program product adapted to a plurality of different unique message ID (UMID1 to UMIDn) processing groups, including respectively: inserting a unique message ID (nl); extracting the inserted unique message ID(s) (n2), comparing the inserted unique message ID(s) with the extractedunique message ID (n3), and(ii) wherein each such unique message ID (UMID1 to UMIDln) is inserted in a message at an originating message sending server (in an example the first server) before forwarding the message to a receiving messaging server (in an example the second server) before forwarding the message to an end destination internet enabled device (e.g. IEMD1 to lEMDln if S2 was origination server and SI receiving server, or IEMD2 to IEMD2n if SI was origination server and S2 receiving server) and to ignore or delete or replace all such messages that do not have a unique message ID before forwarding any such message to its end destination, such end destination corresponding to a respective user account of an internet enabled device of the system.
5. A system including at least one or more internet enabled devices (e.g. wireless devices, smartphones, portable / desktop computers), each device including a built-in camera, the system including at least one or more internet enabled servers (SI to Sn), wherein: each respective internet enabled device (IEMD1 to lEMDn) includes a first respective non-transitory storage medium (NTSM IEMDl to NTSM IEMDn), and a first respective computer program product embodied on the first respective non-transitory storage medium, the first respective computer program product executable on the respective internet enabled device to communicate with a server (SI) of the one or more internet enabled servers, each server of the one or more internet enabled servers including a respective second non-transitory storage medium (NTSM_1 to NTSM_n), and a respective second computer program product embodied on the respective second non-transitory storage medium, the respective second computer program product executable on the respective server to communicate with a first internet enabled device (IEMD1) of the one or more internet enabled devices, and wherein a respective first computer program product is executable on the first internet enabled device to use the built-in camera (BIDCI) to extract digital data from an image recorded and stored using the built-in camera (DC1) of the first internet enabled device, and to transmit such digital data through communication channels (CCH1 and CCH S1) between the first internet enabled device and the internet and between the internet and the server, wherein the server is configured to receive the digital data, and the respective second computer program product is executable on the server toprocess the received digital data, wherein the respective second computer program product is executable on the server to detect new data being received from the one or more internet enabled devices (IEMD1 to lEMDn), and to store the detected new data in the respective second non-transitory storage medium and to index the stored detected new data such that each such stored detected new data is associated to an originating user account of an internet enabled device of the one or more internet enabled devices (IEMD1 to lEMDn), and wherein(i) each server includes a respective third non-transitory storage medium including a respective third computer program product adapted to include a plurality of sub-program products, each respective sub-program product adapted to a respective different ethnical imaging processing task of different ethnical imaging processing tasks, the different ethnical imaging processing tasks including:(nl) separating each input image per its corresponding ethnical group; (n2) separating the human facial biometric of the input data from the rest of the image; (n3) separating shapes and objects from the rest of the image; (n4) separating colours in colour pallets of areas of the image; the sub-program products respectively executable to process new data (BIDCI to BIDCn) in parallel for each new input data, resulting in a respective plurality of outputs referred to as “encoding data 1 to n” (EDI to n), and(ii) each server includes a respective fourth non-transitory storage medium including a respective fourth computer program product, wherein each such respective plurality of outputs “encoding data 1 to n” is used as input to a respective fourth computer program product executable on a respective server (SI to Sn) to compare all the “encoding data 1 to n” and to ignore all such outputs that are inconclusive, and to generate a new modified output of conclusive outputs, referred to as “auto modified encoding data” (AMED), and to store such “auto modified encoding data” in the respective second non-transitory storage medium indexed to a user account corresponding to a respective internet enabled device (IEMD1 to lEMDn) of the system, and(iii) wherein the digital data from the camera (DC1) of the first internet enabled device corresponds to the a user frame(s) of the user showing his face and part of shoulders obtained through the built-in camera (BIDCI) when performing a login through the first computer program product (Application) of the first internet enableddevice, such login being performed independently of the first internet enabled device access using biometrics or PIN number, meaning the login herein separates the plurality of internet devices access from the plurality of internet devices access (login) to a user account in a server of the plurality of servers, wherein the “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) at each login of an internet enabled device user into an account (in example a bank account) of the plurality of servers is verified and compared against the second non-transitory storage medium (NTSM Sl) facial biometrical data of the corresponding first internet enabled device login user account when it was created and optionally against any past successful login “auto modified encoding data” (AMED) stored in the second non-transitory storage medium (NTSM Sl) of that same user account, therefore not allowing a login when a user can access (login) the first internet enabled device but the user does not pass the verification of the actual facial biometrics of the account holder of the account of the first server that the user wants to access (login) which are verified independently of the device login access for the plurality of internet enabled devices.
6. The system of Claim 5 wherein the respective third computer program products (CPP S1 to CPP Sn) of the respective servers, are pretrained neural network program products (PTNNPP1 to PTNNPP4), wherein each such pretrained neural network program product is pretrained for a specific task or sub-task to generate respectively a digital representation of (i) (nl) each input image per its corresponding ethnical group; (n2) the human facial biometric of the input data; (n3) the shapes and objects on the image; (n4) colours bundled in colour range of predefined areas of the image, resulting in a plurality of outputs referred to as “auto modified encoding data” (AMED1 to AEMDn); wherein the nl image data is (i) auto rotated such that the detected human face is put always in the same order of eyes up to nose down and mouth below nose, and such rotation left or right degrees is stored for use in next n2 to n4 rotation, (ii) the digital image file is resized down to a pre-programmed file size of Y Mb if the file size is bigger than Y Mb, where Y is the maximum file size preprogramed in the respective third computer program product, (iii) the respective third computer program product is executable to verify if one or both eyes are present and open, andwherein the respective third computer program product (i) is executable to rotate the n2 to n4 data in the same rotation direction and angle as was applied to the nl image data, and wherein a respective computer program product of a respective internet enabled device is executed automatically when, (i) in one case, right after a successful biometrical device login by the respective internet enabled device and / or (ii) the biometrical device login / access by the respective internet enabled device is skipped; in both cases, for example when user of the respective internet enabled device opens a bank application the respective computer program product is executable to auto initiate a real time streaming from the respective internet enabled device to the first server, wherein at least one or more frames are extracted and processed by the pretrained neural network program products (PTNNPP1) into nl to n4 and compared against that a user account of the respective internet enabled device, in particular a user account stored date when that account was first opened, and against recent successful logins and allowed access to the first server user account of the respective internet enabled device, if the predefined threshold levels for nl to n4 are reached.
7. A computer-implemented method of processing image data recorded by a first internet enabled wireless mobile device including a display, the method including using a built-in camera facing in the same direction as the display, and using at least one internet enabled server device, wherein, the first internet enabled wireless mobile device includes a first non-transitory storage medium, and a first computer program product embodied on the first non-transitory storage medium, the first computer program product executing on the first internet enabled wireless mobile device to communicate with the internet enabled server device, and the internet enabled server device includes a second non-transitory storage medium, and a second computer program product embodied on the second non-transitory storage medium, the second computer program product executing on the internet enabled server device to communicate with at least the first internet enabled wireless mobile device and optionally with more internet enabled wireless mobile device(s), and wherein the first computer program product executes on the first internet enabledwireless mobile device to operate a data communication with the server, and, wherein, the first computer program product executes on the first internet enabled wireless mobile device, during a remote login by the wireless device userl into a remote account in the server, uses the built-in camera to take a live video stream of X frames per second (FPS) (wherein X is a predefined parameter, in example 30 FPS) of a subject person in near proximity in front of the built-in camera, and wherein the first computer program product restricts / prohibits the use of any external camera interfacing with the first mobile device, and streams the live video stream to the server wherein, in the event the server second computer program executing on the server detects data received from the first internet enabled wireless mobile device or from a second internet enabled wireless mobile device, the data is stored in the server non-transitory storage medium indexed such that each data is associated to an originating user account (in example userl) of the first internet enabled wireless mobile device user or of a second internet enabled wireless mobile device user (user2), for further processing and wherein, the second computer program product executes on the internet enabled server, during a remote login by the wireless device user into a remote account in the server, to extract the following frames from the live video stream,- a first frame (F0) at time TO and then stores the first frame (F0) in the second non- transitory storage medium of userl account, and- a second frame (Fl) which is a next first frame after TO where a face is detected in the second frame defined as time Tl, and the time between TO and T1 is stored together with Fl in the second non-transitory storage medium of userl account, and- optionally a third frame (F2) Y sec after Tl (such parameter Y predefined in the second computer program product, in example Y=0.2 sec) defined as time T3, and wherein in the first frame (F0) and in the second frame (Fl) and in the third frame (F2) several predefined areas (al to an) in the background and several points on the body (not face, but for example the shoulders) of the user are checked against each other and, (i) if the match of the respective areas (FO al to an and Fl al to an and F2_al_to_an) is equal to or above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), it’s a pass to proceed to the next step and ifless than the threshold Zl%, then the login is denied, and (ii) if the predominant colours are within the same colour range for the respective areas (FO al to an and Fl al to an and F2_al_to_an) with a match equal to or above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), it’s a pass to proceed to the next step and if less than the threshold Z2%, then the login is denied, and wherein the detected person’s face first frame (F0) and second frame (Fl) and third frame (F2) are checked against each other and, if the match of respective areas (FO facel and Fl_face2 and F2_face3) is equal to or above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), it’s a pass to proceed to the next step but if less than the threshold Z3%, then the next step is denied; where in the event of a pass, then the method continues to the final step (STn), wherein the final step includes extracting from the server database the accountl KYC (know your customer) picture PO accountl corresponding with the picture of the face of the user of accountl when the accountl was first opened, and optionally of more recent accountl user face picture(s) Pn accountl of the corresponding account of the first internet enabled wireless mobile device user and to compare pictures PO accountl and Pn accountl with the first frame (F0) and the second frame (Fl) and the third frame (F2), and if the match in each comparison is equal to or above a threshold Z4% (wherein Z4 is a predefined parameter, in example Z4=95 percent), it’s a pass to allow login but if a match in a comparison is less than the threshold Z4%, then the login is denied, wherein in the event of a pass, the method continued to an actual remote login, wherein the second computer program product on the internet enabled server executes to allow the userl a remote login (in example login on a remote banking service provider) by the wireless device user into the remote userl account in the server and informs and passes the userl account data to the first computer program product on the first internet enabled wireless mobile device.
8. The method of Claim 7 wherein, in the event of a pass, then before going to the final step STn, the first computer program product on the first internet enabled wireless mobile device stops the first live video streaming and restarts a second video streaming with different setting of the camera and repeats the complete process ofClaim 7, and wherein the following additional steps are done,- (i) if the match of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming is above a threshold Zl% (wherein Z1 is a predefined parameter, in example Zl=80 percent), if equal or above Z1 it’s a pass to the next step and if less, then the login is denied, and (ii) if the predominant colours are within the same colour range of respective areas FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are with a match above a threshold Z2% (wherein Z2 is a predefined parameter, in example Z2=85 percent), if equal or above Z2 it’s a pass to the next step and if less, then the login is denied, and, wherein the detected person’s face frame FO al to an of first live video streaming and FO al to an of second live video streaming, and Fl al to an of first live video streaming and Fl al to an of second live video streaming and F2_al_to_an of first live video streaming and F2_al_to_an of second live video streaming are checked against each other and, if the match of respective areas FO facel of first live video streaming and FO facel of second live video streaming and Fl_face2 of first live video streaming and Fl_face2 of second live video streaming and F2_face3 of first live video streaming and F2_face3 of second live video streaming is above a threshold Z3% (wherein Z3 is a predefined parameter, in example Z3=90 percent), if equal or above Z3 it’s a pass and it continues to the final step STn of Claim 7 but if less than Z3, then the final step is denied.
9. The method of Claim 7, wherein, prior to the final step STn of Claim 7, the second computer program product when executed retrieves all the pictures of the frames of each live stream video and stores them in the second non-transitory storage medium of the server, and executes the following steps,(i) detects in all those frame pictures with a face the presence of eyes (FPE l to n), and then(ii) detects which of those FPE l to n have both pupils present, and then(iii) compares in the order of first to last in time if the eyes close and open at least one time and if no eyes closing and opening sequence is detected the next step is denied, and if at least one eyes closing and opening sequence is found then it’s a pass (person present liveness) and proceeds to the final step STn.
10. The method of Claim 7 including using the first internet enabled wireless mobile device including an additional built-in camera2, on the reverse side of the device to the side with the display, the additional built-in camera2 facing away from the device user, and using at least one server, wherein, prior to the steps of Claim 7, the first computer program product when executing on the first internet enabled wireless mobile device, executes the following steps,(i) the device takes a picture Pl with camera2 of a passbook or of an account holder identification item, wherein a passbook is defined as a standard bank or building society banking transactions printed booklet or any such other account holder identification item (such as a live video or image (e.g. selfie) of the account holder) of a client with an account in the server and wherein at least the account holder’s full name and bank account details are printed, and(ii) such picture Pl is sent by the first computer program product executing on the first internet enabled wireless mobile device to the second computer program product and is stored in the second non- transitory storage medium for processing of the next steps,(ii) extracting the full account holder’s name and bank account details “accountl” from picture Pl, and(iii) then proceed to Claim 7 full method execution to allow or deny a remote login by the userl of the first internet enabled wireless mobile device into the corresponding account of the server corresponding to that passbook or account holder identification item.
Citation Information
Patent Citations
Systems and methods for fraud prevention
WO2022219351A1
System and method for an antifraud scoring system
WO2023139367A1
Credit Card Auto-Fill
US20150347859A1
Fraud deterrence and / or identification using multi-faceted authorization procedures
US20190295084A1
Novel ensemble method for face recognition deep learning models
US20210150534A1
Cited By
Authenticating a user in liveness testing using a trusted camera
US20260236567A1