Visual integrity check

Abstract

Embodiments of the present disclosure relate to visual integrity checking. An apparatus, method and computer program are described comprising: means for capturing a presentation (106) on a display (104a), where the presentation (106) comprises data representing a first hash (108); means for generating a second hash (111) from the captured presentation (122); means for reading a digital code (107) from the display (104a) or the captured presentation (122), the digital code (107) comprising a first hash (108) and a digital signature (109) associated with the first hash (108); means for comparing the first hash (108) and the second hash (111); and means for sending an indication as to whether the digital signature (109) is legal in response to the first hash (108) and the second hash (111) at least substantially matching.

Description

Technical Field

[0001] Various example embodiments may be directed to systems, methods, and / or computer programs for performing a visual integrity check on a request from a service to a user device. Background Art

[0002] In the current digital world, where users access services from cloud platforms or distributed computing systems via their user devices, there is now always uncertainty about trusting requests from seemingly trusted services. This is a scenario that has become very familiar: a user receives a request, email, message, communication, application request, or service request that appears to originate from a service that the user recognizes and trusts. For example, the service may include, but is not limited to, for example, an email client, a retail website, an online banking application, a social media platform application, an information technology (IT) department application, an operating system and / or a mobile provider application / service, and / or any other trusted source / service that the user can use via their user device. However, when using these services, there is always uncertainty about whether the user truly trusts the service request or whether the request has been tampered with.

[0003] For example, cybercriminals can intercept communications from trusted entities / services and / or impersonate trusted entities / services and manipulate users into inadvertently revealing sensitive information (e.g., login credentials, confidential information, company secrets, etc.) and / or taking actions to install malware disguised as genuine updates or software, and / or taking "recommended" actions that result in malicious behavior. It is desirable to have an integrity checking apparatus, method, and / or system that robustly detects communications from cybercriminals while being simple and convenient to operate for users operating their user devices. Summary of the invention

[0004] The scope of protection sought by various embodiments of the present invention is set out by the independent claims. Embodiments and features described in this specification that do not fall under the scope of the independent claims, if any, should be interpreted as examples that help understand the various embodiments of the present invention.

[0005] In a first aspect, the present specification describes an apparatus comprising: means for capturing a representation on a display, wherein the representation includes data representing a first hash; means for generating a second hash from the captured representation; means for reading a digital code from the display or the captured representation, the digital code including the first hash and a digital signature associated with the first hash; means for comparing the first hash and the second hash; and means for sending an indication as to whether the digital signature is legitimate in response to the first hash and the second hash at least substantially matching.

[0006] In some example embodiments, the display is part of a user device separate from the apparatus, the apparatus comprising an imaging component for capturing a presentation on the display of the user device.

[0007] In some example embodiments, the display is part of the apparatus, wherein capturing the presentation on the display of the apparatus further comprises means for capturing a screenshot of the presentation on the display, wherein the screenshot is the captured presentation.

[0008] In some example embodiments, the presentation includes one or more from the following group: an image for presentation on a display; multiple images for presentation on a display; a video for presentation on a display; or data representing a user prompt, interaction prompt, or dialog box for rendering on a display.

[0009] In some example embodiments, the captured representation includes a first image associated with the first hash and a second image associated with the digital code, and wherein: the means for generating the second hash of the captured representation further includes means for applying the hash algorithm to the first image to output the second hash; and the means for reading the digital code from the display further includes means for reading the digital code from the second image.

[0010] In some example embodiments, further included are means for cropping the captured presentation to generate a first image and means for cropping the captured presentation to generate a second image.

[0011] In some example embodiments, the digital code also includes one or more from the following group: a barcode; a two-dimensional matrix barcode; a quick response (QR) code; a high capacity color two-dimensional (HCC2D) code; a digital code embedded in a presentation or image; a digital code represented by a presentation or image; any other encoding using an image; or any other encoding that can be presented on the display 104a and decoded from the presented encoding when captured.

[0012] In some example embodiments, the means for comparing the first hash and the second hash further comprises: means for calculating a distance between the first hash and the second hash; and means for determining that the first hash and the second hash at least substantially match when the distance is less than or equal to a predetermined distance threshold or an error threshold.

[0013] In some example embodiments, the distance is one or more from the group of: a Hamming distance between the first hash and the second hash; a Euclidean distance between the first hash and the second hash; or any other distance metric between the first hash and the second hash used by a hashing algorithm that generates the first hash and the second hash.

[0014] In some example embodiments, the first hash and the second hash are generated using a hash algorithm based on one or more hash algorithms from the following group: average hash (aHash) algorithm; median hash (mHash) algorithm; perceptual hash (pHash) algorithm; difference hash (dHash) algorithm; block hash (bHash) algorithm; wavelet hash (wHash) algorithm; color moment hash algorithm; color hash (Colorhas) algorithm; Marr-Hildreth (marrhildreth) hash algorithm; fuzzy hash algorithm; video hash algorithm; locality sensitive hash algorithm; machine learning hash algorithm; a pair of machine learning hash models; or any other hash algorithm configured to generate a hash from one or more images.

[0015] In some example embodiments, wherein: the hash algorithm used to generate the first hash and the second hash is the same hash algorithm; or the hash algorithm used to generate the first hash and the second hash is a similar or different hash algorithm that generates the same hash.

[0016] In some example embodiments, the digital code also includes data representing a seed associated with the digital signature, wherein the seed is used by the digital signature algorithm to verify whether the digital signature is legitimate or valid.

[0017] In some example embodiments, the presentation is associated with a prompt or request from a service for a user action or information from a user of the apparatus and / or a user device of a user using the service, and the component for sending an indication to the user also includes: a component for providing an indication to the user via the apparatus or user device as to whether to perform the prompt or request for the user action or information on the apparatus or user device, respectively, based on whether the digital signature from the service is legitimate or valid and in response to the first hash and the second hash substantially matching.

[0018] In some example embodiments, the request for user action or information includes one or more from the following group: a request from the service for input from a user or an action to be performed by a user; a request for the user to enter sensitive user information; a request for the user to scan sensitive information; a request for the user to scan a user passport, a user photograph, or a contactless credit card; a request for user credentials or login credentials from the user; a request for confidential information of the user or confidential information associated with the user; a request for organizational information associated with the user; a request for the user to accept and / or proceed with installation of software on the user device; a request for a conversation related to downloading, accepting and / or opening a file attachment received from the service; or any other request requiring user input on the user device via a client service application associated with the service.

[0019] In some example embodiments, the apparatus comprises at least: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause performance of the apparatus.

[0020] In a second aspect, the present specification describes a method comprising: capturing a representation presented to a user on a display, wherein the representation includes data representing a first hash; generating a second hash from the captured representation; reading a digital code from the display or the captured representation, the digital code including a first hash associated with the representation and a digital signature associated with the first hash; comparing the first hash and the second hash; and in response to the first hash and the second hash at least substantially matching, sending an indication to the user as to whether the digital signature is legitimate or valid.

[0021] In a third aspect, the present specification describes a computer program comprising instructions for causing a device to perform at least the following operations: capture a representation presented to a user on a display, wherein the representation includes data representing a first hash; generate a second hash from the captured representation; read a digital code from the display or the captured representation, the digital code including a first hash associated with the representation and a digital signature associated with the first hash; compare the first hash and the second hash; and in response to the first hash and the second hash at least substantially matching, send an indication to the user as to whether the digital signature is legitimate or valid.

[0022] In a fourth aspect, the present specification describes an apparatus for a service, comprising: a component for generating a representation and a digital code associated with requesting a user action or information at a user device using the service, wherein the digital code comprises a hash associated with the representation and a digital signature associated with the hash; and a component for sending a request for the user action or information to the user device, wherein the request comprises data representing the representation and the digital code for display on the user device to authenticate the request from the service by the user.

[0023] In a fifth aspect, the present specification describes a method for a service device, comprising: generating a representation and a digital code associated with requesting a user action or information at a user device of a user using a service, wherein the digital code comprises a hash associated with the representation and a digital signature associated with the hash; and sending a request for the user action or information to the user device, wherein the request comprises data representing the representation and the digital code for display on the user device to authenticate the request from the service by the user.

[0024] In a sixth aspect, the present specification describes an apparatus comprising: a component for receiving a request for a user action or information at a user device of a user from a service, the request for the user action comprising data representing a presentation and a digital code for display on the user device to authenticate the request from the service, wherein the presentation and the digital code are generated by the service, the digital code comprising a hash of the presentation and a digital signature associated with the hash; a component for displaying the presentation and the digital code on the user device; a component for performing a visual integrity check on the request using the displayed presentation and the digital code; a component for receiving an indication of whether the digital code and presentation are authentic or legal; a component for notifying a user to perform the requested user action in response to an indication that the digital code and presentation are authentic or legal; and a component for notifying a user to refuse to perform the requested user action in response to an indication that the digital code and presentation are unauthentic or illegal.

[0025] In some embodiments, the components for performing a visual integrity check on the request also include components for using another user device that is caused to perform a visual integrity check based on capturing the presentation and display of the digital code presented on a display of the user device, wherein the other user device is separate from the user device.

[0026] In some embodiments, the component for performing a visual integrity check on the request also includes a component for performing the visual integrity check on a user device, wherein the user device also includes a component for performing the visual integrity check based on capturing a screenshot of the presentation and the digital code presented on a display of the user device.

[0027] In some embodiments, the components for performing a visual integrity check also include: components for capturing a representation for presentation to a user on a display, the representation associated with a first hash; components for generating a second hash from the captured image; components for reading a digital code from the display or the captured image, the digital code comprising the first hash associated with the representation and a digital signature associated with the first hash; components for comparing the first hash and the second hash; and components for sending an indication to the user as to whether the digital signature is legitimate in response to the first hash and the second hash at least substantially matching.

[0028] In some example embodiments, the presentation is associated with a prompt or request from a service for a user action or information from a user device using the service, and the component for sending an indication to the user further includes: a component for providing an indication to the user as to whether to perform the prompt or request for the user action or information on the user device based on whether the digital signature from the service is legitimate or valid, in response to the first hash and the second hash substantially matching.

[0029] In some example embodiments, a visual integrity check is performed according to the functionality of the first aspect and / or the second aspect and / or modifications thereof.

[0030] In some example embodiments, components of a user device include at least: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause performance of the device.

[0031] In a seventh aspect, the present specification describes a method for a user device, comprising: receiving a request for a user action or information at the user device from a service, the request for the user action or information comprising data representing a presentation and a digital code for display to the user on the user device to verify the request from the service, wherein the presentation and the digital code are generated by the service, the digital code comprising a hash associated with the presentation and a digital signature associated with the hash; displaying the presentation and the digital code to the user on the user device; performing a visual integrity check on the request using the displayed presentation and the digital code; receiving an indication of whether the digital code and the presentation are authentic or legal; in response to an indication that the digital code and the presentation are authentic or legal, notifying the user to perform the requested user action; and in response to an indication that the digital code and the presentation are unauthentic or illegal, notifying the user to refuse to perform the requested user action.

[0032] In an eighth aspect, the present specification describes a system comprising: a server device providing services according to the fourth aspect; and a user device according to the sixth aspect, wherein when a user uses the service via the user device, the server device providing services and the user device communicate through a network.

[0033] In some embodiments, the user device includes components for performing a visual integrity check according to any of the features and / or functions of the first aspect and / or second aspect in response to the user device receiving a request from a service and displaying a presentation and digital code associated with the request.

[0034] In some example embodiments, another user device according to the features and / or functionality of the first aspect and / or the second aspect, wherein the other user device includes a component for performing a visual integrity check in response to the user device receiving a request from a service and displaying a representation and a digital code associated with the request.

[0035] In another aspect, the present specification relates to an embodiment of a non-transitory computer-readable medium including program instructions stored thereon for performing at least the following operations: capturing a representation presented to a user on a display, wherein the representation includes data representing a first hash; generating a second hash from the captured representation; reading a digital code from the display or the captured representation, the digital code including a first hash associated with the representation and a digital signature associated with the first hash; comparing the first hash and the second hash; and in response to the first hash and the second hash at least substantially matching, sending an indication to the user as to whether the digital signature is legitimate or valid.

[0036] In yet another aspect, the present specification relates to an embodiment of a non-transitory computer-readable medium including program instructions stored thereon for performing at least the following operations: generating a representation and a digital code associated with requesting a user action or information at a user device of a user using a service, wherein the digital code includes a hash associated with the representation and a digital signature associated with the hash; and sending a request for the user action or information to the user device, wherein the request includes data representing the representation and the digital code for display on the user device to authenticate the request from the service by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various exemplary embodiments will now be described by way of non-limiting examples with reference to the accompanying drawings, in which:

[0038] Figure 1a is a schematic diagram of an example visual integrity inspection system useful for understanding example embodiments;

[0039] Figure 1b is a schematic diagram of another example visual integrity inspection system useful for understanding example embodiments;

[0040] Figure 1c is a schematic diagram showing an example of a visual integrity check for understanding an example embodiment;

[0041] Figure 2a is a flow chart illustrating an example service visual integrity checking method useful for understanding example embodiments;

[0042] Figure 2b is a flow chart illustrating an example user device visual integrity check method useful for understanding example embodiments;

[0043] Figure 2c is a flow chart illustrating an example visual integrity checking method of a user device or an external apparatus for understanding example embodiments;

[0044] Figure 3a is a diagram for understanding the exemplary embodiments. Figure 1aA signal flow diagram of an example visual integrity check signal flow for performing a visual integrity check by a user on a request from a service for a user action as shown in;

[0045] Figure 3b is a diagram for understanding the exemplary embodiments. Figure 1b A signal flow diagram of another example visual integrity check signal flow for performing a visual integrity check by a user on a request from a service for a user action, as shown in;

[0046] Figure 4 is a flow chart of another example visual integrity check system for use by a user having a user device and a mobile device for performing a visual integrity check useful for understanding example embodiments;

[0047] Figure 5 is a schematic diagram of an example device useful for understanding example embodiments; and

[0048] Figure 6 is a schematic diagram of an example computer readable medium useful for understanding example embodiments. DETAILED DESCRIPTION

[0049] The scope of protection sought by various embodiments of the present invention is set out in the claims. Embodiments and features described in the specification that do not fall under the scope of the independent claims, if any, should be interpreted as examples that aid in understanding the various embodiments of the present invention.

[0050] Like reference numerals refer to like elements throughout the specification and drawings.

[0051] Figure 1a An example visual integrity check (VIC) system and process 100 for use in verifying the authenticity of a service request 101 transmitted from a service 103 to a user device 104 of a user 105 over a communication channel 102 is shown. The service request 101 may be a request 101 from the service 103 to the user 105 and / or the user device 104 for requesting an action by the user 105 of the user device 104 or for requesting information from the user 105 of the user device 104. The service request 101 is presented to the user 105 on a display 104a of the user device 104. The service 103 may be implemented on one or more servers remote from the user device 104, wherein the user 105 accesses the service 103 via a client service application (or client application (app)) on the user device 104. The client application is associated with the service 103 and / or one or more services provided to the user 105. During operation of the client application by the user 105 on the user device 104, the client application communicates with the service 103 over the communication channel 102.

[0052] For example, service 103 sends one or more service requests 101 for actions or information that require user input to a client application of user 105. Each service request 101 for an action or information may be, but is not limited to, for example, a request from service 103 that requires input from or an action by user 105; a request for user 105 to enter sensitive user information; a scan of sensitive information such as a user passport, a user photo, and / or a contactless credit card; a request for login credentials from user 105; a request for confidential information of or associated with user 105; a request for organizational information associated with user 105; a request for user 105 to accept and / or proceed with installation of software on user device 104; a dialog request related to downloading, accepting, and / or opening a file attachment received from service 103; and / or any other request that requires user input 105 on user device 104 via a client service application associated with service 103, or any combination thereof, as required by the application.

[0053] The user device or apparatus 104 may include or represent any computing device of the user 105 that is capable of displaying at least the representation 106 and / or the digital code 107 associated with the service request 101. For example, the user device or apparatus 104 may include, but is not limited to, for example, a mobile communication device, a smart phone, a mobile phone, a tablet computer, a laptop computer, a personal computer, a television, a bank terminal, a vehicle display, a virtual reality (VR) display, an augmented reality display, an electronic billboard, a display device, any public / shared device capable of displaying the service request 101 and the representation 106 / digital code 107, and / or any other computing device capable of displaying the service request 101, the representation 106 and / or the digital code 107, or any combination thereof.

[0054] In the example VIC system 100, the service 103 generates a presentation 106 (e.g., p1) associated with a service request 101 and a digital code 107 (e.g., DC) associated with the service request 101, wherein the service request 101 is used to request a user action / information from the user 105 at the user device 104 when using the service 103. The user action may include, but is not limited to, for example, requiring the user to accept / approve information and / suggested actions, requiring the user to enter user credentials; requiring the user to enter sensitive information; requiring the user to download and / or install software and / or software updates; requiring the user to open an attachment and / or sign an attachment; requiring the user to enter data and / or provide any other request for permission for the user device 104 to perform a function or process associated with the service request 101, or any combination thereof.

[0055] Service 103 uses a hash-based algorithm that generates a first hash 108 (e.g., v1) of representation 106, service 103 also generates a digital signature 109 (e.g., σ1) associated with first hash 108, and in some implementations also generates a seed 110 (e.g., S1) associated with digital signature 109. Service 103 generates digital code 107 by digitally encoding first hash 108 (e.g., v1) of representation 106, digital signature 109 (e.g., σ1) associated with first hash 108, and seed 110 (e.g., S1) associated with digital signature 109 in some implementations. Service 103 sends service request 101 including data representing representation 106 and digital code 107 to user device 104 of user 105 via communication channel 102.

[0056] The user device 104 receives a service request 101 for a user action / information of a user 105 sent from the service 103 via the communication channel 102. The service request 101 includes data representing a presentation 106 and a digital code 107 for display on the user device 104, for authenticating the service request 101 received from the service 103. The user device 104 displays the presentation 106 associated with the service request 101 on a display 104a of the user device 104. The user device 104 also displays the digital code 107 on the display 104a of the user device 104. The presentation 106 and the digital code 107 may be displayed on the display 104a of the user device 104 simultaneously or substantially simultaneously. Alternatively, the presentation 106 and the digital code 107 may be displayed on the display 104a of the user device 104 at different times, for example, sequentially and / or by separate requests from the user 105. For example, digital code 107 may be displayed before displaying representation 106, and representation 106 may be displayed for capture once digital code 107 is read / captured and / or decoded. Alternatively, representation 106 may be displayed before digital code 107, and once representation 106 is captured, digital code 107 may be displayed for capture / reading and decoding.

[0057] In this example, the user device 104 performs a visual integrity check (VIC) process related to the service request 101 based on displaying the representation 106 and the digital code 107 associated with the service request 101 to the user 105 on the display 104a of the user device 104. In this example, the VIC process is implemented or performed at an external device or second user device 112 accessible to the user 105, which provides an indication of whether the service request 101 is legitimate / authentic based on capturing the representation 106 and the digital code 107 displayed on the user device 104 at the second user device 112. The external device or second user device 112 can be any type of device capable of capturing the display 104a of the user device 104. For example, an external device or second user device 112 having an imaging component (e.g., a camera sensor) and a communication component includes, but is not limited to, for example, a digital camera, a mobile communication device, a smart phone, a laptop computer, a personal computer with a camera (e.g., a webcam), a video camera, and / or any other computing device capable of capturing the display 104a of the user device 104, or any combination thereof. In some examples, the second user device 112 may be as shown in FIG. Figure 1b A user device 104 is depicted.

[0058] The second user device 112 uses the captured presentation 122 and the captured digital code to verify whether the service request 101 is legitimate / authentic. In the example, the second device 112 sends an indication to the user device 104 that the service request 101 is legitimate / authentic, the indication is displayed to the user 105, and the user 105 can proceed based on the indication. Alternatively, the second device 112 displays whether the service request 101 is legitimate / authentic on the display of the second device 112 based on the VIC process performed at the second device 112. The user 105 can view the indication displayed on the second user device 112 and proceed accordingly to act on the service request 101. In some use cases, the action is a manual operation of the user 105 and / or an input to the user device 104.

[0059] In this example, the second user device 112 includes a VIC application and / or a VIC device operating on the second user device 112, which is configured to perform a VIC process on the second user device 112. For example, the second user device 112 includes functionality for performing the VIC process in a trusted environment of the second user device 112. The VIC process can be based on, but not limited to, for example, a VIC application operating on the second device 112 in a sandbox or trusted environment of the second device 112, a VIC device or component operating on a trusted hardware module of the second device 112, and / or functionality of the VIC operating within an operating system of the second device 112 or an operating system component of the second device 112. This provides the advantage of reducing the risk of the VIC process being maliciously altered and / or changed by a third party, and increases the credibility of the VIC indication output from the VIC process when the VIC process is executed by the second user device 112.

[0060] The VIC process includes the second user device 112 capturing the presentation 106 displayed on the display 104a of the user device 104 as the captured presentation 122. The captured presentation 122 can be displayed on the second user device 112 to obtain clarity and indication of successful capture. The second user device 112 also captures the digital code 107 presented on the display 104a of the user device 104 to decode into the first hash 108, digital signature 109 and / or seed 110 (or other required information in the digital code 107) encoded by the digital code 107. The second user device 112 has the captured presentation 122 and the decoded first hash 108, digital signature 109 and / or seed 110. The captured presentation 122 includes data representing the first hash 108. The second user device 112 uses a hash algorithm to process the relevant portion of the captured presentation 122 to generate a second hash 111 (e.g., v2) associated with the presentation 106. The hash algorithm used by the second device 112 is of the same and / or substantially similar type as the hash algorithm used by the service 103 when generating the first hash 108 of the presentation 106. The type of hash algorithm and any other hash parameters / keys that the second user device 112 should use may be communicated as additional information in the digital code 107 from the service 103. The VIC process of the second user device 112 compares the first hash 108 (e.g., v1) to the second hash 111 (e.g., v2).

[0061] The VIC process of the second user device 112 determines whether the first hash 108 and the second hash 111 at least substantially match each other. For example, the comparison performed by the second user device 112 may include calculating a distance between the first hash 108 and the second hash 111, wherein when the distance between the first hash 108 and the second hash 111 is within a predetermined threshold, the first hash 108 and the second hash 111 are considered to substantially match. The distance may be calculated based on a distance metric associated with the type of hash algorithm used by the service 103 and / or the second user device 112. The predetermined distance threshold and / or the type of distance metric used for the comparison may also be communicated via the digital code 107 from the service 103. Alternatively, the distance metric and / or the distance threshold may be set by the user of the second user device 112 according to the level of trust that the user 105 is prepared to accept.

[0062] In another example, a hash algorithm that processes text and / or strings can be used to generate a first hash 108 of the representation 106 based on text within the representation 106. In this case, the second user device 112 detects and generates text and / or strings from the captured representation 122 (e.g., using optical character recognition), and generates a second hash 111 on the generated text and / or strings using the hash algorithm. In this case, the comparison can detect whether the first hash 108 and the second hash 111 match exactly.

[0063] In another example, a hash algorithm for processing an image may be used, based on the reference image in the representation 106. Figure 1c The image of the user prompt 130 described generates a first hash 108 of the presentation 106. In this case, the second user device 112 detects and generates an image of the user prompt 130 from the captured presentation 122, and generates a second hash 111 on the generated image of the user prompt 130 using a hashing algorithm. In this case, the comparison can detect whether the first hash 108 and the second hash 111 at least substantially match. For example, the comparison performed by the second user device 112 can include calculating a distance between the first hash 108 and the second hash 111, wherein when the distance between the first hash 108 and the second hash 111 is within a predetermined threshold, the first hash 108 and the second hash 111 are considered to substantially match (i.e., the image of the user prompt used to generate the first hash 108 substantially matches the image of the user prompt generated from the captured presentation used to generate the first hash 108). The distance can be calculated based on a distance metric associated with the type of hashing algorithm used to process the hash of the image by the service 103 and / or the second user device 112. The predetermined distance threshold and / or the type of distance metric used for the comparison can also be communicated via the digital code 107 from the service 103.

[0064] Once the VIC process determines that the first hash 108 and the second hash 111 substantially match (or exactly match), the second user device 112 uses the digital signature 109 and the seed 110 to determine whether the digital signature 109 is legitimate. If it is determined that the digital signature 109 is legitimate and the first hash and the second hash substantially match, the second user device 112 determines that the service request 101 is legitimate / authentic. If a) it is determined that the first hash 108 and the second hash 111 do not substantially match (e.g., the distance is greater than a predetermined threshold, or the first hash and the second hash do not match); or b) it is determined that the digital signature 109 is not legitimate / authentic, the VIC process of the second user device 112 determines that the service request 101 is not legitimate / authentic. The VIC process indicates whether the service request 101 is legitimate. The second user device 112 can indicate or render a VIC indication to the user 105 (e.g., a positive mark or check mark indicating that the service request 101 is legitimate, or a negative mark or cross mark indicating that the service request 101 is not legitimate). Once the VIC indication is displayed and / or known to the user 105, the user 105 may take appropriate actions related to the service request 101 on the user device 104 while using the service 103. For example, if the VIC indication indicates that the service request 101 is legitimate, the user 105 of the user device 104 may take actions on the service request 101, such as, but not limited to, for example, entering user credentials (e.g., entering a user login name and password), providing requested information (e.g., entering sensitive user information or company information), performing a requested action (e.g., downloading / installing a software update or opening an attachment, etc.), or accepting a prompt rendered in the presentation 106 on the display 104a (e.g., pressing a related button and / or entering requested information), or any combination thereof.

[0065] In another example, when the second user device 112 performs the VIC of the service request 101 using the representation 106 and the digital code 107, the second user device 112 may send a VIC indication to the user device 104 regarding whether the representation 106 and the digital code 107, and therefore the service request 101, are authentic or legitimate. Upon receiving the VIC indication of whether the service request 101 is authentic or legitimate (e.g., both the digital code and the representation are determined to be authentic or legitimate), the user device 104 may render (e.g., display) the VIC indication to the user 105 of the user device 104. For example, in response to the indication of the digital code 107 and the representation 106, and therefore the service request 101, are authentic or legitimate, the user device 104 may notify the user 105 that the request 101 is from a trusted source, and therefore the user 105 may perform the requested user action of the service request 101 from the service 103. Alternatively, in response to the VIC indication of digital code 107 and presentation 106 and therefore service request 101 being unauthentic or illegitimate, user device 104 may notify user 105 that request 101 should not be trusted and that user 105 should refuse to perform the requested user action of service request 101 .

[0066] The presentation 106 may include, but is not limited to, for example, an image for presentation on the display 104a of the user device 104 of the user 105, multiple images for presentation on the display 104a of the user device 104; video or multimedia for presentation on the display 104a of the user device 104; data representing a user prompt associated with the service request 101 or a dialog box for rendering on the display 104a of the user device 104, text information for presentation on the display 104a of the user device 104, and / or any other information for presentation on the display 104a of the user device 104, or any combination thereof, and can be used by the service 103 to generate the first hash 108.

[0067] In the example, presentation 106 is an image generated by service 103, which may be an image associated with service request 101. For example, image 106 may represent service request 101 or how service request 101 will be rendered and / or presented / displayed to user 105 on display 104a of user device 104. Presentation 106 may be a user prompt, interactive prompt, or dialog box (e.g., a graphical user interface (GUI) element) requesting user action and / or information. For example, in some applications, GUI elements may be generated by service 103 at the service side (e.g., via simulation or pre-rendering at the service end). Alternatively, presentation 106 may be generated from an image captured on user device 104, which is shared with service 103, where service 103 generates a digital code (e.g., a QR code) to send to user device 104 so that it is presented to user 105 by user device 104 on display 104a of user device 104.

[0068] The representation 106 generated by the service 103 includes one or more of the following: an image, a video, a graphic, a text, other imaging data, and / or a rendered user prompt associated with the service request 101 or a dialog box check box (e.g., a GUI element), or any combination thereof. A first hash 108 may be generated from the representation 106 using a hashing algorithm capable of generating a hash from an image, a video, a text, a graphic, other imaging data, and / or a rendered user prompt associated with the service request 101 or a dialog box check box (e.g., a GUI element), or the like, or any combination thereof. For example, a hashing algorithm is used to generate the first hash 108 from the representation 106, and the same or substantially similar hashing algorithm is used to generate the second hash 111 from the captured representation 122. As an example, when the presentation 106 includes one or more of an image, a video, a graphic, text, any other imaging data, or any combination thereof, the first hash 108 and the second hash 111 may be generated using a hash algorithm based on, but not limited to, one or more hash algorithms from, for example, the following group: an average hash (aHash) algorithm; a median hash (mHash) algorithm; a perceptual hash (pHash) algorithm; a difference hash (dHash) algorithm; a block hash (bHash) algorithm; a wavelet hash (wHash) algorithm; a color moment hash algorithm; a color hash (Colorhas) algorithm; a Marr-Hildreth (marrhildreth) hash algorithm; a fuzzy hash algorithm; a video hash algorithm; a locality sensitive hash algorithm; a machine learning hash algorithm; a pair of machine learning hash models; or any other hash algorithm configured to generate a hash from one or more images, or any combination thereof. In this case, a distance metric (e.g., a Hamming distance and / or a Euclidean distance, or other distance metrics associated with a hash algorithm) may be used to determine or estimate the distance between the first hash 108 and the second hash 111. When the determined or estimated distance is less than or equal to a predetermined distance threshold or error threshold, the first hash 108 and the second hash 111 are considered to at least substantially match. When the determined or estimated distance is greater than the predetermined distance threshold or error threshold, the first hash 108 and the second hash 111 are considered to substantially not match or not match.

[0069] As described, the hashing algorithm used to generate the first hash 108 associated with the presentation 106 and / or the second hash 111 associated with the captured presentation 122 can be based on a machine learning (ML) model and / or an ML algorithm. For example, a Siamese neural network can be trained and used to compare the first hash 108 associated with the presentation 106 and the second hash 111 associated with the captured presentation 122. Alternatively, instead of using hashes 108 and 111, the captured presentation 122 can be sent from the second user device 112 to the user device 104, where the trained Siamese neural network is used to distinguish whether the captured presentation 122 is the same as or at least substantially matches the presentation 106 presented on the display of the user device 104. Alternatively, the presentation 106 can be sent from the user device 104 to the second user device 112, where the trained Siamese neural network is used to distinguish whether the captured presentation 122 is the same as or at least substantially matches the presentation 106 presented on the display of the user device 104. This can be used to determine whether the captured presentation 122 substantially matches the presentation 106. The output of each network can be compared with Euclidean distance or other distance metrics. Alternatively, the output of the Siamese neural network can be a classification indicating a match or mismatch (e.g., true or false), where the distance comparison is performed within the Siamese neural network.

[0070] The digital code 107 may include, but is not limited to, for example, data representing one or more digital codes from the following group: a barcode; a two-dimensional matrix barcode; a quick response QR code; a high-capacity color two-dimensional HCC2D code; a digital code embedded in a presentation 106 or an image; a digital code represented by a presentation 106 or an image; or any other digital code that encodes the first hash 108, the digital signature 109, and the seed 110, and can be decoded from data captured from the display 104a of the user device 104 during the VIC process, where the presentation 106 and the digital code 107 are displayed on the display 104a of the user device 104, or any combination thereof.

[0071] The digital code 107 includes at least data representing the first hash value 108, the digital signature 109, and the seed 110 associated with the digital signature 109. The digital signature 109 and the seed 110 embedded or encoded in the digital code 107 are used as additional verification to check whether the first hash 108 and / or the service request 101 associated with the digital code 107 are generated by the service 103. In this case, the service 103 includes the seed 110 in the digital code 107, which can be a unique seed each time the service request 101 is generated, and / or the seed changes regularly or irregularly. The seed 110 is used to strengthen the digital signature 109. For example, the seed 110 (e.g., S1) is a unique string generated by the service 103 to introduce more meta information, which brings uniqueness to each call, each user, each day, etc. The seed 110 also has the further advantage of reducing potential replay attacks or spoofing on the presentation 106 and / or the digital code 107. For example, the seed 110 may be based on a timestamp of the generated digital signature 109 and / or have an expiration time at which the digital signature 109 is deemed invalid. For further security, the seed 110 may be further combined with a version number and a salt generated by a secure random number generator. Other examples of seeds include metadata about the company providing the service 103 or the username of the target user 105. The metadata may be shown in a text format on the second user device 112 with strong cryptographic properties.

[0072] Since the service request 101 and presentation 106 (e.g., dialog box) may be sent to multiple users, there is a risk of a replay attack, where the presentation 106 shown to a first user (e.g., user A) may be observed by an adversary K. The adversary K then has a time window to launch their attack before showing the same presentation 106 (e.g., dialog box) to a second user (e.g., user B). The seed 110 mitigates this problem because the seed contains useful metadata, such as an expiration date, version number, etc., ensuring that the digital signature 109 (e.g., σ1) does not remain valid for too long.

[0073] In another example, the representation 106 generated by the service 103 may be based on text or other non-image data, such as a message associated with the service request, a user prompt or dialogue or other phrases, paragraphs or sentences associated with the service request 101 used in a user prompt, or the like, or any combination thereof. In this case, a first hash 108 may be generated from the text or non-image data of the representation 106 using a hashing algorithm capable of generating a hash from the text or non-image data of the representation. The captured representation 122 of the representation 106 presented on the display 104a of the user device 104 may be further processed using, but not limited to, optical character recognition, for example, to generate the text or non-image data of the representation 106 at the second user device 112. The second user device 122 then generates a second hash 111 by using the same or substantially similar hashing algorithm used by the service 103 on the generated text or non-image data. For example, a hashing algorithm is used to generate the first hash 108 from the representation 106, and the same or substantially similar hashing algorithm is used to generate the second hash 111 from the captured representation 122. As an example, when the presentation 106 includes text or non-image data, the first hash 108 and the second hash 111 may be generated using a hash algorithm associated with hashing the text or non-image data, the hash algorithm being based on, but not limited to, one or more hash algorithms from, for example, the following group: a non-perceptual hash algorithm; a secure hash algorithm (e.g., SHA-0, SHA-1, SHA-2, SHA-3, SHA-256, etc.); a cryptographic hash algorithm (e.g., MD4, MD5, MD6, etc.); a cryptographic hash function; a Blake-based hash algorithm; a Poseidon-based hash algorithm; a one-way hash algorithm; any other hash algorithm configured to generate a unique hash from text and / or non-image data, or any combination thereof. When hashing text or non-image data, a distance metric may not be necessary, and instead a comparison for an exact match between the first hash 108 and the second hash 111 may be performed. When there is an exact match, then the first hash 108 and the second hash 111 are deemed to match and therefore substantially match. When there is no exact match between the first hash 108 and the second hash 111 , then the first hash 108 and the second hash 111 are considered to not match, and therefore do not substantially match.

[0074] For example, the service 103 is configured to generate a first hash 108 from a user-prompted text associated with the service request 101 using a non-perceptual hashing algorithm, wherein the presentation 106 of the text is an image of the user-prompted text, which is sent to the user device 104 in the service request 101. During the VIC process, an image of the text of the presentation 106 displayed on the display 104a of the user device 104 is captured by a user device 112, wherein the user device 112 uses optical character recognition (OCR) on the captured image to generate the text. The second device 112 generates a second hash 111 from the OCR'ed text using a non-perceptual hashing algorithm. The first hash 108 and the second hash 111 are then compared. The user-prompted text may include unique portions, so that a unique first hash 108 is generated for each user. The text may also require a hash of a random number and a signature, which are included in the digital code 107 sent from the service 103 to the user device 104.

[0075] Figure 1b Another example VIC system and process 120 for use in verifying the authenticity of a service request 101 transmitted from a service 103 to a user device 104 of a user 105 over a communication channel 102 is shown. Figure 1a The VIC system and process 100 is further modified because the VIC process is performed by a user device 104. In this example, the user device 104 performs the VIC process associated with the service request 101 based on displaying a presentation 106 and a digital code 107 associated with the service request 101 to a user 105 on a display 104a of the user device 104.

[0076] For example, the VIC process is performed at the user device 104, which indicates whether the service request 101 is legal / authentic based on capturing the presentation 106 and the digital code 107 displayed on the user device 104. The user device 104 can be any type of device capable of capturing the display 104a of the user device 104. For example, when the presentation 106 and / or the digital code 107 are displayed on the display 104a, the user device 104 can perform a screen capture or screenshot of the display 104a of the user device 104. The user device 104 uses the captured presentation and the captured digital code to verify whether the service request 101 is legal / authentic. In the example, the user device 104 displays a notification on the user device 104 based on whether the VIC service request 101 is legal / authentic. The user 105 can view the notification displayed on the user device 104 and continue to act on the service request 101 accordingly.

[0077] In this example, the user device 104 includes a VIC application and / or a VIC device operating on the user device 104, which is configured to execute on the user device 104 as follows: Figure 1aFor example, the user device 104 includes functionality for executing the VIC process in a trusted environment of the user device 104. The VIC process can be implemented based on, but not limited to, for example, a VIC application operating on the user device 104 in a sandbox or trusted environment of the user device 104, a VIC device or component operating on a trusted hardware module of the user device 104, and / or functionality of the VIC process operating within an operating system of the user device 104 or an operating system component of the user device 104, or any combination thereof. This provides the advantage of reducing the risk of the VIC process being maliciously altered and / or changed by a third party, and increases the credibility of the VIC indication output from the VIC process. The VIC process can be implemented in a manner similar to the reference VIC application. Figure 1a The described method operates on the user device 104 in a similar manner.

[0078] Although the digital code 107 is described as being sent with the service request 101 by the service 103 to be presented to the user 105 on the display 104a of the user device 104, this is by way of example only, and the VIC process is not limited thereby, and those skilled in the art will recognize that the digital code 107 may be an encryption of the first hash 108, the digital signature 109, the seed 110 and / or other required information using a digital encryption algorithm, which may be sent to the user device 104 with the service request 101. In this case, the digital code 107 does not need to be presented on the display 104a of the user device 104, but the VIC process of the user device 104 is instead configured to decrypt the received digital code 107 based on the corresponding digital decryption algorithm. Therefore, the user device 104 can retrieve the first hash 108, the digital signature 109 and the seed 110 by decrypting the received digital code 107. Only the presentation 106 needs to be presented to the user 105 via the display 104a of the user device 104. The VIC process includes the user device 104 capturing the presentation 106 displayed on the display 104a of the user device 106 as the captured presentation 122, wherein the capturing uses, for example, a screenshot or screen capture function of the display 104a of the user device 104. Once the user device 104 has the captured presentation and the decrypted first hash 108, digital signature 109 and / or seed 110, the VIC process proceeds with the reference presentation 122. Figure 1a Continue in a similar manner as described.

[0079] Although the VIC process can be Figure 1a and Figure 1bThe VIC process described in the foregoing is implemented on the user device 104 or the second user device 112, but this is only by way of example, and the VIC process is not limited thereto. Those skilled in the art should recognize that the VIC process can be implemented in, but not limited to, the following ways: on a device, other hardware, software, on a chip, on a security dongle or a universal serial bus (USB) device, on a smart card with a chip, in a trusted execution environment (TEE) of the user device 104 and / or the second user device 112, in a trusted platform module (TPM) of the user device 104 and / or the second user device 112, in a secure environment of the user device 104 and / or the second user device 112; and / or any other device, hardware and / or software component that can provide a sandbox, tamper-proof, secure and / or trusted environment.

[0080] The example VIC system 100 or 120 enables a user 105 of a user device 104 to easily authenticate or increase their perception of trust in a service request 101 sent from a service 103 and received at the user device 104 when the user 105 is using a service client associated with a service 103. The VIC system and system 100 provide the following advantages: a user-friendly authentication technology solution for each service request 101; minimizes the risk of confusion / uncertainty for the user 105 to trust each service request 101; minimizes the user attention required to authenticate each service request 101; and is sufficiently robust and reliable to minimize the security risk of each service request 101 being spoofed and the user 105 accidentally / unintentionally sharing sensitive user information or elevated credentials with malicious actors instead of the service 103.

[0081] Figure 1c is shown with reference to Figure 1a and Figure 1b Schematic diagram of an example of a user prompt 130 associated with a service request 101 of the described VIC process. Figure 1a and Figure 1b106. In this example, it is assumed that the service request 101 includes a user prompt 130 as a presentation 106. The user prompt 130 may be generated at the service 103 and represented as an image, from which the first hash 108 is generated, wherein the image is sent from the service 103 to the user device 104 as the presentation 106 in the service request 101. Alternatively, the user prompt 130 may be generated at the service 103, and the image is rendered and used to generate the first hash 108 at the service 103, but data representing instructions, text data, and / or markup language (which may be used by the user device 104 in rendering the image of the user prompt 130 on the display) may be sent in the service request 101 as the presentation 106. In any case, once the service request 101 is received at the user device 104, the presentation 106 is displayed on the display 104a of the user device 104 as the user prompt 130. Similarly, the digital code 107 sent in the service request 101 is also displayed on the display 104a of the user device 104 as a digital code image. In this example, the digital code image is a QR code that represents an encoding of the digital code 107, including the first hash 108, the digital signature 109 and the seed 110, and / or any other required information for verifying the first hash 108 and / or the digital signature 109 by the user device 104 and / or the second user device 112.

[0082] In this case, as in reference Figure 1a and Figure 1b During the described VIC process, the user-prompted presentation 106 and the digital code image displayed on the display 104a of the user device 104 are captured by the second user device 112, such as Figure 1a , or executed by user device 104, such as Figure 1b. The captured presentation 122 is formed by: a first image portion 124, which represents a captured user prompt image of the user prompt 130 displayed on the display 104a, and a second image portion 126, which represents the captured digital code image displayed on the display 104a. For example, the captured presentation 122 is cropped to separate the first image portion 124 and the second image portion 126 to ensure that the captured user prompt image is processed by the hash algorithm 132 while ensuring that the captured digital code image is not processed by the hash algorithm 132. Hereinafter, the first image portion 124 is referred to as the captured user prompt image 124, and the second image portion 126 is referred to as the captured digital code image 126. The captured presentation 122 is processed to detect the user prompt 130 that does not include the digital code image, and is cropped to generate a captured user prompt image 124, which represents the user prompt 130 displayed on the display 104a, but does not include the digital code image displayed on the display 104a. The captured presentation 122 is further processed to detect a digital code image that does not include the user prompt 130 and is cropped to generate a digital code image 126 representing the digital code image displayed on the display 104a, wherein the captured digital code image 126 can be decoded using a decoding algorithm 128 (e.g., a QR decoding algorithm).

[0083] In this example, the captured presentation 122 of the user prompt 130 and the digital code image includes a captured user prompt image 124 representing the user prompt image of the user prompt 130 and associated with the first hash 108, and a captured digital code image 126 associated with the digital code 107. The captured digital code image 126 is, for example, a captured QR code representing the digital code 107. The captured digital code image 126 is read and decoded by the QR reader / software 128 on the second device 112 to output the first hash 108 (e.g., v1), the digital signature 109 (e.g., σ1), and the seed 110 (e.g., S1) and / or other information required to verify the first hash 108 and / or the digital signature 109.

[0084] The second hash 111 (eg, v2) is generated by applying a hash algorithm 132 to the captured user prompt image 124 representing the user prompt 130. For example, as shown in FIG. Figure 1a or Figure 1b As described, the VIC process may use a hash algorithm 132 to generate a hash from an image (e.g., a perceptual hash algorithm). The captured user-prompted image 124 is processed by the hash algorithm 132 to generate a second hash 111 (e.g., v2) associated with the representation 106 of the user-prompted image generated by the service 103 when generating the first hash 108. As described with reference to Figure 1a and Figure 1b As described, the first hash 108 (e.g., v1) and the second hash 111 (e.g., v2) are compared to determine whether they at least substantially match each other. Likewise, the digital signature 109 associated with the first hash 108 is verified to be authentic based on the seed 110 (e.g., S1). If the first hash 108 and the second hash 111 substantially match and the digital signature is authentic, a VIC indication indicating that the service request 101 is legitimate or authentic is sent. If the first hash 108 and the second hash 111 do not substantially match and / or the digital signature is not authentic or invalid, a VIC indication indicating that the service request 101 is not legitimate or authentic is sent.

[0085] Although the example VIC system and process 100 or 120 is described with reference to the service 103 and the user 105 using the service 103 via the user device 104, this is by way of example only and the VIC system and process 100 or 120 is not limited thereby. Those skilled in the art will recognize that the VIC system / process 100 or 120 and / or as referred to herein may be used in any manner. Figures 2a to 4 What is described may be used in client-server implementations and / or application-specific implementations (e.g., an app in a smart device), and by, but not limited to, for example, software-as-a-service applications, email applications, retail services / websites and / or applications, any subscription services / applications, online banking services / applications, social media platforms / applications, information technology (IT) department applications, enterprise / organization services and / or applications, operating systems and / or mobile provider applications / services, and / or any other trusted source / service that a user 105 may use via their user device 104 and / or in response to a service request from a service 103 accessible to the user 105.

[0086] For example, in a client-server implementation, the VIC system 100 or 120 includes a VIC server application associated with a service 103 operating on a server of a centralized or distributed server system and / or a cloud platform, etc., wherein a service client associated with the service 103 operates on a user device 104 of a user. Figure 1a or Figure 1b As described, the VIC server application associated with the service 103 can perform the corresponding service-side (or server-side) steps of the VIC process 100 or 120 at the server operating the service 103 (e.g., prompting and generating the first hash 108, etc.). Figure 1a or Figure 1bAs described, the VIC client application can perform corresponding steps of the VIC process 100 or 120 on the second user device 106 or the user device 104 (e.g., capturing the user prompt and generating the second hash 111, etc.), and / or be incorporated into a service client on the user device 104. Application-specific implementations may include services that use the VIC application to perform service-side operations of the VIC process 100 / 120, where the corresponding VIC application operates on the user device 104 and / or the second device 112, as described in reference to Figure 1a to Figure 1c Described and / or as described herein.

[0087] Figure 2a is shown for processing by service 103 or Figure 1a to Figure 1c Flowchart of an example service VIC process 200 used by a server of service 103. Where applicable, reuse Figure 1a and Figure 1b The service VIC process 200 performed by the service 103 or a server processing the service 103 includes the following steps:

[0088] In step 202, a presentation 106 associated with a service request 101 requesting a user action at a user device 104 of a user 105 is generated, and a digital code 107 associated with the service request 101 is generated. The digital code 107 includes a first hash 108 associated with the presentation, a digital signature 109, and a seed 110 for verifying the digital signature. The presentation 106 may be an image associated with the request for the user action, which may include, but is not limited to, for example, a user prompt, an interactive user prompt, a GUI element, and / or any other dialog or image associated with the first hash 108. The user action may include, but is not limited to, for example, a request for a user to enter user credentials; a request for a user to enter sensitive information; a request for a user to download and / or install software and / or software updates; a request for a user to open an attachment and / or sign an attachment; any other request for a user to enter data and / or provide permission for the user device 104 to perform a function or process.

[0089] In an example, presentation 106 including images, interactive prompts and / or GUI elements, etc. is generated by service 103, which may simulate rendering the images, interactive prompts and / or GUI elements at service 103 and generating images associated therewith based on the rendering.

[0090] In step 204, a service request 101 for user action / information is sent to the user device 104. The service request 101 includes data representing presentation 106 and a digital code 107 for presentation on a display 104a on the user device 104 to authenticate the service request 101 from the service 103 by the user 105.

[0091] Alternatively, steps 202 and 204 of the service VIC process 200 may be further modified, wherein the service request 101 may be sent to the user device 104, and the user device 104 may render an image of the service request 101 on a display of the user device 104. The user device 104 may share or send a captured image of the displayed service request 101 to the service 103, and the service 103 then generates a first hash 108 from the captured image. In addition, the service 103 generates a digital code 107 including the first hash 108, the digital signature 109, and / or the seed 110. The digital code 107 is sent to the user device 104 to be displayed on the user device 104 as a digital code image (e.g., a QR code or other digital code image).

[0092] Figure 2b is shown for Figure 1a to Figure 1c Flowchart of an example user device VIC process 210 used by the user device 104 of FIG. 104. Where applicable, reuse of Figure 1a and Figure 1b The user device VIC process 210 performed by the user device 104 includes the following steps:

[0093] In step 212, a service request 101 for a user action by a user 105 at a user device 104 is received from a service 103. In an example, the service request 101 for the user action includes data representing a representation 106 (e.g., an image) and a digital code 107 for display on the user device 104. The representation 106 and the digital code 107 are for use in authenticating the service request 101 from the service 103. Figures 1a to 2a As described in , service 103 generates a first hash 108 of representation 106 and generates a digital code 107 from at least the first hash 108 and a digital signature 109 associated with the first hash 108. Other information may be included in the digital code 107, such as a seed 110 associated with the first hash 108 and / or information for verifying the digital signature 109.

[0094] Representation 106 may be an image generated by service 103 , and digital code 107 includes data representing a first hash 108 of the image and a digital signature 109 associated with first hash 108 and / or other information used to verify first hash 108 and / or digital signature 109 , such as, for example, seed information 110 .

[0095] In step 214, presentation 106 and digital code 107 are displayed on user device 104. When presentation 106 includes an image associated with first hash 108, the image is displayed on display 104a of the user device. In an example, digital code 107 is displayed as a digital code image along with the display of the image of presentation 106 on user device 104. In another example, digital code 107 may be displayed before or after presentation 106 is displayed on user device 104.

[0096] In step 216, a visual integrity check (VIC) process is performed on the service request 101 using the representation 106 and the digital code 107. For example, as shown in FIG. Figure 1a to Figure 1c The described VIC process(es) may be performed by the user device 104 of the user 105 and / or the user 105 or an external device or second user device 112 that is separate from the user device 104 of the user 105 and accessible by the user 105. In another example, the Figure 2c The VIC process 230 may be executed on the user device 104. Alternatively, the VIC process may be executed on the second user device 112 of the user 105, or may be accessed by the user 105. In either case, the VIC process may output or send a VIC indication indicating whether the presentation 106 and digital code 107 displayed on the user device 104 are legitimate or authentic.

[0097] In step 218, data representing a VIC indication indicating whether the digital code 107 and the image 106 are authentic or legal is indicated or received. In response to the VIC indication indicating that the digital code 107 and the image 106 are authentic or legal (e.g., "Y"), the process proceeds to step 220, otherwise in response to the VIC indication indicating that the digital code and the image are not authentic or legal (e.g., "N"), the process proceeds to step 222.

[0098] In step 220, the user 105 of the user device 104 is indicated that a user action of the service request 101 may be performed. For example, this may include the user device 104 notifying the user 105 of the user device 104 to perform the requested user action in the service request 101. Alternatively, this may include the second user device 112 indicating and / or notifying the user 105 to perform the requested user action in the service request 101 on the user device 104.

[0099] In step 222, the user 105 of the user device 104 is instructed to reject the service request 101 and cancel the execution of the requested user action or not execute the requested user action. For example, this may include the user device 104 notifying the user 105 of the user device 104 not to execute or cancel the execution of the requested user action in the service request 101. Alternatively, this may include the second user device 112 instructing and / or notifying the user 105 not to execute or cancel the execution of the requested user action in the service request 101 on the user device 104.

[0100] Figure 2c is a flow chart illustrating an example VIC process 230 performed by the user device 104 or an external device or a second user device 112. Figure 1a and Figure 1b The VIC process 230 is, for example, Figure 2b The user device VIC process 210 is performed in step 216, and / or by reference Figure 1a to Figure 1c The VIC process 230 described herein is performed. The VIC process 230 includes the following steps:

[0101] In step 232, a presentation 106 presented on a display 104a of a user device 104 used by a user 105 is captured to form a captured presentation 122. The presentation 106 includes data representing a first hash 108 generated by a hash algorithm at the service 103. For example, the presentation 106 may be an image associated with a service request 101 sent from the service 103 and / or data representing the service request 101 for rendering as a user prompt, an image, and / or an interactive user prompt on the display 104a of the user device 104. The presentation 106, when displayed or rendered on the display 104a of the user device 104, may be captured via an imaging function, such as, for example, an imaging component or camera of an external device or a second user device 112, a screenshot or screen save function of the user device 104a for capturing one or more portions of the display 104a of the user device 104. The resulting captured image of the presentation 106 and / or the digital code 107 forms the captured presentation 122.

[0102] The captured representation 122 may include data representing the representation 106 of the service request 101 and also the digital code 107 when displayed on the display 104a of the user device 104. This means that the captured representation 122 includes data representing at least the first hash 108. When the digital code 107 is also captured, then the captured representation 122 also includes data representing at least the first hash 108, the digital signature 109 associated with the first hash 108, and the seed 110. The digital code 107 may also include additional information associated with the type of hash algorithm or hash parameters / key used by the service 103 to generate the first hash 108, and / or the type of hash algorithm that the VIC process 230 should use to generate the second hash 111 based on the captured representation 122.

[0103] In step 234, a second hash 111 is generated from the captured representation 122. In the example, the representation 106 represents an image that is applied to a hash algorithm by the service 103 to generate a first hash 108 of the image. The hash algorithm is capable of generating the first hash 108 from the image of the representation 106. The same or substantially similar hash algorithm is used in step 234 to generate the second hash 111 from the image representing the captured representation 122. The hash algorithm can be any type of hash algorithm for generating a hash from an image, such as, for example, as described in reference to FIG. Figure 1a to Figure 1c The perceptual hashing algorithm described herein, etc. In another example, representation 106 may represent text data or non-image data associated with service request 101, which may include specific user data, wherein service 103 generates first hash 108 from the text data or non-image data using a non-image-based hashing algorithm. In this example, captured representation 122 includes an image of text data or non-image data representing service request 101, wherein captured representation 122 is OCR processed to generate text data or non-image data (e.g., binary data), wherein the generated text data is applied to a hashing algorithm to generate a second hash 111, wherein the hashing algorithm is the same as or substantially similar to the algorithm used by service 103. The hashing algorithm may be a non-image hashing algorithm, such as, for example, as described in reference Figure 1a to Figure 1c Describes the text or data hashing algorithm.

[0104] In step 236, the digital code 107 is read from the display 104a of the user device 104 or from the captured presentation 122. Once read, the digital code 107 is decoded to form data representing the first hash 108, the digital signature 109 associated with the first hash 108, and the seed 110 associated with the digital signature 109. The digital code 107 may include further information associated with the first hash 108 and / or the digital signature 109, etc. (e.g., data representing the type of hash algorithm used to generate the first hash 108 and / or other information required by the second device 112 or the user device 104 when performing the VIC process 230).

[0105] In step 238, it is determined whether the digital code 107 is associated with the second hash 111 based on comparing the first hash 108 with the second hash 111. If the first hash 108 and the second hash 111 at least substantially match each other, it is determined that the second hash 111 is associated with the digital code 107. This is because the digital code 107 includes data representing the first hash 108. When the hash algorithm used to generate the first hash 108 and the second hash 111 is an image-based hash algorithm (e.g., a perceptual hash algorithm), the comparison between the first hash 108 and the second hash 111 can be based on calculating the distance between the first hash 108 and the second hash 111. If the calculated distance is within a predetermined distance threshold or error threshold, the first hash 108 and the second hash 111 are considered to be at least substantially matched. This can also be applied to non-image-based hash algorithms, such as text hash algorithms, which can obtain an exact match (i.e., the same), in which case the distance between the first hash 108 and the second hash 111 is 0 and will be within or below the predetermined distance threshold or error threshold. If the distance is above a predetermined distance threshold or error threshold, then it is determined that the first hash 108 and the second hash 111 do not substantially match. For a text hash algorithm, this may be a strict threshold, where when the first hash 108 and the second hash 111 are the same, they are a match, and where when the first hash 108 and the second hash 111 are not the same, they are a mismatch. In addition, using a digital signature verification algorithm, based on the seed 110 read from the digital code 107, it is determined whether the digital signature 109 associated with the first hash 108 is valid or authentic.

[0106] In step 240 , in response to digital signature 109 of digital code 107 being legitimate or valid and the first hash and the second hash at least substantially matching, a VIC indication is sent indicating that digital code 107 and presentation 106 displayed on user device 104 are legitimate or authentic.

[0107] In step 242, in response to a) the digital signature 109 of the digital code 107 being illegal or invalid; and / or b) the first hash 108 and the second hash 111 at least substantially mismatching, a VIC indication is sent indicating that the digital code 107 or representation 106 displayed on the user device 104 is illegal, unauthentic or invalid.

[0108] The VIC process 230 may be performed by an external device, such as the second user device 112 and / or the user device 104 , during the user device VIC process 210 , which may proceed to step 218 using an indication or result of steps 240 or 242 .

[0109] Further modifications may be applied wherein in step 232, the captured representation 122 includes a first image 124 representing the representation 106 and associated with the first hash 108 and a second image 126 representing the digital code 107. In step 234, generating a second hash 111 of the captured representation 122 includes applying a hash algorithm to the first image to output the second hash 111. This may include performing OCR on the first image 124 if the representation 106 represents text data, and applying the hash algorithm to the generated text data to output the second hash 111. In step 236, reading the digital code from the display 104a may also include processing the second image 126 using a digital decoding algorithm associated with the digital code 107 (e.g., when the digital code 107 is a QR code, a QR reading algorithm may be applied to decode the encoded information within the digital code 107).

[0110] Figure 3a is shown for Figure 1a 100 is a signal flow diagram of an example VIC signal flow 300 of VIC being executed by a user 105 in response to a service request 101 for a user action / information from a service 103. Figure 1a to Figure 1c In this example, the VIC signal flow 300 uses the same reference numerals as in the reference Figure 1a The service 103, the user device 104 and the external device or the second user device 112 are executed. The service 103 can be implemented as shown in FIG. Figure 2a as described and / or as referenced Figure 1a The functions of the service VIC process 200 described above. The user device 104 may implement the Figure 2b as described and / or as referenced Figure 1a The external device or second user device 112 may implement the functions of the user device VIC process 210 as described in reference Figure 2c as described and / or as referenced Figure 1aThe functionality of the VIC process 230 described. In this example, the user device 104 operates a service client application that communicates with the service 103. The service client application provides functions and / or services associated with the service 103 to the user 105. The service 103 operates remotely on a server or cloud platform, etc., and communicates with the service client application of the user device 104 through the communication channel 102 to provide services to the user 105. In this example, the service 103 requires the credentials of the user 105, and therefore issues a service request 101, which represents a prompt for a user action associated with the user 105 to enter the user credentials to log in to the service 103 and / or use some security or subscription functions associated with the service 103. In this case, before the user 105 enters the user credentials, the user 105 wants to know whether the service request 101 believed to be received from the service 103 is legitimate. If it is legitimate, the user 105 can enter their user credentials with more confidence knowing that the service request 101 is authentic and legitimate. Otherwise, the user 105 may reject the service request 101 and notify the service 103 or ignore the service request 101 until a genuine or legitimate service request 101 is received and verified through the VIC process as described herein. In this example, the VIC signal flow 300 includes the following operations:

[0111] In operation 302, the service 103 determines that a service request 101 is required, for example, the service requires user credentials in order for the service 103 to log a user into the service 103 and / or provide secure or subscription-based functionality to a corresponding service client application operated by the user 105 on the user device 104. The service 103 prepares the service request 101 including a user prompt for the user credentials. In preparing the service request 101, the service 103 generates a representation 106, such as data representing, for example, an image of the user prompt, and a first hash (e.g., v1) of the representation or image. The service 103 also generates a digital signature 109 (e.g., σ1) associated with the first hash 108 based on a seed (e.g., s1) 110 associated with the digital signature 109 or a digital signature algorithm.

[0112] In operation 304, the service 103 invokes a user prompt for a user action to present to the user 105 on the display 104a of the user device 104, which includes sending a service request 101 to the user device 104 via the communication channel 102, the service request 101 including data representing the presentation 106 (e.g., an image or instructions for rendering an image) and a digital code representing the first hash 108, the digital signature 109, and the seed 110.

[0113] In operation 310 , the user device 104 receives the service request 101 representing data of the presentation 106 (eg, an image or instructions for rendering an image) and a digital code (eg, DC{s1, v1, σ1}) representing the first hash 108 , the digital signature 109 , and the seed 110 .

[0114] In operation 312 , the user device 104 displays data representing the presentation 106 (eg, the image p1 associated with the user prompt) and the digital code 107 on the display 104 a of the user device 104 .

[0115] In operation 316, the user device 104 requests to perform a VIC process based on displaying the presentation 106 and the digital code 107 on the display 104a of the user device 104. In this example, in response to the request, the user 105 operates a VIC application and / or a VIC device on an external device or a second user device 112 (e.g., mobile). The VIC application and / or the VIC device may operate in a trusted environment and / or a sandbox environment on the second user device 112 (e.g., mobile). The VIC process operations performed during operation 316 are based on the reference Figure 2a The VIC process 230 includes the following operations:

[0116] In operation 330, the user 105 opens the VIC application and / or VIC device to execute the VIC process on the second device 112. The VIC application and / or VIC device operates in a trusted or tamper-proof environment on the second user device 112, which reduces the risk of the VIC process being corrupted or maliciously altered by an unauthorized third party, etc.

[0117] In operation 332 , the VIC process captures the displayed representation 106 (eg, p1 ) displayed on the user device 104 to generate a captured representation 122 (eg, p2 ), and also reads the displayed digital code 107 (eg, DC{v1,σ1,s1}).

[0118] In operation 334 , the VIC process computes a second hash 111 (eg, v2 ) using the captured representation 122 (eg, p2 ).

[0119] In operation 338, the VIC process calculates the distance between the first hash 108 (e.g., v1) and the second hash 111 (e.g., v2), and determines whether the distance between them is acceptable. For example, if the first hash 108 and the second hash 111 at least substantially match each other, the distance is acceptable, where they substantially match each other when the distance is equal to or below a predetermined distance threshold or error threshold set by the service 103 and / or the user 105. When the first hash 108 and the second hash 111 are the same, the distance is 0 and is considered an exact match.

[0120] In operation 339 , when the distance between the first hash 108 and the second hash 111 is acceptable (eg, they substantially match), the VIC process performs verification on whether the digital signature 109 (eg, σ1 ) using the seed 110 (eg, s1 ) is valid or legal.

[0121] In operation 340, if the distance is acceptable and the digital signature 109 is verified as valid or legal, the service request 101 is acceptable and legal, and the VIC application and / or VIC device sends a positive indication to the user device 104 and / or indicates (e.g., displays) a positive indication that the service request 101 associated with the presentation 106 and the digital code 107 is legal or authentic and can be trusted. Otherwise, if the distance is not acceptable or the digital signature 109 is verified as invalid or illegal, the service request 101 is unacceptable, illegal or unauthentic, and the VIC application and / or VIC device sends a negative indication to the user device 104 and / or indicates (e.g., displays) at the second user device 112 that the service request 101 associated with the presentation 106 and the digital code 107 is illegal or unauthentic and should not be trusted.

[0122] At the user device 104, in operation 318, if the second user device 112 (e.g., mobile) sends a positive indication or displays a positive indication in operation 340, the user device 104 allows the user 105 to perform the prompt for the user action in the service request 101. Alternatively or additionally, the user device 104 may notify the user 105 that the service request 101 is trusted, and therefore the user 105 may perform the user action associated with the service request 101. In this case, the user 105 may enter their user credentials to submit to the service 103 to use security and / or subscription functions, etc. If the second user device 112 sends a negative indication or displays a negative indication in operation 340, the user device 104 may prohibit the user 105 from performing the prompt for the user action in the service request 101. Alternatively or additionally, the user device 104 may notify the user 105 not to perform the user action in the service request 101. In this case, the user 105 may refuse to enter their user credentials. The user device 104 may send a negative confirmation or notification indicating the rejection of the service request 101 to the service 103, wherein the service 103 may generate a new service request 101, as described in operation 302. If the service request 101 is an unprompted service request from the service 103, the user 105 may wait until another service request 101 is issued by the service 103 and perform a VIC on the other service request 101.

[0123] In operation 320 , the user device 104 performs a user action based on the user input in response to the execution service request 101 .

[0124] In operation 324 , the user device 104 sends a confirmation or notification to the service 103 that the user action associated with the service request 101 has been performed.

[0125] Figure 3b is shown for Figure 1b 103 for a user action / information request 101 from a service 103. Figure 1a to Figure 1c In this example, the VIC signal flow 350 is as shown in the reference Figure 1b The VIC process described above is implemented at the user device 104 to modify the example VIC signal flow 300. The service 103 may be implemented as described in reference Figure 2a as described and / or as referenced Figure 1b The functions of the service VIC process 200 described above. The user device 104 may implement the Figure 2b as described and / or as referenced Figure 1a The functionality of the user device VIC process 210 described herein, and also as described in reference Figure 2c , Figure 3a as described and / or as referenced Figure 1b The VIC signal flow 350 uses the service 103 and / or the user device 104 to perform operations 352, 354, 360 / 362, 366, 368, 370, and 374. In this example, the operations 352 and 354 performed at the service 103 are the same as those described in reference Figure 3a The operations 302 and 304 performed by the service 103 are substantially the same as described above. The operations 360 and 362 performed at the user device 104 are the same as described above with reference to Figure 3a The operations 310 and 312 described as being performed at the user device 104 are substantially the same.

[0126] In operation 366, the user device 104 performs a VIC process based on the presentation 106 and the digital code 107 displayed on the display 104a of the user device 104. In this example, the user device 104 includes a VIC application and / or a VIC device (e.g., hardware or software capable of executing a VIC application or its process in a trusted environment). For example, the VIC application can operate in a trusted environment and / or a sandbox environment on the user device 104. The trusted environment can be a trusted platform module, a trusted execution environment, a secure execution environment and / or a sandbox environment, and / or a part or component of the operating system of the user device 104. The VIC process operations performed during operation 366 are based on reference Figure 2a VIC Process 230, and also Figure 3a VIC Process 316. For example, Figure 3a Operations 330 - 340 of the VIC process 316 are performed on the user device 104 . Figure 3a Operation 332 may be further modified by the user device 104 capturing the display using a screenshot or screen grab function rather than a camera or the like, but otherwise operations 334 to 340 are the same or similar, but are implemented on the user device 104 .

[0127] At the user device 104, in operation 368, if the VIC application, apparatus, and / or process 366 sends a positive indication in operation 340 or indicates, for example, displays a positive indication on the display 104a of the user device 104, the user device 104 allows the user 105 to perform the prompt for the user action in the service request 101. Alternatively or additionally, the user device 104 may notify the user 105 that the service request 101 is trusted, and therefore the user 105 may perform the user action associated with the service request 101. In this case, the user 105 may enter their user credentials to submit to the service 103 to use security and / or subscription functions, etc. If the VIC application, apparatus, and / or process 366 sends a negative indication in operation 340 or displays a negative indication on the display 104a of the user device 104, the user device 104 may prohibit the user 105 from performing the prompt for the user action in the service request 101. Alternatively or additionally, the user device 104 may notify the user 105 not to perform the user action in the service request 101. In this case, the user 105 may refuse to enter their credentials or cancel the prompt to enter their user credentials, etc. The user device 104 may send a negative confirmation or notification to the service 103 indicating the rejection of the service request 101, wherein the service 103 may generate a new service request 101, as described in operation 302. If the service request 101 is an unprompted service request from the service 103, the user 105 may wait until another service request 101 is issued by the service 103 and perform a VIC on the other service request 101.

[0128] In operation 370 , the user device 104 performs a user action based on the user input in response to the execution service request 101 .

[0129] In operation 374 , the user device 104 sends a confirmation or notification to the service 103 that the user action associated with the service request 101 has been performed.

[0130] Figure 4 is a flow chart illustrating another example VIC system 400 for use by a user 105 having a user device 104 (e.g., a laptop computer) and a second user device 112 (e.g., a mobile communication device) for performing VIC on a request from a service 103 implemented on a server or cloud platform, etc. Figure 1a to Figure 1c In this example, the VIC system 400 includes a laptop computer 104 being operated by a user 105 and a mobile device 112 (eg, an external device) that can also be operated by the user 105 .

[0131] In this example, a user 105 receives a service request 101 from a service 103 or from or via an email service in the form of an email message requesting user credentials, account information, and / or sensitive data. The email message may be sent from an email service of an email provider, which may be associated with information technology (IT) support for an organization representing the user 105. Once the email message is prepared by the service 103 or email service, the service 103 or email service generates a presentation 106 of the email message. In this example, the email service renders the email message in a form that it would appear on the display 104a of the laptop computer 104. The service 103 or email service generates a rendered image of the email message, which forms the presentation 106. The service 103 or email service then applies, for example, a type-aware hash algorithm 415 to generate a first hash 108 (e.g., 292cc7948c2ac144 ...) representing the presentation of the rendered email message. The service 103 or email service also generates a digital signature 109 (e.g., 36f776a69636f631 ...) associated with the first hash 108. The service 103 or email service encodes at least the first hash 108 and the digital signature 109 using a digital encoding algorithm 417 (e.g., a QR encoding algorithm) to generate a digital code 107 (e.g., a QR code). In this case, the encoding algorithm 417 is an image encoding algorithm for encoding the first hash 108 and the digital signature 109 into a code that can be read and decoded from a display or graphic. The email service sends an email message (e.g., a service request 101), and when the user 105 opens the email message using an email client on a laptop computer, the email message will be displayed on the display 104 of the laptop computer 104 together with the digital code 107 based on the presentation 106.

[0132] In step 401, when the user device 104 of the user 105 receives the email message, the user device 104 presents a representation 106 of the email message on the display 104a of the user device 104 in a manner similar to the manner in which the service 103 or the email service renders or generates the representation 106 of the email message when generating the first hash 108. The user device 104 also displays a digital code 107 on the display 104a of the user device 104. The digital code 107 can be displayed outside the area of ​​the representation 106 of the email message, or displayed in a solid color area / region of the representation 106 of the email message. In order to proceed safely, i.e., perform VIC on the email message, the user 105 operates a mobile device 112 with a camera. The mobile device 112 has stored thereon a visual integrity application (e.g., a VIC mobile app) operating in a sandbox or trusted environment.

[0133] In step 402, user 105 uses the VIC mobile app of mobile device 112 to instruct the camera of mobile device 112 to capture laptop display 104a to generate captured representation 122. Captured representation 122 includes email message image 124 and digital code image 126.

[0134] In step 403, the VIC mobile app generates a second hash 111 (e.g., 292cc7948c2ac144 ...) from the captured email message image 124 of the presentation 122 using a hash algorithm. The hash algorithm used by the VIC mobile app is at least substantially similar to or identical to the hash algorithm 415 used by the service 103 or email service to generate the first hash 108. In this case, before using the hash algorithm, the VIC mobile app may crop the captured presentation 122 to generate the email message image 126 by removing the digital code image 126 of the code image 126 representing the digital code 107 that may be captured by the camera of the mobile phone 112 and / or replacing the cropped digital code image 126 with, for example, an image portion that matches the background surrounding the digital code image 126 of the digital code 107. This is to minimize the impact on the image of the digital code image 126 when performing the hash algorithm on the email message image 124 to generate the second hash 111.

[0135] In step 404, the VIC mobile app reads a digital code image 126 (e.g., a QR code), which includes a first hash 108 (e.g., 292cc7948c2ac144 ...) and a digital signature 109 associated with the first hash 108 (e.g., 36f776a69636f631 ...). Given that the hash algorithm is a one-way function that produces a unique first hash 108, this ensures that the same QR code cannot be reused for another prompt or email message to the user 105, which helps prevent replay attacks. In addition, the digital code 107 may also include a unique seed associated with the digital signature 109 for use in generating / verifying the digital signature 109, so that the digital signature 109 is different for different prompts / email messages. This further helps prevent replay attacks because it makes each QR code unique and different. The digital code 107 and / or first hash 108 associated with the alert or service request (e.g., email message) is unique and is pre-generated by the service 103 or email service (e.g., a trusted source / sender) when the email message, alert, service request that requires a VIC is specifically created.

[0136] In step 405, the VIC mobile app verifies whether the first hash 108 and the second hash 111 match each other, and whether the digital signature 109 is legal and / or valid. When the distance (e.g., Hamming distance, Euclidean distance, similarity, etc.) between the first hash 108 and the second hash 111 is lower than a predetermined distance threshold or error threshold, the first hash 108 and the second hash 111 are considered to match each other. The distance between the first hash 108 and the second hash 111 can be calculated according to a distance metric applicable to the hash algorithm used to generate the first hash 108 and the second hash 111. The VIC mobile app uses a digital signature verification algorithm to verify whether the digital signature 109 is legal and valid based on a seed, which may be included in the digital code 107.

[0137] In step 406, the VIC mobile app notifies the user 105 whether the received email message is legitimate. In this case, a positive VIC indication 406a (e.g., a check mark) is presented to the user 105 on the display of the mobile device 112 because both the first hash 108 and the second hash 111 match and the digital signature 109 is also verified as legitimate or valid. However, when the first hash 108 and the second hash 111 do not match (e.g., the distance is above a predetermined distance threshold) and the digital signature 109 is also verified as illegal or invalid, the VIC mobile app notifies the user 105 whether the received email message is illegitimate. In this case, a negative VIC indication 406a (e.g., a cross) may be displayed to the user 105 on the display of the mobile device 112.

[0138] In step 406, when the VIC mobile app provides a positive VIC indication that the email message is legitimate (e.g., the first hash 108 and the second hash 111 match, and the digital signature 109 is valid), the user can confidently enter their user credentials (e.g., their login name and password) in response to the email message prompt in step 407.

[0139] Although service request 101 is an email message requesting user credentials, this is by way of example only and service request 101 is not limited thereto, and those skilled in the art should recognize that due to the variety of different services and different possible request styles, service request 101 can be in any format and / or any form, wherein service request 101 can be presented as an interactive user prompt, a dialog box check box, an email message and / or any other type of display requesting the user to perform an action when the application requires it.

[0140] Although the example VIC system 400 is described with respect to an email service / application, this is by way of example only, and the VIC system 400 or the description herein is not limited thereby, and one skilled in the art will recognize that, as referenced herein, Figures 1a to 4The described VIC process / system can be used in client-server implementations and / or application-specific implementations (e.g., apps in smart devices), including use cases such as, but not limited to, software-as-a-service applications, email applications, retail websites and / or applications, any subscription services / applications, online banking applications, social media platforms / applications, IT department applications, enterprise and / or organizational applications, operating system and / or mobile provider applications / services, and / or any other trusted source / service that a user may use via their user device when accessing or using the associated service.

[0141] Figure 5 An apparatus 500 according to some example embodiments is shown, which may include a user device, a mobile or user device, or a service server as described herein. The apparatus may be configured to perform the operations described herein, such as the operations described with reference to any disclosed process. The apparatus includes at least one processor 502 and at least one memory 504 directly or closely connected to the processor 502. The memory 504 includes at least one random access memory (RAM) 504a and at least one read-only memory (ROM) 504b. Computer program code (software) 505 is stored in the ROM 504b. The apparatus may be connected to a transmitter (TX) and / or a receiver (RX). The apparatus may be connected to a user interface (UI) to instruct the apparatus and / or output data. The apparatus may include an imaging device or camera for capturing an image of a display 104a of a user device 104, or functionality for taking a screenshot of a display 104a of the apparatus. In addition, the apparatus includes at least one processor 502, wherein at least one memory 504 stores instructions, such as computer program code 505, which, when executed by at least one processor, causes the apparatus to perform at least the method / process, such as, for example, as described with respect to Figure 1a to Figure 1c , Figure 2a to Figure 2c , Figure 3a to Figure 3b and Figure 4 In addition, in some other implementations, the device includes one or more circuits to enable the device to at least perform the method / process, for example, as described in relation to Figure 1a to Figure 1c , Figure 2a to Figure 2c , Figure 3a to Figure 3b and Figure 4 and its related features disclosed and described.

[0142] As used in this application, the term "circuitry" may refer to one, more, or all of the following:

[0143] (a) hardware circuit implementation only (e.g., implementation in analog and / or digital circuitry only) and

[0144] (b) a combination of hardware circuitry and software, such as (where applicable):

[0145] (i) a combination of analog and / or digital hardware circuits and software / firmware and

[0146] (ii) any portion of hardware processor(s) with software (including digital signal processor(s), software and memory(s) that work together to enable a device such as a mobile phone or server to perform various functions) and hardware circuit(s) and or processor(s), such as microprocessor(s) or portion(s) of microprocessor(s) that requires software (e.g. firmware) for operation, but where software is not required for operation, the software may be absent.”

[0147] This definition of circuitry applies to all uses of the term in this application, including in any claims. As another example, as used in this application, the term circuitry also covers an implementation of only a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. For example and if applicable to a particular claim element, the term circuitry also covers a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0148] Figure 6 A non-transitory medium 600 according to some embodiments is shown. The non-transitory medium 600 is a computer-readable storage medium. It can be, for example, a compact disk (CD), a digital versatile disk (DVD), a universal serial bus (USB) stick, a Blu-ray disk, a secure digital (SD) card, etc. The non-transitory medium 600 stores computer program code that causes the device to perform a method of any of the aforementioned processes, such as disclosed with respect to the flow charts, signal flow diagrams, and related features thereof.

[0149] The names of network elements, protocols and methods are based on current standards. In other versions or other technologies, the names of these network elements and / or protocols and / or methods may be different, as long as they provide corresponding functions. For example, embodiments can be deployed in 2G (second generation) / 3G / 4G / 5G and / 6G or any future generation and later networks and subsequent generations of 3GPP, and can also be deployed in non-3GPP radio networks such as Wi-Fi.

[0150] The memory may be volatile or non-volatile. It may be, for example, RAM, SRAM, Flash memory, FPGA block RAM, SD, DVD, CD, USB stick and Blu-ray disc.

[0151] Unless otherwise stated or clear from the context, a statement that two entities are different means that they perform different functions. This does not necessarily mean that they are based on different hardware. That is, each of the entities described in this specification may be based on different hardware, or some or all of the entities may be based on the same hardware. It does not necessarily mean that they are based on different software. That is, each of the entities described in this specification may be based on different software, or some or all of the entities may be based on the same software. Each of the entities described in this specification may be embodied in the cloud.

[0152] As non-limiting examples, implementation of any of the above blocks, devices, systems, techniques or methods includes implementation as hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof. Some embodiments may be implemented in the cloud.

[0153] It should be understood that what has been described above is what is presently considered to be the preferred embodiment. However, it should be noted that the description of the preferred embodiment is given by way of example only and that various modifications may be made without departing from the scope defined by the appended claims.

Claims

1. An apparatus (112) for visual integrity inspection, comprising: means for capturing a representation (106) on a display (104a), wherein the representation (106) includes data representing a first hash (108); means for generating a second hash (111) from the captured presentation (122); means for reading a digital code (107) from the display (104a) or the captured representation (122), the digital code (107) comprising: the first hash (108), and a digital signature (109) associated with the first hash (108); means for comparing the first hash (108) and the second hash (111); and Means for sending an indication as to whether the digital signature (109) is legitimate in response to the first hash (108) and the second hash (111) at least substantially matching.

2. The apparatus of claim 1, wherein the display (104a) is part of a user device (104) separate from the apparatus (112), the apparatus (112) comprising: An imaging component for capturing the presentation (106) on the display (104a) of the user device (104).

3. The device of claim 1, wherein the display (104a) is part of the device (112), wherein the means for capturing the presentation (106) on the display (104a) of the device (112) further comprises: Means for capturing a screenshot of the presentation (106) on the display, wherein the screenshot is the captured presentation (122).

4. The apparatus according to any of the preceding claims, wherein the presentation (106) comprises one or more from the group consisting of: an image for presentation on the display (104a); a plurality of images for presentation on the display (104a); A video for presentation on the display (104a); or Data representing a user prompt, interaction prompt or dialog box for rendering on the display (104a).

5. The apparatus of any preceding claim, wherein the captured presentation (122) comprises: a first image (124) associated with the first hash (108), and a second image (126) associated with the digital code (107), and wherein: The means for generating a second hash (111) of the captured representation (122) further comprises: means for applying a hash algorithm to the first image (124) to output the second hash (111); and The means for reading the digital code (107) from the display (104a) further comprises means for reading the digital code (107) from the second image (126).

6. The apparatus according to claim 5, further comprising: Means for cropping the captured presentation (122) to generate the first image (124), and means for cropping the captured presentation (122) to generate the second image (126).

7. The device according to any one of the preceding claims, the digital code (107) further comprising one or more from the following group: Barcode; 2D matrix barcode; Quick response QR code; High-capacity color two-dimensional HCC2D code; Digital codes embedded in displays or images; A digital code represented by a display or image; Use any other encoding of the image; or Any other encoding capable of being presented on the display (104a) and decoded from the presented encoding at the time of capture.

8. The apparatus according to any one of the preceding claims, wherein the means for comparing the first hash (108) and the second hash (111) further comprises: means for calculating a distance between the first hash (108) and the second hash (111); as well as Means for determining that the first hash (108) and the second hash (111) at least substantially match when the distance is less than or equal to a predetermined distance threshold or error threshold.

9. The apparatus of claim 8, wherein the distance is one or more from the group consisting of: a Hamming distance between the first hash (108) and the second hash (111); the Euclidean distance between the first hash (108) and the second hash (111); or any other distance metric between the first hash (108) and the second hash (111) used by a hashing algorithm that generates the first hash (108) and the second hash (111).

10. The apparatus according to any of the preceding claims, wherein the first hash (108) and the second hash (111) are generated using a hash algorithm based on one or more hash algorithms from the group consisting of: Average hashing aHash algorithm; Median hash mHash algorithm; Perceptual hashing pHash algorithm; Differential hashing dHash algorithm; Block hash bHash algorithm; Wavelet hash wHash algorithm; Color moment hashing algorithm; Color hashing algorithm, Marr-Hildreth, marrhildreth, hashing algorithm; Fuzzy hashing algorithm; Video hashing algorithm; Locality sensitive hashing algorithm; Machine learning hashing algorithms; A pair of machine learning hashing models; or Any other hashing algorithm configured to generate a hash from one or more images.

11. The device according to claim 10, wherein: The hash algorithm used to generate the first hash (108) and the second hash (111) are the same hash algorithm; or The hash algorithms used to generate the first hash (108) and the second hash (111) are similar or different hash algorithms that generate the same hash.

12. The device according to any one of the preceding claims, wherein the digital code (107) further comprises: Data representing a seed (110) associated with the digital signature (109), wherein the seed is used by a digital signature algorithm to verify whether the digital signature (109) is legitimate or valid.

13. The apparatus of any of the preceding claims, wherein the presentation (106) is associated with a prompt or request from a service (103) for a user action or information from the user (105) of the apparatus (112) and / or a user device (104) of the user (105) using the service (103), and the means for sending an indication to the user (105) further comprises: Means for providing an indication to the user (105) via the apparatus (112) or the user device (104) as to whether to perform the prompt or request for user action or information on the apparatus (112) or the user device (104), respectively, based on whether the digital signature (109) from the service (103) is legitimate or valid and in response to the first hash (108) and the second hash (111) substantially matching.

14. The apparatus of any one of the preceding claims, wherein the request for user action or information comprises one or more from the group consisting of: The request from the service (103) is an input from the user (105) or a request for an action to be performed by the user (105); Requesting the user (105) to input sensitive user information; Requesting a user (105) to scan sensitive information; Requesting the user (105) to scan a user passport, a user photo, or a contactless credit card; a request for user credentials or login credentials from the user (105); a request for confidential information of the user (105), or confidential information associated with the user (105); a request for organizational information associated with the user (105); Requesting the user (105) to accept and / or proceed with the installation of software on the user device (104); a dialog request associated with downloading, accepting and / or opening a file attachment received from said service (103); or Any other request requiring user input (105) on the user device (104) via a client service application associated with the service (103).

15. The device according to any one of the preceding claims, wherein the components comprise at least: at least one processor (502); as well as At least one memory (504) storing instructions that, when executed by the at least one processor (502), cause execution of the device (112).

16. A method for visual integrity inspection, comprising: capturing (232) a representation (106) for presentation to a user (105) on a display (104a), wherein the representation (106) includes data representing the first hash (108); generating (234) a second hash (111) from the captured presentation (122); reading (236) a digital code (107) from the display (104a) or the captured representation (122), the digital code (107) comprising: the first hash (108) associated with the representation (106), and a digital signature (109) associated with the first hash (108); comparing the first hash (108) and the second hash (111); and In response to the first hash (108) and the second hash (111) at least substantially matching, an indication is sent to the user (105) as to whether the digital signature (109) is legitimate or valid.

17. A computer program comprising instructions for causing a device (104, 106) to perform at least the following operations: capturing (232) a presentation (106) for presentation to a user (105) on a display (104a), wherein the presentation includes data representing the first hash (108); generating (234) a second hash (111) from the captured representation (122); A digital code (107) is read (236) from the display (104a) or captured representation (122), the digital code (107) comprising: the first hash (108) associated with the representation (106), and a digital signature (109) associated with the first hash (108); comparing the first hash (108) and the second hash (111); and In response to the first hash (108) and the second hash (111) at least substantially matching, an indication is sent to the user (105) as to whether the digital signature (109) is legitimate or valid.

18. An apparatus for serving (103), comprising: Means for generating (200) a representation (106) and a digital code (107) associated with requesting a user action or information at a user device (104, 112) using the service (103), wherein the digital code (107) includes: a hash (108) associated with the representation (106), and a digital signature (109) associated with the hash (108); and Means for sending a request (101) for said user action or information to said user device (104, 112), wherein said request (101) comprises: data representing said presentation (106), and a digital code (107) for display on said user device (104, 112) to authenticate said request (101) from said service (103) by said user (105).

19. A method for servicing a device, comprising: generating (202) a representation (106) and a digital code (107) associated with requesting a user action or information at a user device (104) of a user (105) using a service (103), wherein the digital code (107) includes: a hash (108) associated with the representation (106), and a digital signature (109) associated with the hash (108); and A request (101) for the user action or information is sent to the user device (104), wherein the request (101) includes: data representing the presentation (106), and a digital code (107) for display on the user device (104) to authenticate the request (101) from the service (103) by the user (105).

20. An apparatus (104) for visual integrity inspection, comprising: Means for receiving (212) from a service (103) a request (101) for a user action or information at a user device (104) of a user (105), said request (101) for a user action comprising: data representing a representation (106), and a digital code (107) for display on said user device (104) to authenticate said request (101) from said service (103), wherein said representation (106) and digital code (107) are generated by said service (103), said digital code (107) comprising: a hash (108) of said representation (106), and a digital signature (109) associated with said hash (108); means for displaying (214) said representation (106) and said digital code (107) on said user device (104); means for performing (216) a visual integrity check of said request (101) using said displayed representation (106) and said digital code (107); means for receiving (218) said digital code (107) and an indication of whether said representation (106) is authentic or legitimate; means for notifying (220) said user (105) to perform a requested user action in response to an indication that said digital code (107) and presentation (106) are authentic or legitimate; and Means for notifying (222) the user to deny performing the requested user action in response to the digital code (107) and the indication that the presentation (106) is unauthentic or illegitimate.

21. The user device of claim 20, wherein the means for performing (216) the visual integrity check on the request (101) further comprises: Means for using another user device (106) that is caused to perform the visual integrity check based on capturing the display of the presentation (106) and the digital code (107) presented on the display (104a) of the user device (104), wherein the another user device (106) is separate from the user device (104).

22. The user device of claim 20, wherein the means for performing the visual integrity check on the request (101) further comprises: Means for performing the visual integrity check on the user device (104), wherein the user device (104) further comprises means for performing the visual integrity check based on capturing a screenshot of the presentation (106) and the digital code (107) presented on the display (104a) of the user device (104).

23. The user device of any one of claims 20 to 22, wherein the means for performing the visual integrity check further comprises: means for capturing (232) a representation (106) for presentation to the user (105) on a display (104a), the representation (106) being associated with a first hash (108); means for generating (234) a second hash (111) from said captured image (122); means for reading (236) a digital code (107) from the display (104a) or the captured image (122), the digital code (107) comprising: the first hash (108) associated with the representation (106), and a digital signature (109) associated with the first hash (108); means for comparing the first hash (108) and the second hash (111); as well as Means for sending an indication to the user (105) as to whether the digital signature is legitimate in response to the first hash (108) and the second hash (111) at least substantially matching.

24. The user device of claim 23, wherein the presentation (106) is associated with a prompt or request (101) from the service (103) for a user action or information from the user (105) of the user device (104) using the service (103), and the means for sending the indication to the user further comprises: Means for providing (220) to the user (105) an indication as to whether to perform the prompt or request (101) for the user action or information on the user device (104) based on whether the digital signature (109) from the service (103) is legitimate or valid, in response to the first hash (108) and the second hash (111) being substantially matched.

25. A user device according to any one of claims 20 to 24, wherein the visual integrity check is performed according to any one of claims 1 to 15.

26. The user equipment (104) according to any one of claims 20 to 25, wherein the components at least comprise: at least one processor (502); as well as At least one memory (504) storing instructions that, when executed by the at least one processor (502), cause performance of the device (104).

27. A method for a user device (104), comprising: receiving (212) from a service (103) a request (101) for a user action or information at said user device (104), said request (101) for said user action or said information comprising: data representing a presentation (106), and a digital code (107) for display to a user (105) on said user device (104) to verify said request (101) from said service (103), wherein said presentation (106) and said digital code (107) are generated by said service (103), said digital code (107) comprising: a hash (108) associated with said presentation (106), and a digital signature (109) associated with said hash (108); displaying (214) the presentation (106) and the digital code (107) to the user (105) on the user device (104); performing (216) a visual integrity check of the request (101) using the displayed representation (106) and the digital code (107); receiving (218) an indication of whether the digital code (107) and the representation (106) are authentic or legitimate; In response to an indication that the digital code (107) and the presentation (106) are authentic or legitimate, notifying (220) the user (105) to perform the requested user action; and In response to an indication that the digital code (107) and the presentation (106) are not authentic or legitimate, the user is notified (222) of a refusal to perform the requested user action.

28. A system (100), comprising: A server device for providing a service (103) according to claim 18; as well as The user device (104) according to claims 20 to 26, wherein when the user (105) uses the service (103) via the user device (104), the server device providing the service (103) and the user device (104) communicate through a network.

29. The system of claim 28, wherein the user device (104) comprises: Means for performing a visual integrity check according to any one of claims 1 to 15 when dependent on claim 2, in response to the user device (104) receiving a request from the service (103) and displaying the representation (106) and the digital code (107) associated with the request (101).

30. The system of claim 28, further comprising another user device (112), the another user device (112) comprising: Means for performing a visual integrity check according to any one of claims 1 to 15 when dependent on claim 3 in response to the user device (104) receiving a request from the service (103) and displaying the representation (106) and the digital code (107) associated with the request (101).

Images