User authentication using original and modified images

Abstract

A method and system for user authentication using original and modified images are provided. The method includes receiving an original image of a user, where the original image is a private image that meets certain set criteria. The method uses a pre-trained Convolutional Neural Network (CNN) model to extract one or more image features of the original image, and feeds the extracted image features to a Generative Adversarial Network (GAN) image generator that realistically modifies the extracted image features to generate the modified image. The method authenticates the user based on recognition of the presented original image or the presented modified image.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to user authentication, and more particularly to user authentication using original and modified images. [Background technology]

[0002] User authentication is required in many situations to confirm a user's authenticity, for example, when accessing social media accounts or other digital platforms.

[0003] Social media accounts are frequently attacked, and once access is stolen, it can have serious consequences for the account owner. Therefore, many authentication mechanisms are in use, and it is important that such mechanisms are robust and easy to use.

[0004] Two-factor authentication, a method commonly used to address this issue, typically relies on something the user knows (such as personal information / a special phrase) and something the user has (such as an additional mobile device). This approach has limitations in that personal information can be harvested or discovered, and the special phrase is often forgotten (since it is rarely used). Furthermore, the additional mobile device may be unavailable or may be lost or stolen. Summary of the Invention

[0005] According to one aspect of the present disclosure, there is provided a computer-implemented method for user authentication using an original image and a modified image, the method including receiving an original image of a user, applying a pre-trained convolutional neural network (CNN) model to extract one or more image features of the original image, and generating a modified image of the original image by feeding the extracted image features to a generative adversarial network (GAN) image generator that realistically modifies the extracted image features to generate the modified image, and authenticating the user based on recognition of the presented original image or the presented modified image.

[0006] According to another aspect of the present disclosure, there is provided a system for user authentication using original and modified images, comprising: a processor; and a memory configured to provide the processor with computer program instructions to perform functions of the components, the system comprising: an image receiving component for receiving an original image of a user; an original image feature extraction component for using a pre-trained convolutional neural network (CNN) model to extract one or more image features of the original image; a modified image generation component for providing the extracted image features to a generative adversarial network (GAN) image generator that realistically modifies the extracted image features to generate modified images; and an image set providing component for providing a set of original images and the modified images to an authentication component for authenticating a user based on recognition of the presented original image or the presented modified image.

[0007] According to another aspect of the present disclosure, there is provided a computer program product for user authentication using an original image and a modified image, the computer program product including a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by a processor to cause the processor to receive an original image of a user; apply a pre-trained convolutional neural network (CNN) model to extract one or more image features of the original image; provide the extracted image features to a generative adversarial network (GAN) image generator that realistically modifies the extracted image features to generate a modified image; and authenticate the user based on recognition of the presented original image or the presented modified image.

[0008] The computer readable storage medium may be a non-transitory computer readable storage medium, and the computer readable program code may be executable by a processing circuit.

[0009] The subject matter which is regarded as the present disclosure is particularly pointed out and distinctly claimed in the concluding portion of this specification. The present disclosure, both as to organization and method of operation, together with its objects, features, and advantages, may best be understood by reference to the following detailed description when read in connection with the accompanying drawings.

[0010] Preferred embodiments of the present disclosure will now be described, by way of example only, with reference to the following drawings: [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating an exemplary embodiment of a method flow in a modeling system according to an aspect of the present disclosure. [Figure 2A] FIG. 1 is a flow diagram of an exemplary embodiment of an aspect of a method according to the present disclosure. [Figure 2B] FIG. 1 is a flow diagram of an exemplary embodiment of an aspect of a method according to the present disclosure. [Figure 3]FIG. 10 is a flow diagram of an exemplary embodiment of another aspect of a method according to the present disclosure. [Figure 4] FIG. 1 is a block diagram of an exemplary embodiment of a system according to the present disclosure. [Figure 5] FIG. 1 is a block diagram of one embodiment of a computer system or cloud server in which the present disclosure may be implemented. [Figure 6] FIG. 1 is a schematic diagram of a cloud computing environment in which the present disclosure may be implemented. [Figure 7] FIG. 1 is a diagram of abstraction model layers of a cloud computing environment in which the present disclosure may be implemented.

[0012] It will be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous features. DETAILED DESCRIPTION OF THE INVENTION

[0013] The described user authentication mechanism defines a security technique that can authenticate a user based on the ability to distinguish genuine images from a collection of both original and modified (fake) images. The images used in this method are a set of private memorable images uploaded by the user.

[0014] This security approach utilizes a deep learning model called a convolutional neural network (CNN), which can be trained on a large set of images to create a generative model. CNNs have many useful applications; once trained, they can be used to generate a set of photo-realistically altered images based on the original image uploaded by the user. Generative adversarial networks (GANs) are the state-of-the-art training method in this area and can be used to realistically alter images. GANs consist of a pair of CNNs, one generative network and one discriminative network. Examples of alterations a GAN can make include changing the background of an image or altering the hair color, facial expression, or pose of the person in the image. These deep learning models can be combined to generate a set of fake, altered images for use in this method to verify the user's identity.

[0015] When authentication is required, the user is presented with a mixture of both the user's original, unaltered image and the modified, generated image, and only the true account owner can identify the true original image.

[0016] Image authentication mechanisms can be used as an additional authentication factor to augment existing authentication methods, for example as an additional random spot check or when a person is trying to recover their account after forgetting their password or after their account has been hacked.

[0017] Referring to FIG. 1, a schematic diagram 100 illustrates an exemplary embodiment of a trained deep learning model 110 of the described method.

[0018] The trained deep learning model 110 is pre-trained with large-scale data. CNN models used in image processing are trained with large-scale data, and these CNN models are used in a technique called transfer learning, which involves removing the top layer of the trained CNN model and feeding the output of the bottom layer into a new network that can use it as extracted features. In this described technique, transfer learning techniques are used to provide extracted image features to different GAN generators. Each GAN generator is trained to perform photo-realistic modifications on a set of training images, making the classifier unable to distinguish between modified and unmodified images. By the end of the training phase, each generator specializes in creating realistic modified images for its specific modification, thereby training each generator to realistically modify images through extracted image features to generate modified images. The generators and classifiers may be referred to as generator networks and classifier networks because they use appropriate network architectures.

[0019] The original image 101 is input into a deep learning model 110 that is trained for the user.

[0020] The trained deep learning model 110 includes a first pre-trained CNN 111 that extracts one or more image features 102 of an original image 101. The image features may be semantic features in the form of objects or concepts within the image.

[0021] The image features 102 are fed to multiple GANs 115, each in the form of a generator network 112 and a classifier network 114, which realistically modify the extracted image features and generate multiple modified images 103. The method uses transfer learning techniques to feed the extracted image features 102 to the generator networks 112 of the GANs 115. The generator networks 112 of the GANs 115 are trained to perform photo-realistic modifications such that the classifier networks 114 cannot distinguish between modified and unmodified images. The set of GANs 115 may be provided with each GAN 115 specialized to perform one of the proposed image modifications, such as changing the background, hair color, facial expression, and pose. Each GAN 115 would have its own generator network 112 and classifier network 114.

[0022] The generated modified image 103 is passed to a second pre-trained CNN 113, and transfer learning techniques are used to again extract image features 104 of the modified image 103, whose output is passed to each classifier network 114 of the GAN 115 trained to recognize the generated modified image.

[0023] The output of the classifier network 114 is used to provide a ranking score 120 of the generated modified images 103 to select the best generated modified image 105 relative to the original image 101 from the various modified images provided by the different GANs 115.

[0024] In an exemplary embodiment, each classifier network 114 provides a score for the generated modified images from its respective generator network 112, and each GAN 115 generates a single decimal score between 0 and 1. These scores are normalized and can be used to rank the generated modified images 103 to generate a ranking score 120 for each generated image 103.

[0025] For example, GAN1: Generator 1 generates modified image 1, and classifier 1 scores image 1 at 0.33 points; GAN2: Generator 2 generates modified image 2, and classifier 2 scores image 2 at 0.74 points; GAN3: Generator 3 generates modified image 3, and classifier 3 scores image 3 at 0.98 points. As a result, the ranking scores are: 1st place (best) image 3, 2nd place image 2, and 3rd place image 1.

[0026] Referring to Figures 2A and 2B, flow diagrams 200, 220 illustrate an exemplary embodiment of an image processing method for a user authentication method.

[0027] A user uploads a set of private, memorable images to the platform. The method receives a set of original images uploaded for the user (201). The original images are uploaded during the initial registration process and may be needed later to maintain the number of images in the set at a desired level.

[0028] The set of original images may be required to meet certain criteria, and to ensure that the original images are useful for future authentication, the images are marked as secret or private and not shared with anyone, even friends on the same social network.

[0029] For original images, the following privacy and recognition criteria may need to be met: · Memorable: The image should not be something that occurs regularly, such as one taken in a pub, as this will be difficult for the user to recall. · They have never been shared before: It is important to ensure that attackers do not have access to these images. · Personal significance: The image has meaning and is recognizable as your own.

[0030] The method can automatically check the social spread of images in the user's original image set (202). This social spread criterion may be determined automatically by the system to ensure that the set of images includes, for example, three or more separate social circles. The original image set as a whole may be determined to include photos of people with ties from three separate social circles, such as friends, family, and coworkers. The strength of this criterion is automatically determined with reference to a social graph of ties that may be provided by a social networking platform. For example, this may be the social networking platform with which the user is authenticated.

[0031] In one embodiment, this automated assessment of social spread can use facial recognition to identify which of a user's friends appear in each image. The distance between social circles in each image can then be identified as the shortest path between two subgraphs in the social graph. The distance between three images in an image set should exceed a configurable distance threshold, including comparing the shortest path to a configurable distance threshold. The larger the distance threshold, the stronger the security of this technology. This information can be fed back to users in real time as they upload images to the image set, informing them of whether they need to add more explicit images.

[0032] The method may also automatically check whether the content of the set of original images has changed sufficiently for the user based on set criteria 203. A set of content change criteria may be provided such that the set of original images includes images that have changed in content to facilitate the application of different modifications using the style transfer techniques employed to generate the modified images.

[0033] For application of the discussed modifications, and for extensibility to future modifications, the following preferred content change criteria may be applied to the set of images uploaded by a user. Image with a clearly recognizable background: You need one image taken outdoors with a clearly recognizable background, not a photo with a pure white background or a "Portrait mode" image with a blurred background. This will allow for changes in weather / seasons and time of day. Images taken at famous locations and landmarks: Tolerates changes in weather / seasons and time of day. Self-portrait or "selfie": This is quick and easy for the user to provide, and is very secure if no one else is present. Additionally, the user may be asked to elicit a memorable facial expression. This is suitable for changing facial expressions, facial features, and hair color. Full-body images: These are likely to be good candidates for pose correction, and as our ability to generatively position objects in images improves, these images may also be good candidates for other such corrections.

[0034] This set of content variation criteria may be more lenient than the privacy and awareness criteria and the social spread criteria. Depending on the volume of images uploaded, a comprehensive subset of the content variation criteria may provide sufficient security. This is configurable.

[0035] The method can determine 204 whether the user's set of original images meets preset criteria, and can request additional original images if they do not.

[0036] Once uploaded and meeting the aforementioned criteria, a user's private set of original images is fed into one or more trained networks, where each image has a set of proposed modifications applied by multiple generators.

[0037] The method may use a deep learning modeling process (210) trained on each original image to extract image features from the original image (205), generate one or more modified images for the original image by modifying different features of the original image (206), and select an optimal modified image for the original image (207).

[0038] A set of modified images is generated for the user's set of original images (208), and the set of original and modified images is stored for future authentication challenges (209).

[0039] The images presented to the user for authentication can be randomly selected from the user's original image and a generated image with optimal modifications. The authentication mechanism involves the user identifying which images are genuine and which are generated. If the user can correctly identify the images within a configurable error threshold, they are successfully authenticated.

[0040] Referring to Figure 2B, a flow diagram 220 illustrates further details of an exemplary embodiment of the modeling process 210 of Figure 2A using a pre-trained CNN model and a technique called transfer learning. CNN models may be pre-trained with large amounts of data. These models can provide significant benefits for downstream tasks such as object / person recognition and semantic segmentation through a technique called transfer learning. In this technique, the top layer of a trained CNN model is removed and the output of the bottom layer is fed into a new network where it can be used as extracted features.

[0041] The original images are passed to a first pre-trained CNN model (221) to extract image features from each original image (222).

[0042] The method uses transfer learning techniques to feed the extracted features to multiple generators (223) of different GANs, each specialized in performing one of the proposed image modifications, such as changing the background, hair color, facial expression, or pose. Each GAN generator is trained to perform photo-realistic modifications on a set of training images, rendering the classifier unable to distinguish between modified and unmodified images. By the end of the training phase, each generator is specialized in creating realistically modified images for a particular modification.

[0043] The generated modified image is passed to a second pre-trained CNN (224), where transfer learning techniques are again used to extract image features (225), but this time the output is passed to multiple trained GAN classifier networks to recognize the generated modified image (226).

[0044] The normalized logit output of each classifier is used to rank the modifications (227) to determine which modification gives the most realistic and best results. This modified image is selected (228) as the best modification to the original image.

[0045] Referring to FIG. 3, a flow diagram 300 illustrates an exemplary embodiment of a method for authenticating a user using an original image of the user and a generated modified image.

[0046] The method may enroll a user with a set of original images 301. The set of original images may be added to maintain a sufficient number of original images in the set for the user, as images may be deleted once used in failed authentication attempts to prevent image disclosure.

[0047] Authentication may be triggered 302, for example, as an additional authentication check for a user of an account, such as a social media account or other online account.

[0048] The method may randomly select an image from a combined set of the user's original image set and a set of modified images based on the original image set generated by the described method (303). The user may be prompted to identify whether the presented image is an original image or a modified image (304).

[0049] The response may be determined to be correct (305). If the response is incorrect, the presentation image is removed from the image set (306) and user authentication fails (307). If image identification fails or is aborted, all displayed presentation images are invalidated. This condition prevents brute force attacks against the mechanism by ensuring that an attacker never sees the same presentation image twice.

[0050] If the response is correct, it is determined whether a sufficient number of presentation images have been correctly identified as configured for authentication (308), which may be configured to require, for example, the correct identification of three images. If a sufficient number of presentation images have not been identified, the method may loop to randomly select (303) the next image for presentation to the user. Once a sufficient number of presentation images have been correctly identified by determining operation 308, the user may be authenticated (309).

[0051] Also, a monitor may be maintained over the number of images available to be presented, in order to request more original images from the user when the supply is nearing exhaustion. If the image set is completely exhausted, the authentication mechanism may be locked.

[0052] Only one image should be presented at a time, and the user should correctly identify that image before presenting another, to prevent user error from quickly exhausting the image set, and only a subset of the displayed images should be invalidated after a failed authentication attempt.

[0053] The described authentication has several advantages: it is stronger than a static set of secondary security questions; these questions can usually be answered by someone who has known the account owner for an extended period of time; and the described method ensures that uploaded images come from different social circles, so it is unlikely that a single individual will have sufficient prior knowledge of the images to pass the security check.

[0054] The described method is a visual authentication method that does not require the user to remember any other information. The described method is very easy to use because it is based on personal but non-sensitive data. Only the genuine account owner can tell which images are modified, i.e., fake, allowing for fast account authentication. It is also important to note that it does not encourage users to write down confidential information, such as passwords or answers to security questions, in files or notes, which may pose a security risk.

[0055] When presented with a set of images that includes at least three of a user's social circle, it becomes less likely that someone who knows the user well will be able to correctly identify all of the images—for example, it's less likely that a partner will be able to identify a coworker's changing facial expression.

[0056] This method is easy for any user to implement, especially in the context of a social media account, since the user has an image they can upload. It also does not depend on the availability of a mobile phone or other device to receive or generate the one-time passcode. Someone who has lost their mobile phone or does not have a mobile phone will not be able to receive this passcode.

[0057] Referring to FIG. 4, a block diagram illustrates an exemplary embodiment of the described system.

[0058] The computing system 400 includes at least one processor 401, circuitry for performing the functions of the described components, which may be a hardware module or a software unit running on the at least one processor. Multiple processors executing parallel processing threads may be provided, allowing for parallel processing of some or all of the functions of the components. The memory 402 may be configured to provide computer instructions 403 to the at least one processor 401 to perform the functions of the components.

[0059] The trained deep learning model 110 may be provided on or accessed from the computing system 400 and includes a first pre-trained CNN 111, multiple GANs 115 each including a generator network 112 and a discriminator network 114, and a second pre-trained CNN 113, as previously described.

[0060] The image user authentication system 410 may be provided in the computing system 400 and may include the following components: An image receiving component 411 may receive original images for a user. The image receiving component 411 may evaluate the received images to ensure they meet the criteria required for suitable original images. A criteria setting component 412 may apply a configurable set of criteria to the set of original images to ensure they are suitable for image feature extraction and modification.

[0061] The received image evaluation component 413 can evaluate the received set of original images to verify that they meet configured criteria. This can include evaluating the social distance between people in each of the subset of original images and verifying that this exceeds a social distance threshold. If the user authentication is for a social media platform, the received image evaluation component 413 can use the social graph of the user's social media account to determine the social distance between people in the original images.

[0062] The original image feature extraction component 414 may use a first pre-trained CNN model to extract one or more image features of the original image. The extracted feature supply component 415 may supply the extracted image features to each of a set of different GAN image generators that perform different sets of image modifications, and the modified image generation component 416 may generate the modified images. The modified image selection component 417 may select a modified image relative to the original image from the multiple modified images using a second pre-trained CNN 113 to extract image features from the multiple modified images and pass them to the classifier network 114 to rank the multiple modified images.

[0063] The image set provisioning component 422 can provide the authentication component 418 with an image set of the user's original image and modified images.

[0064] The authentication component 418 may or may not authenticate the user based on recognition of the presented original image or the presented modified image. The authentication component 418 includes a random selection component 419 for randomly selecting an image from a combination of the set of original images and the set of modified images for user authentication, and a presentation component 420 for presenting the image to the user.

[0065] The set update component 421 may remove images from the combination of the set of original images and the set of modified images after they are disabled as not recognized by the user for authentication, and may add additional original images to the set of original images when the number in the set reaches a threshold.

[0066] 5 illustrates a block diagram of components of a computing system 400 such as may be used in an image user authentication system 410 and a trained deep learning model 110 according to an embodiment of the present disclosure. It should be understood that FIG. 5 is merely illustrative of one implementation and does not imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

[0067] The computing system may include one or more processors 502, one or more computer-readable RAMs 504, one or more computer-readable ROMs 506, one or more computer-readable storage media 508, device drivers 512, read / write drives or interfaces 514, and network adapters or interfaces 516, all interconnected over a communications fabric 518. The communications fabric 518 may be implemented with any architecture designed to pass data and / or control information between processors (such as microprocessors, communications and network processors), system memory, peripheral devices, and any other hardware components in the system.

[0068] One or more operating systems 510, such as the image user authentication system 410 and the trained deep learning model 110, and application programs 511 are stored in one or more computer-readable storage media 508 for execution by one or more of the processors 502 via one or more (typically including cache memory) of the respective RAMs 504. In the illustrated embodiment, each of the computer-readable storage media 508 may be a semiconductor storage device such as an internal hard drive magnetic disk storage device, a CD-ROM, a DVD, a memory stick, a magnetic tape, a magnetic disk, an optical disk, a RAM, a ROM, an EPROM, a flash memory, or any other computer-readable storage medium capable of storing computer programs and digital information, in accordance with an embodiment of the present disclosure.

[0069] The computing system may also include a R / W drive or interface 514 for reading from and writing to one or more portable computer-readable storage media 526. Application programs 511 on the computing system may be stored on one or more portable computer-readable storage media 526 and read via the respective R / W drive or interface 514 and loaded onto the respective computer-readable storage media 508.

[0070] The computing system may also include a network adapter or interface 516, such as a TCP / IP adapter card or a wireless communication adapter. Application programs 511 on the computing system may be downloaded to the computing device from an external computer or external storage device via a network (e.g., the Internet, a local area network or other wide area network or a wireless network) and the network adapter or interface 516. The programs may be loaded from the network adapter or interface 516 into the computer-readable storage medium 508. The network may include copper wire, optical fiber, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0071] The computing system may also include a display screen 520, a keyboard or keypad 522, and a computer mouse or touchpad 524. The device driver 512 interfaces to the display screen 520 for imaging, to the keyboard or keypad 522, to the computer mouse or touchpad 524, or to the display screen 520 for alphanumeric character entry and pressure sensing of user selections, or a combination thereof. The device driver 512, the R / W drive or interface 514, and the network adapter or interface 516 may comprise hardware and software stored in the computer-readable storage medium 508 or in the ROM 506, or both.

[0072] The present disclosure may be a system, method, or computer program product, or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium having stored thereon computer-readable program instructions for causing a processor to perform aspects of the present disclosure.

[0073] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, by way of example only, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROMs, DVDs, memory sticks, floppy disks, mechanically encoded devices having instructions recorded thereon, such as punch cards or ridge-in-groove structures, and suitable combinations thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0074] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof). The network may be comprised of copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0075] Computer-readable program instructions for carrying out the operations of the present disclosure may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language and similar programming languages. The computer-readable program instructions may be executable entirely on the user's computer, as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize them in order to carry out aspects of the present disclosure.

[0076] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0077] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing apparatus to create a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored in a computer-readable storage medium connectable to a computer, programmable data processing apparatus, or other device, or combination thereof, that functions in a particular way, such that the computer-readable storage medium having the instructions stored thereon configures one of the products including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0078] Computer-readable program instructions, such as instructions to perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams on a computer, other programmable apparatus, or other device, can also be loaded into a computer, other programmable data processing apparatus, or other device to perform a series of operational steps on the computer, other programmable apparatus, or other device to generate a computer-implemented process.

[0079] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of executable implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which constitute one or more executable instructions for implementing the specified logical function(s). In some alternative embodiments, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be accomplished as a single step, executed concurrently, substantially concurrently, partially, or fully in a time-overlapping manner, or the blocks may be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.

[0080] <Cloud Computing> Although this disclosure includes detailed descriptions of cloud computing, implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present disclosure can be practiced in conjunction with any other type of computing environment now known or developed in the future.

[0081] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five characteristics, at least three service models, and at least four implementation models.

[0082] The characteristics are as follows:

[0083] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time or network storage, automatically as needed, without the need for human interaction with the service provider.

[0084] Broad network access: Computing power is available over the network and can be accessed through standard mechanisms, facilitating use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, PDAs).

[0085] Resource Pooling: Computing resources from a provider are pooled and offered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated based on demand. Consumers generally have no control or knowledge of the exact location of the resources they are provided with, resulting in a sense of location independence. However, consumers may be able to determine location at a higher level of abstraction (e.g., country, state, data center).

[0086] Rapid Elasticity: Computing capacity can be provisioned quickly and elastically, sometimes automatically, to instantly scale out and quickly release to instantly scale in. To the consumer, the computing power available for provisioning often appears unlimited, and can be purchased at any time and in any quantity.

[0087] Metered Services: Cloud systems leverage measurement capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user count) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of utilized services.

[0088] The service model is as follows:

[0089] Software as a Service (SaaS): The functionality offered to the consumer is the availability of a provider's applications running on a cloud infrastructure that can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., webmail). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functionality, except for limited user-specific application configuration settings.

[0090] Platform as a Service (PaaS): The capability offered to consumers is to deploy applications they create or acquire using programming languages ​​and tools supported by the provider onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of their hosting environment.

[0091] Infrastructure as a Service (IaaS): The functionality offered to consumers is the provisioning of processors, storage, networking, and other basic computing resources on which they can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating system, storage, and deployed applications, and in some cases partial control over some network components (e.g., host firewalls).

[0092] The deployment model is as follows:

[0093] Private Cloud: This cloud infrastructure is dedicated to a specific organization and can be managed by that organization or a third party, and can exist on-premise or off-premise.

[0094] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by those organizations or a third party and can exist on-premises or off-premises.

[0095] Public cloud: This cloud infrastructure is available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0096] Hybrid cloud: This cloud infrastructure combines two or more cloud models (private, community, or public), each of which retains its inherent nuances but is bound by standards or specific technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0097] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0098] Referring to FIG. 6, an exemplary cloud computing environment 50 is depicted. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices used by cloud consumers (e.g., a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof) can communicate. The nodes 10 can communicate with each other. The nodes 10 can be physically or virtually grouped (not shown) in one or more networks, such as the private, community, public, or hybrid clouds described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure, platform, or software as a service, or combinations thereof, for which cloud consumers are not required to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N depicted in FIG. 6 are merely exemplary, and that the computing nodes 10 and the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network-addressable connection (e.g., using a web browser), or both.

[0099] Referring to Figure 7, a set of functional abstraction model layers provided by cloud computing environment 50 (Figure 6) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 7 are merely exemplary, and embodiments of the present disclosure are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0100] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0101] The virtualization layer 70 provides an abstraction layer from which the following virtual entities can be provided, for example: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0102] By way of example, the management layer 80 may provide the following functions: Resource provisioning 81 enables dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment; Metering and pricing 82 enables cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources; By way of example, these resources may include application software licenses; Security enables identification and verification of cloud consumers and tasks, as well as protection for data and other resources; User portal 83 provides consumers and system administrators with access to the cloud computing environment; Service level management 84 enables allocation and management of cloud computing resources so that requested service levels are met; Service level agreement (SLA) planning and fulfillment 85 enables advance arrangement and procurement of anticipated future cloud computing resources required in accordance with SLAs.

[0103] The workload layer 90 provides examples of functionality available to a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and image correction and user authentication processing 96.

[0104] The computer program product of the present disclosure comprises one or more computer-readable hardware storage devices having computer-readable program code stored thereon, said program code being executable by one or more processors to perform the methods of the present disclosure.

[0105] The computer system of the present disclosure comprises one or more processors, one or more memories, and one or more computer-readable hardware storage devices, the one or more hardware storage devices containing program code executable by the one or more processors via the one or more memories to perform the methods of the present disclosure.

[0106] The description of various embodiments of the present disclosure is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. It will be apparent to those skilled in the art that many modifications and variations are possible without departing from the scope of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements to technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0107] Improvements and modifications can be made to the foregoing without departing from the scope of the present disclosure.

Claims

1. 1. A computer-implemented method for user authentication using an original image and a modified image, comprising: receiving an original image of a user; applying a pre-trained convolutional neural network (CNN) model to extract one or more image features from the original image, and generating a modified image of the original image by feeding the extracted one or more image features to a generative adversarial network (GAN) image generator that modifies the extracted one or more image features to generate the modified image; authenticating the user based on recognition of the submitted original image or the submitted modified image; A method comprising:

2. feeding the extracted one or more image features to a set of different generative adversarial network (GAN) image generators, each performing a different set of image modifications, to generate multiple modified images from a single original image; selecting a modified image for the original image from the plurality of modified images; The method of claim 1 , comprising:

3. 3. The method of claim 2, wherein selecting a modified image from the plurality of modified images uses a second pre-trained CNN to extract image features from the plurality of modified images and pass them to a classifier to rank the plurality of modified images.

4. receiving a set of original images of the user; generating a set of modified images based on said set of original images; Randomly selecting an image from the combination of the set of original images and the set of modified images for authentication of the user; The method of claim 1 , comprising:

5. removing an image from the combination of the set of original images and the set of modified images after it has been used to fail authentication of the user; adding additional original images to the set of original images when the number in the set reaches a threshold; generating a modified image of the additional original image; The method of claim 4, comprising:

6. evaluating the received set of original images to assess the social distance between people in each of the subset of original images and to confirm that this exceeds a social distance threshold. The method of claim 4, comprising:

7. To evaluate is using facial recognition to identify people appearing in each of at least a subset of said original images; referencing a social graph of the user's connections; determining the social distance between the people in each original image as the shortest path between subgraphs of each image in the social graph; comparing the shortest path to a configurable distance requirement threshold; The method of claim 6, comprising:

8. applying a set of configurable criteria to the set of original images to verify that the set of original images is suitable for the extraction and modification of the image features. The method of claim 4, comprising:

9. The method of claim 1 , wherein the original image is a private image that is not shared.

10. 1. A system for user authentication using original and modified images, comprising: a processor and a memory configured to provide computer program instructions to the processor to perform functions of one or more components; an image receiving component for receiving an original image of a user; an original image feature extraction component for extracting one or more image features from the original image using a pre-trained convolutional neural network (CNN) model; a modified image generation component for providing the extracted one or more image features to a generative adversarial network (GAN) image generator that modifies the extracted one or more image features to generate a modified image; an image set providing component for providing a set of original images and modified images to an authentication component for authenticating the user based on recognition of the submitted original images or the submitted modified images; A system comprising:

11. an extracted feature feeding component for feeding the extracted image features to each of a set of different generative adversarial network (GAN) image generators that perform different sets of image modifications to generate multiple modified images from a single original image; a selection component for selecting a modified image from the plurality of modified images; The system of claim 10, comprising:

12. 12. The system of claim 11, wherein the selection component for selecting a modified image from the plurality of modified images uses a second pre-trained CNN to extract image features from the plurality of modified images and pass them to a classifier to rank the plurality of modified images.

13. 11. The system of claim 10, wherein an image receiving component receives a set of original images of the user, the modified image generating component generates a set of modified images based on the set of original images, and an image set providing component provides a combined set of the set of original images and the set of modified images for authentication of the user.

14. The system of claim 13 , wherein the authentication component comprises a random selection component for randomly selecting an image from the combination of the set of original images and the set of modified images for the authentication of the user.

15. a set update component for removing images from the combination of the set of original images and the set of modified images after being used in the authentication of the user, and for adding additional original images to the set of original images when a number in the set reaches a threshold. The system of claim 13, comprising:

16. a received image evaluation component for evaluating the received set of original images to evaluate the social distance between people in each of the subset of original images and to verify that this exceeds a social distance threshold; The system of claim 13, comprising:

17. a criteria setting component for applying a set of configurable criteria to the set of original images to verify that the set of original images is suitable for the extraction and modification of the image features; The system of claim 13, comprising:

18. 17. The system of claim 16, wherein the authentication component is for a social media platform, and the received image evaluation component uses a social graph of the user's social media account to determine social distance between people in the original image.

19. The system of claim 10 , wherein the authentication component is triggered as an additional authentication check of the user.

20. 1. A computer program for user authentication using an original image and a modified image, the computer program comprising program instructions executable by a processor, the program instructions comprising: receiving an original image of a user; applying a pre-trained convolutional neural network (CNN) model to extract one or more image features from the original image; providing the extracted one or more image features to a generative adversarial network (GAN) image generator that modifies the extracted one or more image features to generate the modified image; authenticating the user based on recognition of the submitted original image or the submitted modified image; A computer program that executes

Images