Security challenge

Abstract

An automated computer-implemented method of generating a security challenge to determine whether a user of a computer is a human, the challenge comprising a plurality of frames; and a computer-implemented method for determining whether an operator of a computing device is a human using a challenge, the challenge comprising a plurality of frames

Description

SECURITY CHALLENGETECHNICAL FIELD

[0001] Methods and programs for generating and performing a security challenge. BACKGROUND

[0002] Digital platforms, such as websites, mobile applications, simulations etc., contain resources that a third party may want or need to access or interact with. The third party may be termed an end user and be either a human or a non-human, e.g., a bot or a script. To limit access to a resource, the end user may be presented with a challenge to determine whether the end user is a human or a non-human and access to the resource being either allowed or denied depending upon the result of the challenge. Limiting access of non-humans to a resource may be for security to prevent unauthorized access, prevent a denial-of-service attack, protect information, or prevent another malicious action.

[0003] An existing solution is a Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA). CAPTCHA is a security measure known as challenge-response authentication. Implementations of CAPTCHA are used to limit access to websites and, in one implementation, require an end user to distinguish between different objects in images. For example, an end user is presented with a grid of images and asked to select images that include a specific object, such as a such as a pedal bike or a traffic light. This form of CAPTCHA was useful when computer vision was unable to correctly identify objects in images, however as computer vision technology has improved, the reliability of this form of CAPTCHA to correctly differentiate between a human and a non-human has decreased. With advancements in artificial intelligence, computer ability to solve a computer vision task, such as comprehending an image or identifying an object in an image based on an abstract prompt, such as “Bike” or “Traffic Light”, has improved. Because of the advancements in computer vision, an image presented to an end user has become increasingly difficult to understand by the end user, however the increased difficulty is perceived by both a human and a non-human as the end user.SUMMARY

[0004] The following presents a simplified summary of the disclosure in order to provide a basic understanding to the reader. This summary is not intended to identify key features or essential features of the claimed subject matter nor is it intended to be used to limit the scope of the claimed subject matter. Its sole purpose is to present a selection of concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.

[0005] An aspect of the invention relates to an automated computer-implemented method of generating a security challenge to determine whether a user of a computer is a human. The method comprising: retrieving a video; separating the video into a plurality of frames; selecting at least one frame of the plurality of frames as a target image; selecting anarea from the target image as a target area; and saving a challenge, the challenge comprising a plurality of images corresponding to the frames, the at least one target image, and the target area; wherein the challenge includes a random path for moving the target area or a random location of the target area.

[0006] Another aspect of the invention relates to a computer-implemented method for determining whether an operator of a computing device is a human. The method comprising: displaying a challenge, the challenge comprising an object, a target image and a current image, wherein the challenge includes a random path for moving the object or a random location for the object; receiving and storing an input to alter the object location or the current image; altering the current image based on the received input, wherein consecutive current images correspond to an animation; identifying if the current image matches a target image or if the object location matches a target object location; based on the identifying, determining that the challenge is complete; and receiving an indication, based on the stored received input, whether the operator of a computing device proving the input is human.

[0007] Many of the attendant features will be more readily appreciated as the same becomes better understood by reference to the following detailed description considered in connection with the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein:FIGs. 1A and 1 B are user interface components;FIG. 2 is a flow diagram of a method for detecting whether an end user is a human;FIG. 3 is a flow diagram of a method for generating challenges;FIG. 4 illustrates an end user challenge requiring an image match;FIGs. 5 and 6 illustrate an end user challenge requiring an image piece match for a static image;FIG. 7 illustrates an end user challenge requiring an image piece match for a moving image;FIGs. 8 and 9 illustrate an end user challenge requiring a reveal operation;FIGs. 10A and 10B illustrate an end user challenge requiring an image rotation operation;FIG. 11 illustrates an end user challenge requiring eye tracking on a static image; FIGs. 12A and 12B illustrate an end user challenge requiring an understanding of physics;FIG. 13 illustrates a first end user challenge requiring audio interaction;FIG. 14 illustrates a second end user challenge requiring audio interaction;FIGs. 15A and 15B illustrate media and an allocated dead-zone of the media;FIG. 16 is a flow diagram of a method for media ingestion; and FIG. 17 is a schematic diagram of a computing-based device.Like reference numerals are used to designate like parts in the accompanying drawings.DETAILED DESCRIPTION

[0009] The detailed description provided below in connection with the appended drawings is intended as a description of the present examples and is not intended to represent the only forms in which the present examples are constructed or utilized. The description sets forth the functions of the examples and the sequence of operations for constructing and operating the examples. However, the same or equivalent functions and sequences may be accomplished by different examples.

[0010] FIG. 1 A illustrates an exemplary user interface (Ul) component 10A prior to a user engaging with the UL The Ul component 10A may be presented in response to an end user attempting to access a resource. A trigger for the Ul component 10A may be specified by an entity controlling the resource or an entity controlling access to the resource. An individual, entity or organization which has control over a resource to be accessed via a platform, or storage location that the individual, entity, or organization owns is termed a resource owner. A trigger may be present to an end user on an interface, e.g., a two-dimensional (2D) interface, such as a web browser, TV screen, PC screen or mobile device screen, a three-dimensional (3D) interface such as a virtual reality (VR) or an augmented reality (AR) user interface, or the like. The Ul component 10A may be overlayed on an existing 2D or 3D Ul and the Ul component 10A may be sized manually by a Ul designer or automatically sized to fit a required placement on a display. In the illustrated example, an end user has moved a cursor 12 to click on a check box field 11 A.

[0011] FIG. 2 is a flow diagram of an exemplary method 200 for detecting whether an end user is a human. The above-described end user clicking “I’m human” corresponds to block 201.

[0012] At block 202, a challenge window opens. The challenge window, which may be a modal window or a frame in another Ul, is to present a challenge to an end user to determine whether the end user is human. An end user may be forced to interact with the challenge window if after the challenge window is triggered, the challenge window is presented on top of all other content visible to the end user. In one example, a shroud will obscure content not used by the window to draw attention of the end user to the challenge window. An attempt to interact with content outside of the challenge window may fail thereby forcing the end user to engage with the challenge window and, consequently, a verification process. Examples of challenge window content are described below and illustrated in the figures.

[0013] At block 203, a challenge is fetched from a server. This may be performed by a software widget for example. The fetching may comprise an API call to request a challenge besent for presentation to an end user, and the API call being returned with an appropriate challenge. The challenge may be one of many challenge types available at the server with each challenge type having a respective method of presenting a challenge to an end user as well as a unique mechanism for collecting data about the end user's solution attempt. The widget may be run by the server, in a browser, on a remote device or locally at an end user device.

[0014] At block 204, the fetched challenge is presented to the end user in a challenge window. A session for the challenge may begin when the challenged is fetched from the server and end when the challenge process is completed, or the challenge window is closed. There may be movement in the challenge window prior to the user engaging with the challenge (block 205). The movement may be an animation of an image, a video playing in the challenge window or an object in the challenge. The received challenge may include random variables relating to a path or location of objects in the challenge window. The movement and randomness described in these examples make it difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge without acting in a clearly iterative and non-human manner therefore including such measures increases security for sites or assets controlled by the challenge system.

[0015] At block 205, the end user attempts to solve the challenge by interacting with the contents of the challenge presented in the challenge window. The challenge is a simple task designed to inform a determination whether the end user is a human.

[0016] At block 205, end user interaction data is collected while the end user attempts to solve the challenge. The interaction data may comprise X and Y coordinates of the interaction on the screen over time, details of interactions with components of the challenge made by the end user, and / or other interaction data. The interaction data relates to a solution provided by the end user and consist of at least one interaction.

[0017] The form and content of the interaction data collected may be dependent upon the end user platform and manner in which the end user interacts with the respective challenge. With a computer mouse, for example, mouse event data is collected. With a touch screen device, for example, touch data of either a finger or a stylus is collected. With devices that use spatial operating systems, for example AR or VR devices, which may be head mounted, eye movement data and / or gesture data from the user's hands or hand controller is collected.

[0018] At block 206, at least a portion of the solution attempt comprising the data collected is sent to the server for processing and verification.

[0019] At block 207, it is determined whether the end user successfully completed the challenge presented to them.

[0020] If no to the determination at block 207, block 203 may be reperformed and a further challenge fetched from the server for the end user to complete. This process may repeat until the end user has successfully completed a challenge or a maximum number of challengeattempts has been made. The maximum number of challenge attempts may be a predetermined value and stored on the server.

[0021] If yes to the determination at block 207, at block 208 the server analyses received interaction data from the end user solution and at block 209, it is determined a likelihood that the end user is a human. The evaluation method used in the determination may be specific to the type of event data collected and may be specific to the type of challenge or the end user device. For example, mouse movement data and eye tracking data are incredibly different and so the respective evaluation methods used to perform respective evaluations will differ. The evaluation may return a likelihood value that the end user is human. The likelihood may be presented as a score between zero and one.

[0022] At block 209, it is determined whether the end user is human. This may comprise evaluating the score against a predetermined threshold.

[0023] If no to the determination at block 209, block 203 may be reperformed and a further challenge fetched from the server for the end user to complete. This process may repeat until the end user has successfully completed a challenge or a maximum number of challenge attempts has been made. The maximum number of challenge attempts may be a predetermined value and stored on the server.

[0024] If yes to the determination at block 209, at block 210 the server provides challenge completion data to another entity. This entity may be a third party of may be a software widget used for a current challenge session.

[0025] Block 211 is optional. At the block it is determined whether there is additional content to display to the end user post successful completion of the challenge and post determination that the end user is human.

[0026] If yes to the determination at block 211 , at block 212 the server provides additional content to be displayed to the user in the challenge window or in anther window. This additional content is termed post verification display and may be one or more of: a message, a security key, an URL, a barcode, a QR code, a link to an application for installation, or an action that relates to a fiduciary transaction. Whether to display additional content may be a decision of the server.

[0027] If no to the determination at block 211 or after the post verification display has been displayed, at block 213 a verifiable token may be made available to a web page, or a device hosting the challenge or associated with the challenge. The token is cryptographically signed and may comprise information about the end user's solution to the challenge. The token contains no sensitive information about the user or the actual solution and may be a digital receipt for the completion of the challenge between the user and the challenge platform / server. This token may be verified using a challenge platform public key and is proof that the challengeplatform / server issue the challenge and that the challenge was successfully completed by an end user that was determined to be human.

[0028] Further in relation to the token, the challenge platform / server may provide a verification mechanism whereby an API call between a backend of a web platform and a challenge platform API may be made to verify that a token was generated by the challenge platform / server and to confirm that the token was not previously verified - the challenge platform / server API will only respond positively to a verification request once per token. Using this additional verification step allows a third party to use the challenge platform technology to be confident that an end user is human and is using the system legitimately. The solution protects the system against replay attacks in which an attacker will complete the challenge once and then attempt to re-use a token until it becomes invalid.

[0029] At block 214, the “I’m human” box is disabled, and the end user is displayed a confirmation of the completion of the challenge.

[0030] FIG. 1 B is a challenge user interface 10B after an end user engaged with the user interface. A tick 11B indicates that field 11A of FIG. 1A was selected by the end user declaring themselves as a human and that the end user was found to be human. FIG. 1 B is an example of a confirmation of the completion of the challenge.

[0031] At block 215, the challenge window closes. Any shroud placed over content is removed. Full operation may be returned to a web platform and the end user can continue with their intended action. The challenge trigger may no longer be actionable either permanently or for a predetermined time to generate a further challenge.

[0032] The server as described above may be a single entity that controls the challenge process and also stores challenges to be completed in a repository or database, or the server may access a separate repository or database of challenges to be completed. The server may also be a network of disparate entities operating together to perform the tasks of the described server.

[0033] The determination for whether a challenge was completed by a human or a nonhuman actor may include a plurality of tests performed on some or all collected data. The results of each test are aggregated and may be weighted to create a confidence score that ranges between zero and one. The closer to one, the more confident the challenge platform is that the end user is a human.

[0034] In some examples, the challenge system utilises both hand-coded logic and Machine Learning algorithms trained on previous challenge data to make a human / non-human determination. The hand-coded algorithms combine checks, such as a check to determine if a challenge completed too quickly for the end user to be human, and / or an application of statistical models unique to each type of end user input device. Machine Learning algorithms each are individually trained models unique to the type of challenge presented. They models aretrained on real user input data to understand what a human input looks like for each challenge type and to identify solution attempts that fall outside of expected patterns.

[0035] FIG. 3 is a flow diagram of a method 300 for generating unique challenges. The method may be performed by a system to enable a resource owner to generate a unique challenge to protect a resource. It will be clear from the below disclosure that two or more challenges that appear visually similar may have vastly different solutions. It may be beneficial to using visually similar media to produce multiple challenges with different solutions. In this way, a nefarious actor will find it more difficult to train an algorithm to successful complete challenges because the training data will appear similar, but the required end user input to pass respective challenges are quite different. To this end, wherever possible, media may be re-used to generate multiple challenges of a same and / or different type. Having a diverse range of possible challenges with context specific instructions that require human level comprehension and solutions that appear to be human make creating a computer program capable of defeating the challenge platform very difficult.

[0036] In the exemplary method 300 illustrated in FIG. 3, media has been added to a media database at block 301. The media may comprise a 3D model, a video and / or collection of static image assets. At block 302, the raw media is fed into a decision tree.

[0037] If (block 303) raw media is a 3D model then it is used (block 304) to generate a plurality of unique videos of the 3D model being rotated, each video for use in a challenge to be put to an end user.

[0038] If (block 305) raw media is a video then it is used (block 306) to be split into a plurality generate a plurality of sets of multiple frames, each set for use in a challenge to be put to an end user.

[0039] If (block 307) raw media is a collection of image assets then it is used (block 308) to generate multiple interactive challenges, each challenge being unique and to be put to an end user.

[0040] Various challenge types are described and illustrated herein in relation to FIGs. 4 to 14. After the challenge types are described, methods for generating individual challenges are discussed in relation to FIGs. 15 and 16.

[0041] FIG. 4 illustrates an example of a video scrubbing style challenge that may be created, presented to an end user with a challenge window and solved according to systems described herein.

[0042] A challenge window 401 contains an image of an interactive frame 402 and an image of a target frame 410. The target frame 410 is unchanging, however the interactive frame 402 is dependent upon the position of a slider icon 404 on a slider track 403. The slider icon 404 indicates an initial direction for the slider to move in the form of an arrow. An instruction 405 to an end user is overlayed on the slider track 403. As an end user moves the slider 404 along thetrack 403, the interactive frame changes so that a video plays from the sequential interactive frames if the user moves the slider 404 to the right. The goal of the end user is to match the interactive frame 402 with the target frame 410 in order to complete the challenge. The instruction 405 may be position elsewhere in the challenge window other than the slider track 405. In one example, the instruction may be an audio prompt or outside the challenge window 401 . The interactive frame 402 and target frame 410 are illustrated as a picture-in-picture, however the interactive and target frames may be merely next to or proximal to each other. In one example, the interactive frame is above, the slider track is below, and the target frame is in between.

[0043] While an end user interacts with the challenge, data of their interaction is collected. A current interactive frame 402 being displayed, a position of the slider along the slider track 403 as well as the X and Y position of the end user input on the display as a whole may be stored. This data is collected multiple times a second to provide an accurate representation of the end user input.

[0044] Optionally, the image displayed in interactive frame 402 is manipulated randomly as the slider 404 is moved. This randomised manipulation is different for each frame and may be generated while the interaction is occurring. As a result, viewing the same frame multiple times will result in a subtly different image every time that increases that difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge.

[0045] This challenge provides enhanced security over existing solutions as the bulk of the security model is provided by analysis of an end user interaction with a challenge system. While it would be relatively simple for a computer to be trained to record each interactive frame sequentially and then move the slider 404 along the track 403 until the images of the interacted and target frames matched, it would be difficult to do that in a way that is convincingly human to the challenge system. This is because computers move in very precise and calculated movements. They take the most optimal path to their destination and move quickly. While it is possible to train a computer to move like a human in specific, isolated examples, it is difficult to do so in a way that would work for all challenge generated according to a video scrubbing style challenge.

[0046] The security of a challenge according to the type illustrated in FIG. 4 may be calculated. In a one-minute video composed of frames at a rate of 30 frames per second (fps) there is a total of 1800 frames present (60 seconds x 30 fps). The potential for unique, nonoverlapping 5-second clips (each clip consisting of 150 frames) is 1651 , assuming extraction at the frame level with no overlap. Consequently, the probability of correctly identifying a specific 5-second clip out of the 1651 unique possibilities is 1 / 1651 , and the odds of subsequently selecting the correct frame within that designated 5-second clip are 1 / 150. A combinedprobability of correctly identifying both the unique 5-second clip and the exact frame within that clip from a one-minute video is calculated by multiplying these two individual probabilities together. Consequently, the odds are 1 in 247,650 (1 / 1651 x 1 / 150). This probability suggests that if an end user is tasked with identifying a single, randomly selected frame from a specific 5- second clip within a one-minute video, the chance of correctly guessing the original 5-second clip and the specific frame within that clip is 1 in 247,650.

[0047] Video media may be uploaded by a resource owner and the video processed into individual frames. Frames are stored on a media server and may be used to generate a challenge. A subset of frames stored on the media server are selected and used, in chronological order. The number of frames in the subset varies and this variation helps to improve the security of a resultant challenge by making it harder for attackers to train a computer to solve challenges. In a subset of frames, the target frame may be selected automatically or manually and this variation further enhances the security of a resultant challenge because a model trained on a first challenge comprising a first subset and a first target frame will have difficulty in solving a second challenge with the same first subset but a different target frame. If the target frame is selected manually, a challenge designer may manually scrub through the frames to select a target frame within the subset.

[0048] As explained above, dividing video content into individual frames is an advantageous way to generate a challenge. A single video may provide a large number of possible challenges to serve to many end users. However, the nature of video content makes it difficult to simply extract all frames from a video and use them for a challenge as described below.

[0049] As an end user moves through interactive frames, thereby viewing individual frames in sequence, the window of possible frames to include in a solution to a challenge is proportional to the number of frames in the original video that are were devoted to a single shot. For example, a video with lots of clips, moving quickly between one another is a suitable candidate for a challenge because there are very few frames that look visually similar to one another. As an end user drags a slider along a slider track, the target frame the end user is searching for is nested amongst a small number of similar looking frames. We can allow the user to land on any one of those similar looking frames to successfully complete the challenge. In contrast, in the case of a much slower video where there is little image change for several seconds of the video, the number of similar frames to a target frame is much greater, hence either the number of similar frames to complete the challenge must be widened, thereby lowering the security of the challenge, or only allow a small number of similar frames may be accepted thereby requiring an end user to guess a target frame between a large number of similar looking frames.

[0050] To address the above problem, the challenge system may apply a vision based artificial intelligence to determine how visually similar adjacent frames are from one another. As described above, when generating a challenge according to an above method, a target frame is selected to be a solution to the challenge. Working outwards from either or both sides of the target frame, the system analyses adjacent frames and may remove adjacent frames that are visually similar to a degree above a predetermined threshold. Ideally, there is a non-zero number of adjacent similar frames to the target frame to decrease the accuracy to a level suitable for a human to pass the challenge, but the number of similar frames is low so that the likelihood of a successful random guess is low. A solution window of two to four similar frames is preferred to provide a solution window.

[0051] The playback of a video to an end user must be sufficiently smooth so that their performance when solving a challenge unaffected, as this could be detrimental to the security afforded by the challenge by increasing the number of false detections of a non-human. Further, by removing ambiguity for an end user in a case where there are a large number of consecutive similar frames, the number of humans failing the challenge is decreased, which affects security in a system that is meant to detect non-humans attempting the challenge. The removal of similar frames would be visually jarring during normal playback of a video, however the same is not true for a human while scrubbing through frames manually and the optimisation is not evident to an end user.

[0052] FIGs. 5 and 6 are exemplary illustrations of an end user challenge requiring an image piece match for a static image. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0053] An end user is presented with a challenge window 501 comprising an image 502. The image 502 has at least one piece of missing content 504 and at least one piece of fill content 503 shaped to fit the missing content 504. The challenge for the end user is to move a slider along a slider path to a position along the slider path when the fill content 503 overlays the missing content 504. The shape of the missing content may vary however the illustrated example has the missing and fill contents shaped as a jigsaw piece. A path the fill content 503 will travel may not be straight, but rather may intentionally be irregular. The path may be automatically randomised by an algorithm used to design a challenge from an input image in a media repository.

[0054] In FIG. 5, a path the fill content 503 will move is illustrated using dotted lines A, B with direction indicated using an arrow at the end of the dotted lines. The dotted lines are not visible to the end user. From a start position of FIG. 5, when the slider is on the left of the slider path, the fill content 503 will move along line A and pass over missing content 504. If an end user stops moving the slider while the fill content 503 is above the missing content 504, the challenge will be complete, however if the end user continues to move the slider along the sliderpath, the fill content 503 will move along line B away from the missing content 504. In another example, the fill content rotates around its central axis while it moves along the predetermined path A, B to further increase the difficulty of a computer vision system attempting to solve the challenge. In a further example, the rate the fill content moves along the predetermined path A, B is not proportional to the rate the slider is moved along the slider path.

[0055] Each instance of the challenge that is created for one image may have: (i) a different path for the missing content to travel along, (ii) a different rate of rotation of the content while it travels along the path, and / or (iii) a different varying rate of travel along the path.

[0056] The challenge is solved as described above; however, the detection of a human / non-human is dependent upon data collected about how the end user arrives as the challenge solution, e.g. the position of the puzzle piece, the position of the slider along the slider track as well as the X and Y position of end user input. This data is stored and collected many times a second to provide an accurate representation of the end user input while solving the challenge.

[0057] The image piece match for a static image enables improvements for human detection rates as it provides for additional data collection relating to how an end user solves the challenge in its control over the slider. While a non-human, i.e., a computer, is capable of determining a current position of a fill content and a position of the missing content, is difficult to determine the required slider position to make the positions align. The difficulty faced by a computer is compounded if a position of a slider is not directly proportional to a position of fill content. The combination of finding a correct solution to the challenge in a manner that is not readily detectable as being performed by a computer is incredibly difficult to achieve thereby providing a secure challenge.

[0058] FIG. 7 is an exemplary illustration of an end user challenge requiring an image piece match for a moving image that may be created, presented to an end user, and solved according to systems described herein.

[0059] The challenge window 801 contains a video that, once loaded, plays within the challenge window 801. Withing the challenge window there is an area of missing content 803 and an area of fill content 802. Optionally, the fill content 802 contains video relating to the missing content area.

[0060] The end user must move the fill content 802 to the missing content location by engaging with a slider 805 that is movable along a slide path 806. The challenge window 801 may include an instruction 804 for the end user for completing the challenge. Optionally, there is a play bar 807 illustrating the progress of the video being played in the window 801.

[0061] The movement of the fill content 802 to the missing content 803 is non-linear and may take a randomly generated path illustrated by a dashed line in. Optionally, the rate of movement of the fill content along the path is non-linear and may be randomly generated.Optionally, the fill content may rotate as it moves along the path. The random path may be generated when the challenge is designed, or the random variables generated to form a random path when the challenge is run for a first time.

[0062] The end user input is limited to dragging the slider 805 along the path however the moving image, varying path, and optional varying speed along the path 806 and optional rotation of the fill content makes the completion of the challenge difficult for a computer. While the end user interacts with the challenge window, data is collected about the interaction. The position of the fill content, the position of the slider along the slider track and the X and Y position of the end user input on the window can be collected for analysis to determine whether the end user is human.

[0063] Humans possess a perception of video that is not replicated by computer systems. A human perceives a video and derives an extraordinary level of understanding and context from it. In contrast, a computing system individually analyses static frames of a video and determine specific details relating to a single frame, such as, for example, making assumptions relating to a weather condition of the single frame. While this may appear to be useful, it cannot equally perform the same task on an entire video, but rather only on individual frames of the video.

[0064] For a computing system to solve a captcha such as that illustrated in FIG. 7, the processing power required would be relatively large therefore it would not be possible for the computing system to perform at any scale that would be useful for bot attack on a captcha secured website or service. In use, an area of content that is moved around by a user as they drag a slider blends into each frame of the image in a manner that the user, as a human, can easily perceive. A computer, viewing individual frames one-by-one cannot perform this anywhere near as effectively.

[0065] A captcha system can analyse small and subtle user input movements and determine how they related a captcha challenge being solved to identify a computing system imitating a human making an automated medium to large scale attach on captcha system computationally expensive.

[0066] FIGs. 8 and 9 are exemplary illustrations of an end user challenge requiring a reveal operation. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0067] The end user is presented with a challenge window 901. Within the challenge window 901 is image content that animates according to a position of a slider 902 along a slider path. Optionally, the challenge window 901 has an instruction for soling the challenge.

[0068] The animation, controlled by the slider 902 position, reveals an image 904 at a point of the slider 902 along the slider path. Further animation caused by further movement ofthe slider 902 will hide the image 904 from view. To solve the challenge, the end user must stop the slider when the image 904 is displayed.

[0069] In the illustrated example, the animation is of twisted fabric that untwists to reveal a further image 904 of a thumbs up and the fabric will continue to twist to hide the image 904 based on further slider 902 movement.

[0070] The pleats of fabric may be randomly manipulated in real-time as the slider is moved. The number of and complexity of folds are randomised to make it harder for a computing system to track and predict object movement occurring between frames.

[0071] In another example, different sections of animation to fade in and out of view in order to reveal an image and the fading is dependent upon a slider position.

[0072] When designing the challenge, different target images from a media bank may be automatically inserted into the animation to be uncovered by the end user.

[0073] The end user input when controlling the slider is captured and used to provide data for a determination of whether the end user is human.

[0074] FIGs. 10A and 10B are exemplary illustrations of an end user challenge requiring an image rotation operation. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0075] A challenge window 1001 presents a 3D object 1002 that is rotatable by an end user engaging with the challenge window. Optionally, there is a control in the challenge window, however the 3D image may be directly dragged in order to make it rotate. A translucent silhouette 1003 of the object in a specific rotation is overlayed on top of the 3D object 1002.

[0076] Instructions 1004 are provided to the end user for how to solve the challenge.

[0077] The end user must manipulate the orientation of the 3D object 1002 until it lines up with the silhouette 1003 thereby completing the challenge.

[0078] A completion message 1005 may be displayed to the end user.

[0079] Optionally, once the challenge is complete the silhouette 1003 disappears to reveal the object 1002 without obstruction and the end user can continue to rotate the 3D object 1002.

[0080] The challenge requires a large amount of user input to control the movement of the 3D object 1002. The user input is used to make a high confidence determination as to whether the end user is a human. Generating a human-like behaviour for the manipulation of a 3D object is a complex task and random or a predetermined movement will be extremely slow in solving the challenge. A statistical analysis may be performed on all three axes and on the interaction as a whole to determine whether the end user is human.

[0081] The starting and target positions of the 3D object 1002 may be randomised, and / or the captcha may require the object to be manipulated in at least two axes to complete the challenge. The additional constraints make it difficult for a computer system to gainsufficient data and context about a challenge to create a model suitable to reliably solve the challenge without acting in a clearly iterative and non-human manner. To add further difficulty for a computer system to solve the challenge, the 3D object 1002 object and / or the target (translucent silhouette 1003) may rotate prior to the user interacting with the challenge. This makes it significantly more difficult for a computer system to gain sufficient data and context about the challenge before it interacts with a challenge. Further, it makes training a model to solve the challenge highly difficult. From the moment interaction begins, data collection of interaction data may begin allowing analysis of the interaction data to aid in a prediction of whether the user is human.

[0082] FIG. 11 is an exemplary illustration of an end user challenge requiring eye tracking on a static image. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0083] Eye tracking hardware (not shown) is required to be used by an end user determine the gaze of the end user.

[0084] A challenge window 1101 presents an image, which may be in 2D or 3D. A 3D image requires a VR or an AR system. The image has multiple objects therein and the end user is provided with an instruction to look at a particular object 1103. In the illustrated example, there is a first representation 1102 of an object to be viewed in the middle of the challenge window 1101. In the challenge window 1101 , there is also a second representation 1103 of the object to be viewed.

[0085] If the end user gazes at the second representation 1103, the area being viewed zooms in on the second representation 1103 and the second representation enlarges. The first and second representations will eventually be overlayed, and the challenge is solved when the relative sizes of the representations 1102, 1103 substantially match. If the end user looks away from the second representation 1103 then the viewed area zooms out.

[0086] Optionally, as the image zooms in, a virtual environment around the user begins to form starting at the edges of the representation 1103 and moving inwards from the edges.

[0087] While the user is reading instructions and looking at the challenge window 1101 , eye tracking data is gathered. Small and subtle movements of the eyes as they scan the images in front of them provide data to perform an analysis of whether the end user is a human.

[0088] The challenge requires the ability to understand a given instruction and then to generate eye movement data while completing the challenge, which is incredibly difficult for a computer to reproduce in a manner deemed human.

[0089] In some examples, an end user may be drawn to multiple points of interest in an environment. This would provide additional eye movement data as well as require additional contextual based understanding of the environment which is difficult for a computer.

[0090] The time from the end user eye movement while reading an instruction and the beginning of an instructed task may be used to aid in the determination of whether the end user is human.

[0091] In some examples, the size and / or location of the object is randomised to make it difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge.

[0092] FIGs. 12A and 12B are exemplary illustrations of an end user challenge requiring an understanding of physics. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0093] A challenge window 1201 is provided to an end user and there is a comprehension element to the challenge in respect of multiple written sets of instructions 1202, 1203 and a comprehension task in respect of the illustrated sporting scenario. First, the challenge window 1201 requires an end user to read an instruction 1202 of “Pick a direction” and then to select an appropriate arrow - the arrow being appropriate based on a location of a goalkeeper image in the challenge, which may vary. The variance of the goalkeeper image may be controlled by random variables of the challenge thereby making it difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge.

[0094] Second, the challenge window 1201 requires an end user to read a further instruction 1203 of “Swipe to shoot” and an indication that the swipe represents a power gauge before swiping with a slider at an appropriate speed to complete the challenge by scoring a goal. Without an understanding of physics and the rules of football / soccer, an end user who is not human, i.e., a computer, will not be able to complete the task except for relying on an exceptionally large number of attempts relying on chance to allow the challenge to be completed.

[0095] The end user input controlling the buttons and the slider are captured and used to provide data for a determination of whether the end user is human.

[0096] FIG. 13 is an exemplary illustration of a first end user challenge requiring audio interaction. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0097] The end user is presented with a challenge window 1301 includes audio content that plays an audio clip. In one example, the clip is one to ten seconds long. In another example, the clip is longer than ten seconds in length. The challenge window 1301 includes a question 1302 related to the audio clip, and a plurality of answers 1303 to the question, whereby one or more of the answers are correct. The challenge is completed by an end user when they have answered a predetermined number of questions correctly. The determination of whether the end user is human depends upon the time taken to answer the questions and / or the enduser button clicking behaviour, e.g., the time taken to click the buttons, or the accuracy of the end user in clicking the buttons. The arrangement of the answers 1303 may be represented by randomly controlled variable so that it difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge.

[0098] FIG. 14 is an exemplary illustration of a second end user challenge requiring audio interaction. The challenge may be created, presented to an end user, and solved according to systems described herein.

[0099] An end user is presented with a challenge window 1401 comprising an image 1402 related to a piece of music and an instruction 1403 to the end user. A short segment of a music plays which may be related to the image 1402; the music may be looped for continuous playing. The tempo and / or pitch of the playing music may be incorrect. Using control(s) provided, the end user must adjust the tempo and / or pitch of the playing music until the audio sounds correct. There is a predetermined accuracy level for each of the variables. During the end user attempt to solve this challenge, data about the control input(s) is collected for analysis. Specifically, the changes presented at the control input device, e.g., a mouse, touch screen, movement of the control(s), movement between controls, and / or the amount of time the end user spends listening to the playing music between changes are collected. The collected information is used to make a determination as to whether the end user is a human. In another example (not illustrated) the end user is presented with two dials - one to control the pitch of the music and the other to control the tempo of the music. The correlation between the tempo / pitch and the amount of adjustment provided for a given amount of user input may be randomised to make it difficult for a computer system to gain sufficient data and context about a challenge to create a model suitable to reliably solve the challenge.

[0100] FIGs. 15A and 15B illustrate an example of media 1600 and the allocation of a dead-zone within the media.

[0101] FIG. 15A illustrates media 1600 that a media owner has uploaded to a media storage system. In this example, the media is a static image, however in other examples the media may be a video or moving image. The media 1600 comprises a first piece of content 1601 , a picture, and a second piece of content 1602, branding. The media is uploaded as it is intended for it to form a challenge to be generated based on the uploaded media 1600. The challenge will require an end user to engage with the media 1600 therefore it is required that the end user is not required to engage with the second piece of content 1602 as the branding includes words and an icon could be confusing or distracting to an end user having to engage with part of the media 1600 in order to solve a challenge. There may be a plurality of challenge types created that incorporate the image therefore the suitability of the image for all types of potential challenges should be ensured.

[0102] FIG. 15B illustrates delineation of the second piece of content 1602, the branding, of the media 1600. The delineation creates a dead-zone - an area of the media that will not be obscured during a challenge if the challenge incorporates the media 1600. In some examples, a dead-zone is an image, text, or an attribution. Dead-zones may be delineated using a basic shape, such as a rectangle, circle or triangle, or a free form drawing tool. In the example illustrated in FIG. 15B, a polygon is used to delineate a dead zone. The polygon comprises points 1603 connected with lines 1603. In some instances, there is a restriction on an area of media 1600 that a dead-zone can cover as a proportion of the total media 1600 area. A deadzone may be limited to 10%, 20%, 25% or 30% of the total media area.

[0103] As part of all challenge generation, the dead zone areas describe areas of the content that cannot be obscured by challenges. For example. Randomised motion paths do not intersect the dead-zones areas.

[0104] FIG. 16 is a flow diagram of an exemplary method 1600 for media ingestion. The method 1600 includes dead-zone allocation.

[0105] At block 1601 , a media owner uploads media. The media may be a static image, a moving image, or a video. Various file formats may be uploaded.

[0106] At block 1602, the raw media is stored in a database.

[0107] At block 1603, a media preview is provided to the media owner enabling the media owner to review the uploaded media.

[0108] At block 1604, the media owner selects whether they wish to identify a deadzone in the media.

[0109] If yes to block 1604 decision, at block 1605 the media specifies a dead-zone in the media. The specification may be according to the example illustrated in relation to FIGs. 15A and15B.

[0110] At block 1606, the media is stored in a media database with any allocated deadzone^).

[0111] Multiple dead-zones may be applied to media in the examples described above. In other examples, media may be bulk uploaded and dead-zones later individually identified.

[0112] Dead-zone identification in videos may be applied for all frames using the abovedescribed methods. An alternative method is that a sub-group of frames from the total number of frames in a video is identified and then a dead-zone identified and limited to the frame subgroup. The frame sub-group method may be used with the media upload method 1600. The maximum frame area applicable to a dead-zone may be the same percentages as described in relation to the method 1600 of FIG. 16.

[0113] Challenges require media assets for their creation. Media assets may be uploaded via method 1600 or uploaded in another manner. The main categories of media that may be used with challenges are static images, videos, 3D objects and audio clips.

[0114] Video media may be incorporated into a challenge that requires video, or frames of the video may be extracted from the video and the extracted frames used as static images for incorporation into one or more challenges requiring one or more static images.

[0115] 3D model media may be incorporated into a challenge that requires a 3D model, or a 3D model may be used in to generate a video by rotating the 3D model, and / or static image(s) may be created from the 3D model based on one or more different perspectives of the 3D model. A single 3D model enables thousands of different challenges to be generated enabling a strong security model based on the 3D model media.

[0116] A system of programmatic generation of media from a 3D model is described. In one example, a 3D model is automatically manipulated into random positions to generate a plurality of static images. The generated static images are then used as static images in a plurality of challenges. In another example, a virtual camera programmatically generates video media by taking multiple images of as a 3D model is rotated around one or more axes. The images are then combined as frames to create video media. The video media can then be divided into a plurality of shorter videos. The video(s) are stored and used in the same manner as uploaded video media.

[0117] In some examples, a challenge is generated from uploaded media that are interrelated, for example a plurality of static images that are interrelated. Multiple challenges may be generated using different combinations of the static images whereby a large number of distinct challenges are generated from a relatively small number of distinct images by using different image combinations and different image placement in each respective challenge.

[0118] Alternatively, or in addition, the functionality described herein is performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that are optionally used include Field- programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), Graphics Processing Units (GPUs).

[0119] FIG. 17 illustrates various components of an exemplary computing-based device 1700 which are implemented as any form of a computing and / or electronic device, and in which the above-described examples may be implemented for an end user operate.

[0120] Computing-based device 1700 comprises one or more processors 1701 which are microprocessors, controllers, or any other suitable type of processors for processing computer executable instructions to control the operation of the device in order to load web pages and interact therewith, for example. Platform software comprising an operating system 1711 or any other suitable platform software is provided at the computing-based device to enable application software 1712, such as a web browser, to be executed on the device 1700.

[0121] The computer executable instructions are provided using any computer-readable media that is accessible by computing-based device 1700. Computer-readable media includes, for example, computer storage media such as memory 1710 and communications media. Computer storage media, such as memory 1710, includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or the like. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), electronic erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that is used to store information for access by a computing device. In contrast, communication media embody computer readable instructions, data structures, program modules, or the like in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media does not include communication media. Therefore, a computer storage medium should not be interpreted to be a propagating signal per se. Although the computer storage media (memory 1710) is shown within the computing-based device 1700 it will be appreciated that the storage is, in some examples, distributed or located remotely and accessed via a network or other communication link (e.g., using communication interface 1702).

[0122] The computing-based device 1700 also comprises an input / output controller 1703 arranged to output display information to a display device 1721 which may be separate from or integral to the computing-based device 1700. The display information may provide a graphical user interface. The input / output controller 1703 is also arranged to receive and process input from one or more devices, such as a user input device 1722 (e.g., a mouse, keyboard, camera, microphone, or other sensor). In some examples the user input device 1722 detects voice input, user gestures or other user actions and provides a natural user interface (Nlll). This user input may be collected and used to determine whether the end user is human. In an example the display device 1721 also acts as the user input device 1722 if it is a touch sensitive display device. The input / output controller 1703 outputs data to devices other than the display device in some examples, e.g., a locally connected printing device (not shown in FIG. 17).

[0123] Any of the input / output controller 1703, display device 1721 and the user input device 1722 may comprise Nlll technology which enables a user to interact with the computingbased device in a natural manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls and the like. Examples of Nlll technology that are provided in some examples include but are not limited to those relying on voice and / or speechrecognition, touch and / or stylus recognition (touch sensitive displays), gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. Other examples of Nlll technology that are used in some examples include intention and goal understanding systems, motion gesture detection systems using depth cameras (such as stereoscopic camera systems, infrared camera systems, red green blue (rgb) camera systems and combinations of these), motion gesture detection using accelerometers / gyroscopes, facial recognition, three dimensional (3D) displays, head, eye and gaze tracking, immersive augmented reality and virtual reality systems and technologies for sensing brain activity using electric field sensing electrodes (electro encephalogram (EEG) and related methods).

[0124] The term 'computer' or 'computing-based device' is used herein to refer to any device with processing capability such that it executes instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the terms 'computer' and 'computing-based device' each include personal computers (PCs), servers, mobile telephones (including smart phones), tablet computers, set-top boxes, media players, games consoles, personal digital assistants, wearable computers, and many other devices.

[0125] The methods described herein are performed, in some examples, by software in machine readable form on a tangible storage medium e.g., in the form of a computer program comprising computer program code means adapted to perform all the operations of one or more of the methods described herein when the program is run on a computer and where the computer program may be embodied on a computer readable medium. The software is suitable for execution on a parallel processor or a serial processor such that the method operations may be carried out in any suitable order, or simultaneously.

[0126] Those skilled in the art will realize that storage devices utilized to store program instructions are optionally distributed across a network. For example, a remote computer is able to store an example of the process described as software. A local or terminal computer is able to access the remote computer and download a part or all of the software to run the program. Alternatively, the local computer may download pieces of the software as needed or execute some software instructions at the local terminal and some at the remote computer (or computer network). Those skilled in the art will also realize that by utilizing conventional techniques known to those skilled in the art that all, or a portion of the software instructions may be carried out by a dedicated circuit, such as a digital signal processor (DSP), programmable logic array, or the like.

[0127] Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person.

[0128] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0129] It will be understood that the benefits and advantages described above may relate to one example or may relate to several examples. The examples are not limited to those that solve any or all of the stated problems or those that have any or all of the stated benefits and advantages. It will further be understood that reference to 'an' item refers to one or more of those items.

[0130] The operations of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the scope of the subject matter described herein. Aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples without losing the effect sought.

[0131] The term 'comprising' is used herein to mean including the method blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.

[0132] The term ‘subset’ is used herein to refer to a proper subset such that a subset of a set does not comprise all the elements of the set (i.e. at least one of the elements of the set is missing from the subset).

[0133] It will be understood that the above description is given by way of example only and that various modifications may be made by those skilled in the art. The above specification, examples and data provide a complete description of the structure and use of examples.Although various examples have been described above with a certain degree of particularity, or with reference to one or more individual examples, those skilled in the art could make numerous alterations to the disclosed examples without departing from the scope of this specification.

Claims

CLAIMS1 . An automated computer-implemented method of generating a security challenge to determine whether a user of a computer is a human, the method comprising: retrieving a video; separating the video into a plurality of frames; selecting at least one frame of the plurality of frames as a target image; selecting an area from the target image as a target area; and saving a challenge, the challenge comprising a plurality of images corresponding to the frames, the at least one target image, and the target area; wherein the challenge includes a random path for moving the target area or a random location of the target area.

2. The computer-implemented method of claim 1 , further comprising: selecting a subset of the plurality of frames, wherein the subset of frames is saved into the challenge.

3. The computer-implemented method of claim 1 or claim 2, further comprising: analysing the plurality of frames to determine a degree of visual similarity between adjacent frames.

4. The computer-implemented method of claim 3, further comprising: increasing the number of target frames based on the determined degree of similarity being above a threshold.

5. The computer-implemented method of claim 3 or claim 4, further comprising: decreasing the number of target frames based on the determined degree of similarity being below a threshold.

6. The computer-implemented method of any preceding claim, further comprising: excluding a frame from the challenge due to the excluded being sufficiently similar to an adjacent frame that is included in the challenge.

7. The computer-implemented method of any preceding claim, wherein the security challenge is completed if a frame adjacent to a target frame is selected.

8. The computer-implemented method of any preceding claim, further comprising: distributing the challenge to a device for performing a security challenge according to the challenge.

9. A computer-implemented method for determining whether an operator of a computing device is a human, the method comprising: displaying a challenge, the challenge comprising an object, a target image and a current image, wherein the challenge includes a random path for moving the object or a random location for the object; receiving and storing an input to alter the object location or the current image; altering the current image based on the received input, wherein consecutive current images correspond to an animation;identifying if the current image matches a target image or if the object location matches a target object location; based on the identifying, determining that the challenge is complete; and receiving an indication, based on the stored received input, whether the operator of a computing device proving the input is human.

10. The method of claim 9, further comprising: generating the random path or generating the random location prior to displaying the challenge.

11. The method of claim 9 or claim 10, wherein the displaying the challenge further comprises changing a current image prior to receiving an input.

12. The method of any of claims 9 to 11 , wherein the input is provided at a user interface using a slider movable along a slider bar.

13. The method of any of claims 9 to 12, wherein the received input moves an animation forwards or backwards.

14. The method of claim 13, wherein the object is part of the target image.

15. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any preceding claim.

Images