Method for evaluating and discovering phishing attempts in an email

Abstract

A method of rating a phishing email's human phishing detection difficulty includes identifying cues in content of the email tending to indicate that the email is a phishing email; assigning weights to identified cues; assessing a relative severity based on the weighted cues; determining an overall human detection difficulty of the email based on the relative severity of the weighted cues and a total number of the identified cues determining a target audience for the phishing email; determining a plurality of sub-groups of the target audience; and determining a plurality of respective premise alignments of the email by characterizing respective pertinences of a message premise of the email with respect to each respective sub-group of the target audience; wherein the step of determining an overall human detection difficulty of the email is additionally based on the respective premise alignments of the email.

Description

[0001] METHOD FOR EVALUATING AND DISCOVERING PHISHING ATTEMPTS IN AN EMAIL

[0002] Related Applications

[0003] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 638,554 (filed April 25, 2024), which is herein incorporated by reference in its entirety.

[0004] Federally-Sponsored Research and Development

[0005] This invention was made with United States Government support from the National Institute of Standards and Technology (NIST), an agency of the United States Department of Commerce. The Government has certain rights in this invention.

[0006] Copyright Notice

[0007] This patent disclosure may contain material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves any and all copyright rights.

[0008] Field of Invention

[0009] The present invention relates generally to network security, and more particularly to a phishing email evaluation metric for providing context to click rates.

[0010] Background

[0011] Phishing is an email-based cybersecurity threat in which cybercriminals attempt to get sensitive information from email recipients. It is a social engineering technique that compels an email recipient to perform an action beneficial to the attacker (e.g., clicking a link to a fraudulent website or downloading a malicious attachment). The phishing cyber threat exploits vulnerabilities in the United States (U.S.) and around the world across private and public sectors. It remains one of the top security threats to these organizations, costing companies billions. As technological safeguards and email filters are not guaranteed to block all incoming malicious emails, humans are often the last line of defense an organization has against a phishing attack. Therefore, it is imperative that employees in these organizations are prepared in case of a real-world phishing scenario.

[0012] To help combat phishing threats, many organizations implement embedded phishing awareness training programs, where simulated phishing emails are sent to employees in an effort to train them to spot real phishing emails they may receive. The personnel executing these types of programs, or “training implementers,” use the results of these programs, in part, to assess the security risk of their organization. These results are usually measured as click rates - the number of people who clicked on a potentially malicious link or attachment out of the total number of people sent the simulated phish.

[0013] Summary of Invention

[0014] Organizations often evaluate the click rates that result from phishing awareness training exercises in a vacuum and are concerned when they do not decline. Click rates do not provide a complete picture of an organization’s phishing risk; they provide a single point of insight - what percentage of people “fell” for the phish. The fact that some phishing emails are more difficult to detect than others should be incorporated into the assessment of a simulated phishing email training exercise. The invention described herein addresses this concern.

[0015] The NIST Phish Scale and other exemplary Phish Scales in accordance with the invention are methods for training implementers to rate an email’s human phishing detection difficulty, thereby providing context to phishing awareness training click rates and giving organizations a more holistic view of their training program results. The Phish Scale assesses both the properties of a phishing email itself and the characteristics of the email’s recipients since user context plays a key role in interpreting click rates. The result is a human phishing detection difficulty rating that can be used as an additional metric in assessing the efficacy of phishing awareness training.

[0016] The Phish Scale may be applied to a whole organization or a specific, typically smaller, target audience. To apply broadly to a whole organization, job families are used when evaluating part of the premise alignment component (Has workplace relevance), ultimately resulting in separate detection difficulty ratings for each job family.

[0017] Additionally, the cues component of the Phish Scale assigns weights to cues when determining human phishing detection difficulty. Cues are weighted based on how people perceive them when checking email: more noticeable cues that alert people, indicating that an email may be a phish have a heavier weight than cues that people don’t typically interpret as being associated with phishing emails. Cue weighting can be accomplished in two primary ways: weighting by cue type or by weighting by individual cue.

[0018] According to an aspect of the invention a method of rating a phishing email’s human phishing detection difficulty includes identifying cues in content of the email tending to indicate that the email is a phishing email; assigning weights to identified cues; assessing a relative severity based on the weighted cues; and determining an overall human detection difficulty of the email based on the relative severity of the weighted cues and a total number of the identified cues.

[0019] Optionally, the step of assigning weights to identified cues is based on how an email viewer would perceive them, giving a relatively higher weight to cues that are relatively more noticeable and / or associated with phishing emails and a relatively lower weight to cues that are relatively less noticeable and / or associated with phishing emails.

[0020] Optionally, the step of assigning weights to identified cues is by cue type.

[0021] Optionally, the cue type is one or more of errors, technical indicators, visual presentation indicators, language and content, or common tactics.

[0022] Optionally, the step of assigning weights to identified cues is by individual cue. Optionally, an individual cue is identified as one or more of spelling errors; grammar irregularities; inconsistency; attachment type; sender display name and email address; URL hyperlinking; domain spoofing in email address; hyperlink domain spoofing; no / minimal branding and logos; branding / logo imitation; out-of-date branding / logos; unprofessional looking design or formatting; security indicators and icons; legal language / copyright info / disclaimers; distracting detail; requests for sensitive information; sense of urgency; threatening language; generic greeting; lack of personalization; lack of signer details; humanitarian appeals; too good to be true offers; you’re special; limited time offer; or poses as friend, colleague, supervisor, authority figure plays a key role in interpreting click rates. Optionally, the method of rating a phishing email’s human phishing detection difficulty also includes the steps of determining a target audience for the phishing email; determining a plurality of sub-groups of the target audience; and determining a plurality of respective premise alignments of the email by characterizing respective pertinences of a message premise of the email with respect to each respective subgroup of the target audience; wherein the step of determining an overall human detection difficulty of the email is additionally based on the respective premise alignments of the email.

[0023] Optionally, each one of the plurality of subgroups is a respective job family.

[0024] Optionally, determining a premise alignment of the email includes assigning a value to each of a plurality of premise alignment elements by characterizing respective pertinences of a message premise of the email.

[0025] Optionally, the plurality of premise alignment elements include; mimics a workplace process or practice; has workplace relevance; aligns with other situations or events, including external to the workplace; engenders concern over consequences for not clicking; and has been the subject of targeted training, specific warnings, or other exposure.

[0026] According to another aspect of the invention, a method of rating a phishing email’s human phishing detection difficulty includes determining a target audience for the phishing email; determining a plurality of sub-groups of the target audience; determining a plurality of respective premise alignments of the email by characterizing respective pertinences of a message premise of the email with respect to each respective sub-group of the target audience; and determining an overall human detection difficulty of the email based on the respective premise alignments of the email.

[0027] Optionally, each one of the plurality of subgroups is a respective job family.

[0028] Optionally, determining a premise alignment of the email includes assigning a value to each of a plurality of premise alignment elements.

[0029] Optionally, the plurality of premise alignment elements include; mimics a workplace process or practice; has workplace relevance; aligns with other situations or events, including external to the workplace; engenders concern over consequences for not clicking; and has been the subject of targeted training, specific warnings, or other exposure.

[0030] Optionally, the plurality of premise alignment elements include; mimics a workplace process or practice; and wherein determining a premise alignment of the email includes assigning a value for the mimics a workplace process or practice element based on: the plausibility of the premise for a work environment of each of the plurality of sub-groups of the target audience, whether the premise aligns for an organization of each of the plurality of sub-groups of the target audience, whether the organization of each of the plurality of sub-groups of the target audience sends / receives emails like this, and whether content, look, and feel of the phishing email mimics legitimate emails using this process / practice for each of the plurality of sub-groups of the target audience.

[0031] Optionally, the plurality of premise alignment elements include: engenders concern over consequences for not clicking; and wherein determining a premise alignment of the email includes assigning a value for the engenders concern over consequences for not clicking element based on: whether the premise evokes concern from an email recipient or includes missed information, a technical concern, an organizational concern, an organizational disciplinary action, a professional concern, or a personal disciplinary action.

[0032] The foregoing and other features of the invention are hereinafter described in greater detail with reference to the accompanying drawings.

[0033] Brief Description of the Drawings

[0034] FIG. 1 shows a block diagram of an exemplary method of rating a phishing email’s human phishing detection difficulty using cue weighting.

[0035] FIG. 2 shows an exemplary method of weighting by cue type.

[0036] FIG. 3 shows a block diagram of an exemplary method of rating a phishing email’s human phishing detection difficulty using job families.

[0037] FIG. 4 shows an exemplary method of job families as sub-groups of whole organization target audience

[0038] FIG. 5 shows a schematic diagram of a suitable computer system for running programs in accordance with exemplary embodiments of the invention.

[0039] Detailed Description

[0040] Using an exemplary Phish Scale to understand the detection difficulty of a phishing email helps phishing awareness training implementors in two primary ways. First, by providing context regarding training message clicks and reporting rates for a target audience. Phishing emails that are Very Difficult to detect would understandably result in high click rates when used in a simulated phishing exercise; Least Difficult emails would likely result in low click rates. However, when a phishing exercise yields unexpected results (e.g., a Least Difficult email that results in high click rates), it may indicate that modified or additional training is needed for the target audience.

[0041] Second, exemplary Phish Scales provide a way to characterize actual phishing threats so training implementors can reduce their organization’s security risk by tailoring training to the types of threats their organization faces while still maintaining a resilient security posture. One benefit of a strong and resilient security posture is safeguarding internal and external trust A robust phishing program should not be a stagnant “check the box” type of exercise, but rather an evolving part of a mature cybersecurity awareness and training program to provide mature, metrics- driven results. Organizations need to tailor their cybersecurity and awareness training programs to their unique environment, and employees' needs and requirements while still meeting their organization’s mission and risk tolerance. The level of security an organization implements should be commensurate with the risk and organizational purpose and operations. In other words, the higher the risk the organization encounters, the higher the level of security the organization should implement.

[0042] Lastly, an organization’s cybersecurity awareness and training program is not a silver bullet cure-all. An organization needs a multi-pronged approach to identifying, reacting to, and reporting suspicious phishing attempts. These tactics can be applied to both a user’s home and work environment. Thus, giving users concrete skills and knowledge can better prepare them to defend against potential phishing attempts. A combination of technology, processes, and people can effectively combat phishing attempts, protecting both the user and their organization.

[0043] An exemplary Phish Scale has two main components used collectively to determine human phishing detection difficulty:

[0044] 1) A scoring system for observable characteristics of the phishing email itself, such as the number of cues, nature of the cues, repetition of cues, and so on.

[0045] 2) A scoring system for alignment of the phishing email premise with respect to a target audience. A phishing email message’s content is measured by the indicators (cues) present in the email that may be used to detect the training phish. The premise measurement is related to current events, the environment of an organization, and the email recipient’s roles and responsibilities. Both components are first measured, then assigned categories representing relative ranges for each. The three categories used to reflect the effect of the “cues” contained in a phishing message are low (equating to fewer opportunities to detect the email as a phish), medium, and high (equating to more opportunities to detect the email as a phish). Similarly, the three categories used to characterize the premise alignment are weak, medium, and strong. The cues and premise alignment categories may then be interpreted collectively (e.g., combining the methods of FIGs. 1 and 3), mapping to a specific human detection difficulty rating for a phishing email.

[0046] Cues Component

[0047] The first component of an exemplary Phish Scale is a rating system for observable characteristics of the phishing email itself, referred to as email cues. Cues are the properties of an email that either compel a user to click on a fraudulent link or attachment or serve as red flags alerting the user that the email may be a phish. The cues in an email’s message provides an objective rating of the email itself; the rating is based on both the number of cues present in an email and the interpretation significance the cues pose when detecting a phish.

[0048] Referring now to FIG. 1, an exemplary method of rating a phishing email’s human phishing detection difficulty is shown at 100. At block 110, cues are identified in content of the email which tend to indicate that the email is a phishing email. At block 120, weights are assigned to identified cues. At block 130 a relative severity based on the weighted cues is assessed. At block 140, an overall human detection difficulty of the email is determined based on the relative severity of the weighted cues and a total number of the identified cues.

[0049] A lower number of insignificant cues in a phishing email indicates an email that is more difficult to detect as a phish; a higher number of significant cues indicates easier detection. The cue rating is categorized and, along with an email’s premise alignment category, is used to determine detection difficulty. When categorizing the number of cues in a phishing email, it may be important to first understand the types of cues that may be present in a phishing email, and where they occur. Phishing email cues may be categorized into five types: Errors - relating to spelling and grammar errors and inconsistencies contained in the message; Technical indicators - pertaining to email addresses, hyperlinks and attachments; Visual presentation indicators - relating to branding, logos, design and formatting; Language and content - such as a generic greeting and lack of signer details, use of time pressure and threatening language; and Common tactics - use of humanitarian appeals, too good to be true offers, time-limited offers, poses as a friend, colleague, or authority figure.

[0050] Each cue type has associated cues, listed in

[0051] Table 1.

[0052] Table 1. List of Cues by Type

[0053] When analyzing an email, an understanding of how to properly identify its cues is important. While the characteristics of each cue are described above, we now move to dissecting the anatomy of an email and highlighting where different types of cues are typically found, in an overall email, there are four major components:

[0054] • Header - includes From, Sent, and To lines

[0055] • Subject

[0056] • Attachment - if present, one or more downloadable files

[0057] • Message - including the Salutation (greeting), Body (primary content), Closing (signature), and Postscript (disclaimers)

[0058] “Error” and “Technical indicator” cue types can be found anywhere in an overall email, depending on the cue. “Visual presentation indicator" cue types are related to how an email is displayed, and therefore are usually found in an email’s message. “Language and content” and “Common tactic” cue types are more about the premise of the email and are located in the subject or message of an email.

[0059] When evaluating a phishing email, exemplary embodiments of the Phish Scale assign weights to cues when determining human phishing detection difficulty, using the list of cues in Table 1 . This evaluation examines the email by counting each instance of the cues listed, accounting for their associated weights. Cues are weighted based on how people perceive them when checking email: more noticeable cues have a heavier weight than cues that people don’t interpret as being associated with phishing emails. The weighting can be accomplished in two ways: by assigning weights to the type of cues or by assigning weights to individual cues within those types. The method of weighting and the weights should be determined prior to Phish Scale application.

[0060] At block 120, the step of assigning weights to identified cues may be based on how an email viewer would perceive them, giving a relatively higher weight to cues that are relatively more noticeable and / or associated with phishing emails and a relatively lower weight to cues that are relatively less noticeable and / or associated with phishing emails.

[0061] At block 120, the step of assigning weights to identified cues may be by cue type. Optionally, the cue type is one or more of errors, technical indicators, visual presentation indicators, language and content, or common tactics.

[0062] Alternatively, at block 120, the step of assigning weights to identified cues may be by individual cue. Optionally, an individual cue is identified as one or more of spelling errors; grammar irregularities; inconsistency; attachment type; sender display name and email address; URL hyperlinking; domain spoofing in email address; hyperlink domain spoofing; no / minimal branding and logos; branding / logo imitation; out-of-date branding / logos; unprofessional looking design or formatting; security indicators and icons; legal language / copyright info / disclaimers; distracting detail; requests for sensitive information; sense of urgency; threatening language; generic greeting; lack of personalization; lack of signer details; humanitarian appeals; too good to be true offers; you’re special; limited time offer; or poses as friend, colleague, supervisor, authority figure plays a key role in interpreting click rates.

[0063] The weighting of cues depends on their relative impact on the target audience in an organization. For example, when weighting by cue type, “common tactics” cues may be harder to detect as a phish than “visual presentation indicator” cues, and therefore be more heavily weighted (see FIG. 2). When weighting by individual cue, a phishing email that appears to come directly from an authority figure in upper management may have a higher weighting relative to the other cues since it elicits serious concerns and a deeper sense of needed action by the email recipient and therefore is harder to detect as a phish.

[0064] Going into more detail now, with regard to a specific example of cue weighting by type in accordance with the method 100 shown in FIG. 1 :

[0065] Step 1: Count instances of each cue present in the email. Note that some email characteristics can be identified and counted as multiple cues. For example, an email with a text link displayed as “www.niist.gov” that has an underlying Uniform Resource Locator (URL) to “commerce.gov” would be counted as both “Spelling errors” and “URL hyperlinking” cues.

[0066] Step 2: Sum the total number of cue instances for each of the five cue types. The sum for each cue type is that type’s count total.

[0067] Step 3: Use a look-up table, e.g., Table 2 to identify the corresponding weight for each cue type. Multiply the count total by the weight for each cue type. This product for each cue type is that type’s weighted total.

[0068] Step 4: Sum the five weighted totals. The result is the weighted cue rating that will be used to determine the cues category based on the mapping in Table 4. Table 2. Cue Type Weights

[0069] Errors I 3 (less difficult)

[0070] Visual presentation indicators 2

[0071] Language and Co 1 (more difficult)

[0072] Common Tactics 1 (more difficult)

[0073] The following equation (1) represents the calculation required to determine the weighted rating for the cues component by cue type.

[0074] For example, if a phishing email has 3 spelling errors, 2 grammatical errors, and 1 out-of-date logo, the CueTypeCountTotals are 5 “Error” cues and 1 “Visual presentation indicator” cue. The WeightedCueRating for the email is 17 ((5x3)+(1x2)).

[0075] Going into more detail now, with regard to a specific example of cue weighting by individual cue, in accordance with the method 100 shown in FIG. 1 :

[0076] Step 1: Count instances of each cue present in the email. The sum for each cue is the cue count total.

[0077] Step 2: Use a look-up table, e.g., Table 3 to identify the corresponding weight for each cue. Multiply the cue count total by the cue weight for each cue. This product for each cue is that cue’s weighted total.

[0078] Step 3: Sum the weighted totals. The result is the weighted cue rating that will be used to determine the cues category based on the mapping in Table 4.

[0079] Table 3. Individual Cue Weights

[0080] The equation (2) below represents the calculation required to determine the weighted rating for the cues component by individual cue.

[0081] WeightedCueRating = Xn=i CueCountTotalnx CueWeightn(2)

[0082] For example, if a phishing email has 3 hyperlinks spoofing a domain, makes 2 requests for sensitive information, includes an attachment, and appears to be from someone posing as an authority figure, the WeightedCueRating is 13 ((3x2)+(2x1)+(1x4)+(1x1)). Independent on the method used to calculate the WeightedCueRating, the mapping in a separate look-up table, e.g., Table 4 may be used to determine the cues category for a phishing email. Emails categorized as “low” are typically more difficult to detect than emails categorized as “high”. This cues category, combined with the premise alignment category is used to determine the overall human phishing detection difficulty of a phishing email.

[0083] Table 4. Cues Category Mapping

[0084] Premise Alignment Component

[0085] The second component of an exemplary Phish Scale focuses on the relationship between user context of the phishing email message - the premise alignment. Evaluating a phishing email’s premise alignment is a process of characterizing the pertinence of the email message premise for a target audience. A strong premise alignment indicates that the phishing email closely matches the work roles or responsibilities of the individual or organization. For example, the stronger an email’s premise alignment, the more difficult it is to detect the email as a phish. Inversely, the weaker an email’s premise alignment, the easier it is to detect the email as a phish. This section details the steps required to evaluate a phishing email’s premise alignment.

[0086] The premise alignment applies to a phishing email for a specific target audience; you can apply the premise alignment at various levels within your organization (e.g., divisions, departments, groups, teams) to contextualize click rates and their direct relationship to specific departments or employees. The premise alignment cannot be evaluated without knowledge of the target population’s context of work with respect to the premise of the phishing email’s message to accurately categorize premise alignment. As such, measuring a phishing email’s premise alignment should be performed by an individual with knowledge of the target audience’s work culture and responsibilities. Referring now to FIGs. 3 and 4, exemplary Phish Scales may also broaden the application of the metric from the narrow application to a target audience to a more holistic whole-organization approach. In doing so, the concept of job families — a group of positions in an organization that have similar knowledge, skills and functional requirements — is hereby introduced, providing greater precision to the evaluation metric.

[0087] A method of rating a phishing email s human phishing detection difficulty is shown at 300 in FIG. 3. At block 310, a target audience for the phishing email is determined. At block 320, a plurality of sub-groups of the target audience is determined. At block 330, a plurality of respective premise alignments of the email is determined by characterizing respective pertinences of a message premise of the email with respect to each respective sub-group of the target audience. At block 340, an overall human detection difficulty of the email is determined based on the respective premise alignments of the email.

[0088] With regard to the plurality of subgroups, each may be, for example, a respective job family.

[0089] Optionally, at block 330, determining a premise alignment of the email may include assigning a value to each of a plurality of premise alignment elements. Optionally, the plurality of premise alignment elements may include: mimics a workplace process or practice; has workplace relevance; aligns with other situations or events, including external to the workplace; engenders concern over consequences for not clicking; and has been the subject of targeted training, specific warnings, or other exposure.

[0090] In particular, for example, the “has workplace relevance” element may be expanded to consider whether there is relevance to a job family. Example job families may include, for example, administrative support, core mission employees (defined based on the workplace), facilities - field employees, facilities - office employees, legal, management, and organization support staff.

[0091] Measuring a phishing email’s premise alignment begins with assigning a numerical value to the five individual premise alignment elements. The premise alignment rating is then calculated using these applicability scores. This final premise alignment rating is then mapped to a strong, medium, or weak premise alignment category, which, considered with the cues category, is used to determine an email’s detection difficulty. Exemplary embodiments including premise alignment with job families, as compared with base embodiments without is compared in Table 5 below.

[0092] Table 5. Premise alignment evaluation flow diagram

[0093] Determine applicability scores for each element i „ u <-4. s.

[0094] ; Determine applicability scores for the remaining four elements

[0095] T xp

[0096] Premise Alignment Rating Calculation i

[0097] Eva holistically to determine Detection Difficulty Rating i

[0098] Evaluate Premise Alignment and Cues Categories holistically to determine Detection Difficulty

[0099] Rating

[0100] Ideally, Job Families should be created prior to application of the Phish Scale and should be customized and tailored to an organization’s work environment and mission. An exemplary embodiment of job families commonly found in many organizations is listed below:

[0101] • Administrative support

[0102] • Core mission employees

[0103] • Facilities -field employees

[0104] • Facilities - office employees

[0105] • Legal

[0106] • Management

[0107] • Organization support staff

[0108] An understanding of the five elements that are the basis for this process is needed before evaluating a phishing email. A description of these elements and how to use them to calculate the premise alignment rating follows below.

[0109] Element 1 mimics a workplace process or practice. This element reflects the relevance of the email's premise to a process or practice of the target audience. Any processes or functions that typically happen in your organization should be considered for this element.

[0110] For example, if the target audience typically receives official organization chat notifications via an app, an email that notifies the recipient of a missed chat message would have a lower applicability score for this element. However, if email is the typical mechanism for chat notifications, that email would have a higher applicability score for this element.

[0111] Element 2 has workplace relevance. This element reflects the relevance of the premise to the work of the target audience - including their roles and responsibilities. It is usually important to have knowledge of your target audience’s job duties and job functions, in order to appropriately evaluate this element.

[0112] For example, if the target audience is the finance department and an email has a premise of a late or missed payment, that email would have a higher applicability score for this element. Another consideration for this element is the sender's domain. If the domain name of the email sender is the same or similar to your organization’s domain, (e.g., john.doe@nist.gov is receiving an email from jane.doe@nist.gov), the email would have a higher applicability score for this element. Similarly, with the prevalence of users using personal email for business purposes, senders with familiar names and public domains should be considered for this element. If the target audience is familiar with an employee at your organization, John Doe, then an email from “john.doe@gmail.com” would have a higher applicability score.

[0113] Also, consider if the email is relevant to the work and responsibilities of all or a subset of the target audience. For example, if the premise of the email has a higher contextual alignment, but only to a small portion of the target audience, then the workplace relevance element may be assigned a mid-range applicability score.

[0114] Element 3 is based on alignment with other situations or events, including external to the workplace. This element is based on timing and reflects the alignment of the premise to internal and external situations or events directly or indirectly affecting your organization.

[0115] Examples of calendar-related external events are Christmas, New Year's Day, and Memorial Day; examples of internal events are a new organization director / president / leader hired, the opening of a new branch office, and the start or end of a fiscal year. If the event is current and relevant for the target audience, then the applicability score for this element should be higher (e.g., a phishing email sent around February 14, related to Valentine’s Day). Conversely, if an event is not current or relevant to the intended audience, then the premise should have a lower applicability score (e.g., a phishing email about an office winter holiday party sent in July).

[0116] Element 4 is based on engendering concern over consequences for NOT clicking. This element reflects potentially harmful ramifications if no action is taken (raising the likelihood of the phishing email recipient clicking on fraudulent links or attachments). Certain email premises elicit this reaction in recipients more than others.

[0117] For example, a phishing email acting on a user’s fear of missing out (e.g., a missed message, an informational notice) may not produce the same response as an email with a more serious allegation (e.g., potential leak of protected health information, exposure of personal information, ransomware). The former would yield a lower rating for this element than the latter.

[0118] Element 5 is based on the effects of training on the target audience, including phishing-related organizational training on phish detection, specific warnings or other exposure. Ideally, employees who have had exposure to some form of IT security training related to phishing would be more judicious in identifying an email as a phish (a higher applicability score for this element) versus those who have not received training (a lower applicability score for this element). This phishing training is not constrained to components within formal IT security training courses; training refers to any awareness materials or guidance to which the target audience has been exposed. Training may refer to:

[0119] • formal IT cybersecurity awareness and training programs;

[0120] • educational materials or seminars on how to identify a phishing email; or

[0121] • organizational emails alerting employees to be on the lookout for phishing attempts or warning against specific types of phishing attacks.

[0122] If the phishing email recipient has had a considerable amount of training in the detection of phishing emails, this element may have a higher applicability score.

[0123] To calculate the premise alignment rating, assign each of the five premise alignment elements a numerical value using the applicability score scale in Table 4. An element with a lower applicability score indicates that there is a complete mismatch in the relevancy of the phishing email to the target audience. Inversely, a high applicability score indicates that the phishing email message is very applicable to the target audience.

[0124] Table 4. Premise Alignment Applicability Score Scale

[0125] • Extreme applicability, alignment, or relevancy 9

[0126] • Moderate applicability, alignment, or relevancy 6

[0127] • Low applicability, alignment, or relevancy 3

[0128] • No applicability, no alignment, or no relevancy 0

[0129] Table 5 provides criteria for the five premise alignment elements. Use these criteria, along with the applicability scale, to determine the applicability score for each element.

[0130] Table 5. Premise Alignment Elements Rating Criteria

[0131] Additional optional features or modifications to embodiments of the Phish Scale include increased granularity for the specification of two premise alignment elements.

[0132] Specifically, the plurality of premise alignment elements may include: mimics a workplace process or practice. In that case, determining a premise alignment of the email may include assigning a value for the mimics a workplace process or practice element based on: 1 ) the plausibility of the premise for a work environment of each of the plurality of sub-groups of the target audience,

[0133] 2) whether the premise aligns for an organization of each of the plurality of subgroups of the target audience, 3) whether the organization of each of the plurality of sub-groups of the target audience sends / receives emails like this, and 4) whether content, look, and feel of the phishing email mimics legitimate emails using this process / practice for each of the plurality of sub-groups of the target audience.

[0134] Element 1 : Mimics a workplace process or practice

[0135] Specification for determining applicability score for this element:

[0136] • 0 score: premise is not plausible for a work environment

[0137] • low score: premise is plausible for a work environment (e.g., new voicemail, package delivery, order confirmation, notice of invoice),

[0138] BUT does not align for the specific organization (therefore not handled by organization in any way) OR organization or external entity would not handle process in this way (e.g., wouldn't send notice / info via email)

[0139] • middle score: premise is plausible for a work environment,

[0140] AND aligns for the specific org,

[0141] AND the org sends / receives emails like this,

[0142] BUT content, look, and / or feel are different from actual org or external entity emails using this process / practice

[0143] • high score: premise is plausible for a work environment,

[0144] AND aligns for the specific org,

[0145] AND the org sends / receives emails like this,

[0146] AND content, look, and feel of phishing email mimics actual org or external entity emails using this process / practice

[0147] Alternatively, for example, the plurality of premise alignment elements may include: engenders concern over consequences for not clicking. In that case, determining a premise alignment of the email may include assigning a value for the engenders concern over consequences for not clicking element based on whether the premise evokes concern from an email recipient or includes missed information, a technical concern, an organizational concern, an organizational disciplinary action, a professional concern, or a personal disciplinary action.

[0148] Element 4: Engenders concern over consequences for NOT clicking

[0149] Specification for determining applicability score for this element:

[0150] • 0 score: premise does not evoke any concern from the email recipient

[0151] • Low score: for phishing emails with a premise around missed information (e.g., missed chat message) or a technical concern (e.g, software updates)

[0152] • Middle score: for phishing emails with a premise around organizational concerns or organizational disciplinary action (e.g., government sanctions) • High score: for phishing emails with a premise around personal professional concerns or personal disciplinary actions (e.g., employment termination)

[0153] The applicability score for each premise alignment element (ElementApplicabilityScore) is used to calculate the final premise alignment rating (PArating) according to equation (3) below. Note: element five pertains to training and helps with detection, therefore, the numerical value assigned to this element is subtracted from the total sum.

[0154] PArating = (Sn=i Element ApplicabilityScoren) — ElementApplicabilityScore^ (3)

[0155] The highest possible premise alignment rating (PArating) in this case is 36, indicating that a phishing email message matches up with the target audience and the target audience has not had any related training nor received any prior alert or warning about an upcoming phishing exercise. The lowest possible premise alignment rating (PArating) in this case is -9, indicating that the phishing email is a complete mismatch with the target audience and they have received prior phishing- related training, alerts or warnings.

[0156] Once the premise alignment rating (PArating) is calculated, it can be mapped to one of the three premise alignment categories:

[0157] • Strong - the alignment of the phishing email’s premise to the target audience is high, making the email difficult to detect a phish

[0158] • Medium - the alignment of the phishing email’s premise to the target audience is moderate

[0159] • Weak - the alignment of the phishing email’s premise to the target audience is low, making the email less difficult to detect as a phish

[0160] The mapping depicted in Table 6 is used to determine the premise alignment category for a phishing email based on the final premise alignment rating for that email.

[0161] Table 6. Premise Alignment Category Mapping

[0162] The weak, medium, or strong premise alignment category carries through into further analysis, along with the cues category. Detection Difficulty Analysis

[0163] The final step in applying the Phish Scale to a phishing email is to determine the overall detection difficulty of an email. The cues categorization and the category for premise alignment for a phishing email are analyzed collectively to determine the email’s detection difficulty, as shown in Table 7. Table 7. The Phish Scale - Detection Difficulty

[0164] Phishing emails with a low cues categorization and a strong premise alignment are more difficult for a human to detect as a phish than those with a high cues categorization and a weak premise alignment. For example, a phishing email categorized as “Low” cues and “Medium" premise alignment is “Very difficult” to detect as a phish; a cues category of “Medium”, with a “Weak” premise alignment rating has an overall detection difficulty rating of “Least difficult”. The groupings, evaluations, comparisons, weightings, and other processes, described herein may be embodied in, and fully automated via, software code modules executed by a computing system that includes one or more general purpose computers or processors. The code modules may be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all the methods may alternatively be embodied in specialized computer hardware. In addition, the components referred to herein may be implemented in hardware, software, firmware, or a combination thereof.

[0165] It should be understood that the calculations may be performed by any suitable computer system, such as that diagrammatically shown in FIG. 5. Data is entered into system 500 via any suitable type of user interface 516, and may be stored in memory 512, which may be any suitable type of computer readable and programmable memory and is preferably a non-transitory, computer readable storage medium. Calculations are performed by processor 514, which may be any suitable type of computer processor and may be displayed to the user on display 518, which may be any suitable type of computer display. Processor 514 may be associated with, or incorporated into, any suitable type of computing device, for example, a personal computer or a programmable logic controller. The display 518, the processor 514, the memory 512 and any associated computer readable recording media are in communication with one another by any suitable type of data bus, as is well known in the art.

[0166] Examples of computer-readable recording media include non-transitory storage media, a magnetic recording apparatus, an optical disk, a magneto-optical disk, and / or a semiconductor memory (for example, RAM, ROM, etc.). Examples of magnetic recording apparatus that may be used in addition to memory 512, or in place of memory 512, include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape (MT). Examples of the optical disk include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc-Read Only Memory), and a CD-R (Recordable) / RW. It should be understood that non-transitory computer-readable media include all computer-readable media except for a transitory, propagating signal.

[0167] Many other variations than those described herein will be apparent from this disclosure. For example, depending on the embodiment, certain acts, events, or functions of any of the algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of the algorithms). Moreover, in certain embodiments, acts or events can be performed concurrently, e.g., through multithreaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially. In addition, different tasks or processes can be performed by different machines and / or computing systems that can function together.

[0168] Any logical blocks, modules, and algorithm elements described or used in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and elements have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. The described functionality can be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.

[0169] The various illustrative logical blocks and modules described or used in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a processing unit or processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can be a microprocessor, but in the alternative, the processor can be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor can include electrical circuitry configured to process computer-executable instructions. In another embodiment, a processor includes an FPGA or other programmable device that performs logic operations without processing computerexecutable instructions. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor may also include primarily analog components. For example, some or all of the signal processing algorithms described herein may be implemented in analog circuitry or mixed analog and digital circuitry. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.

[0170] The elements of a method, process, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module stored in one or more memory devices and executed by one or more processors, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of non- transitory computer-readable storage medium, media, or physical computer storage known in the art. An example storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The storage medium can be volatile or nonvolatile.

[0171] While one or more embodiments have been shown and described, modifications and substitutions may be made thereto without departing from the spirit and scope of the invention. Accordingly, it is to be understood that the present invention has been described by way of illustrations and not limitation. Embodiments herein can be used independently or can be combined.

[0172] All ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other. The ranges are continuous and thus contain every value and subset thereof in the range. Unless otherwise stated or contextually inapplicable, all percentages, when expressing a quantity, are weight percentages. The suffix (s) as used herein is intended to include both the singular and the plural of the term that it modifies, thereby including at least one of that term (e.g., the colorant(s) includes at least one colorants). Option, optional, or optionally means that the subsequently described event or circumstance can or cannot occur, and that the description includes instances where the event occurs and instances where it does not. As used herein, combination is inclusive of blends, mixtures, alloys, reaction products, collection of elements, and the like. As used herein, a combination thereof refers to a combination comprising at least one of the named constituents, components, compounds, or elements, optionally together with one or more of the same class of constituents, components, compounds, or elements.

[0173] All references are incorporated herein by reference.

[0174] The use of the terms “a,” “an,” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. It can further be noted that the terms first, second, primary, secondary, and the like herein do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. For example, a first current could be termed a second current, and, similarly, a second current could be termed a first current, without departing from the scope of the various described embodiments. The first current and the second current are both currents, but they are not the same condition unless explicitly stated as such.

[0175] The modifier about used in connection with a quantity is inclusive of the stated value and has the meaning dictated by the context (e.g., it includes the degree of error associated with measurement of the particular quantity). The conjunction or is used to link objects of a list or alternatives and is not disjunctive; rather the elements can be used separately or can be combined together under appropriate circumstances.

[0176] Although the invention has been shown and described with respect to a certain embodiment or embodiments, it is obvious that equivalent alterations and modifications will occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In particular regard to the various functions performed by the above described elements (components, assemblies, devices, compositions, etc.), the terms (including a reference to a "means") used to describe such elements are intended to correspond, unless otherwise indicated, to any element which performs the specified function of the described element (i.e., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary embodiment or embodiments of the invention. In addition, while a particular feature of the invention may have been described above with respect to only one or more of several illustrated embodiments, such feature may be combined with one or more other features of the other embodiments, as may be desired and advantageous for any given or particular application.

Claims

Claims1. A method of rating a phishing email’s human phishing detection difficulty comprising the steps of: identifying cues in content of the email tending to indicate that the email is a phishing email; assigning weights to identified cues; assessing a relative severity based on the weighted cues; and determining an overall human detection difficulty of the email based on the relative severity of the weighted cues and a total number of the identified cues.

2. The method of rating a phishing email’s human phishing detection difficulty of claim 1 , wherein the step of assigning weights to identified cues is based on how an email viewer would perceive them, giving a relatively higher weight to cues that are relatively more noticeable and / or associated with phishing emails and a relatively lower weight to cues that are relatively less noticeable and / or associated with phishing emails.

3. The method of rating a phishing email’s human phishing detection difficulty of claim 1 , wherein the step of assigning weights to identified cues is by cue type.

4. The method of rating a phishing email’s human phishing detection difficulty of claim 3, wherein the cue type is one or more of errors, technical indicators, visual presentation indicators, language and content, or common tactics.

5. The method of rating a phishing email’s human phishing detection difficulty of claim 1 , wherein the step of assigning weights to identified cues is by individual cue.

6. The method of rating a phishing email’s human phishing detection difficulty of claim 5, wherein an individual cue is identified as one or more of spelling errors; grammar irregularities; inconsistency; attachment type; sender display name and email address; URL hyperlinking; domain spoofing in email address; hyperlinkdomain spoofing; no / minimal branding and logos; branding / logo imitation; out-of-date branding / logos; unprofessional looking design or formatting; security indicators and icons; legal language / copyright info / disclaimers; distracting detail; requests for sensitive information; sense of urgency; threatening language; generic greeting; lack of personalization; lack of signer details; humanitarian appeals; too good to be true offers; you’re special; limited time offer; or poses as friend, colleague, supervisor, authority figure plays a key role in interpreting click rates.

7. The method of rating a phishing email’s human phishing detection difficulty of claim 1 , further comprising the step of: determining a target audience for the phishing email; determining a plurality of sub-groups of the target audience; and determining a plurality of respective premise alignments of the email by characterizing respective pertinences of a message premise of the email with respect to each respective sub-group of the target audience; wherein the step of determining an overall human detection difficulty of the email is additionally based on the respective premise alignments of the email.

8. The method of rating a phishing email’s human phishing detection difficulty of claim 7, wherein each one of the plurality of subgroups is a respective job family.

9. The method of rating a phishing email’s human phishing detection difficulty of claim 7, wherein determining a premise alignment of the email includes assigning a value to each of a plurality of premise alignment elements by characterizing respective pertinences of a message premise of the email.

10. The method of rating a phishing email’s human phishing detection difficulty of claim 9, wherein the plurality of premise alignment elements include: mimics a workplace process or practice; has workplace relevance; aligns with other situations or events, including external to the workplace; engenders concern over consequences for not clicking; and has been the subject of targeted training, specific warnings, or other exposure.

11. A method of rating a phishing email’s human phishing detection difficulty comprising the steps of: determining a target audience for the phishing email; determining a plurality of sub-groups of the target audience; determining a plurality of respective premise alignments of the email by characterizing respective pertinences of a message premise of the email with respect to each respective sub-group of the target audience; and determining an overall human detection difficulty of the email based on the respective premise alignments of the email.

12. The method of rating a phishing email’s human phishing detection difficulty of claim 11 , wherein each one of the plurality of subgroups is a respective job family.

13. The method of rating a phishing email’s human phishing detection difficulty of claim 11 , wherein determining a premise alignment of the email includes assigning a value to each of a plurality of premise alignment elements.

14. The method of rating a phishing email’s human phishing detection difficulty of claim 13, wherein the plurality of premise alignment elements include; mimics a workplace process or practice; has workplace relevance; aligns with other situations or events, including external to the workplace; engenders concern over consequences for not clicking; and has been the subject of targeted training, specific warnings, or other exposure.

15. The method of rating a phishing email’s human phishing detection difficulty of claim 11, wherein the plurality of premise alignment elements include; mimics a workplace process or practice; and wherein determining a premise alignment of the email includes assigning a value for the mimics a workplace process or practice element based on: the plausibility of the premise for a work environment of each of the plurality of sub-groups of the target audience, whether the premise aligns for an organization of each of the plurality of sub-groups of the target audience,whether the organization of each of the plurality of sub-groups of the target audience sends / receives emails like this, and whether content, look, and feel of the phishing email mimics legitimate emails using this process / practice for each of the plurality of sub-groups of the 5 target audience.

16. The method of rating a phishing email’s human phishing detection difficulty of claim 11, wherein the plurality of premise alignment elements include: engenders concern over consequences for not clicking; and10 wherein determining a premise alignment of the email includes assigning a value for the engenders concern over consequences for not clicking element based on: whether the premise evokes concern from an email recipient or includes missed information, a technical concern, an organizational concern, 15 an organizational disciplinary action, a professional concern, or a personal disciplinary action.

Images