Unstructured text classification

Abstract

The technique described herein uses a classifier that is both accurate and fast to identify malicious URLs. This technique is particularly well-suited for use as a real-time URL security analysis tool because it can process URLs quickly and issue warnings when malicious URLs are identified. The fast processing speed of the technique described herein is partly achieved by using only a single input signal, the URL itself. The high accuracy produced by the technique described herein is achieved through analysis of unstructured text at both the character-level and word-level. The technique described herein utilizes both character-level and word-level information from the incoming URL.

Description

[0001] Related applications This application is a divisional application of the invention patent application with application number 202180026164.X and invention title "Unstructured Text Classification". Background Technology

[0002] Electronic content such as web pages and emails are widely used communication tools, providing a fast and inexpensive way for individuals and groups to communicate. A growing number of spam emails and web pages, phishing emails and web pages, and emails containing or linking to malicious programs such as malware and ransomware pose risks to users, companies, and computing devices. In the past, efforts have been made to identify unwanted and / or malicious content before users receive it using filtering systems. Filtering systems can use natural language processing or other mechanisms to categorize emails or web pages based on various characteristics. Many of these characteristics are constantly changing. Coarse filters using a "blacklist" of malicious characteristics miss a large number of newly added malicious content. For example, it is easy to create and start using new Uniform Resource Locators (URLs) that will not appear in any lists. Currently, no filtering system is perfect. Some benign content is misclassified as malicious, while other malicious content reaches the user. Furthermore, real-time filters used during browsing can significantly increase latency in the browsing experience. Summary of the Invention

[0003] The invention summary is provided to present a selection of concepts in a simplified form, which will be further described in the detailed description below. This invention summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter.

[0004] The technique described herein uses a classifier to identify malicious URLs or other unstructured text. This classifier uses computational resources more efficiently by eliminating signals typically processed when classifying URLs. This technique can use the URL itself as the sole input. Using a single input is more efficient than using multiple inputs. Using a single input also reduces latency because less information is processed, and less time is required.

[0005] This technique uses a single input, whereas existing techniques typically use multiple inputs (e.g., context, metadata) that each needs to be processed. Systems using multiple signals may not achieve the speed required for real-time URL evaluation. URLs are difficult to evaluate because they do not follow grammatical conventions and include meaningless words and terms that would otherwise be impossible to classify accurately using natural language processing techniques trained and built using grammatical text as input. The high accuracy produced by the technique described here is achieved by analyzing unstructured text at both the character-by-character and word-by-word levels. The technique described here uses character-level and word-level information from the incoming URL and does not rely on manually crafted features or feature engineering. Processing URLs into character groups and one or more words using a classifier provides higher accuracy compared to evaluating URLs as only characters or words.

[0006] Parallel layers evaluating words can each find groups of words of varying lengths. For example, the first layer might evaluate a single word, the second layer might evaluate a group of two words, and so on. These layers can be parallel. The outputs of the parallel text processing layers are ultimately merged in a classifier and processed together. The parallel use of convolutional layers allows the technique to generalize better to new URLs. The technique described here achieves a true positive rate with an extremely low false positive rate, surpassing traditional text classification methods, which is crucial for healthy operation at internet scale. This is achieved by processing characters and words from URLs in parallel and combining them with other features described herein.

[0007] This technology is particularly well-suited for use as a real-time URL security analysis tool because it can process URLs quickly and generate warnings or other automated responses when malicious URLs are identified. Real-time URL security analysis assessments can be used on the client or server side for multiple applications, including evaluating links during a user's browser session, when an email is received, when a social media post containing a URL is published, and when it is liked. Attached Figure Description

[0008] The aspects of the present invention will now be described in detail with reference to the accompanying drawings, in which: Figure 1 This is a block diagram of an example operating environment applicable to various aspects of implementing this technology; Figure 2 This is a diagram illustrating the selection of an environment according to one aspect of the technology described herein; Figure 3 This is a diagram illustrating a convolution operator applied to embedding analogy symbol vectors in analogical sentences; Figure 4 This is an exemplary process flowchart illustrating the process of forming the output feature vector; Figure 5 This is an exemplary process flowchart illustrating the recognition process of forming decision statistics from the output feature vectors generated by an exemplary text processing block; Figure 6 This is a block diagram of an exemplary computing environment applicable to implementing one aspect of this technology; Figure 7 This is a flowchart illustrating a method for classifying URLs according to one aspect of the technology described herein; Figure 8 This is a flowchart illustrating a method for classifying URLs according to one aspect of the technology described herein; and Figure 9 This is a flowchart illustrating a method for classifying URLs according to one aspect of the technology described herein. Detailed Implementation

[0009] The subject matter of various aspects of the technology is specifically described herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways to include different steps or combinations of steps similar to those described herein, and in combination with other current or future technologies. Furthermore, although the terms “step” and / or “block” may be used herein to denote different elements of the method employed, these terms should not be construed as implying any particular order between or between the various steps disclosed herein, unless and only if such order explicitly describes the individual steps.

[0010] The technique described here uses a classifier that is both accurate and fast to identify malicious URLs. Certain aspects of this technique are particularly well-suited for use as a real-time URL security analysis tool because it can process URLs quickly and generate alerts or other automated responses when a malicious URL is identified. Real-time URL security analysis assessments can be used on the client or server side for multiple applications, including evaluating links during a user's browser session, when an email is received, when a social media post containing a URL is published, and when it is liked.

[0011] The technique described herein uses computational resources more efficiently because the URL to be classified may be the only input to the classifier. Using a single input also reduces latency because less information needs to be processed. This technique uses a single input, whereas existing techniques typically use multiple inputs (e.g., context, metadata) that each requires processing. The fast processing speed of the technique described herein is partly achieved by using only a single input signal, which is the URL itself. In some respects, the URL is the only input to the classifier. Using only a single signal speeds up the evaluation process by eliminating the processing steps required to evaluate each input signal. Systems using multiple signals may not be able to achieve the speed required to evaluate URLs in real time. Some aspects of this technique are not limited to use with URLs and can also be used with other unstructured text in electronic documents, such as emails, social media posts, and web pages. In some respects, the single unstructured text from the content to be classified is the only input to the classification process. As used herein, unstructured means without metadata.

[0012] The technique described herein produces improved accuracy. The high accuracy achieved by this technique is obtained by analyzing unstructured text at both the character-by-character and word-by-word levels. The technique described herein uses character-level and word-level information from the incoming URL and does not rely on manually crafted features or feature engineering. Processing URLs into character groups and one or more words using a classifier provides greater accuracy compared to evaluating URLs simply as characters or words.

[0013] The technique described herein uses multiple parallel convolutional layers to process words and characters from a URL simultaneously. On one hand, the classifier architecture includes word-level paths running in parallel with character-level paths. The word-level paths receive word-level embedding vectors as input, while the character-level paths receive character-level embedding vectors as input.

[0014] In the character-level path, accuracy is improved by using parallel character-level layers, each receiving n-grams of different lengths. For example, one parallel layer might process an n-gram of two characters (i.e., a bigram), another might process an n-gram of three characters, yet another would process four characters, and so on. The outputs of the parallel character processing layers are ultimately merged in the classifier and processed together.

[0015] Parallel layers evaluating words can each find groups of words of varying lengths. For example, the first layer might evaluate a single word, the second layer might evaluate a group of two words, and so on. These layers can be parallel. The outputs of the parallel text processing layers are ultimately merged in a classifier and processed together. The parallel use of convolutional layers enables this technique to generalize better to new URLs. The technique described here achieves a true positive rate with an extremely low false positive rate, surpassing traditional text classification methods, which is crucial for healthy operation at internet scale. This is achieved by processing characters and words from URLs in parallel and combining them with other features described herein.

[0016] Many computer security-related services require verifying that a URL (Uniform Resource Locator) does not provide the location of a malicious webpage on the Internet. Specifically, this technique can generate URL classifications to detect phishing attacks. Phishing can be broadly defined as a social engineering attack conducted through electronic channels that tricks humans into performing certain actions for the attacker's benefit. These actions might include obtaining passwords or bank account numbers, which are then sold on the black market. To this end, the technique described herein provides a novel character-level and word-level deep learning model that learns to detect malicious URLs associated with these phishing websites.

[0017] Many signals can be considered features of a model for detecting phishing pages, including the requester's URL, URI, static HTML content, the page's DOM, and screenshots of the page. Most of these raw signals require extensive transformations before they can be used as useful features, which typically involves significant time-consuming feature engineering. The technique described here takes a URL as input and predicts whether that URL belongs to a phishing attack or other malicious activity. This technique can also be applied to other unstructured text on web pages, emails, and other digital content. While the technique described here is not limited to use with URLs, much of the following description will use a URL as example input.

[0018] In addition to real-time classification, the techniques described here are useful for detecting “potential” phishing domains. Potential domains are newly registered domains that are not yet hosting malicious content, but are very likely to do so in the near future. In this case, the domain does not host any content, and the only signal that can be used is usually the URL.

[0019] The technique described herein utilizes contextual embeddings. The basic idea of ​​contextual embeddings is to construct a vocabulary of words from a text corpus and assign a low-dimensional, randomly initialized dense vector to each word. During unsupervised training, each sentence in the corpus is labeled to extract words, and the algorithm aims to predict the surrounding words of a given word (i.e., skipgram architecture) or predict a single word based on its surrounding words or context (i.e., continuous bag-of-words, or CBOW architecture). The vector assigned to each word is adjusted to optimize this task, and the trained embedding vectors are expected to reflect contextual relationships. After training, words with similar meanings are closely clustered in the embedding space. Word embeddings trained on large text corpora can be used as input to downstream models. However, such pre-trained vectors are useless in the phishing detection problem because the words appearing in a URL corpus are dissimilar to any language corpus. Therefore, the technique described herein constructs a unique set of embeddings for word-based analysis, essentially building a unique vocabulary for text strings found in a URL corpus. In the context of a URL, "words" can be a series of n-grams of varying sizes found in an actual URL corpus. The words in the URL "msn.com" might include "msn", ".", and "com". N-grams that reappear in the URL corpus at frequencies above a threshold will be added to the URL vocabulary. In one aspect, a word is represented by a series of n-grams. For example, the word "accommodation" can be represented by a vector of 4-grams: <"acco", "ccom", "comm", "ommo", "mmod", "moda", "odat", "dati", "atio", "tion">. In another aspect, word embedding vectors are generated based on n-grams. In the sense of having similar n-grams, close words will have similar embedding vectors.

[0020] Training a classifier requires training data. In this case, the training data might be a set of URLs and a binary phishing rating (phishing / benign) for each URL. The rating is used as a label for training and evaluating the model used in the classification. The training data can be based on phishing classification of real-world URLs. Typically, it might be necessary to build a training set where at least 5% of the data instances are phishing URLs.

[0021] As previously mentioned, the technique described here uses character and word embeddings that are known to be more efficient than the traditional Bag-of-Words (BOW) technique. For character embeddings, an alphabet of all characters appearing in the training corpus is constructed, and then a lookup table is assigned that takes the integer index of each character in the alphabet and returns a low-dimensional dense vector trained during network training. Two additional indices are reserved for unknown characters and whitespace. The technique also defines the maximum number of characters to be processed in each URL. Longer URLs are pruned, and shorter URLs are padded with empty characters to meet this maximum value. Using these hyperparameter settings, each input URL is mapped to a dense matrix of character embeddings, as referenced later. Figure 3 More detailed description.

[0022] definition Artificial neural networks (ANNs) are computational processing modules in hardware or software that are inspired by elements similar to those in biological neurons. For example, an output with scalar elements of length N, v1, v2, ... . On one hand, nonlinearity can be achieved through functions such as the sign function, the hyperbolic tangent (tanh) function, a function that restricts the maximum and / or minimum values ​​to a programmable output, or the ReLU function. An ANN can produce an output equal to h(v1, v2, ...). wl+v2 w2+...+ vN The output of the network is wN + b0. This network is "learned" based on the input and weight adjustment method. The weights can be iteratively adjusted based on evaluating the ANN on the dataset, while also modifying the weights according to the learning object. One or more classification techniques can be used, such as gradient boosting decision tree binary classification. On the one hand, decision trees can be used as classifiers because they may be easier to visualize and integrate into practical solutions.

[0023] A convolutional layer is a layer in a Convolutional Neural Network (CNN) hierarchical structure. A layer is a group of adjacent neurons with small, contiguous receptive fields. A CNN may have many defined layers. In one respect, the number of network layers can be defined within a CNN component, such as 6, 16, 19, or 38 layers.

[0024] A Convolutional Neural Network (CNN) is an ANN that performs operations using convolutional operations, typically for image data. A CNN may have several layers stacked together to reflect higher-level neuron processing. CNN neurons may be fully or partially connected to subsequent layers. One or more layers can be skipped when feeding neuron outputs to higher layers. Convolutions can be performed at the same resolution as the input, or with a stride different than 1 for data reduction. The output of a layer can be reduced in resolution by pooling layers. A CNN may consist of several adjacent neurons that process inputs only in a receptive field much smaller than the entire image. Examples of CNN components include ZF Net, AlexNet, GoogLeNet, LeNet, VGGNet, VGG, ResNet, DenseNet, etc.

[0025] A corpus is a collection of similar data samples. For example, a corpus of URLs associated with web traffic is a set of URLs monitored by a security component over a period of time. The security component can perform analysis on the URLs and associate tags from the security analysis with each URL. The result is a set of pairs containing URLs and associated tags. The collection of these pairs is the corpus.

[0026] Norms are generally positive length measures over vector spaces. In one respect, norms include semi-norms. The 2-norm is the square root of the sum of the squares of the elements (2-norm). The 1-norm is the sum of the absolute values ​​of the vector elements. The p-norm is a quantity raised to the power of 1 / p, comprising the sum of the absolute values ​​of the vector elements, where each absolute value is raised to the power of p. The infinity norm is the maximum value of the absolute value of each vector element.

[0027] A residual neural network is an artificial neural network that feeds neural outputs to layers outside of adjacent layers, skipping one or more layers, so that the receiving layer can form a result that includes neural inputs from non-adjacent layers.

[0028] Having briefly described an overview of the aspects of the technology described herein, an exemplary operating environment that can implement the aspects of the technology described herein is described below.

[0029] Now turn to the diagram. (Reference) Figure 1A block diagram illustrating an operating environment 100 in which various aspects of this disclosure may be employed is provided. It should be understood that such and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, commands, and function groups) may be used to supplement or replace those shown, and some elements may be omitted entirely for clarity. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in combination with other components, and implemented in any suitable combination and location. The various functions described herein as being performed by one or more entities can be performed by hardware, firmware, and / or software. For example, some functions can be performed by a processor executing instructions stored in memory.

[0030] Among other components not shown, the example operating environment 100 includes multiple user devices, such as user devices 102a and 102b to 102n; multiple websites, such as websites 104a and 104b to 104n; and a search server 106. It should be understood that... Figure 1 The environment 100 shown is an example of a suitable operating environment. Each component shown in Figure 1... Figure 1 This can be achieved using any type of computing device, such as combined with Figure 6 The described computing device 600, for example, allows these components to communicate with each other via a network 110, which may include, but is not limited to, one or more local area networks (LANs) and / or wide area networks (WANs). In an exemplary implementation, network 110 includes the Internet and / or cellular networks, as well as any of a variety of possible public and / or private networks.

[0031] It should be understood that, within the scope of this disclosure, any number of user devices, servers, and websites can be employed within the operating environment 100. Each can include a single device or multiple devices cooperating in a distributed environment. For example, search server 106 can be provided by multiple devices deployed in a distributed environment that collectively provide the functionality described herein. Additionally, other components not shown may also be included in the distributed environment.

[0032] User devices 102a and 102b to 102n can be client devices on the client side of operating environment 100, while search server 106 can be the server side of operating environment 100. Server 106 may include server-side software designed to work collaboratively with the client software on user devices 102a and 102b to 102n to implement any combination of features and functions discussed in this disclosure. This division of operating environment 100 is provided to illustrate an example of a suitable environment, and for each implementation, it is not required that search server 106 and any combination of user devices 102a and 102b to 102n remain separate entities.

[0033] User equipment 102a and 102b to 102n may include any type of computing device that a user can use. For example, in one aspect, user equipment 102a to 102n may be related to... Figure 1 The type of computing device described. 6 Here. As an example and not a limitation, a user device can be a personal computer (PC), laptop computer, mobile or mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, global positioning system (GPS) or device, video player, handheld communication device, gaming device or system, entertainment system, in-vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronic device, workstation, or any combination of these defined devices, or any other suitable device that can present notifications. A user can associate with one or more user devices. A user can communicate with search server 106, websites 104a and 104b to 104n through the user device.

[0034] One objective of the technology described herein is to ensure that user devices are warned of malicious websites. Websites 104a and 104b to 104n may include both benign and malicious websites, such as phishing websites. User devices 102a and 102b to 102n can access the websites via a browser. Users can find the websites through search results provided by search service 106 in response to a query. Users of user devices 102a and 102b to 102n can navigate to websites 104a and 104b to 104n by clicking hyperlinks embedded in web pages, emails, social media posts, through applications, etc. Users can also simply enter a URL into the browser's address bar. The technical aspects described herein will evaluate the URL and provide a warning when the URL is classified as malicious. The analysis and warnings can be performed entirely on user devices 102a and 102b to 102n, by search server 106 alone, or by a combination thereof. Search server 106 can also be used to collect training data for a classifier.

[0035] In addition to warning users, security components on servers or clients can take direct action or act as an alternative. For example, browsers may block access to websites associated with URLs classified as phishing. Client- or server-based email systems may disable hyperlinks to URLs classified as phishing. Search engines may block search results displaying links to URLs classified as phishing. Social media platforms or applications may remove or otherwise block access to social media posts that include links to URLs classified as phishing. Social media users who post offensive social media posts may be warned. Social media users may be added to a list of social media users requiring further investigation to determine whether the account is a phishing account or a fraudulent account. Social media accounts may be automatically suspended when the number of social media posts linking to phishing URLs exceeds a threshold.

[0036] Operating environment 100 can be used to implement Figure 2 The system 200 described herein includes one or more components, including components for classifying URLs and alerting users.

[0037] refer to Figure 1 And now refer to Figure 2 This document provides a block diagram illustrating an exemplary computing system architecture suitable for implementing various aspects of the technology, and is generally designated as System 200. System 200 represents only one example of a suitable computing system architecture. Other arrangements and elements may be used in addition to those shown or instead of those shown, and some elements may be omitted entirely for clarity. Furthermore, as with operating environment 100, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or combined with other components, and in any suitable combination and location.

[0038] At a higher level, system 200 includes a security service 210 that receives a URL 201 and returns a classification 202 indicating whether the URL is malicious or benign. Here, "malicious" is described as anything non-benign, such as a phishing website. Security service 210 may be implemented on one or more servers, such as search server 106. Various aspects of the search service may be distributed across user devices, such as device 102a. On one hand, classifier 226 is trained on the server by classifier trainer 250 and then transmitted to the user device.

[0039] Example system 200 includes security service 210 (including its components 212 and 220) and classifier trainer 250. Security service 210 (and its components) and classifier trainer 250 (and its components) may be implemented as a set of compiled computer instructions or functions, program modules, computer software services, or in one or more computer systems (e.g., in combination). Figure 6 The process arrangement executed on the described computing device 600.

[0040] In one aspect, the functions performed by the components of system 200 are associated with one or more personal assistant applications, browsers, services, or routines. Specifically, such applications, services, or routines may run on one or more user devices (e.g., user device 102a), servers (e.g., search server 106), may be distributed across one or more user devices and servers, or may be implemented in the cloud. Furthermore, in some aspects, these components of system 200 may be distributed across a network, including one or more servers (e.g., server 106) and client devices (e.g., user device 102a), in the cloud, or may reside on a user device, such as user device 102a. Moreover, these components, the functions performed by these components, or the services performed by these components may be implemented at an appropriate abstraction layer of the computing system, such as the operating system layer, application layer, hardware layer, etc. Alternatively or additionally, the functions of these components and / or aspects of the techniques described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, exemplary hardware logic component types that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc. Furthermore, although the functionality of these components is described herein with reference to the specific components shown in example system 200, it is anticipated that in some respects, the functionality of these components may be shared or distributed across other components.

[0041] Before classifier 226 can classify URLs, it needs to be trained by classifier trainer 250. Initially, classifier 226 can have Figure 4 and Figure 5 The architecture described in the document is applicable, but it cannot accurately classify URLs. The classifier trainer 250 takes the original training data instance 252 (which includes URLs and corresponding labels) from the training data as input to build a trained classifier 226. Once trained, the classifier 226 can classify unlabeled URLs.

[0042] The layers in classifier 226 consist of neurons. In this specific case, input layer neurons receive character or word embeddings from a URL and pass the data derived from the embeddings to neurons in multiple hidden layers. Neurons in the hidden layers pass their computations to additional layers until the results reach the output layer. The output layer then generates probability values ​​for each individual segment classification. Different types of layers and networks connect neurons in different ways.

[0043] A neuron has an intrinsic activation function that computes its output given an input (a vector of numbers) multiplied by another vector of numbers (called weights). Weights are adjustable parameters that cause the neural network to produce the correct output given a previously known match between the input and output. For example, if the training URL is labeled benign, the correct output is to classify the URL as benign. Weights are adjusted during training. Once trained, the weights associated with a given neuron can remain fixed. Other data passed between neurons can change in response to a given input (such as a URL). Retraining the network with additional training data can update one or more weights in one or more neurons.

[0044] In each type of deep model, training is used to fit the model's output to the training data. Specifically, the weights associated with each neuron in the model can be updated during training. Initially, the model may contain random weight values ​​that are adjusted during training. In this case, training is done in multiple iterations, each iteration containing multiple steps: forward propagation, loss function calculation, and backpropagation, where the weights are updated based on the errors made by the neural network during training. This process is repeated for multiple batches of training images. The goal is to update the weights of each neuron (or other model component) so that the model generates an output that maps to the correct labels of as many URLs as possible. The training data consists of labeled URLs. Each labeled URL is fed into the model and used to train it. Training can be stopped once a sufficient number of training URLs have been fed into the model used by the classifier 226 and the model stops improving or improves slowly during training. The model can then be used to classify unlabeled images (e.g., not training documents).

[0045] The original training data 252 may include anonymous browsing telemetry data consisting primarily of URLs with phishing ratings. On one hand, the phishing rating is an integer between 0 and 5. On the other hand, the phishing rating is 0 for safe and 1 for suspicious. The ratings are used as labels for training and evaluating classifier 226. A portion of the corpus is reserved for training data, while another portion is reserved for testing the accuracy of classifier 226. When class imbalance exists in the original data, benign classes are downsampled to improve the balance of the training set. On one hand, the threat category accounts for 5% of the total data used in training. A portion of the corpus with the same class balance is constructed by selecting a portion of the corpus through stratified sampling.

[0046] During training, data from the training corpus is presented to classifier 226 as a sequence of URLs. Each URL is evaluated and classified by classifier 226 without knowing the real data. If classifier 226 misclassifies a URL, the connections, weights, parameters, and biases of one or more ANNs within classifier 226 are adjusted to improve classifier 226.

[0047] In one aspect, classifier 226 is trained using the PyTorch deep learning framework. An exemplary configuration includes a binary cross-entropy loss function and an SGD optimizer with momentum equal to 0.9 and a learning rate initialized to 0.01. In another aspect, training is performed for 30 epochs, with the minimum batch size set to 128, and a validation score is obtained at the end of each training epoch. The model with the best validation loss is returned after the final model for each experiment. The learning rate can be halved every 5 training epochs.

[0048] The character embedding component 254 constructs character vectors for training inputs, but can also construct a vector space for character understanding. The word embedding component 256 constructs word vectors for training inputs, but can also construct a vector space for word understanding.

[0049] A URL is an example of unstructured text containing a set of symbols. Symbols can be characters or words, analogous to sentences in unstructured data. As mentioned, the techniques described herein perform both word-based and character-based analysis on URLs.

[0050] As the initial step performed by the character embedding component 254, characters within the URL are identified. On one hand, each character in the alphabet is assigned an integer index. Additional indices are reserved for unknown characters and spaces. Each character in the alphabet is assigned an M-dimensional embedding vector, such as a 32-dimensional numeric array. On one hand, the dimensions are scalars, such as integers, floating-point numbers, or scalar values ​​on fields. On one hand, a lookup table is formed by storing the embedding vectors associated with each symbol defined for the alphabet. On one hand, the elements of the embedding vectors are randomly assigned.

[0051] In one approach, each character in the URL's length L matches a character from the alphabet. In another, long URLs are trimmed to length L, for example, by taking the first L characters. In yet another, short URLs are padded with null characters to length L. In yet another, L is controlled as a parameter. In one approach, L = 1000, 500, or 200.

[0052] In one aspect, an embedding vector is assigned to each character in a URL of length L, and a consecutive column is assigned to each embedding vector in the order of the characters in the URL to generate a character embedding matrix.

[0053] Symbol embedding matrix in Figure 3 The matrix 300 is shown. The symbol embedding matrix is, for example, a character embedding matrix or a word embedding matrix. The character embeddings are generated by the character embedding component 254, while the word embedding component 256 constructs the word vectors. Figure 1 Only character vectors are shown in Figure 3, but word embedding vectors work in a similar way. An example of a character embedding matrix is ​​shown directly in... Figure 3 The matrix 300 is shown for M=5 and L=7 and for the URL “msn.com”. Each column of the character embedding matrix 300 shows an N-dimensional vector. The value of each element of the N-dimensional vector is shown as a shaded level, reflecting an integer value approximately from 0 to 6. The column under the first “m” represents the five-dimensional embedding vector for the character “m”, the column under “s” represents the five-dimensional embedding vector for the character “s”, and so on. The letters n, c, o, m and the period symbol each have similar vectors.

[0054] In one aspect, analogy symbols are words formed from adjacent characters in a URL. The feature processor identifies analogy symbols, such as words in a URL. In another aspect, the word embedding component 256 identifies words in a URL. URLs are tagged as analogy words. In one aspect, special characters in each analogy sentence, including “? / -_\=%@+;”, are used to tag each URL to obtain analogy words. The analogy words used here are strings of adjacent characters that appear repeatedly within different URLs in the training data. Analogy words do not necessarily have to be words in any language. In one aspect, these special characters are also included in the model vocabulary. Each tagged word is compared with a predefined word model library encountered in the URL design corpus. An analogy word vocabulary is constructed by analyzing the frequency of a word's occurrence in the URL corpus. In one aspect, newly encountered words are added to the model vocabulary. In one aspect, the model library is derived from the training corpus of URLs of all possible classes (benign or malicious). In one aspect, words that appear more than a user-defined number of times in the training corpus in the training set are added to the model library.

[0055] An N-dimensional embedding vector is assigned to each analogy word in the URL to form a symbol matrix such as matrix 300, where each column of matrix 300 is associated with a word, rather than with... Figure 3 The single character shown is associated with an embedding vector. In one aspect, the embedding vector is assigned via a lookup operation that takes a tokenized word and returns a defined embedding vector from a table. In one aspect, word embedding component 256 assigns the embedding vector. In one aspect, N = 32. In one aspect, a random value is assigned to each element of each embedding vector, which is assigned to a unique word in the model library. In one aspect, when a certain number of n-gram words match, a unique embedding vector is assigned to multiple words in the model library. In one aspect, the contextual word embeddings are learned from a URL corpus.

[0056] Once the vector space is established, URLs can be used to translate unlabeled URLs into word embeddings by the word embedding component 224. Similarly, character embeddings can be constructed by the character embedding component 222 in the same manner. The result is that the embeddings used to train the classifier 226 take the same form as the embeddings input to the classifier when analyzing URLs.

[0057] Go to Figure 4 Processing block 410 shows an exemplary component in a URL processing block used to form the output feature vector. (See reference...) Figure 5As explained, the output feature vector can be used to generate a classification. The output feature vector, such as the output of link block 415, is formed by connections occurring within link block 415, which takes input from component processing branches (CPLs). In the example shown in Figure 4, five different CPLs are used, but only three are shown. The presence of the other two is indicated by dots. From left to right, the five CPLs include CPL-A 401, CPL-B 402, CPL-C, CPL-D, and CPL-E 403. Three CPLs are explicitly shown. Figure 4 In CPL-A 401, CPL-B 402, and CPL-E 403, the vertical path serves as the connection between input block 411 and link block 415. CPL-C is a block similar to CPL-B 402 but with parameter F=4, where F is the filter size. CPL-D is a block similar to CPL-B 402 but with parameter F=5. Each CPL, such as CPL-B 402, employs a symbolic embedding matrix 300 and performs a convolution operation at 412, one or more normalization operations at 413, and a nonlinear operation 414 to form the component feature matrix. Different parameters are defined for each CPL, determining the processing performed in a branch. For example, the parameters for CPL-B 401, denoted as PS-B, include the filter size F, expansion D, input channel size i, output channel size o, maximum pool window width W, group size j, and a set of j filter kernel definitions {KD1, KD2, ... KDj}. Similarly, a set of parameters is defined for each of the other branches. The parameter set for CPL-A401 is PS-A. The parameter set for CPL-C is PS-C. The parameter set for CPL-D is PS-D. The parameter set for CPL-E 403 is PS-E. A kernel definition is a set of filter terms that define a filter. In one respect, the kernel defined for filtering is one-dimensional. In another respect, the filter values ​​defining the kernel terms are learned during training. In one respect, the output size o is specified as an integer multiple of the input size i, thus defining the number of kernels. That is, o = j i.

[0058] The operations performed within the CPL can generally be understood through the processing performed in CPL-B 402. On one hand, a one-dimensional convolution operation is performed at 416 through independent operations performed on each row of the symbol embedding matrix 300. For example, row 311 of the convolution output 340 is formed by filtering row 301 using kernel KD1. Row 321 of the convolution output 340 is formed by filtering row 301 using kernel KD2. Row 312 of the convolution output 340 is formed by filtering row 302 using kernel KD1. Row 322 of the convolution output 340 is formed by filtering row 302 using kernel KD2. Similarly, rows 313 and 323 are formed by convolution processing on row 303 of the symbol matrix 300 using kernels KD1 and KD2. Rows 314 and 324 are formed by convolution processing on row 304 of the symbol matrix 300 using kernels KD1 and KD2. Rows 315 and 325 are formed by convolving kernels KD1 and KD2 on row 305 of the symbol matrix 300. The convolution output 340 feature matrix is ​​formed in relation to the selected filter length F.

[0059] exist Figure 3 In the example shown, the filter length F=3, the dilation D=1, i=5, o=10. The terms K1 and K2 were learned during training. When generating the elements of the 340 feature matrix of the convolutional output, three symbols are passed through the filter kernel at a time to produce a single element. For example, the output at row 311 and column 331 is taken from the 301-row embedding elements under “m”, “s”, and “n” respectively, as a dot product with the elements of kernel KD1. The output at row 311 and column 332 forms a dot product with the elements of kernel KD1 by taking the 301-row embedding elements under “s”, “n”, and “.” respectively. Figure 3 The code provides a handling without zero padding, so a URL of length 7 results in a convolution output of line 311 with 5 elements, resulting in a line of 340. Alternatively, zero padding can be used to perform the convolution.

[0060] On one hand, column 331 of the convolutional output matrix 340 is formed by processing a portion of the embedding vector matrix 300 through a dedicated mapping filter. For example, 15 elements in rows 301-305 of the embedding vector matrix 300 (which are also located under the characters "msn") can be processed by the dedicated mapping filter to form the 10 outputs found in column 331. Examples of dedicated mapping filters include multi-output adaptive networks, recurrent neural networks, and long short-term memory networks (LSTM).

[0061] Normalization block 413 performs one or more normalization operations, such as batch normalization (BN) and max pooling, on the convolutional output matrix 340. CNN training is typically performed in mini-batches. In batch normalization, each element in the input vector is scaled so that the mean and unit variance of all corresponding elements in the training batch are zero. In the pooling operation, a window of size W is pooled within a portion of the component feature matrix (e.g., row 311), and the output of the window is determined by an operation such as averaging or a ranking order such as max, median, or rank-m. In the rank-m operation, the m-th ranked value within the window is output.

[0062] The nonlinear operation 414 receives the output of the normalization block 413 and forms the component eigenvalue matrix (CFM) for CFL-B, which is represented as CFM-B. In one aspect, the rectified linear unit (ReLU) is used for nonlinearity. In another aspect, a bias term is applied before applying the ReLU function.

[0063] The processing of other branches (such as CPL-A 401, CPL-C, CPL-D, and CPL-E 403) is similar to that performed in CPL-B 401. Therefore, branch CPL-A 401 produces the component feature matrix CFM-A. Branch CPL-C produces CFM-C. Branch CPL-D produces CFM-D. Branch CPL-E 403 produces CFM-D. Link block 415 produces an output sequence of o-dimensional vectors from the links of CFM-A, CFM-B, CFM-C, CFM-D, and CFM-E.

[0064] Block 410 is a Basic Text Processing Block (TBP), which is parameterized by a set of parameters defining the processing operations, such as parameter sets PS-A, PS-B, PS-C, PS-D, and PS-E.

[0065] Now go to Figure 5 Intuitively, an ID convolution with a filter size of N over a sequence of character embeddings is analogous to extracting N-grams from the input text. The advantage of using convolutional layers in parallel, each with a different filter size, is that this arrangement captures text patterns better compared to multiple consecutive layers with fixed filter sizes.

[0066] The complete model 500 includes two parallel paths: a character-level path 501 for extracting character-level information, and a word-level path 502 for processing word-level information. The character-level path 501 first transforms the original URL into a dense matrix of character embedding vectors, as shown in the reference... Figure 3As described. Next, multiple convolutional blocks (e.g., 512, 516, 519) are applied, followed by adaptive max-pooling layers (e.g., 513, 517, 520), which constrain the size of the output of the last block (e.g., 514, 518, 521) to a user-defined value.

[0067] The word-level path 502 first tags the incoming URL using the same regular expression (regex) keys used in the model construction. Then, the list of words in the URL is matched against a maximum word length, pruning longer word lists and padding shorter ones with zeros. This list is then passed through a word embedding layer 507 to produce a sequence of word vectors, followed by multiple convolutional blocks (e.g., 532, 536, and 539). The output of the word path is also constrained using an adaptive max-pooling operator.

[0068] The outputs of character-level path 501 and word-level path 502 are concatenated at 592 and fed into multiple fully connected layers (e.g., 594, 596, and 598) that ultimately produce class probabilities. Layers with more nodes have higher loss probabilities. Note that users can choose from a variety of architectures: deeper networks with more consecutive convolutional blocks, wider networks with more parallel layers, using words with pre-trained weights and random weights, or even no words at all.

[0069] Figure 5 illustrates the formation of a decision score at the output of a sigmoid (S-shaped function) 599 from the output of one or more text processing blocks. This decision score is used to assign a category probability, such as malicious or benign probability. Such a decision score provides the basis for determining the selected category, which is described by the resulting category probability. As an alternative to the sigmoid function, the softmax function can be used. The softmax function can be particularly useful when the output specifies multiple categories (e.g., benign, phishing, spam, etc.). The output of layer 598 is then the final set of N identified features v1, v2, ... vN and their corresponding weights w1, w2, ... wN, which can be processed using a defined offset constant b0, and will be processed by the nonlinear h(.) in sigmoid 599 to produce a decision score d. A representative ANN can produce an output d that is equal to h(v1, v2, ... vN). wl+v2 w2+...+ vN wN+ b0). An example is shown in Figure 5. In the example in Figure 5, the feature length leaving layer 598 is N=128, and it takes 512 input elements from layer 596, thus achieving 25% dropout. Layer 596 produces 512 outputs from a fully connected layer that takes 1028 inputs from layer 594. Layer 596 also has a non-linear layer at the output of layer 598, achieving 50% dropout. Layer 594 produces 1028 outputs to layer 596 by processing the output from connection block 592, while connection block 592 provides 1536 inputs. For example, layer 594 is a fully connected convolutional layer with non-linear outputs, such as ReLU, and achieves a 75% dropout rate. On one hand, the link block 592 generates 1536 outputs by taking 512 outputs from the adaptive max pool block 591 and linking them with 1024 outputs from the adaptive max pool block 593.

[0070] On one hand, max-pooling block 591 receives 512 inputs from two structurally identical TBPs, namely TPB 550 and TPB 510, which are concatenated. On the other hand, the convolution of TPB 550 allows for column growth, such that the convolutional block within 550 has 64 inputs and 128 outputs. In the example, TPB 510 is configured with four branches as shown in the figure. Convolutional block 512 has F=2. Convolutional block 516 has F=3. Convolutional block 519 has F=4. Convolutional block 522 has F=5. The variable F corresponds to the input filter size. Therefore, the block with a filter size of 4 will take four adjacent characters from the URL as input.

[0071] Each convolutional block is connected to input 511 and configured for a 32-dimensional input vector and a 64-dimensional output vector. Each convolutional block is followed by corresponding normalization blocks 513, 517, 520, and 523, and corresponding nonlinear blocks 514, 518, 521, and 524, which are then concatenated to form a vector of length 256 at the output of concatenation block 515. Embedding block 505 creates a character embedding matrix 300 based on the original input string received from URL recognition block 503 and passes the input to TPB 510.

[0072] On one hand, max-pooling block 593 receives 1024 inputs from two structurally identical TPBs, namely TPB 570 and TPB 530, which are concatenated. On another hand, the convolution of TPB 570 allows for column growth, such that the convolutional block within 570 has 64 inputs and 128 outputs. On yet another hand, TPB 530 includes three parallel processing paths for 32-dimensional word embedding vectors. On yet another hand, TPB 510 includes convolutional block 532 configured with F=1, i=32, o=64, j=2, D=1. F describes the filter size. A filter size of 1 indicates that the input will be a single word embedding vector. A word embedding vector represents a single word and can contain different numbers of characters, or even just one character. On yet another hand, convolutional filter block 536 is configured with F=3, o=64, j=2, D=1. A filter size of 3 indicates that the input will be three word embedding vectors. On one hand, convolutional filter block 539 is configured with F=5, o=64, j=2, D=1. A filter size of 5 indicates that the input will be five word embedding vectors. TPB 530 takes the word embedding matrix derived from the original URL and performs convolution. Word embedding block 507 creates a word embedding matrix, such as matrix 300, based on the original input string received from analogy sentence recognition block 503, and passes the input to TPB 530. Input processing block 531 passes the word embedding matrix to each of convolutional blocks 532, 536, and 539. Convolutional block 532 performs a specified convolution on the input and passes the output to normalization block 533, which passes its output to connection block 535 via nonlinear block 534. Convolutional block 536 performs a specified convolution on the input and passes the output to normalization block 537, which passes its output to connection block 535 via nonlinear block 538. Convolution block 539 performs a specified convolution on the input and passes the output to normalization block 540, which then passes its output to connection block 535 via nonlinear block 541.

[0073] On one hand, adaptive max-pooling blocks such as 591 allow users to specify a fixed output size. On another hand, adaptive max-pooling block 593 operates with an output size of 32. On another hand, adaptive max-pooling block 593 operates with an output size of 16. On yet another hand, adaptive max-pooling block 591 operates with an output size of 32. In one aspect, the decision score generated by sigmoid 599 is based on the output feature matrix from TPB 510. In another aspect, the decision score generated by sigmoid 599 is based on the output feature matrix from TPB 530. In yet another aspect, the score generated by decision sigmoid 599 is based on both the output feature matrices from TPB 510 and TPB 530. In yet another aspect, when TPB 550 is cascaded, the decision score generated by sigmoid 599 is based on the output feature matrix from TPB 550 to obtain the input feature matrix from TPB 510. In yet another aspect, when TPB 570 is cascaded, the decision score generated by sigmoid 599 is based on the output feature matrix from TPB 570 to obtain the input feature matrix from TPB 530. On the one hand, the decision score generated by sigmoid 599 is based on deep learning from two or more convolutional layers derived from one or more TPBs such as TPB 510, TPB 530, TPB 550 and TPB 570.

[0074] In one aspect, the input is a URL being evaluated for different threat levels. In another aspect, a decision score is used to classify the threat level of the URL. The decision score is compared to a threshold and assigned a level, or integer level. The output of sigmoid 599 is the decision score d. The decision score d is compared to one or more thresholds to determine a Threat Level (TL) category, such as an integer between 0 and 5, which indicates the danger level of the URL, where 0 represents the lowest threat and 5 represents the highest threat. If the threshold is set to TL>3, then if the sigmoid decision score indicates TL=4 or TL=5, the URL is considered threatening. When sigmoid 599 determines TL>3, the computer monitor displays a warning in an alert box, indicating on the graphical user interface that the URL requested by the computer user may be associated with a phishing attack. In one aspect, TL=5 is indicated to the user as a high threat level, TL=4 as a medium-high threat level, TL=3 as a medium threat level, TL=2 as a low threat level, and TL=1 as a very low threat level. TL=0 indicates no threat. Ten threat levels are used.

[0075] refer to Figure 6The computing device 600 includes a bus 610 that directly or indirectly couples to the following devices: a memory 612, one or more processors 614, one or more presentation components 616, one or more input / output (I / O) ports 618, one or more additional I / O components 620, and a schematic power supply 622. Bus 610 can represent one or more buses (e.g., an address bus, a data bus, or a combination thereof). Although various blocks are shown in the figure. For clarity, Figure 6 The lines illustrate that these blocks actually represent logic, not necessarily physical components. For example, a presentation component such as a display device can be considered an I / O component. Furthermore, the processor has memory. The inventors recognize this as being of the nature in the art and reiterate the diagram in Figure 1. Figure 6 This description only illustrates exemplary computing devices that can be used in conjunction with one or more aspects of this technology. There is no distinction between categories such as "workstation," "server," "laptop," and "handheld device," as all of these are envisioned in... Figure 1 Within the scope. 6. And refer to "Computing Devices".

[0076] Computing device 600 typically includes a variety of computer-readable media. Computer-readable media can be any available medium that can be accessed by computing device 600, and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0077] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disk (DVD) or other optical disc storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by the computing device 600. Computer storage media itself does not include signals.

[0078] Communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals (such as carrier waves or other transmission mechanisms), and include any information transmission medium. The term "modulated data signal" refers to a signal having one or more of its characteristics, which are set or altered in such a way that information is encoded in the signal. As an example and not a limitation, communication media includes wired media, such as wired networks or direct wired connections, and wireless media, such as acoustic, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0079] Memory 612 includes computer storage media in the form of volatile and / or non-volatile memory. The memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Computing device 600 includes one or more processors 614 that read data from various entities such as memory 612 or I / O components 620. 616 presents data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc.

[0080] I / O port 618 allows computing device 600 to be logically coupled to other devices, including I / O components 620, some of which may be built-in. Illustrative components include microphones, joysticks, game controllers, satellite antennas, scanners, printers, wireless devices, etc.

[0081] I / O component 620 provides a Natural User Interface (NUI) that processes air gestures, voice, or other physiological input generated by the user. In some cases, the input can be transmitted to appropriate network elements for further processing. The NUI can implement any combination 600 of voice recognition, touch and stylus recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with a display on the computing device. The computing device 600 may be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, and combinations thereof, for gesture detection and recognition. Furthermore, the computing device 600 may be equipped with an accelerometer or gyroscope capable of detecting motion. The output of the accelerometer or gyroscope can be provided to the display of the computing device 600 to present immersive augmented reality or virtual reality.

[0082] Some aspects of computing device 600 may include one or more radio devices 624 (or similar wireless communication components). Radio 624 transmits and receives radio or wireless communications. Computing device 600 may be a wireless terminal adapted to receive communications and media over various wireless networks. Computing device 600 may communicate with other devices via wireless protocols such as Code Division Multiple Access (“CDMA”), Global System for Mobile Communications (“GSM”), or Time Division Multiple Access (“TDMA”). Radio communication may be a short-range connection, a long-range connection, or a combination of both short-range and long-range wireless telecommunication connections. When we refer to “short” and “long” types of connections, we are not referring to a spatial relationship between two devices. Rather, we generally refer to short-range and long-range as different categories or types of connections (i.e., primary and secondary connections). As an example and not a limitation, a short-range connection may include a Wi-Fi® connection to a device that provides access to a wireless communication network (e.g., a mobile hotspot), such as a WLAN connection using the 802.11 protocol; a Bluetooth connection to another computing device is a second example of a short-range connection or near-field communication connection. As an example and not a limitation, remote connections may include connections using one or more of the following protocols: CDMA, GPRS, GSM, TDMA, and 802.16.

[0083] Turning now to the diagram, method 700 is a class indicator for displaying unstructured text in a URL. Method 700 can be performed by the previously described system 200.

[0084] In step 702, the method includes receiving a URL. The URL may be received as part of on-demand URL analysis. On-demand analysis is performed in real time. On-demand analysis may first compare the URL with a list of known malicious or benign URLs. If the URL is not found in the list, the analysis described subsequently may continue. On-demand analysis may be initiated by a user entering a URL in a browser, receiving an email, text message, or social media content with embedded URLs (e.g., hyperlinks), navigating to a website with a URL link, or responding to other actions. The techniques described herein are not limited to use with on-demand services. For example, the technique may evaluate new URLs registered in a domain name service.

[0085] In step 704, the method includes forming a first contextual word embedding from the URL, which represents a first word identified in the URL and the context of that first word in the URL. (See previous examples for instance.) Figures 2-5 It describes the embedding of contextual words.

[0086] In step 706, the method includes forming a second contextual word embedding from the URL, which represents a second word identified in the URL and the context of that second word in the URL. (See previous examples for instance.) Figures 2-5 It describes the embedding of contextual words.

[0087] In step 708, the method includes processing a first contextual word embedding in a first parallel path within a word-level path of the URL classifier, the first parallel path having a first filter length. (Previously, for example, references...) Figures 2-5 It describes how to handle embeddings.

[0088] In step 710, the method includes processing a second contextual word embedding in a second parallel path within the word-level path of the URL classifier, the second parallel path having a second filter length greater than the first filter length. (Previously, for example, references...) Figures 2-5 It describes how to handle embeddings.

[0089] In step 712, the method includes generating word-level output from word-level paths.

[0090] In step 714, the method includes generating a decision input vector using the word-level output. In one aspect, the word-level output can be combined with the character-level output from parallel character-level paths via concatenation. (See previously, for example, [reference needed]). Figures 2-5 Generate decision input vectors.

[0091] In step 716, the method includes generating decision statistics using the decision input vector. This has been previously illustrated, for example, with reference to... Figures 2-5 Generate decision statistics.

[0092] In step 718, the method includes determining whether decision statistics indicate that the URL is malicious. Decision statistics can be numerical. A number within a certain range may indicate that the URL is malicious. If it falls outside this range, the URL may be benign.

[0093] In step 720, the method includes blocking access to the URL in response to decision statistics indicating that the URL is malicious. Web browsers may disable links to web pages associated with unstructured text. Communication programs may isolate emails, texts, social media posts, videos, blogs, or other content associated with unstructured text. For example, emails with links associated with unstructured text may be moved to the spam folder. On one hand, the source of the unstructured text is prevented from sending any additional content to the entity to which the unstructured text is being classified. For example, email addresses, IP addresses, social media accounts, and internet domains may be added to the block list. Entities may be required to confirm adding the source to the block list. On the other hand, all code on the content source (e.g., a web page) associated with the unstructured text is automatically blocked.

[0094] Warnings can also be provided. Warnings can take many forms, such as turning the link to the URL red, turning the address bar red, or displaying a pop-up alert. In addition to warnings, links to the URL can be disabled, and web pages found at the URL can be blocked from loading. This URL can be submitted to services that track malicious URLs.

[0095] Turning now to the diagram, method 800 is a class indicator for displaying unstructured text in a URL. Method 800 can be executed by the previously described system 200.

[0096] In step 802, the method includes receiving a URL. The URL may be received as part of on-demand URL analysis. On-demand analysis is performed in real time. On-demand analysis may first compare the URL with a list of known malicious or benign URLs. If the URL is not found in the list, the analysis described subsequently may continue. On-demand analysis may be triggered by a user entering a URL in a browser, receiving an email, text message, or social media content with embedded URLs (e.g., hyperlinks), navigating to a website with a URL link, or responding to other actions. The techniques described herein are not limited to use with on-demand services. For example, the technique may evaluate new URLs registered in a domain name service.

[0097] In step 804, the method includes forming a contextual word embedding from the URL by identifying strings in the URL that form words in a URL-specific vocabulary. (See previous examples for instance.) Figures 2-5 It describes the embedding of contextual words.

[0098] In step 806, the method includes processing contextual word embeddings in the word-level paths of the URL classifier. This has been previously illustrated, for example, by referring to... Figures 2-5 The input embedding is described.

[0099] In step 808, the method includes generating word-level output from word-level paths. (See previous examples for instance.) Figures 2-5 This describes how to generate output from word-level paths.

[0100] In step 810, the method includes generating a decision input vector using word-level outputs as input. (See previous examples for instance.) Figures 2-5 It describes the generation of decision input vectors.

[0101] In step 812, the method includes generating decision statistics using the decision input vector. (See previously, for example, reference...) Figures 2-5 The statistics for generating decisions are described.

[0102] In step 814, the method includes determining whether a decision statistic indicates that the URL is malicious. The decision statistic can be a number. A number within a certain range may indicate that the URL is malicious. If it is not within the malicious range, the URL may be benign.

[0103] In step 816, the method includes automatically blocking access to content associated with unstructured text in response to classification. Web browsers may disable links to web pages associated with unstructured text. Communication programs may isolate emails, texts, social media posts, videos, blogs, or other content associated with unstructured text. For example, emails with links associated with unstructured text may be moved to the spam folder. On one hand, the source of the unstructured text is blocked from sending any additional content to the entity to which the unstructured text is being classified. For example, email addresses, IP addresses, social media accounts, and internet domains may be added to the block list. The entity may be required to confirm adding the source to the block list. On the other hand, all code on the content source (e.g., a web page) associated with the unstructured text is automatically blocked.

[0104] Blocking may be accompanied by warnings. Warnings can take many forms. They can be provided by turning the link to the URL red, turning the address bar red, or displaying a pop-up warning. In addition to warnings, you can also disable the link to the URL and prevent the webpage found at that URL from loading. This URL can be submitted to a service that tracks malicious URLs.

[0105] Turning now to the diagram, a method 900 for displaying category indications for unstructured text. Method 900 can be performed by the previously described system 200.

[0106] In step 902, the method includes receiving unstructured text. In one respect, unstructured text is not a combination of natural language expressions that follows grammatical conventions. Instead, unstructured text may be non-grammatical. Unstructured text may include strings that do not constitute words in a known language. Unstructured text may not contain any metadata. Unstructured text can be URLs, email addresses, usernames, social media account titles, trademarks, hashtags, etc.

[0107] In step 904, the method includes forming a first context character embedding of a first number of characters from the unstructured text. (See previously, for example, reference...) Figures 2-5 It describes context character embedding.

[0108] In step 906, the method includes forming a second context character embedding of a second number of characters from the unstructured text, wherein the first number is less than the second number. (See previously, for example, references...) Figures 2-5 It describes context character embedding.

[0109] In step 908, the method includes processing the first context character embedding in a first parallel path within the character-level path of the classifier. This has been previously referenced, for example... Figures 2-5 It describes how to handle embeddings.

[0110] In step 910, the method includes processing the second context character embedding in a second parallel path within the character-level path of the classifier. (Previously, for example, referenced...) Figures 2-5 It describes how to handle embeddings.

[0111] In step 912, the method includes generating a first character-level output from the character-level path. (See previously, for example, [reference needed]). Figures 2-5 It describes how to generate character-level output.

[0112] In step 914, the method includes generating a decision input vector using the first character-level output as input. (Previously, for example, referenced...) Figures 2-5 It describes the generation of decision input vectors.

[0113] In step 916, the method includes generating decision statistics using the decision input vector. (Previously, for example, referenced...) Figures 2-5 The statistics for generating decisions are described.

[0114] In step 918, the method includes determining a decision statistic that indicates the unstructured text is classified as a security risk. The decision statistic can be numerical. A number within a certain range may indicate that the URL is malicious. If it is not within the malicious range, the URL may be benign.

[0115] In step 920, the method includes automatically blocking access to content associated with unstructured text in response to classification. For example, a web browser may disable links to web pages associated with unstructured text. Communication programs may isolate emails, texts, social media posts, videos, blogs, or other content associated with unstructured text. For example, emails with links associated with unstructured text may be moved to the spam folder. On one hand, the source of the unstructured text is blocked from sending any additional content to the entity whose unstructured text is being classified. For example, email addresses, IP addresses, social media accounts, and internet domains may be added to the block list. The entity may be required to confirm adding the source to the block list. On another hand, all code on the content source (e.g., a web page) associated with the unstructured text is automatically blocked.

[0116] Warnings can also be provided. Warnings can take many forms. If the unstructured text is a URL, warnings can be provided by turning the link pointing to the URL red, turning the address bar red, or displaying a pop-up alert. In addition to warnings, links pointing to the URL can be disabled, and web pages found at the URL can be blocked from loading. This URL can be submitted to services that track malicious URLs.

[0117] Many different arrangements of the various components depicted, and those not shown, are possible without departing from the scope of the following claims. Aspects of the technology described are for illustrative purposes only and not for limitation. Alternative aspects will become apparent to the reader of this disclosure after reading it, and because of reading it. Alternative methods of implementing the foregoing can be accomplished without departing from the scope of the following claims. Certain features and sub-combinations are useful and can be employed and contemplated within the scope of the claims without reference to other features and sub-combinations.

Claims

1. One or more computer storage media having computer-executable instructions contained thereon, said instructions, when executed by one or more processors, causing said one or more processors to perform a method for detecting malicious URLs, said method comprising: Receive URL; A first contextual word embedding is formed from the URL, the first contextual word embedding representing a first word identified in the URL and the context of the first word in the URL; A second contextual word embedding is formed from the URL, the second contextual word embedding representing a second word identified in the URL and the context of the second word in the URL; The first context word embedding is processed in the first parallel component processing branch in the word-level path of the URL classifier. The first parallel component processing branch has a first filter length, a first convolutional layer, and a first max pooling layer following the first convolutional layer. The second context word embedding is processed in the second parallel component processing branch in the word-level path of the URL classifier. The second parallel component processing branch has a second filter length greater than the first filter length, a second convolutional layer, and a second max pooling layer following the second convolutional layer. Using the first result from the processing branch of the first parallel component and the second result from the processing branch of the second parallel component, generate word-level output from the word-level path; The word-level output is used to generate the decision input vector; Decision statistics are generated using the aforementioned decision input vector; The decision statistics indicate that the URL is malicious; as well as In response to the decision statistics indicating that the URL is malicious, a URL warning is output.

2. The medium according to claim 1, further comprising: Contextual character embeddings are formed from the URL; The context characters are embedded into the character-level path of the URL classifier, wherein the input layer of the character-level path includes multiple parallel convolutional layers; as well as Generate character-level output from the character-level path.

3. The medium of claim 2, wherein each of the plurality of parallel convolutional layers in the character-level path has an input filter with a different number of characters.

4. The medium according to claim 1, wherein the input layer of the word-level path comprises a plurality of parallel convolutional layers.

5. The medium of claim 4, wherein each of the plurality of parallel convolutional layers in the word-level path has an input filter for words comprising different numbers of characters.

6. The medium of claim 1, wherein the first word is identified by recognizing a string in the URL that matches a word in a URL-specific vocabulary.

7. The medium of claim 6, wherein the URL-specific vocabulary is generated by decomposing a corpus of URLs into n-grams, and then adding n-grams that appear more than a threshold number of times in the corpus to the URL-specific vocabulary.

8. A method for displaying category indicators of unstructured text in a URL, the method comprising: Receive URL; Contextual word embeddings are formed from the URL by identifying strings in the URL that match words in a URL-specific vocabulary; The contextual word embedding is processed in the word-level path of the URL classifier, wherein the word-level path includes a first text processing block and a second text processing block arranged in series, wherein the first text processing block includes a first set of parallel convolutional layers and a first connection layer, the first set of parallel convolutional layers including a convolutional layer and a subsequent max pooling layer, the first connection layer being used to generate a first vector from the output of the first set of parallel convolutional layers, and wherein the second text processing block includes a second set of parallel convolutional layers and a second connection layer, the second set of parallel convolutional layers including a convolutional layer and a subsequent max pooling layer, the second connection layer being used to generate a second vector from the output of the second set of parallel convolutional layers; The second matrix is ​​used to generate word-level output from the word-level path; The word-level output is used as input to generate a decision input vector; Decision statistics are generated using the aforementioned decision input vector; Determine the classification of the decision statistical indicators; as well as Output the indication of the classification.

9. The method of claim 8, wherein the URL-specific vocabulary is generated by decomposing a corpus of URLs into n-grams, and then adding n-grams that appear more than a threshold number of times in the corpus to the URL-specific vocabulary.

10. The method of claim 8, wherein the input layer of the word-level path comprises a plurality of parallel convolutional layers.

11. The method of claim 10, wherein each of the plurality of parallel convolutional layers in the word-level path has an input filter for words comprising different numbers of characters.

12. The method of claim 8, wherein, apart from training data, the URL is the only input processed to generate the decision statistics.

13. The method of claim 8, wherein the method further comprises: Context character embeddings are formed from the URL; The context characters are embedded into the character-level path of the URL classifier; Generate character-level output from the character-level path; as well as The character-level output is also used to form the decision input vector.

14. The method of claim 13, wherein the input layer of the character-level path comprises a plurality of parallel convolutional layers, and wherein each of the plurality of parallel convolutional layers in the character-level path has an input filter with a different number of characters.

15. The method of claim 14, wherein the contextual word embedding represents a first word identified in the URL and the context of the first word in the URL.

16. A method for displaying category indicators of unstructured text, the method comprising: Receive unstructured text; A first context character embedding of a first number of characters is formed from the unstructured text; A second context character embedding of a second number of characters is formed from the unstructured text, wherein the first number is less than the second number; The first context character embedding is processed in a first parallel component processing branch within the character-level path of the classifier. The first parallel component processing branch has a first filter length, a first convolutional layer, and a first max pooling layer following the first convolutional layer. The second context character embedding is processed in a second parallel component processing branch within the character-level path of the classifier. The second parallel component processing branch has a second filter length, a second convolutional layer, and a second max pooling layer following the second convolutional layer. A first character-level output is generated from the character-level path using a first result from the first parallel component processing branch and a second result from the second parallel component processing branch; Use the first character-level output as input to generate a decision input vector; Decision statistics are generated using the aforementioned decision input vector; The decision statistics indicate that the unstructured text conforms to a first category; and Output the indication for the first category.

17. The method of claim 16, wherein the input layer of the character-level path comprises a plurality of parallel convolutional layers.

18. The method of claim 16, further comprising: Contextual word embeddings are formed from the unstructured text; The context words are embedded into the word-level path of the classifier; Generate word-level output from the word-level path; and The word-level outputs are also used to form the decision input vector.

19. The method of claim 18, wherein the input layer of the word-level path comprises a plurality of parallel convolutional layers, and each of the plurality of parallel convolutional layers in the word-level path has an input filter for words comprising a different number of characters.

20. The method of claim 16, wherein, apart from training data, the unstructured text is the sole input processed to generate the decision statistics.

Images