A malicious URL identification method, device, equipment and storage medium

Abstract

The application discloses a malicious URL identification method and device, equipment and a storage medium, the URL data to be identified is preprocessed to obtain a URL string, the URL string is vectorized to obtain character vectors of each character in the URL string, a pre-trained malicious URL identification model is used to identify the character vectors to obtain an identification result of the URL data, wherein the malicious URL identification model comprises a convolutional neural network, a non-local operation, a long short-term memory neural network and a multilayer perception connected in sequence, the use of a one-dimensional convolutional neural network can enable the model to first focus on local information, and the non-local operation is used to capture global context information, so that the accuracy and efficiency of malicious URL identification can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method, apparatus, device, and storage medium for identifying malicious URLs. Background Technology

[0002] Existing malicious URL identification technologies mainly fall into two categories. One is to analyze the content of the URL's specified location and process information such as text, images, audio, and video to determine whether the URL is malicious. The other is to judge based on the URL itself, which can be further subdivided into three schemes: matching through blacklists; constructing features based on prior human knowledge and then using traditional machine learning methods for judgment; and directly using neural networks to simultaneously complete feature extraction and result judgment.

[0003] However, existing malicious URL detection methods have the following shortcomings: content-based malicious URL detection methods require processing numerous information elements, involve large computational loads, and have low detection efficiency. While blacklist-based detection methods are highly efficient, they can evade detection by changing keywords or website addresses, making them ineffective at identifying newly created malicious websites. Other existing neural network-based malicious URL detection methods often introduce complex preprocessing operations to transform URLs into structures that existing neural network models can directly process, such as converting one-dimensional URL strings into two-dimensional images before using convolutional neural networks for discrimination. These methods, due to the introduction of additional steps, often reduce the efficiency of malicious URL identification. Summary of the Invention

[0004] To address the problems existing in the prior art, embodiments of the present invention provide a method, apparatus, device, and storage medium for identifying malicious URLs, which can effectively improve the accuracy and efficiency of malicious URL identification.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying malicious URLs, including:

[0006] The URL data to be identified is preprocessed to obtain the URL string;

[0007] The URL string is vectorized to obtain the character vector of each character in the URL string;

[0008] The character vector is identified using a pre-trained malicious URL identification model to obtain the identification result of the URL data; wherein, the malicious URL identification model includes a convolutional neural network, a non-local operation, a long short-term memory neural network, and a multilayer perceptron connected in sequence.

[0009] As an improvement to the above solution, the step of vectorizing the URL string to obtain the character vector of each character in the URL string includes:

[0010] Adjust the length of the URL string to a preset length;

[0011] The adjusted URL string is segmented into multiple characters at the character level.

[0012] Each character is encoded to obtain the corresponding character's numerical code;

[0013] The numerical encoding is vectorized to obtain the character vector of the corresponding character.

[0014] As an improvement to the above solution, adjusting the length of the URL string to a preset length includes:

[0015] Determine whether the length of the URL string is greater than the preset length;

[0016] When the length of the URL string is greater than the preset length, the URL string is truncated starting from the first character and according to the preset length, so that the length of the truncated URL string is equal to the preset length.

[0017] When the length of the URL string is less than the preset length, the end of the URL string is padded with a set character until the length of the padded URL string is equal to the preset length.

[0018] As an improvement to the above scheme, the training of the malicious URL identification model includes the following steps:

[0019] The pre-collected URL sample data is preprocessed to obtain multiple URL sample strings;

[0020] Each of the URL sample strings is vectorized to obtain the character vector of each character in the corresponding URL sample string; wherein, all the character vectors corresponding to each URL sample string are used as a training sample.

[0021] The training samples are used to train a pre-built malicious URL identification model.

[0022] As an improvement to the above scheme, the step of training the pre-built malicious URL identification model using the training samples includes:

[0023] For each training sample, the character vector of the training sample is subjected to local feature extraction by the convolutional neural network to obtain the static vector features of each character in the corresponding URL sample string; wherein, the convolutional neural network is a one-dimensional convolutional neural network.

[0024] The static vector features are extracted globally through the nonlocal operation to obtain the first dynamic vector features of each character in the corresponding URL sample string;

[0025] The first dynamic vector features of each character in the corresponding URL sample string are aggregated by the long short-term memory neural network to obtain the second dynamic vector features of the corresponding URL sample string.

[0026] The second dynamic vector features are used to identify URLs through the multilayer perceptron to obtain the malicious URL prediction results of the corresponding training samples;

[0027] Based on the URL prediction results, the model parameters of the malicious URL identification model are updated using a backpropagation mechanism to obtain the updated malicious URL identification model. The next round of iteration is then performed based on the updated malicious URL identification model until the preset iteration conditions are met, resulting in a trained malicious URL identification model.

[0028] As an improvement to the above scheme, the preprocessing of the URL data to be identified to obtain a URL string includes:

[0029] The protocol header in the URL data is filtered to obtain the URL string of the corresponding URL data.

[0030] As an improvement to the above scheme, the preprocessing of the URL data to be identified to obtain a URL string further includes:

[0031] The domain name prefix in the URL string is filtered a second time to obtain the final URL string.

[0032] Secondly, embodiments of the present invention provide a malicious URL identification device, comprising:

[0033] The first preprocessing module is used to preprocess the URL data to be identified to obtain the URL string;

[0034] The first character vectorization module is used to vectorize the URL string to obtain the character vector of each character in the URL string;

[0035] The URL recognition module is used to identify the character vector using a pre-trained malicious URL recognition model to obtain the recognition result of the URL data; wherein, the malicious URL recognition model includes a convolutional neural network, a non-local operation, a long short-term memory neural network, and a multilayer perceptron connected in sequence.

[0036] Thirdly, embodiments of the present invention provide a malicious URL identification device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the malicious URL identification method as described in any one of the first aspects.

[0037] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the malicious URL identification method as described in any one of the first aspects.

[0038] Compared to existing technologies, the malicious URL identification method, apparatus, device, and storage medium of this invention preprocess the URL data to be identified to obtain a URL string; vectorize the URL string to obtain character vectors for each character in the URL string; and use a pre-trained malicious URL identification model to identify the character vectors to obtain the identification result of the URL data. The malicious URL identification model includes a convolutional neural network, nonlocal operations, a long short-term memory neural network, and a multilayer perceptron connected in sequence. By using a one-dimensional convolutional neural network, the model can focus on local information first and capture global contextual information using nonlocal operations, thereby effectively improving the accuracy and efficiency of malicious URL identification. Attached Figure Description

[0039] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a malicious URL identification method provided in an embodiment of the present invention;

[0041] Figure 2 This is a structural block diagram of the malicious URL identification model provided in the embodiments of the present invention;

[0042] Figure 3 This is a schematic diagram of the character vectorization processing flow provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the training process of the malicious URL identification model provided in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the non-local operation provided in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the overall URL discrimination process provided in the embodiments of the present invention;

[0046] Figure 7 This is a schematic diagram of a malicious URL identification device provided in an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of a malicious URL identification device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] Please see Figure 1 This is a flowchart of a malicious URL identification method provided in an embodiment of the present invention. The malicious URL identification method specifically includes:

[0051] S1: Preprocess the URL data to be identified to obtain the URL string;

[0052] S2: Vectorize the URL string to obtain the character vector of each character in the URL string;

[0053] S3: The character vector is identified using a pre-trained malicious URL identification model to obtain the identification result of the URL data; wherein, the malicious URL identification model includes a convolutional neural network, a non-local operation, a long short-term memory neural network, and a multilayer perceptron connected in sequence.

[0054] For example, see Figure 2The structural block diagram of the malicious URL recognition model is shown below. The malicious URL recognition model consists of a convolutional neural network (CNN), non-local operations, a long short-term memory (LSTM) neural network, and a multilayer perceptron. The CNN serves as the input layer, receiving the processed URL data to be identified, i.e., the character vectors of each character. It extracts the static vector features of each character vector in the URL string. The CNN allows the malicious URL recognition model to focus on the local features of the URL data, forming a concept similar to words, introducing a certain inductive bias into the model. This avoids the problem of requiring a large amount of data for pre-training when using a self-attention mechanism. The output of the CNN serves as the input to the non-local operations, extracting the first dynamic vector features of each character in the corresponding URL string. Non-local operations are an attention mechanism for extracting long-range dependencies, suitable for processing sequential data, and capable of capturing global contextual information, not just local adjacency relationships. The output of the non-local operations... As input to the Long Short-Term Memory (LSTM) neural network, the first dynamic vector features of each character in the corresponding URL string are aggregated to obtain the second dynamic vector features representing the entire URL string. Using the LSM neural network for aggregation does not lose important sequence information and can consider the importance of characters at different positions in the sequence, rather than simply averaging the vectors corresponding to characters in the sequence. The output of the LSM neural network serves as input to the Multilayer Perceptron (MLP), used to determine whether the URL data belongs to a malicious URL based on the final representation of the entire URL string obtained by the LSM neural network. For example, the second dynamic vector features representing the entire URL string from the output of the LSM neural network are used to perform final classification calculations to obtain the predicted probabilities of belonging to a malicious URL and a normal URL, and the determination of whether the URL data belongs to a malicious URL is based on the predicted probabilities, thus obtaining the identification result of the URL data. The working principles and specific network structures of the convolutional neural network, the nonlocal operation, the LSM neural network, and the MLP can be found in existing technologies and will not be elaborated here.

[0055] URL data is a sequence of characters, such as "http: / / www.yidongsheji.com:82". This invention preprocesses the URL data to be identified, extracting useful information such as domain name, path name, and file name to obtain a useful URL string, for example, "yidongsheji.com:82". Then, the obtained useful URL string is vectorized at the character level to obtain character vectors for each character. Finally, the character vectors of each character are input into the malicious URL identification model for identification, obtaining the result of whether the URL data to be identified is a malicious URL. Compared to existing technologies that use special symbols (such as "@", "-", " / ") as delimiters for word segmentation, which may lose information about these special symbols that are crucial for URL classification, and because URL naming lacks specific rules, the segmented results may not strictly align with the concepts of words in natural language, affecting the recognition results, this invention takes special symbols in the URL into account. It first extracts useful information from the URL, and then performs word segmentation and vectorization on the extracted useful information at the character level. Considering that most single letters lack semantic meaning, this invention combines convolutional neural networks and nonlocal operations to address this issue. Specifically, it employs a one-dimensional convolutional neural network and nonlocal operations. The one-dimensional convolutional neural network allows the malicious URL identification model to focus on the URL first. Local features of the data are used to form word-like concepts. Then, non-local operations are used to capture global contextual information. By stacking one-dimensional convolutional neural networks and non-local networks, the malicious URL identification model can achieve dynamic feature extraction based on contextual information when processing information at each location, thereby effectively improving the accuracy of malicious URL identification. Furthermore, this invention directly uses the URL string for identification, without needing to access the target URL or process the content information corresponding to the URL, thus accelerating the detection speed of malicious URLs. It also eliminates the need for complex data preprocessing steps that could introduce additional noise or cause information loss, simplifying the detection process and effectively improving the efficiency and accuracy of malicious URL identification. In addition, compared to existing technologies that use a large number of stacked convolutional layers for global information capture, resulting in complex models, the malicious URL identification model provided by this invention has a simpler structure.

[0056] Specifically, step S1: preprocess the URL data to be identified to obtain a URL string, including:

[0057] The protocol header in the URL data is filtered to obtain the URL string of the corresponding URL data.

[0058] Further, step S1: preprocessing the URL data to be identified to obtain a URL string, also includes:

[0059] The domain name prefix in the URL string is filtered a second time to obtain the final URL string.

[0060] For example, it can filter out protocol headers that do not affect the identification of malicious URLs to obtain the URL string of the corresponding URL data. It can also filter out the domain name prefix in the URL data, that is, filter out both the protocol header and the domain name prefix in the URL data to obtain the final URL string.

[0061] Since the numerous "http: / / ", "https: / / ", and "www" segments at the beginning of the URL data do not significantly affect the distinction between malicious and legitimate URLs, and the URL protocol headers contain segments such as "http: / / " and "https: / / ", while the domain name prefix contains segments such as "www", filtering out these segments makes the dataset more concise and reduces the time and resource consumption in subsequent steps.

[0062] Specifically, see Figure 3 Step S2: Vectorize the URL string to obtain the character vectors of each character in the URL string, including:

[0063] S21: Adjust the length of the URL string to a preset length;

[0064] S22: Tokenize the adjusted URL string at the character level to obtain multiple characters;

[0065] S23: Encode each character to obtain the corresponding character's numerical code;

[0066] S24: Vectorize the digital encoding to obtain the character vector of the corresponding character.

[0067] For example, since malicious URL identification models built on deep learning frameworks cannot directly process URL strings, this embodiment of the invention converts the preprocessed URL string into a vector. First, considering that neural networks require a fixed length for each input, while the length of a URL string is variable, the URL string is truncated or padded to adjust its length to a preset length, ensuring a fixed length. Then, the URL string is segmented into individual characters at the character level; for example, the URL string “yidongsheji.com:82” is segmented into “y”, “i”, “d”, etc. Next, label encoding is used to convert each character into a corresponding numerical code based on a specific correspondence. Each character corresponds to a unique number; for example, “y” corresponds to 25, “i” to 9, and “d” to 4. Finally, an embedding layer is used to convert each numerical code into a corresponding character vector.

[0068] This invention directly treats URL data as a character sequence, which is then input into the malicious URL identification model after a conventional vectorization process. This eliminates the need for complex data preprocessing steps that could introduce additional noise, such as converting the data into a two-dimensional image, thus improving identification accuracy. Furthermore, it avoids the need for segmentation using special symbols, preventing the information contained in special symbols from being overlooked. Experience shows that malicious URLs contain more special symbols than normal URLs, especially "@" and "-". This invention incorporates special symbol information, crucial for malicious URL classification, further enhancing the accuracy of malicious URL identification.

[0069] Further, step S21: Adjusting the length of the URL string to a preset length includes:

[0070] Determine whether the length of the URL string is greater than the preset length;

[0071] When the length of the URL string is greater than the preset length, the URL string is truncated starting from the first character and according to the preset length, so that the length of the truncated URL string is equal to the preset length.

[0072] When the length of the URL string is less than the preset length, the end of the URL string is padded with a set character until the length of the padded URL string is equal to the preset length.

[0073] It should be noted that the preset length and the setting of the set character are not specifically limited in this embodiment of the invention, and users can customize the settings. Taking the number "0" as the set character as an example, the length of the URL string obtained after preprocessing is judged. When the length of the URL string is greater than the preset length, it is truncated starting from the first character of the URL string according to the preset length, so that the length of the truncated URL string is equal to the preset length. When the length of the URL string is less than the preset length, the end of the URL string is padded with the number 0 until the length of the padded URL string is equal to the preset length. For example, padded with the number 0, the string "yidongsheji.com:82" results in the fixed-length string "yidongsheji.com:820000". The string "yidongsheji.com:82 / 82341755" is truncated to obtain the fixed-length string "yidongsheji.com:82 / 823".

[0074] In one optional embodiment, training the malicious URL identification model includes the following steps:

[0075] The pre-collected URL sample data is preprocessed to obtain multiple URL sample strings;

[0076] Each of the URL sample strings is vectorized to obtain the character vector of each character in the corresponding URL sample string; wherein, all the character vectors corresponding to each URL sample string are used as a training sample.

[0077] The training samples are used to train a pre-built malicious URL identification model.

[0078] It should be noted that the preprocessing and vectorization process for each URL sample data can be found in the URL data preprocessing and vectorization process described above, and the network structure of the malicious URL identification model can be found in the above description. Figure 2 As shown, it will not be elaborated further here.

[0079] Specifically, see Figure 4 As shown, training the pre-built malicious URL identification model using the training samples includes:

[0080] S100: For each training sample, the character vector of the training sample is subjected to local feature extraction by the convolutional neural network to obtain the static vector features of each character in the corresponding URL sample string; wherein, the convolutional neural network is a one-dimensional convolutional neural network.

[0081] S200: Global feature extraction is performed on the static vector features through the non-local operation to obtain the first dynamic vector features of each character in the corresponding URL sample string;

[0082] S300: Aggregate the first dynamic vector features of each character in the corresponding URL sample string through the long short-term memory neural network to obtain the second dynamic vector features of the corresponding URL sample string;

[0083] S400: The second dynamic vector feature is identified by the multilayer perceptron to obtain the malicious URL prediction result of the corresponding training sample;

[0084] S500: Based on the URL prediction results, the model parameters of the malicious URL identification model are updated using a backpropagation mechanism to obtain the updated malicious URL identification model. The next round of iteration is then performed based on the updated malicious URL identification model until the preset iteration conditions are met, thus obtaining the trained malicious URL identification model.

[0085] For example, for each URL sample string, the character vectors corresponding to all characters of the URL sample string obtained after character-level word segmentation and vectorization are input into a one-dimensional convolutional neural network. This allows the malicious URL identification model to focus on the local first, obtain the static vector features of each character in the corresponding URL sample string, and form a concept similar to words. This introduces a certain inductive bias into the malicious URL identification model, which can avoid the problem that existing technologies that simply use self-attention mechanisms require a large amount of data for pre-training. Thus, a high-performance malicious URL identification model can be trained with a small amount of URL sample data.

[0086] Nonlocal operations are attention mechanisms used to extract long-range dependencies. They are suitable for processing sequential data and can capture global contextual information, not just local adjacency relationships. Attention mechanisms are used in deep learning models, particularly in the context of natural language processing, to capture relationships between different parts of a sequence. In this mechanism, n tokens are taken as input. Each token (which can be understood as an element vector) is transformed into three parts: query, key, and value, with the data structure as follows:<query,key,value> These vector tokens are obtained by multiplying the input sequence by the learned weight matrix. To update the information, the attention mechanism first calculates the attention weights, i.e., the dot product of the key and query. Then, a softmax function is applied to the dot product result to normalize it, making its sum equal to 1. Afterward, the final output is obtained by weighted summing of the values, and this is fed into the next layer of the malicious URL identification model, i.e., the Long Short-Term Memory neural network. The attention mechanism can be mathematically expressed as follows:

[0087]

[0088] Where X is the input sequence, XW Q XW K XW v Let d represent the weight matrices of the query, key, and value vectors, respectively. k It is the dimension of the key vector.

[0089] See Figure 5 The diagram shows the structure of the non-local operation; let X represent the input to the malicious URL identification model, and set the feature map size of X to T×H×W×1024, where W=Width, H=height, T=temporal, and 1024 represents the size of X; through three 1×1 convolutions (i.e., Figure 5The channels of θ (1×1×1), φ (1×1×1), and g (1×1×1) are reduced by half, resulting in an output feature map size of T×H×W×512, which helps reduce computation. Then, a reshape function is used to adjust the rows, columns, and dimensions of the output T×H×W×512, yielding THW×512, 512×THW, and THW×512 respectively. Next, a matrix multiplication operation is performed between the THW×512 output corresponding to θ (1×1×1) and the 512×THW output corresponding to φ (1×1×1), resulting in THW×THW, which is then fed into a softmax layer for processing. The softmax layer's function is to normalize the output. The normalized THW×THW is multiplied by the THW×512 output of the 1×1×1 convolutional layer g to obtain THW×512. This THW×512 is then reshaped using a reshape function to adjust its rows, columns, and dimensions, restoring it to T×H×W×512. The T×H×W×512 obtained through the above operations is then passed through a 1×1×1 output convolutional layer to restore the number of channels, resulting in T×H×W×1024. Finally, the output of the output convolutional layer is element-wise added to the input of the non-local operation to obtain the final output Z. Wherein, in Figure 5 middle, This indicates an element addition operation; This represents a matrix multiplication operation. Non-local operations can comprehensively consider useful information such as the URL's domain name, path, and filename. Since these parts are geographically distant, compared to existing technologies that simply use convolutional operations to allow the model to consider global information (which requires stacking a large number of convolutional layers and greatly increases model complexity), this invention uses non-local operations to easily improve the performance of this part of URL recognition.

[0090] For the first dynamic vector features of each character in the URL sample string output by the non-local operation, the first dynamic vector features of each character in the URL sample string are aggregated sequentially using the Long Short-Term Memory Neural Network to obtain the second dynamic vector features of the corresponding URL sample string. Compared with the existing technology that uses average pooling as the aggregation method to obtain the representation of the entire URL, which simply averages the vectors corresponding to the characters in the sequence, resulting in the dilution of useful information, the present invention uses the Long Short-Term Memory Neural Network for aggregation without losing important sequence information, and can consider the importance of characters at different positions in the sequence, rather than simply averaging the vectors corresponding to the characters in the sequence.

[0091] Finally, the multilayer perceptron is used to determine whether the URL sample data belongs to a malicious URL based on the final representation (i.e., the second dynamic vector feature) of the entire URL sample data obtained by the long short-term memory neural network. After obtaining the prediction result, the model parameters in the malicious URL identification model are continuously trained and updated through the backpropagation mechanism, and the model is optimized using the cross-entropy loss function. The working principle of using the backpropagation mechanism to train and update the model parameters and using the cross-entropy loss function to optimize the model can be found in existing technologies and will not be elaborated here.

[0092] Furthermore, during the iterative training of the malicious URL identification model, iteration conditions can be set. When these conditions are met, training ends, and the final malicious URL identification model is output. For example, the iteration conditions can be set as the number of iterations reaching a preset iteration threshold or the accuracy of the malicious URL identification model reaching a preset accuracy threshold. After the model training is completed, for any URL to be identified, steps S1-S3 described above can be used to identify the malicious URL and determine whether it is a malicious URL. See [link to URL identification process] for the overall URL identification process. Figure 6 As shown, after the URL sample data is preprocessed, the corresponding URL characters are converted into corresponding character vectors. Then, a neural network model based on non-local operations and Long Short-Term Memory (LSTM) is trained. After that, the URL data to be identified is input into the trained malicious URL identification model to obtain the identification result.

[0093] Compared with the prior art, the malicious URL identification method of this invention preprocesses the URL data to be identified, extracts useful information from the URL data to be identified, then vectorizes the obtained useful URL string at the character granularity to obtain the character vector of each character in the useful URL string, and finally inputs the character vector of each character into the malicious URL identification model for identification to obtain the identification result of whether the URL data to be identified is a malicious URL. Compared to existing technologies that use special symbols as delimiters for word segmentation, which may lose information about special symbols that are crucial for URL classification, and because URL naming lacks specific rules, the segmented results may not strictly align with the concepts of words in natural language, affecting the recognition results, this invention incorporates special symbols in the URL. It first extracts useful information from the URL, then performs word segmentation and vectorization on the extracted useful information at the character level. Combining a one-dimensional convolutional neural network and nonlocal operations, the one-dimensional convolutional neural network allows the malicious URL recognition model to first focus on the local features of the URL data, forming concepts similar to words. Then, nonlocal operations capture global contextual information. The stacked one-dimensional convolutional neural network and nonlocal network enable the malicious URL recognition model to achieve dynamic feature extraction based on contextual information when processing information at each location, thereby effectively improving the accuracy of malicious URL recognition. Furthermore, this invention directly uses the URL string for recognition, without needing to access the target URL or process the content information corresponding to the URL, accelerating the detection speed of malicious URLs. It also eliminates the need for complex data preprocessing steps that may introduce additional noise or lead to information loss, simplifying the detection process and effectively improving the efficiency and accuracy of malicious URL recognition. Furthermore, compared to existing technologies that capture global information by stacking a large number of convolutional layers, resulting in complex models, the malicious URL identification model provided in this embodiment of the invention has a simpler structure.

[0094] Example 2

[0095] See Figure 7 This invention provides a malicious URL identification device, comprising:

[0096] The first preprocessing module 1 is used to preprocess the URL data to be identified to obtain a URL string;

[0097] The first character vectorization module 2 is used to vectorize the URL string to obtain the character vector of each character in the URL string;

[0098] URL recognition module 3 is used to recognize the character vector using a pre-trained malicious URL recognition model to obtain the recognition result of the URL data; wherein, the malicious URL recognition model includes a convolutional neural network, a non-local operation, a long short-term memory neural network, and a multilayer perceptron connected in sequence.

[0099] In an optional embodiment, the first character vectorization module 2 includes:

[0100] A string length adjustment unit is used to adjust the length of the URL string to a preset length;

[0101] The word segmentation unit is used to segment the adjusted URL string into multiple characters at the character level.

[0102] A digital encoding unit is used to encode each character to obtain the digital code of the corresponding character;

[0103] The vectorization processing unit is used to vectorize the digital encoding to obtain the character vector of the corresponding character.

[0104] In one optional embodiment, the string length adjustment unit includes:

[0105] The length determination subunit is used to determine whether the length of the URL string is greater than the preset length;

[0106] The string truncation subunit is used to truncate the URL string according to the preset length, starting from the first character of the URL string, when the length of the URL string is greater than the preset length, so that the length of the truncated URL string is equal to the preset length.

[0107] The string padding subunit is used to pad the end of the URL string with a set character until the length of the padded URL string is equal to the preset length when the length of the URL string is less than the preset length.

[0108] In an optional embodiment, the device further includes:

[0109] The second preprocessing module is used to preprocess multiple URL sample data to be pre-collected to obtain multiple URL sample strings;

[0110] The second character vectorization module is used to vectorize each of the URL sample strings to obtain the character vector of each character in the corresponding URL sample string; wherein, all the character vectors corresponding to each URL sample string are used as a training sample.

[0111] The model training module is used to train a pre-built malicious URL identification model using the training samples.

[0112] In one optional embodiment, the model training module is specifically used for:

[0113] For each training sample, the character vector of the training sample is subjected to local feature extraction by the convolutional neural network to obtain the static vector features of each character in the corresponding URL sample string; wherein, the convolutional neural network is a one-dimensional convolutional neural network.

[0114] The static vector features are extracted globally through the nonlocal operation to obtain the first dynamic vector features of each character in the corresponding URL sample string;

[0115] The first dynamic vector features of each character in the corresponding URL sample string are aggregated by the long short-term memory neural network to obtain the second dynamic vector features of the corresponding URL sample string.

[0116] The second dynamic vector features are used to identify URLs through the multilayer perceptron to obtain the malicious URL prediction results of the corresponding training samples;

[0117] Based on the URL prediction results, the model parameters of the malicious URL identification model are updated using a backpropagation mechanism to obtain the updated malicious URL identification model. The next round of iteration is then performed based on the updated malicious URL identification model until the preset iteration conditions are met, resulting in a trained malicious URL identification model.

[0118] In an optional embodiment, the first preprocessing module 1 includes:

[0119] The filtering unit is used to filter the protocol header in the URL data to obtain the URL string of the corresponding URL data.

[0120] It should be noted that the working process of each module in the malicious URL identification device described in the embodiments of the present invention can refer to the working process of the malicious URL identification method described in the above embodiments, and the technical effect achieved is the same as that of the malicious URL identification device described in the above embodiments, so it will not be repeated here.

[0121] Example 3

[0122] See Figure 8 , Figure 8This is a schematic diagram of a malicious URL identification device provided in an embodiment of the present invention. The malicious URL identification device includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the various malicious URL identification method embodiments described above, such as steps S1 to S3.

[0123] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the malicious URL identification device.

[0124] The malicious URL identification device may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of a malicious URL identification device and does not constitute a limitation on the device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the malicious URL identification device may also include input / output devices, network access devices, buses, etc.

[0125] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the malicious URL identification device, connecting all parts of the device via various interfaces and lines.

[0126] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the malicious URL identification device by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0127] If the modules / units integrated into the malicious URL identification device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0128] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0129] The above description is a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, many improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A malicious URL identification method characterized by, The method comprises the following steps: Preprocessing URL data to be identified to obtain a URL string; Vectorizing the URL string to obtain character vectors of each character in the URL string; Using a pre-trained malicious URL identification model to identify the character vectors to obtain an identification result of the URL data; wherein the malicious URL identification model comprises a convolutional neural network, a non-local operation, a long short-term memory neural network, and a multi-layer perceptron connected in sequence. 2.The malicious URL identification method of claim 1, wherein, The vectorization of the URL string to obtain character vectors of each character in the URL string comprises: Adjusting the length of the URL string to a preset length; Segmenting the adjusted URL string by character granularity to obtain a plurality of characters; Encoding each character to obtain a numerical encoding of the corresponding character; Vectorizing the numerical encoding to obtain a character vector of the corresponding character.

3. The malicious URL recognition method of claim 2, wherein, The step of adjusting the length of the URL string to a preset length comprises: Determining whether the length of the URL string is greater than the preset length; When the length of the URL string is greater than the preset length, the first character of the URL string is taken as the starting point and the URL string is truncated according to the preset length, so that the length of the truncated URL string is equal to the preset length; When the length of the URL string is less than the preset length, a set character is filled at the end of the URL string until the length of the filled URL string is equal to the preset length. 4.The malicious URL identification method of claim 1, wherein, The training of the malicious URL identification model comprises the following steps: Preprocessing a plurality of URL sample data to be pre-collected to obtain a plurality of URL sample strings; Vectorizing each URL sample string to obtain character vectors of each character in the corresponding URL sample string; wherein all character vectors corresponding to each URL sample string serve as a training sample; Training a pre-constructed malicious URL identification model using the training sample.

5. The malicious URL recognition method of claim 4, wherein, The step of training a pre-constructed malicious URL identification model using the training sample comprises: For each training sample, the convolutional neural network is used to extract local features from the character vectors of the training sample to obtain static vector features of each character in the corresponding URL sample string; wherein the convolutional neural network is a one-dimensional convolutional neural network; The non-local operation is used to extract global features from the static vector features to obtain first dynamic vector features of each character in the corresponding URL sample string; The long short-term memory neural network is used to aggregate the first dynamic vector features of each character in the corresponding URL sample string to obtain second dynamic vector features of the corresponding URL sample string; The multi-layer perceptron is used to identify the URL based on the second dynamic vector features to obtain a malicious URL prediction result of the corresponding training sample; According to the URL prediction result, model parameters of the malicious URL identification model are updated by using a back propagation mechanism to obtain an updated malicious URL identification model, and the next round of iteration is updated based on the updated malicious URL identification model until a preset iteration condition is met to obtain a trained malicious URL identification model. 6.The malicious URL identification method of claim 1, wherein, The pre-processing of the URL data to be identified to obtain a URL string comprises: Filtering a protocol header in the URL data to obtain a URL string of the corresponding URL data.

7. The malicious URL recognition method of claim 6, wherein, The pre-processing of the URL data to be identified to obtain a URL string further comprises: Secondly filtering a domain name prefix in the URL string to obtain a final URL string.

8. A malicious URL recognition apparatus characterized by comprising: It comprises: A first pre-processing module configured to pre-process the URL data to be identified to obtain a URL string; A first character vectorization module configured to vectorize the URL string to obtain a character vector of each character in the URL string; A URL identification module configured to identify the character vector by using a pre-trained malicious URL identification model to obtain an identification result of the URL data; wherein the malicious URL identification model comprises a convolutional neural network, a non-local operation, a long short-term memory neural network and a multi-layer perception machine connected in sequence.

9. A malicious URL recognition device characterized by comprising: It comprises: A processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the malicious URL identification method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program controls the device where the computer readable storage medium is located to execute the malicious URL identification method according to any one of claims 1 to 7 when running.

Images