A malicious URL detection and classification method based on capsule neural network

Abstract

This invention relates to a malicious URL detection and classification method based on capsule neural networks, belonging to the interdisciplinary fields of network security, deep learning, and natural language processing. First, a high-speed network is used to integrate embedding representations of URL strings at different granularities. Then, a convolutional neural network is used to extract local semantic and structural features within different ranges of the URL. Finally, these local features are input into the capsule neural network, converting scalar features into vector capsules, enriching the feature representation of the URL. A dynamic routing mechanism between capsule layers effectively aggregates low-level features and outputs the classification results. Compared with existing technologies, this invention fully utilizes information at different granularities of the URL string, introducing capsule neural networks to extract more accurate and richer discriminative features from the URL string, improving the performance of malicious URL classification, accurately identifying malicious URL attack types, and effectively addressing the diversity of obfuscation techniques.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method for detecting and classifying malicious URLs based on capsule neural networks. Background Technology

[0002] In recent years, the rapid development of the internet has greatly enriched people's material and cultural lives, promoted the continuous progress of the information society, and facilitated the rapid growth and development of online banking, e-commerce, social networks, and e-government services. Currently, governments, businesses, and individuals all rely on internet services provided by the World Wide Web. However, technological advancements and the development of online services have also attracted a large number of attackers who use malicious websites to launch cyberattacks against governments, businesses, and individual users for illegal profit, threatening cybersecurity and causing huge economic losses every year.

[0003] A Uniform Resource Locator (URL) is a unique identifier for information resources on the internet, used to locate and access those resources. As the sole entry point to a website, the URL is most easily exploited by malicious attackers to spread fake websites and malicious programs. Cybersecurity experts possess extensive knowledge and experience in this field, making the identification of malicious URLs relatively easy. However, for ordinary users, without sufficient expertise and experience, accurately identifying malicious URL attacks is extremely difficult. Therefore, timely and efficient automatic detection of malicious URLs to protect vulnerable users from cyberattacks is of great practical significance. Furthermore, malicious URL attacks take many forms. Beyond correctly distinguishing between malicious and benign URLs, further classification of malicious URL attack types is essential. This helps users prioritize and direct their defensive actions, allocating higher priority to attacks with greater risk and addressing those potentially causing greater damage first.

[0004] Traditional methods for detecting malicious URLs involve deploying website blacklists in web browsers and identifying malicious URLs by checking if the target URL is on the blacklist. However, blacklist-based detection methods rely on manual feedback to maintain a large blacklist of known malicious URLs, making them unreliable, time-consuming, and difficult to update in a timely manner. With the rise of machine learning, statistical model-based machine learning methods have begun to be widely used in the detection and classification of malicious URLs. While machine learning-based methods have achieved some success in the field of malicious URL detection and classification, they heavily rely on expert knowledge for manual feature design and selection, resulting in significant drawbacks such as being time-consuming, unable to fully extract valuable information from URL strings, and struggling to cope with constantly changing attack strategies.

[0005] In recent years, deep learning methods have become increasingly popular and have achieved state-of-the-art performance in numerous classification tasks across various domains. Deep learning aims to automatically extract appropriate features from raw, unstructured data and use these features to train classification models. Deep learning helps reduce complex feature engineering and builds effective classification models without any domain-specific expertise, making it a mainstream research method in malicious URL detection and classification. Currently, many studies have applied deep learning methods to malicious URL detection and classification, but existing methods still have room for improvement.

[0006] (1) The classification of malicious URL attack types is ignored. Most existing works focus on using binary classification to distinguish between malicious URLs and benign URLs, while few attempt to use deep learning-based methods to further identify the attack types of malicious URLs.

[0007] (2) The information provided by the URL string is not fully utilized. Embedding representations of different granularities can produce different information. Word-level embeddings can better extract overall structural information from the URL string, while character-level embeddings are more sensitive to subtle changes in words and can effectively handle rare words in the URL. Most existing methods simply concatenate embedding representations of different granularities, which makes character-level information easily buried by word-level information and fails to fully utilize the useful information provided by the URL string.

[0008] (3) Insufficient ability to extract discriminative features from URL strings, making it difficult to effectively cope with the diversity of obfuscation techniques. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are currently the most common deep learning models in the field of malicious URL detection. However, these common deep learning models are unable to fully mine discriminative features from URL strings, are not sensitive enough to subtle changes in URL strings, and cannot effectively cope with the diversity of obfuscation techniques. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a malicious URL detection and classification method based on capsule neural network. It utilizes a high-speed network to effectively integrate word-level and character-level embedding information of URL strings, constructs a feature extraction module based on capsule neural network, captures rich and accurate discriminative features from URL strings, and achieves high-precision performance in malicious URL classification tasks.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] A method for detecting and classifying malicious URLs based on capsule neural networks includes the following steps:

[0012] S1. Input dataset, which includes various types of malicious URLs and their category labels. Process the duplicate or disordered data in the dataset and divide the dataset into training set and test set.

[0013] S2. Obtain word-level and character-level embedding representations of URL strings in the dataset, and integrate information at different embedding granularities using a high-speed network to obtain the filtered URL string embedding representations.

[0014] S3. Use a convolutional neural network to extract local semantic and structural features in different ranges from the embedded representation of the filtered URL string;

[0015] S4. Utilize capsule neural networks to process the local semantic and structural features, enrich the URL feature representation, aggregate low-level features, and finally output classification labels through the classification capsule layer.

[0016] Further, step S1 includes the following sub-steps:

[0017] S101. Mix different types of malicious URLs and their category labels to form a dataset with multiple different types of malicious URLs;

[0018] S102. For URL data with disordered format in the dataset, convert the URLs to UTF-8 format; remove duplicate and invalid URL data;

[0019] S103. Divide the training set and the test set according to a specific ratio.

[0020] Further, step S2 includes the following sub-steps:

[0021] S201. Obtain all distinct words that appear in the training set, separated by special characters (e.g., ' / ', '?', '0', etc.), and use all the distinct words... <pad>Mark and <unk>The tags form a word dictionary, generating a word-level embedding matrix;

[0022] For URL sequences whose length is less than the specified maximum length, use <pad>Fill with markers; use <unk>Mark words that appear only once; at the same time, use <unk>The marker indicates words that appear in the test set but not in the training set;

[0023] S202. Using the word-level embedding matrix, obtain the embedding vector of each word in the URL string, and concatenate the embedding vectors of all words in the URL string to form the word-level embedding representation of the entire URL string.

[0024] S203, using l uppercase and lowercase letters, m digits, and n special characters. <pad>Mark and <unk>The tags form a character dictionary, generating a character-level embedding matrix;

[0025] S204. Using the character-level embedding matrix, obtain the embedding vector of each character in the URL string, and concatenate the embedding vectors of all characters in the URL string to form the character-level embedding representation of the entire URL string.

[0026] S205. The word-level embedding representation and the character-level embedding representation are concatenated to obtain the concatenated embedding representation; the concatenated embedding representation is adaptively filtered through a high-speed network to obtain a URL string embedding representation that integrates word-level and character-level information, denoted as the filtered URL string embedding representation.

[0027] Furthermore, the calculation formula for the high-speed network is as follows:

[0028] z wc =H*x wc W H )·T(x wc W T )+x wc ·C(x wc W C )

[0029] Where H(·) is the nonlinear transformation of the traditional neural network, T(·) is the transformation gate, C(·) is the carry gate, and W H W T W C Both are weight matrices, x wc For the concatenated embedded representation, z wc This is an embedded representation of the filtered URL string.

[0030] Furthermore, step S3 includes the following sub-steps:

[0031] S301. The filtered URL string is embedded and input into the parallel convolution module to capture the local semantic and structural features of characters or words in different ranges of the URL string and obtain the differential information in different ranges; the parallel convolution module includes four convolution kernels of different sizes;

[0032] S302. The local semantic and structural features extracted from different ranges by the parallel convolution module are concatenated, and the feature dimension and number of parameters are reduced by two concatenated convolutional layers to generate a low-level feature representation of the URL, denoted as scalar feature.

[0033] Further, step S4 includes the following sub-steps:

[0034] S401. The scalar features output by the convolutional neural network are converted into multidimensional vector capsules through the primary capsule layer;

[0035] S402. Utilize a dynamic routing mechanism to iterate multiple times, adjusting the weights between the primary capsule layer and the classification capsule layer. Connect the vector capsules output by the primary capsule layer to the classification capsule layer to aggregate the features of the primary capsule layer. Use the Squashing function to compress the magnitude of the vector capsules output by the classification capsule layer to the range of (0, 1) and treat it as the probability of the classifier outputting each category label, ultimately obtaining the classification label of the input URL.

[0036] S403. The parameters in the model are continuously trained and updated through the backpropagation mechanism, and the model is optimized using the Margin loss function.

[0037] Further, in step S401, the calculation formula for converting the scalar features output by the convolutional neural network into multidimensional vector capsules through the primary capsule layer is as follows:

[0038]

[0039] in, This represents a convolution operation, where b1 is the bias term and r is the value of the convolution term. i For scalar features, u i ∈R d Let W be a d-dimensional vector capsule, where squash is the squashing function. B For filters.

[0040] Furthermore, step S402 specifically includes the following sub-steps:

[0041] Using the weight matrix W ij Transform the individual vector capsules in the primary capsule layer into vectors.

[0042] The output of the classification capsule layer is obtained by multiplying the vector capsule of each primary capsule layer with the coupling coefficient obtained after three iterations of dynamic routing, and then summing the results.

[0043] The Squashing function compresses the magnitude of the output vector capsules of the classification capsule layer to the range (0, 1), resulting in the final output probabilities for each category label:

[0044]

[0045] Furthermore, the iterative formula for the coupling coefficient in the dynamic routing mechanism is as follows:

[0046]

[0047]

[0048] Where b ij c is the iteration parameter, initialized to 0. ij v is the coupling coefficient. j The output capsule is the compressed classification capsule layer, and exp is the exponentiation operation with base e.

[0049] Furthermore, the formula for calculating the Margin loss function is as follows:

[0050] L c =Tc c max(0,m + -||v c ||) 2 +λ(1-T c max(0,||v) c ||-m - ) 2

[0051] Among them, v c For the c-th output capsule in the category capsule layer, T is true if and only if the current URL belongs to category c. c =1;m + =0.9, m - =0.1; λ=0.25 is used to ensure numerical stability during training;

[0052] The total loss is the sum of the losses of all classification vector capsules.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. This invention aims to achieve accurate classification of malicious URLs using deep learning methods. First, a high-speed network is used to integrate embedding representations of URL strings at different granularities. Then, a convolutional neural network is used to extract local semantic and structural features within different ranges of the URL. Finally, the local features are input into a capsule neural network to convert scalar features into vector capsules, which enriches the feature representation of the URL. The low-level features are effectively aggregated through a dynamic routing mechanism between capsule layers, which can effectively output classification results and has the advantages of high efficiency and accuracy.

[0055] 2. This invention can further accurately identify different types of malicious URL attacks, based on the distinction between malicious and benign URLs.

[0056] 3. This invention utilizes high-speed networks to combine and filter word-level and character-level embedding representations of URL strings, effectively integrating and fully utilizing information at different embedding granularities.

[0057] 4. This invention utilizes primary capsule layers to convert scalar features into vector capsules, enriching the feature representation of URLs. Through a dynamic routing mechanism between capsule layers, it effectively aggregates low-level features, extracting more accurate and richer discriminative features from URL strings, thereby improving the classification performance of the model and effectively addressing the diversity of obfuscation techniques. Attached Figure Description

[0058] Figure 1 This is a flowchart of the malicious URL detection and classification method based on capsule neural network of the present invention;

[0059] Figure 2 This is a schematic diagram of the URL embedding representation generation module in an embodiment of the present invention;

[0060] Figure 3 This is a diagram of the primary capsule layer architecture in an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the dynamic routing process between capsule network layers in an embodiment of the present invention. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0063] Example 1:

[0064] In this embodiment, see Figure 1 This invention discloses a method for detecting and classifying malicious URLs based on capsule neural networks, comprising the following steps:

[0065] Step 1: Input a dataset consisting of various types of malicious URLs and their category labels, process duplicate or disordered data accordingly, and divide the dataset into training and test sets;

[0066] Step 2: Obtain the word-level and character-level embedding representations of the URL string, and effectively integrate information at different embedding granularities using high-speed networks;

[0067] Step 3: Use a convolutional neural network to extract local semantic and structural features within different ranges of the URL string;

[0068] Step 4: Use capsule neural networks to process local features, further enrich the URL feature representation, effectively aggregate low-level features, and finally output classification labels through classification capsule layers.

[0069] In this embodiment, step 1 processes the duplicate or disordered data in the malicious URL dataset accordingly and divides it into training and test sets, including the following steps:

[0070] Step 1.1: Mix different types of malicious URLs and their category labels to create a dataset with multiple different types of malicious URLs;

[0071] Step 1.2: For the URLs in the dataset that have some disordered formatting, convert them to UTF-8 format; remove duplicate and invalid URLs.

[0072] Step 1.3: Divide the training set and the test set into an 8:2 ratio.

[0073] In this embodiment, see Figure 2 Step 2 involves obtaining word-level and character-level embedding representations of the URL string and effectively integrating information at different embedding granularities using a high-speed network, including the following steps:

[0074] Step 2.1: Obtain all distinct words that appear in the training set, separated by special characters (e.g., ' / ', '?', '.', etc.). Use all the distinct words... <pad>Mark and <unk>The tags form a word dictionary of N words, generating a word-level embedding matrix. Where k = 256 represents the embedding vector dimension; specifically, the model uses a fixed-length URL as input, and for URLs shorter than the specified maximum length, it uses... <pad>Fill in the blanks with markers; words that appear only once are called "rare words" and are marked with... <unk>These rare words are replaced with tags to reduce memory usage and interference from rare words on the model. Simultaneously, words not present in the training set will appear in the test set; these "unknown words" are also represented using tags. <unk>The marker indicates.

[0075] Step 2.2: Set the string length to a fixed L = 100 words. URLs shorter than 100 words should use... <pad>Markup is used to prune URLs longer than 100 words, truncating at the 100th word. Unknown words in the URL are pruned. <unk>Tag Representation. Using a word-level embedding matrix, the embedding vector of each word in the URL string is obtained. Then, the embedding vectors of all words in the URL string are concatenated to form the word-level embedding representation of the entire URL string.

[0076] Step 2.3: Use 52 uppercase and lowercase letters, 10 numbers, and 32 special characters. <pad>Mark and <unk>The tags form a character dictionary, generating a character-level embedding matrix. Where k = 256 represents the embedding vector dimension.

[0077] Step 2.4: Set the string length to a fixed L = 100 characters. URLs shorter than 100 characters should use... <pad>Markup is used to truncate URLs longer than 100 characters at the 100th character. Unknown characters in the URL are marked with padding. <unk>Tag representation. Using a character-level embedding matrix, the embedding vector of each character in the URL string is obtained. Then, the embedding vectors of all characters in the URL string are concatenated to form the character-level embedding representation of the entire URL string.

[0078] Step 2.5: Concatenate the word-level embedding representation and the character-level embedding representation using the concat method to obtain the embedding representation. Subsequently, the concatenated embedding representation is adaptively filtered using a high-speed network to obtain a URL string embedding representation that effectively integrates word-level and character-level information. The formula for calculating high-speed networks is:

[0079] z wc =H(x) wc W H )·T(x wc W T )+x wc ·C(x wc W C )

[0080] Where H(·) is the nonlinear transformation of a traditional neural network, T(·) is the transform gate, C(·) is the carry gate, and W... H W T W C All are weight matrices.

[0081] In this embodiment, step 3 utilizes a convolutional neural network to extract local semantic and structural features within different ranges of the URL string, including the following steps:

[0082] Step 3.1: Embed the filtered URL string to represent z wc The input is fed into a parallel convolutional module consisting of four different sizes of convolutional kernels to capture the semantic and structural features of characters or words in different ranges within the URL string and obtain the differential information in different ranges.

[0083] Step 3.2: Concatenate the local features extracted from different ranges by the parallel convolutional module, and reduce the feature dimension and number of parameters through two concatenated convolutional layers to improve detection efficiency. This ultimately generates a low-level feature representation r of the URL.

[0084] In this embodiment, step 4 utilizes a capsule neural network to process local features, further enriching the URL feature representation, effectively aggregating low-level features, and finally outputting classification labels through a classification capsule layer, including the following steps:

[0085] Step 4.1: See Figure 3 Through the primary capsule layer, using filter W B The scalar features (i.e., low-level feature representation r) output by the convolutional neural network are transformed into a d-dimensional vector capsule u. i ∈R d This is achieved by using rich URL features to obtain more distinctive high-level features; the calculation formula is as follows:

[0086]

[0087] in, This represents a convolution operation, where b1 is the bias term and r is the value of the convolution term. i For scalar features, u i ∈R d Let be a d-dimensional vector capsule, and squash be the Squashing function.

[0088] Step 4.2: See Figure 4 By iteratively adjusting the weights between the primary capsule layer and the classification capsule layer using a dynamic routing mechanism, the vector capsules output by the primary capsule layer are connected to the classification capsule layer to effectively aggregate the features of the primary capsule layer. The Squashing function is used to compress the magnitude of the output vector capsules of the classification capsule layer to the range of (0,1) and treat it as the probability of the classifier outputting each category label, finally obtaining the category label of the input URL;

[0089] First, using the weight matrix W ij Transform the individual vector capsules in the primary capsule layer into vectors. Secondly, the output of the classification capsule layer is obtained by multiplying the vector capsule of each primary capsule layer with the coupling coefficient obtained after three iterations of dynamic routing and then summing the results: Finally, the Squashing function compresses the magnitude of the classification capsule output vector to the range of (0,1) to obtain the final output probabilities of each class label:

[0090]

[0091] The iterative formula for the coupling coefficient in dynamic routing is:

[0092]

[0093]

[0094] Where b ij c is the iteration parameter, initialized to 0. ij v is the coupling coefficient. j The output capsule is the compressed classification capsule layer, and exp is the exponentiation operation with base e.

[0095] Step 4.3: The parameters in the model are continuously trained and updated using the backpropagation mechanism, and the model is optimized using the Margin loss function. The formula for calculating the Margin loss function is:

[0096] L c =T c max(0,m + -||v c ||) 2 +λ(1-T c max(0,||v) c ||-m - ) 2

[0097] Among them, v c For the c-th output capsule in the category capsule layer, T is true if and only if the current URL belongs to category c. c =1;m + =0.9, m - =0.1; λ=0.25 are used to ensure numerical stability during training.

[0098] The total loss is the sum of the losses of all classification vector capsules.

[0099] It is worth noting that the dataset constructed in this embodiment includes five different categories of URLs, which are captured from real network environments. These different categories of URLs represent different attack types: legitimate and benign URLs, phishing URLs, malicious URLs, spam URLs, and defaced URLs. The category labels are {0, 1, 2, 3, 4}, resulting in five output capsules. The loss for each output capsule is calculated using the Margin loss function, so c can be any value from {0, 1, 2, 3, 4}. In other implementations, different numbers of URL types can be set according to specific needs.

[0100] In summary, this invention presents a malicious URL detection and classification method based on capsule neural networks, aiming to achieve accurate classification of malicious URLs using deep learning methods. First, a high-speed network effectively integrates embedding representations of URL strings at different granularities, fully utilizing useful information at different levels. Second, a primary capsule layer transforms scalar features into vector capsules, enriching the feature representation of the URL. A dynamic routing mechanism between capsule layers effectively aggregates low-level features, extracting more accurate and richer discriminative features from the URL string. Finally, this improves the model's classification performance, effectively addressing the diversity of malicious URL obfuscation techniques. This is of great significance for effectively protecting users from malicious URL attacks, and its application prospects are promising.

[0101] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.< / unk> < / pad> < / unk> < / pad> < / unk> < / pad> < / unk> < / unk> < / pad> < / unk> < / pad> < / unk> < / pad> < / unk> < / unk> < / pad> < / unk> < / pad>

Claims

1. A capsule neural network based malicious URL detection and classification method, characterized in that, Includes the following steps: S1. Input dataset, which includes various types of URLs and their category labels. Process duplicate or disordered data in the dataset and divide the dataset into training set and test set. S2. Obtain word-level and character-level embedding representations of URL strings in the dataset, and integrate information at different embedding granularities using a high-speed network to obtain the filtered URL string embedding representation. S3. Use a convolutional neural network to extract local semantic and structural features in different ranges from the embedded representation of the filtered URL string; S4. Use capsule neural networks to process the local semantic and structural features, enrich the URL feature representation, aggregate low-level features, and finally output classification labels through the classification capsule layer. Step S1 includes the following sub-steps: S101. Mix different types of URLs and their category labels to form a dataset with multiple different types of URLs; S102. For URL data with disordered format in the dataset, convert the URLs to UTF-8 format; remove duplicate and invalid URL data; S103. Divide the training set and the test set according to a specific ratio; Step S2 includes the following sub-steps: S201, obtaining all different words separated by special characters appearing in the training set, using all different words, <pad>marker and <unk> The tags form a word dictionary, generating a word-level embedding matrix;< / unk> < / pad> where, for URL sequences of length less than the prescribed maximum length, use <pad>The marker is filled; with <unk>The marked words are replaced with the number of times they appear; at the same time, the <unk> The marker indicates words that appear in the test set but not in the training set;< / unk> < / unk> < / pad> S202. Using the word-level embedding matrix, obtain the embedding vector of each word in the URL string, and concatenate the embedding vectors of all words in the URL string to form the word-level embedding representation of the entire URL string. S203、using l one uppercase letter, m one number, n one special character, <pad>marker and <unk> The tags form a character dictionary, generating a character-level embedding matrix;< / unk> < / pad> S204. Using the character-level embedding matrix, obtain the embedding vector of each character in the URL string, and concatenate the embedding vectors of all characters in the URL string to form the character-level embedding representation of the entire URL string. S205. The word-level embedding representation and the character-level embedding representation are concatenated to obtain the concatenated embedding representation; the concatenated embedding representation is adaptively filtered through a high-speed network to obtain a URL string embedding representation that integrates word-level and character-level information, denoted as the filtered URL string embedding representation. The calculation formula for the high-speed network is as follows: in, For nonlinear transformations of traditional neural networks, For transformation gate, For carrying doors, , , Both are weight matrices. This is the concatenated embedded representation. The filtered URL string is embedded as a representation; Step S3 includes the following sub-steps: S301. The filtered URL string is embedded and input into the parallel convolution module to capture the local semantic and structural features of characters or words in different ranges of the URL string and obtain the differential information in different ranges; the parallel convolution module includes four convolution kernels of different sizes; S302. The local semantic and structural features extracted from different ranges by the parallel convolution module are concatenated, and the feature dimension and number of parameters are reduced by two concatenated convolutional layers to generate a low-level feature representation of the URL, denoted as scalar feature. Step S4 includes the following sub-steps: S401. The scalar features output by the convolutional neural network are converted into multidimensional vector capsules through the primary capsule layer; S402. Utilize a dynamic routing mechanism to iterate multiple times, adjusting the weights between the primary capsule layer and the classification capsule layer. Connect the vector capsules output by the primary capsule layer to the classification capsule layer to aggregate the features of the primary capsule layer. Use the Squashing function to compress the magnitude of the vector capsules output by the classification capsule layer to the range of (0, 1) and treat it as the probability of the classifier outputting each category label, ultimately obtaining the classification label of the input URL. S403. The parameters in the model are continuously trained and updated through the backpropagation mechanism, and the model is optimized using the Margin loss function; In step S401, the formula for converting the scalar features output by the convolutional neural network into multidimensional vector capsules through the primary capsule layer is as follows: in, This represents the convolution operation. For bias terms, As a scalar feature, for dimensional vector capsule, For the Squashing function, For filters; Step S402 specifically includes the following sub-steps: Utilizing a weight matrix Converting individual vector capsules in a primary capsule layer into vectors : ; The output of the classification capsule layer is obtained by multiplying the coupling coefficients obtained after 3 iterations of dynamic routing of the vector capsules of each primary capsule layer and then adding them together: ; The Squashing function compresses the magnitude of the output vector capsules of the classification capsule layer to the range of (0, 1), resulting in the final output probabilities for each category label: ； The iterative formula for the coupling coefficient in the dynamic routing mechanism is: wherein is an iteration parameter, initialized to 0, is a coupling coefficient, is an output capsule of the compressed capsule layer of the classification, is an exponential operation with base is an exponential operation with base The formula for calculating the Margin loss function is as follows: in, For the classification of capsule layers An output capsule is generated if and only if the current URL belongs to the class. hour, ; , ; Used to ensure numerical stability during the training process; The total loss is the sum of the losses of all classification vector capsules.

Images