Domain name retrieval method and device based on webpage icon similarity calculation and medium

Abstract

The application relates to the technical field of network security analysis, in particular to a domain name retrieval method and device based on webpage icon similarity calculation and a medium. The method comprises the following steps: when detecting a webpage icon to be retrieved, determining the graphic features and semantic features of the webpage icon, wherein the graphic features comprise a color histogram, ORB features, shape moment features and edge features; performing multi-modal fusion on the graphic features and the semantic features to obtain multi-modal fusion features; performing similarity calculation on the multi-modal fusion features and pre-stored suspicious icon features respectively, and performing suspicious domain name retrieval according to suspicious icon features with a similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names. The method solves the problems of poor retrieval accuracy and low efficiency of an icon retrieval system caused by the lack of semantic understanding.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network security analysis, and in particular to a domain name retrieval method and device based on webpage icon similarity calculation and a medium. BACKGROUND

[0002] Under the background of accelerating digital development, online assets (domain names, servers, IPs, etc.) of enterprises, institutions and individuals have become the main targets of network attacks. Attackers usually use domain name deception (such as Typosquatting), brand imitation, phishing websites, malicious redirection and other means to steal user data, commit financial fraud or spread malicious software. Therefore, accurately identifying and classifying the attack surface assets of enterprises is crucial for security defense.

[0003] Existing webpage icon retrieval systems mainly include two categories: 1. Retrieval systems based on traditional image processing and local features, which mainly use OpenCV to extract color histograms, SIFT / SURF, etc. Local features are combined with inverted indexes (such as Faiss) for similarity matching, but this system lacks understanding of global semantics and cannot solve the robustness problem of icon deformation and illumination changes; 2. Single model retrieval systems based on deep learning, which use CNN (such as ResNet) or ViT model to extract image features, combined with cosine similarity calculation, but this system is difficult to capture global structure, has low retrieval efficiency, cannot be optimized for large-scale vector libraries, and has high time complexity. SUMMARY

[0004] Therefore, the present application provides a domain name retrieval method and device based on webpage icon similarity calculation, which solves the problem of low retrieval efficiency caused by the lack of understanding of global semantics in webpage icon retrieval systems.

[0005] According to a first aspect of an embodiment of the present application, a domain name retrieval method based on webpage icon similarity calculation is provided, which comprises:

[0006] When a webpage icon to be retrieved is detected, the graphical features and semantic features of the webpage icon are determined, the graphical features including color histograms, ORB features, shape moment features and edge features;

[0007] The graphical features and semantic features are subjected to multi-modal fusion to obtain multi-modal fusion features;

[0008] The multi-modal fusion features are subjected to similarity calculation with a plurality of pre-stored suspicious icon features, and suspicious domain names are retrieved according to suspicious icon features with similarity greater than a pre-set similarity threshold, so as to be managed according to the retrieved suspicious domain names.

[0009] In a second aspect, the embodiment of the present application discloses a domain name retrieval device based on webpage icon similarity calculation, which comprises:

[0010] An icon feature determination module is configured to determine the graphical features and semantic features of the webpage icon when detecting the webpage icon to be retrieved, wherein the graphical features comprise a color histogram, ORB features, shape moment features and edge features.

[0011] An icon feature fusion module is configured to perform multi-modal fusion on the graphical features and the semantic features to obtain multi-modal fusion features.

[0012] A retrieval result determination module is configured to perform similarity calculation on the multi-modal fusion features and a plurality of suspicious icon features stored in advance, and perform suspicious domain name retrieval according to the suspicious icon features with a similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names.

[0013] In a third aspect, the embodiment of the present application discloses an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the computer program.

[0014] In a fourth aspect, the embodiment of the present application discloses a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0015] When detecting the webpage icon to be retrieved, the embodiment of the present application determines the graphical features and semantic features of the webpage icon, wherein the graphical features comprise a color histogram, ORB features, shape moment features and edge features. After the fusion of the graphical features and the semantic features, similarity calculation is performed on a plurality of suspicious icon features, and suspicious domain name retrieval is performed according to the icon features with a similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names. This way of combining image features and semantic features for domain name retrieval solves the problems of poor retrieval accuracy and low efficiency caused by the lack of semantic understanding in icon retrieval, and can accurately identify and efficiently match the icon (ICON) features in the Internet assets, thereby improving the comprehensiveness of attack surface management, the real-time performance of risk monitoring and the accuracy of vulnerability positioning. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate one embodiment consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0017] Figure 1 A flowchart of a domain name retrieval method based on webpage icon similarity calculation is provided for an embodiment of the present application.

[0018] Figure 2 A shape moment feature extraction flowchart is provided in the domain name retrieval method based on webpage icon similarity calculation according to an embodiment of the present application.

[0019] Figure 3 An application flowchart of the domain name retrieval method based on webpage icon similarity calculation is provided according to an embodiment of the present application.

[0020] Figure 4 A processing diagram of image features in the application flowchart of the domain name retrieval method based on webpage icon similarity calculation is provided according to an embodiment of the present application.

[0021] Figure 5 A structure diagram of the domain name retrieval device based on webpage icon similarity calculation is provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0023] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart.

[0024] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments.

[0025] According to an embodiment of the present application, a domain name retrieval method based on webpage icon similarity calculation is provided, as shown in Figure 1 The method includes steps S101, S102 and S103.

[0026] Step S101: When the webpage icon to be retrieved is detected, the graphical features and semantic features of the webpage icon are determined, the graphical features including color histogram, ORB feature, shape moment feature and edge feature.

[0027] Specifically, the retrieval platform can provide a variety of webpage icon acquisition interfaces. For example, the webpage icon to be retrieved uploaded by the user is acquired through a preset image upload interface; for another example, a domain name editing interface can be provided, the retrieval platform loads the website content after acquiring the domain name input by the user through the domain name editing interface, and outputs the candidate icon according to the loading result, and determines the candidate icon as the webpage icon to be retrieved when detecting the selection operation of the user on the candidate icon.

[0028] Specifically, when the retrieval platform receives the webpage icon to be retrieved, a pre-configured feature extraction algorithm can be invoked to perform feature extraction on the webpage icon, or the webpage icon can be sent to a third party to obtain image features and semantic features of the webpage icon by using tools provided by the third party. The embodiment utilizes the highly optimized open source library OpenCV of computer vision application program and semantic feature encoder to analyze the visual content of the webpage icon, and extracts key information for similar icon identification.

[0029] Step S102: Multi-modal fusion is performed on the graphic features and semantic features to obtain multi-modal fusion features.

[0030] Specifically, the fusion of the graphic features and the semantic features is achieved by splicing the two.

[0031] Step S103: Similarity calculation is performed on the multi-modal fusion features and pre-stored multiple suspicious icon features respectively, and suspicious domain name retrieval is performed according to suspicious icon features with similarity greater than a preset similarity threshold, so as to perform management according to the retrieved suspicious domain names.

[0032] Specifically, the similarity threshold can be preset, or the similarity threshold can be set after obtaining the similarity calculation result. In the case where the similarity threshold can be preset, multiple levels of retrieval accuracy can be provided in advance, such as high similarity retrieval, higher similarity retrieval, and general similarity retrieval, wherein the similarity threshold of high similarity retrieval can be set to 90%, that is, icons with similarity greater than 90% are similar icons; or the similarity threshold is determined according to the parameters input by the user in advance. In the case where the similarity threshold is set after obtaining the similarity calculation result, a plurality of similarity threshold options can be provided in advance, and the similarity calculation result is updated according to the user's selection operation after obtaining the similarity calculation result.

[0033] Specifically, the retrieved suspicious domain names can be taken as management objects to perform management operations such as monitoring, or the retrieved suspicious domain names can be first output in a list for visualization for user selection, and the management objects are determined by detecting the user's selection operation on the suspicious domain names in the list.

[0034] When the icon of the webpage to be searched is detected by the embodiment of the application, the graphical features and semantic features of the icon of the webpage are determined, the graphical features include color histogram, ORB feature, shape moment feature and edge feature, similarity calculation is performed on the fused graphical features and semantic features and the plurality of suspicious icon features, and suspicious domain name searching is performed according to the icon features with similarity greater than a preset similarity threshold, so as to perform management according to the searched suspicious domain name. In this way, the icon searching is combined with the semantic features to perform domain name searching, which solves the problems of low searching accuracy and low efficiency caused by lack of semantic understanding in icon searching, and can accurately identify and efficiently match the icon (ICON) features in the Internet asset, thereby improving the comprehensiveness of attack surface management, the real-time performance of risk monitoring and the accuracy of vulnerability positioning.

[0035] In some implementations, before step S101, the method further includes: step S1011 (not shown in the figure): converting the preprocessed webpage icon into an RGB image, a BGR image and a grayscale image; and step S101 of determining the semantic features of the webpage icon further includes: step S1012 (not shown in the figure): inputting the RGB image into a pre-trained semantic feature model to obtain semantic features; step S101 of determining the graphical features of the webpage icon further includes: step S1013 (not shown in the figure): performing image channel statistical analysis on the BGR image to obtain a color histogram; step S1014 (not shown in the figure): performing corner detection on the grayscale image by using an improved FAST detector, determining the main direction of the detected corner, performing direction alignment on the coordinates of the selected first predetermined number of point pairs after filtering, calculating the variance of the first predetermined number of point pair coordinates after filtering, and generating descriptors for the second predetermined number of point pair coordinates in the front of the order to obtain ORB features; step S1015 (not shown in the figure): calculating the average brightness of the grayscale image, converting the grayscale image into a binary image according to the average brightness, performing denoising, extracting the maximum contour of the denoised binary image, and performing logarithmic compression after calculating the spatial moment and hu moment according to the maximum contour to obtain shape moment features; and step S1016 (not shown in the figure): performing edge detection on the preprocessed webpage icon to obtain edge features.

[0036] Specifically, a detector can be created based on the ORB algorithm in OpenCV, and the detectAndComputer method is used to detect key points and calculate feature descriptors to complete the extraction of ORB features in step S101.

[0037] In the ORB feature extraction, the direction compensation is determined by calculating the image moment of the neighborhood (i.e. the surrounding predetermined area range, for example, a 31x31 pixel window) of the key point, and the specific steps include centroid calculation and main direction calculation. The formula for centroid calculation is: and where I(x, y) is the intensity (grayscale value) of pixel (x, y), the numerator is the weighted sum of coordinates, and the denominator is the total intensity (for normalization). The principal direction calculation is through the principal direction angle calculated from the centroid offset, where the principal direction angle is calculated as: The principal direction angle represents the principal direction of the local area grayscale gradient, and is used to rotate the BRIEF point pair coordinates so that the descriptor has rotation invariance. In the embodiments of the present application, the FAST focal point detection method is used for ORB. Compared with other corner detection methods, the FAST focal point detection has controllable parameters, can limit the maximum number of feature points detected by ORB through the setting of nfeatures, and avoid the overfitting problem of texture regions; at the same time, the corner points detected by FAST can be directly input into the improved BRIEF to generate a fixed-length descriptor, and the compatibility with the method used for binary compression of the descriptor (such as mean compression) is better.

[0038] Specifically, the shape moment feature can be extracted with reference to Figure 2 First, the web icon is preprocessed using the tool OpenCV, such as converting the web icon to a target size (such as 224*224), and then converting to a grayscale image to start the process of shape moment feature extraction. First, a dynamic binarization function such as adaptive_binarization can be called to calculate the average brightness, and different binarization strategies can be selected according to different average brightness (such as mean<50 or mean>200), for example, if the average threshold is less than 50, dark background threshold processing is performed, if the average threshold is greater than 200, high-light threshold processing is performed, otherwise OTSU automatic threshold processing is performed. Dark images (low mean) set a lower threshold to retain details, and bright images (high mean) set a higher threshold to avoid overexposure; the OTSU automatic threshold is suitable for high-contrast images. When applied, if the average brightness is less than 50, assuming that the dark background threshold is 20, then the pixels with a brightness higher than 20 are set to 255 (white), and the pixels with a brightness lower than 20 are set to 0 (black); the OTSU algorithm automatically determines the best segmentation threshold by maximizing the inter-class variance of the foreground and background, and is suitable for images with bimodal histograms. After binarization processing, a binary image is returned, and then a morphological opening operation method is used to denoise and detect whether the contour exists, if yes, the largest contour is selected and spatial moments and Hu moments are calculated, and then logarithmic compression processing is performed to enhance the details of the low brightness area and suppress the excessive changes of the high brightness area, thereby improving the stability of the image features.

[0039] In the above implementation, the semantic feature extraction model includes a convolutional neural network (CNN) and a Transformer hierarchical encoder, and the step S1012 further includes: inputting the webpage icon into the convolutional neural network (CNN) for processing to obtain initial embedding vectors of image blocks; and inputting the initial embedding vectors after position encoding into the Transformer hierarchical encoder to obtain the semantic features.

[0040] Specifically, the image of a predetermined size can be input into the convolutional neural network (CNN), i.e., the webpage icon is first converted into a predetermined size (such as 224x224) and then input into the convolutional neural network (CNN), and the icon is divided into 14x14=196 blocks by a convolution kernel (such as 16x16) of the convolutional neural network (CNN), each block having a size of 16x16x3 and containing RGB channels. Then, the blocks are embedded, and the embedding dimension is 768. Each block is mapped to a 768-dimensional vector, i.e., after converting the 224x224 image into a 14x14x768 feature map, the 14x14x768 feature map is flattened and spliced, the 14x14x768 feature map is flattened into a 196x768 sequence, i.e., 196 blocks, and each block corresponds to a 768-dimensional vector. Each block is described by a 768-dimensional vector to provide local details of semantic information, and the position encoding provides features of the overall layout, which provides a basis for subsequent global analysis by the Transformer hierarchical encoder.

[0041] In the above embodiment, the Transformer hierarchical encoder includes a plurality of encoding layers, each encoding layer includes a first sub-layer and a second sub-layer, the first sub-layer includes a first normalization component, a multi-head attention component and a first residual connection component connected in sequence, the second sub-layer includes a second normalization component, a feedforward network and a second residual connection component connected in sequence, the initial embedding vectors are input into the Transformer hierarchical encoder after position encoding to obtain the semantic features, including: inputting the initial embedding vectors after position encoding into the first sub-layer, the first sub-layer performing normalization and global dependency calculation by the first normalization component and the multi-head attention component respectively, and performing residual connection by the first residual connection component to obtain a residual connection result, taking the residual connection result as an input of the second sub-layer, the second sub-layer performing normalization and nonlinear transformation enhancement by the second normalization component and the feedforward network respectively to obtain a feedforward output feature, and connecting the residual connection result and the feedforward output feature by the second residual connection component to obtain the semantic features.

[0042] Specifically, the input image size of the Transformer hierarchical encoder is generally 224x224. The Transformer encoder adopts 12 layers of Transformer encoding layers, each layer containing a multi-head self-attention (MHSA) and a feedforward network (FFN). The 768-dimensional vector corresponding to the class token of the last layer is taken as the global semantic feature.

[0043] In some implementations, step S102 further includes:

[0044] The color histogram, ORB feature, shape moment feature, and edge feature are fully connected to obtain a structure feature;

[0045] The structure feature and the semantic feature are weighted and fused, and then dimensionally reduced to obtain a multi-modal fusion feature.

[0046] Specifically, the weights of the structure feature and the semantic feature can be adjusted according to external input (threat intelligence, user configuration, etc.). When applied, the weights can be determined according to a predefined weight adjustment rule library as shown in Table 1. For example, if image blurring is detected (structure feature reliability decreases), the weight of the structure feature is reduced; if the text label confidence is high (the semantic feature is more reliable), the weight of the semantic feature is increased.

[0047] Table 1 - Weight adjustment and its trigger condition correspondence

[0048]

[0049] Specifically, after preprocessing the webpage icon (such as image size adjustment), the image sharpness score of the webpage icon can be obtained by a preset image sharpness evaluation algorithm, such as the Laplacian gradient method. When applied, if the image sharpness score trigger condition, i.e., the image sharpness score is less than the image sharpness threshold, the weights of the structure feature and the semantic feature are adjusted to 0.3 and 0.7, respectively; otherwise, the default weights of the structure feature and the semantic feature are both 0.5. Specifically, the semantic feature confidence of the webpage icon is calculated after preprocessing the webpage icon (such as image size adjustment). The semantic feature confidence is used to measure the certainty of the model's understanding of the image content semantics. The embodiment of the present application uses a semantic feature extraction model ViT model to output global semantic features. The confidence is calculated based on the probability distribution of the classification layer, and the calculation method uses the probability entropy method (normalized confidence). The more concentrated the probability distribution, the lower the entropy value, and the higher the confidence. For example, if the semantic feature confidence is greater than the semantic feature confidence threshold, the weight is increased by 0.2 based on the default weight. After the increase, the structure feature and the semantic feature can be normalized or the weight of the structure feature is reduced.

[0050] In some embodiments, step S103 further includes:

[0051] The similarity of the multimodal fusion features included in the local Faiss cache is calculated for multiple suspicious icon features;

[0052] If the similarity between the multimodal fusion feature and any suspicious icon feature in the local Faiss cache is less than the similarity threshold, the multimodal fusion feature is forwarded to the Milvus cluster for similarity calculation.

[0053] Obtain the similarity calculation results from the Milvus cluster, and take the icons whose similarity is greater than the preset similarity threshold in the similarity calculation results from the Milvus cluster as similar icons, and search according to the domain name corresponding to the similar icons.

[0054] Specifically, the local Faiss cache can be adjusted based on pre-configured strategies. For example, the local Faiss cache can be for icon features that are matched within a predetermined time period (such as the most recent hour).

[0055] In this embodiment, the local Faiss cache stores the icon features of recent high-frequency queries, which can achieve millisecond-level response and reduce the dependence on the Milvus cluster. The Milvus cluster has the function of high-concurrency real-time retrieval, which can meet large-scale real-time requests and perform domain name icon similarity calculation across the entire network.

[0056] This application achieves a collaborative division of labor between online and offline services through local Faiss caching and Milvus clusters, effectively balancing high-concurrency real-time retrieval with low-latency local caching.

[0057] In some embodiments, before forwarding the multimodal fusion feature to the Milvus cluster, the method further includes: determining the domain name type corresponding to the webpage icon; forwarding the multimodal fusion feature to the Milvus cluster includes: calculating the similarity of the multimodal fusion feature with multiple suspicious icon features included in the Milvus database corresponding to the domain name type.

[0058] Specifically, the domain name type can be determined based on the domain name suffix. For example, if the domain name is "bank.com", when the local Faiss cache query fails, the data stored on the node with the domain name type "bank.com" in the Milvus cluster will be used to calculate similarity, thereby speeding up the icon retrieval.

[0059] In some embodiments, forwarding the multimodal fusion feature to the Milvus cluster further includes: determining the domain name metadata of the webpage icon; forwarding the multimodal fusion feature and the domain name metadata to the Milvus cluster, so as to perform similarity calculation on the multimodal fusion feature after filtering the Milvus cluster based on the domain name metadata.

[0060] Specifically, the mapping relationship between the feature vector database and the metadata database can be established, such as the mapping relationship between the domain_name field and the feature vector, so that when searching, vector similarity searching can be performed first, and then the vector and the metadata are combined to achieve the purpose of accurate searching according to the metadata filtering result.

[0061] The following will be described in combination with Figure 3 and Figure 4 The present application is described. For example, in the counterfeit asset identification scene, input the web icon to be queried which may be counterfeited, after OpenCV preprocessing (such as image normalization, graying, noise removal, and geometric correction) of the web icon, double-path features are extracted, that is, ViT global embedding features are extracted through the Transformer semantic branch, and color histogram, ORB feature, shape moment feature, and edge feature are extracted through the OpenCV local feature branch. Among them, the color histogram adopts an RGB three-channel joint quantization strategy, a three-dimensional histogram is calculated through cv2.calcHist, each channel is quantized into 8 intervals (total dimension 8x8x8=512), and normalized to eliminate light difference. The ORB feature adopts ORB (Oriented FAST and Rotated BRIEF), and a 32-bit binary vector is generated through FAST corner detection and rBRIEF descriptor. Compared with traditional ORB feature extraction (the maximum number of features is 500, and the original descriptor is generated), the present application limits the maximum number of feature points of ORB detection to nfeatures=50, and combines mean compression to compress the descriptor into a 32-dimensional fixed-length vector. In terms of single-frame processing time, memory occupation, and matching accuracy, the system performance is better. The Hu shape moment feature can convert the image to a grayscale image (cv2.COLOR_BGR2GRAY) through preprocessing, and then calculate 7 Hu invariants based on the image grayscale moment, output a 7-dimensional vector to directly represent the global shape feature. These features have translation, rotation, and scaling invariance, and the redundant information is eliminated through nonlinear combination of central moments. For example Figure 4 Taking an input image of 224*224*3 as an example, the color feature extracts a 512-dimensional RGB image, the shape feature extracts a 7-dimensional Hu invariant moment, the texture feature extracts a 32-dimensional ORB feature, and the edge feature extracts a 64-dimensional edge feature. The fusion of these features obtains a 615-dimensional structure vector feature. The structure vector feature and the semantic vector are horizontally stacked by np.hstack to obtain a composite vector, which retains the complementary information of the two modalities.

[0062] One embodiment of the present application provides a domain name retrieval device based on web icon similarity calculation, as shown in Figure 5As shown, the apparatus 50 comprises an icon feature determination module 501, an icon feature fusion module 502, and a search result determination module 503.

[0063] The icon feature determination module 501 is configured to determine the graphical features and semantic features of the web icon when the web icon to be searched is detected, wherein the graphical features include a color histogram, an ORB feature, a shape moment feature, and an edge feature.

[0064] The icon feature fusion module 502 is configured to perform multi-modal fusion on the graphical features and the semantic features to obtain multi-modal fusion features.

[0065] The search result determination module 503 is configured to perform similarity calculation on the multi-modal fusion features and a plurality of suspicious icon features respectively, and perform suspicious domain name search according to suspicious icon features with a similarity greater than a preset similarity threshold, so as to perform management according to the searched suspicious domain names.

[0066] When the web icon to be searched is detected, the embodiment of the present application determines the graphical features and semantic features of the web icon, wherein the graphical features include a color histogram, an ORB feature, a shape moment feature, and an edge feature. After the fusion of the graphical features and the semantic features, similarity calculation is performed on a plurality of suspicious icon features, and suspicious domain name search is performed according to icon features with a similarity greater than a preset similarity threshold, so as to perform management according to the searched suspicious domain names. This way of combining image features and semantic features for domain name search solves the problem of poor search accuracy and low efficiency caused by the lack of semantic understanding in icon search, and can accurately identify and efficiently match the icon (ICON) features in the Internet assets, thereby improving the comprehensiveness of attack surface management, the real-time performance of risk monitoring, and the accuracy of vulnerability positioning.

[0067] Further, before determining the graphical features and semantic features of the web icon, the icon feature determination module further comprises:

[0068] The image preprocessing submodule is configured to convert the preprocessed web icon into an RGB image, a BGR image, and a grayscale image.

[0069] The icon feature determination module comprises a first branch feature extraction submodule and a second branch feature extraction submodule, wherein the first branch feature extraction submodule is configured to input the RGB image into a pre-trained semantic feature model to obtain semantic features; the second branch feature extraction submodule comprises a color histogram determination unit configured to perform image channel statistical analysis on the BGR image to obtain a color histogram; an ORB feature determination unit configured to perform corner detection on the grayscale image based on an improved FAST detector, determine the main direction of the detected corner points, perform direction alignment on the coordinates of the selected first predetermined number of point pairs, filter the first predetermined number of point pairs, calculate the variance of the filtered first predetermined number of point pairs, and generate descriptors for the second predetermined number of point pairs in the top order to obtain ORB features; a shape moment feature determination unit configured to calculate the average brightness of the grayscale image, convert the grayscale image into a binary image based on the average brightness, denoise the binary image, extract the maximum contour of the denoised binary image, and perform logarithmic compression after calculating the spatial moments and hu moments based on the maximum contour to obtain shape moment features; and an edge feature determination unit configured to perform edge detection on the preprocessed web icon to obtain edge features.

[0070] Further, the semantic feature extraction model comprises a convolutional neural network (CNN) and a Transformer hierarchical encoder, and the first branch feature extraction submodule comprises:

[0071] a CNN processing unit configured to input the web icon into the convolutional neural network (CNN) for processing to obtain initial embedding vectors of image blocks;

[0072] a Transformer processing unit configured to input the initial embedding vectors after position encoding into the Transformer hierarchical encoder to obtain semantic features.

[0073] Further, the Transformer hierarchical encoder comprises a plurality of encoding layers, each encoding layer comprises a first sublayer and a second sublayer, the first sublayer comprises a first normalization component, a multi-head attention component and a first residual connection component connected in sequence, the second sublayer comprises a second normalization component, a feedforward network and a second residual connection component connected in sequence, and the Transformer processing unit is configured to input the initial embedding vectors after position encoding into the first sublayer, the first normalization component and the multi-head attention component of the first sublayer are used for normalization and global dependency calculation respectively, and the first residual connection component is used for residual connection, the residual connection result is taken as the input of the second sublayer, the second normalization component and the feedforward network of the second sublayer are used for normalization and nonlinear transformation enhancement respectively to obtain a feedforward output feature, and the second residual connection component is used for connecting the residual connection result and the feedforward output feature to obtain semantic features.

[0074] Further, the icon feature fusion module comprises:

[0075] The first fusion processing submodule is configured to fully connect the color histogram, the ORB feature, the shape moment feature, and the edge feature to obtain a structure feature.

[0076] The second fusion processing submodule is configured to perform dimension reduction on the weighted fusion of the structure feature and the semantic feature to obtain a multi-modal fusion feature.

[0077] Further, the retrieval result determination module comprises:

[0078] The offline result determination submodule is configured to perform similarity calculation on the multi-modal fusion feature and a plurality of suspicious icon features included in the local Faiss cache.

[0079] The online processing submodule is configured to, if the similarity between the multi-modal fusion feature and any suspicious icon feature included in the local Faiss cache is less than a similarity threshold, forward the multi-modal fusion feature to the Milvus cluster to perform similarity calculation on the multi-modal fusion feature by the Milvus cluster.

[0080] The online result determination submodule is configured to obtain the similarity calculation result fed back from the Milvus cluster, and determine an icon pointed to by an icon feature with a similarity greater than a preset similarity threshold in the similarity calculation result fed back by the Milvus cluster as a similar icon, and perform retrieval according to the domain name corresponding to the similar icon.

[0081] Further, the online processing submodule further comprises, before forwarding the multi-modal fusion feature to the Milvus cluster:

[0082] The domain name type determination unit is configured to determine the domain name type corresponding to the webpage icon.

[0083] The online processing submodule comprises:

[0084] The first offline result determination unit is configured to perform similarity calculation on the multi-modal fusion feature and a plurality of suspicious icon features included in the Milvus database corresponding to the domain name type.

[0085] Further, the online processing submodule further comprises:

[0086] The metadata determination unit is configured to determine the domain name metadata of the webpage icon.

[0087] The second offline result determination unit is configured to forward the multi-modal fusion feature and the domain name metadata to the Milvus cluster to perform similarity calculation on the multi-modal fusion feature after filtering the Milvus cluster according to the domain name metadata.

[0088] The domain name retrieval apparatus based on webpage icon similarity calculation in the embodiment can execute the domain name retrieval method based on webpage icon similarity calculation shown in the embodiment of the application, and the implementation principle is similar, which will not be described here.

[0089] Another embodiment of the application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method.

[0090] Specifically, the processor can be a CPU, a general processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in combination with the disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, DSP and microprocessor combinations, etc.

[0091] Specifically, the processor is connected with the memory through a bus, and the bus can include a channel for transmitting information. The bus can be a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0092] The memory can be a ROM or other types of static storage devices that can store static information and instructions, a RAM or other types of dynamic storage devices that can store information and instructions, an EEPROM, a CD-ROM or other optical disc storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but not limited to this.

[0093] Optionally, the memory is used to store the code of the computer program for executing the scheme of the application, and the processor is used to control the execution. The processor is used to execute the application program code stored in the memory to realize the actions of the device provided in the above embodiments.

[0094] Another embodiment of the application provides a computer readable storage medium storing computer executable instructions for executing the method provided in the above embodiments.

[0095] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0096] As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the methods disclosed above can be embodied in software, firmware, hardware, and / or suitable combinations thereof. Some or all of the physical components can be implemented with software executed by a processor, such as a central processing unit, a digital signal processor, or microprocessor, or can be implemented as hardware, or as an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.

[0097] The above description is that of the preferred embodiments of the present application. Various equivalents substitutions of the techniques described herein can be implemented, both currently known or later developed, without departing from the spirit and scope of the application. Such equivalents substitutions are also intended to be encompassed by the claims of the application.

Claims

1. A domain name retrieval method based on webpage icon similarity calculation, characterized in that, include: When a webpage icon to be retrieved is detected, the graphic features and semantic features of the webpage icon are determined. The graphic features include color histogram, ORB features, shape moment features, and edge features. Multimodal fusion is performed on the graphic features and the semantic features to obtain multimodal fused features; The multimodal fusion feature is compared with a number of pre-stored suspicious icon features for similarity calculation, and suspicious domain names are retrieved based on the suspicious icon features with similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names; this step further includes: comparing the multimodal fusion feature with a number of suspicious icon features included in the local Faiss cache for similarity calculation. If the similarity between the multimodal fusion feature and any suspicious icon feature included in the local Faiss cache is less than the similarity threshold, the multimodal fusion feature is forwarded to the Milvus cluster for similarity calculation. The similarity calculation result fed back from the Milvus cluster is obtained, and the icons pointed to by the icon features with similarity greater than the preset similarity threshold in the similarity calculation result fed back from the Milvus cluster are taken as similar icons, and a search is performed based on the domain name corresponding to the similar icon.

2. The method according to claim 1, characterized in that, Before determining the graphic and semantic features of the webpage icon, the method further includes: The webpage icons are preprocessed and converted into RGB, BGR, and grayscale images; Determine the semantic features of webpage icons, including: The RGB image is input into a pre-trained semantic feature model to obtain the semantic features; Determine the graphic characteristics of webpage icons, including: Perform image channel statistical analysis on the BGR image to obtain the color histogram; Corner detection is performed on the grayscale image using an improved FAST detector. After determining the main direction of the detected corners, the coordinates of a first predetermined number of selected point pairs are oriented and filtered. The variance of the first predetermined number of point pairs after filtering is calculated, and descriptors are generated for the first predetermined number of point pairs in the sorted order to obtain the ORB features. The average brightness of the grayscale image is calculated. The grayscale image is then converted into a binary image based on the average brightness and denoised. The maximum contour of the denoised binary image is extracted, and the spatial moments and Hu moments are calculated based on the maximum contour. Logarithmic compression is then performed to obtain the shape moment feature. Edge detection is performed on the preprocessed webpage icon to obtain the edge features.

3. The method according to claim 2, characterized in that, The semantic feature model includes a convolutional neural network (CNN) and a Transformer hierarchical encoder. The step of inputting the RGB image into the pre-trained semantic feature model to obtain the semantic features includes: The webpage icon is input into the convolutional neural network (CNN) for processing to obtain the initial embedding vector for image blocks; The initial embedding vector is positionally encoded and then input into the Transformer hierarchical encoder to obtain semantic features.

4. The method according to claim 3, characterized in that, The Transformer hierarchical encoder includes multiple encoding layers, each comprising a first sub-layer and a second sub-layer. The first sub-layer includes a first normalization component, a multi-head attention component, and a first residual connection component connected in sequence. The second sub-layer includes a second normalization component, a feedforward network, and a second residual connection component connected in sequence. The initial embedding vector is positionally encoded and then input into the Transformer hierarchical encoder to obtain semantic features, including: The initial embedding vector after position encoding is input to the first sub-layer. The first sub-layer performs normalization and global dependency calculation using the first normalization component and the multi-head attention component, respectively, and then performs residual connection using the first residual connection component. The residual connection result is used as the input to the second sub-layer. The second sub-layer performs normalization and nonlinear transformation enhancement using the second normalization component and the feedforward network, respectively, and then obtains feedforward output features. Finally, the second residual connection component is used to connect the residual connection result and the feedforward output features to obtain the semantic features.

5. The method according to claim 1, characterized in that, The process of multimodal fusion of the graphic features and the semantic features to obtain multimodal fused features includes: A full connection is performed on the color histogram, the ORB feature, the shape moment feature, and the edge feature to obtain the structural feature; The structural features and semantic features are weighted and fused, and then dimensionality is reduced to obtain the multimodal fusion features.

6. The method according to claim 1, characterized in that, Before forwarding the multimodal fused features to the Milvus cluster, the method further includes: Determine the domain name type corresponding to the webpage icon; The step of forwarding the multimodal fusion features to the Milvus cluster includes: The similarity calculation is performed on the multimodal fusion features of the Milvus database corresponding to the domain name type.

7. The method according to claim 1, characterized in that, The step of forwarding the multimodal fusion features to the Milvus cluster further includes: Determine the domain name metadata of the webpage icon; The multimodal fusion feature and the domain name metadata are forwarded to the Milvus cluster so that the Milvus cluster is filtered based on the domain name metadata and then the multimodal fusion feature is used to calculate similarity.

8. A domain name retrieval device based on webpage icon similarity calculation, characterized in that, include: The icon feature determination module is used to determine the graphic and semantic features of the webpage icon when the icon to be retrieved is detected. The graphic features include color histogram, ORB features, shape moment features, and edge features. The icon feature fusion module is used to perform multimodal fusion of the graphic features and the semantic features to obtain multimodal fused features; The retrieval result determination module is used to calculate the similarity between the multimodal fusion features and multiple pre-stored suspicious icon features, and to retrieve suspicious domain names based on suspicious icon features with similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names. The search results determination module includes: The offline result determination submodule is used to calculate the similarity of multiple suspicious icon features included in the local Faiss cache of the multimodal fusion features; The online processing submodule is used to forward the multimodal fusion feature to the Milvus cluster if the similarity between the multimodal fusion feature and any suspicious icon feature included in the local Faiss cache is less than the similarity threshold, so that the Milvus cluster can perform similarity calculation on the multimodal fusion feature. The online result determination submodule is used to obtain the similarity calculation results from the Milvus cluster, and to identify icons whose similarity scores are greater than a preset similarity threshold in the similarity calculation results from the Milvus cluster as similar icons, and to perform a search based on the domain names corresponding to the similar icons.

9. A computer-readable storage medium storing computer-executable instructions for performing the method according to any one of claims 1 to 7.

Images