Domain name retrieval method and apparatus based on webpage icon similarity calculation, and medium

By combining the graphic and semantic features of web page icons for multimodal fusion, the problems of low efficiency and poor robustness in existing web page icon retrieval systems are solved, and accurate identification and efficient matching of Internet asset icons are achieved.

WO2026237363A1PCT designated stage Publication Date: 2026-11-19ICLOUDSHIELD SECURITY TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/143387
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-13
Filing Date
2025-12-18
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing web icon retrieval systems lack an understanding of global semantics, resulting in low retrieval efficiency and difficulty in handling robustness issues related to icon deformation and lighting changes.

Method used

By detecting the graphic and semantic features of web page icons, including color histograms, ORB features, shape moment features, and edge features, multimodal fusion is performed. Similarity is calculated by combining the features of pre-stored suspicious icons, thus achieving accurate identification and efficient matching.

Benefits of technology

It improves the comprehensiveness of attack surface management, the real-time nature of risk monitoring, and the accuracy of vulnerability location, and solves the problems of poor retrieval accuracy and low efficiency caused by the lack of semantic understanding in icon retrieval.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025143387_19112026_PF_FP_ABST
    Figure CN2025143387_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of network security analysis, and in particular relates to a domain name retrieval method and apparatus based on webpage icon similarity calculation, and a medium. The method comprises: when a webpage icon to be subjected to retrieval is detected, determining a graphical feature and semantic feature of the webpage icon, wherein the graphical feature comprises a color histogram, an ORB feature, a shape moment feature and an edge feature; performing multimodal fusion on the graphical feature and the semantic feature, so as to obtain a multimodal-fused feature; and performing similarity calculation on the multimodal-fused feature and each of a plurality of pre-stored suspicious icon features, and on the basis of a suspicious icon feature which has a similarity greater than a preset similarity threshold, performing suspicious domain name retrieval, so as to perform management on the basis of a retrieved suspicious domain name. Therefore, the problems of an icon retrieval system having a poor retrieval accuracy and low efficiency due to the lack of semantic understanding are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Domain Name Retrieval Method, Apparatus, and Medium Based on Webpage Icon Similarity Calculation Technical Field

[0001] This application relates to the field of network security analysis, and in particular to a domain name retrieval method, apparatus and medium based on webpage icon similarity calculation. Background Technology

[0002] With the accelerated pace of digital development, the online assets (domains, servers, IPs, etc.) of enterprises, institutions, and individuals have become primary targets of cyberattacks. Attackers typically use techniques such as domain spoofing (e.g., typosquatting), brand imitation, phishing websites, and malicious redirects to steal user data, commit financial fraud, or spread malware. Therefore, accurately identifying and classifying an enterprise's attack surface assets is crucial for security defense.

[0003] Existing web icon retrieval systems mainly fall into two categories: First, retrieval systems based on traditional image processing and local features. These systems primarily use OpenCV to extract local features such as color histograms, SIFT / SURF, etc., and combine them with inverted indexes (such as Faiss) for similarity matching. However, such systems lack an understanding of global semantics and cannot solve the robustness issues caused by icon deformation and lighting changes. Second, single-model retrieval systems based on deep learning. These systems use CNN (such as ResNet) or ViT models to extract image features and combine them with cosine similarity calculations. However, these systems struggle to capture global structure, have low retrieval efficiency, cannot be optimized for large-scale vector libraries, and have high time complexity. Summary of the Invention

[0004] In view of this, this application provides a domain name retrieval method, apparatus, device and medium based on webpage icon similarity calculation, which solves the problem of low retrieval efficiency caused by the lack of understanding of global semantics in webpage icon retrieval systems.

[0005] According to a first aspect of the embodiments of this application, a domain name retrieval method based on webpage icon similarity calculation is provided, the method comprising:

[0006] When a webpage icon to be retrieved is detected, its graphic and semantic features are determined. The graphic features include color histogram, ORB features, shape moment features, and edge features.

[0007] Multimodal fusion of graphic features and semantic features is performed to obtain multimodal fused features;

[0008] The multimodal fusion features are compared with multiple pre-stored suspicious icon features to calculate their similarity. Suspicious domain names are retrieved based on suspicious icon features with similarity greater than a preset similarity threshold, and then managed according to the retrieved suspicious domain names.

[0009] A second aspect of this application discloses a domain name retrieval device based on webpage icon similarity calculation, the device comprising:

[0010] The icon feature determination module is used to determine the graphic and semantic features of the webpage icon when it is detected. The graphic features include color histogram, ORB features, shape moment features and edge features.

[0011] The icon feature fusion module is used to perform multimodal fusion of graphic features and semantic features to obtain multimodal fused features;

[0012] The retrieval result determination module is used to calculate the similarity between the multimodal fusion features and multiple pre-stored suspicious icon features, and to retrieve suspicious domain names based on the suspicious icon features with similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names.

[0013] A third aspect of this application discloses an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0014] A fourth aspect of this application discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] When the embodiment of this application detects a webpage icon to be retrieved, it determines the graphic features and semantic features of the webpage icon. The graphic features include color histogram, ORB features, shape moment features, and edge features. After fusing the graphic features and semantic features, it calculates the similarity with multiple suspicious icon features. Suspicious domain names are retrieved based on icon features with similarity greater than a preset similarity threshold. The retrieved suspicious domain names are then managed. This method of combining image features and semantic features for domain name retrieval solves the problem of poor retrieval accuracy and low efficiency caused by the lack of semantic understanding in icon retrieval. It can improve the comprehensiveness of attack surface management, the real-time nature of risk monitoring, and the accuracy of vulnerability location by accurately identifying and efficiently matching icon features in Internet assets. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] Figure 1 is a flowchart illustrating a domain name retrieval method based on webpage icon similarity calculation provided in an embodiment of this application;

[0018] Figure 2 is a schematic diagram of the shape moment feature extraction process in a domain name retrieval method based on webpage icon similarity calculation provided in an embodiment of this application;

[0019] Figure 3 is a schematic diagram of the application process of a domain name retrieval method based on webpage icon similarity calculation according to an embodiment of this application;

[0020] Figure 4 is a schematic diagram of image feature processing in the application process of a domain name retrieval method based on webpage icon similarity calculation provided in an embodiment of this application;

[0021] Figure 5 is a schematic diagram of the structure of a domain name retrieval device based on webpage icon similarity calculation provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] According to one embodiment of this application, a domain name retrieval method based on webpage icon similarity calculation is provided, as shown in Figure 1. The method includes steps S101, S102 and S103.

[0026] Step S101: When a webpage icon to be retrieved is detected, determine the graphic features and semantic features of the webpage icon. The graphic features include color histogram, ORB features, shape moment features, and edge features.

[0027] Specifically, the search platform can provide various interfaces for obtaining webpage icons. For example, it can obtain the webpage icon uploaded by the user through a preset image upload interface; or it can provide a domain name editing interface, through which the search platform obtains the domain name entered by the user, loads the website content, outputs candidate icons based on the loading results, and determines the webpage icon to be searched when it detects the user's selection of the candidate icon.

[0028] Specifically, when the retrieval platform receives a webpage icon to be retrieved, it can call a pre-configured feature extraction algorithm to extract features from the webpage icon, or it can send the webpage icon to a third party to utilize the tools provided by the third party to obtain the image and semantic features of the webpage icon. This embodiment utilizes the highly optimized open-source library OpenCV and a semantic feature encoder for computer vision applications to analyze the visual content of the webpage icon and extract key information for similar icon recognition.

[0029] Step S102: Perform multimodal fusion on the graphic features and semantic features to obtain multimodal fused features.

[0030] Specifically, the fusion of graphic features and semantic features is achieved by combining them.

[0031] Step S103: Calculate the similarity between the multimodal fusion features and the pre-stored multiple suspicious icon features, and search for suspicious domain names based on the suspicious icon features with similarity greater than the preset similarity threshold, so as to manage the suspicious domain names found.

[0032] Specifically, the similarity threshold can be preset or set after the similarity calculation results are obtained. When the similarity threshold can be preset, multiple levels of search precision can be provided in advance, such as high similarity search, relatively high similarity search, and general similarity search. The similarity threshold for high similarity search can be set to 90%, meaning icons with a similarity greater than 90% are considered similar icons. Alternatively, the similarity threshold can be determined based on parameters pre-input by the user. When setting the similarity threshold after the similarity calculation results are obtained, several similarity threshold options can be provided in advance, and the similarity calculation results can be updated based on the user's selections after the similarity calculation results are obtained.

[0033] Specifically, suspicious domains retrieved can be used as management objects to perform monitoring and other management operations. Alternatively, the suspicious domains retrieved can be visualized as a list for users to select, and the management object can be determined by detecting the user's selection of suspicious domains in the list.

[0034] When the embodiment of this application detects a webpage icon to be retrieved, it determines the graphic features and semantic features of the webpage icon. The graphic features include color histogram, ORB features, shape moment features, and edge features. After fusing the graphic features and semantic features, it calculates the similarity with multiple suspicious icon features. Suspicious domain names are retrieved based on icon features with similarity greater than a preset similarity threshold. The retrieved suspicious domain names are then managed. This method of combining image features and semantic features for domain name retrieval solves the problem of poor retrieval accuracy and low efficiency caused by the lack of semantic understanding in icon retrieval. It can improve the comprehensiveness of attack surface management, the real-time nature of risk monitoring, and the accuracy of vulnerability location by accurately identifying and efficiently matching icon features in Internet assets.

[0035] In some implementations, before step S101, the method further includes: step S1011 (not shown in the figure): preprocessing the webpage icon and converting it into an RGB image, a BGR image, and a grayscale image; determining the semantic features of the webpage icon in step S101 further includes: step S1012 (not shown in the figure): inputting the RGB image into a pre-trained semantic feature model to obtain semantic features; determining the graphic features of the webpage icon in step S101 further includes: step S1013 (not shown in the figure): performing image channel statistical analysis on the BGR image to obtain a color histogram; step S1014 (not shown in the figure): performing angular analysis on the grayscale image using an improved FAST detector. Point detection: After determining the main direction of the detected corner points, the coordinates of the selected first predetermined number of point pairs are aligned by direction and then filtered. The variance of the first predetermined number of point pairs after filtering is calculated, and descriptors are generated for the first predetermined number of point pairs in the sorting, to obtain ORB features; Step S1015 (not shown in the figure): The average brightness of the grayscale image is calculated. The grayscale image is converted into a binary image based on the average brightness and then denoised. The maximum contour of the denoised binary image is extracted, and the spatial moment and Hu moment are calculated based on the maximum contour and then logarithmically compressed to obtain shape moment features; Step S1016 (not shown in the figure): Edge detection is performed on the preprocessed web page icon to obtain edge features.

[0036] Specifically, a detector can be created based on the ORB algorithm in OpenCV, and the detectAndComputer method can be used to detect key points and calculate feature descriptors to complete the extraction of ORB features in step S101.

[0037] In ORB feature extraction, orientation compensation determines the principal orientation by calculating the image moments of the keypoint's neighborhood (i.e., a predetermined surrounding area, taking a 31×31 pixel window as an example). The specific steps include centroid calculation and principal orientation calculation. The formula for centroid calculation is: and Where I(x,y) is the intensity (grayscale value) of pixel (x,y), the numerators in the two formulas are the weighted sum of the coordinates, and the denominator is the total intensity (used for normalization). The principal direction is calculated using the principal direction angle. Calculated from the centroid offset, where the principal direction angle is... The formula is: Main direction angle The principal direction of the gray-level gradient in the local region is used to rotate the coordinates of the BRIEF points, making the descriptor rotation invariant. In this embodiment, the ORB uses the FAST focus detection method. Compared with other corner detection methods, FAST focus detection has controllable parameters. The maximum number of feature points detected by ORB can be limited by setting nfeatures, avoiding overfitting in textured regions. At the same time, the corner points detected by FAST can be directly input into the improved BRIEF to generate fixed-length descriptors, and have better compatibility with methods used for descriptor binary compression (such as mean compression).

[0038] Specifically, shape moment features can be extracted as shown in Figure 2. First, the webpage icon is preprocessed using OpenCV, such as converting it to the target size (e.g., 224*224) to obtain a grayscale image before starting the shape moment feature extraction process. First, a dynamic binarization function, such as `adaptive_binarization`, can be called to calculate the average brightness. Different binarization strategies can be selected based on different average brightness levels (e.g., mean < 50 or mean > 200). For example, if the average threshold is less than 50, dark background thresholding is applied; if the average threshold is greater than 200, bright background thresholding is applied; otherwise, OTSU automatic thresholding is applied. Dark images (low mean) are set with lower thresholds to preserve details, while bright images (high mean) are set with higher thresholds to avoid overexposure. OTSU automatic thresholding is suitable for high-contrast images. In application, if the average brightness is less than 50, assuming a dark background threshold of 20, pixels with a brightness higher than 20 are set to 255 (white), and those lower are set to 0 (black). The OTSU algorithm automatically determines the optimal segmentation threshold by maximizing the inter-class variance between the foreground and background, making it suitable for images with bimodal histograms. After binarization, a binary image is returned. Then, morphological opening is used to denoise and detect the existence of contours. If a contour is found, the largest contour is selected and its spatial moments and Hu moments are calculated before logarithmic compression is performed to enhance the details in low-brightness areas and suppress excessive changes in high-brightness areas, thereby improving the stability of image features.

[0039] In the above implementation, the semantic feature extraction model includes a convolutional neural network (CNN) and a Transformer hierarchical encoder. Step S1012 further includes: inputting the web page icon into the CNN for processing to obtain the initial embedding vector of the image block; and inputting the initial embedding vector into the Transformer hierarchical encoder after position encoding to obtain semantic features.

[0040] Specifically, an image of a predetermined size can be input into a Convolutional Neural Network (CNN). First, the webpage icon is converted to a predetermined size (e.g., 224×224) and then input into the CNN. The CNN's convolutional kernels, such as 16×16, divide the icon into 14×14=196 patches, each patch being 16×16×3 in size and containing RGB channels. Next, the patches are embedded with an embedding dimension of 768. Each patch is mapped to a 768-dimensional vector. That is, after converting the 224×224 image into a 14×14×768 feature map, it is flattened and stitched together, flattening the 14×14×768 feature map into a 196×768 sequence, i.e., 196 patches, each corresponding to a 768-dimensional vector. Each patch is described by a 768-dimensional vector, providing semantic information and local details. Positional encoding provides features of the entire layout, laying the foundation for subsequent global analysis by the Transformer hierarchical encoder.

[0041] In the above embodiments, the Transformer hierarchical encoder includes multiple encoding layers, each including a first sub-layer and a second sub-layer. The first sub-layer includes a first normalization component, a multi-head attention component, and a first residual connection component connected in sequence. The second sub-layer includes a second normalization component, a feedforward network, and a second residual connection component connected in sequence. After positional encoding of the initial embedding vector, it is input into the Transformer hierarchical encoder to obtain semantic features. This process includes: inputting the positionally encoded initial embedding vector into the first sub-layer; the first sub-layer performs normalization and global dependency calculation using the first normalization component and the multi-head attention component, respectively; then performing residual connection using the first residual connection component; using the residual connection result as the input of the second sub-layer; the second sub-layer performs normalization and nonlinear transformation enhancement using the second normalization component and the feedforward network, respectively, and obtains feedforward output features; finally, the second residual connection component connects the residual connection result and the feedforward output features to obtain semantic features.

[0042] Specifically, the input image size of the Transformer hierarchical encoder is typically 224x224. The Transformer encoder uses 12 Transformer coding layers, each containing a multi-head self-attention (MHSA) network and a feedforward network (FFN). The last layer outputs a 768-dimensional vector corresponding to the class token as global semantic features.

[0043] In some implementations, step S102 further includes:

[0044] Fully connect the color histogram, ORB features, shape moment features, and edge features to obtain structural features;

[0045] After weighted fusion of structural and semantic features, dimensionality reduction is performed to obtain multimodal fusion features.

[0046] Specifically, the weights of structural and semantic features can be adjusted based on external inputs (threat intelligence, user configuration, etc.). In application, these weights can be determined using a predefined weight adjustment rule base as shown in Table 1. For example, if image blurring is detected (decreased reliability of structural features), the weight of structural features is reduced; if text label confidence is high (more reliable semantic features), the weight of semantic features is increased.

[0047] Table 1 - Correspondence between Weight Adjustment and its Triggering Conditions

[0048]

[0049] Specifically, after preprocessing the webpage icon (such as adjusting image size), a preset image sharpness evaluation algorithm, such as the Laplacian gradient method, can be used to score the image sharpness of the webpage icon. In application, if the image sharpness scoring triggers a condition—that is, the image sharpness score is less than the image sharpness threshold—then the structural features... and semantic features Adjust them to 0.3 and 0.7 respectively; otherwise, use the system's initial default weights and structural features. and semantic features The default weights are all 0.5. Specifically, after preprocessing the webpage icon (e.g., adjusting image size), the semantic feature confidence score of the webpage icon is calculated. The semantic feature confidence score is used to measure the certainty of the model's understanding of the semantic content of the image. This application's embodiment uses the ViT semantic feature extraction model to output global semantic features. Its confidence score is calculated based on the probability distribution of the classification layer, using the probability entropy method (normalized confidence score). Information entropy measures that the more concentrated the probability distribution, the lower the entropy value and the higher the confidence score. For example, if the semantic feature confidence score is greater than the semantic feature confidence score threshold, it is increased by 0.2 based on the default weight. This increase can improve the understanding of structural features. and semantic features Perform normalization, or simultaneously reduce the weight of structural features.

[0050] In some embodiments, step S103 further includes:

[0051] The similarity of the multimodal fusion features included in the local Faiss cache is calculated for multiple suspicious icon features;

[0052] If the similarity between the multimodal fusion feature and any suspicious icon feature in the local Faiss cache is less than the similarity threshold, the multimodal fusion feature is forwarded to the Milvus cluster for similarity calculation.

[0053] Obtain the similarity calculation results from the Milvus cluster, and take the icons whose similarity is greater than the preset similarity threshold in the similarity calculation results from the Milvus cluster as similar icons, and search according to the domain name corresponding to the similar icons.

[0054] Specifically, the local Faiss cache can be adjusted based on pre-configured strategies. For example, the local Faiss cache can be for icon features that are matched within a predetermined time period (such as the most recent hour).

[0055] In this embodiment, the local Faiss cache stores the icon features of recent high-frequency queries, which can achieve millisecond-level response and reduce the dependence on the Milvus cluster. The Milvus cluster has the function of high-concurrency real-time retrieval, which can meet large-scale real-time requests and perform domain name icon similarity calculation across the entire network.

[0056] This application achieves a collaborative division of labor between online and offline services through local Faiss caching and Milvus clusters, effectively balancing high-concurrency real-time retrieval with low-latency local caching.

[0057] In some embodiments, before forwarding the multimodal fusion feature to the Milvus cluster, the method further includes: determining the domain name type corresponding to the webpage icon; forwarding the multimodal fusion feature to the Milvus cluster includes: calculating the similarity of the multimodal fusion feature with multiple suspicious icon features included in the Milvus database corresponding to the domain name type.

[0058] Specifically, the domain name type can be determined based on the domain name suffix. For example, if the domain name is "bank.com", when the local Faiss cache query fails, the data stored on the node with the domain name type "bank.com" in the Milvus cluster will be used to calculate similarity, thereby speeding up the icon retrieval.

[0059] In some embodiments, forwarding the multimodal fusion feature to the Milvus cluster further includes: determining the domain name metadata of the webpage icon; forwarding the multimodal fusion feature and the domain name metadata to the Milvus cluster, so as to perform similarity calculation on the multimodal fusion feature after filtering the Milvus cluster based on the domain name metadata.

[0060] Specifically, the mapping relationship between the feature vector database and the metadata database can be established, such as by establishing a mapping relationship between the domain_name field and the feature vector. In this way, during the search, vector similarity search can be performed first, and then the results can be filtered based on the metadata. The vector and metadata can be combined to achieve the purpose of accurate search.

[0061] The following description, in conjunction with Figures 3 and 4, illustrates this application. For example, in a counterfeit asset identification scenario, the input is a potentially counterfeited webpage icon. After OpenCV preprocessing (e.g., image normalization, grayscale conversion, noise removal, geometric correction), dual-path features are extracted. Specifically, ViT global embedding features are extracted via the Transformer semantic branch, and color histogram, ORB features, shape moment features, and edge features are extracted via the OpenCV local feature branch. The color histogram employs a joint RGB three-channel quantization strategy, calculating a three-dimensional histogram using cv2.calcHist. Each channel is quantized into eight intervals (total dimensions 8×8×8=512), and normalization is applied to eliminate illumination differences. The ORB feature uses ORB (Oriented FAST and Rotated BRIEF), generating a 32-bit binary vector through FAST corner detection and the rBRIEF descriptor. Compared to traditional ORB feature extraction (maximum feature count of 500, generating raw descriptors), this application limits the maximum number of feature points detected by ORB to nfeatures=50 and combines mean compression to compress the descriptor into a 32-dimensional fixed-length vector. In terms of single-frame processing time, memory usage, and matching accuracy, the system performance is superior. The Hu shape moment feature can be obtained by preprocessing the image to grayscale (cv2.COLOR_BGR2GRAY), then calculating seven Hu invariant moments based on the image grayscale moments, outputting a 7-dimensional vector that directly represents the global shape features. These features are invariant to translation, rotation, and scaling, and redundant information is eliminated through nonlinear combination of central moments. As shown in Figure 4, taking a 224*224*3 input image as an example, color features are extracted as 512-dimensional RGB images, shape features as 7-dimensional Hu invariant moments, texture features as 32-dimensional ORB features, and edge features as 64-dimensional edge features. These features are then fused to obtain a 615-dimensional structural vector feature. The structural vector features and semantic vectors are horizontally stacked using np.hstack to obtain a composite vector, preserving the complementary information of the two modalities.

[0062] One embodiment of this application provides a domain name retrieval device based on webpage icon similarity calculation, as shown in FIG5. The device 50 includes: an icon feature determination module 501, an icon feature fusion module 502, and a retrieval result determination module 503.

[0063] The icon feature determination module 501 is used to determine the graphic features and semantic features of the webpage icon when the icon to be retrieved is detected. The graphic features include color histogram, ORB features, shape moment features and edge features.

[0064] Icon feature fusion module 502 is used to perform multimodal fusion of graphic features and semantic features to obtain multimodal fused features;

[0065] The retrieval result determination module 503 is used to calculate the similarity between the multimodal fusion features and multiple pre-stored suspicious icon features, and to retrieve suspicious domain names based on the suspicious icon features with similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names.

[0066] When the embodiment of this application detects a webpage icon to be retrieved, it determines the graphic features and semantic features of the webpage icon. The graphic features include color histogram, ORB features, shape moment features, and edge features. After fusing the graphic features and semantic features, it calculates the similarity with multiple suspicious icon features. Suspicious domain names are retrieved based on icon features with similarity greater than a preset similarity threshold. The retrieved suspicious domain names are then managed. This method of combining image features and semantic features for domain name retrieval solves the problem of poor retrieval accuracy and low efficiency caused by the lack of semantic understanding in icon retrieval. It can improve the comprehensiveness of attack surface management, the real-time nature of risk monitoring, and the accuracy of vulnerability location by accurately identifying and efficiently matching icon features in Internet assets.

[0067] Furthermore, before determining the graphic and semantic features of the webpage icon, the icon feature determination module also includes:

[0068] The image preprocessing submodule is used to preprocess web page icons and convert them into RGB images, BGR images, and grayscale images;

[0069] The icon feature determination module includes: a first branch feature extraction submodule and a second branch feature extraction submodule. The first branch feature extraction submodule inputs the RGB image into a pre-trained semantic feature model to obtain semantic features. The second branch feature extraction submodule includes: a color histogram determination unit, used to perform image channel statistical analysis on the RGB image to obtain a color histogram; an ORB feature determination unit, used to perform corner detection on the grayscale image using an improved FAST detector, determine the main direction of the detected corners, perform direction alignment and filtering on the selected first predetermined number of point pairs, calculate the variance of the first predetermined number of point pairs after filtering, and generate descriptors for the first predetermined number of point pairs with the highest ranking, thus obtaining ORB features; a shape moment feature determination unit, used to calculate the average brightness of the grayscale image, convert the grayscale image to a binary image based on the average brightness, perform denoising, extract the maximum contour of the denoised binary image, and perform spatial moment and Hu moment calculations based on the maximum contour, followed by logarithmic compression, to obtain shape moment features; and an edge feature determination unit, used to perform edge detection on the preprocessed webpage icon to obtain edge features.

[0070] Furthermore, the semantic feature extraction model includes a convolutional neural network (CNN) and a Transformer hierarchical encoder. The first branch feature extraction submodule includes:

[0071] The CNN processing unit is used to input web page icons into the convolutional neural network (CNN) for processing to obtain the initial embedding vectors of image blocks.

[0072] The Transformer processing unit is used to perform positional encoding on the initial embedding vector and then input it into the Transformer hierarchical encoder to obtain semantic features.

[0073] Furthermore, the Transformer hierarchical encoder includes multiple encoding layers, each comprising a first sub-layer and a second sub-layer. The first sub-layer includes a first normalization component, a multi-head attention component, and a first residual connection component connected in sequence. The second sub-layer includes a second normalization component, a feedforward network, and a second residual connection component connected in sequence. The Transformer processing unit is used to input the position-encoded initial embedding vector into the first sub-layer. The first sub-layer uses the first normalization component and the multi-head attention component to perform normalization and global dependency calculation, respectively, and then uses the first residual connection component to perform residual connection. The residual connection result is used as the input to the second sub-layer. The second sub-layer uses the second normalization component and the feedforward network to perform normalization and nonlinear transformation enhancement, respectively, and then uses the second residual connection component to connect the residual connection result and the feedforward output feature to obtain semantic features.

[0074] Furthermore, the icon feature fusion module includes:

[0075] The first fusion processing submodule is used to perform full connection on the color histogram, ORB features, shape moment features and edge features to obtain structural features;

[0076] The second fusion processing submodule is used to perform dimensionality reduction after weighted fusion of structural features and semantic features to obtain multimodal fusion features.

[0077] Furthermore, the search result determination module includes:

[0078] The offline result determination submodule is used to calculate the similarity of multiple suspicious icon features included in the local Faiss cache of the multimodal fusion features;

[0079] The online processing submodule is used to forward the multimodal fusion feature to the Milvus cluster if the similarity between the multimodal fusion feature and any suspicious icon feature included in the local Faiss cache is less than the similarity threshold, so that the Milvus cluster can perform similarity calculation on the multimodal fusion feature.

[0080] The online result determination submodule is used to obtain the similarity calculation results from the Milvus cluster, and to identify icons whose similarity scores are greater than a preset similarity threshold in the similarity calculation results from the Milvus cluster as similar icons, and to perform a search based on the domain names corresponding to the similar icons.

[0081] Furthermore, before forwarding the multimodal fusion features to the Milvus cluster, the online processing submodule also includes:

[0082] Domain type determination unit, used to determine the domain type corresponding to the webpage icon;

[0083] The online processing submodule includes:

[0084] The first offline result determination unit is used to calculate the similarity of the multimodal fusion features with multiple suspicious icon features included in the Milvus database corresponding to the domain name type.

[0085] Furthermore, the online processing submodule also includes:

[0086] Metadata determination unit, used to determine the domain name metadata of the webpage icon;

[0087] The second offline result determination unit is used to forward the multimodal fusion features and domain name metadata to the Milvus cluster, so as to perform similarity calculation on the multimodal fusion features after filtering the Milvus cluster based on the domain name metadata.

[0088] The domain name retrieval device based on webpage icon similarity calculation in this embodiment can execute the domain name retrieval method based on webpage icon similarity calculation shown in the embodiment of this application. The implementation principle is similar, and will not be described again here.

[0089] Another embodiment of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0090] Specifically, the processor can be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0091] Specifically, the processor connects to the memory via a bus, which may include a path for transmitting information. The bus can be a PCI bus or an EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc.

[0092] The memory may be ROM or other types of static storage devices that can store static information and instructions, RAM or other types of dynamic storage devices that can store information and instructions, or EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0093] Optionally, the memory stores the code of a computer program that executes the scheme of this application, and the execution is controlled by a processor. The processor executes the application code stored in the memory to implement the operation of the apparatus provided in the above embodiments.

[0094] Another embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for performing the methods provided in the above embodiments.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0097] The above is a detailed description of the preferred embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A domain name search method based on web icon similarity calculation, characterized in that, include: When a webpage icon to be retrieved is detected, the graphic features and semantic features of the webpage icon are determined. The graphic features include color histogram, ORB features, shape moment features, and edge features. Multimodal fusion is performed on the graphic features and the semantic features to obtain multimodal fused features; The multimodal fusion features are compared with a number of pre-stored suspicious icon features for similarity calculation. Suspicious domain names are retrieved based on suspicious icon features with similarity greater than a preset similarity threshold, and then managed based on the retrieved suspicious domain names.

2. The method according to claim 1, characterized in that, Before determining the graphic and semantic features of the webpage icon, the method further includes: The webpage icons are preprocessed and converted into RGB, BGR, and grayscale images; Determine the semantic features of webpage icons, including: The RGB image is input into a pre-trained semantic feature model to obtain the semantic features; Determine the graphic characteristics of webpage icons, including: Perform image channel statistical analysis on the BGR image to obtain the color histogram; Corner detection is performed on the grayscale image using an improved FAST detector. After determining the main direction of the detected corners, the coordinates of a first predetermined number of selected point pairs are oriented and filtered. The variance of the first predetermined number of point pairs after filtering is calculated, and descriptors are generated for the first predetermined number of point pairs in the sorted order to obtain the ORB features. The average brightness of the grayscale image is calculated. The grayscale image is then converted into a binary image based on the average brightness and denoised. The maximum contour of the denoised binary image is extracted, and the spatial moments and Hu moments are calculated based on the maximum contour. Logarithmic compression is then performed to obtain the shape moment feature. Edge detection is performed on the preprocessed webpage icon to obtain the edge features.

3. The method of claim 2, wherein, The semantic feature extraction model includes a convolutional neural network (CNN) and a Transformer hierarchical encoder. The step of inputting the RGB image into the pre-trained semantic feature model to obtain the semantic features includes: The webpage icon is input into the convolutional neural network (CNN) for processing to obtain the initial embedding vector for image blocks; The initial embedding vector is positionally encoded and then input into the Transformer hierarchical encoder to obtain semantic features.

4. The method of claim 3, wherein, The Transformer hierarchical encoder includes multiple encoding layers, each comprising a first sub-layer and a second sub-layer. The first sub-layer includes a first normalization component, a multi-head attention component, and a first residual connection component connected in sequence. The second sub-layer includes a second normalization component, a feedforward network, and a second residual connection component connected in sequence. The initial embedding vector is positionally encoded and then input into the Transformer hierarchical encoder to obtain semantic features, including: The initial embedding vector after position encoding is input to the first sub-layer. The first sub-layer performs normalization and global dependency calculation using the first normalization component and the multi-head attention component, respectively, and then performs residual connection using the first residual connection component. The residual connection result is used as the input to the second sub-layer. The second sub-layer performs normalization and nonlinear transformation enhancement using the second normalization component and the feedforward network, respectively, and then obtains feedforward output features. Finally, the second residual connection component is used to connect the residual connection result and the feedforward output features to obtain the semantic features.

5. The method of claim 1, wherein, The process of multimodal fusion of the graphic features and the semantic features to obtain multimodal fused features includes: A full connection is performed on the color histogram, the ORB feature, the shape moment feature, and the edge feature to obtain the structural feature; The structural features and semantic features are weighted and fused, and then dimensionality is reduced to obtain the multimodal fusion features.

6. The method of claim 1, wherein, The step of calculating the similarity between the multimodal fusion features and a plurality of pre-stored suspicious icon features, and performing suspicious domain name retrieval based on suspicious icon features with similarity greater than a preset similarity threshold, includes: The similarity of the multimodal fusion features included in the local Faiss cache is calculated for multiple suspicious icon features; If the similarity between the multimodal fusion feature and any suspicious icon feature included in the local Faiss cache is less than the similarity threshold, the multimodal fusion feature is forwarded to the Milvus cluster so that the Milvus cluster can perform similarity calculation on the multimodal fusion feature. Obtain the similarity calculation results from the Milvus cluster, and take the icons whose similarity is greater than a preset similarity threshold in the similarity calculation results from the Milvus cluster as similar icons, and perform a search based on the domain name corresponding to the similar icons.

7. The method according to claim 6, characterized in that, Before forwarding the multimodal fused features to the Milvus cluster, the method further includes: Determine the domain type corresponding to the webpage icon; The step of forwarding the multimodal fusion features to the Milvus cluster includes: The similarity calculation is performed on the multimodal fusion features of the Milvus database corresponding to the domain name type.

8. The method of claim 6, wherein, The step of forwarding the multimodal fusion features to the Milvus cluster further includes: Determine the domain name metadata of the webpage icon; The multimodal fusion feature and the domain name metadata are forwarded to the Milvus cluster so that the Milvus cluster is filtered based on the domain name metadata and then the multimodal fusion feature is used to calculate similarity.

9. A domain name search device based on similarity calculation of web page icons, characterized by, include: The icon feature determination module is used to determine the graphic and semantic features of the webpage icon when the icon to be retrieved is detected. The graphic features include color histogram, ORB features, shape moment features, and edge features. The icon feature fusion module is used to perform multimodal fusion of the graphic features and the semantic features to obtain multimodal fused features; The retrieval result determination module is used to calculate the similarity between the multimodal fusion features and multiple pre-stored suspicious icon features, and to retrieve suspicious domain names based on suspicious icon features with similarity greater than a preset similarity threshold, so as to manage the retrieved suspicious domain names.

10. A computer-readable storage medium storing computer-executable instructions for performing the method according to any one of claims 1 to 8.