Website identification method and device, electronic equipment and storage medium

By combining text, image, and metadata information in a multimodal recognition method, and using pre-trained models and meta-learners for dynamic weight adjustment, the problem of low accuracy and insufficient adaptability in website recognition in existing technologies is solved, achieving higher recognition accuracy and robustness.

CN120849740APending Publication Date: 2025-10-28CHINA UNITED NETWORK COMM GRP CO LTD +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510947265.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-09
Publication Date
2025-10-28

AI Technical Summary

Technical Problem

Existing technologies for website recognition suffer from problems such as low recognition accuracy, incomplete modal coverage, inefficient cross-modal fusion, insufficient image analysis depth, and insufficient dynamic adaptability, resulting in a high false positive rate and difficulty in adapting to the rapidly changing Internet environment.

Method used

By acquiring textual, image, and metadata information, multimodal information fusion is performed using pre-trained models and meta-learners. A voting mechanism and dynamic weight adjustment are adopted to improve the accuracy and robustness of website type recognition.

Benefits of technology

It enhances the accuracy and reliability of website type identification, adapts to the rapidly changing internet environment, reduces misjudgments, and improves the robustness and generalization ability of identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120849740A_ABST
    Figure CN120849740A_ABST
Patent Text Reader

Abstract

The invention provides a website identification method and device, electronic equipment and a storage medium, relates to the technical field of computers, and can solve the problem of low identification accuracy. The method comprises the following steps: obtaining text information, image information and metadata information of a target website, processing the text information, the image information and the metadata information of the target website through a pre-trained model to obtain first probability distribution, second probability distribution and website risk probability, and obtaining the risk probability of the target website through a meta-learner. And obtaining the website type of the target website. According to the method and the device, the website type of the target website can be obtained in combination with the information of multiple modalities, the problem of misjudgment caused by incomplete modalities is reduced, the accuracy and the credibility of website type identification are enhanced, and the robustness and the generalization ability of website type identification can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a website identification method, apparatus, electronic device, and storage medium. Background Technology

[0002] In today's era of rapid development of internet technology, accurate identification of website types plays a crucial role in several key areas such as network security protection, anti-fraud operations, and content ecosystem governance.

[0003] Currently, website type identification is achieved by extracting features from the website's text, images, and internet content provider (ICP) information. However, this method suffers from low accuracy. Summary of the Invention

[0004] This application provides a website identification method, apparatus, electronic device, and storage medium that can improve the accuracy of website identification.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a website identification method, which includes: acquiring text information, image information, and metadata information of a target website, wherein the metadata information includes ICP filing information and WHOIS domain name query protocol information; processing the text information using a pre-trained text model to obtain a first probability distribution, wherein the first probability distribution is used to indicate the probability that the target website belongs to each website type based on the text information; processing the image information using a pre-trained image model to obtain a second probability distribution, wherein the second probability distribution is used to indicate the probability that the target website belongs to each website type based on the image information; processing the metadata information using a pre-trained data model to obtain a website risk probability, wherein the website risk probability is used to indicate the risk level of the target website; and processing the first probability distribution, the second probability distribution, and the website risk probability using a meta-learner to obtain the website type of the target website.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, a meta-learner processes the first probability distribution, the second probability distribution, and the website risk probability to obtain the website type of the target website. This includes: using the voting mechanism of the meta-learner, weighting and summing the first probability distribution, the second probability distribution, and the website risk probability to obtain a comprehensive score for the target website belonging to each website type; and selecting the website type with the highest comprehensive score as the website type of the target website.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, after processing the image information using a pre-trained image model to obtain a second probability distribution, the method further includes: if the maximum probability in the second probability distribution is less than a preset probability, re-extracting features from the image information to obtain website image features; and processing the website image features using a pre-trained image model to obtain an updated second probability distribution.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, after processing the text information through a pre-trained text model to obtain a first probability distribution, the method further includes: if the number of the largest probabilities in the first probability distribution is greater than 1, based on a preset website type priority, increasing the probability of website types with higher priority, and using the increased probability distribution as the first probability distribution.

[0010] In conjunction with the first aspect mentioned above, in one possible implementation, before processing the first probability distribution, the second probability distribution, and the website risk probability through a meta-learner to obtain the website type of the target website, the method further includes: adjusting the weight corresponding to the website risk probability to a preset weight if the metadata information meets the weight adjustment conditions.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, the metadata information includes the number of times the filing entity has changed, the filing expiration time, the expiration time of the Secure Socket Layer (SSL) certificate, and the registration time; the weight adjustment conditions include: the number of times the filing entity has changed is greater than the number of changes threshold; or, the filing expiration time is less than a first preset time; or, the SSL certificate expiration time is less than a second preset time; or, the registration time is less than a third preset time; or the metadata information is missing.

[0012] Secondly, this application provides a website identification device, comprising: an acquisition unit for acquiring text information, image information, and metadata information of a target website, the metadata information including ICP filing information and WHOIS information; a first processing unit for processing the text information using a pre-trained text model to obtain a first probability distribution, the first probability distribution indicating the probability that the target website, obtained based on the text information, belongs to each website type; a second processing unit for processing the image information using a pre-trained image model to obtain a second probability distribution, the second probability distribution indicating the probability that the target website, obtained based on the image information, belongs to each website type; a third processing unit for processing the metadata information using a pre-trained data model to obtain a website risk probability, the website risk probability indicating the risk level of the target website; and an identification unit for processing the first probability distribution, the second probability distribution, and the website risk probability using a meta-learner to obtain the website type of the target website.

[0013] In conjunction with the second aspect above, in one possible implementation, the identification unit is used to: through the voting mechanism of the meta-learner, perform a weighted summation of the first probability distribution, the second probability distribution, and the website risk probability to obtain a comprehensive score for the target website belonging to each website type; and take the website type with the highest comprehensive score as the website type of the target website.

[0014] In conjunction with the second aspect above, in one possible implementation, the device further includes: a comparison unit, used to re-extract features from the image information to obtain website image features when the maximum probability in the second probability distribution is less than a preset probability; and to process the website image features using a pre-trained image model to obtain an updated second probability distribution.

[0015] In conjunction with the second aspect above, in one possible implementation, the device further includes: a first adjustment unit, used to increase the probability of website types with higher website type priority based on a preset website type priority when the number of the largest probabilities in the first probability distribution is greater than 1, and to use the increased probability distribution as the first probability distribution.

[0016] In conjunction with the second aspect above, in one possible implementation, the device further includes: a second adjustment unit, used to adjust the weight corresponding to the website risk probability to a preset weight when the metadata information meets the weight adjustment conditions.

[0017] In conjunction with the second aspect above, in one possible implementation, the metadata information includes the number of times the filing entity has changed, the filing expiration time, the expiration time of the Secure Sockets Layer (SSL) certificate, and the registration time; the weight adjustment conditions include: the number of times the filing entity has changed is greater than the number of changes threshold; or, the filing expiration time is less than the first preset time; or, the SSL certificate expiration time is less than the second preset time; or, the registration time is less than the third preset time; or the metadata information is missing.

[0018] Thirdly, this application provides an electronic device, including: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the website identification method as described in the first aspect and any possible implementation of the first aspect.

[0019] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the website identification method as described in the first aspect and any possible implementation thereof.

[0020] Fifthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the website identification method as described in the first aspect and any possible implementation thereof.

[0021] In a sixth aspect, this application provides a chip including a processor and a communication interface coupled to the processor. The processor is used to run computer programs or instructions to implement the website identification method as described in the first aspect and any possible implementation thereof.

[0022] Specifically, the chip provided in this application also includes a memory for storing computer programs or instructions.

[0023] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the device, or it may be packaged separately from the processor of the device; this application does not impose any limitation on this.

[0024] In a seventh aspect, this application provides a website identification system, comprising: an electronic device and a terminal, wherein the electronic device is used to perform the website identification method as described in the first aspect and any possible implementation thereof.

[0025] The descriptions of aspects two through seven in this application can be referenced to the detailed description of aspect one; and the beneficial effects of the descriptions of aspects two through seven can be referenced to the analysis of the beneficial effects of aspect one, which will not be repeated here.

[0026] In this application, the name of the aforementioned website identification device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0027] These or other aspects of this application will become more readily apparent in the following description.

[0028] The above-mentioned solution brings at least the following beneficial effects: Based on the above technical solution, the website identification method provided in this application, by acquiring text information, image information, and metadata information, can capture the influence of multiple modalities on website identification at a fine-grained level. Combining information from multiple modalities yields the website type of the target website, reducing misjudgments caused by incomplete modalities and enhancing the accuracy and reliability of website type identification. By using a pre-trained model and a meta-learner to obtain the website type of the target website, it can adapt to the rapidly changing internet environment and provide an accurate website type for the target website, thereby improving the robustness and generalization ability of website type identification. Attached Figure Description

[0029] Figure 1 This application provides an architectural diagram of a website identification system.

[0030] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0031] Figure 3 A flowchart illustrating a website identification method provided in this application embodiment;

[0032] Figure 4 A flowchart illustrating another website identification method provided in this application embodiment;

[0033] Figure 5 A schematic diagram illustrating model pre-training as provided in an embodiment of this application;

[0034] Figure 6 This application provides a schematic diagram of the module division of a website identification system according to an embodiment of the present application.

[0035] Figure 7 This is a schematic diagram of the structure of a website identification device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0038] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0039] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0040] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0041] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0042] In today's era of rapid development in internet technology, accurate website type identification plays a crucial role in several key areas, including cybersecurity protection, anti-fraud efforts, and content ecosystem governance. Traditional identification methods primarily rely on single-modal data, such as text keyword matching or filtering using rule bases. However, these methods have significant limitations: their feature dimensions are relatively singular, leading to a high false positive rate, and they often struggle to cope with complex and ever-changing real-world scenarios, failing to meet practical needs.

[0043] In recent years, multimodal recognition technology has gradually become a research hotspot. By integrating multi-dimensional information such as text, images, and web metadata, multimodal recognition technology aims to improve the accuracy of website type identification. However, although multimodal recognition technology has improved accuracy compared to traditional methods, existing technologies still have significant shortcomings in terms of modality fusion depth, cross-modal correlation analysis capabilities, and generalization ability, requiring further improvement and refinement.

[0044] The following two multimodal recognition methods will be used as examples to illustrate this.

[0045] I. In a website identification method based on multimodal features, malicious websites are identified by fusing webpage text features, user information features, webpage resource information features, and image features.

[0046] Specifically, we first use a short text pre-trained language model (bidirectional encoder representations from transformers, BERT) and an attention mechanism to extract web page text features from the web page text.

[0047] Then, using techniques such as scale-invariant feature transform (SIFT) feature similarity calculation, optical character recognition (OCR) text, QR code recognition, and object detection, image features are extracted.

[0048] Finally, the extracted webpage text features, user information features, webpage resource information features, and image features are simply concatenated, and then the concatenated features are input into the classification model for recognition.

[0049] However, the aforementioned methods underutilize network metadata. During website identification, crucial network metadata such as domain name registration information and WHOIS information are not fully integrated. This metadata provides global support for website type determination; its absence leads to a lack of comprehensiveness and accuracy in website identification results. Furthermore, there is a weakness in cross-modal semantic relationships. In the feature fusion stage, only a simple concatenation method is used, without dynamic weighting or attention mechanisms. This makes it difficult to effectively capture deep relationships between different modalities, thus affecting the accuracy of website identification. In addition, the depth of image analysis is limited. During image analysis, it mainly relies on a predefined harmful image library for similarity matching, failing to combine deep learning models for fine-grained analysis of image content, such as in-depth analysis of scenes, objects, and behaviors within the images.

[0050] II. In a multimodal harmful link identification method based on text and images, multimodal classification training of text and images is used to identify harmful links. The model is trained using massive sample data to improve identification efficiency.

[0051] However, the aforementioned methods primarily focus on determining "harmfulness" and fail to design differentiated feature extraction strategies for different website types. This limits their ability to identify different types of websites, resulting in weak type generalization. Furthermore, the identification process lacks network metadata, failing to incorporate information such as domain registration time and the entity responsible for filing. Since network metadata provides contextual support for the identification results within the business scenario, the absence of this information diminishes the accuracy and reliability of the results.

[0052] Despite combining the above multimodal recognition methods, there are still many shortcomings in the current website recognition.

[0053] (1) Incomplete modal coverage

[0054] Existing methods only integrate textual and image features during website identification, neglecting the characteristics of network metadata. For example, network metadata could include domain registration time, changes in the registration entity, etc. This network metadata is crucial for comprehensively and accurately determining website type. Specifically, current website identification processes do not dynamically correlate WHOIS information with image content, thus failing to identify the impact of historical domain changes on type, resulting in a lack of global credibility in type determination.

[0055] (2) Inefficient cross-modal fusion

[0056] Most existing methods employ feature concatenation or static weighted fusion, without designing dynamic attention mechanisms or cross-modal alignment strategies. This simple fusion approach results in weak semantic connections between different modalities, making it difficult to capture fine-grained mapping relationships between objects and types.

[0057] (3) Insufficient depth of image analysis

[0058] Existing methods for image analysis primarily rely on similarity matching from predefined harmful image libraries or only extract global features from images. This approach fails to combine object detection and scene understanding models for fine-grained analysis of image content. For example, while referring image segmentation (RIS) can improve the alignment between language and images, it lacks type-specific object detection modules, leading to the omission of key visual information and affecting the accurate determination of website types. For instance, specific objects might include unique images or business identifiers found on websites in specific industries.

[0059] (4) Insufficient dynamic adaptability

[0060] In practical applications, the quality of multimodal data may change, such as due to delays in network metadata updates or upgrades in image camouflage techniques. Existing methods lack real-time response mechanisms to changes in multimodal data quality, failing to address these changes in a timely manner. For example, while low-quality data fusion techniques can handle noise and incomplete data, they do not dynamically adjust weights based on network metadata features, leading to lags in recognition in emerging business scenarios.

[0061] (5) Multimodal data quality issues

[0062] In practical applications, multimodal data often faces problems such as noise (e.g., interference from web page text advertisements), imbalance (e.g., text features dominating decision-making), and incompleteness (e.g., missing network metadata). However, existing methods lack corresponding robustness enhancement strategies and cannot effectively address these problems, leading to increased false positive rates and affecting the accuracy and reliability of website type identification.

[0063] Therefore, the website identification method provided in this application, by acquiring text information, image information, and metadata information, can capture the impact of multiple modalities on website identification at a fine-grained level. Combining information from multiple modalities yields the website type of the target website, reducing misjudgments caused by incomplete modalities and enhancing the accuracy and reliability of website type identification. By using a pre-trained model and a meta-learner to obtain the website type of the target website, it can adapt to the rapidly changing internet environment and provide an accurate website type for the target website, thereby improving the robustness and generalization ability of website type identification.

[0064] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0065] Figure 1 This is a schematic diagram of the architecture of a website identification system provided in an embodiment of this application. Figure 1 As shown, the architecture includes: terminal 101 and electronic device 102.

[0066] Terminal 101 can be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, laptop computer, Internet of Things device, edge device, etc. This application embodiment does not limit it.

[0067] In some embodiments, terminal 101 has communication capabilities. For example, terminal 101 can send text information, image information, and metadata information of a target website to electronic device 102.

[0068] It should be noted that the number of terminals 101 can be one or more. This application embodiment does not limit this.

[0069] Electronic device 102 can be a mobile phone, tablet, computer with wireless transceiver capabilities, virtual reality (VR) terminal, augmented reality (AR) terminal, wireless terminal, computer, independent physical server, server cluster consisting of multiple physical servers, or at least one of the following cloud servers providing basic cloud computing services: cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data or artificial intelligence platforms. This application embodiment does not limit this specific application. Of course, electronic device 102 can also include other functions to provide more comprehensive and diversified services.

[0070] In some embodiments, the electronic device 102 has communication capabilities. For example, communication between the electronic device 102 and the terminal 101.

[0071] In other embodiments, the electronic device 102 has processing capabilities. For example, the electronic device 102 can process text information using a pre-trained text model to obtain a first probability distribution. The first probability distribution indicates the probability that the target website, obtained based on the text information, belongs to each website type. As another example, the electronic device 102 can process image information using a pre-trained image model to obtain a second probability distribution, which indicates the probability that the target website, obtained based on the image information, belongs to each website type. As yet another example, the electronic device 102 can process metadata information using a pre-trained data model to obtain a website risk probability, which indicates the risk level of the target website. Furthermore, the electronic device 102 can use a meta-learner to process the first probability distribution, the second probability distribution, and the website risk probability to obtain the website type of the target website.

[0072] The electronic device 102 can be one or more, and this embodiment of the application does not limit this. For ease of understanding, Figure 1 Only one is shown in the image.

[0073] Terminal 101 and electronic device 102 are connected via a communication link. This communication link can be a wired communication link or a wireless communication link, and this embodiment does not limit it.

[0074] When implemented in hardware, electronic devices can provide, for example... Figure 2 The hardware structure shown is, specifically, Figure 2This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Figure 2 As shown, the electronic device includes at least one processor 201, a communication line 202, and at least one communication interface 204, and may also include a memory 203. The processor 201, memory 203, and communication interface 204 are connected via the communication line 202.

[0075] The processor 201 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0076] Communication line 202 is used to transmit information between the aforementioned components.

[0077] The communication interface 204 is used to communicate with other devices or communication networks. It can use any transceiver-like device, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0078] The memory 203 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of including or storing desired program code having the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0079] In one possible design, the memory 203 can exist independently of the processor 201, meaning the memory 203 can be an external memory of the processor 201. In this case, the memory 203 can be connected to the processor 201 via a communication line 202 to store execution instructions or application code, and its execution is controlled by the processor 201 to implement the website identification method provided in the following embodiments of this application. In another possible design, the memory 203 can also be integrated with the processor 201, meaning the memory 203 can be an internal memory of the processor 201. For example, the memory 203 can be a cache, which can be used to temporarily store some data and instruction information.

[0080] As one possible implementation, processor 201 may include one or more CPUs, for example Figure 2 CPU0 and CPU1 in the example. As another possible implementation, the electronic device may include multiple processors, such as... Figure 2 The processors 201 and 207 are included. As another possible implementation, the electronic device may also include an output device 205 and an input device 206.

[0081] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.

[0082] Figure 3 A flowchart illustrating a website identification method provided in this application embodiment, which can be applied to, for example... Figure 2 In the electronic device shown. For example... Figure 3 As shown, this method can be implemented through S301 to S305.

[0083] S301. Obtain text information, image information, and metadata information from the target website.

[0084] The metadata information includes ICP filing information and WHOIS information.

[0085] The target website can be a website of the type to be identified.

[0086] For example, the text information of the target website can be a Hypertext Markup Language (HTML) file.

[0087] For example, the image information of the target website could be a screenshot of the target website's RGB three-channel page. This screenshot could be in portable network graphics (PNG) format.

[0088] For example, ICP filing information includes the organizer, the nature of the organizer, the license number, the ICP filing update time, and the website registration domain name.

[0089] For example, WHOIS information includes the registrant, WHOIS server, Domain Name System (DNS) server, update time, expiration time, etc.

[0090] The metadata information can be stored in a JavaScript object notation (JSON) file. The metadata information can be for a preset time period.

[0091] In one possible implementation, electronic devices can use web crawling technology to obtain text information, image information, and metadata information of a target website based on its URL or domain name.

[0092] S302. The text information is processed through a pre-trained text model to obtain the first probability distribution.

[0093] The first probability distribution is used to indicate the probability that the target website, obtained based on text information, belongs to each website type.

[0094] For example, the pre-trained text model could be a RoBERTa model tuned using PyTorch. For instance, the RoBERTa model might have a learning rate of 1e-5 and a step size (batch size) of 32.

[0095] For example, the pre-trained text model can be an ALBERT model trained in layers. For instance, during the layered training of the ALBERT model, the first six layers can be frozen, and only the last six layers can be trained. In this case, the learning rate in the ALBERT model is 5e-5.

[0096] For another example, the pre-trained text model can be a DeBERTa model that has been hyperparameterized through gradient checkpointing. For instance, in a DeBERTa model, the learning rate is 5e-5 and the step size (batch_size) is 16.

[0097] It should be noted that the pre-trained text model can consist of a single model or multiple models, and this application embodiment does not limit this. When processing text information using the pre-trained text model, multiple models can be input sequentially to obtain the first probability distribution, or multiple models can be input in parallel to obtain the first probability distribution.

[0098] For example, the website type in the first probability distribution can be a pre-agreed website type, or it can be a website type obtained by clustering text information.

[0099] S303. The image information is processed by a pre-trained image model to obtain the second probability distribution.

[0100] The second probability distribution is used to indicate the probability that the target website, obtained based on image information, belongs to each website type.

[0101] For example, the pre-trained image model can be a pre-trained ResNet50 model; or, the pre-trained image model can be a pre-trained EfficientNet-B4 model.

[0102] It should be noted that the pre-trained image model can consist of a single model or multiple models, and this application embodiment does not limit this. When processing image information using the pre-trained text image model, multiple models can be input sequentially to obtain the second probability distribution, or multiple models can be input in parallel to obtain the second probability distribution.

[0103] For example, the website type in the second probability distribution can be a pre-agreed website type, or it can be a website type obtained by clustering image information.

[0104] It should be noted that the website types in the first probability distribution are the same as those in the second probability distribution.

[0105] S304. The metadata information is processed through a pre-trained data model to obtain the website risk probability.

[0106] Among them, website risk probability is used to indicate the level of risk of the target website.

[0107] For example, the pre-trained data model can be a LightGBM model that has been pre-trained and hyperparameter-tuned. For instance, in a LightGBM model, the number of leaves (num_leaves) is 31, the maximum tree depth (max_depth) is 5, and the learning rate (learning_rate) is 0.05.

[0108] It should be noted that there is no strict order between steps S302, S303, and S304. All three steps can be executed simultaneously or sequentially. When the three steps are executed sequentially, there is no strict order of execution. This application does not impose specific limitations on this.

[0109] S305. Using a meta-learner, process the first probability distribution, the second probability distribution, and the website risk probability to obtain the website type of the target website.

[0110] For example, the meta-learner can be an XGBoost classifier. In the XGBoost classifier, the maximum tree depth (max_depth) is 6, the number of base learners (n_estimators) is 100, and the learning rate is 0.05.

[0111] In one possible implementation, the electronic device calculates the first probability distribution, the second probability distribution, and the website risk probability through a voting mechanism of the meta-learner to obtain the website type of the target website.

[0112] For example, the website type could be "malicious website", "financial website", "e-commerce website", etc.

[0113] Based on the aforementioned website identification method, by acquiring textual, image, and metadata information, the influence of multiple modalities on website identification can be captured at a fine-grained level. Combining information from multiple modalities yields the website type of the target website, reducing misjudgments caused by incomplete modalities and enhancing the accuracy and reliability of website type identification. Through pre-trained models and meta-learners, the website type of the target website can be obtained, adapting to the rapidly changing internet environment and providing accurate website types, thus improving the robustness and generalization ability of website type identification.

[0114] The following is about the above. Figure 3 The process of obtaining the website type of the target website in step S305 is described below. In one possible implementation, the process of obtaining the website type of the target website in step S305 can be implemented by steps 11 to 12.

[0115] Step 11: Through the voting mechanism of the meta-learner, the first probability distribution, the second probability distribution, and the website risk probability are weighted and summed to obtain the comprehensive score of the target website belonging to each website type.

[0116] For example, the weights corresponding to the first probability distribution, the second probability distribution, and the website risk probability can be preset fixed values ​​or values ​​calculated by the meta-learner. This application embodiment does not limit this.

[0117] For example, the weight corresponding to the first probability distribution is 0.4, the weight corresponding to the second probability distribution is 0.3, and the weight corresponding to the website risk probability is 0.3.

[0118] Step 12: Select the website type with the highest overall score as the target website type.

[0119] In this way, by using the voting mechanism of the meta-learner, the comprehensive score of the target website belonging to each website type can be obtained, which can reduce human intervention in the website identification process, enhance the generalization ability of website identification, and take the website type corresponding to the highest comprehensive score as the website type of the target website. This avoids judging the website type by simply splicing features, and can improve the accuracy of website identification.

[0120] Following step S303, the method further includes steps 21 to 22.

[0121] Step 21: If the maximum probability in the second probability distribution is less than the preset probability, re-extract features from the image information to obtain website image features.

[0122] Understandably, if the maximum probability in the second probability distribution is less than the preset probability, it means that the probability of belonging to each website type in the second probability distribution is relatively low. This can be interpreted as a low confidence level in classifying the target website into this website type based on image information. Feature extraction needs to be performed again to generate more reliable results, thereby improving the accuracy of identifying the website type of the target website.

[0123] A preset probability is used to determine whether feature extraction should be performed again. For example, the preset probability is 60%.

[0124] Understandably, the overall score for each website type can also be interpreted as the confidence level of that website type.

[0125] Step 22: Process the website image features using a pre-trained image model to obtain an updated second probability distribution.

[0126] It should be noted that after re-extracting features from the image information to obtain an updated second probability distribution, if the maximum probability in the updated second probability distribution is still less than the preset probability, then feature extraction needs to be performed again to generate a second probability distribution, until the maximum probability in the latest generated second probability distribution is greater than or equal to the preset probability.

[0127] Thus, when the maximum probability in the second probability distribution is less than the preset probability, by re-extracting features from the image information to obtain an updated second probability distribution, the problem of ambiguity in the identification of multiple website types due to low confidence can be reduced, the uncertainty of website identification can be reduced, and the accuracy of website identification can be improved.

[0128] Following step S302, the method further includes step 31.

[0129] Step 31: If the number of the largest probabilities in the first probability distribution is greater than 1, based on the preset website type priority, increase the probability of website types with higher priority, and use the increased probability distribution as the first probability distribution.

[0130] For example, in the first probability distribution, the number of cases where the maximum probability is greater than 1 could be that the probability of a website being financial and the probability of a website being e-commerce are the same, and that is the maximum value.

[0131] Understandably, since the first probability distribution includes probabilities corresponding to multiple website types, there will be cases where the number of the largest probabilities in the first probability distribution is greater than 1. That is, there will be cases where multiple probabilities are the same in the first probability distribution and are the maximum values.

[0132] For example, multiple website types are sorted according to their priority to generate a first probability distribution. If the number of the highest probability in the first probability distribution is greater than 1, the website type that ranks first has the highest priority, thus increasing the probability of that website type.

[0133] For example, among multiple website types, each website type corresponds to a priority index, and the priority of the website type is determined according to the priority index. If the number of the highest probabilities in the first probability distribution is greater than 1, the website type with the highest priority index has the highest priority, and the probability of that website type is increased.

[0134] For example, the probability of increasing the probability of a website type with a high priority can be a preset fixed value, or the probability of increasing the probability can be a value calculated from the data. This application embodiment does not limit this.

[0135] In this way, if the number of the largest probabilities in the first probability distribution is greater than 1, the value of the first probability distribution can be adjusted, and the website type of the target website can be obtained based on the adjusted first probability distribution, which can improve the accuracy of website identification.

[0136] Prior to step S305, the method further includes step 41.

[0137] Step 41: If the metadata information meets the weight adjustment conditions, adjust the weight corresponding to the website risk probability to the preset weight.

[0138] For example, the preset weight can be 0.6, 0.7, etc.

[0139] In one possible implementation, the metadata information includes the number of times the filing entity has changed, the filing expiration date, the SSL certificate expiration date, and the registration date. In this case, the weight adjustment conditions include at least one of the following five conditions.

[0140] Scenario 1: The number of changes to the filing entity exceeds the threshold for the number of changes.

[0141] For example, the threshold for the number of changes could be 2.

[0142] Understandably, if the number of changes to the registration entity exceeds the threshold, it indicates that the target website has been changing frequently within a preset time, which may pose a risk to the authenticity or legality of the target website and increase the probability that the target website is a malicious website. By adjusting the weights corresponding to the website risk probability, the website type can be identified more accurately.

[0143] Scenario 2: The expiration date of the filing is less than the first preset time.

[0144] Understandably, if the registration expiration date is less than the first preset time, meaning the target website's registration has expired within the first preset timeframe, then adjusting the weights corresponding to the website's risk probability can more accurately reflect the target website's risk level, thereby improving the accuracy of website identification.

[0145] Scenario 3: The SSL certificate expires less than the second preset time.

[0146] Understandably, if the SSL certificate expires before the second preset time, meaning the target website's SSL certificate has already expired, then adjusting the weights corresponding to the website's risk probability can more accurately reflect the target website's risk level, thereby improving the accuracy of website identification.

[0147] Scenario 4: The registration time is less than the third preset time.

[0148] For example, the registration time can be the domain name registration time.

[0149] For example, the third preset time period could be 3 months.

[0150] Scenario 5: Missing metadata information.

[0151] For example, missing metadata information includes missing ICP filing information and / or missing WHOIS information.

[0152] By clearly defining the conditions for weight adjustment, the weight corresponding to the website's risk probability can be adjusted when one of these five conditions is met.

[0153] In this way, by adjusting the weights corresponding to the website risk probability, the risk level of the target website can be more accurately reflected, reducing the problem of weak correlation caused by static weights, thereby improving the accuracy of website identification. In scenarios where metadata information changes frequently, expires, or is not accurate, adjusting the weights can improve the identification response time.

[0154] Figure 4 A flowchart illustrating another website identification method provided in this application embodiment. (See attached flowchart.) Figure 4 As shown, this method can be implemented through S401 to S406.

[0155] S401. Obtain text information, image information, and metadata information from the target website.

[0156] S402. Perform data preprocessing on the text information, image information and metadata information of the target website to obtain preprocessed text information, preprocessed image information and preprocessed metadata information.

[0157] In one possible implementation, the text information of the target website is cleaned, segmented, and truncated to obtain preprocessed text information.

[0158] For example, text cleaning can be performed by extracting the main text paragraphs using the BeautifulSoup algorithm.

[0159] It should be noted that after obtaining the text information of the target website, the text information can be represented in the form of multiple paragraphs, each paragraph corresponding to a tag. This tag is used to characterize the website type. That is, the tag corresponds to the website type in the first probability distribution. This tag is retained during the extraction of the main text paragraphs using the BeautifulSoup algorithm.

[0160] For example, text segmentation can be performed by filtering or removing non-Chinese characters from the text using regular expressions, while retaining Chinese characters, numbers, punctuation, etc. For instance, non-Chinese characters could include advertising codes, stop words, etc.

[0161] For example, truncating text information can be done by cutting the text information to a preset number of characters. For instance, the preset number of characters could be 512. If the text information is less than the preset number of characters, it is padded with the preset number of characters to complete the cutoff.

[0162] For example, the preprocessed text information can be a UTF-8 encoded text file, where each line represents a piece of data.

[0163] In one possible implementation, the image information of the target website is normalized, target detected, dimensionality reduced, and clustered to obtain preprocessed image information.

[0164] The target website's image information resolution is greater than or equal to a preset resolution. For example, the preset resolution could be 1024*768.

[0165] For example, normalizing image information can be achieved by using bilinear interpolation to adjust the image to 512*512 pixels.

[0166] For example, object detection in an image can be performed by histogram equalization followed by YOLOv11 image detection, and then cropping the image to a preset size. Histogram equalization can be implemented using the CLAHE algorithm, where the contrast limit threshold (clip_limit) can be 2.0 and the tile grid size (tile_grid_size) can be 8*8. Alternatively, histogram equalization can be augmented by adding Gaussian noise to the image to increase robustness. The Gaussian noise has a mean (μ) of 0 and a standard deviation (σ) of 0.1.

[0167] For example, dimensionality reduction of image information can be achieved by extracting features using a ResNet50 model to obtain a 1024-dimensional feature vector, and then using principal component analysis (PCA) to reduce the dimensionality of the feature vector to 128 dimensions. PCA dimensionality reduction preserves 95% of the variance.

[0168] For example, image information can be clustered using the K-means algorithm. For instance, k in the K-means algorithm can be 50, 20, etc.

[0169] It should be noted that after obtaining the image information of the target website, each image corresponds to a label, which is used to characterize the website type. That is, the label corresponds to the website type in the second probability distribution. This label is retained during the clustering process using the K-means algorithm. For example, the label can be in CSV format, and could be something like "Cluster_25: Electronics Products".

[0170] By using supplementary image information for dimensionality reduction and clustering, more fine-grained features can be extracted from images, avoiding the use of fully connected layers for feature extraction. For example, PCA dimensionality reduction reduces image feature dimensions by 87.5%, improves training speed by 3 times, and significantly enhances label interpretability.

[0171] It should be noted that when the maximum probability in the second probability distribution is less than the preset probability, the process of re-extracting features from the image information can be to re-perform PCA dimensionality reduction and clustering on the image information.

[0172] In one possible implementation, missing value filling and time-series encoding are performed on the metadata information of the target website to obtain preprocessed metadata information.

[0173] For example, missing value filling for metadata information can be achieved by using the forward filling method to fill in the missing metadata information with the character "unknown".

[0174] For example, time-series encoding of metadata information can be performed by using the following metadata information within a preset time period: organizer, organizer type, license number, ICP filing update time, website registration domain name, registrant, WHOIS server, DNS server, update time, and expiration time, as preprocessed 10-dimensional metadata information. The preset time can be the time interval between the registration time and today, and the preset time can be 30 days.

[0175] It should be noted that when preprocessing the text information, image information and metadata information of the target website, the text information, image information and metadata information of the target website can be processed simultaneously, or they can be preprocessed in a sequential order. This application embodiment does not limit this.

[0176] S403. The pre-trained text model is used to process the pre-processed text information to obtain the first probability distribution.

[0177] During the pre-training process of a text model, methods such as bagging sampling, PyTorch fine-tuning, and cross-validation can be used to pre-train the text model.

[0178] For example, the bagging sampling process can involve performing five rounds of bagging sampling on the original text data training set to generate three differentiated training sets. The sampling rate is 80%.

[0179] For example, PyTorch fine-tuning can be used to fine-tune parameters such as the learning rate, stride, and number of training layers of a text model.

[0180] For example, cross-validation could be 5-fold cross-validation.

[0181] For example, a pre-trained text model can consist of a hyperparameter-tuned RoBERTa model, an ALBERT model, and a DeBERTa model.

[0182] For example, the first probability distribution can be a 20-dimensional vector.

[0183] S404. The pre-trained image model is used to process the pre-processed image information to obtain the second probability distribution.

[0184] In one possible implementation, the second probability distribution is obtained by inputting the dimensionality-reduced image features into a pre-trained image model for processing.

[0185] For example, the second probability distribution could be a 20-dimensional vector.

[0186] S405. The pre-processed metadata information is processed using a pre-trained data model to obtain the website risk probability.

[0187] Understandably, the website risk probability is a one-dimensional vector.

[0188] It should be noted that during the pre-training of text models, image models, and data models, the GridSearch algorithm can be used to optimize the model parameters.

[0189] Table 1 shows the model parameters and their value ranges.

[0190] Table 1

[0191] parameter Range of values Learning rate [1e-5, 5e-5] gagging sampling rate [0.6,0.8] Tree depth [3,7]

[0192] During the optimization of model parameters using the GridSearch algorithm, each parameter combination is validated using 5-fold cross-validation. The parameter combination with the highest weighted F1-score is selected as the optimal combination. For example, the optimal combination could be learning rate = 3e-5, sampling rate = 0.8, and tree depth = 5.

[0193] S406. Using a meta-learner, process the first probability distribution, the second probability distribution, and the website risk probability to obtain the website type of the target website.

[0194] For example, after tuning the parameters of the XGBoost classifier, the 20-dimensional first probability distribution, the 20-dimensional second probability distribution, and the 1-dimensional website risk probability are input into the XGBoost classifier to obtain the website type of the target website.

[0195] The process of tuning the XGBoost classifier parameters can be achieved by performing 5-fold cross-validation on the training set using the base model.

[0196] Figure 5 This is a schematic diagram illustrating a model pre-training method provided in an embodiment of this application. Figure 5 As shown, the model pre-training process includes pre-training the text model, the image model, the data model, and the XGBoost classifier.

[0197] The text model pre-training process includes bagging sampling, PyTorch fine-tuning, and cross-validation.

[0198] Image model pre-training includes image model training.

[0199] Data model pre-training includes LightGBM model parameter tuning.

[0200] Based on the aforementioned website identification method, a heterogeneous base learner is constructed that integrates text, image, and metadata information. The model parameters are optimized using bagging and GridSearch algorithms, and combined with a meta-learner. This addresses the issues of fixed model parameters and inefficient fusion in traditional models, achieving global coverage and dynamic association of multimodal data. Through object detection, dimensionality reduction, and clustering, fine-grained features of website types are extracted from image information, avoiding the visual information omissions caused by relying on predefined image libraries or global features. Processing is implemented to address scenarios such as noisy text information, low-resolution images, and incomplete metadata, reducing the false positive rate, improving model robustness, and enhancing the accuracy of identifying emerging disguised websites.

[0201] As a concrete example, text information, image information, and metadata information are obtained through the target website's URL or domain name. After clustering the image information, one tag is identified as "malicious website." The metadata information includes the number of times the registration entity has changed, which is 3 times.

[0202] After inputting the aforementioned text information, image information, and metadata information into the pre-trained text model, pre-trained image model, and pre-trained data model, respectively, a first probability distribution, a second probability distribution, and a website risk probability are obtained. In the first probability distribution, the probability of the website type being "malicious website" is the highest at 85%; in the second probability distribution, the probability of the website type being "malicious website" is the highest at 78%; and the website risk probability is 92%, indicating that the target website is a high-risk website.

[0203] Since the number of changes to the registration entity exceeds the threshold for the number of changes (3 > 2), the weight corresponding to the website risk probability will be adjusted from 0.3 to 0.6.

[0204] The target website that was identified as a "bad website" by the meta-learner had the highest overall score. That is, the overall score of the target website that was identified as a "bad website" was (0.85×0.2)+(0.78×0.2)+(0.92×0.6)=0.878, and the target website was identified as a "bad website".

[0205] In this embodiment, the website identification system can be divided into functional modules according to the website identification method shown in the above method embodiments. For example, Figure 6 This is a schematic diagram illustrating the module division of a website identification system provided in an embodiment of this application. For example... Figure 6 As shown, the website recognition system includes an input layer, a feature processing layer, an ensemble learning layer, and an output layer.

[0206] The input layer includes text information, image information, and metadata information.

[0207] The feature processing layer includes data cleaning, word segmentation, truncation; normalization, object detection, dimensionality reduction, clustering; missing value imputation, and temporal coding.

[0208] The ensemble learning layer includes a pre-trained text model, a pre-trained image model, a pre-trained data model, and a meta-learner.

[0209] The output layer includes the website type of the target website.

[0210] This application embodiment can divide the website identification device into functional modules or functional units according to the above method examples. For example, each function can be divided into its own functional modules or functional units, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module or functional unit. The module or unit division in this application embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation.

[0211] Figure 7This is a schematic diagram of the structure of a website identification device 70 provided in an embodiment of this application. The website identification device 70 includes: an acquisition unit 701, used to acquire text information, image information, and metadata information of a target website, the metadata information including ICP filing information and WHOIS information; a first processing unit 702, used to process the text information through a pre-trained text model to obtain a first probability distribution, the first probability distribution indicating the probability that the target website belongs to each website type based on the text information; a second processing unit 703, used to process the image information through a pre-trained image model to obtain a second probability distribution, the second probability distribution indicating the probability that the target website belongs to each website type based on the image information; a third processing unit 704, used to process the metadata information through a pre-trained data model to obtain a website risk probability, the website risk probability indicating the risk level of the target website; and an identification unit 705, used to process the first probability distribution, the second probability distribution, and the website risk probability through a meta-learner to obtain the website type of the target website.

[0212] In one possible implementation, the identification unit 705 is used to: use the voting mechanism of the meta-learner to perform a weighted summation of the first probability distribution, the second probability distribution, and the website risk probability to obtain a comprehensive score for the target website to belong to each website type; and take the website type with the highest comprehensive score as the website type of the target website.

[0213] In one possible implementation, the device further includes: a comparison unit, used to re-extract features from the image information to obtain website image features when the maximum probability in the second probability distribution is less than a preset probability; and to process the website image features using a pre-trained image model to obtain an updated second probability distribution.

[0214] In one possible implementation, the device further includes: a first adjustment unit, configured to, when the number of the largest probabilities in the first probability distribution is greater than 1, increase the probability of website types with higher website type priority based on a preset website type priority, and use the increased probability distribution as the first probability distribution.

[0215] In one possible implementation, the device further includes a second adjustment unit, used to adjust the weight corresponding to the website risk probability to a preset weight when the metadata information meets the weight adjustment conditions.

[0216] In one possible implementation, the metadata information includes the number of times the filing entity has changed, the filing expiration time, the SSL certificate expiration time, and the registration time; the weight adjustment conditions include: the number of times the filing entity has changed is greater than the number of changes threshold; or, the filing expiration time is less than a first preset time; or, the SSL certificate expiration time is less than a second preset time; or, the registration time is less than a third preset time; or the metadata information is missing.

[0217] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0218] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the website identification method described in the above method embodiments.

[0219] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the website identification method in the method flow shown in the above method embodiments.

[0220] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0221] Since the website identification device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above method, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.

[0222] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0223] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0225] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A website identification method, characterized in that, The method includes: Obtain text information, image information, and metadata information from the target website. The metadata information includes ICP filing information of the network content provider and WHOIS information of the domain name query protocol. The text information is processed by a pre-trained text model to obtain a first probability distribution, which is used to indicate the probability that the target website belongs to each website type based on the text information. The image information is processed by a pre-trained image model to obtain a second probability distribution, which is used to indicate the probability that the target website belongs to each website type based on the image information. The metadata information is processed by a pre-trained data model to obtain the website risk probability, which is used to indicate the risk level of the target website. The first probability distribution, the second probability distribution, and the website risk probability are processed by a meta-learner to obtain the website type of the target website.

2. The method according to claim 1, characterized in that, The process of using a meta-learner to process the first probability distribution, the second probability distribution, and the website risk probability to obtain the website type of the target website includes: The first probability distribution, the second probability distribution, and the website risk probability are weighted and summed through the voting mechanism of the meta-learner to obtain the comprehensive score of the target website belonging to each website type. The website type with the highest overall score is selected as the website type of the target website.

3. The method according to claim 1, characterized in that, After processing the image information using a pre-trained image model to obtain the second probability distribution, the method further includes: If the maximum probability in the second probability distribution is less than the preset probability, the image information is re-extracted to obtain website image features; The website image features are processed using the pre-trained image model to obtain an updated second probability distribution.

4. The method according to claim 1, characterized in that, After processing the text information using a pre-trained text model to obtain the first probability distribution, the method further includes: If the number of the highest probabilities in the first probability distribution is greater than 1, based on the preset website type priority, the probability of website types with higher website type priority is increased, and the increased probability distribution is used as the first probability distribution.

5. The method according to claim 1, characterized in that, Before processing the first probability distribution, the second probability distribution, and the website risk probability using a meta-learner to obtain the website type of the target website, the method further includes: If the metadata information meets the weight adjustment conditions, the weight corresponding to the website risk probability will be adjusted to the preset weight.

6. The method according to claim 5, characterized in that, The metadata information includes the number of times the filing entity has changed, the filing expiration date, the expiration date of the Secure Sockets Layer (SSL) certificate, and the registration date. The weight adjustment conditions include: The number of changes to the filing entity exceeds the threshold for the number of changes; or, The expiration time of the filing is less than the first preset time; or, The SSL certificate expiration time is less than the second preset time; or, The registration time is less than the third preset time; or The metadata information is missing.

7. A website identification device, characterized in that, The device includes: The acquisition unit is used to acquire text information, image information, and metadata information of the target website. The metadata information includes ICP filing information of the network content provider and WHOIS information of the domain name query protocol. The first processing unit is used to process the text information through a pre-trained text model to obtain a first probability distribution, wherein the first probability distribution is used to indicate the probability that the target website belongs to each website type based on the text information. The second processing unit is used to process the image information through a pre-trained image model to obtain a second probability distribution, which is used to indicate the probability that the target website belongs to each website type based on the image information. The third processing unit is used to process the metadata information through a pre-trained data model to obtain the website risk probability, which is used to indicate the risk level of the target website. The identification unit is used to process the first probability distribution, the second probability distribution, and the website risk probability through a meta-learner to obtain the website type of the target website.

8. An electronic device, characterized in that, include: A processor and a communication interface; the communication interface is coupled to the processor, the processor being used to run computer programs or instructions to implement the website identification method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the website identification method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on a computer, cause the computer to perform the website identification method as described in any one of claims 1-6.

Citation Information

Cited By

  • Website classification method based on search engine

    CN121188306A

  • A search engine-based website classification method

    CN121188306B