Website grading detection method and device based on multi-modal large model

Abstract

The invention provides a website grading detection method and device based on a multi-modal large model, and relates to the artificial intelligence fields of deep learning, large models, natural language processing, computer vision and the like. The method comprises the following steps: acquiring a target website to be processed, and acquiring target detection information corresponding to the target website; according to the target detection information, carrying out preliminary detection on the target website by utilizing a knowledge base; in response to determining that the preliminary detection is passed, performing secondary detection on the target website by using a lightweight small model according to the target detection information; and in response to the fact that the target website is determined as a suspicious website through secondary detection, performing deep detection on the target website by using the multi-modal large model according to the target detection information to obtain the following deep detection result: a credible website or an uncredible website. By applying the scheme provided by the invention, the recall rate, the accuracy rate and the like of website detection can be remarkably improved on the premise that the cost is controllable.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of deep learning, large models, natural language processing and computer vision, and especially to a method and apparatus for hierarchical detection of URLs based on multimodal large models. Background Technology

[0002] In website detection scenarios, black and gray market information is characterized by strong concealment and rapid evolution. Furthermore, with the increasing complexity of internet content, black and gray market information is exhibiting trends towards multimodality, adversarial nature, and long-tail characteristics. Consequently, traditional detection methods based on text filtering and keyword matching are no longer sufficient, leading to numerous missed and false positives. Summary of the Invention

[0003] This disclosure provides a method and apparatus for URL hierarchical detection based on a multimodal large model.

[0004] A URL hierarchical detection method based on a multimodal large model includes:

[0005] Obtain the target URL to be processed, and obtain the target detection information corresponding to the target URL;

[0006] Based on the target detection information, the target URL is preliminarily detected using a knowledge base;

[0007] In response to the determination that the preliminary detection has passed, a second detection is performed on the target URL using a lightweight small model based on the target detection information;

[0008] In response to the determination of the target URL as a suspicious URL through the secondary detection, a multimodal large model is used to perform deep detection on the target URL based on the target detection information, and the following deep detection results are obtained: trusted URL or untrusted URL.

[0009] A URL hierarchical detection device based on a multimodal large model includes: a preprocessing module, a preliminary detection module, a secondary detection module, and a deep detection module;

[0010] The preprocessing module is used to obtain the target URL to be processed and to obtain the target detection information corresponding to the target URL;

[0011] The preliminary detection module is used to perform preliminary detection on the target URL using a knowledge base based on the target detection information;

[0012] The secondary detection module is used to perform a secondary detection on the target URL based on the target detection information using a lightweight small model, in response to determining that the preliminary detection has passed.

[0013] The deep detection module is used to respond to the secondary detection determining the target URL as a suspicious URL, and to perform deep detection on the target URL using a multimodal large model based on the target detection information, to obtain the following deep detection results: trusted URL or untrusted URL.

[0014] An electronic device, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described above.

[0018] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0019] A computer program product includes a computer program / instructions that, when executed by a processor, implement the method described above.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a flowchart of the first embodiment of the URL hierarchical detection method based on a multimodal large model as described in this disclosure;

[0023] Figure 2 This is a flowchart of an embodiment of the method for detecting text information as described in this disclosure;

[0024] Figure 3 This is a flowchart of an embodiment of the method for detecting image information as described in this disclosure;

[0025] Figure 4 This is a flowchart of the second embodiment of the URL hierarchical detection method based on a multimodal large model as described in this disclosure;

[0026] Figure 5 This is a schematic diagram of the composition structure of Embodiment 500 of the URL hierarchical detection device based on a multimodal large model as described in this disclosure;

[0027] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Furthermore, it should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0030] Figure 1 This is a flowchart of the first embodiment of the URL hierarchical detection method based on a multimodal large model as described in this disclosure. Figure 1 As shown, the specific implementation methods are as follows.

[0031] In step 101, the target URL to be processed is obtained, and the target detection information corresponding to the target URL is obtained.

[0032] In step 102, the target URL is preliminarily detected using a knowledge base based on the target detection information.

[0033] In step 103, in response to determining that the preliminary detection has passed, a secondary detection of the target URL is performed using a lightweight small model based on the target detection information.

[0034] In step 104, in response to the target URL being identified as a suspicious URL through secondary detection, a multimodal large model is used to perform deep detection on the target URL based on the target detection information, resulting in the following deep detection results: trusted URL or untrusted URL.

[0035] The scheme described in the above-described method embodiments employs a multi-level detection mechanism, combining a knowledge base, rapid detection using a small model, and deep understanding using a multimodal large model. This reduces missed detections and false detections. Furthermore, the multimodal large model is used for further detection only on websites that pass the initial detection or whose credibility cannot be determined by a second detection, thereby reducing implementation costs. In other words, the scheme described in the above-described method embodiments can fully combine the efficiency of the knowledge base and lightweight small models with the accuracy of the multimodal large model, significantly improving the recall and accuracy of website detection while keeping costs under control, thus effectively enhancing the ecological governance capabilities of internet content.

[0036] The target URL to be processed can refer to the URL of the Uniform Resource Locator (URL) to be processed.

[0037] For a target URL, preprocessing can be performed to obtain the target detection information corresponding to that URL. The specific information included in the target detection information can be determined according to actual needs, and may include the URL and other information obtained from the URL. This other information may include: webpage content collected by a web crawler using a highly adversarial page optimization automated testing framework (Playwright), such as Hypertext Markup Language (HTML) snapshots, Document Object Model (DOM) structure, inline frame (iframe) content, page screenshots, website favicons, metadata information, and webpage feature fingerprints. Webpage feature fingerprints may include structural fingerprints and iframe MD5. MD5 refers to Message Digest Algorithm version 5. Additionally, plain text information can be extracted based on HTML snapshots and iframe content.

[0038] Based on the target detection information, the target website can be classified for detection. The detection methods for each level are introduced below.

[0039] 1) Preliminary testing

[0040] Based on the target detection information, the target URL can be initially detected using a knowledge base.

[0041] In some embodiments of this disclosure, target detection information can be compared with black and gray market intelligence databases of different dimensions. In response to determining that the target detection information includes content that matches any black and gray market intelligence database, it can be determined that the preliminary detection has failed and the target URL can be identified as an untrusted URL. In response to determining that the target detection information does not include content that matches any black and gray market intelligence database, it can be determined that the preliminary detection has passed.

[0042] The black and gray market intelligence database may include:

[0043] A. Black and gray market URL / domain databases and webpage fingerprint databases;

[0044] B. Database of fraudulent phone numbers, email addresses, and social media accounts;

[0045] C. Blacklist of IP / Account / Device Accounts;

[0046] D. Black and gray industry keyword database.

[0047] Accordingly, phone numbers, email addresses, social media accounts, etc., can be extracted from target detection information (such as plain text information) and compared with relevant black and gray market intelligence databases. Additionally, keywords can be extracted from plain text information and compared with relevant black and gray market intelligence databases. Furthermore, the domain name and subdomain of the URL can be obtained and compared with relevant black and gray market intelligence databases. Moreover, webpage feature fingerprints can be compared with relevant black and gray market intelligence databases. Screenshots of the page are not required for initial detection.

[0048] Once it is determined that the target detection information includes content that matches any black and gray market intelligence database, the initial detection is deemed to have failed, and the target URL can be identified as an untrusted URL. The process can then be terminated, meaning that no further steps will be taken.

[0049] Using a black and gray market intelligence database for initial detection enables millisecond-level rapid matching and interception based on known high-confidence threat intelligence. This allows for the immediate filtering out of most explicit and patterned untrusted URLs, reducing the detection pressure on subsequent lightweight small models and multimodal large models. Consequently, it significantly reduces the overall system computational cost and response latency, providing an efficient first line of defense for the hierarchical detection system.

[0050] If the initial detection is successful, a secondary detection of the target URL can be performed using a lightweight small model based on the target detection information.

[0051] (ii) Secondary testing

[0052] In some embodiments of this disclosure, the target detection information may include text information and image information. Accordingly, the text information can be detected to obtain the target text detection result, and the image information can be detected to obtain the target image detection result. Then, the target text detection result and the target image detection result can be combined to generate a secondary detection result.

[0053] For example, text information can refer to the aforementioned plain text information, and image information can refer to the aforementioned page screenshots. Accordingly, the target URL can be detected from two different dimensions: text and image, thereby further improving the accuracy of the detection results.

[0054] 1) Text dimension

[0055] In some embodiments of this disclosure, the text information can be determined as text to be processed, and the following predetermined processing can be performed: a first text detection is performed on the text to be processed using a rule matching method to obtain a first text detection result; in response to determining that the first text detection result is a rule mismatch, a second text detection is performed on the text to be processed using a text classification model to obtain a second text detection result, wherein the second text detection result is that the target URL is risky, does not have risk, or is uncertain; and the second text detection result is determined as the target text detection result.

[0056] In some embodiments of this disclosure, the predetermined processing may further include: determining the target URL as an untrusted URL in response to determining that the first text detection result is a hit rule, and determining the target URL as a suspicious URL in response to determining that the first text detection result is a suspected hit rule.

[0057] Rule matching is primarily used to identify highly structured, stable content that is difficult to circumvent through variations, or long-tail content that multi-classification models cannot cover, such as complex and ever-changing fraud scenarios. Specifically, rule matching can include rule-based matching, regular expression matching, and template matching.

[0058] For example, the fraud scenarios mentioned may include: bank card fraud: card number, issuing bank, transfer instructions; verification code hijacking: verification code, login link; contact information: customer service number, referral contact information; loan fraud: combination of keywords such as interest rate, credit limit, approval, credit check, etc.

[0059] If the rule is matched, the target URL can be directly identified as an untrusted URL, and subsequent processing can be terminated. If the rule is not matched, a second text detection can be performed using a text classification model. If the rule is suspected to be matched, the target URL can be identified as a suspicious URL.

[0060] Text classification models can be lightweight multi-class text classification models, such as Text Convolutional Neural Network (TextCNN) or FastText.

[0061] Using a text classification model, multi-label / category prediction can be performed on text information. Categories can include high-frequency risks such as borderline content, traffic diversion, fraud, confidentiality violations, and low quality. The model can output the following three results:

[0062] A. Risks exist.

[0063] For example, obvious fraudulent template phrases with clear semantics ("low-interest loans arrive instantly", "loans disbursed immediately without credit check") can be directly identified as risky, and the probability of misjudgment is extremely low.

[0064] B. No risk exists.

[0065] For example, generic, risk-free content from large websites, such as "Welcome to the State Taxation Administration of ** Province Electronic Tax Bureau".

[0066] C. Uncertain

[0067] For example, "I've sent you the verification code," but without any prior information, it's impossible to determine whether there is any risk.

[0068] Based on the above introduction, Figure 2 This is a flowchart illustrating an embodiment of the method for detecting text information as described in this disclosure. Figure 2 As shown, the specific implementation methods are as follows.

[0069] In step 201, the text information is identified as the text to be processed.

[0070] In step 202, the text to be processed is subjected to first text detection using rule matching to obtain the first text detection result.

[0071] In step 203, it is determined whether the first text detection result is a rule hit, a rule miss, or a suspected rule hit. If it is a rule hit, step 204 is executed; if it is a rule miss, step 205 is executed; if it is a suspected rule hit, step 206 is executed.

[0072] In step 204, the target URL is determined to be an untrusted URL, and then the process ends.

[0073] In step 205, a second text detection is performed on the text to be processed using a text classification model to obtain the second text detection result, and the second text detection result is determined as the target text detection result, and then the process ends.

[0074] In step 206, the target URL is identified as a suspicious URL, and then the process ends.

[0075] By combining text classification models with rule matching, the advantages of both semantic understanding and precise interception are taken into account. This ensures the detection generalization ability while achieving intelligent identification and interception with higher coverage and faster response for text risks.

[0076] 2) Image Dimensions

[0077] In some embodiments of this disclosure, a first image detection can be performed on the image information using a first detection method. In response to determining that the first image detection result is either risky or not risky, the first image detection result can be determined as the target image detection result. In response to determining that the first image detection result is uncertain, a second image detection can be performed on the image information using a second detection method. In response to determining that the second image detection result is either risky or not risky, the second image detection result can be determined as the target image detection result. In response to determining that the second image detection result is uncertain, a third image detection can be performed on the image information using a third detection method, and the third image detection result can be determined as the target image detection result. The third image detection result is either risky, not risky, or uncertain.

[0078] The three-level serial + early-stop strategy can be used to detect image information. The early-stop strategy can effectively improve detection efficiency. That is, if the current detection method makes a reliable judgment, the subsequent process can be terminated directly to avoid unnecessary computational overhead.

[0079] In some embodiments of this disclosure, the method of performing first image detection on image information by means of a first detection method may include: performing first image detection using a first visual model based on the overall visual features of the image information; the method of performing second image detection on image information by means of a second detection method may include: performing second image detection using a second visual model based on the local structural features of the image information.

[0080] In addition, in some embodiments of this disclosure, the method of performing third image detection on image information through a third detection method may include: extracting text information from the image information, determining the extracted text information as the text to be processed, performing the predetermined processing, and determining the obtained second text detection result as the third image detection result. Specifically, a rule matching method can be used to perform first text detection on the text to be processed to obtain a first text detection result. In response to determining that the first text detection result is a rule mismatch, a text classification model can be used to perform second text detection on the text to be processed to obtain a second text detection result. Furthermore, in response to determining that the first text detection result is a rule match, the target URL is determined to be an untrusted URL; in response to determining that the first text detection result is a suspected rule match, the target URL is determined to be a suspicious URL.

[0081] The first visual model can be a CNN model. CNN models excel at processing overall visual features, such as overall style, overall color saturation, overall layout shape, and user interface (UI) typography. Using the first visual model, image information with obvious visual risks and image information that is clearly risk-free can be quickly screened out. Accordingly, if the first image detection result generated by the first visual model indicates that there is risk or no risk, the first image detection result can be determined as the target image detection result, and the "three-level serial" processing can be terminated. Obvious visual risks can refer to the presence of large-area QR code advertisements, the presence of common promotional images, etc.

[0082] If the first image detection result is uncertain, a second visual model can be used for second image detection. The second visual model can be a real-time target detection (YOLO, You Only Look Once) model, which can be used to detect local high-risk targets (i.e., key risk target structures), such as referral QR codes, financial UIs (bank cards, transfer vouchers), documents, seals, customer service chat boxes, odds tables, etc. If it can be clearly determined whether there is a risk or not, a second image detection result indicating whether there is a risk or not can be directly generated, and the second image detection result can be determined as the target image detection result, ending the "three-level serial" processing; otherwise, an uncertain second image detection result can be generated.

[0083] If the second image detection result is uncertain, optical character recognition (OCR) technology can be used to extract text content from the image information. Based on the text content, the predetermined processing can be performed to obtain a third image detection result, which can then be identified as the target image detection result. The text classification model can be a TextCNN model, etc.

[0084] By using OCR+TextCNN, we can effectively extract and identify hidden textual risk information in image information, transforming visual modalities into analyzable semantic content. This compensates for the blind spots of purely visual models in detecting textual risks (such as lead generation scripts and fraudulent texts) within image information, thereby enhancing the comprehensive judgment ability for complex and disguised pages.

[0085] Based on the above introduction, Figure 3 This is a flowchart illustrating an embodiment of the method for detecting image information as described in this disclosure. Figure 3 As shown, the specific implementation methods are as follows.

[0086] In step 301, based on the overall visual features of the image information, the first visual model is used to perform first image detection on the image information to obtain the first image detection result.

[0087] In step 302, it is determined whether the first image detection result is risky, non-risky, or uncertain. If it is risky or non-risky, step 303 is executed; if it is uncertain, step 304 is executed.

[0088] In step 303, the first image detection result is set as the target image detection result, and then the process ends.

[0089] In step 304, based on the local structural features of the image information, a second visual model is used to perform second image detection on the image information to obtain the second image detection result.

[0090] In step 305, it is determined whether the second image detection result is risky, non-risky, or uncertain. If it is risky or non-risky, step 306 is executed; if it is uncertain, step 307 is executed.

[0091] In step 306, the second image detection result is set as the target image detection result, and then the process ends.

[0092] In step 307, text information is extracted from the image information, and the predetermined processing is performed on the extracted text information. The obtained second text detection result is determined as the target image detection result, and then the process ends.

[0093] After determining the target text detection result and the target image detection result respectively, the two can be combined to generate the secondary detection result.

[0094] In some embodiments of this disclosure, the secondary detection result may include: untrusted URL, trusted URL, and suspicious URL. Accordingly, in response to determining that at least one of the target text detection result and the target image detection result is at risk, the secondary detection result is determined to be an untrusted URL. In response to determining that the following second determination condition is met: neither the target text detection result nor the target image detection result is at risk, the secondary detection result is determined to be a trusted URL. In response to determining that neither the first determination condition nor the second determination condition is met, the secondary detection result is determined to be a suspicious URL.

[0095] It can be seen that the lightweight small model is only responsible for determining the "confident result", while the detection of highly difficult content can be handled by the multimodal large model.

[0096] In practical applications, more complex content filtering can be performed on suspicious URLs. The following explains how this is done.

[0097] III) High-Difficulty Content Filtering

[0098] In some embodiments of this disclosure, in response to determining that a target URL is identified as a suspicious URL through secondary detection, a multimodal large model can be directly used to perform deep detection on the target URL. Alternatively, the target URL can be triggered for identification based on predetermined triggering conditions. In response to determining that the target URL meets any triggering condition based on the identification result, a multimodal large model can be used to perform deep detection on the target URL. In response to determining that the target URL does not meet any triggering condition based on the identification result, the target URL can be identified as a trustworthy URL.

[0099] The specific method used can be determined according to actual needs, offering great flexibility and convenience. Preferably, the latter method can be adopted. By introducing a high-difficulty content filtering operation, high-difficulty target URLs can be identified through trigger conditions and routed to the multimodal large model for processing. This ensures that the resources of the multimodal large model are only allocated to the most complex target URLs that require the deepest reasoning, thereby further reducing implementation costs.

[0100] In some embodiments of this disclosure, the triggering conditions may include one or any combination of the following: having model uncertainty, having cross-modal conflict, having abnormal risk content, or belonging to the long-tail high-risk type.

[0101] Model uncertainty can include: classification confidence below a threshold, high category entropy, large distance from the training distribution, and activation space close to the category decision boundary. The model can refer to the aforementioned text classification model and / or visual model. Taking the text classification model in 1) as an example, classification confidence below a threshold means that the model's predicted probabilities for each category do not meet the preset standard; high category entropy means that the distribution of the predicted probabilities for each category tends to be uniform and has no clear bias; large distance from the training distribution means that the features of the target URL differ significantly from the data pattern learned during model training; and activation space close to the category decision boundary means that the activation vector corresponding to the target URL falls near the boundary. In the feature space of a neural network model, different categories usually have clear boundaries.

[0102] Cross-modal conflict refers to contradictory analysis results of different modalities (such as text and image information) corresponding to a target URL. For example, if the text detection result indicates no risk, but the image detection result is uncertain, this inconsistency suggests that the content is highly suspicious or deliberately disguised, and deep detection must be triggered for a final decision.

[0103] Abnormal risk content may include: hitting variant features (such as similar-looking characters), abnormal text length (too short or too long), abnormal spelling of high-frequency risk words, involving sensitive content (such as QR codes, chat boxes), and intentional blurring, obscuring, or mosaicking.

[0104] Long-tail high-risk types can include those involving fraud (such as customer service impersonation, investment scams, and lead generation) and sensitive information. This type is not well-suited for handling by lightweight, small-scale models.

[0105] IV) Depth Detection

[0106] In some embodiments of this disclosure, the multimodal large model may include: a multimodal large model fine-tuned based on training samples corresponding to the URL detection scenario. In addition, the method of using the multimodal large model to perform deep detection on the target URL may include: constructing a prompt based on the target detection information, inputting the prompt into the multimodal large model, and obtaining the output deep detection result.

[0107] There are no restrictions on how training samples are constructed. For example, training samples can include text and image information corresponding to a certain URL, and can also include corresponding tags, such as borderline, traffic-driving, fraudulent, confidential, low-quality, etc. The training samples can then be used to fine-tune the multimodal large model, such as performing low-rank adaptation (LoRA) fine-tuning, to improve the deep semantic reasoning capabilities of the multimodal large model.

[0108] Accordingly, prompt words can be constructed based on the text and image information of the target URL. The prompt words can be input into a finely tuned multimodal large model to obtain the output depth detection results, thereby ultimately determining whether the target URL is a trustworthy or untrustworthy URL. If it is an untrustworthy URL, the risk category can also be output at the same time, such as fraud or confidentiality.

[0109] By introducing the powerful cross-modal reasoning and deep semantic understanding capabilities of multimodal large models, it can effectively identify highly complex content such as hidden expressions, long-tail high-risk types, cross-modal camouflage, and adversarial variants, thereby significantly improving the coverage and detection accuracy of complex risk content. It enables accurate discovery, real-time early warning, and rapid handling of untrusted websites under the premise of controllable cost, and comprehensively enhances the system's defense robustness and response timeliness.

[0110] Based on the above introduction, Figure 4 This is a flowchart of the second embodiment of the URL hierarchical detection method based on a multimodal large model described in this disclosure. Figure 4 As shown, the specific implementation methods are as follows.

[0111] In step 401, the target URL to be processed is obtained, and the target detection information corresponding to the target URL is obtained.

[0112] In step 402, based on the target detection information, the target URL is preliminarily detected using a knowledge base, and it is determined whether the preliminary detection passes. If not, step 403 is executed; if yes, step 404 is executed.

[0113] In step 403, the target URL is intercepted, and then the process ends.

[0114] This means that the target URL can be identified as an untrusted URL and then blocked.

[0115] In step 404, the text information in the target detection information is detected by rule matching to obtain the first text detection result.

[0116] In step 405, it is determined whether the first text detection result is a rule hit, a rule miss, or a suspected rule hit. If it is a rule hit, step 403 is executed; if it is a rule miss, step 406 is executed; if it is a suspected rule hit, step 410 is executed.

[0117] In step 406, a second text detection is performed on the text information using a text classification model to obtain the target text detection result.

[0118] In step 407, the image information in the target detection information is detected to obtain the target image detection result.

[0119] Among them, the detection of image information can adopt the aforementioned three-level cascaded + early termination strategy.

[0120] In step 408, the target text detection result and the target image detection result are combined to determine whether the first judgment condition is met, the second judgment condition is met, or neither judgment condition is met. If the first judgment condition is met, step 403 is executed. If the second judgment condition is met, step 409 is executed. If neither judgment condition is met, step 410 is executed.

[0121] Meeting the first criterion means that at least one of the target text detection results and the target image detection results is at risk.

[0122] Meeting the second criterion means that both the target text detection result and the target image detection result are free of risk.

[0123] In step 409, the target URL is allowed, and then the process ends.

[0124] This allows you to determine if the target URL is a trusted URL and to allow access to it.

[0125] In step 410, the target URL is identified based on trigger conditions of different dimensions to determine whether the target URL meets any trigger condition. If yes, step 411 is executed; otherwise, step 409 is executed.

[0126] In step 411, prompt words are constructed based on the target detection information, and the prompt words are input into the multimodal large model to obtain the output depth detection results.

[0127] In step 412, it is determined whether the depth detection result is a trusted or untrusted URL. If it is a trusted URL, step 409 is executed; if it is an untrusted URL, step 403 is executed.

[0128] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure. Furthermore, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0129] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0130] Figure 5This is a schematic diagram of the structural composition of Embodiment 500 of the URL hierarchical detection device based on a multimodal large model as described in this disclosure. Figure 5 As shown, it includes: a preprocessing module 501, a preliminary detection module 502, a secondary detection module 503, and a depth detection module 504.

[0131] The preprocessing module 501 is used to obtain the target URL to be processed and to obtain the target detection information corresponding to the target URL.

[0132] The preliminary detection module 502 is used to perform preliminary detection on the target URL based on the target detection information and a knowledge base.

[0133] The secondary detection module 503 is used to perform a secondary detection on the target URL based on the target detection information, using a lightweight small model, in response to the determination that the preliminary detection has passed.

[0134] The depth detection module 504 is used to respond to the determination of the target URL as a suspicious URL through secondary detection, and to perform depth detection on the target URL using a multimodal large model based on the target detection information, to obtain the following depth detection results: trusted URL or untrusted URL.

[0135] In some embodiments of this disclosure, the preliminary detection module 502 can compare the target detection information with black and gray industry intelligence databases of different dimensions. In response to determining that the target detection information includes content that matches any black and gray industry intelligence database, the preliminary detection can be determined to have failed and the target URL can be determined to be an untrusted URL. In response to determining that the target detection information does not include content that matches any black and gray industry intelligence database, the preliminary detection can be determined to have passed.

[0136] In some embodiments of this disclosure, the target detection information may include text information and image information. The secondary detection module 503 may perform secondary detection on the target URL in the following ways: detect the text information to obtain the target text detection result, detect the image information to obtain the target image detection result, and combine the target text detection result and the target image detection result to generate a secondary detection result.

[0137] In some embodiments of this disclosure, the secondary detection module 503 detects text information and obtains the target text detection result in the following ways: determining the text information as text to be processed, and performing the following predetermined processing: performing a first text detection on the text to be processed using a rule matching method to obtain a first text detection result; in response to determining that the first text detection result is a rule mismatch, performing a second text detection on the text to be processed using a text classification model to obtain a second text detection result, wherein the second text detection result is that the target URL is risky, does not have risk, or is uncertain; and determining the second text detection result as the target text detection result.

[0138] In some embodiments of this disclosure, the predetermined processing may further include: determining the target URL as an untrusted URL in response to determining that the first text detection result is a hit rule; and determining the target URL as a suspicious URL in response to determining that the first text detection result is a suspected hit rule.

[0139] In some embodiments of this disclosure, the secondary detection module 503 detects image information and obtains a target image detection result in the following ways: performing a first image detection on the image information using a first detection method, and determining the first image detection result as the target image detection result in response to determining that the first image detection result is either risky or not risky; performing a second image detection on the image information using a second detection method in response to determining that the second image detection result is either risky or not risky, and determining the second image detection result as the target image detection result in response to determining that the second image detection result is uncertain, performing a third image detection on the image information using a third detection method, and determining the third image detection result as the target image detection result, wherein the third image detection result is either risky, not risky, or uncertain.

[0140] In some embodiments of this disclosure, the secondary detection module 503 may perform first image detection on image information using a first detection method, which may include: performing first image detection using a first visual model based on the overall visual features of the image information; the secondary detection module 503 may perform second image detection on image information using a second detection method, which may include: performing second image detection using a second visual model based on the local structural features of the image information.

[0141] In some embodiments of this disclosure, the secondary detection module 503 may perform third image detection on image information by means of a third detection method, including: extracting text information from image information, determining the extracted text information as text to be processed, performing the predetermined processing, and determining the obtained second text detection result as the third image detection result.

[0142] In some embodiments of this disclosure, the secondary detection results include: trusted URLs, untrusted URLs, and suspicious URLs. Accordingly, the secondary detection module 503 may generate secondary detection results by combining the target text detection results and the target image detection results in the following manner: in response to determining that at least one of the target text detection results and the target image detection results is at risk, the secondary detection result is determined to be an untrusted URL; in response to determining that the following second determination condition is met: neither the target text detection result nor the target image detection result is at risk, the secondary detection result is determined to be a trusted URL; in response to determining that neither the first determination condition nor the second determination condition is met, the secondary detection result is determined to be a suspicious URL.

[0143] In some embodiments of this disclosure, the depth detection module 504, in response to determining the target URL as a suspicious URL through secondary detection, can also trigger identification of the target URL according to predetermined trigger conditions. In response to determining that the target URL meets any trigger condition based on the identification result, it can use a multimodal large model to perform depth detection on the target URL. In response to determining that the target URL does not meet any trigger condition based on the identification result, it can determine the target URL as a trustworthy URL.

[0144] In some embodiments of this disclosure, the triggering conditions may include one or any combination of the following: having model uncertainty, having cross-modal conflict, having abnormal risk content, or belonging to the long-tail high-risk type.

[0145] In addition, in some embodiments of this disclosure, the multimodal large model may include: a multimodal large model fine-tuned based on training samples corresponding to the URL detection scenario. Furthermore, the method by which the depth detection module 504 uses the multimodal large model to perform depth detection on the target URL may include: constructing prompt words based on the target detection information, inputting the prompt words into the multimodal large model, and obtaining the output depth detection result.

[0146] The specific workflow of the above-described device embodiments can be found in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.

[0147] Furthermore, the solution described in this disclosure can be applied to different scenarios and has broad adaptability. For example:

[0148] 1. Search Engine / Browser Risk Control: By analyzing webpage text, screenshots, structural layout, and QR code redirection relationships using multimodal methods, risky landing pages can be identified in advance during the search display process, effectively reducing the exposure rate of malicious results and enhancing the security capabilities of browser and search engine entry points.

[0149] 2. Social Media Platform Content Security: Identify the URL content of social media posts, comments, private messages, and posts to detect risks of black and gray market activities such as fraudulent promotions and coded traffic redirection.

[0150] 3. Ad Placement Review: In scenarios such as ad placement and creative material review, it can identify situations where QR codes on landing pages lead to fraudulent websites, thereby significantly reducing the success rate of gray and black market activities in attracting traffic, committing fraud, and circumventing compliance reviews through advertising channels.

[0151] 4. Anti-fraud system: Malicious URLs in telecommunications network fraud are characterized by strong resistance, strong concealment, and rapid updates. It can quickly identify fraudulent landing pages, improve the accuracy and coverage of early warnings, etc.

[0152] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly deep learning, large-scale models, natural language processing, and computer vision. Artificial intelligence is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware and software technologies. Artificial intelligence hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0153] Furthermore, the various image and text information in the embodiments described in this disclosure are not targeted at any specific user and are not intended to reflect the personal information of any specific user. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions of this disclosure all comply with relevant laws and regulations and do not violate public order and good morals.

[0154] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0155] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0156] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0157] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0158] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as those described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods described in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the methods described herein by any other suitable means (e.g., by means of firmware).

[0159] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0161] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0163] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0164] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0165] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0166] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A URL hierarchical detection method based on a multimodal large model, comprising: Obtain the target URL to be processed, and obtain the target detection information corresponding to the target URL; Based on the target detection information, the target URL is preliminarily detected using a knowledge base; In response to the determination that the preliminary detection has passed, a second detection is performed on the target URL using a lightweight small model based on the target detection information; In response to the determination of the target URL as a suspicious URL through the secondary detection, a multimodal large model is used to perform deep detection on the target URL based on the target detection information, and the following deep detection results are obtained: trusted URL or untrusted URL.

2. The method according to claim 1, wherein, The preliminary detection of the target URL using a knowledge base includes: The target detection information is compared with black and gray market intelligence databases of different dimensions; In response to determining that the target detection information includes content that matches any black and gray market intelligence database, the preliminary detection is determined to have failed, and the target URL is identified as the untrusted URL; In response to determining that the target detection information does not contain any content that matches any black and gray market intelligence database, the preliminary detection is determined to be successful.

3. The method according to claim 1, wherein, The target detection information includes: text information and image information; The secondary detection of the target URL includes: The text information is detected to obtain the target text detection result; The image information is detected to obtain the target image detection result; A secondary detection result is generated by combining the target text detection result and the target image detection result.

4. The method according to claim 3, wherein, The step of detecting the text information to obtain the target text detection result includes: The text information is identified as the text to be processed, and the following predetermined processing is performed: the text to be processed is subjected to a first text detection using a rule matching method to obtain a first text detection result; in response to determining that the first text detection result is a rule not hit, the text to be processed is subjected to a second text detection using a text classification model to obtain a second text detection result; the second text detection result is that the target URL is risky, does not have risk, or is uncertain. The second text detection result is determined as the target text detection result.

5. The method according to claim 4, wherein, The pre-processing also includes: In response to determining that the first text detection result is a hit rule, the target URL is identified as the untrusted URL; In response to determining that the first text detection result is a suspected rule hit, the target URL is identified as the suspicious URL.

6. The method according to claim 5, wherein, The step of detecting the image information to obtain the target image detection result includes: The image information is subjected to a first image detection using a first detection method. In response to determining that the first image detection result indicates the presence of risk or the absence of risk, the first image detection result is determined as the target image detection result. In response to determining that the first image detection result is uncertain, a second image detection is performed on the image information using a second detection method. In response to determining that the second image detection result is either that there is a risk or that there is no risk, the second image detection result is determined as the target image detection result. In response to determining that the second image detection result is uncertain, a third image detection is performed on the image information using a third detection method, and the third image detection result is determined as the target image detection result, wherein the third image detection result is the presence of risk, the absence of risk, or the uncertainty.

7. The method according to claim 6, wherein, The first image detection of the image information by the first detection method includes: performing the first image detection using a first visual model based on the overall visual features of the image information; The second image detection method includes performing the second image detection using a second visual model based on the local structural features of the image information.

8. The method according to claim 6, wherein, The third image detection of the image information using the third detection method includes: Extract text information from the image information; The extracted text information is identified as the text to be processed, and the predetermined processing is performed. The obtained second text detection result is identified as the third image detection result.

9. The method according to claim 6, wherein, The secondary detection results include: the trusted URL, the untrusted URL, and the suspicious URL; The step of generating a secondary detection result by combining the target text detection result and the target image detection result includes: In response to determining that at least one of the target text detection result and the target image detection result meets the following first determination condition: the secondary detection result is determined to be the untrusted URL; In response to determining that the second judgment condition is met: both the target text detection result and the target image detection result indicate that there is no risk, the secondary detection result is determined to be the trusted URL; In response to the determination that neither the first determination condition nor the second determination condition is met, the secondary detection result is determined as the suspicious URL.

10. The method according to claim 1, further comprising: In response to identifying the target URL as the suspicious URL through the secondary detection, the target URL is triggered for identification according to predetermined triggering conditions; In response to determining that the target URL meets any triggering condition based on the identification result, the multimodal large model is used to perform the depth detection on the target URL; In response to determining, based on the identification result, that the target URL does not meet any triggering condition, the target URL is identified as the trusted URL.

11. The method according to claim 10, wherein, The triggering conditions include one or any combination of the following: model uncertainty, cross-modal conflict, abnormal risk content, or belonging to the long-tail high-risk type.

12. The method according to claim 1, wherein, The multimodal large model includes: a multimodal large model fine-tuned based on training samples corresponding to the URL detection scenario; The method of performing deep detection on the target URL using a multimodal large model includes: constructing prompt words based on the target detection information, inputting the prompt words into the multimodal large model, and obtaining the output deep detection result.

13. A URL hierarchical detection device based on a multimodal large model, comprising: The system consists of a preprocessing module, a preliminary detection module, a secondary detection module, and a depth detection module. The preprocessing module is used to obtain the target URL to be processed and to obtain the target detection information corresponding to the target URL; The preliminary detection module is used to perform preliminary detection on the target URL using a knowledge base based on the target detection information; The secondary detection module is used to perform a secondary detection on the target URL based on the target detection information using a lightweight small model, in response to determining that the preliminary detection has passed. The deep detection module is used to respond to the secondary detection determining the target URL as a suspicious URL, and to perform deep detection on the target URL using a multimodal large model based on the target detection information, to obtain the following deep detection results: trusted URL or untrusted URL.

14. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

16. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the method of any one of claims 1-12.

Images