A method and apparatus for identifying risky websites

Abstract

This invention relates to the field of website data processing technology, and provides a method and apparatus for identifying risky websites. The method includes: performing binary classification data annotation on multimodal data obtained from multidimensional website information; using LoRA fine-tuning technology to perform supervised instruction fine-tuning and weight merging on a large multimodal model to obtain a first risky website identification model; performing multi-class data annotation to establish a second training dataset; combining GaLore fine-tuning technology to perform supervised instruction fine-tuning and weight merging on a large multimodal model to obtain a second risky website identification model; receiving current website data and inputting it into the first risky website identification model to obtain a website identification result; when the obtained website identification result is a risky website, inputting the current website data into the second risky website identification model to obtain a risky website category result. This invention achieves progressive identification of risky websites, significantly improving the accuracy of risky website identification in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of website information processing technology, and in particular to a method and apparatus for identifying risky websites. Background Technology

[0002] Faced with increasingly complex risky website scenarios, traditional identification methods are mostly based on single-modal data. Such single-modal independent identification methods lack the ability to comprehensively analyze multi-source information and understand complex semantics and patterns, which easily leads to misjudgment and missed judgment.

[0003] Traditional deep learning models exhibit poor generalization performance in scenarios with small samples or heterogeneous data, making them unable to adapt to the dynamic evolution of increasingly complex risky websites and to promptly identify the types and characteristics of newly emerging risky websites. Furthermore, cutting-edge technologies such as multimodal large models have not yet been used to complete the task of identifying risky websites; therefore, existing methods cannot comprehensively and accurately determine the nature and risk status of websites.

[0004] Therefore, the present invention provides a new method and apparatus for identifying risky websites to solve the above-mentioned problems. Summary of the Invention

[0005] This invention aims to provide a method and apparatus for identifying risky websites, addressing the technical problems of existing identification models having low efficiency and accuracy in complex application scenarios, the lack of identification models based on multimodal data such as text, images, and videos, poor generalization performance of existing deep learning models in small sample or heterogeneous data scenarios, and the inability to promptly detect the types and characteristics of newly emerging risky websites due to their inability to adapt to the dynamic evolution of complex risky websites. The technical problems to be solved by this invention are achieved through the following technical solutions.

[0006] The first aspect of this invention proposes a method for identifying risky websites, comprising: collecting multi-dimensional website information to form multimodal data; based on the obtained multimodal data, performing binary classification data annotation of websites according to predefined website types to establish a first training dataset, and generating a first instruction fine-tuning dataset using zero-shot thought chain technology according to a first-level business task, and performing instruction-supervised fine-tuning and weight merging on a large multimodal model using LoRA fine-tuning technology to obtain a first risky website identification model; based on the obtained multimodal data, performing multi-class classification data annotation of websites according to predefined website types to establish a second training dataset, and performing instruction-supervised fine-tuning and weight merging on a large multimodal model using a multi-thought chain inference mode and combined with GaLore fine-tuning technology according to a second-level business task to obtain a second risky website identification model; receiving current website data based on a model inference service call interface, inputting it into the first risky website identification model to obtain a website identification result; when the obtained website identification result is a risky website, inputting the current website data into the second risky website identification model to obtain a risky website category result.

[0007] The second aspect of this invention proposes a risky website identification device, which executes the risky website identification method described in the first aspect of this invention. The risky website identification device includes: a data collection module for collecting multi-dimensional website information to form multimodal data; a first construction module for, based on the obtained multimodal data, performing binary classification data annotation of websites according to predefined website types to establish a first training dataset, and generating a first instruction fine-tuning dataset using zero-shot thought chain technology according to a primary business task, and using LoRA fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain a first risky website identification model; and a second construction module based on... The obtained multimodal data is labeled with multi-classification data of websites according to predefined website types to establish a second training dataset. Based on the secondary business tasks, a multi-thinking chain reasoning mode is used, and GaLore fine-tuning technology is combined to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain a second risk website identification model. The first determination module receives the current website data based on the model inference service call interface, inputs it into the first risk website identification model, and obtains the website identification result. The second determination module, when the obtained website identification result is a risky website, inputs the current website data into the second risk website identification model to obtain the risk website category result.

[0008] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the risky website identification method described in the first aspect of the present invention.

[0009] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the risky website identification method described in the first aspect of the present invention.

[0010] The embodiments of the present invention have the following advantages: Compared with existing technologies, this invention constructs a progressive two-level risk website identification model based on multimodal data such as text, images, and videos, thereby further improving identification efficiency and accuracy. It breaks through the limitations of traditional single-modal identification, realizes comprehensive and in-depth feature mining of risk websites, and thus significantly improves the accuracy of risk website identification in complex scenarios.

[0011] Furthermore, by integrating multimodal data such as text, images, and audio from a website, it breaks through the limitations of traditional single-modal recognition. It can simultaneously analyze guiding words in web page text, suspicious logos or advertisements in images, and risk warnings contained in audio, comprehensively and accurately characterizing website features.

[0012] In addition, a multimodal large model technology framework is introduced, specifically to comprehensively process risk website information using a multimodal large model. This framework integrates multi-source data such as text and images, and simultaneously inputs multimodal information from images and text into the multimodal large model. This differs from models that input predefined independent modalities separately, allowing for fine-tuning and training of multimodal data while breaking the isolation of traditional single-modal analysis.

[0013] Furthermore, customized prompts (i.e., Prompts) were designed for risk website identification tasks. By precisely guiding the model to focus on key information, such as the type of risky website and the correlation of multimodal data, the multimodal big data model was guided to explore the correlation between website type and website features, as well as the hidden correlations and features between various modal data. For example, it was used to identify whether the text and image content were consistent and to make a comprehensive judgment based on the combination of text and image information. This fully utilized the powerful modality processing and semantic understanding capabilities of the multimodal big data model, significantly improving the model's ability to understand and judge complex risk scenarios. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating an example of the risky website identification method of the present invention; Figure 2 This is a schematic diagram of the process for constructing the first risky website identification model in the risky website identification method of the present invention; Figure 3 This is a schematic diagram of the process for constructing the second risky website identification model in the risky website identification method of the present invention; Figure 4 This is a structural block diagram of the risk website identification device of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer-readable medium embodiment according to the present invention. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] In view of the above problems, this invention proposes a risk website identification method. For the intelligent identification of risk websites, based on multimodal data such as text, images, and videos, a progressive two-level risk website identification model (a first risk website identification model and a second risk website identification model) based on a large multimodal model is constructed. This achieves progressive identification of risk websites, thereby improving identification efficiency and accuracy. Specifically, the large multimodal model is used to comprehensively analyze the acquired multimodal data, more comprehensively and accurately judging various complex risk website types and providing classification reasons. It also supports the identification of novel risk website types beyond the preset classifications.

[0017] It should be noted that the risk website identification method of the present invention has wide applications, is particularly suitable for identifying risk websites in complex scenarios, and is also suitable for use in conjunction with anti-fraud big data model platforms to support intelligent identification of risk websites. The method of the present invention will be described in detail below with specific examples.

[0018] Example 1 The following reference Figure 1 , Figure 2 and Figure 3 The present invention will be described in detail below.

[0019] Figure 1 This is a flowchart illustrating an example of the risky website identification method of the present invention.

[0020] like Figure 1 As shown, in step S101, multi-dimensional website information is collected to obtain multimodal data.

[0021] Multimodal data is generated by crawling multidimensional website information from target websites. This multidimensional website information includes website name, website URL, website OCR text content, website image data, and website hyperlink URLs.

[0022] Specifically, website URLs are a crucial foundation for identifying risky websites. The domain name, path, and parameters they contain serve as the first line of defense in risk assessment. For example, some risky websites use incorrect domain names and confuse users through subtle character substitutions (such as changing "baidu" to "baidul"); while unusual path and parameter settings indicate risky redirects or data loss, which are often difficult to detect.

[0023] Website text content is a direct indicator of a website's nature, encompassing page titles, body text, link text, and meta tags. Page titles often directly reflect the website's core theme; if they contain irrelevant or risky content, it indicates potential risks. Link text reveals the website's redirection intent; links inconsistent with the website's theme pose various potential risks. Keywords and descriptions in meta tags that significantly contradict the actual content indicate the website is misleading search engines and users.

[0024] Website image data includes website logos, advertising images, promotional images, etc. Some risky websites will imitate the logos of legitimate websites, taking advantage of users' trust in well-known brands to put users in a risky situation; other websites will use exaggerated advertising images to attract users to click, thus putting users in a risky situation as well.

[0025] Website hyperlink URLs serve as links between websites. Data collection includes whether hyperlinked URLs redirect to risky or illegal websites, as well as the quantity, distribution, and target URLs of these hyperlinks. In practice, some risky websites hide numerous malicious hyperlinks or use deceptive design to guide users to click on them, potentially leading to information leaks or cyberattacks.

[0026] The aforementioned multi-dimensional website information is integrated and processed to obtain multimodal data. This integration process includes removing invalid website data, standardizing formatting, adjusting file size, handling pixel limitations, and text cleaning (including removing HTML tags and special characters), among other things.

[0027] For example, multimodal formats need to be standardized. Image files should all be formatted as JPEG, with the extension .jpg or .jpeg. Individual image files should not exceed 10 MB in size. Image width and height should both be greater than 10 pixels, with a maximum aspect ratio less than 200:1 or 1:200, and the total number of pixels less than 12 million. If an image exceeds size or pixel limits, the `imencode` function from the OpenCV library and the `resize` function from the Pillow library can be used to compress the image size and pixels to obtain an image or text that meets the size or pixel limits for subsequent model input, i.e., it should meet the model input format requirements. The collected text should be cleaned to remove HTML tags and special characters.

[0028] Multimodal data obtained by crawling and integrating the above-mentioned multidimensional website information can more accurately identify risky websites.

[0029] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0030] Next, in step S102, based on the obtained multimodal data, binary classification data annotation of websites is performed according to the predefined website types to establish the first training dataset. According to the first-level business task, the first instruction fine-tuning dataset is generated using zero-shot thinking chain technology, and LoRA fine-tuning technology is used to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the first risk website identification model.

[0031] Specifically, a multi-level risk website classification system is established for data annotation and model classification. This includes a primary classification task (i.e., a binary classification task) for risk websites and a secondary classification task (i.e., a multi-classification task) for specific risk website types. Table 1 below shows the specific multi-level risk website classification system: The primary classification divides risk websites into risk-free websites and risky websites, performing a coarse risk classification; the secondary classification, based on the specific risk website type, divides risk websites into multiple subcategories (e.g., 18 subcategories), including risk-free websites, risky department websites, risky service websites, risky operator websites, risky application websites, risky software download websites, risky audio-visual resource websites, risky financial resource websites, risky e-commerce transaction websites, risky social communication websites, risky payment platform websites, risky recruitment websites, risky news websites, risky fundraising websites, risky agency websites, risky part-time job platform websites, risky currency websites, or other risky websites, etc.

[0032] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention. Other embodiments also include risky websites with other types of risks.

[0033] Table 1

[0034] In subsequent website data annotation, based on the multimodal data obtained in step S101, and according to the website types predefined in Table 1 above, binary classification data annotation is performed to establish a first classification dataset. Then, based on the primary business task (i.e., the binary classification task), zero-shot thought chain technology is used to generate a first instruction fine-tuning dataset for subsequent model training. This first classification dataset contains multimodal data labeled with risky website tags, focusing on basic risk identification of websites.

[0035] To further optimize the data, and considering the characteristics of risky websites, a zero-shot thought chain technique was used to generate a first instruction fine-tuning dataset for subsequent model training, in order to optimize the first pre-trained polymorphic large model. See details below. Figure 2 .

[0036] Extract partial data of website URLs and website screenshots from the first training dataset, and divide them into a first training set, a first validation set, and a first test set for binary classification of risky websites according to a first specified ratio (e.g., 7:1:2). These sets are used for fine-tuning, iterative validation, and testing of the first pre-trained polymorphic large model.

[0037] To further enhance the reasoning ability of the first pre-trained multimodal large model in solving the risk website classification task (i.e., binary classification task), a first instruction fine-tuning dataset for the binary classification task is constructed based on the Zero-shot Chain of Thought (Zero-shot CoT) technology. The website classification reasoning knowledge of the first fine-tuning task dataset is then transferred to the downstream first pre-trained multimodal large model.

[0038] It should be noted that the thought chain technology breaks down a logical reasoning problem into multiple steps, resulting in a clearer logical chain in the generated results, thus enhancing interpretability.

[0039] Specifically, a first inference model (e.g., DeepSeek-R1) is selected as the teacher model. Website information (corresponding to the aforementioned multi-dimensional website information, specifically including website URL, website OCR text content, website image data, website hyperlink URLs, website names, etc.) and classification results are extracted from the first training dataset and used as input data to the first inference model. The above input data of the first inference model is converted into a question template (e.g., "Q: [Question]. A: [Trigger Sentence]"). By attaching a trigger sentence (e.g., "Let's think about it step by step") to the end of the set question, the first inference model can generate a thought chain to answer the question, guiding the model to generate the reason for classifying risky websites. For example, given a website screenshot image in the path "data / website_CLS_2 / 1.jpg" of the first classification dataset data / website_CLS_2 and the website URL "https: / / 101.32.23.152", the label of the website is 1 (i.e., risky website), the following question is set as input to the first inference model to generate the classification reason:

[0040] Q: The image is a screenshot of a snapshot of the website https: / / 09n25k.660xi.com, which was identified as a <risky website>.

[0041] A: Let's think about it step by step. The first inference model generates the following output based on this prompt: "(Output) The URL of this website contains numbers with unknown meaning, and the website involves a large number of risky keywords and risky images. These factors indicate that the website has a certain degree of risk, therefore it is a risky website." The first inference model generates a thought chain to answer questions, guiding it in generating reasons for risk website classification. This first inference model can generate inference prompts for different risk website types, creating a thought chain corresponding to the model's input data—a binary classification reason thought chain. These inference prompts for various risk website types help the downstream pre-trained multimodal model improve its inference capabilities for risk website classification.

[0042] The generated binary classification cause-of-death thought chain and the risky website labels in the model input data are further combined to construct the first instruction fine-tuning dataset. For example, given a website screenshot image in the path "data / website_CLS_2 / 1.jpg" of the first classification dataset data / website_CLS_2 and the website URL "https: / / 101.32.23.152", the website label is 1 (i.e., a risky website), the specific form of the combined fine-tuning data is as follows: { "messages": [ { "content": " This image is a screenshot of a website, https: / / 101.32.23.152. Is it a risky or safe website? "role": "user" }, { "content": "<Risk website>", "role": "assistant" }, { "content": " What are the reasons for this classification? Let's think about it step by step. "role": "user" }, { "content": "The website's URL contains numbers with unknown meaning, and the website involves a large number of risky keywords and images. These factors indicate that the website carries a certain level of risk, therefore it is a risky website." "role": "assistant" } ], "images": [ "data / website_CLS_2 / 1.jpg" ] }

[0043] Preferably, special characters (such as "$") are added before and after the URL portion of the input data to help the model more accurately identify the boundaries of the URL.

[0044] A Chinese-supporting multimodal large-scale model, Qwen2-VL-2B, was used as the base model for the first pre-trained multimodal large-scale model, and fine-tuning was performed on it. The required large-scale model fine-tuning framework, LLaMa-Factory, the large-scale model inference acceleration engine, MindIE, and their related dependencies were determined. The weights of the pre-trained Qwen2-VL-2B were determined (e.g., initial weights downloaded from the official website) and loaded into the local environment. LoRA (Low-Rank Adaptation) fine-tuning was performed based on the LLaMa-Factory large-scale model fine-tuning framework, transferring the knowledge contained in the first instruction fine-tuning dataset to the model weights of the first pre-trained multimodal large-scale model, resulting in the fine-tuned model weight parameters. LoRA uses low-rank decomposition to characterize the calculation of the parameter update matrix, thus significantly reducing the number of parameters that need to be trained. Specifically, the original weight matrix of the first pre-trained multimodal large-scale model... The update amount is characterized by the product of two low-rank matrices:

[0045] (1) Where B represents the first low-rank matrix and A represents the second low-rank matrix. , , , This represents a weight matrix with d rows and r columns. This represents a weight matrix with r rows and k columns. The dimension parameter of the low-rank matrix represents the number of rows or columns in the low-rank matrix, and d represents the original weight matrix. The number of rows, k represents the original weight matrix. The number of columns; This represents the task adaptive hyperparameters for the current iteration. This indicates the adaptive hyperparameters for the next iteration, which are adaptively adjusted based on the accuracy of the risk website identification task after fine-tuning. (2) Among them, This represents the task adaptive hyperparameters for the next iteration. This represents the task adaptive hyperparameters for the current iteration. This indicates the accuracy of the first risk website identification model regarding the risk website label (i.e., label 1) in the current iteration after fine-tuning. , This represents the accuracy rate of the risky website tag (i.e., the tag with a value of 1) in the previous iteration. Adjust the speed. , This represents the adjustment factor used to prevent the denominator from being zero, and has a minimum value of 1.0e-7; To prevent extreme values, the function specifically restricts the value of x to a range of 1000. Within a certain range to prevent extreme values ​​from affecting model stability, where x = , The value is 1. Values d represents the original weight matrix The number of rows; k represents the original weight matrix. The number of columns, through The relative rate of change of the model's recognition accuracy can be calculated, allowing for dynamic perception of model performance improvement or decline, and thus adjustment of the values ​​of the task's adaptive hyperparameters.

[0046] The model iteration process is optimized using the above formula. After final tuning, the adaptive hyperparameter of the task is set to 1 to 16, with 8 being the preferred value.

[0047] When the training loss stabilizes, that is, the change in parameter update is less than a set threshold. When the training loss converges, fine-tuning stops. The first low-rank matrix is ​​then monitored and calculated. Second low-rank matrix The Frobenius norm, if it remains below a set threshold This indicates that parameter updates have reached saturation, so fine-tuning updates should be stopped. A threshold is set. The set of values ​​for is {5.0e-3, 1.0e-3, 5.0e-4, 1.0e-4, 5.0e-5}, and after training, 1.0e-4 is preferred. Alternatively, fine-tuning can be stopped when the number of training epochs num_train_epochs reaches the set hyperparameter value (e.g., 5).

[0048] Subsequently, the update values ​​determined by the two low-rank matrices are merged with the original weight matrix using the following expression to generate the final model weights: ,in, This represents the final model weights generated by merging the update values ​​determined by the two low-rank matrices with the original weight matrix. Let B represent the original weight matrix of the first pre-trained multimodal large model; let A represent the first low-rank matrix; and let B represent the second low-rank matrix. The training objective becomes optimizing only the first low-rank matrix. Second low-rank matrix .

[0049] It should be noted that the explanation for A and B is the same as that for expression (1) above, so the explanation of the same part is omitted here.

[0050] After merging the model weights, the first pre-trained multimodal large model is evaluated using a validation dataset to check if the model's accuracy, recall, and other metrics meet expectations, ensuring its performance is consistent with that during fine-tuning. See also Figure 2 The final first-risk website identification model is denoted as "Qwen2-VL-Web-CLS-2". The fine-tuned model is tested on a test dataset, and the five test results with an accuracy rate of over 85% are selected as the final first-risk website identification model (see details). Figure 2 (This is used to identify whether a website is risky).

[0051] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0052] Next, in step S103, based on the obtained multimodal data, multi-class data annotation of websites is performed according to predefined website types to establish a second training dataset. Then, according to the secondary business task, a multi-thinking chain reasoning mode is used, and GaLore fine-tuning technology is combined to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the second risk website identification model.

[0053] Based on the multimodal data obtained in step S101, and according to the website types predefined in Table 1 above, multi-classification data annotation is performed on the websites to establish a second classification dataset. Then, based on the multi-classification business tasks, a second instruction fine-tuning dataset is generated using multi-thinking chain technology for subsequent model training. The second training dataset contains multimodal data labeled with multiple risk website tags, used for detailed annotation of different types of risks.

[0054] Because multi-class classification tasks involve a large number of categories and are quite complex, requiring stronger reasoning and analysis capabilities, a Plan-and-Solve CoT technique is used to generate a second instruction fine-tuning dataset for training a second pre-trained multimodal large model.

[0055] Specifically, a second instruction fine-tuning dataset is generated based on a planning and solving multi-thinking chain. A second inference model (e.g., DeepSeek-R1) is selected as the teacher model. The website multimodal data and classification results from the second classification dataset are input into the second inference model. The planning and solving thinking chain guides the second inference model to generate the reasons for the multi-classification task of risky website types. An instruction fine-tuning dataset for the multi-classification task is constructed, and its multi-classification inference knowledge for risky website types is transferred to the downstream second pre-trained multimodal model.

[0056] Specifically, the second classification dataset for a multi-classification task, divided into risk website types according to a second specified ratio (e.g., 7:1:2), forms a second training set, a second validation set, and a second test set, which are used for fine-tuning, iterative validation, and testing of the second pre-trained multimodal model, respectively.

[0057] A second inference model (e.g., DeepSeek-R1) is selected as the teacher model. Website multimodal data and multi-classification results from the second classification dataset are used as input data to the second inference model. A planning and solving mind chain guides the model to generate reasons for the multi-classification task of risk website types. This planning and solving mind chain consists of multiple sub-tasks decomposed for each risk website classification task. It guides the second inference model to generate analytical answers and classification reasons corresponding to these sub-tasks, thus generating a planning and solving mind chain corresponding to the model input data—that is, a multi-classification reasoning mind chain—to further generate (i.e., construct) the second task instruction fine-tuning dataset. Specifically, its multi-classification inference knowledge for risk website types is transferred to the downstream second pre-trained multimodal model.

[0058] Specifically, for the multi-classification problem of risky website types, previous experiments have shown that directly allowing a large model to predict the type can lead to significant errors or illusions. Therefore, a planning and solving chain technique is adopted to decompose complex tasks or problems into multiple sub-tasks or sub-problems, guiding the large inference model to predict the correct risk category. Based on the question template "Q: [Question]. A: [Trigger]", combined with multi-dimensional website information, including website URLs and hyperlinks, extracted text, and images, the task planning process and sub-problem decomposition prompts are defined in the trigger sentence section. For example, given website data in the second training set directory "data / website_CLS_Multi / 1", including the website URL "https: / / 101.32.23.152" and hyperlinks... <hyperlink>Extracting text from a website <text>and website snapshot images And website multi-category tags: 5 risky software download websites, set the following questions to input the reasoning model to generate classification reasons: Q: The image is a screenshot of a snapshot of the website https: / / 101.32.23.152, and the hyperlinks extracted from that website are... <hyperlink>The text content is <text>Please analyze why it was identified as a "risky software download website".

[0059] A: Let's think about it step by step.

[0060] Task planning: Decompose the current problem into multiple sub-problems and answer them separately. Based on the comprehensive analysis of the answers to each sub-problem, obtain the final answer to the current problem.

[0061] Decomposition of subproblems: Question 1: Please check the website's URL structure and hyperlinks, including whether it contains risky characters (such as %20, @, or meaningless numeric strings), and whether it imitates a certain brand but contains spelling errors.

[0062] Question 2: Please analyze the website's text content, extract risky keywords (including whether it contains high-risk words) and perform semantic analysis (including whether it asks users to provide sensitive information, contains instructions to download attachments or click links, whether the text is consistent with the website's functions, and whether the text content involves risky industries, etc.).

[0063] Question 3: Please analyze the website's image content and perform visual identification, including whether it imitates a brand's logo / interface, whether it contains sensitive images, whether it contains floating ads, and whether the website's text and image content are consistent.

[0064] Question 4: Based on the above three questions, analyze why this website was identified as a <risky software download website>. Based on this prompt, the large inference model generates a classification cause analysis, which, after being processed using a standardized format, yields the following output: "Question 1: URL structure and hyperlink analysis: URLs containing meaningless random number combinations (such as `101.32.23.152`) are used to circumvent keyword filtering or quickly register temporary sites; hyperlinks pointing to hidden paths (` / login?ref=risk`) further increase the risk; hyperlinks containing a message-level domain (TLD) are common suffixes used by high-risk businesses or organizations, such as high-risk registration platforms."

[0065] Question 2: Text content analysis: Contains directly risky words such as "modifier", "cracked version", "emulator", etc.; Contains hidden guiding words such as "free unlock" and "member exclusive".

[0066] Question 3: Image content analysis: Contains visually sensitive features, the page is full of pop-up ads (such as "limited time free"), the ad images are high-profile, and some images are not fully displayed.

[0067] Question 4: Reasons for being classified as a "risky software download website": This website meets the core characteristics of a risky software download website because its domain name contains meaningless random characters, its text contains keywords that guide users to download risky software, its images contain images, and its floating advertisements are also risky software download websites. Therefore, it has been classified as a risky software download website. Furthermore, the multi-category cause-and-effect thought chain generated by the second inference model (i.e., the teacher model) and multiple risk website tags from the model input data are combined to generate (i.e., construct) the second task instruction fine-tuning dataset. (See [link to relevant documentation]). Figure 3 .

[0068] The Qwen2-VL-7B multimodal large model supporting Chinese is used as the base model for the second risky website identification model. Based on the second task instruction fine-tuning dataset obtained above, the second risky website identification model is fine-tuned. Compared to Qwen2-VL-2B, the second risky website identification model has more parameters and is more suitable for more complex tasks. Similar to the previous step, the required model running framework is installed and deployed locally. The base model is fine-tuned and weights are merged using GaLore (Gradient Low-Rank Projection) fine-tuning technology to obtain the second risky website identification model (e.g., represented as "Qwen2-VL-Web-CLS-Multi"). The core step of GaLore is to perform low-rank projection on the gradient weight matrix (unlike LoRA which targets the original weight matrix), using low-rank gradients to represent the gradient matrix, and preserving the original gradient through singular value decomposition (SVD). One principal component is used to reduce memory consumption without sacrificing model performance.

[0069] Specifically, this is reflected in the following steps: For the original gradient weight matrix... ( Let represent a gradient weight matrix with d rows and k columns. Performing a complete SVD decomposition, the resulting expression is: ,in, for orthogonal matrix, , This represents a gradient weight matrix with d rows and k columns. for An orthogonal matrix of dimension 1 , This represents a gradient weight matrix with k rows and k columns. The elements on the diagonal of a diagonal matrix are called a matrix. singular values, , This represents a gradient weight matrix with k rows and k columns. Sorting the singular values ​​from largest to smallest, retaining the top... The lowest-rank representation is obtained by finding the largest singular value and its corresponding singular vector, and is expressed by the following expression:

[0070]

[0071] in, Represents the original gradient weight matrix The low-rank characterization value, express The positive definite matrix of the low-rank singular value decomposition of dimension 1. , This indicates that the number of rows is d and the number of columns is The gradient weight matrix; express The diagonal matrix of the low-rank singular value decomposition of dimension . , Indicates the number of rows. The number of columns is The gradient weight matrix; It means The transpose of the positive definite matrix of a low-rank singular value decomposition of dimension 1. , Indicates the number of rows. The number of columns is The gradient weight matrix, where, This represents the number of singular values ​​retained and is used as a dimension parameter of the low-rank matrix. Specifically, it is the number of rows or columns of the low-rank matrix, with a value ranging from 1 to 18, and a preferred value of 16.

[0072] when When the dimensionality parameter is larger, the low-rank gradient retains more complete gradient information, but the computational cost is higher; when the dimensionality parameter is larger... The smaller the value, the lower the storage and computational complexity of the gradient matrix. Down to ,therefore The value of is crucial, where d represents the number of rows in the original gradient weight matrix and k represents the number of columns in the original gradient weight matrix. The method for determining the value is the same as the method for selecting the task adaptive hyperparameter r mentioned above, but the task accuracy rate used for calculation is the average accuracy rate of all risk categories (excluding risk-free websites with a label of 0). After final optimization... Set it to 16.

[0073] In addition, the parameter galore_update_interval is set to update the step interval of the GaLore projection, with a default value of 200; the parameter galore_scale is the scaling factor of GaLore, with a default value of 0.25.

[0074] When the training loss stabilizes, i.e., the magnitude of change is below a set threshold. At this point, the training loss converges. The Frobenius norm of the low-rank gradient after low-rank projection is monitored; if the monitored low-rank gradient remains below a set threshold, i.e. This indicates that parameter updates within the current subspace are saturated, so fine-tuning updates should be stopped, i.e., fine-tuning of the base model of the second risk website identification model should cease. Alternatively, fine-tuning updates should be stopped when the number of training epochs (num_train_epochs) reaches the set hyperparameter value of 5. Hyperparameter threshold. The set of values ​​for is {5.0e-3, 1.0e-3, 5.0e-4, 1.0e-4, 5.0e-5}, and 5.0e-4 is preferred based on the accuracy of risk website identification.

[0075] Subsequently, the parameter weights are merged, and the parameter weights are updated based on the low-rank gradient using the following expression:

[0076] in, This represents the final model weights generated after updating the original weight matrix using low-rank gradients. This represents the original weight matrix of the second pre-trained multimodal large model; The hyperparameter learning rate is taken in the range {5.0e-4, 1.0e-4, 5.0e-5, 1.0e-5, 5.0e-6, 1.0e-6}, preferably 5.0e-6; Represents the original gradient weight matrix The low-rank characterization of the gradient value.

[0077] After merging the model weights, the merged model is validated. The second validation set is used to evaluate the second pre-trained multimodal model, checking whether the model's accuracy, recall, and other metrics meet expectations, ensuring its performance is consistent with that during fine-tuning. The fine-tuned weights are then merged with the weights of the second pre-trained multimodal large model to obtain the final second risky website identification model (e.g., denoted as "Qwen2-VL-Web-CLS-Multi"). The fine-tuned model is tested based on the second test set, and the five sets with the highest test accuracy (above 85%) are selected as the final second risky website identification model.

[0078] In addition, for new types of risky websites, continuous data updates and model iterations are needed to improve the system's adaptability and real-time risk identification capabilities.

[0079] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0080] Next, in step S104, the current website data is received based on the model inference service call interface, the first risk website identification model is input, and the website identification result is obtained.

[0081] For the first risk website identification model, deploy the finely tuned and saved model "Qwen2-VL-Web-CLS-2" in the MindIE inference image and configure the inference service call interface.

[0082] Obtain multidimensional website information of the website to be predicted, perform data preprocessing to obtain multimodal website data (i.e., multimodal data) as described above, input it into the first risk website identification model (i.e. model "Qwen2-VL-Web-CLS-2"), and predict the classification result of "risk-free website" or "risky website".

[0083] For primary classification tasks (i.e., binary classification tasks), design a templated prompt text input. This should include the definition of the role in the multimodal large model, the binary classification category of the risky website, the description of the binary classification task, the website data input, and the model output format. For example, given a website URL of "http: / / owcyz.care", an example of the specific format for the model input is as follows:

[0084] { Role: You are a risk website monitor, responsible for identifying risky websites on the Internet.

[0085] Category: Risky websites are categorized as follows: 0 - No risk; 1 - Risky.

[0086] Goal: When a user enters a website URL and a screenshot of the website's snapshot, please determine the appropriate action based on the defined Category. The website's category.

[0087] Input: The image is a screenshot of a website, $http: / / owcyz.care$.

[0088] Output: Please return only the following JSON format, for example: {"CLS": "1 Risky Website"\}.

[0089] }".

[0090] Based on the model's JSON output, the website identification result for the website to be predicted (or the current website) is obtained.

[0091] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0092] Next, in step S105, when the obtained website identification result is a risky website, the current website data is input into the second risky website identification model to obtain the risky website category result.

[0093] For the second risk website identification model, specifically deploy the finely tuned and saved model "Qwen2-VL-Web-CLS-Multi" in the MindIE inference image and configure the calling interface of the inference service.

[0094] When the website identification result is a non-risk website (i.e., "0 No Risk Website"), no processing is performed. When the website identification result is a risk website (i.e., "1 Risky Website"), the multimodal data of the current website and the website identification result (i.e., the current website data) are input into the second risk website identification model to obtain the specific risk website type result. A templated prompt text input needs to be designed for multi-classification tasks. For example, if a website URL is "http: / / owcyz.care" and its website identification result is "1 Risky Website", an example of the specific input format for this website is as follows:

[0095] { Role: You are a risk website monitor, responsible for identifying risky websites on the Internet.

[0096] Category: Categories of risky websites and their definitions: 0. Risk-free websites (risk-free or safe websites); 1. Risky department websites (risky websites related to relevant departments); ...; 17. Other risky websites (other risky websites not included in the above types).

[0097] Goal: When a user enters a website URL and hyperlinks, a website snapshot screenshot, and website text content, please determine the risk category of the website based on the defined Category.

[0098] Input: The image is a screenshot of a snapshot of the website $https: / / 09n25k.660xi.com$, from which hyperlinks were extracted. <hyperlink>The text content is <text>.

[0099] Output: Please return the results in the following format, for example: \{"CLS": "1 Risk Department Website"\}.

[0100] }".

[0101] The categories and definitions of risky websites are shown in Table 1, "Multi-level Classification of Risky Websites." The risky website category results are obtained based on the model's JSON output.

[0102] In the model prediction and inference stage, after inputting the current website information, the probability value of the current website belonging to each risk category will be calculated based on the extracted website features. The risk category with the highest predicted probability will be selected as the risk website type determination result for the current website, thereby realizing the automatic identification and classification of risk websites.

[0103] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0104] Furthermore, the accompanying drawings are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes shown in the drawings do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0105] Compared with existing technologies, this invention constructs a progressive two-level risk website identification model based on multimodal data such as text, images, and videos, thereby further improving identification efficiency and accuracy. It breaks through the limitations of traditional single-modal identification, realizes comprehensive and in-depth feature mining of risk websites, and thus significantly improves the accuracy of risk website identification in complex scenarios.

[0106] Furthermore, by integrating multimodal data such as text, images, and audio from a website, it breaks through the limitations of traditional single-modal recognition. It can simultaneously analyze guiding words in web page text, risk symbols or advertising images in images, and risk warnings contained in audio, comprehensively and accurately characterizing website features.

[0107] In addition, a multimodal large model technology framework is introduced, specifically to comprehensively process risk website information using a multimodal large model. This framework integrates multi-source data such as text and images, and simultaneously inputs multimodal information from images and text into the multimodal large model. This differs from models that input predefined independent modalities separately, allowing for fine-tuning and training of multimodal data while breaking the isolation of traditional single-modal analysis.

[0108] Furthermore, customized prompts (i.e., Prompts) were designed for risk website identification tasks. By precisely guiding the model to focus on key information, such as the type of risky website and the correlation of multimodal data, the multimodal big data model was guided to explore the correlation between website type and website features, as well as the hidden correlations and features between various modal data. For example, it was used to identify whether the text and image content were consistent and to make a comprehensive judgment based on the combination of text and image information. This fully utilized the powerful modality processing and semantic understanding capabilities of the multimodal big data model, significantly improving the model's ability to understand and judge complex risk scenarios.

[0109] Example 2 The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.

[0110] Figure 4 This is a schematic diagram of an example of the risk website identification device of the present invention.

[0111] Reference Figure 4 The second aspect of this disclosure provides a risky website identification device that performs the risky website identification method described in the first aspect of the invention. The risky website identification device 500 includes a data collection module 510, a first construction module 520, a second construction module 530, a first determination module 540, and a second determination module 550.

[0112] Specifically, the data collection module 510 collects multi-dimensional website information to form multimodal data. The first construction module 520, based on the obtained multimodal data, performs binary classification data annotation of websites according to predefined website types to establish a first training dataset. Based on the primary business task, it uses zero-shot thought chain technology to generate a first instruction fine-tuning dataset, and uses LoRA fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain a first risk website identification model. The second construction module 530, based on the obtained multimodal data, performs multi-class classification data annotation of websites according to predefined website types to establish a second training dataset. Based on the secondary business task, it uses a multi-thought chain inference mode and combines GaLore fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain a second risk website identification model. The first determination module 540, based on the model inference service call interface, receives the current website data, inputs it into the first risk website identification model, and obtains the website identification result. The second determination module 550, when the obtained website identification result is a risk website, inputs the current website data into the second risk website identification model to obtain the risk website category result.

[0113] According to an optional implementation, the multimodal large model Qwen2-VL-2B is used as the base model of the first pre-trained multimodal large model. The base model of the first pre-trained multimodal large model is fine-tuned, and the knowledge contained in the first instruction fine-tuning dataset is transferred to the model weights of the first pre-trained multimodal large model. The original weight matrix of the first pre-trained multimodal large model is shown below. The update amount is characterized by the product of two low-rank matrices:

[0114]

[0115] in, This represents the original weight matrix of the first pre-trained multimodal large model. The update amount; B represents the first low-rank matrix; A represents the second low-rank matrix. , , , This represents a weight matrix with d rows and r columns. This represents a weight matrix with r rows and k columns. The dimension parameter of the low-rank matrix is ​​represented by the number of rows or columns in the low-rank matrix, and d represents the original weight matrix. The number of rows, k represents the original weight matrix. The number of columns; This represents the task adaptive hyperparameters for the current iteration. This indicates the adaptive hyperparameters for the next iteration, which are adaptively adjusted based on the accuracy of the risk website identification task after fine-tuning. in, This represents the task adaptive hyperparameters for the next iteration. This represents the task adaptive hyperparameters for the current iteration. This indicates the accuracy of the first risk website identification model regarding the risk website label (i.e., label 1) in the current iteration after fine-tuning. , This represents the accuracy of the risk website label (i.e., label 1) in the previous iteration, where λ is the adjustment rate. , This represents the adjustment factor used to prevent the denominator from being zero, and has a minimum value of 1.0e-7; To prevent extreme values, the function specifically restricts the value of x to a range of 1000. Within a certain range to prevent extreme values ​​from affecting model stability, where x = , The value is 1. Values d represents the original weight matrix The number of rows; k represents the original weight matrix. The number of columns, through The relative rate of change of the model's recognition accuracy can be calculated, allowing for dynamic perception of model performance improvement or decline, and thus adjustment of the values ​​of the task's adaptive hyperparameters.

[0116] According to an optional implementation, the step of establishing a first training dataset and generating a first instruction fine-tuning dataset using zero-shot thought chain technology based on a primary business task includes: selecting a first inference big model as the teacher model, extracting website information and classification results from the first training dataset as model input data to input into the first inference big model, the first inference big model will generate a thought chain to answer questions, guiding the first inference model to generate reasons for risk website classification, specifically using the first inference big model to generate inference prompt information with classification for various risk website types, so as to generate a thought chain to answer questions corresponding to the model input data, i.e., a binary classification reason thought chain.

[0117] The generated binary cause thought chain and the risk website tags in the model input data are further combined to construct the first instruction fine-tuning dataset.

[0118] According to an optional implementation, the first low-rank matrix is ​​expressed using the following expression. Second low-rank matrix The update values ​​determined by the two low-rank matrices are merged with the original weight matrix to generate the final model weights: ,in, This represents the final model weights generated by merging the update values ​​determined by the two low-rank matrices with the original weight matrix. Let B represent the original weight matrix of the first pre-trained multimodal large model; let A represent the first low-rank matrix; and let B represent the second low-rank matrix. The training objective becomes optimizing only the first low-rank matrix. Second low-rank matrix .

[0119] According to an optional implementation, the step of using a multi-thinking chain reasoning mode based on the secondary business task, and combining GaLore fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the second risk website identification model includes: using the multimodal large model Qwen2-VL-7B as the base large model of the second risk website identification model, and using the generated second task instruction fine-tuning dataset to fine-tune the base large model of the second risk website identification model.

[0120] A low-rank projection is performed on the gradient weight matrix, and the low-rank gradient is used to represent the gradient matrix. The norm of the low-rank gradient after low-rank projection is monitored. If the monitored low-rank gradient remains below a set threshold, i.e. If the parameter update is saturated in the current subspace, then stop fine-tuning the update, that is, stop fine-tuning the base model of the second risk website identification model.

[0121] According to the optional implementation method, the second inference big model is selected as the teacher model. The website multimodal data and multi-classification results in the second classification dataset are used as the model input data to input the second inference big model. Combined with the planning and solving thinking chain, the second inference model is guided to generate the reasons for the multi-classification task of risk website types. The planning and solving thinking chain is a series of sub-tasks decomposed for each risk website classification task. The second inference model is guided to generate analysis answers and classification reasons corresponding to the multiple sub-tasks, so as to generate the planning and solving thinking chain corresponding to the model input data, that is, the multi-classification reason thinking chain.

[0122] The generated multi-category cause thought chain and multiple risk website tags in the model input data are further combined to construct the second instruction fine-tuning dataset.

[0123] According to an optional implementation, the parameter weights are updated based on the low-rank gradient using the following expression:

[0124] in, This represents the final model weights generated after updating the original weight matrix using low-rank gradients. This represents the original weight matrix of the second pre-trained multimodal large model; The learning rate for the second pre-trained multimodal model; Represents the original gradient weight matrix The low-rank characterization of the gradient value.

[0125] According to the optional implementation, the model input for the first risk website identification model includes the following restrictions: the image file format in the modal data is adjusted to JPEG, the size of a single image file does not exceed 10 MB; the image width and height are both greater than 10 pixels, the maximum aspect ratio is less than 200:1 or 1:200, and the total number of image pixels is less than 12 million.

[0126] It should be noted that the risk website identification method performed by the risk website identification device in Embodiment 2 is the same as that in Embodiment 1, therefore, the description of the same parts is omitted.

[0127] Compared with existing technologies, this invention constructs a progressive two-level risk website identification model based on multimodal data such as text, images, and videos, thereby further improving identification efficiency and accuracy. It breaks through the limitations of traditional single-modal identification, realizes comprehensive and in-depth feature mining of risk websites, and thus significantly improves the accuracy of risk website identification in complex scenarios.

[0128] Furthermore, by integrating multimodal data such as text, images, and audio from a website, it breaks through the limitations of traditional single-modal recognition. It can simultaneously analyze guiding words in web page text, risk symbols or advertising images in images, and risk warnings contained in audio, comprehensively and accurately characterizing website features.

[0129] In addition, a multimodal large model technology framework is introduced, specifically to comprehensively process risk website information using a multimodal large model. This framework integrates multi-source data such as text and images, and simultaneously inputs multimodal information from images and text into the multimodal large model. This differs from models that input predefined independent modalities separately, allowing for fine-tuning and training of multimodal data while breaking the isolation of traditional single-modal analysis.

[0130] Furthermore, customized prompts (i.e., Prompts) were designed for risk website identification tasks. By precisely guiding the model to focus on key information, such as the type of risky website and the correlation of multimodal data, the multimodal big data model was guided to explore the correlation between website type and website features, as well as the hidden correlations and features between various modal data. For example, it was used to identify whether the text and image content were consistent and to make a comprehensive judgment based on the combination of text and image information. This fully utilized the powerful modality processing and semantic understanding capabilities of the multimodal big data model, significantly improving the model's ability to understand and judge complex risk scenarios.

[0131] Figure 5 This is a schematic diagram of an embodiment of an electronic device according to the present invention.

[0132] like Figure 5 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0133] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0134] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0135] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0136] It should be understood that Figure 5 The electronic device shown is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0137] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 6 As shown, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several commands to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of the present invention.

[0138] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0139] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with a command execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0140] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0141] The aforementioned computer-readable medium carries one or more programs (e.g., computer executable programs) that, when executed by a device, cause the computer-readable medium to implement the methods of this disclosure.

[0142] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified to be uniquely different from one or more devices in this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0143] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several commands to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of the present invention.

[0144] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.< / text> < / hyperlink> < / text> < / hyperlink> < / text> < / hyperlink>

Claims

1. A method for identifying risky websites, characterized in that, include: Collect information from multiple websites to form multimodal data; Based on the obtained multimodal data, binary classification data labeling of websites is performed according to predefined website types to establish the first training dataset. According to the first-level business task, the first instruction fine-tuning dataset is generated using zero-shot thinking chain technology. LoRA fine-tuning technology is used to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the first risk website identification model. Based on the obtained multimodal data, multi-classification data annotation of websites is performed according to predefined website types to establish a second training dataset. Then, according to the secondary business tasks, a multi-thought chain reasoning model is used, combined with GaLore fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the large multimodal model, resulting in a second risky website identification model, including: The multimodal large model Qwen2-VL-7B is used as the base large model of the second risk website identification model. The base large model of the second risk website identification model is fine-tuned using the generated second task instruction fine-tuning dataset. Perform a low-rank projection on the gradient weight matrix and use the low-rank gradient to characterize the gradient matrix. Monitor the norm of the low-rank gradient after low-rank projection. If the monitored low-rank gradient remains below a set threshold, i.e. If this indicates that parameter updates within the current subspace are saturated, then fine-tuning updates should be stopped, meaning fine-tuning of the base model of the second risk website identification model should cease; further including: The parameter weights are updated based on the low-rank gradient using the following expression: in, This represents the final model weights generated after updating the original weight matrix using a low-rank gradient. This represents the original weight matrix of the second pre-trained multimodal large model; The learning rate for the second pre-trained multimodal model; Represents the original gradient weight matrix The low-rank characterization of the gradient value; Based on the model inference service call interface, the current website data is received, the first risk website identification model is input, and the website identification result is obtained; When the obtained website identification result is a risky website, the current website data is input into the second risky website identification model to obtain the risky website category result.

2. The risky website identification method according to claim 1, characterized in that, The method of using LoRA fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on a multimodal large model includes: The multimodal large model Qwen2-VL-2B is used as the base model of the first pre-trained multimodal large model. The base model of the first pre-trained multimodal large model is fine-tuned, and the knowledge contained in the first instruction fine-tuning dataset is transferred to the model weights of the first pre-trained multimodal large model. The original weight matrix of the first pre-trained multimodal large model The update amount is characterized by the product of two low-rank matrices: in, This represents the original weight matrix of the first pre-trained multimodal large model. The update amount; B represents the first low-rank matrix; A represents the second low-rank matrix. , , , This represents a weight matrix with d rows and r columns. This represents a weight matrix with r rows and k columns. The dimension parameter of the low-rank matrix represents the number of rows or columns in the low-rank matrix, and d represents the original weight matrix. The number of rows, k represents the original weight matrix. The number of columns; This represents the task adaptive hyperparameters for the current iteration. This indicates the adaptive hyperparameters for the next iteration, which are adaptively adjusted based on the accuracy of the risk website identification task after fine-tuning. in, This represents the task adaptive hyperparameters for the next iteration. This represents the task adaptive hyperparameters for the current iteration. This indicates the accuracy of the first risk website identification model regarding the risk website label (i.e., label 1) in the current iteration after fine-tuning. , This represents the accuracy rate of the risky website tag (i.e., the tag with a value of 1) in the previous iteration. To adjust the rate, , This represents the adjustment factor used to prevent the denominator from being zero, and has a minimum value of 1.0e-7; To prevent extreme values, the function specifically restricts the value of x to a range of 1000. Within a certain range to prevent extreme values ​​from affecting model stability, where x = , The value is 1. Values d represents the original weight matrix The number of rows; k represents the original weight matrix. The number of columns, through The relative rate of change of the model's recognition accuracy is calculated to adjust the values ​​of the task-adaptive hyperparameters.

3. The risk website identification method according to claim 2, characterized in that, The establishment of the first training dataset and the generation of the first instruction fine-tuning dataset using zero-shot mind chain technology based on the primary business task include: The first reasoning model is selected as the teacher model. The website information and classification results in the first training dataset are extracted as the model input data and input into the first reasoning model. The first reasoning model will generate a thought chain to answer the question and guide the first reasoning model to generate the reason for classifying risky websites. Specifically, the first reasoning model can generate reasoning prompts for classifying various types of risky websites to generate a thought chain to answer the question corresponding to the model input data, that is, a binary classification reason thought chain. The generated binary cause thought chain and the risk website tags in the model input data are further combined to construct the first instruction fine-tuning dataset.

4. The risky website identification method according to claim 2, characterized in that, Further includes: The first low-rank matrix is ​​expressed using the following expression. Second low-rank matrix The update values ​​determined by the two low-rank matrices are merged with the original weight matrix to generate the final model weights: ,in, This represents the final model weights generated by merging the update values ​​determined by the two low-rank matrices with the original weight matrix. B represents the original weight matrix of the first pre-trained multimodal large model; A represents the first low-rank matrix; B represents the second low-rank matrix. The training objective becomes optimizing only the first low-rank matrix. Second low-rank matrix .

5. The risky website identification method according to claim 1, characterized in that, Further includes: The second inference model is selected as the teacher model. The website multimodal data and multi-classification results in the second classification dataset are used as the model input data to input the second inference model. The planning and solving thinking chain guides the model to generate the reasons for the multi-classification task of risk website types. The planning and solving thinking chain is a series of sub-tasks decomposed for each risk website classification task. The second inference model is guided to generate analysis answers and classification reasons corresponding to the multiple sub-tasks, so as to generate the planning and solving thinking chain corresponding to the model input data, that is, the multi-classification reason thinking chain. The generated multi-category cause thought chain and multiple risk website tags in the model input data are further combined to construct the second instruction fine-tuning dataset.

6. The risky website identification method according to claim 1, characterized in that, Further includes: The model input for the first-risk website identification model includes the following constraints: In the modal data, the image file format is adjusted to JPEG, the size of a single image file does not exceed 10 MB, the image width and height are both greater than 10 pixels, the maximum aspect ratio is less than 200:1 or 1:200, and the total number of pixels in the image is less than 12 million.

7. A risky website identification device, characterized in that, The method for identifying risky websites according to any one of claims 1 to 6, wherein the risky website identification device comprises: The data collection module is used to collect multi-dimensional website information and form multimodal data; The first construction module, based on the obtained multimodal data, performs binary classification data labeling of websites according to predefined website types to establish the first training dataset. According to the first-level business task, the first instruction fine-tuning dataset is generated using zero-shot thinking chain technology. The LoRA fine-tuning technology is used to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the first risk website identification model. The second construction module, based on the obtained multimodal data, performs multi-classification data annotation of websites according to predefined website types to establish a second training dataset. Based on the secondary business tasks, it uses a multi-thinking chain reasoning mode and combines GaLore fine-tuning technology to perform instruction-supervised fine-tuning and weight merging on the multimodal large model to obtain the second risk website identification model. The first determination module, based on the model inference service call interface, receives the current website data, inputs the first risk website identification model, and obtains the website identification result; The second determination module, when the obtained website identification result is a risky website, inputs the current website data into the second risky website identification model to obtain the risky website category result.

8. The risk website identification device according to claim 7, characterized in that, include: The multimodal large model Qwen2-VL-2B is used as the base model of the first pre-trained multimodal large model. The base model of the first pre-trained multimodal large model is fine-tuned, and the knowledge contained in the first instruction fine-tuning dataset is transferred to the model weights of the first pre-trained multimodal large model. The original weight matrix of the first pre-trained multimodal large model The update amount is characterized by the product of two low-rank matrices: in, This represents the original weight matrix of the first pre-trained multimodal large model. Update volume; The parameter represents the dimension of the low-rank matrix, where B represents the first low-rank matrix and A represents the second low-rank matrix. , , , This represents a weight matrix with d rows and r columns. This represents a weight matrix with r rows and k columns. The dimension parameter of the low-rank matrix is ​​denoted by d, and the original weight matrix is ​​d. The number of rows, k represents the original weight matrix. The number of columns; This represents the task adaptive hyperparameters for the current iteration. This indicates the adaptive hyperparameters for the next iteration, which are adaptively adjusted based on the accuracy of the risk website identification task after fine-tuning. in, This represents the task adaptive hyperparameters for the next iteration. This represents the task adaptive hyperparameters for the current iteration. This indicates the accuracy of the first risk website identification model regarding the risk website label (i.e., label 1) in the current iteration after fine-tuning. , This represents the accuracy rate of the risky website tag (i.e., the tag with a value of 1) in the previous iteration. , , This represents the adjustment factor used to prevent the denominator from being zero, and has a minimum value of 1.0e-7; To prevent extreme values, the function specifically restricts the value of x to a range of 1000. Within a certain range to prevent extreme values ​​from affecting model stability, where x = , The value is 1. Values d represents the original weight matrix The number of rows; k represents the original weight matrix. The number of columns, through The relative rate of change of the model's recognition accuracy is calculated to adjust the values ​​of the task-adaptive hyperparameters.

Images