Risky website detection method and apparatus based on large language model

Abstract

Provided are a risky website detection method and apparatus based on a large language model, the method comprising: inputting website information into a detection model to obtain a discrimination result, which is output by the detection model, so as to determine whether a website to be subjected to detection is an impersonation website, wherein the detection model is obtained by constructing a reward function by means of a group-wise reciprocal rank index, constructing unsupervised contrastive feedback on the basis of a set of similar websites, and training a large language model. The present application efficiently and accurately identifies and prevents impersonation cyber fraud activities by using strong semantic understanding and generation capabilities of the large language model and in view of a group-wise contrastive feedback reinforcement learning mechanism and fine-grained analysis of latent semantic enhancement.

Description

A Method and Apparatus for Detecting Risky Websites Based on Large Language Models

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411482697.0, filed on October 23, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of information security technology, specifically to a method and apparatus for detecting risky websites based on a large language model. Background Technology

[0004] With the advent of the Internet age, as the Internet becomes a way of life for the people, various hidden risks have emerged, especially fraudulent online resources (fraudulent websites, fraudulent apps, etc.), which have a wide reach, cause great harm, and change rapidly, bringing new challenges to network security governance.

[0005] Currently, methods for detecting internet fraud websites primarily rely on traditional machine learning and deep learning models, such as support vector machines, random forests, and neural networks. These models typically analyze static features of website content (e.g., domain name, page layout, URL structure) and dynamic behaviors (e.g., user interaction, traffic patterns). While these methods perform well with simple fraudulent data, they have limitations in understanding and processing complex logical reasoning (especially in impersonation scams). For example, traditional models may fail to accurately capture semantic nuances and potential deceptive hints in the text content of web pages, making them prone to failure when faced with elaborately disguised fraudulent websites. Summary of the Invention

[0006] This application provides a method and apparatus for detecting risky websites based on a large language model.

[0007] This application provides a risky website detection method based on a large language model, the method comprising:

[0008] Obtain website information of the website to be tested;

[0009] The website information is input into the detection model to obtain the discrimination result output by the detection model, so as to determine whether the website to be detected is a counterfeit website;

[0010] The construction process of the detection model includes the following operations:

[0011] Construct a reward function;

[0012] Based on the reward function, an unsupervised comparison feedback signal is constructed;

[0013] The detection model is obtained by training the original detection model based on the unsupervised comparison feedback signal.

[0014] In one embodiment, constructing the reward function includes:

[0015] The reward function is constructed using the group reciprocal rank index.

[0016] In one embodiment, the group reciprocal rank index is obtained as follows:

[0017] Based on the original detection model, the model output corresponding to any website in the suspected fraud website corpus is obtained, as well as the model output corresponding to other websites in the suspected fraud website corpus.

[0018] The group reciprocal rank index is obtained based on the similarity between the model output corresponding to any one website and the model output corresponding to the other websites.

[0019] In one embodiment, constructing an unsupervised contrastive feedback signal based on the reward function includes:

[0020] Based on the reward function, the penalty term, and the hyperparameters used to balance the group reciprocal rank index and the penalty term, the unsupervised contrastive feedback signal is constructed.

[0021] The penalty term is used to measure the relative entropy between the trained detection model and the original detection model.

[0022] In one embodiment, the discrimination result includes the discrimination result of the backbone information;

[0023] Accordingly, the website information is input into the detection model to obtain the discrimination result output by the detection model, including:

[0024] The website information and the first prompt word template are input into the detection model to obtain the backbone information;

[0025] The main information, supplementary main search content, and discrimination prompt template are input into the detection model to obtain the discrimination result of the main information.

[0026] In one embodiment, the discrimination result further includes discrimination results with fine-grained information;

[0027] Accordingly, the website information is input into the detection model to obtain the discrimination result output by the detection model, including:

[0028] When only one piece of fine-grained information is included, the operation of obtaining the discrimination result of the fine-grained information includes:

[0029] The website information, the main information, and the second prompt word template are input into the detection model to obtain the fine-grained information;

[0030] The fine-grained information, the fine-grained search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the fine-grained information;

[0031] When i ≥ 2, the operation of obtaining the discrimination result of the i-th fine-grained information includes:

[0032] The website information, the main information, the second prompt word template, and the previous fine-grained information are input into the detection model to obtain the i-th fine-grained information;

[0033] The i-th fine-grained information, the i-th fine-grained search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the i-th fine-grained information.

[0034] In one embodiment, the discrimination result further includes the discrimination result of the implicit information;

[0035] Accordingly, the website information is input into the detection model to obtain the discrimination result output by the detection model, including:

[0036] The website information, the backbone information, the i-th fine-grained information, and the third prompt word template are input into the detection model to obtain the hidden information;

[0037] The implicit information, the implicit search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the implicit information.

[0038] In one embodiment, determining whether the website to be detected is a counterfeit website includes:

[0039] Based on at least one of the discrimination results of the backbone information, the discrimination results of the fine-grained information, and the discrimination results of the implicit information, a comprehensive discrimination model is constructed to determine whether the website to be detected is a counterfeit website. Optionally, obtaining the website information of the website to be detected includes:

[0040] The website information is obtained by semantically extracting the website content of the website to be detected using a web crawler tool.

[0041] The website information includes, but is not limited to, page titles, body text, link text, and script information; the website content includes, but is not limited to, webpage content, Uniform Resource Locator (URL) structure, domain registration information, and metadata.

[0042] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described risk website detection methods based on a large language model.

[0043] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the risk website detection methods based on large language models described above.

[0044] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the risk website detection methods based on large language models described above. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this application or the current related technologies, the accompanying drawings used in the description of the embodiments or the current related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 is a flowchart of the risk website detection method based on a large language model provided in an embodiment of this application;

[0047] Figure 2 is a schematic diagram of the model training process provided in the embodiment of this application;

[0048] Figure 3 is a schematic diagram illustrating the application of the detection model provided in the embodiments of this application;

[0049] Figure 4 is a schematic diagram of the physical structure of the electronic device provided in the embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Current technical solutions suffer from the following drawbacks: large language models struggle to understand implicit information and deeper meanings within text, and their ability to identify social engineering tactics used by fraudulent websites is limited. Knowledge base-based methods for accurately identifying counterfeit websites are costly and time-consuming. Due to the uninterpretable nature of the models and their dependence on datasets, they may only provide a judgment on whether a website is fraudulent, without offering specific criteria. Current models may fail to accurately capture subtle semantic nuances and potential deceptive cues, making them prone to failure when faced with sophisticatedly disguised fraudulent websites. Furthermore, current technologies lack sufficient logical and common-sense reasoning capabilities beyond the current training set when dealing with novel fraudulent techniques, hindering their ability to effectively identify unseen deception patterns.

[0052] To address the aforementioned issues, embodiments of this application provide a risk website detection method based on a large language model. In practical applications, this method is primarily used in anti-fraud intelligent systems in the public security field. It integrates the language generation capabilities and fine-grained semantic information understanding capabilities of a large language model, and significantly improves the accuracy and intelligence level of detecting counterfeit and fraudulent websites after group comparison feedback.

[0053] Figure 1 is a flowchart of the risky website detection method based on a large language model provided in an embodiment of this application. As shown in Figure 1, the method includes:

[0054] Operation 100: Obtain website information of the website to be tested.

[0055] In the process of obtaining website information of the website to be tested, web crawling tools can be used to extract semantics from the website content of the website to obtain the website information.

[0056] The website information includes, but is not limited to, page titles, body text, link text, and script information; the website content includes, but is not limited to, webpage content, Uniform Resource Locator (URL) structure, domain registration information, and metadata.

[0057] Operation 101: Input the website information into the detection model and obtain the discrimination result output by the detection model to determine whether the website to be detected is a counterfeit website.

[0058] The construction process of the detection model includes the following operations:

[0059] Construct a reward function;

[0060] Based on the reward function, an unsupervised comparison feedback signal is constructed;

[0061] The detection model is obtained by training the original detection model based on the unsupervised comparison feedback signal.

[0062] The detection model mentioned in this embodiment is obtained through training with an unsupervised contrastive feedback signal. The unsupervised contrastive feedback signal can be constructed using a reward function. Specifically, the construction method may include: constructing the unsupervised contrastive feedback signal based on the reward function, a penalty term, and hyperparameters used to balance the group reciprocal rank index and the penalty term; wherein the penalty term is used to measure the relative entropy between the trained detection model and the original detection model.

[0063] Furthermore, the reward function can be constructed using a group reciprocal rank index. The specific process includes: based on the original detection model, obtaining the model output corresponding to any website in the suspected fraud website corpus, and the model output corresponding to other websites in the suspected fraud website corpus; and obtaining the group reciprocal rank index based on the similarity between the model output corresponding to any website and the model output corresponding to other websites.

[0064] In this embodiment, after obtaining website information, the website information can be input into a pre-trained detection model, and the discrimination result output by the detection model can be obtained to determine whether the website to be detected is a counterfeit website. The detection model is obtained by constructing a reward function through a group reciprocal rank index, constructing unsupervised contrastive feedback based on a set of similar websites, and training a large language model. This application utilizes the powerful semantic understanding and generation capabilities of the large language model, combined with a group contrastive feedback reinforcement learning mechanism and fine-grained analysis of implicit semantic enhancement, to efficiently and accurately identify and prevent counterfeit online fraud activities.

[0065] In this embodiment, a group contrastive feedback reinforcement learning mechanism is constructed. A combination of similar fraudulent websites is built in a suspected fraud corpus. A reward function is constructed through the group reciprocal rank index. An unsupervised contrastive feedback is constructed based on the set of similar website information. The large language model is trained to generate output results that can distinguish subtle differences between similar frauds, thereby enhancing the understanding and discrimination ability of the large language model (LLM) for fraudulent information.

[0066] This application provides a risk website detection method and apparatus based on a large language model. Leveraging the powerful semantic understanding and generation capabilities of large language models, combined with a group contrastive feedback reinforcement learning mechanism and fine-grained analysis of implicit semantic enhancement, it constructs a high-precision, high-efficiency, and intelligent system for detecting counterfeit and fraudulent websites. This effectively addresses the increasingly complex counterfeit online fraud threats that current detection schemes struggle to capture. Current detection schemes utilize large language model alignment techniques that calculate reward scores individually, relying solely on single inputs and outputs. This fails to reflect the uniqueness of the model's output. In contrast, the group reciprocal rank index, calculated through group grouping, helps the model effectively identify subtle differences within a set of counterfeit samples. The reasoning ability of large language models compensates for the shortcomings of traditional models in logical analysis, improving the intelligence level of the detection system.

[0067] The methods provided in the embodiments of this application are used in systems that enable telecommunications operators to combat fraud, significantly improving the system's intelligence level and performance indicators.

[0068] The reward function constructed using the group reciprocal rank index is as follows:

[0069] Furthermore, based on the reward function constructed above, an unsupervised contrastive feedback signal is constructed:

[0070] Where β is a hyperparameter used to balance the group reciprocal rank index and the penalty term, and the penalty term is used to measure the number of parameters. The KL divergence between the trained large language model and the original large language model with parameter π;

[0071] The group reciprocal rank index is obtained as follows: Based on the original detection model, the model outputs for any website in the suspected fraud website corpus and other websites in the suspected fraud website corpus are obtained. Then, based on the similarity between the model outputs, the group reciprocal rank index K is obtained. GR Specifically, in the above formula, K GR d is the group reciprocal rank exponent. i As a sample of counterfeit websites, d j As a sample of similar websites, To make d i The output of the large language model is taken as input. To make d j As the output of the large language model after input, S(·,·) is the similarity calculation function. A collection of similar website samples. λ is an indicator function, and λ is an adjustable hyperparameter.

[0072] Furthermore, the discrimination result may include three levels of discrimination results: the discrimination result of the main information, the discrimination result of the fine-grained information, and the discrimination result of the implicit information.

[0073] Accordingly, in the operation of inputting the website information into the detection model and obtaining the discrimination result output by the detection model, the operation of obtaining the discrimination result for the backbone information includes:

[0074] The website information T and the first prompt word template P1 are input into the detection model to obtain the backbone information T1;

[0075] Supplement T1 and the main search content with T s1 And the discrimination prompt template P L The information is input into the detection model to obtain the discrimination result L1 of the backbone information.

[0076] Furthermore, in the operation of inputting the website information into the detection model and obtaining the discrimination result output by the detection model, the operation of obtaining the discrimination result for the fine-grained information includes:

[0077] In the case of including only one piece of fine-grained information, the discrimination result L of the fine-grained information 21 The acquisition operations include:

[0078] The fine-grained information T is obtained by inputting T, T1, and the second prompt word template P2 into the detection model. 21 ;

[0079] T 21 The fine-grained retrieval content supplement T s1 And the discrimination prompt template P L The data is input into the detection model to obtain the discrimination result L of the fine-grained information. 21 ;

[0080] When including i fine-grained information, i≥2, the discrimination result L of the i-th fine-grained information is... 2i The acquisition operations include:

[0081] Combine the T, T1, the second prompt word template P2, and the previous fine-grained information T 2(i-1) The information is input into the detection model to obtain the i-th fine-grained information T. 2i ;

[0082] T 2i Supplementing the i-th fine-grained search content T s2i And the discrimination prompt template P L The input is fed into the detection model to obtain the discrimination result L of the i-th fine-grained information. 2i.

[0083] In the operation of inputting the website information into the detection model and obtaining the discrimination result output by the detection model, the operation of obtaining the discrimination result of the implicit information includes:

[0084] The T, the T1, the T 2i The third prompt word template P3 is input into the detection model to obtain the hidden information T3;

[0085] Supplement T3 and the implicit search content to T s3 And the discrimination prompt template P L The information is input into the detection model to obtain the discrimination result L3 of the hidden information.

[0086] Accordingly, determining whether the website to be detected is a counterfeit website includes:

[0087] Based on at least one of the discrimination results of the backbone information, the discrimination results of the fine-grained information, and the discrimination results of the implicit information, a comprehensive discrimination model is constructed to determine whether the website to be detected is a counterfeit website. For example, a comprehensive discrimination model f(L1,L...) is constructed. 21 ,L 22 ...,L3), set thresholds to make classification decisions and determine whether the website to be detected is a counterfeit website.

[0088] The following detailed explanation, using a specific example, illustrates the training process and practical application of the detection model. In practical business applications, this embodiment is primarily used in intelligent anti-fraud systems in the public security field. It proposes a risk website detection method based on a large language model, leveraging the powerful semantic understanding and generation capabilities of the large language model, combined with a group contrastive feedback reinforcement learning mechanism and fine-grained analysis of implicit semantic enhancement, to efficiently and accurately identify and prevent counterfeit online fraud activities.

[0089] Operation 1: Data preprocessing and information extraction.

[0090] Training samples are obtained based on Operation 1. Specifically, webpage content, URL structure, domain registration information, and website metadata of suspected fraudulent websites are collected and organized from Deep Packet Inspection (DPI) data. Web crawling tools are used to perform semantic extraction on the website content, extracting key information and semantic features that may be involved in fraud, including but not limited to page titles, body text, link text, and script information, which are then concatenated to form website information. A corpus of potentially fraudulent websites is created (similar websites are grouped together).

[0091] Operation 2: Large language model group comparative feedback reinforcement learning.

[0092] Model training is performed based on Operation 2. Specifically, current general-purpose large language models often generate responses that are generic and fail to capture the uniqueness of each input, especially when the input content is similar. This makes it difficult for LLMs to extract or identify information about fraudulent websites. Therefore, this embodiment constructs an unsupervised comparative feedback based on a set of similar websites and trains a large language model to generate outputs that can distinguish subtle differences between similar websites.

[0093] Figure 2 is a schematic diagram of the model training process provided in the embodiment of this application. The overall process of this operation is shown in Figure 2. The information of each website in each group is input into the LLM to obtain the corresponding model output. Comparative feedback is constructed from it. A group mutual inverse rank reward function is designed and adopted. Finally, the standard policy gradient method (Proximal Policy Optimization, PPO) algorithm is used to optimize the large language model to enhance its ability to capture the unique features of counterfeit websites.

[0094] Let d be the website obtained from the corpus of suspected fraudulent websites, and let G be the set of similar websites. d The website's information obtained from the LLM output is O. d A similarity calculation model is used to obtain the document vector representation (e.g., using the pre-trained model SimBERT to convert text data into corresponding 768-dimensional vectors), and the similarity calculation function is denoted as S(·,·). The formula is expressed as follows:

[0095] Next, for each group of similar websites G={d∪G d The predefined instruction I and website information are concatenated, and LLM is used to generate corresponding outputs for each group of websites (the outputs can be of any type; this proposal sets them as a specific text summarization task). The formulaic expression is:

[0096] By mapping the output of a large model and website information into a semantic space, and then computing each document... and The similarity between them. If the trained LLM captures d i and The key information that distinguishes it from other websites, then d i and The similarity score between them will be the highest. Based on this, embodiments of this application propose a group reciprocal rank index K. GR This is used as the reward function for reinforcement learning, and then the PPO algorithm is used to maximize it to achieve group contrastive feedback, thereby optimizing the information extraction and discrimination capabilities of LLM for fraudulent activities. The formulaic expression is:

[0097] in, Let λ be the indicator function, and λ be an adjustable hyperparameter. The complete reward function is:

[0098] Where β is a hyperparameter used to balance the group reciprocal rank index and the penalty term, and the penalty term is used to measure the number of parameters. The KL divergence between the optimized large language model and the initial large language model with parameter π can limit the policy model to produce outputs that are significantly different from the LLM, thus preserving the language capabilities of the policy model.

[0099] Operation 3: Fine-grained analysis of implicit semantic enhancement based on the optimized LLM.

[0100] Figure 3 is a schematic diagram of the application of the detection model provided in the embodiment of this application. As shown in Figure 3, in operation 3, the website information is parsed into three levels: main information, fine-grained information and implicit information. Among them, fine-grained information is secondary information. The large language model optimized in operation 2 is used to perform in-depth analysis on each level.

[0101] (1) Core Information Identification: Based on the powerful summarizing ability of the large language model, a more reasonable extraction method was designed to extract information from website content. Assuming the initial input website information text is T, the extraction prompt template is P1, and the extracted core information is T1, then:

[0102] External knowledge is introduced to supplement the main information by retrieving factual information related to the core information through search engines to verify the authenticity of the website page content. Let's assume the collected retrieval information is T. s1 The corresponding discrimination prompt word template is P. L To further enhance the reasoning ability of the large language model, the result of the main information discrimination is:

[0103] (2) Fine-grained information discrimination: Many counterfeit and fraudulent websites change their service content significantly by altering details while keeping the main website information unchanged. To detect this type of fine-grained counterfeit and fraudulent information, effectively unleash the potential of the large language model in content understanding, and after each fine-grained piece of information is given, a query template with a termination condition is repeatedly used 3 times to confirm that the information extraction is complete and all corresponding fine-grained information is obtained. Specifically, the extraction prompt word template used is P2, and the extracted i-th fine-grained information is T. 2i The result of the i-th query to determine whether to terminate is F. 2i , This represents a non-terminating set of information. Therefore, when At that time, we can obtain:

[0104] Further supplementary external knowledge is introduced by retrieving factual information related to fine-grained details through search engines, using similar discriminative cue words P. L Each piece of fine-grained information is evaluated, and supplementary search content T for each part is added before the evaluation. s2i The discrimination results for each set of fine-grained information are obtained:

[0105] (3) Implicit Information Reasoning: In addition to the core and fine-grained information mentioned above, many fraudulent websites also use language guidance and implicit assumptions in their text to contain certain fraudulent assumptions and hints. This type of information is difficult to detect due to its high degree of concealment and complex logic. Therefore, this proposal utilizes the implicit information reasoning capabilities of the large language model to provide more perspectives for the judgment process.

[0106] Specifically, by using the extracted cue word template P3 as input, along with the extracted core information and fine-grained information, to force the large language model to provide a hidden content T3 implied behind the website content, then:

[0107] Further supplementary content T obtained by adding implicit content s3 Finally, the discriminant prompt template P is used. L The final judgment result is as follows:

[0108] Operation 4: Comprehensive Judgment and Decision-Making.

[0109] Based on the fine-grained analysis and discrimination results of implicit semantic enhancement obtained in Operation 3, a comprehensive discrimination model f(L1,L) is constructed. 21 ,L 22 ...,L3), set thresholds to make classification decisions and determine whether a website is a counterfeit or fraudulent website.

[0110] One example is: after obtaining the results of all the above branches, if two or more branches in the results are identified as counterfeit websites, then the website will be identified as a counterfeit website, and the specific information of the corresponding counterfeit part will also be obtained; if only 0 or 1 branches are identified as counterfeit websites, then the website will be identified as a real and legitimate website.

[0111] In this embodiment, the optimized LLM fully explores and utilizes the summarizing and reasoning capabilities of the generative large language model, and performs hierarchical deduction according to the order of main information, fine-grained information and implicit information to comprehensively identify counterfeit fraudulent websites.

[0112] By decomposing the large language model into tasks and performing hierarchical derivation in the order of backbone information, fine-grained information, and implicit information, the embodiments of this application maximize the capabilities of LLM and improve the ability to detect counterfeit fraud. At the same time, the method also has a certain degree of interpretability and can provide a basis for detection.

[0113] Figure 4 is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application. As shown in Figure 4, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute the risk website detection method based on a large language model provided in the above embodiments.

[0114] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the operations described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the risk website detection method based on the large language model provided in the above embodiments.

[0116] In another aspect, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being processed to execute the risk website detection method based on a large language model provided in the above embodiments.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A risky website detection method based on a large language model, wherein, The method includes: Obtain website information of the website to be tested; The website information is input into the detection model to obtain the discrimination result output by the detection model, so as to determine whether the website to be detected is a counterfeit website; The construction process of the detection model includes the following operations: Construct a reward function; Based on the reward function, an unsupervised comparison feedback signal is constructed; The detection model is obtained by training the original detection model based on the unsupervised comparison feedback signal.

2. The risk website detection method based on a large language model according to claim 1, wherein, The constructed reward function includes: The reward function is constructed using the group reciprocal rank index.

3. The risk website detection method based on a large language model according to claim 2, wherein, The group reciprocal rank index is calculated by grouping groups.

4. The risk website detection method based on a large language model according to claim 2, wherein, The group reciprocal rank index is obtained as follows: Based on the original detection model, the model output corresponding to any website in the suspected fraud website corpus is obtained, as well as the model output corresponding to other websites in the suspected fraud website corpus. The group reciprocal rank index is obtained based on the similarity between the model output corresponding to any one website and the model output corresponding to the other websites.

5. The risk website detection method based on a large language model according to claim 1, wherein, The construction of the unsupervised comparative feedback signal based on the reward function includes: Based on the reward function, the penalty term, and the hyperparameters used to balance the group reciprocal rank index and the penalty term, the unsupervised contrastive feedback signal is constructed. The penalty term is used to measure the relative entropy between the trained detection model and the original detection model.

6. The risk website detection method based on a large language model according to claim 5, wherein, The discrimination results include the discrimination results of the backbone information; Accordingly, the website information is input into the detection model to obtain the discrimination result output by the detection model, including: The website information and the first prompt word template are input into the detection model to obtain the backbone information; The main information, supplementary main search content, and discrimination prompt template are input into the detection model to obtain the discrimination result of the main information.

7. The risk website detection method based on a large language model according to claim 6, wherein, The discrimination result also includes discrimination results with fine-grained information; Accordingly, inputting the website information into the detection model to obtain the discrimination result output by the detection model also includes: When only one piece of fine-grained information is included, the operation of obtaining the discrimination result of the fine-grained information includes: The website information, the main information, and the second prompt word template are input into the detection model to obtain the fine-grained information; The fine-grained information, the fine-grained search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the fine-grained information; When i ≥ 2, the operation of obtaining the discrimination result of the i-th fine-grained information includes: The website information, the main information, the second prompt word template, and the previous fine-grained information are input into the detection model to obtain the i-th fine-grained information; The i-th fine-grained information, the i-th fine-grained search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the i-th fine-grained information.

8. The risk website detection method based on a large language model according to claim 7, wherein, The discrimination result also includes discrimination results containing implicit information; Accordingly, inputting the website information into the detection model to obtain the discrimination result output by the detection model also includes: The website information, the backbone information, the i-th fine-grained information, and the third prompt word template are input into the detection model to obtain the hidden information; The implicit information, the implicit search content supplement, and the discrimination prompt template are input into the detection model to obtain the discrimination result of the implicit information.

9. The risk website detection method based on a large language model according to claim 8, wherein, Determining whether the website to be detected is a counterfeit website includes: Based on at least one of the discrimination results of the backbone information, the discrimination results of the fine-grained information, and the discrimination results of the implicit information, a comprehensive discrimination model is constructed to determine whether the website to be detected is a counterfeit website.

10. The risk website detection method based on a large language model according to claim 9, wherein, The determination of whether the website to be detected is a counterfeit website includes: Based on the comprehensive discrimination model, a threshold is set to make a classification decision and determine whether the website to be detected is a counterfeit website.

11. The risk website detection method based on a large language model according to claim 10, wherein, The step of setting a threshold based on the comprehensive discrimination model to make classification decisions and determine whether the website to be detected is a counterfeit website includes: If two or more branches appear in the results of the discrimination results of the main information, the fine-grained information, and the implicit information, the website to be detected is determined to be a counterfeit website. If a website is identified as a counterfeit website if only 0 or 1 branches are found in the results of the main information, the fine-grained information, and the implicit information, then the website to be detected is identified as a legitimate website.

12. The risk website detection method based on a large language model according to any one of claims 1 to 11, wherein, The process of obtaining website information of the website to be detected includes: The website information is obtained by semantically extracting the website content of the website to be detected using a web crawler tool. The website information includes, but is not limited to, page titles, body text, link text, and script information; the website content includes, but is not limited to, webpage content, Uniform Resource Locator (URL) structure, domain registration information, and metadata.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the risk website detection method based on a large language model as described in any one of claims 1 to 12.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the risk website detection method based on a large language model as described in any one of claims 1 to 12.

15. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the risk website detection method based on a large language model as described in any one of claims 1 to 12.

Images