Website multi-dimensional detection method and device, electronic equipment and storage medium

Abstract

The invention provides a multi-dimensional detection method and device for a website, electronic equipment and a storage medium, and the method comprises the steps: obtaining a website link of a to-be-detected website, and obtaining website data of the to-be-detected website; based on the first large language model, analyzing multiple data dimensions of the website data, and determining a target data dimension, needing to be expanded, of the website data; calling a second large language model based on the target data dimension, and expanding the website data to obtain expanded website data; and calling a third language model, carrying out anomaly identification on the expanded website data, and determining whether the website is an abnormal website. The website data of the to-be-detected website is subjected to multi-data-dimension analysis in combination with the large language model, and the multi-dimension analysis process of the website data is achieved. And based on a decoupling analysis mode that different large language models undertake different responsibilities, a high-precision anomaly recognition process of the to-be-detected website is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a multi-dimensional detection method, device, electronic device and storage medium for a website. Background Art

[0002] In existing methods, abnormal websites are generally detected based on URL detection or web page content detection. Among them, the URL detection method mainly uses two methods, blacklist mechanism and machine learning technology, to identify abnormal websites. Web page content detection is generally based on text detection and image detection to identify abnormalities.

[0003] In existing methods, the detection of abnormal websites often only considers one dimension of information, resulting in low detection accuracy. Summary of the invention

[0004] The present invention provides a multi-dimensional detection method, device, electronic device and storage medium for websites, so as to improve the accuracy of abnormal website identification.

[0005] The present invention provides a multi-dimensional detection method, device, electronic device and storage medium for a website, comprising the following steps: Obtaining a website link of a website to be detected, and based on the website link, obtaining website data of the website to be detected; Based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Based on the target data dimension, calling the second largest language model, expanding the website data to obtain expanded website data; The third language model is called to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0006] According to a multi-dimensional detection of a website provided by the present invention, based on the target data dimension, calling the second largest language model to expand the website data to obtain the expanded website data includes: Based on the category of the target data dimension, a target prompt word template corresponding to the target data dimension is acquired from a prompt word template library, wherein the prompt word template library contains prompt word templates corresponding to all data dimensions; Based on the target prompt word template, the second largest language model is called to expand the website data to obtain expanded website data.

[0007] According to a multi-dimensional detection of a website provided by the present invention, the website data is analyzed in multiple data dimensions based on the first language model to determine the target data dimension that needs to be expanded for abnormal identification of the website data, including: Pre-building a prompt word template, the prompt word template is used to determine whether data expansion is required based on analyzing the website data in multiple data dimensions; Based on the prompt word template, the website data is combined, and the combined data is input into the first language model to obtain the target data dimension required to be expanded for abnormality identification of the website data output by the first language model.

[0008] According to a multi-dimensional detection of a website provided by the present invention, the second largest language model is called based on the target data dimension to expand the website data to obtain the expanded website data, and further includes: Based on the first language model, analyzing the expanded website data to determine whether the expanded website data can perform an abnormality identification judgment process; In the case that the expanded website data cannot perform the judgment process of abnormal identification, further analyzing the expanded website data to determine the data dimension to be further expanded; Based on the further expanded data dimension, the second largest language model is called to further expand the expanded website data.

[0009] According to a multi-dimensional detection of a website provided by the present invention, after determining whether the website to be detected is an abnormal website, the method further includes: In the case where it is determined that the website to be detected is an abnormal website, obtaining feedback information of the user on the output of the third language model; Based on the expanded website data and the feedback information, the third language model is trained, and the third language model is updated based on the training result.

[0010] According to a multi-dimensional detection of a website provided by the present invention, the data dimensions include a website link dimension, a website access method dimension, a text information dimension, and an image information dimension.

[0011] The present invention also provides a multi-dimensional detection device for a website, comprising the following modules: A data acquisition module, used to acquire a website link of a website to be detected, and based on the website link, acquire website data of the website to be detected; An expansion identification module, used to analyze the website data in multiple data dimensions based on the first language model, and determine the target data dimension that needs to be expanded for abnormality identification of the website data; A data expansion module, used to call the second largest language model based on the target data dimension to expand the website data to obtain expanded website data; The anomaly identification module is used to call the third language model to perform anomaly identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a multi-dimensional detection method for a website as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multi-dimensional detection methods for websites described above.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the multi-dimensional detection methods for websites described above.

[0015] The multi-dimensional detection method, device, electronic device and storage medium of a website provided by the present invention realizes a multi-dimensional analysis process of the website data by combining a large language model to analyze the website data of the website to be detected in multiple data dimensions and determine the target data dimension that needs to be expanded. Based on the target data dimension, the second largest language model is called to expand the website data, and the third largest language model is called to perform abnormality identification on the expanded website data. Based on the decoupled analysis method in which different large language models assume different responsibilities, a high-precision abnormality identification process of the website to be detected is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the multi-dimensional detection method of a website provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the multi-agent system detection framework structure provided by the present invention.

[0019] Figure 3 This is a diagram showing a comparison of the processes of different frameworks provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the multi-dimensional detection device for websites provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Related methods detect abnormal websites, which are generally implemented based on URL detection or web page content detection.

[0024] URL detection methods mainly use blacklist mechanism and machine learning technology to identify abnormal websites.

[0025] Blacklist mechanism: This method efficiently and accurately identifies abnormal websites by maintaining a list of known abnormal URLs. However, its limitation is that it cannot effectively identify newly emerging abnormal websites because they can evade detection by changing the URL address. In addition, as new abnormal websites continue to emerge, it becomes increasingly difficult to maintain a comprehensive and exhaustive blacklist.

[0026] Deep learning technology: Using deep learning algorithms to identify URLs can generalize new abnormal URLs to a certain extent. Nevertheless, deep learning models are prone to errors when faced with URLs that are significantly different from the training data. In addition, the decision-making process of deep learning models is often viewed as a "black box" that lacks transparency and explainability, making it difficult for auditors to understand the basis for the model's judgment. When the model does not work well, a common means of improvement is to retrain by collecting more sample data, but in practice this can be time-consuming and inefficient.

[0027] Web page content detection can be divided into two aspects: text detection and image detection.

[0028] Text detection: includes monitoring sensitive keywords and using deep learning models to classify entire texts.

[0029] Sensitive keyword monitoring: Faces similar challenges as blacklists. It may evade detection by changing the text (such as converting between simplified and traditional Chinese, using variant characters, etc.), which may also cause a large number of false positives.

[0030] Deep learning classification: When processing web pages that only contain images but no text, or when the abnormal content only accounts for a small part of the total text, the deep learning model has the risk of missing reports due to the lack of sufficient judgment basis.

[0031] Image detection: Image detection has shown high accuracy in identifying abnormal images, but it still has certain limitations in identifying abnormal websites. This is due to: Limitations of the dataset: Due to the lack of sufficient datasets of abnormal images, it is very difficult to train a model that can effectively identify these contents. The incompleteness of the dataset limits the generalization ability and recognition accuracy of the machine learning model.

[0032] Difference in the nature of the task: Recognition of an entire website and recognition of a single image are essentially two different tasks. A website usually contains a variety of content and contextual information, while a single image lacks such context. Therefore, it is necessary to develop detection techniques that can understand the overall structure of the website and the relationship between the content.

[0033] In addition, both text detection and image detection face the same dilemma, that is, the problem of web page jumps faced by abnormal website detection based on content. In fact, abnormal websites are often blocked for review through a series of intermediate jump pages.

[0034] In addition, there are a series of methods that use large language models or multimodal large language models to perform content review, such as using a multimodal large language model to convert images into text, and then using a text classification model to make judgments, and using large language models and prompt engineering prompts, lora fine-tuning, text preprocessing and other technologies to identify abnormal emails.

[0035] The techniques used in related methods to identify abnormal websites often only consider information of one dimension, such as URL dimension, text dimension, image dimension, etc., but ignore the comprehensive analysis of these multi-dimensional information. The discriminant model is used to make judgments based on single-dimensional information. Because there is too little content to base on, it is inevitable that there will be misjudgments and missed judgments. In order to reduce missed alarms, the system may increase sensitivity, which may lead to an increase in the false alarm rate; conversely, in order to reduce false alarms, the sensitivity may be reduced, increasing the risk of missed alarms.

[0036] In addition, it is possible to exploit the weaknesses of a single model and bypass detection through targeted methods. For example, the page that jumps to the web page may be packaged as a normal web page, or almost no content is displayed, and the content-based analysis method is circumvented by automatically jumping to the abnormal website after a few seconds.

[0037] With the continuous advancement and application of large language model technology, abnormal content review methods based on these models have also been widely adopted. The related methods combine the content to be reviewed with a carefully designed prompt word template, which may require the large language model to complete several subtasks at the same time, and then input it into a large language model fine-tuned by LoRa to identify whether the content is compliant. However, this method still essentially regards the large language model as a more capable natural language processing classifier and does not fully utilize the full potential of the large language model. There are several major problems with this approach: The prompt design is too complex: If the prompt requires the large language model to complete several subtasks at the same time, the prompt may be too complex and the model may find it difficult to accurately capture and execute all instructions. This will not only reduce the response quality of the model, but may also cause the model's output to deviate from the expected goal.

[0038] Limited scalability: After fine-tuning a specific prompt template using LoRa, it is only effective for a specific prompt template and task. This will cause the model to perform poorly when faced with new prompts that have not been seen or slightly different contexts.

[0039] Lack of multiple rounds of thinking: Even when the model does not have enough evidence to make a reasonable judgment, this paradigm allows the large language model to still make a judgment, rather than making a judgment through multiple rounds of reasoning and thinking and trying to obtain additional information.

[0040] Insufficient flexibility in calling tools: Since multiple subtasks need to be completed at the same time and the input context (prompt + content to be tested) is too long, it will cause great interference to the accurate knowledge retrieval using the plug-in knowledge base. Therefore, this method cannot efficiently utilize the large language model's ability to call and coordinate different tools and information sources, thereby limiting the model's potential in handling complex tasks.

[0041] In view of the defects of the related methods, the present invention provides a multi-dimensional detection method for a website. Figure 1 is a flow chart of the multi-dimensional detection method of a website provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 110, obtaining a website link of a website to be detected, and based on the website link, obtaining website data of the website to be detected; Step 120: Analyze the website data in multiple data dimensions based on the first language model to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Step 130, based on the target data dimension, calling the second largest language model to expand the website data to obtain expanded website data; Step 140, calling the third language model to perform anomaly recognition on the expanded website data to determine whether the website to be detected is an abnormal website.

[0042] The execution subject of the multi-dimensional detection method of a website provided by the present invention may be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (NAS), or a personal computer (PC), etc., which is not specifically limited by the present invention.

[0043] The following takes a computer executing the multi-dimensional detection method of a website provided by the present invention as an example to describe the technical solution of the present invention in detail.

[0044] In step 110, a website link of a website to be detected is obtained, and based on the website link, website data of the website to be detected is obtained.

[0045] After obtaining the website link, the website to be detected is accessed based on the website link to obtain the website data of the website to be detected.

[0046] The website data of the website to be detected may include the website's domain name information, the website's text content information, the website's IP history change data, the website's image content data, and the like.

[0047] In step 120, based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for anomaly identification of the website data.

[0048] After obtaining the website data, the website data is analyzed in different data dimensions based on the first language model. Optionally, the different data dimensions may include website link dimension, website access method dimension, text information dimension, image information dimension, etc.

[0049] Optionally, the first language model can use pre-built prompt word templates to perform specific analysis processes of different data dimensions, including domain name analysis, website content analysis, access method analysis, and IP history change analysis.

[0050] After the first language model analyzes each data dimension, it determines whether it has obtained sufficient basis to determine whether the website to be tested is an abnormal website. If the basis for judgment is insufficient, it analyzes which data dimensions have insufficient data and determines the target data dimensions that need to be expanded to identify anomalies in the website data.

[0051] In step 130, based on the target data dimension, the second largest language model is called to expand the website data to obtain expanded website data.

[0052] The second largest language model is used for the data expansion process. For website data that needs to be expanded in different data dimensions, the corresponding data expansion process is implemented.

[0053] After data expansion of the target data dimension is performed on the website data, the expanded website data is obtained to realize a further collection process of the website data.

[0054] In step 140, the third language model is called to perform anomaly identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0055] The third language model is used in the final analysis and judgment process of the website. It will identify anomalies in the expanded website data, make a final summary and judgment, and realize the judgment process of whether the website to be detected is an abnormal website.

[0056] Optionally, a multi-agent system based on setting up a scheduling agent, an analysis agent group, and a summary agent can be used to implement a multi-dimensional detection process for the website to be detected. Among them, the analysis agent group includes agents that analyze and expand multiple data dimensions. The specific implementation framework can be as follows Figure 2 The multi-agent system detection framework structure provided by the present invention is shown in the schematic diagram.

[0057] The scheduling agent is a first-class language model built on the ReAct (Synergizing Reasoning and Acting in LanguageModels) framework, which is a method that combines reasoning and action. The ReAct framework draws on the human thinking mode when facing complex tasks, that is, before taking any action, we first conduct careful logical reasoning.

[0058] like Figure 3The process comparison of different frameworks provided by the present invention is shown in the figure. In the ReAct framework, the large language model first explicitly performs reasoning, explains the tools to be called and the reasons for calling them, and then analyzes the output results of these tools in a single round of interaction. Through continuous interaction with the tools, the model can eventually come up with a more accurate answer.

[0059] Compared with the ReAct framework, other frameworks such as CoT (Chain of Thought) rely on more complex prompt designs. When a prompt attempts to guide a large language model to complete multiple subtasks, the instructions may become too complex, increasing the difficulty for the large language model to accurately understand and execute all task instructions. In addition, the CoT framework mainly relies on the internalized knowledge of the large language model for reasoning, which may cause the generated information to lack factual basis, thereby creating hallucinations. The Act-only framework only executes a single tool call, does not pay attention to the correctness of the call result, and does not consider whether the most appropriate tool is selected, which limits the model's potential in complex task processing.

[0060] As the core of the multi-agent system, the scheduling agent connects the analysis agent group and the summary agent. The specific workflow of the scheduling agent based on the ReAct framework is as follows: Step 1, enter the URL of the website to be tested; Step 2, through the pre-built prompt, tell the scheduling agent how to schedule the analysis agent group and which agents can be scheduled, including domain name analysis, website content analysis, access method analysis, and IP history change analysis.

[0061] Step 3: The dispatching agent will determine whether it has obtained sufficient evidence to determine whether the website to be detected is an abnormal website based on the known information. If the evidence is insufficient, it will analyze which analysis agents in the analysis agent group need to be called for analysis. Step 4: As needed, the scheduling agent calls the corresponding analysis agent to perform analysis and data expansion for specific tasks. After the expansion is completed, the results are fed back to the scheduling agent.

[0062] In step 5, the scheduling agent will input the returned analysis results into the model again and repeat steps 3 and 4 until enough information is collected to make a judgment. Once the scheduling agent determines that sufficient basis for judgment has been obtained, it will summarize all the collected information and input it into the summary agent to make a final judgment.

[0063] Among them, the prompt word template can be set to; You are a website content reviewer who needs to review categories A, B, and C. Your task is to use tools to collect information from various dimensions of the website to be reviewed until you have enough information to make a judgment. The agents you can call are as follows: {agents}: Please answer the questions in the following format: Question: The website that needs to be reviewed; Thought: To answer this, what do I need to do? Action: The agent to be called must be in {agents_name}; Observation: Select the results returned by the tool; ...(this thought / action / observation can be repeated N times); Thought: I have now acquired enough basis for judgment; Here are some examples

[0064] {examples}

[0065] Start answering now, and remember to follow the specified format to make step-by-step reasoning before giving your final answer.

[0066] Question: {input}.

[0067] Some examples of the content filled in the {examples} field in the above prompt can be: Question: The URL of the website to be reviewed is w……cc; Thought: I need to analyze the URL of the webpage; Action: URL analysis agent [http: / / w……3.cc]; Observation: There is w...8.cc in the blacklist. Similar to it, the top-level domain .cc is not a common suffix and has a high abuse rate.

[0068] Thought: This alone is not enough to make a judgment, I need to visit the website and get the content of the website; Action: Access method analysis agent [http: / / w……cc]

[0069] Observation: The website can be accessed normally without any redirects; Thought: This alone is not enough to make a judgment, I need to analyze the content of the page; Action: content analysis agent [w……cc]; Observation: The content of the website shows that it is a login portal for the mailbox; Thought: This alone is not enough to make a judgment, I need to analyze the IP changes of the website; Action: ip query agent [w……cc]; Observation: The website's IP address is 111.11. … …, its location is A, its operator is Company C, and its IP address type is data center. In the past two months, the website has been bound to 9 IP addresses, and the changes are relatively frequent. Thought: I have now acquired enough basis for judgment; Question: The URL of the website to be reviewed is app.3…xyz; Thought: I need to analyze the URL of the webpage; Action: URL analysis agent [app.3…xyz]; Observation: The URL is in the blacklist; Thought: I have gained enough basis for judgment now.

[0070] Many large language models have a context length limit when processing input. If there are too many examples, this limit may be exceeded, causing the model to be unable to process all the information. At the same time, too many examples may distract the model and reduce its sensitivity and understanding of the core tasks. Therefore, in addition to listing representative and wide-coverage examples in the prompt, it is also necessary to use the correct reasoning action trajectories similar to the above examples as training samples for fine-tuning the large language model to improve the accuracy of the scheduling agent.

[0071] The agents in the analysis agent group called by the scheduling agent are determined based on the second largest language model and are all packaged in the style of tools. Therefore, the large language model used by the scheduling agent must support tool calls.

[0072] The analysis agent group can include URL analysis agents, web page access method analysis agents, web page content analysis agents, and network IP change agents, which can be flexibly added according to the audit needs.

[0073] For the URL analysis agent, after obtaining the domain name of the website to be detected, it is vectorized using a pre-trained sentence embedding model such as m3e. The vectorized representation is used to search in the blacklist knowledge base and the domain name structure knowledge base respectively. The data in the knowledge base should also be vectorized to facilitate similarity calculation.

[0074] The purpose of searching in the blacklist database is to find the same or similar URL domain name structure knowledge base in the blacklist database. The knowledge base contains the top-level domain names and the corresponding information of some common domain names. It should be noted that the vectorized part and the corresponding text part of the domain name structure knowledge base for similarity calculation are separated, just to avoid the interference of a large amount of text content on the similarity calculation.

[0075] After retrieving the corresponding content in the knowledge base, it is spliced ​​into the prompt and input into the large language model to obtain an answer.

[0076] The structure of Prompt can be: Please analyze the URL of the website to be tested, mainly based on the knowledge base search results to analyze whether it is in the blacklist or whether it is similar to the URL in the blacklist. At the same time, analyze the domain name structure in the URL; Please output in the following format; URL of the website to be tested; Similar URLs in the blacklist are: analyze; Here are some examples: URL of the website to be tested: w…cc; Similar URLs in the blacklist include: w…8.cc, w…vip, w…9.vip; Please start the analysis and output the results.

[0077] URL of the website to be tested: {URL}; Similar URLs in the blacklist are: {similar_URLs}; Structure: {dns-structure}.

[0078] For the web access method analysis agent, after obtaining the domain name of the website to be detected, the web access method analysis agent first uses a simulated browser to access it. The simulated browser can simulate the browsing behavior of real users and circumvent the website's anti-crawler strategy. During the analysis process, the agent records the returned HTTP status code to determine whether the access is successful or not and its nature (such as 200 for success, 404 for not found, 500 for server error, etc.).

[0079] If the initial access fails, the agent will automatically switch to using a proxy server to try again. The purpose of this step is to check whether the website is geographically restricted or IP blocked, and can only be successfully accessed through a proxy. The agent will record in detail whether a proxy server is required for successful access.

[0080] If the access is successful, the agent will wait for 5 seconds to see if the web page will automatically jump. This jump may indicate that the website is redirecting or loading additional content. The agent will record the target URL of the jump so that the scheduling agent can direct the web content analysis agent to directly access the URL after the jump.

[0081] For the web content analysis agent, the agent obtains a screenshot of the entire web page through crawler technology. This step ensures that the complete visual presentation of the web page is captured, including all text and images. Then the agent extracts all text elements from the web page, including the title, body, etc. This step ensures that all text information can be analyzed.

[0082] The prompt word template prompt corresponding to the web page content analysis agent can be: You are a website content reviewer. Please analyze the uploaded screenshot of the web page based on the text content of the web page to see if it contains any abnormal content, including but not limited to: Category A, Category B, Category C, etc.

[0083] The text of the webpage is as follows: {text}; Please start the analysis and output the results.

[0084] For the network IP change agent, after obtaining the domain name of the website to be detected, the ping command, nslookup command, ip138 query online API, etc. are called to obtain the IP address bound to the domain name, the location, operator, IP type (data center / personal IP), the number of IPs bound in the past, etc. These results are usually output in JSON format and can be organized into text by the large language model.

[0085] The summarizing agent is the last link in the detection process. It is also built based on the third language model. It accepts the judgment basis collected by the scheduling agent and makes the final summary and judgment. Its essence is to use the large language model to complete a text classification task.

[0086] To ensure that the agent's response is aligned with the auditor's intention, we need to use the collected correct reasoning examples to fine-tune LoRA. In order to make the model's judgment unbiased, the selection of positive and negative samples must be balanced.

[0087] Since the prompt template contains analysis text of abnormal content, which may cause the model to refuse to answer, the Baichuan large language model with weaker ethical restrictions can be used as the base model for LoRA fine-tuning.

[0088] Among them, the prompt word template corresponding to the third language model can be set as: "You are a website content reviewer and need to review websites containing abnormal content such as Category A, Category B, and Category C. Your task is to summarize and draw conclusions based on the information obtained in each dimension and output them in JSON format. Please answer the questions in the following format: Input: collected information on various dimensions of the website; Output: {

[0089] "Summary": summarize the information of each dimension of the website and make inferences for drawing conclusions

[0090] “Conclusion”: [yes / no]

[0091] }

[0092] Here are some examples: {examples}

[0093] Input: {information}

[0094] ”.

[0095] The saved content after processing will be reviewed by human experts. The saved content is a collection of observations of the scheduling agent, and the summary process and conclusions of the summary agent.

[0096] Experts will analyze high-quality answers and typical error cases, and use these cases to schedule agents, analyze agent groups, and summarize further fine-tuning training of agents, or directly add them to prompts to optimize the model's capabilities.

[0097] The multi-dimensional detection method of a website provided by the present invention realizes a multi-dimensional analysis process of the website data by combining a large language model to analyze the website data of the website to be detected in multiple data dimensions and determining the target data dimension that needs to be expanded. Based on the target data dimension, the second largest language model is called to expand the website data, and the third largest language model is called to perform abnormality identification on the expanded website data. Based on the decoupled analysis method in which different large language models assume different responsibilities, a high-precision abnormality identification process of the website to be detected is realized.

[0098] In one embodiment, based on the target data dimension, calling the second largest language model to expand the website data to obtain the expanded website data includes: based on the category of the target data dimension, acquiring a target prompt word template corresponding to the target data dimension from a prompt word template library, the prompt word template library containing prompt word templates corresponding to all data dimensions; based on the target prompt word template, calling the second largest language model to expand the website data to obtain the expanded website data.

[0099] A variety of prompt word template libraries can be pre-set and combined with the second largest language model to realize the expansion process of website data of different data dimensions.

[0100] The prompt word template library is a pre-built resource that contains prompt word templates for different data dimensions. These templates are designed to guide the language model to generate content that meets the requirements of specific data dimensions.

[0101] Apply the matched target cue word template to the input data. This can be achieved by combining the cue word template with the input data to form a new input sequence. The role of the cue word template is to guide the large language model to generate content related to a specific data dimension.

[0102] The input data processed by the prompt word template is input into the second largest language model. The model will generate new text content that meets the requirements of the target data dimension based on the input data and the prompt word template to realize the expansion process of the website data in the target data dimension.

[0103] In one embodiment, based on the first large language model, the website data is analyzed in multiple data dimensions to determine the target data dimensions that need to be expanded for abnormality identification of the website data, including: pre-building a prompt word template, the prompt word template is used to determine whether data expansion is needed based on the analysis of multiple data dimensions of the website data; based on the prompt word template, the website data is combined, and the combined data is input into the first large language model to obtain the target data dimensions that need to be expanded for abnormality identification of the website data output by the first large language model.

[0104] For each data dimension, a series of prompt word templates are designed. These prompt word templates can guide the first language model to identify the data features of a specific dimension and help determine whether the dimension needs to be expanded.

[0105] Combine the relevant information in the website data according to the prompt word template. This can be achieved through text matching, semantic analysis, etc. Perform multiple data dimension analysis on the combined data.

[0106] The combined and analyzed website data is input into the first language model. Based on the input data, the first language model will generate new text content to expand the target data dimension. These generated contents can make up for the deficiencies in the original data and improve the completeness and accuracy of the data.

[0107] In one embodiment, based on the target data dimension, calling the second largest language model to expand the website data to obtain the expanded website data, it also includes: based on the first largest language model, analyzing the expanded website data to determine whether the expanded website data can perform the judgment process of abnormal identification; if the expanded website data cannot perform the judgment process of abnormal identification, further analyzing the expanded website data to determine the further expanded data dimension; based on the further expanded data dimension, calling the second largest language model to further expand the expanded data.

[0108] After preliminary website data expansion, the website data is analyzed based on the first language model to determine whether the anomaly identification process can be performed on the expanded website data.

[0109] If the first language model determines that the expanded website data is sufficient to perform anomaly identification, it can directly proceed to the subsequent analysis or processing steps. If the judgment result is insufficient, that is, the data is insufficient to support anomaly identification, it is necessary to further analyze the data to determine the data dimensions that need to be expanded.

[0110] Conduct in-depth analysis of the expanded website data to identify which data dimensions or features are missing or insufficient, resulting in the inability to perform anomaly identification. Based on the results of further analysis, determine the data dimensions that need to be expanded. Use the data dimensions that need to be expanded as input and call the second largest language model for further expansion.

[0111] It is understandable that analyzing the expanded website data based on the first language model and further data expansion when necessary is an iterative and meticulous process. Through reasonable model selection, data analysis and expansion strategies, the integrity and accuracy of the data can be effectively improved, thereby supporting more accurate anomaly identification tasks.

[0112] In one embodiment, after determining whether the website to be detected is an abnormal website, it also includes: when it is determined that the website to be detected is an abnormal website, obtaining user feedback information on the output of the third language model; based on the expanded website data and the feedback information, training the third language model, and updating the third language model based on the training result.

[0113] Once a website is determined to be abnormal, the feedback collection mechanism is immediately triggered. User feedback can include their recognition of the model output, their understanding of the abnormal website, possible errors or omissions, etc.

[0114] Use the augmented website data (including data augmented by prompt word templates and the top two language models) as part of the training data. Integrate user feedback and convert it into a format that the model can understand. This may involve converting user text feedback into feature vectors or combining it with existing data labels.

[0115] The third language model is trained using the prepared data. The training process may involve adjusting the parameters of the model, optimizing its structure, or introducing new feature representation methods.

[0116] Based on the training results and evaluation feedback, the third largest language model is updated, including adjusting the model's weights, replacing certain layers, or introducing new training strategies.

[0117] In one embodiment, the data dimensions include website link dimension, website access method dimension, text information dimension and image information dimension.

[0118] It is understandable that multi-dimensional data analysis can more finely identify the type, degree and scope of website anomalies. By analyzing the correlation and mutual influence between different dimensions, we can gain a deeper understanding of the root causes of website anomalies.

[0119] The multi-dimensional detection device for a website provided by the present invention is described below. The multi-dimensional detection device for a website described below and the multi-dimensional detection method for a website described above can be referenced to each other.

[0120] like Figure 4 As shown, the device comprises: The data acquisition module 410 is used to acquire a website link of a website to be detected, and based on the website link, acquire website data of the website to be detected; An expansion identification module 420 is used to analyze the website data in multiple data dimensions based on the first language model, and determine the target data dimension that needs to be expanded for abnormality identification of the website data; A data expansion module 430 is used to call the second largest language model based on the target data dimension to expand the website data to obtain expanded website data; The abnormality identification module 440 is used to call the third language model to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0121] The multi-dimensional detection device for websites provided by the present invention realizes a multi-dimensional analysis process of website data by combining a large language model to analyze the website data of the website to be detected in multiple data dimensions and determining the target data dimension that needs to be expanded. Based on the target data dimension, the second largest language model is called to expand the website data, and the third largest language model is called to perform abnormality identification on the expanded website data. Based on the decoupled analysis method in which different large language models assume different responsibilities, a high-precision abnormality identification process of the website to be detected is realized.

[0122] In one embodiment, the data expansion module 430 is specifically used to: Based on the target data dimension, calling the second largest language model to expand the website data to obtain the expanded website data includes: Based on the category of the target data dimension, a target prompt word template corresponding to the target data dimension is acquired from a prompt word template library, wherein the prompt word template library contains prompt word templates corresponding to all data dimensions; Based on the target prompt word template, the second largest language model is called to expand the website data to obtain expanded website data.

[0123] In one embodiment, the expansion identification module 420 is specifically used to: The website data is analyzed in multiple data dimensions based on the first language model to determine the target data dimensions that need to be expanded for abnormality identification of the website data, including: Pre-building a prompt word template, the prompt word template is used to determine whether data expansion is required based on analyzing the website data in multiple data dimensions; Based on the prompt word template, the website data is combined, and the combined data is input into the first language model to obtain the target data dimension required to be expanded for abnormality identification of the website data output by the first language model.

[0124] In one embodiment, the data expansion module 430 is further specifically used for: After the second largest language model is called based on the target data dimension to expand the website data and the expanded website data is obtained, the method further includes: Based on the first language model, analyzing the expanded website data to determine whether the expanded website data can perform an abnormality identification judgment process; In the case that the expanded website data cannot perform the judgment process of abnormal identification, further analyzing the expanded website data to determine the data dimension to be further expanded; Based on the further expanded data dimension, the second largest language model is called to further expand the expanded website data.

[0125] In one embodiment, the anomaly identification module 440 is specifically used to: After determining whether the website to be detected is an abnormal website, the method further includes: In the case where it is determined that the website to be detected is an abnormal website, obtaining feedback information of the user on the output of the third language model; Based on the expanded website data and the feedback information, the third language model is trained, and the third language model is updated based on the training result.

[0126] In one embodiment, the expansion identification module 420 is further specifically configured to: The data dimensions include website link dimension, website access method dimension, text information dimension and image information dimension.

[0127] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the multi-dimensional detection method of the website, the method comprising: obtaining a website link of the website to be detected, and based on the website link, obtaining the website data of the website to be detected; Based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Based on the target data dimension, calling the second largest language model, expanding the website data to obtain expanded website data; The third language model is called to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0128] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0129] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the multi-dimensional detection method of the website provided by the above methods, the method includes: obtaining a website link of the website to be detected, and based on the website link, obtaining the website data of the website to be detected; Based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Based on the target data dimension, calling the second largest language model, expanding the website data to obtain expanded website data; The third language model is called to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0130] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the multi-dimensional detection method of the website provided by the above methods, the method comprising: obtaining a website link of a website to be detected, and based on the website link, obtaining website data of the website to be detected; Based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Based on the target data dimension, calling the second largest language model, expanding the website data to obtain expanded website data; The third language model is called to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-dimensional detection method for a website, characterized in that: The method comprises: Obtaining a website link of a website to be detected, and based on the website link, obtaining website data of the website to be detected; Based on the first language model, the website data is analyzed in multiple data dimensions to determine the target data dimension that needs to be expanded for abnormality identification of the website data; Based on the target data dimension, calling the second largest language model, expanding the website data to obtain expanded website data; The third language model is called to perform abnormality identification on the expanded website data to determine whether the website to be detected is an abnormal website.

2. The multi-dimensional detection method of a website according to claim 1, characterized in that: Based on the target data dimension, calling the second largest language model to expand the website data to obtain the expanded website data includes: Based on the category of the target data dimension, a target prompt word template corresponding to the target data dimension is acquired from a prompt word template library, wherein the prompt word template library contains prompt word templates corresponding to all data dimensions; Based on the target prompt word template, the second largest language model is called to expand the website data to obtain expanded website data.

3. The multi-dimensional detection method of a website according to claim 1, characterized in that: The website data is analyzed in multiple data dimensions based on the first language model to determine the target data dimensions that need to be expanded for abnormality identification of the website data, including: Pre-building a prompt word template, the prompt word template is used to determine whether data expansion is required based on analyzing the website data in multiple data dimensions; Based on the prompt word template, the website data is combined, and the combined data is input into the first language model to obtain the target data dimension required to be expanded for abnormality identification of the website data output by the first language model.

4. The multi-dimensional detection method of a website according to claim 1, characterized in that: After the second largest language model is called based on the target data dimension to expand the website data and the expanded website data is obtained, the method further includes: Based on the first language model, analyzing the expanded website data to determine whether the expanded website data can perform an abnormality identification judgment process; In the case that the expanded website data cannot perform the judgment process of abnormal identification, further analyzing the expanded website data to determine the data dimension to be further expanded; Based on the further expanded data dimension, the second largest language model is called to further expand the expanded website data.

5. The multi-dimensional detection method of a website according to claim 1, characterized in that: After determining whether the website to be detected is an abnormal website, the method further includes: In the case where it is determined that the website to be detected is an abnormal website, obtaining feedback information of the user on the output of the third language model; Based on the expanded website data and the feedback information, the third language model is trained, and the third language model is updated based on the training result.

6. The multi-dimensional detection method of a website according to claim 1, characterized in that: The data dimensions include website link dimension, website access method dimension, text information dimension and image information dimension.

7. A multi-dimensional detection device for a website, characterized in that: include: A data acquisition module, used to acquire a website link of a website to be detected, and based on the website link, acquire website data of the website to be detected; An expansion identification module, used to analyze the website data in multiple data dimensions based on the first language model, and determine the target data dimension that needs to be expanded for abnormality identification of the website data; A data expansion module, configured to call a second language model based on the target data dimension to expand the website data to obtain expanded website data; The anomaly identification module is used to call the third language model to perform anomaly identification on the expanded website data to determine whether the website to be detected is an abnormal website.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-dimensional detection method for a website as claimed in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-dimensional detection method for a website as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-dimensional detection method for a website as claimed in any one of claims 1 to 6 is implemented.

Images