Malicious Website Detection Using LVM Prompt Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) and large vision models (LVMs) face challenges in accurately processing complex information, particularly in domain-specific and knowledge-intensive tasks, such as detecting malicious applications or websites, due to limitations in image comprehension and the propensity for hallucinations, which are resource-intensive and costly to mitigate.
Innovation Solution
A computer-implemented method and system that utilizes a Large Vision Model (LVM) with retrieval-augmented generation (RAG) and black-box access, incorporating domain-expert knowledge to extract features from user-provided images and text, and employs hallucination mitigation techniques to provide accurate risk assessments, using a risk classification model and rule-based or machine learning algorithms for decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If pre-trained Large Vision Models (LVMs) are used for domain-specific detection tasks, then the model possesses vast knowledge in wide ranges of fields, but the model's performance deteriorates significantly when the number of task-relevant documents during pre-training is reduced
Solution Approach 1:
The system performs preliminary action by curating and storing domain-specific malicious content indicators (malicious URLs, file hashes, suspicious patterns) in an external knowledge base before the actual detection task. This pre-prepared knowledge is then retrieved and integrated with the LVM's general capabilities, ensuring the model has access to task-relevant information without requiring extensive domain-specific pre-training.
Solution Approach 2:
An external knowledge base serves as an intermediary between the LVM's general knowledge and the specific detection task. The knowledge base stores curated malicious content indicators and provides them to the model during inference, acting as a bridge that supplies task-relevant information without requiring the model to be pre-trained on extensive domain-specific data.
2Loss of information
If state-of-the-art LVMs are used for complex image comprehension tasks, then the model achieves remarkable natural language processing capabilities, but the model inadequately describes the contents of the provided image
Solution Approach 1:
The system segments the detection task into two parts: (1) the LVM processes the screenshot and extracts general visual information, and (2) the external knowledge base provides specific malicious content indicators. This segmentation allows each component to focus on its strength - the LVM on general image understanding and the knowledge base on specific detection patterns - thereby improving overall image content description accuracy.
Solution Approach 2:
The system replaces the mechanical approach of training LVMs extensively on domain-specific image data with a knowledge-based retrieval approach. Instead of relying on the model to learn malicious content patterns through extensive training, the system substitutes this with retrieval of pre-curated indicators from an external knowledge base, which are then integrated with the LVM's visual processing capabilities.
3Measurement precision
If professional consultation is sought for analyzing complex information, then accurate processing of complex information is achieved, but the cost becomes highly expensive
Solution Approach 1:
The system enables self-service by providing automated detection capabilities that allow users to independently analyze screenshots and detect malicious content without requiring expensive professional consultation. The automated system combines LVM image processing with knowledge base retrieval to deliver accurate analysis at low cost, making professional-level detection accessible to individual users.
4Adaptability or versatility
If LVMs are used for detecting malicious content, then the system can process both text and images effectively, but hallucination mitigation becomes resource-intensive and costly
Solution Approach 1:
The external knowledge base acts as an intermediary that reduces the computational burden on the LVM for hallucination mitigation. By providing pre-curated, verified malicious content indicators, the knowledge base gives the model reliable reference information to anchor its predictions, reducing the need for resource-intensive verification and mitigation processes while maintaining multi-modal processing capabilities.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Computer-implemented method for detecting a malicious application or website by a transaction processing application, the method comprising: inputting a transactional request, by said transaction processing application, comprising a transactional data record and a screenshot; sending the input data record and input screenshot to a backend controller; requesting to a prompt selector, a string comprising a feature-extraction prompt; sending the input screenshot and the received prompt string to a Large Vision Model, LVM; receiving a string comprising risk classification features from said LVM; verifying the received string by a format parser; if the received string fails the verification, requesting by the backend controller, to the prompt selector, a string comprising a feature-extraction prompt which explicitly mentions format parsing compatibility, and repeating the preceding steps; sending the received string to a risk classification model for providing a risk classification; sending the risk classification to the backend controller; determining by the backend controller if the application or website is determined as malicious, and accepting or rejecting the transactional request accordingly.