Malicious URL Detection From Cross-Service User Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing detection techniques for malicious sites lack high detection accuracy, speed, and range, particularly in the context of user-generated content on the Web, such as moving images and blogs, which are often generated in real time and spread rapidly during events.
Innovation Solution
A detection device that processes user-generated content to extract key phrases, collects content using generated queries, calculates feature amounts, performs learning on normal and malicious content, and determines malignancy, accessing URLs to output threat information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a search engine is used to detect malicious sites and recursively create queries, then detection range is improved, but detection speed deteriorates
Solution Approach 1:
The system performs preliminary actions by collecting user generated content and generating search queries in advance, before actual malicious site detection is needed. This allows the detection model to be pre-trained and prepared, improving both detection speed and range when actual threats need to be identified
Solution Approach 2:
The system creates copies of detection capabilities by training a machine learning model on collected user generated content and search results. This model can then rapidly detect malicious sites without requiring real-time recursive searching, maintaining detection range while improving speed
2Adaptability or versatility
If feature amount design is expanded to capture various attacks, then detection range is improved, but detection accuracy deteriorates
Solution Approach 1:
The system applies local quality by selecting and weighting specific features differently based on the type of content and service being analyzed. Instead of treating all features equally across all contexts, the model adapts feature importance locally to maintain high accuracy while covering various attack types
Solution Approach 2:
The system changes parameters by dynamically adjusting feature selection and weighting based on the specific service and content type being analyzed. This allows the detection model to maintain high accuracy across different contexts while expanding detection range to capture various attack methodologies
3Measurement precision
If all URLs are accessed for feature amount calculation, then detection accuracy is improved, but detection speed deteriorates
Solution Approach 1:
The system applies partial action by accessing only a representative subset of URLs rather than all URLs in user generated content. The machine learning model is trained to detect malicious patterns from this partial sampling, maintaining sufficient detection accuracy while dramatically improving processing speed
Solution Approach 2:
The system enables self-service by using the collected user generated content and search results to automatically train and refine the detection model without requiring manual analysis of every URL. The model learns to identify malicious patterns efficiently, improving both speed and accuracy over time
4Measurement precision
If specialized feature amount is used for specific fraud types, then detection accuracy is improved, but detection range deteriorates
Solution Approach 1:
The system achieves universality by designing a multi-functional detection model that can handle various types of malicious content across different services and platforms. The model learns general malicious patterns from diverse user generated content while maintaining the ability to detect specific fraud types, achieving both broad detection range and high accuracy
Data Source
AI summary
A detection device includes processing circuitry configured to acquire user generated content generated in each service in a predetermined period, generate a search query using words appearing in the user generated content for each service, collect the user generated content generated in a plurality of services using the generated search query, calculate a feature amount of the collected user generated content of a predetermined service, perform learning using the feature amount of the user generated content generated by a normal user and a feature amount of content generated by a malicious user, determine whether the user generated content is generated by a malicious user based on a learned model, and access an entrance URL described in the user generated content and output a feature of an attack of the user generated content as threat information when the user generated content is determined to be generated by a malicious user.