Multimodal Knowledge Graph Linking for Illegal Commodity Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying illegal commodities on e-commerce platforms are limited by high establishment costs, inability to cover all implied words, and inability to identify new cheating methods, with supervised learning models requiring high labeling costs and limited to known situations.
Innovation Solution
A method utilizing a multi-modal knowledge graph to link commodity images and text to a knowledge graph, employing entity linking and a discriminative model to determine illegality based on shortest paths or subgraphs, reducing the need for illegality semantic databases and multi-modal feature learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an illegality semantic database is established to identify illegal commodities by comparing commodity attribute information words, then identification coverage for known illegal words is improved, but the establishment cost is high and merchants can easily change words to avoid monitoring
Solution Approach 1:
The patent introduces a multi-modal knowledge graph as an intermediary structure that connects commodity images, text descriptions, and legal knowledge bases. Instead of directly comparing commodity words against a static illegal words database, the system uses the knowledge graph to infer illegality through relationships between multiple entities, enabling indirect identification that is more robust to word changes while maintaining comprehensive coverage
Solution Approach 2:
The multi-modal knowledge graph serves multiple functions simultaneously: it stores legal knowledge, processes commodity images, analyzes text descriptions, and performs illegality inference. This universal structure replaces the need for separate databases and processing systems, reducing overall establishment cost while improving identification reliability through multi-dimensional analysis
2Reliability
If a supervised learning artificial intelligence model is adopted to learn multi-modal features of illegal commodities, then identification capability for known situations is improved, but the labeling cost is high and the method cannot identify new cheating words and pictures
Solution Approach 1:
The system performs preliminary actions by pre-processing and indexing commodity images and text descriptions before illegality determination. The multi-modal knowledge graph is pre-built with legal knowledge and commodity relationships, allowing the system to directly query and infer illegality without requiring extensive labeled training data for each new commodity type, thus reducing labeling cost while maintaining high identification capability
Solution Approach 2:
The knowledge graph system enables self-service identification by automatically inferring illegality based on pre-established knowledge relationships. Instead of requiring supervised learning models to be retrained for each new cheating method, the system uses its pre-built knowledge structure to adapt to new patterns autonomously, eliminating the need for costly manual labeling of new illegal commodity types
3Adaptability or versatility
If merchants use metaphors and hints to avoid platform monitoring, then their ability to evade detection is improved, but the identification system's robustness against new cheating methods deteriorates
Solution Approach 1:
The patent transitions from single-dimensional word matching to multi-dimensional analysis by processing both image and text modalities simultaneously. The multi-modal knowledge graph creates additional dimensions of analysis by establishing relationships between visual features, textual descriptions, and legal knowledge, making it difficult for merchants to evade detection through simple word changes or metaphors in one dimension
Data Source
AI summary
A method, a device, computer equipment and a storage medium for identify an illegal commodity. The method comprises: firstly, constructing a multi-modal knowledge graph according to a multi-modal knowledge graph data set, and extracting visual features of all visual modality entities and text features of all text modality entities in the knowledge graph; then obtaining a commodity image and a commodity text according to a database; then, generating commodity visual feature according to the commodity image; then generating the commodity text feature according to the commodity text; secondly, according to the visual features and text features, as well as the commodity visual feature and the commodity text feature, linking the commodity image and the commodity text to the knowledge graph by using an entity linking method; finally, obtaining the correlation between the commodity image and the commodity text according to the linked knowledge graph to determine the illegality of the commodity.