Skill Extraction System Disambiguation and De-duplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Social networking services face challenges in accurately identifying and ranking user skills due to subjective and potentially fraudulent member-submitted skill lists, lacking indication of proficiency, and ambiguity in skill descriptions.
Innovation Solution
A method and system for extracting, disambiguating, and de-duplicating skill seed phrases from member profiles, using a combination of natural language processing, machine learning, and crowdsourcing to create a standardized list of skills, tagging members with skills, and ranking them based on implicit and explicit factors, including connections and external activities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If member-submitted skill lists are used to identify user skills, then the system can quickly gather skill information, but the accuracy and reliability of skill identification deteriorates due to subjectivity and potential fraud
Solution Approach 1:
The patent introduces an intermediary verification mechanism where skill information submitted by members is cross-checked against external data sources (employer profiles, education institutions, project repositories) before being accepted into the skill database. This intermediary validation layer maintains the efficiency of user-submitted data collection while significantly improving accuracy by filtering out fraudulent or inaccurate entries through automated verification against authoritative external sources.
2Quantity of substance
If detailed skill descriptions are collected from member profiles, then more comprehensive skill information is obtained, but ambiguity in skill descriptions increases making standardization difficult
Solution Approach 1:
The patent replaces manual skill description standardization with an automated natural language processing system. The NLP system analyzes unstructured skill descriptions from member profiles, extracts key skill entities, and automatically maps them to standardized skill taxonomy categories. This substitution of manual mechanical standardization with automated computational processing maintains comprehensive skill information while eliminating ambiguity through consistent algorithmic classification.
3Reliability
If skill lists are standardized for consistency, then skill identification reliability improves, but the complexity of the skill extraction and processing system increases
Solution Approach 1:
The patent implements preliminary action by pre-establishing a comprehensive skill taxonomy database with standardized skill categories, relationships, and validation rules before the skill extraction process begins. This pre-configured framework allows the system to automatically map and standardize incoming skill descriptions without requiring complex real-time processing logic, thereby maintaining high reliability while managing system complexity through advance preparation of the classification structure.
4Measurement precision
If multiple data sources are integrated to verify skills, then fraud detection capability improves, but the time and computational resources required for skill verification increase
Solution Approach 1:
The patent applies partial action by implementing a tiered verification approach where not all skill entries undergo the full verification process. High-frequency, well-established skills from reputable sources are accepted with minimal verification, while unusual or high-risk skill claims trigger more intensive multi-source verification. This selective application of verification intensity maintains high accuracy for critical cases while reducing overall processing time and resource consumption by avoiding exhaustive verification of all entries.
Data Source
AI summary
In an example, disclosed is a machine automated method of identifying a set of skills. In some examples, the method includes extracting a plurality of skill seed phrases from a plurality of member profiles of a social networking site, creating a plurality of disambiguated skill seed phrases by disambiguating the plurality of skill seed phrases using one or more computer processors, and de-duplicating the plurality of disambiguated skill seed phrases to create a plurality of de-duplicated skill seed phrases.