Sentiment Analysis Using CNN and Co-occurrence Graph Bigrams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current big data systems face challenges in analyzing data due to noise, duplication, and heterogeneity, requiring substantial manual processes for data cleaning and curation, which become impractical as data volumes increase, and existing sentiment analysis techniques struggle to accurately capture the meaning and sentiment of words, especially in relation to other words or arrangements.
Innovation Solution
A computer-based system employing convolutional neural networks (CNNs), co-occurrence graphs, and bigrams to improve sentiment analysis accuracy by processing data through a semantic pipeline that includes ingestion, preparation, enrichment, and publication stages, automating data onboarding and enabling efficient sentiment analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual processes are used to clean and curate data, then data quality is improved, but productivity deteriorates as data volumes increase
Solution Approach 1:
The system uses automated machine learning models and algorithms to perform data cleaning and curation tasks independently without human intervention. The system self-services by automatically detecting noise, duplicates, and formatting issues in data, then applying appropriate cleaning transformations to improve data quality while maintaining high productivity even as data volumes scale.
Solution Approach 2:
Manual mechanical processes of data cleaning are replaced with automated computational systems using machine learning models, natural language processing algorithms, and automated data quality assessment tools. This substitution enables the system to process large volumes of data efficiently while maintaining or improving data quality through intelligent automated detection and correction.
2Measurement precision
If existing sentiment analysis techniques are used, then processing speed is maintained, but measurement precision deteriorates in capturing word meaning and sentiment
Solution Approach 1:
The sentiment analysis system combines multiple analytical approaches and data sources into a composite analysis framework. It integrates traditional sentiment analysis with advanced natural language processing techniques, contextual understanding models, and co-occurrence network analysis to capture word meanings and sentiments more accurately. This composite approach improves measurement precision by leveraging the strengths of multiple methods while managing complexity through systematic integration.
3Adaptability or versatility
If data from multiple heterogeneous sources is merged, then adaptability is improved, but difficulty of detecting and measuring worsens due to heterogeneity
Solution Approach 1:
The system implements a universal data processing framework that can handle multiple heterogeneous data sources through common processing pipelines. It uses standardized data ingestion interfaces, unified data models, and multi-functional processing algorithms that work across different data types and formats. This universality enables seamless integration of diverse data sources while maintaining the ability to detect and measure data meaning through consistent analytical approaches.
Solution Approach 2:
The system introduces intermediary processing layers including data normalization routines, contextual embedding models, and semantic interpretation algorithms that mediate between heterogeneous data sources and the analysis engine. These intermediaries translate diverse data formats and structures into a unified representation that preserves meaning and enables accurate detection and measurement across different data types.
Data Source
AI summary
Techniques are provided for performing sentiment analysis on words in a first data set. An example embodiment includes generating a word embedding model including a first plurality of features. A value indicating sentiment for the words in the first data set can be determined using a convolutional neural network (CNN). A second plurality of features are generated based on bigrams identified in the data set. The bigrams can be generated using a co-occurrence graph. The model is updated to include the second plurality of features, and sentiment analysis can be performed on a second data set using the updated model.