Text Analysis System Using Semantic Vector Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to accurately determine the audience interested in an event by analyzing massive network data, lacking efficiency in identifying relevant users based on text similarity and user attributes.
Innovation Solution
A text analyzing method and device that utilize a semantic similarity algorithm for short texts, combined with Elasticsearch and word2vec models, to calculate cosine similarity between search texts and target texts, and statistically analyze user attributes to identify interested users within a preset time frame.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional text analysis methods are used to determine event audience, then the analysis process is simple, but the accuracy of determining interested users is low
Solution Approach 1:
The patent segments the text analysis process into multiple stages: first using Elasticsearch for initial similarity filtering, then applying word2vec for semantic vectorization, and finally calculating cosine similarity for precise matching. This multi-stage segmentation allows the system to achieve high accuracy while managing complexity through modular processing.
Solution Approach 2:
The patent introduces semantic vectors as an intermediary representation between raw text and similarity calculation. By converting text into word2vec vectors before computing cosine similarity, the system creates a mediating layer that captures semantic meaning, thereby improving accuracy without directly complicating the final comparison operation.
2Measurement precision
If semantic similarity algorithms are applied to all search texts, then the accuracy of audience identification is improved, but the computational time and resources increase
Solution Approach 1:
The patent applies partial action by using Elasticsearch to pre-filter search texts before applying the more computationally intensive word2vec and cosine similarity algorithms. Only texts that pass the initial filtering stage undergo full semantic analysis, reducing the overall computational burden while maintaining accuracy for relevant matches.
Solution Approach 2:
The patent performs preliminary action by using Elasticsearch to calculate initial similarity scores and filter out obviously unrelated search texts before applying the more accurate but time-consuming word2vec and cosine similarity methods. This preliminary filtering step reduces the number of texts requiring full semantic analysis.
Data Source
AI summary
A text analyzing method and device, and a server are provided by the present disclosure. The method includes: acquiring a target text and a first set of texts to be compared, wherein the first set of texts to be compared comprises at least one text to be compared, and each text to be compared is a search text entered by a user in one search; calculating a text similarity between the text to be compared and the target text based on a semantic similarity algorithm for short text, and determining a text to be compared with a text similarity higher than a first threshold as an object to be analyzed; and determining a target user entering the object to be analyzed; and statistically analyzing the target user based on at least one attribute of the target user, to generate and output an analyzing result of the target text respective to the target user.