Intelligent Text Cleaning via BiLSTM-CNN Hybrid Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual removal of improper line breaks in combined texts from different formats is inefficient, non-reproducible, and costly, affecting presentation and semantic analysis.
Innovation Solution
An intelligent text cleaning method using a hybrid BiLSTM-CNN model that preprocesses text into word vectors, applies full-text matrix numeralization, and utilizes a semantic recognition classifier with word2vec reverse recovery to produce a standard text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual methods are used to remove improper line breaks, then the cleaning process can be performed, but the efficiency is low and cost-effectiveness is poor
Solution Approach 1:
The patent replaces manual mechanical text cleaning operations with an automated neural network system. The BiLSTM-CNN model automatically identifies and removes improper line breaks through semantic understanding, eliminating the need for manual intervention while significantly improving cleaning efficiency and reducing time loss.
Solution Approach 2:
The text cleaning system performs self-service by automatically detecting and correcting improper line breaks without human assistance. The neural network model independently processes the text, identifies semantic boundaries, and removes incorrect line breaks autonomously, making the system self-sufficient and highly efficient.
2Measurement precision
If manual methods are used to remove improper line breaks, then the cleaning can be performed, but the accuracy and flexibility are insufficient
Solution Approach 1:
The patent transforms text cleaning from a manual parameter-based approach to a semantic parameter approach using neural networks. The BiLSTM-CNN model dynamically adjusts its processing based on semantic context, allowing it to accurately detect improper line breaks while adapting flexibly to different text formats and sources without requiring manual reconfiguration.
Solution Approach 2:
The text cleaning system employs dynamic semantic analysis through the BiLSTM-CNN model, which adapts its detection criteria based on the contextual meaning of the text. This dynamic approach enables the system to maintain high accuracy across different text types and sources, providing versatility that static manual methods cannot achieve.
3Reliability
If manual methods are used to remove improper line breaks, then the process can be completed, but it is not reproducible across different cases
Solution Approach 1:
The patent creates a reproducible text cleaning solution by training the BiLSTM-CNN model on diverse text data. Once trained, the model can be copied and deployed across different systems and cases, ensuring consistent and reproducible cleaning results without requiring manual intervention for each new case, while maintaining ease of implementation through automated processing.
Data Source
AI summary
An intelligent text cleaning method includes: acquiring a text set, and preprocessing the text set to obtain a word vector text set; subjecting the word vector text set to a full-text matrix numeralization to generate a principal word vector matrix and a text word vector matrix; inputting the principal word vector matrix to a BiLSTM model to generate an intermediate text vector; inputting the text word vector matrix to a convolution neural network model to generate a target text vector; and concatenating the intermediate text vector and the target text vector to obtain combined text vectors, inputting the combined text vectors to a pre-constructed semantic recognition classifier model, outputting an aggregated text vector, subjecting the aggregated text vector to reverse recovery using a word2vec reverse algorithm, and outputting a standard text. The present application realizes accurate text cleaning.