Text Verticalization Model Using N-gram Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional verticalization processes in text translation systems are error-prone and require significant human effort for quality assurance, as they rely on rule-based systems that often produce unacceptable results.
Innovation Solution
A system is trained to predict whether to verticalize translated text using a model that maps text features to flag values, where n-grams are used to determine correct replacements, reducing the need for human review by automatically finalizing verticalization based on generated flag values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a rule-based system is used for verticalization, then the process can be automated, but the accuracy and quality of the replacement texts deteriorate
Solution Approach 1:
The patent replaces the rule-based mechanical system with a machine learning model that learns patterns from training data. The model uses n-gram features and supervised learning to predict appropriate verticalization replacements, substituting the rigid rule-based approach with a data-driven statistical system that captures linguistic patterns more effectively.
Solution Approach 2:
The patent changes the parameters of the verticalization system by introducing multiple n-gram features (unigrams, bigrams, trigrams) and using a machine learning model with adjustable parameters that are optimized during training. This allows the system to adapt to different contexts and improve prediction accuracy compared to fixed rule-based parameters.
2Extent of automation
If a rule-based verticalization system is used, then automation is achieved, but human review effort increases due to poor quality results
Solution Approach 1:
The patent replaces the rule-based mechanical system with a machine learning model that learns patterns from training data. The model uses n-gram features and supervised learning to predict appropriate verticalization replacements, substituting the rigid rule-based approach with a data-driven statistical system that captures linguistic patterns more effectively.
Solution Approach 2:
The system performs self-correction and self-improvement by using training data that includes feedback on correct and incorrect replacements. The machine learning model automatically learns from these examples and improves its predictions without requiring continuous human intervention, enabling the system to serve itself and reduce dependency on human reviewers.
3Productivity
If machine learning is used to predict verticalization, then human effort is reduced, but system complexity increases
Solution Approach 1:
The patent segments the text into n-gram features (unigrams, bigrams, trigrams) that are processed independently by the machine learning model. This segmentation allows the complex prediction task to be broken down into manageable feature extraction and classification steps, making the system more tractable and easier to implement despite the underlying complexity.
Solution Approach 2:
The patent introduces n-gram features as intermediary representations between the input text and the final prediction. These features serve as a bridge that transforms raw text into a structured format suitable for machine learning processing, simplifying the overall system architecture by creating a clear pipeline from input to output.
Data Source
AI summary
Systems and methods include acquisition of a plurality of text segments, each of the text segments associated with a flag value indicating whether the text segment is associated with a correct replacement text or an incorrect replacement text, determination of one or more n-grams of each text segment of the plurality of text segments, generation, based on the one or more n-grams of each text segment and the flag value associated with each text segment, a model to determine a flag value based on one or more input n-grams, reception of an input text segment, determination of a second one or more n-grams of the input text segment, determination, using the model, of an output flag value based on the determined second one or more n-grams, and presentation of the input text segment and the output flag value on a display.