Automated Data Reorganization Using Classification Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing and reorganizing large amounts of unstructured data from sources like social media, emails, business processes, and IoT devices is challenging, especially in extracting useful information from logs with different formats, which requires significant effort and is not easily manageable.
Innovation Solution
A computer-implemented method and system that classifies original data into types using a trained type classification model, analyzes severity using a trained severity classification model, and outputs messages indicating the severity, enabling efficient data reorganization and log summarization through division and summarization modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual processing and reorganization of unstructured data is performed, then data can be categorized and organized, but the effort and time required increase significantly
Solution Approach 1:
The patent replaces manual mechanical data processing with automated machine learning models. The type classification model and severity classification model automatically categorize and prioritize unstructured data without human intervention, substituting the mechanical process of manual sorting and analysis with intelligent automated systems that process data much faster and with consistent accuracy.
2Loss of information
If all unstructured data is processed in detail, then comprehensive information is extracted, but the complexity and resources required increase
Solution Approach 1:
The patent segments the data processing task into distinct classification stages. The type classification model first categorizes data into broad types, then the severity classification model further prioritizes within those types. This segmentation allows the system to process data comprehensively without overwhelming complexity, as each model focuses on a specific aspect of classification rather than attempting to analyze all data dimensions simultaneously.
3Reliability
If traditional log analysis methods are used, then all logs can be reviewed, but the effort required is not easily manageable
Solution Approach 1:
The patent implements self-service log analysis through automated classification models that independently process and prioritize logs without requiring human reviewers to manually examine each entry. The system serves itself by automatically identifying critical issues, assigning severity levels, and presenting only the most important logs for human review, making the process manageable even with large volumes of logs.
Data Source
AI summary
A method, system, and computer program product for data reorganization and logs reorganization. The method includes receiving, by one or more processing units, original data. The method also includes classifying, by the one or more processing units, the original data into different types based on a trained type classification model. The method also includes generating, by the one or more processing units, at least one severity for at least part of the original data based on a trained severity classification model, the at least part of the original data corresponding to at least one type. The method also includes outputting, by the one or more processing units, at least one message, the at least one message indicating the severity of the at least part of the original data.