Telecom Log Preprocessing for DNN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for analyzing software logs in 5G telecommunications networks are non-systematic, inefficient, and result in resource wastage due to incorrect analysis, failing to utilize historical data and leading to incorrect network modifications.
Innovation Solution
A preprocessing system that cleans raw data by converting markup language to text format, extracts relevant data, removes unnecessary files and data, and trains a deep neural network (DNN) model with the cleaned data to generate a trained DNN model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current techniques are used to analyze software logs, then analysis can be performed, but resource wastage occurs due to non-systematic and inefficient processing
Solution Approach 1:
The patent applies preliminary action by performing data cleaning, filtering, and preprocessing operations before the actual analysis. The system removes unnecessary files, extracts relevant data, and prepares cleaned datasets in advance, which significantly improves subsequent analysis efficiency and reduces resource wastage during the analysis process itself.
Solution Approach 2:
The patent segments the log analysis process into distinct stages: data collection, cleaning, filtering, preprocessing, and analysis. By dividing the raw log data into manageable components and processing them systematically through different modules, the system improves overall efficiency and reduces resource consumption compared to handling all data uniformly.
2Quantity of substance
If all raw data is processed without filtering, then complete data is available, but processing time and resources increase significantly
Solution Approach 1:
The patent extracts only the relevant data from the massive volume of raw logs, removing unnecessary files and filtering out irrelevant information. This extraction process maintains the quality and completeness of necessary data while dramatically reducing the total data volume that requires processing, thereby reducing processing time and resource consumption.
Solution Approach 2:
The patent applies local quality by treating different portions of the data differently based on their relevance. Rather than uniformly processing all data, the system identifies and applies specific cleaning and filtering operations to different data segments based on their characteristics and importance, optimizing the balance between data completeness and processing efficiency.
3Reliability
If historical data is not utilized, then processing is simpler, but valuable information is lost leading to incorrect analysis
Solution Approach 1:
The patent merges historical data with current log data through systematic preprocessing and integration. By combining relevant historical information with current operational data and applying consistent cleaning and filtering operations, the system improves analysis reliability and accuracy while managing complexity through standardized processing procedures.
4Productivity
If incorrect analysis is performed, then processing can be completed quickly, but wrong conclusions are drawn leading to incorrect network modifications
Solution Approach 1:
The patent performs preliminary data cleaning, validation, and preprocessing operations before analysis to ensure data quality. By preparing accurate and clean data in advance, the system enables both fast and accurate analysis, preventing incorrect conclusions while maintaining high productivity through efficient preprocessing pipelines.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A device may receive software logs identifying raw data and may convert the raw data to a text format, to generate text data. The device may extract pre-log data from the text data and may remove files with less than a threshold quantity of lines from the text data to generate modified text data. The device may extract UE data from the modified text data and may decode RRC messages in the modified text data to generate decoded RRC messages. The device may extract marker data from the modified text data and may remove files associated with timestamps and test cases from the modified text data to generate further modified text data. The device may extract test case data from the further modified text data and may generate final data that includes the pre-log data, the UE data, the decoded RRC messages, the marker data, and the test case data.