Transformer Model Positional Encoding for Telecommunications Event Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for detecting events in telecommunications networks, such as those using regular ML models or deep RNNs, are inaccurate and inefficient, particularly in high-speed streaming networks, due to their inability to effectively capture sequential behavior and perform real-time predictions.
Innovation Solution
A computer-implemented method that organizes data items from telecommunications network features into sequences based on time and encodes them with positional information for use in training a machine learning model, enabling more accurate and efficient event prediction by utilizing a transformer model for sequence learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a deep RNN is used to learn contexts from sequential behaviors in the network, then some contexts can be captured, but it performs poorly for longer sequences and has high training cost
Solution Approach 1:
The patent uses positional encoding to create a copy of position information and adds it to the data items in the sequence. This allows the model to access position information without requiring complex sequential processing, enabling parallel computation while maintaining the ability to capture contextual relationships in long sequences.
2Productivity
If a regular ML model is used for event detection, then training is fast, but it cannot capture sequential behavior characteristics in the network
Solution Approach 1:
The patent segments the sequential data into individual data items while preserving their position information through encoding. This allows the model to process each item independently in parallel (maintaining training speed) while still capturing sequential relationships through the positional encoding information.
3Reliability
If a deep RNN is used for real-time prediction of events in the network, then sequential contexts can be learned, but inference time is too long for practical application
Solution Approach 1:
By copying position information into the data items through positional encoding, the model can process all sequence positions simultaneously during inference without the sequential dependency that slows down RNNs, achieving both high accuracy and fast real-time prediction.
Data Source
AI summary
There is provided a computer-implemented method for processing data items for use in training a machine learning model to identify a relationship between the data items. The data items correspond to one or more features of a telecommunications network. For each feature of the one or more features, the corresponding data items are organised into a sequence according to time to obtain at least one sequence of data items. A single sequence of data items having the at least one sequence of data items is encoded to obtain an encoded sequence of data items. The single sequence of data items is encoded with information indicative of a position of data items in the single sequence of data items. The encoded sequence of data items is for use in training the machine learning model to identify the relationship between the data items.