Multi-Sequence Vector Logic Encoder for Source Code Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art feature encoding methods for logical trees are either lossy, leading to poor inference accuracy, as they either lose important topology details or require complex parsing, which decelerates training and decreases accuracy in natural language processing (NLP) models like BERT.
Innovation Solution
A novel approach that uses a multi-sequence vector generated through multiple tree traversal algorithms to train a logic encoder within a neural network, allowing for high-fidelity encoding of logical trees by concatenating distinct sequences, which can be used with existing NLP models like BERT for anomaly detection in database or browser scripts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If state of the art feature encoding is used to encode logical trees, then the encoding can be performed, but important tree topology details are lost (lossy flattening)
Solution Approach 1:
The patent segments the tree encoding process into multiple distinct sequences generated by different traversal algorithms (e.g., preorder, postorder, levelorder). Each sequence captures different aspects of tree topology, and their combination provides comprehensive encoding without loss of important structural details.
Solution Approach 2:
The patent merges multiple tree traversal sequences into a single multi-sequence vector representation. This combination of sequences from different traversal algorithms preserves comprehensive tree topology information while maintaining encoding efficiency, resolving the contradiction between information preservation and encoding simplicity.
2Measurement precision
If complex parsing is performed to capture tree structure, then encoding accuracy improves, but training time increases (decelerates training)
Solution Approach 1:
The patent performs tree parsing and sequence generation as preliminary actions during the data preparation phase. By pre-computing multiple traversal sequences for the training data, the actual training process does not need to perform complex parsing during each training iteration, thus maintaining high encoding accuracy while preventing training deceleration.
3Quantity of substance
If sparse encoding is used for logical trees, then space is saved, but inference accuracy decreases
Solution Approach 1:
The patent transitions from traditional sparse encoding to a multi-dimensional dense encoding representation. The multi-sequence vector creates a richer feature space that captures tree topology details, allowing the model to achieve high inference accuracy without relying on sparsity, effectively using dimensional expansion to overcome the accuracy-space tradeoff.
Data Source
AI summary
In an embodiment, a computer generates a multi-sequence vector that contains a plurality of distinct sequences of distinct nodes of a parse tree of source logic. Based on the multi-sequence vector, the computer trains a logic encoder. After training and in a production environment, the logic encoder infers a fixed-size encoded logic from new source logic. Based on the fixed-size encoded logic, the new source logic is detected as anomalous by an anomaly detector. Both of the logic encoder and the anomaly detector are machine learning models and, herein, they may be separately trained. In an embodiment, the logic encoder is based on a natural language processing (NLP) language model architecture such as bidirectional encoder representations from transformers (BERT), or novel training herein may be self-supervised according to skip-gram for use with an unlabeled training corpus.