Hybrid Embeddings for Source Code Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies fail to effectively combine contextual, semantic, and structural aspects of source code into unified representations, limiting their applicability in tasks like source code change prediction, search, quality assessment, and analytics.
Innovation Solution
The integration of sequence-processing machine learning models with graph-based data structures, such as abstract syntax trees, to generate semantically-rich and structurally-informative joint representations, known as hybrid embeddings, which combine token embeddings and graph data to capture both short-term and long-term dependencies in source code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only language domain data (token embeddings) is used to represent source code, then semantic understanding is improved, but structural information is lost
Solution Approach 1:
The patent merges language domain data (token embeddings from sequence-processing models) with graph domain data (structural representations from ASTs) to create unified joint representations. This combining approach preserves both semantic meaning from token embeddings and structural relationships from graph data, resolving the contradiction between maintaining semantic understanding and retaining structural information.
2Loss of information
If graph-based data structures are used to capture source code structure, then structural information is improved, but contextual and semantic information is lost
Solution Approach 1:
The patent combines graph domain data (which captures structural information) with language domain data (which contains contextual and semantic information from token embeddings). This merger ensures that structural relationships from the graph are preserved while simultaneously retaining the contextual and semantic understanding from the sequence-processing model, resolving the information loss contradiction.
3Device complexity
If separate domain data are used for source code analysis, then processing simplicity is improved, but representation completeness is worsened
Solution Approach 1:
The patent merges separate language domain data and graph domain data into unified joint representations, achieving complete source code representation that includes both semantic and structural information. The merging process maintains processing efficiency by using established sequence-processing models and graph-based structures as inputs, thus resolving the contradiction between processing simplicity and representation completeness.
Data Source
AI summary
Implementations are described herein for generating embeddings of source code using both the language and graph domains, and leveraging combinations of these semantically-rich and structurally-informative embeddings for various purposes. In various implementations, tokens of a source code snippet may be applied as input across a sequence-processing machine learning model to generate a plurality of token embeddings. A graph may also be generated based on the source code snippet. A joint representation may be generated based on the graph and the incorporated token embeddings. The joint representation generated from the source code snippet may be compared to one or more other joint representations generated from one or more other source code snippets to make a determination about the source code snippet.