Word Embeddings for Sensitive Text Detection in Code
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The sheer volume of programming code scripts in large organizations makes manual inspection for sensitive information, such as passwords and credentials, onerous and inefficient, especially given the dynamic nature of code bases.
Innovation Solution
A computer-implemented method using a neural network trained on a corpus of programming code scripts to generate word embeddings, which allows for the identification of sensitive text by calculating proximity in a vector or matrix space, thereby automating the scanning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual inspection of programming code scripts is performed, then sensitive information can be detected, but the process becomes onerous and inefficient due to the sheer volume of code scripts
Solution Approach 1:
The patent replaces manual mechanical inspection with an automated computational system using neural networks and word embeddings. The system automatically scans code scripts by converting code into text, generating word embeddings, and comparing them against sensitive information patterns in vector space, eliminating the need for manual review while maintaining detection accuracy.
Solution Approach 2:
The patent introduces word embeddings as an intermediary representation layer between the code scripts and the detection process. By converting code into text representations and then into vector embeddings, the system creates a mediating format that enables efficient automated comparison and pattern matching against sensitive information signatures.
2Adaptability or versatility
If the code base is made dynamic with ongoing changes and additions, then the system remains adaptable, but manual inspection becomes increasingly onerous
Solution Approach 1:
The patent enables continuous automated scanning that can operate alongside ongoing code changes. The system processes code scripts dynamically as they are added or modified, maintaining continuous protection without requiring periodic manual re-inspection, thus adapting to the dynamic nature of code bases while minimizing time loss.
3Productivity
If automated scanning using traditional methods is used, then processing speed increases, but the ability to detect nuanced sensitive information decreases
Solution Approach 1:
The patent transforms the detection parameters by moving from exact string matching to semantic similarity measurement in vector space. By changing the parameter space from discrete text patterns to continuous vector representations, the system achieves both high-speed automated processing and nuanced detection of sensitive information through cosine similarity calculations.
Data Source
AI summary
Exemplary embodiments may use word embeddings to enhance scanning of programming code scripts for sensitive subject matter, such as confidential subject matter. The scanning may be performed by a neural network in some exemplary embodiments. The neural network initially may be trained on a corpus of programming code scripts to identify keywords relating to sensitive subject matter, such as passwords, tokens or credentials. The neural network may not only identify instances of the keywords but also may identify related terms as well. The output of the scan may be a ranked list of terms in the programming code script that may relate to sensitive subject matter.