Automated Source Code Documentation via AST and Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current software documentation is often incomplete, inconsistent, and outdated, leading to inefficiencies and inaccuracies in tracing functionality, and existing automatic Natural Language Generation (NLG) techniques fail to provide reliable and accurate documentation due to limitations in semantic and syntactic analysis, resulting in poor performance and high maintenance costs.
Innovation Solution
A system and method that utilize both programming language specification knowledge and open-source code knowledge to generate comprehensive natural language documentation through machine learning models, incorporating advanced summarization techniques and neural networks to create accurate and up-to-date documentation by parsing source code, building syntax trees, and consolidating documentation at various levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual documentation is prepared by people without technical documentation experience, then documentation can be created quickly, but the documentation becomes hard to read, vague, incomplete, and inaccurate
Solution Approach 1:
The patent replaces manual documentation preparation (human mechanical process) with an automated system using Abstract Syntax Tree (AST) analysis and sentence template processing. The system automatically extracts code elements, applies templates, and generates documentation without human intervention, thereby eliminating the trade-off between speed and quality that plagues manual documentation by non-experts.
Solution Approach 2:
The system enables code to document itself through automated AST analysis and template-based generation. The documentation system serves itself by automatically processing source code, extracting meaningful elements, and producing documentation without requiring external human expertise, thus achieving both high productivity and high accuracy simultaneously.
2Loss of time
If documentation is updated infrequently to save time, then maintenance costs are reduced, but the documentation becomes obsolete and outdated
Solution Approach 1:
The automated documentation system enables continuous synchronization between source code and documentation. By integrating AST analysis and template processing into the development workflow, the system maintains documentation currency automatically without requiring periodic manual updates, thus eliminating the trade-off between maintenance frequency and documentation obsolescence.
Solution Approach 2:
The system establishes a feedback loop where changes in source code automatically trigger documentation regeneration through AST analysis. This continuous feedback mechanism ensures documentation remains synchronized with the latest code version, eliminating the need for infrequent manual updates while maintaining high reliability and currency.
3Extent of automation
If existing IR-based automatic NLG techniques are used, then documentation generation is automated, but the accuracy is poor and duplication occurs in summary sentences
Solution Approach 1:
The patent segments the documentation generation process into distinct stages: AST analysis to extract code elements, sentence template application to generate accurate documentation, and consolidation to merge results. This segmentation allows each stage to be optimized independently, achieving both high automation and high accuracy while avoiding the duplication issues that plague monolithic IR-based approaches.
Solution Approach 2:
The system replaces Information Retrieval (IR) based automatic NLG with a deterministic template-based generation approach driven by AST analysis. This substitution eliminates the probabilistic nature of IR methods that lead to duplication and inaccuracy, providing precise control over documentation generation while maintaining full automation through programmatic template processing.
4Reliability
If comprehensive documentation is created manually to ensure completeness, then documentation quality improves, but the cost and time required increases significantly
Solution Approach 1:
The patent replaces complex manual documentation processes with automated AST analysis and template-based generation. The system automatically traverses the code structure, extracts all necessary elements, and applies templates to generate comprehensive documentation without human intervention, thus achieving completeness while reducing the operational complexity of the documentation system.
Solution Approach 2:
The automated system performs multiple functions simultaneously: it analyzes code structure through AST, extracts various code elements (variables, functions, parameters), applies appropriate templates, and generates comprehensive documentation in a single unified process. This multi-functionality eliminates the need for separate manual processes for different documentation aspects, reducing system complexity while maintaining completeness.
Data Source
AI summary
Systems and methods for automatically generating natural language documentation for source code files are provided. An exemplary method includes parsing a source code file to extract a function and generate an abstract syntax tree, generating first natural language documentation for each of a plurality of program statements within the function using a programming language neural network model, generating second natural language documentation for the function as a whole by processing a code snippet of the function using a function documentation neural network model, consolidating the first natural language documentation and the second natural language documentation at a function level, a source file level, and a project level to create consolidated natural language documentation, and summarizing multiple sentences of the consolidated natural language documentation into an abstract summary of the source code file by applying a set of rules.