Programming Evaluation System Parallel Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated development environments (IDEs) and automated test tools lack the ability to accurately evaluate the quality of computer programs and compare programs developed by different software developers, as they rely on manual configuration and lack metrics for assessing program quality.
Innovation Solution
A computer programming evaluation system that compiles and executes user-submitted source code, using multiple programming servers for parallel execution, load balancing, and performance tuning to ensure security and scalability, while generating a graph-based quantified knowledge map to evaluate programming quality based on metrics and complexity data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated test tools are used to verify program accuracy, then testing capability is improved, but manual configuration complexity increases
Solution Approach 1:
The system automatically generates test cases from programming task descriptions without requiring manual configuration. The automated test tool executes source code submissions and compares outputs against expected results derived from the task specification, enabling self-service verification of program accuracy
Solution Approach 2:
The programming evaluation system serves multiple functions: it acts as an IDE for code submission, an automated test tool for execution and verification, and an evaluation system for quality assessment. This multi-functionality eliminates the need for separate manual configuration of test tools
2Reliability
If traditional IDE or automated test tools are used, then code execution verification is possible, but programming quality evaluation capability is lost
Solution Approach 1:
The system provides comprehensive feedback by evaluating multiple dimensions of programming quality: correctness (whether the program passes test cases), efficiency (time and space complexity analysis), and code quality (readability, maintainability, and best practices adherence). This multi-dimensional feedback enables precise measurement of programming quality
Solution Approach 2:
The system transforms the evaluation from binary pass/fail to a multi-parameter assessment including correctness, efficiency, and code quality metrics. By changing the evaluation parameters from simple execution status to comprehensive quality indicators, the system achieves precise programming quality measurement
3Measurement precision
If multiple developers' programs are compared, then quality assessment capability is improved, but system complexity increases
Solution Approach 1:
The system segments the evaluation process into distinct components: correctness checking (test case execution), efficiency analysis (time and space complexity), and code quality assessment (readability and maintainability). This segmentation allows comprehensive multi-developer comparison while managing system complexity through modular evaluation
Data Source
AI summary
Techniques are described for analysis and evaluation of different computer programs developed by different users to accomplish programming tasks. A computer programming evaluation system provides users with one or more programming tasks and ability to input source code to solve the programming tasks. In an embodiment, using a client computer system, a user initiates a computer programming session with a programming system. As a result of the session, the client computer system is provided with user interface elements allowing the user to view one or more programming tasks and submit to the programming system source code for the one or more programming tasks. In an embodiment, different modes are used by the programming system to evaluate quality of the computer programming performed by a user in terms of accomplishing the programming task. By collecting result metrics from evaluations and comparing with complexity data of the programming tasks, the programming system generates a graph-based quantified knowledge map of computer programming for a user. Non-limiting examples of modes for evaluation of the computer programming include a practice mode, an arcade mode, versus mode, a bot mode and a bot training mode. In an embodiment, to execute source codes in parallel, a programming system may include multiple programming servers. Each programming server is a separate computer system that, in some embodiments, is a cloud computer system allocated from cloud services. Executing different instances of source code on different programming servers improves both the security and scalability of the programming system.