Automated Grading of Algorithm Code Using Isolated Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing solutions for grading computer science assignments, particularly those involving big data analysis, are inadequate as they cannot efficiently evaluate and grade algorithms in a secured manner, as they rely on predefined correct answers and fail to account for the complexity and variability of big data insights, which may have infinite logically equivalent solutions and require execution in a computing environment, posing risks from malicious code.

Innovation Solution

A grading system that utilizes secured isolated execution environments for code processing, implements fuzzy logic functions for approximate grading, and includes features like validity checking and machine learning metrics to evaluate the completeness and accuracy of submitted code, ensuring secure and comprehensive assessment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If existing web-based grading solutions are used, then grading can be automated for questions with predefined answers, but they cannot evaluate programming assignments where correct answers are not predetermined

Engineering Contradiction:

Improvegrading automationVSAvoidapplicability to programming assignments

Core Design Contradiction:

Extent of automationVSAdaptability or versatility

Solution Approach 1:

The patent introduces an intermediary execution environment that acts as a mediator between the student's code and the grading system. This environment executes the code and captures its output, allowing the system to evaluate programming assignments without requiring predefined correct answers. The execution environment serves as the intermediary that transforms ungraded code into evaluable results.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If code execution is allowed for grading, then programming assignments can be evaluated, but malicious code can tamper with the grading program and harm the execution environment

Engineering Contradiction:

Improvecapability to evaluate codeVSAvoidsecurity risks from malicious code

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent implements beforehand cushioning by creating a protected execution environment with pre-established security measures. The system prepares the execution environment in advance with safeguards that prevent malicious code from harming the grading program. This includes isolation mechanisms and validation protocols that are in place before code execution begins, cushioning the system against potential attacks.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Solution Approach 2:

The execution environment serves as an intermediary layer that separates student code from the grading system. This mediator allows code evaluation while preventing direct access to and tampering with the grading program. The intermediary captures and validates code execution without exposing the underlying grading infrastructure to malicious actions.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If multiple code processing engines are used, then code can be executed in isolated environments, but system complexity increases

Engineering Contradiction:

Improvesecurity and isolation of code executionVSAvoidnumber of processing engines

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the code execution function into separate processing engines. Each engine operates as an isolated unit that can execute code independently. This segmentation provides security and reliability by containing potential issues within individual engines while maintaining the overall system's integrity. The segmented architecture allows parallel processing without inter-engine contamination.

Inventive Principle:

Principle #1Segmentation

4Adaptability or versatility

If approximate grading functions are used, then big data algorithms with infinite correct answers can be evaluated, but grading precision decreases

Engineering Contradiction:

Improveability to grade open-ended algorithmsVSAvoidgrading accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent implements self-service by allowing the code execution results to serve as the primary basis for grading. Instead of requiring the system to predefined all correct answers, the execution environment captures the actual output of student code and compares it against expected outcomes. This self-service approach enables evaluation of open-ended algorithms while maintaining reasonable grading precision through automated result validation.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS10140886B2Automated assessment and grading of computerized algorithms

Publication Date: 2018.11.27 DATA SCIENCE EVANGELISTS INC

AI summary

A grading system and method for grading a user solution to a computing assignment are presented. The method includes receiving a program code submitted by a user, wherein the received program code is the user solution to the computing assignment; activating at least one code processing engine, wherein each code processing engine is a secured isolated execution environment; executing the program code in the at least one activated code processing engine to produce an answer; determining a grade for the answer based on an expected answer and an approximate grading function, wherein the approximate grading function is determined based on a type of the computing assignment; and returning the grade to the user.