Automated Examination Assessment System with Targeted Feedback for Mathematical and Algorithmic Queries

Abstract

An automated examination marking system is disclosed for assessing student answers to mathematical or algorithmic questions. The system places particular emphasis on both the answer and the accompanyi

Description

[002] In the ever-evolving landscape of education technology, a consistent challenge faced by educators and institutions alike has been the effective and efficient evaluation of students' comprehension and application of mathematical or algorithmic concepts. Traditional exam grading systems primarily focus on the end-result, i.e., the final answer submitted by the student, often overlooking the process or workings leading to that answer. This singular focus on the result often leaves both educators and students in the dark about the nuances of the student's understanding, mistakes, and misconceptions. Furthermore, manually grading student's workings is timeconsuming, labor-intensive, and subject to human error or inconsistency, making it less reliable and scalable for larger student populations.

[003] Software solutions developed to address this challenge have provided limited relief. Automated exam marking software in the past has largely replicated the limitations of manual grading by primarily evaluating the correctness of the final answer. While some of these software tools have incorporated mechanisms to analyze students' workings, their analysis has been rudimentary at best, lacking in depth and understanding of the students' thought processes. Such systems are unable to provide constructive feedback, leaving students without clarity on their mistakes or guidance on how to rectify them. Simply put, these systems, while automating the grading process, have failed to enhance the learning experience.

[004] Another significant drawback in the previous systems has been the inflexibility in student input methods. With the increasing integration of technology into classrooms, students employ diverse methods for problem-solving, from traditional paper and pen to digital note-taking. The lack of versatility in input methods in prior software solutions constrains students' natural problem-solving approaches and can even introduce additional barriers to accurate assessment.

[005] The motivation for developing the present invention stems from these glaring inadequacies in the existing automated examination marking systems. The inventors recognized an urgent need for a comprehensive software solution that doesn't just mechanize the grading process, but elevates it. A system that seamlessly blends with students' diverse problem-solving methods, dives deep into their workings, identifies misconceptions, and provides targeted, constructive feedback. By addressing these unmet needs, the current invention aims to transform not just the grading process, but the very essence of learning from evaluations.

[006] It is within this context that the present invention is provided. Summary

[007] The present invention provides a novel and advanced software method tailored for automarking student answers to mathematical or algorithmic examination questions. At its core, this system delves deep into not just the final answers but also the student's workings, identifying misconceptions and dispatching pre-defined feedback to aid the student's understanding. This multi-dimensional approach to grading is transformative, offering educators and students an unparalleled depth of insight into the learning and problem-solving process, thus significantly enhancing the educational experience.

[008] In some embodiments, the invention encompasses a designated database wherein mathematical or algorithmic questions are recorded. Such a structured organization allows for streamlined question retrieval and systematic correlation with potential misconceptions, streamlining the evaluation process.

[009] In certain embodiments, misconceptions relevant to a particular question are not only listed but also intricately linked with the question within the database. This integrated approach ensures that each question has a tailor-made set of misconceptions, allowing for a more nuanced and precise analysis of student answers.

[010] In other embodiments, the invention identifies specific student answer inputs that signify a particular misconception. By preemptively mapping potential errors to misconceptions, the system achieves a heightened level of precision in its feedback, ensuring that students receive targeted and relevant guidance. [Oil] In some iterations, the system is equipped to craft and display feedback for each identified misconception. This feedback typically elucidates what the student did incorrectly juxtaposed against the correct approach. By visually differentiating the erroneous steps (in red text) from the correct ones (in green text), students receive clear and actionable feedback, thus facilitating better comprehension and retention.

[012] In certain configurations, the system incorporates an optional hint mechanism relevant to each misconception. These hints serve as gentle nudges, directing students towards the right track without directly revealing the solution. This feature enhances the learning experience by providing students with an opportunity to rectify their errors based on minimal guidance, fostering independent problem-solving skills.

[013] In specific embodiments, the software solution offers flexibility in the mode of submission analysis. Whether it's evaluating answers post-submission or analyzing answers line-by-line as students input their workings, the system adapts to varying examination setups. This dynamic mode of operation ensures timely feedback, thereby potentially increasing student engagement and comprehension during exams.

[014] In some versions, the invention expands its horizons beyond traditional text input boxes. Recognizing the varied problem-solving methodologies employed by students, the system is equipped to transcribe handwritten workings from images, ensuring that students can operate within their comfort zones while still benefiting from the advanced analytical prowess of the system.

[015] The present invention addresses the critical gaps in existing educational technology, offering a comprehensive, flexible, and nuanced approach to auto-marking student answers. Brief Description of the Drawings

[016] Various embodiments of the invention are disclosed in the following detailed description and accompanying drawings.

[017] FIG.l illustrates a flowchart depicting the primary methodological steps of the invention.

[018] FIG.2A provides a screenshot showcasing a student's interface while answering a question, emphasizing the freeform text workings box and specialized buttons for mathematical expressions.

[019] FIG.2B displays a screenshot of an answered question that has been fully marked, emphasizing the line-by-line analysis of the student's workings.

[020] FIG.2C reveals a screenshot where the system has identified a common mathematical misconception, presenting feedback on the correct method and a detailed explanation.

[021] FIG.3 presents a screenshot demonstrating the application of the invention's method to a student's handwritten answer, converted via Optical Character Recognition (OCR) and subsequently analyzed by the system.

[022] Common reference numerals are used throughout the figures and the detailed description to indicate like elements. One skilled in the art will readily recognize that the above figures are examples and that other architectures, modes of operation, orders of operation, and elements / functions can be provided and implemented without departing from the characteristics and features of the invention, as set forth in the claims. Detailed Description and Preferred Embodiment

[023] The following is a detailed description of exemplary embodiments to illustrate the principles of the invention. The embodiments are provided to illustrate aspects of the invention, but the invention is not limited to any embodiment. The scope of the invention encompasses numerous alternatives, modifications and equivalent; it is limited only by the claims.

[024] Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. However, the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured. DEFINITIONS:

[025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. 4

[026] As used herein, the term "and / or" includes any combinations of one or more of the associated listed items.

[027] As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well as the singular forms, unless the context clearly indicates otherwise.

[028] It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. DESCRIPTION OF DRAWINGS

[030] The present invention pertains to a method and system aimed at auto-marking student answers to mathematical or algorithmic examination questions. Unlike traditional systems that largely focus on the final answer, the invention delineates itself by paying heed to the students' intermediate workings and the thought processes that precede the conclusion. By leveraging this unique approach, the system offers insights into the understanding and misconceptions held by the student, thereby promoting more tailored feedback.

[031] Central to the system is a comprehensive database storing questions and associated misconceptions. Each question is tagged with potential misconceptions that students might harbor. The questions may be predefined or may be created using generative algorithms.

[032] Upon receipt of a student's response, the invention evaluates the final answer to determine its correctness and scours the intermediate workings, assigning marks for permutations of correct workings and seeking any patterns that align with the pre-stored misconceptions. In the event a match is found, the system generates targeted feedback. This feedback, often bifurcated into what the student did and what they ought to have done, offers clear delineations and learning paths.

[033] The method generally works via free text input, but also handles hand-scribed responses through image uploads. The latter, upon upload, undergoes optical character recognition (OCR) technology, converting handwritten content into machine-readable text.

[034] Referring now to FIG. 1, a flowchart representing the method of the invention is illustrated. The process initiates with a start 100, which leads to the question input 102. At this stage, a mathematical or algorithmic question is entered into the system and stored within an associated database. This database ensures structured storage and easy retrieval of questions, contributing to the system's efficiency.

[035] Once the question is inputted and saved, the system progresses to the misconception database creation 104. Here, for the provided question, potential misconceptions are identified and stored. Each misconception is linked to its corresponding question.

[036] Upon setting up the misconception database, the method advances to the student response input. This stage offers dual options to accommodate diverse student preferences. Firstly, the typed response 106 allows students to manually type their answer and workings into the designated interface.

[037] Alternatively, an image upload 108 can be employed for students using traditional handwritten methods. They can easily scribe their workings on paper, capture an image, and upload it to the system. Should the image upload option be chosen, the process transitions to OCR conversion 110. The optical character recognition (OCR) technology converts the handwritten contents of the uploaded image into textual data.

[038] Following the student response input, either typed or converted from the image, the answer analysis 112 is initiated. The system checks the final answer while concurrently evaluating the workings, line-by-line.

[039] The next phase is the misconception identification 114. Leveraging the earlier created misconception database, the student's response undergoes a comparison to detect any matching misconceptions.

[040] Once a misconception, if any, is pinpointed, the method moves to the feedback generation. Tailored feedback, founded on the detected misconception, is shown. In some embodiments, an initial hint display 116 is presented first. This hint gently guides the student towards the correct approach without directly revealing the answer.

[041] If hints are not employed or once the student interacts with the hint, the full feedback display 118 is showcased. This feedback is comprehensive, detailing "what the student has done" juxtaposed against "what the student should have done". By color-coding these segments - the former in red and the latter in green - the system further aids in clarity and visual understanding.

[042] Finally, the process concludes, marking its end 120. The entire flow illustrated in FIG. 1 exemplifies an advanced auto-marking system enriched with feedback mechanisms that prioritize student understanding and learning.

[043] FIG. 2A portrays a screenshot of a student's interface while they answer a question. Displayed prominently on the screen is the question followed by an expansive box 202 designated for the student's workings. In this box, students are able to draft their intermediate steps and thought processes, entering them in freeform text.

[044] Complementing the workings box are a series of buttons 204 positioned below. These buttons are facilitate the input of algebraic and mathematical expressions into the text box. Further down, another box 206 is provided for the student to input their final answer.

[045] Proceeding to FIG. 2B, a screenshot is shown of a question that has undergone the marking process. The student's response has been evaluated, including their workings in the previously mentioned freeform text box 202. The system's algorithm has undertaken a line-by-line scrutiny of these workings, juxtaposing each line with a repository of correct permutations of workings in the database, with marks assigned as a result indicated by markings 208 — two marks for the correct workings and an additional mark for the final answer in box 206.

[046] In FIG. 2C, a screenshot shows a scenario where the algorithm has discerned an error in the student's approach to fraction addition.

[047] The answer provided aligns with a frequently incorrect response stored in the database. Absence of detailed workings in box 202 compounds the error, allowing the algorithm to promptly map the answer to a common misconception. As a result, the student is shown a notification 210 that states "misconception found." Alongside, the notification is the correct method of fraction addition 212 in green, an example of the incorrect method in red 214, and an accompanying explanation 216.

[048] FIG. 3 shows the system's capability to accommodate handwritten answers, a mode commonly prevalent in traditional examination settings. Featured predominantly on the left of the screenshot is an original image 302, which captures a student's handwritten response to a question, inscribed on paper.

[049] Optical Character Recognition (OCR) technology is then used to transcribe the handwritten content of the original image 302 into machine-readable text 304. Following this OCR process, the machine-readable text undergoes the same method as described above, comparing the derived workings and answers with the comprehensive database, ensuring each line is scrutinized and evaluated for its correctness. The confidence rating 306 of the OCR process is also shown.

[050] In this example, the student has achieved full marks for their workings and answer. An indication 308 adjacent to the transcribed workings affirms that they have been awarded full marks for their working process.

[051] Although the examples described above utilize free text and OCR image recognition as input methods, a person of skill in the art reading the present application will understand that any other suitable input methods could be used, including but not limited to interactive widgets, checkboxes, and other computer interface mechanisms. NETWORK COMPONENTS

[052] The methods and systems described herein can be performed by any suitable type of computer.

[053] A computer may be a uniprocessor or multiprocessor machine. Accordingly, a computer may include one or more processors and, thus, the aforementioned computer system may also include one or more processors. Examples of processors include sequential state machines, microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on a chip (SoC), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, programmable control boards (PCBs), and other suitable hardware configured to perform the various functionality described throughout this disclosure.

[054] Additionally, the computer may include one or more memories. Accordingly, the aforementioned computer systems may include one or more memories. A memory may include a memory storage device or an addressable storage medium which may include, by way of example, random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), electronically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), hard disks, floppy disks, laser disk players, digital video disks, compact disks, video tapes, audio tapes, magnetic recording tracks, magnetic tunnel junction (MTJ) memory, optical memory storage, quantum mechanical storage, electronic networks, and / or other devices or technologies used to store electronic content such as programs and data. In particular, the one or more memories may store computer executable instructions that, when executed by the one or more processors, cause the one or more processors to implement the procedures and techniques described herein. The one or more processors may be operably associated with the one or more memories so that the computer executable instructions can be provided to the one or more processors for execution. For example, the one or more processors may be operably associated to the one or more memories through one or more buses. Furthermore, the computer may possess or may be operably associated with input devices (e.g., a keyboard, a keypad, controller, a mouse, a microphone, a touch screen, a sensor) and output devices such as (e.g., a computer screen, printer, or a speaker).

[055] The computer may advantageously be equipped with a network communication device such as a network interface card, a modem, or other network connection device suitable for connecting to one or more networks.

[056] A computer may advantageously contain control logic, or program logic, or other substrate configuration representing data and instructions, which cause the computer to operate in a specific and predefined manner as, described herein. In particular, the computer programs, when executed, enable a control processor to perform and / or cause the performance of features of the present disclosure. The control logic may advantageously be implemented as one or more modules. The modules may advantageously be configured to reside on the computer memory and execute on the one or more processors. The modules include, but are not limited to, software or hardware components that perform certain tasks. Thus, a module may include, by way of example, components, such as, software components, processes, functions, subroutines, procedures, attributes, class components, task components, object-oriented software components, segments of program code, drivers, firmware, micro code, circuitry, data, and / or the like.

[057] The control logic conventionally includes the manipulation of digital bits by the processor and the maintenance of these bits within memory storage devices resident in one or more of the memory storage devices. Such memory storage devices may impose a physical organization upon the collection of stored data bits, which are generally stored by specific electrical or magnetic storage cells.

[058] The control logic generally performs a sequence of computer-executed steps. These steps generally require manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, or otherwise manipulated. It is conventional for those skilled in the art to refer to these signals as bits, values, elements, symbols, characters, text, terms, numbers, files, or the like. It should be kept in mind, however, that these and some other terms should be associated with appropriate physical quantities for computer operations, and that these terms are merely conventional labels applied to physical quantities that exist within and during operation of the computer based on designed relationships between these physical quantities and the symbolic values they represent.

[059] It should be understood that manipulations within the computer are often referred to in terms of adding, comparing, moving, searching, or the like, which are often associated with manual operations performed by a human operator. It is to be understood that no involvement of the human operator may be necessary, or even desirable. The operations described herein are machine operations performed in conjunction with the human operator or user that interacts with the computer or computers.

[060] It should also be understood that the programs, modules, processes, methods, and the like, described herein are but an exemplary implementation and are not related, or limited, to any particular computer, apparatus, or computer language. Rather, various types of general-purpose computing machines or devices may be used with programs constructed in accordance with some of the teachings described herein. In some embodiments, very specific computing machines, with specific functionality, may be required.

[061] Unless otherwise defined, all terms (including technical terms) used herein have the same meaning as commonly understood by one having ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[062] The disclosed embodiments are illustrative, not restrictive. While specific configurations of the method and system have been described in a specific manner referring to the illustrated embodiments, it is understood that the present invention can be applied to a wide variety of solutions which fit within the scope and spirit of the claims. There are many alternative ways of implementing the invention.

[063] It is to be understood that the embodiments of the invention herein described are merely illustrative of the application of the principles of the invention. Reference herein to details of the illustrated embodiments is not intended to limit the scope of the claims, which themselves recite those features regarded as essential to the invention.

Claims

What is claimed is:

1. A computer-implemented method for marking a student's response to mathematical or algorithmic questions, the method comprising:Storing a question within a database;Associating, within said database, a list of misconceptions relevant to said question;Identifying specific student answer inputs that signify the misconceptions;Generating a feedback notification corresponding to each identified misconception;Displaying the feedback notification to the student.

2. The method of Claim 1, further comprising:Crafting a hint corresponding to each misconception, wherein said hint provides guidance to redirect the student towards the correct methodology without unveiling the complete solution; andDisplaying the hint to the student prior to displaying the full feedback notification upon identification of a misconception.

3. The method of Claim 1, wherein said feedback notification's error illustrating segment is highlighted in a first color and the correct methodology segment is highlighted in a second color.

4. The method of Claim 1, wherein the student inputs their answer and workings through a multi-line text area interface.

5. The method of Claim 1, further comprising:Employing optical character recognition (OCR) technology to transcribe handwritten student workings captured in an image format into textual data; andImporting said textual data into an input box for analysis and feedback generation.

6. The method of Claim 1, wherein the system evaluates the student's submission either post the final submission or on a line-by-line basis as the student inputs their workings.

7. The method of Claim 6, wherein in the line-by-line evaluation mode, the student is provided with an opportunity to correct their answer upon the identification of a misconception prior to a final submission.

8. The method of Claim 1, wherein said misconceptions are pre-defined, and each misconception is associated with one or more specific student answer inputs that signify the misconception.

9. The method of Claim 1, wherein said feedback notification further comprises an explanatory section, providing an in-depth understanding of the underlying subject matter associated with the identified misconception.

10. The method of Claim 1, further comprising analyzing both the student's final answer and the individual lines of their workings to determine if any part corresponds to the predefined misconceptions.

11. The method of Claim 1, wherein the question is a mathematical or algorithmic question.

12. The method of Claim 1, wherein said feedback notification comprises a segment illustrating the student's error and a segment indicating the correct methodology.

13. The method of Claim 1, wherein the method further comprises carrying out an automatic line by line marking process on the student answer inputs.

Images