A system enabling a student to work with a large language model while ensuring integrity of student's work

Abstract

A system for ensuring the integrity of student work and providing feedback includes a professor module. The professor module creates a lesson formed from at least one scaffold containing information used to provide feedback. Scaffolds are a series of structured assignment steps. The system includes a student module in communication with the professor module. The student module prepares the assignments. An intelligent system module communicates with the professor module and the student module. The intelligent system module monitors activity of the student performed by the student in preparing the assignment defined by the scaffold. The intelligent system module, in response thereto, determines whether a student activity aligns with a student written response forming the assignment.

Description

A SYSTEM ENABLING A STUDENT TO WORK WITH A LARGE LANGUAGEMODEL WHILE ENSURING INTEGRITY OF STUDENT’S WORKCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 557,769 filed February 26, 2024, the entirety of which is incorporated by reference herein as if fully set forth.BACKGROUND OF THE INVENTION

[0002] The present invention is directed to a system for making use of artificial intelligence and large language models in schoolwork, and more particularly, a system enabling a professor, student and large language model to work together while ensuring the integrity of the student’s work.

[0003] In academia it has always been an issue whether the work being done is original work, that it is the work of the author, and that it has complied with academic standards of scholarly works in the field. The scholastic system has relied upon the skill and experience of either a panel of reviewing peers, or in the classroom, the experience of the professor. This has substantially worked in the past.

[0004] In the academic field professors can also rely on the honor system to help enforce that the work of a student is original and reflects their work. However, the advent of generative pretrained transformers (GPTs), and similar artificial intelligence, have produced large language models that are able to be prompted on a certain topic (or multiple), wherein GPTs can substantially respond to prompts based on querying metadata or data that has been processed.However, with widespread availability of GPT technology, students may be more inclined to forgo more traditional, learning based research / writing process methodologies in exchange for allowing GPTs to generate assignment work product for students. This inclination can lead to an abuse of GPT technology and circumventing learning and / or more traditional educational approaches.

[0005] As GPT technologies advance it becomes more tempting to use by students on the one hand, and more difficult to spot as non-student work as fakes get better and better on the other. The system learns. For now, the users of GPT for all of their work, have anecdotally been identified by the similarity between two GPT generated papers including the same inaccuracies. Soon it will be extremely difficult to spot, beyond the skill set of the professor, enabling students to get the credit for GPT generated work without the effort.

[0006] Accordingly, there is a need for a system which enables educators and learners to be able to collaborate in a setting that advances learning and understanding while, at the same time, monitoring learners to ensure they are keeping up with pre-set standards and not violating academic or other ethical codes.SUMMARY OF THE INVENTION

[0007] A system for ensuring the integrity of student work and providing automatic feedback includes a professor module. The professor module is enabled to create a lesson formed from at least one scaffold containing information used to provide feedback. Each scaffold is a series of structured assignment steps. The system includes a student module in communication with the professor module. The student module is enabled to prepare the assignments. An intelligent system module communicates with the professor module and the student module. The intelligent system module is enabled to monitor activity of the student at the student module performed in preparingthe assignment defined by the scaffold. The intelligent system module, in response thereto, determines whether a student activity aligns with a student written response forming the assignment.

[0008] In one embodiment of the invention, the intelligent system module is further enabled to learn from communication between the professor module and student module to simulate the professor when providing automatic feedback.

[0009] In yet another embodiment of the invention, a rubric is created, as a function of the scaffold, as a grading structure for the assignments. The rubric determines which items in the student created documents are applicable to a student grade.

[0010] In a further embodiment of the invention the student module may be simultaneously used by more than one student.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure will be better understood by reading the written description with reference to the accompanying drawing Figures in which like reference numerals denote similar structure and refer to like elements throughout in which:

[0012] FIG. l is a schematic view of a system for implementing the invention;

[0013] FIG. 2 is an operation diagram of the system in accordance with the invention;

[0014] FIG. 3 is a flow diagram of the course creation process in accordance with the invention;

[0015] FIG. 4 is a flow diagram of the assignment creation process in accordance with the invention;

[0016] FIG. 5 is a flow diagram of the scaffold creation process in accordance with the invention;

[0017] FIG. 6 i s a flow diagram of the assignment feedback process in accordance with the invention;

[0018] FIG. 7 is a flow diagram of the user research and note taking process in accordance with the invention;

[0019] FIG. 8 is a flow diagram of the document writing process in accordance with the invention;

[0020] FIG 9 is a flow diagram of the assignment collaboration process in accordance with the invention;

[0021] Fig. 10 is a flow diagram of the show your work process in accordance with the invention; and

[0022] Fig. 11 is a flow diagram of the grading process in accordance with the invention.DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Reference is first made to FIGs. 1 and 2 in which a system enabling a student to work with a large language model (LLM) while ensuring the integrity of the work constructed in accordance with the invention, is provided. System 100 includes a student module 102 enabling a student to communicate, and cooperate with, a large language model 106, such as GPT in a non-limiting exemplary embodiment. System 100 includes a professor module 104 enabling a professor to communicate, and cooperate with, student module 102 and the large language model 106. As will be discussed in detail below, the modules cooperate with each other to enable the student to complete assignments from the professor module 104, making use of the GPT large language model 106, while complying with academic standards for the assignments.

[0024] Reference is now made more particularly to Fig. 2 in which system 100 is explained in greater detail. The set up and operation of the system begins with professor module 104 and the creation and use of a scaffold-by-scaffold creator submodule 112.

[0025] Professor module 104 may create a new course in a step 108 or utilize a previously created scaffold in a step 122. A scaffold is a series of structured assignment steps to guide the student through the steps of an assignment. If a professor decides to create a new course in step 108, professor module 104 creates an assignment(s) in a step 110 by creating the appropriate scaffold. Within step 112, professor module 104 creates a new scaffold assignment (requirement, lesson) for the course in a step 110.

[0026] The assignments are used as inputs to create the scaffold in step 112. The created assignment is formed as a series of steps, forming the scaffold. The assignment is input to scaffold creator 112. A new scaffold step is created as a function of the assignment in a step 1 14. As the assignments and lessons of the scaffold are presented to the student, as will be discussed in greater detail, a presentation style may be selected in a step 116. Questions, developed by the professor, or with aid of the LLM, may be added to the scaffold in a step 118. A description of the scaffold step is added in a step 120.

[0027] The process is then repeated as needed or the scaffold is then customized for the particular needs of an assignment in a step 124. It should be noted that even where a scaffold is selected from a library of scaffolds in a step 122, the scaffold may undergo customization in step 124 if desired by the professor.

[0028] Once the scaffold is completed, a rubric is created for that particular scaffold in a step 126. A rubric is a scaffold specific set of rules; a grading structure for the assignments making up the scaffold. It determines what items in the student created documents are applicable to a student grade as a function of the criteria making up the rubric. By way of non-limiting example,for a paper, exemplary nonlimiting criteria may be how well the argument was made, grammar, spelling, organization or the like. Once the scaffolding and rubrics are complete the assignment is made available for access in a step 128.

[0029] As further seen in Fig. 2 large language module 106 includes a large language module / artificial intelligence sub module 162, a show your work submodule 180, and an instructor interaction submodule 170. Artificial intelligence (LLM) sub module 162 receives the rubric created in step 126 and the scaffold steps from a scaffold prompts 130. LLM sub module 162 includes an engine for receiving scaffold prompts 130, rubrics developed by professor module 104, descriptions and content to train and then tune the LLM in a step 166. Additionally, LLM Submodule 162 receives instructor feedback from instructor interaction module 170 and fine tunes the LLM in a step 164 by an iterative loop and provides in an input to the engine in 166.

[0030] In show your work submodule 180, student input document changes are recorded in a step 184. This submodule may leverage a combination of artificial intelligence methods to implement the features discussed below. In a step 186, LLM module 106 monitors student research as conducted through student module 102 and aligns the research with the student written responses to assignments. In a step 188, LLM module 106 monitors student work in response to assignments as compared suggestions provided by the system.

[0031] In a step 190, LLM module 106 monitors student inputs to detect undeclared, disallowed use of generative Al to create text. This may make use of a separate module designed for detecting these types of text, and may include not just detection of static text, but detection of anomalous behavior such as copy / paste, or irregular typing.

[0032] The recorded document changes of step 182, alignment observations relative to student work product obtained in steps 184, 186, and 188 are input to instructor interaction module 170. The LLM detections obtained in step 190 are also input to instructor interaction module 170. Instructor interaction module 170 continuously monitors a state of the document in a step 172, and the determination is issued in part to provide feedback in a step 178. In a step 176, professor interaction module 104 views student actions as a function of time by monitoring the outputs from show your work submodule 180, particularly the outputs in steps 184, 186, 188, 190. These are used to determine appropriate feedback to the student directly or as part of the feedback to the LLM sub module 162 to fine tune LLM sub module 162 in step 164.

[0033] Student module 102 operates on inputs from professor module 104 as interactions with LLM module 106. In a step 130, a student utilizing student module 102 enters an assignment developed in step 128 by professor module 104. In a step 132, student module 102 enters a scaffold step corresponding to the assignment. A student may utilize a research module 134 within student module 102 to perform an assignment. The operation of research module 134 is within system 100 and can be monitored by LLM module 106. To this end, the operation of research module 134 is recorded.

[0034] The student may search for sources in a step 136. Working with LLM sub module 162, the sources may be summarized by LLM module 106 in a step 160, and input to research module 134 where the student reads the summary in a step 138. The source is then added to a student workspace within system 100 and, if being used in the assignment, cites from the source are stored and the process repeats itself at step 136.

[0035] If research is considered completed, then the assignment is written and revised in a step 144. Research and writing are iterative processes which may be repeated throughout an assignment or evenwithin one part of an assignment, until that portion is considered completed. As part of the writing process, a student may request feedback from the LLM module 162 as to structure, content, and compliance with the rubric. LLM module 162 will provide feedback to the student. Additionally, in a step 148, the student may request feedback from the professor which may be given in accordance with step 178 based upon the real time observations occurring in LLM module 106. As part of a collaborative effort, the student may text third parties such as other students in a step 150, video chat in a step 152, or receive other types of assistance with respect to grammar, spelling and tone in a step 154. All of steps 144 -154 are recorded by system 100 to ensure the integrity of the assignment process.

[0036] Once a scaffold step of the assignment is completed it is submitted in a step 156 to a grading module 191, the step is finished in a step 158, and if the entire assignment is not completed the process returns to a new scaffold step in step 132.

[0037] An interactive grading module 191, for use by the instructor, may review each assignment and provide relative progress re[ports. In a step 192, grading module 191 views a student document (assignment) in progress. In a step 194 grading module views the scaffolding step, associated rubric and any prompts received from LLM module 106 or the professor. In a step 196 grading module reviews the work as recorded over the timeline to determine whether any information, formatting or other characteristics of the paper do not conform to what was recorded; this being an indicator of use of materials outside the shown work. In a step 198, as function of the review it is determined how the completed assignment corresponds to the associated rubric, a grade is given and input to the student module 102 and the student grade may be sent to a learning management system (“LMS”) , in a step 199.

[0038] Reference is now made to Fig. 3 in which the process for course creation performed by professor module 104 of system 100 is provided in greater detail. In a step 302, an instructor chooses to create a new course. The instructor may, in a step 304, choose to import an existing course from a learningmanagement system or may create a new course. If the professor decides to utilize a preexisting course in step 304, then they may make use of an import wizard for importing an existing course in a step 306. The professor signs into the LMS of their choice in a step 308, and selects a course to import to system 100 in a step 310. In a step 311, the LMS imports the course including the course descriptions, rubrics, assignments and other sources. The course information is stored in a step 330, and is accessible including, as it may change, in a feedback loop with the help of LMS 326. In a step 326, student participation is automatically stored in a student database 332.

[0039] If the professor decides to create their own course in step 304, then, in a step 314, the instructor names the course. In a step 316, the professor creates a course description . This information is then stored, preferably locally, in course database 330. If the course is being created manually, then the professor must also add the students to the course in step 318 to be stored in student database 332.

[0040] A professor created course is then exported to LMS 326 in a step 320. This act starts an export wizard in a step 322, arranging a handshake with a selected LMS in a step 324 to export all relevant information such as course descriptions, rubrics, assignments and other resources to LMS 326.

[0041] Reference is now made to Fig. 4 in which an operational diagram of the assignment creation process performed within system 100 by professor module 104 is provided. A professor chooses to create a new assignment in a step 402. In a step 404, the professor determines whether they wish to select a preexisting scaffold in a step 406, or not to select a preexisting scaffold and utilize a scaffold creation wizard to create a custom scaffold in a step 406.

[0042] If the professor chooses not to select a scaffold, they may make use of a scaffold creation wizard in step 406, which initiates the scaffold creation process 408, described in detail below in connection with Fig. 5, and proceeds with the newly created basic formatted scaffold in a step 410. In a step 412, thescaffold is edited with details such as creating the assignment name in a step 412, setting a due date in step 420, and creating a description in a step 418 and assignment type in a step 424.

[0043] The due date maybe set for each step in the scaffolding process in a step 422. The assignment type selected in step 424, as will be described in connection with Fig. 6, and affects the automatic feedback LLM 162 offered students in a step 426. In a step 428, the professor can enable or disable certain types of feedback.

[0044] If, in step 404, the professor selects a scaffold, then in a step 406, the professor selects one of a preexisting set of scaffolds. The professor still maintains control over the process as the selected scaffold is edited in step 417, and the process flows from there. Whether the scaffold is selected by the professor, or is created with the help of an LMS, all of the information is combined in a step 460 and the process is moved onto editing of the scaffold steps in a step 430.

[0045] For each step in the scaffold, the professor may edit the scaffold step name in a step 432; edit the scaffold step description in a step 434, edit the step due date in a step 436, and even add step requirements in a step 438. In a step 440, a rubric is created as a function of the information. The professor decides in a step 442 whether the rubric will be imported from existing rubrics or created from scratch.

[0046] If a rubric is to be imported, then a rubric import wizard may be used in a step 444 to connect the professor with an LMS in a step 446, to enable selection of a rubric to import in a step 448. The selected rubric may then be edited, as needed, along with the course description and scoring algorithms in a step 480. If the professor has created a rubric in step 440, then editing occurs in step 480 upon completion.

[0047] The process is repeated until the professor has completed editing all of the scaffold steps in a step 450. The finished assignment is then published to the system in a step 452, which saves the details in a step 454, and makes the assignment accessible to students in a step 456, by database access, or bybroadcast to the students. It should be noted that the assignment can be saved at any time during the process in a step 482, in database 484, for continued creation at a later date.

[0048] Reference is now made to Fig. 5 where the scaffold creation process, within system 100, in accordance with the invention is provided. In a step 502, a professor begins the process to create a new scaffold. The process includes naming the scaffold in a step 504, and selecting a scaffold type in a step 506. In a step 508, LLM module 162 provides feedback to students based on the scaffold type. In a step 510, prewritten feedback questions, used in the scaffold for the student, are selected. These questions are provided at least in part by LLM module 162, as a function of the scaffold type. LLM module 162 incorporates the selected feedback questions in a step 512.

[0049] In a step 514, per assignment reflective questions are written. These reflective questions are plain English language questions incorporated, in a step 516 as part of the prompts sent to LLM module 162 to provide automatic feedback to help guide the student through the writing process. These questions are also provided at least in part by LLM module 162, as a function of the scaffold type.

[0050] In a step 518, the individual scaffold steps are created. The process includes the step of naming the scaffold step in a step 520. It also includes selecting the type of organization of the scaffold step in a step 522, which affects the way the student facing document is presented in a step 524. By way of nonlimiting example, the document may be in the form of bullet points, distinct sections for brainstorming ideas, or full paper by way of non-limiting example.

[0051] In a step 526, per step reflective questions are written. Reflective questions are written to automatically provide feedback in a step 528, for each respective scaffold step. Reflective questions may also be toggled in a step 530 to populate each step with the same information in a step 532.

[0052] A professor creates a rubric in step 534 and in step 536, the professor may choose to import a rubric or not. If a rubric is to be imported, system 100 utilizes a rubric wizard in a step 538 to connect toan LMS in a step 540, and then selects a rubric to import in a step 542. Whether a rubric is imported, or created by system 100, the rubric may be edited further in a step 541.

[0053] In a step 544, it is determined whether the process needs to be returned to step 18 to create each scaffold step or is complete. If complete, then the scaffold is published to system 100, and is saved in a step 548. It should be noted that this scaffold can be saved in a step 550, at any intermediate step or time, as a scaffold draft in a step 552.

[0054] Reference is now made to Fig. 6, the manner in which the LLM module 126 provides automatic feedback in accordance with the invention. In a step 602, a student requests feedback on their assignment from either LLM Module 162 or the professor. In a step 604, it is determined whether the request was for the professor or the LLM module 162. If the professor provides feedback in a step 606, the feedback is stored along with the question in a step 610. The feedback is formatted, in a step 612, for a fast LLM improvement method, such as retrieval augmented generation by way of nondimiting example. The stored data leads, in step 614, to the creation of an instructor specific LLM model which provides feedback in a tone and manner closer to the professor using this system. This model is used to augment, in a step 616, future responses, providing more accurate and appropriate feedback to students.

[0055] If the student requests LLM based feedback in step 604, LLM module 162 receives and combines information from a number of sources in a step 624. The LLM module collects and combines the instructor model augmented responses created in step 616, scaffold rubric and description 618, reflective questions 620, and the student’s paper 622, which reflects the content in question. The information is synthesized into a prompt in a step 626. A response is generated by LLM module 162 and parsed into a feedback format in a step 628. The LLM module 162 returns the feedback in a step 630, and the feedback from at least one of the professors and the LLM module 162 is displayed to the student in a step 608.

[0056] Reference is now made to Fig. 7, in which the process for research and note taking in accordance with invention is provided. A student opens a research tool in student module 102 in a step 702. In a step 704, the student, within student module 102, searches, in a step 704, for sources on the topic corresponding to the assignment. System 100, in response to a student query, searches for this topic across all integrated academic sources in a step 706. LLM 162 automatically summarizes the sources in a step 708. In a step 710, the student reads the summaries and selects the pertinent, as determined by the student, sources to add to a source library in a step 712 to be stored for future use in a step 714.

[0057] In a step 724, a student chooses to browse saved sources and cites the relied upon sources in a step 726. System 100 automatically inserts the cite to the document in the correct format for the assignment in a step 728.

[0058] Alternatively, the student may open the notes in a step 716 and begin taking notes based on the source in a step 718. In a step 720, the student copies the content from the notes into the document created for the assignment. As discussed above, this portion of the process is tracked and recorded in a step 730. System 100 tracks the flow of information from the source to the notes to the document in a step 722. This recording process enables the ability to perform a show your work process in step 732.

[0059] Reference is now made to Fig. 8 wherein the student document writing procedure in accordance with the invention is shown. A student may enter the document writing process in one of two ways. They may create a new document unrelated to any assignment in a step 802 or create a document in response to an assignment in a step 812. If a student creates a new document unrelated to any assignment, they pick a paper format in a step 804; free form or using a predetermined format. If the predefined format is selected then the student can edit specific document settings such as margins, in a step 808, if needed. If free form is selected, then the user sets every document setting in a step 806.

[0060] Whether the student creates a document using free form edits, using a predetermined format, or in response to an assignment, then a paper is being created in a step 810. Once a paper is created, it is a work in progress, and the student then enters the scaffold in a step 812; as described in Fig. 5. The student creates content as a function of the scaffold requirements in a step 814.

[0061] Creation occurs by first writing a draft document, or revising an existing document if returning, in a step 836. Additionally, as part of the writing process, the student may request feedback in a step 816 utilizing the feedback system protocols (Fig. 6) from LLM 162 in a step 824. This feedback may be used to edit the document in step 836. Similarly, in a step 818, the student may cite a source from previously saved sources in a step 826 from the research system as described in connection with Fig. 7, in a step 830. Again, these sources may be used to edit the document in step 836. The student may incorporate their notes into the document at step 836, in a step 820, by accessing the saved notes in a step 822 from the note system discussed above in a step 832. Again, these notes may be used to edit the document in step 836. The student may receive automatically generated editing suggestions created in a step 834, as a function of monitoring the paper in a step 822. These suggestions may be with respect to the tone of the document, style, completeness or the like.

[0062] The user completes the scaffold step in a step 838, and the completed work is submitted for review in a step 840. The step may be reviewed by the grading module in a step 846 and feedback is provided to the student. The scaffold step is then competed in a step 842. If it is determined that this is not the final scaffold step in a step 844, the student enters the process at step 812 for the next scaffold step and the process is repeated. If it is the final step the assignment is considered finished in a step 848, and feedback is provided by the grading module 191.

[0063] It should be noted that as with other processes in the document writing process, all activity is stored in an ongoing step 852 by the show your work module 180 in a step 850, further enhancing theintegrity of the process. Additionally, a draft of the work can be affirmatively saved at any time in steps 854 and 856. In this way all writing activity, including edits, citations, notes, feedback and quick suggestions are part of the “Show Your Work” process embodied by module 191.

[0064] Reference is now made to Fig. 9, wherein the process for using the system to enable collaboration on a document amongst students is provided. This operation of the system 100 enables students to invite others to collaborate, all within the confines of system 100. To begin the process a student will invite other students within system 100 to collaborate in a step 902, or a professor may assign collaboration teams in a step 904.

[0065] Once collaborators are identified, they join a document as users in a step 906, and create content for the document in a step 912. Activities are assigned by the group to each team member or by the professor in a step 908. The activity and progress of each collaborator is tracked, in a step 910, by student content creation modulel02 within the application, by the students and the professor.

[0066] As with operation for a sole student, each of the collaborators may take notes, either individually or collaboratively, in a step 924. These notes are recorded in a synced note document in a step 926, then stored in a step 928. This collaboration may occur in a number of ways. Collaborators may text chat with other collaborators in a step 914, or video chat with other collaborators in a step 916, to be audio transcribed in a step 918. The collaborative cooperation (discussions) is stored in a step 920 as part of the show your work module 180 in a step 922.

[0067] Reference is now made to Fig. 10, where in the operation of the show your work module 180 is shown . This system and methodology enable the tracking of the creation of a document by a student or group of students from start to finish while also allowing the professor to confirm that a paper was constructed in a proper way that conforms to academic integrity requirements. One way in which this isaccomplished is that the student part of the process is recorded. This includes feedback, research and notes, “quick suggestions,” and interaction with and by collaborators.

[0068] The student activity is recorded in a step 1002 as discussed above. The student has written content in a step 1004 as described above. In step 1016, student written comment is then temporarily aligned with the student action inputs such as receiving feedback 1006, performing research 1008, taking notes, 1010, use of quick suggestions 1012, and work with collaborators 1014. These inputs are semantically aligned (compared with the document at that state), using natural language processing, with student actions in a step 1018. A timeline is created by show your work module 180 in a step 1020, which i s links between edits, actions, actions and how they progress over time. LLM detection models are run, in a step 1022, of the generated text for the document, for suspicious activity. This may be pasting text without citation or use of automatic text generation within the document. Areas of determined suspicious activity are highlighted on the timeline in a step 1024. A check your work score may automatically be generated in a step 1025 as a function of show your work module 180 estimation of the originality and quality of the work in creating the document.

[0069] The professor may also interact at the show your work operation to review for suspicious activity in a step 1026 making use of the timeline, as well as to monitor and understand the document from start to finish. The professor can select any link in the timeline in a step 1028. The system shows the professor a document state at that period of time in which the relevant sections is highlighted in a step 1030. The professor may also manually highlight a section for specific feedback or manual flagging in a step 1032. In summary, the suspicious activity flags are a subset of the timeline; instructors may review the entire timeline or focus on suspicious activity. Reviewing the timeline may enable the instructor to flag specific parts of the timeline for feedback.

[0070] In a step 1034, the professor decides what to do with suspicion flags. They may remove them as non-suspicious in a step 1038 which increases the grading score in a step 1038. The professor may highlight the suspicious activity in a step 1036 for further review during grading.

[0071] Reference is now made to Fig. 11, where in the operation of grading module 191 is provided. A professor selects an assignment in a step 1102, and views the document in a step 1104. The check your work score is automatically calculated in a step 1112 utilizing the show your work module 180, and displayed to the professor as part of the step 1104 display. In a step 1106, suspicious flags within the document can be toggled to be shown as highlights and can be hovered over to show an explanation in a step 1108. At any time in the grading process the current grade may be saved in steps 1136, 1138.

[0072] The professor then grades entering scores and feedback for each rubric in a step 1116. As part of the process, the professor may review the show your work timeline in a step 1118. From this the final score is calculated in a step 1120. The professor then bundles their overall feedback in a step 1122 for the assignment. The instructor may export the feedback to an LMS in a step 1124. In a step 1128, an export wizard facilitates the instructor utilizing an LMS of choice in a step 1130, where the grade, final document, and feedback are exported to the LMS, where it is stored and operated upon in a step 1134. Otherwise, the professor releases the grade and feedback in a step 1126.

[0073] By providing the professor module, the professor module may allow for the creation, customization, and planning of a learning module. The learning module may be provided to at least one student module (or a student through the student module, thereby allowing the student module to receive the learning module). The learning module can be customized to include various education components such as assignments, quizzes, reading materials, and interactive activities in the form of steps in the scaffold. The professor module also can enable professors to set specific goals, deadlines, and assignment criteria for each learning module (including a rubric input or asa rubric input), to attempt to ensure that educational content is tailored to the needs of their students. Furthermore, the professor module may include tools for real -monitoring and feedback (which can be, but does not have to be part of the oversight module), allowing professors to track student progress by viewing how the student used the student module over time and throughout a learning module, identify areas where students may need additional support, and provide timely and constructive feedback by transmitting messages to the student module (or otherwise leaving messages, which can be placed on specific parts of assignments or projects within a learning module).

[0074] Additionally, the professor module can connect to or otherwise be in electronic communication with at least one external LMS. The at least one external LMS might be software / services such as, but not limited to, Blackboard, Canvas, Moodle, and / or other software / services that facilitate online learning. By being so connected to, or otherwise be in electronic communication with, at least one external learning management system, the professor module can communicate data to the at least one external learning management system, such as, but not limited to, grades, assignments, learning modules, student progress reports, attendance records, and other educational materials. The professor module may also receive data from the external learning management system, such as student enrollment information, course prerequisites, and other records, which can be used to tailor the learning modules (and / or assignments therein) to the specific needs and backgrounds of students.

[0075] As previously mentioned, in the professor module, the professor module can comprise a rubric input. A scaffold input may describe a pre-defined set of goals / steps to complete a learning module (or portions thereof), such as, a. brainstorm, b. research, c. confirm results of research, d. write a first draft, e. re-research, e. write a second draft, f. check in with professor, g. write a thirddraft, and so on (otherwise, progress points). A rubric input may be applied to each scaffold portion and define the rules by which the professor grades an assignment within the scaffold. As such, the scaffold enables for the learning module to become a series of steps that a student can systematically complete and carry out so as to progress through and ultimately complete a learning module. The scaffold input can alternatively allow for a professor, through the use of the professor module to input descriptions of a series of steps or methods (or select from some steps, or input some steps and select some steps) that the professor would like the student to carry out in the form of the scaffold. As such, the scaffold can be customizable to allow a professor to set up the process flow of an assignment (such as including some or more of the aforementioned steps above). That said, the scaffold, as described by the rubric input, can comprise a list of quantifiable tasks that can be measured, tracked, and / or categorized by the aforementioned oversight model so as to allow the system to provide feedback, utilizing the rubric input, to consider the tasks performed within the scaffold.

[0076] The student module can be a platform to allow students to complete learning tasks as scaffold steps, Additionally, the student module can allow for querying of a professor, via instant messaging, video or voice calling, or other communication means should a student desire to reach out to an professor, perhaps with a question or to seek clarification. As such, the student module can allow the student to systematically / sequentially complete scaffold steps (through the guidance of a rubric input). A student may systematically complete scaffold steps. Also, the student module can include a collaborative workspace, enabling a student to work together on group assignments or projects (wherein multiple students are progressing through a learning module as a team), fostering teamwork and peer-to-peer learning. As such, the system can be equipped with tools for document sharing and joint editing. Further, the student module can be in electroniccommunication with the oversight provided by the system, allowing the system, by recording key processes, to oversee a student’s (or multiple students’, if working in a group) progress throughout a learning module. Moreover, the student module can access additional resources, submit assignments, and receive grades and feedback, creating a comprehensive and interconnected learning experience.

[0077] The collaborative writing capabilities of the student module may leverage an oversight capability of the system to carry out these functional modules / functions and provide feedback to the student based on student content. The system monitors and evaluates student interactions within the student module and the applications as the student progresses through a learning module. To this end, the system monitors any and all interactions a student has with a student module. The system can also operate by analyzing student inputs in relation to a rubric input (i.e. a series of systematic, quantifiable steps that can be understood by rules and standards making up the rubric). As such, the model can utilize algorithmic processes to compare and contrast the student’s activities against the rubric input criteria. As such, this comparison can go beyond a “check-box” style criteria, but can involve a nuanced analysis of how closely the student’s input aligns with the expected progression and outcomes as defined in the scaffold input (such as, but not limited to research methods, time spent on certain tasks, drafting and editing skills and time spent thereon, time spent on revisions to particular sections, citation and academic integrity characteristics, and / or use of features within the student module). As oversight is provided, it can provide a professor with updates as to how a student is progressing (or it may, for example, notify a professor as to predictions of a language model analyzer for example).

[0078] Further as to oversight of student work in progress, use may be made of a language model analyzer. The analyzer can be an algorithmic tool able to scrutinize various aspects of a student’sinput, including but not limited to tone, writing style, grammar, spelling, and input speed. The purpose of this multifaceted analysis is twofold. Primarily, it can serve as a quality control mechanism, ensuring that the student’s submissions meet the expected rubric input standards (as part of the oversight model). Additionally, it can function as a detector of anomalies in the student’s input pattern. By tracking and analyzing changes in writing style, time, sudden shifts in input speed, and other measurable input factors, the model can identify patterns that may indicate the use of unauthorized assistance or plagiarism (such as GPT models).

[0079] In addition to the above, the oversight model can interact with external learning management systems to transfer data such as grades, assignments, and student progress reports. Further, the oversight model can feed data it accumulates to a structure model, which has the capability to refine the oversight model or make changes to what pre-defined selections a rubric input model may incorporate.

[0080] The language model analyzer scrutinizes the students’ inputs, focusing on aspects such as tone, writing style, grammar, spelling, and input speed over time. It can detect significant deviations in these parameters over time, which could indicate potential integrity issues of a departure from the leaner’s typical submission pattern. For instance, if a student’s draft shows a sudden improvement in language quality or a change in writing style, the analyzer flags this for the professor’s review.

[0081] Throughout the assignment, the professor can monitor student progress in real-time through the professor module. This includes viewing draft submissions, providing feedback, and assessing the quality of research. The professor can offer targeted feedback at each stage, guiding students through the research and writing process. This continuous monitoring and feedback loopensures that students are supported throughout their assignment and can make necessary improvements before the final submission.

[0082] In a second example, a professor utilizes the collaborative writing module abilities to set up a group assignment focusing on a research project. The professor configures the rubric input to include stages such as topic selection, collaborative research, joint drafting, peer review, and final submission. Each stage has specific objectives and deliverables, such as collaborative topic selection, division of research areas, joint drafting responsibilities, peer review guidelines, and final presentation criteria.

[0083] Students, working in groups, access the module through their respective student modules. They may utilize a text editor application for drafting their sections of the project. The module's collaborative workspace is used for communication, document sharing, and tracking the contributions of each group member. This workspace facilitates seamless collaboration, allowing group members to edit, comment, and review each other's work in real-time. Students also make use of the research components.

[0084] The ability of the system to monitor in real time plays a role in this setup. It monitors each group's progress, analyzing inputs for consistency and collaboration. The model checks for equitable participation by analyzing the frequency and quality of each member's contributions. This ensures that all group members are actively participating and contributing to the project. The professor can view the collaborative efforts and individual contributions through the professor module, ensuring that the group dynamics are balanced and productive.

[0085] In a scenario where a group member might use a GPT tool to generate a section of the project, the various modules detect this by identifying anomalies in writing style or sudden shifts in input quality / speed. For example, if a section of the project significantly differs in style orsophistication from previous submissions by the same student, the analyzer flags this inconsistency. The analyzer can also detect which of the students each input (or flagged input) arose from. The professor is then alerted to review this section more closely, potentially uncovering the use of unauthorized Al tools.

[0086] Throughout the project, the professor can provide feedback and guidance, mediated through the professor module. The final submission is reviewed against the criteria set in the rubric input, with the professor providing a collective grade and individual feedback based on each member's contribution, informed by the oversight model's analysis. The professor can also use the oversight model's data to identify areas where the group excelled or struggled, providing insights for future instructional strategies.

[0087] In a third example, a professor focuses on individual writing skills through a series of structured assignments. The learning module created in the professor module is tailored to develop and assess various aspects of academic writing, with a detailed rubric input emphasizing writing quality at stages like brainstorming, outline creation, multiple draft revisions, and final submission.

[0088] Students access the module via their student modules, starting with brainstorming and progressing to drafting. Here, the oversight model, particularly the language model analyzer, becomes crucial. It evaluates the student's inputs for writing quality, scrutinizing grammar, spelling, and adherence to the chosen writing style. Additionally, it monitors for sudden changes in writing style or input speed.

[0089] In a situation where a student might attempt to use a GPT tool for drafting a section of their assignment, a language model analyzer used by the show your work feature when performing show your work detection, is designed to detect this. For instance, if a draft submission shows a marked improvement in language complexity or a shift in writing style that does not align with the student'susual submissions, the analyzer flags these discrepancies. This detection prompts the professor to review the submission more closely, potentially identifying the use of GPT technology. This feature ensures that the student's work is original and aligns with their developmental trajectory in writing skills.

[0090] Throughout the assignment, the professor tracks each student's progress and development through the professor module, which provides detailed reports from the oversight functionality. The professor can offer personalized feedback, focusing on areas of improvement. The final submission is evaluated against the comprehensive criteria set in the rubric, with the professor providing a grade and detailed feedback, relying on the insights provided by the oversight model and its language analyzer.

[0091] The professor can also use the data from the oversight model to understand each student's writing patterns, strengths, and areas needing improvement. This data-driven approach allows for more targeted instruction and support, helping students to develop their writing skills more effectively. The oversight model's detailed analysis of each student's work provides a comprehensive view of their progress, ensuring that the professor can guide them effectively towards achieving the learning objectives.

[0092] It will thus be seen that the objects set forth above, among those made apparent from the preceding description, are efficiently attained and, since certain changes may be made in carrying out the above method and in the construction set forth without departing from the spirit and scope of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

[0093] It is also to be understood that the following claims are intended to cover all of the generic and specific features of the invention herein described, and all statements of the scope of the invention which, as a matter of language, might be said to fall there between.

Claims

CLAIMS1. A system for ensuring the integrity of student work and providing automatic feedback comprises: a professor module; the professor module configured to create a lesson formed from at least one scaffold; the scaffold configured to contain information used to provide feedback to a student; each scaffold being at least one assignment formed as a series of structured assignment steps; a student module in communication with the professor module; the student module configured to prepare the at least one assignment; and an intelligent system module communicating with the professor module and the student module; the intelligent system module configured to monitor activity of the student at the student module performed in preparing the assignment defined by the scaffold; the intelligent system module, in response thereto, determining whether a student activity aligns with a student written response forming the assignment.

2. The system of claim 1, wherein the professor module comprises a scaffold by scaffold submodule configured to create a scaffold as the series of structured assignment steps.

3. The system of claim 2, wherein the scaffold by scaffold submodule is further configured to create the scaffold as a function of a presentation style.

4. The system of claim 2, wherein the scaffold by scaffold submodule is configured as a function of at least one question developed by one of a professor and the intelligent system module.

5. The system of claim 1, wherein the professor module is configured to create a rubric as a set of rules, the professor module further being configured to create the at least one assignment as a function of the set of rules.

6. The system of claim 1, wherein the intelligent system module comprises a large language module, the large language module being configured to receive the rubric and the scaffold steps, and in response to scaffold prompts, the large language module trains the large language module as a function thereof .

7. The system of claim 6, wherein the intelligent system module further comprises an instructor interaction module configured to allow the professor to provide an instructor feedback to the large language module, the large language module utilizing an interactive loop to process the instructor feedback and the large language module trains the large language module as a function thereof.

8. The system of claim 1, wherein the intelligent system module comprises a show your work module configured to receive a student input in response to an assignment and record document changes, the intelligent system module comprises a large language module configured to monitor the student input and align research performed by the student with a student generated response to the assignment.

9. The system of claim 1, wherein the intelligent system module comprises a show your work module configured to receive a student input in response to an assignment and record document changes, the intelligent system module comprises a large language module configured to provide a suggestion in response to assignments, the large language model configured to compare the suggestion to a student generated response to the assignment.

10. The system of claim 8, wherein the large language module is configured to determine whether a student input is impermissible text as being one of undeclared text or text created by generative artificial intelligence.11 . The system of claim 10, wherein the large language module determines whether the text is impermissible text as a function of a copy and pasted text or irregular typing.

12. The system of claim 1, wherein the intelligent system module comprises a show your work module configured to receive a student input in response to an assignment and record document changes, the intelligent system module comprises a large language module configured to monitor the student input and align research performed by the student with a student generated response to the assignment; and large language module being further configured to provide a suggestion in response to assignments, the large language model configured to compare the suggestion to a student generated response to the assignment, and the large language module being further configured to determine whether a student input is impermissible text; and the professor module receiving an output of the show your work module and a determination whether the input is impermissible text from the large language module; the professor module providing a feedback to the student as a function of the output of the show your work module and the determination.

13. The system of claim 1, wherein the intelligent system module comprises a show your work module configured to receive a student input in response to an assignment and record document changes, the intelligent system module comprises a large language module configured to monitor the student input and align research performed by the student with a student generated response to the assignment; and the large language module being further configured to provide a suggestion in response to assignments, the large language model configured to compare the suggestion to a student generated response to the assignment, and the large language module being further configured to determine whether a student input is impermissible text; and the large language model receiving an output of the show your work module and adetermination whether the input is impermissible text from the large language module; the large language model providing a feedback to the student as a function of the output of the show your work module and the determination; the professor module communicating with the show your work module, the show your work module having an output as a function of monitoring a student action, and the intelligent system module providing feedback to the student as a function of at least one of the suggestion in response to assignments, the determination whether a student input is impermissible text, and the output.

14. The system of claim 1, wherein the intelligent system module comprises a show your work module configured to receive a student input in response to an assignment and record document changes, the intelligent system module comprises a large language module configured to monitor the student input and align research performed by the student with a student generated response to the assignment; and the large language module being further configured to provide a suggestion in response to assignments, the large language model configured to compare the suggestion to a student generated response to the assignment, and the large language module being further configured to determine whether a student input is impermissible text; and the large language model receiving an output of the show your work module and a determination whether the input is impermissible text from the large language module; the large language model providing a feedback to the student as a function of the output of the show your work module and the determination; the professor module communicating with the show your work module, the show your work module having an output as a function of monitoring a student action, and the intelligent system module providing feedback to the large language model as a function of at least one of the suggestion in response to assignments, the determination whether a student input is impermissible text, and the output.

15. The system of claim 1 , wherein the student module receives at least one input from the professor module, and interacts with a scaffold as a function thereof; the student module comprising a research module , the research module performing the at least one assignment by searching for sources.

16. The system of claim 15, wherein the large language model summarizes the sources.

17. The system of claim 15, wherein the large language model provides feedback to a student’s performance of the at least one assignment.

Images