Online lecture support system

Abstract

An online lecture support system based on integrated reference documents for each online classroom is disclosed. The online lecture support system includes: a reference document generation module that converts audio from lecture videos into text, extracts text from lecture material files including PDF / PPT materials, and collects lecture-related announcements in text format; a prompt engine that provides a method for efficiently searching by splitting documents into small chunks and converting each chunk into a vector when finding answers to specific questions in large documents; and an AI response module that generates a natural language response to said user question using a long-term contextual language model as input of a user question and a dynamic prompt generated by said prompt engine, wherein the prompt engine includes: a document chunk splitter that receives the entire text of a reference document, divides it into tokens of a certain size, and outputs a chunk list; a vector embedding converter that converts document chunks into vector embeddings; an indexing builder that receives a vector list of vectors corresponding to each chunk and builds an index using an IVF (Inverted File) algorithm with a similarity search library; and a dynamic prompt assembler that selects the top 3 chunks most similar to said question when given a user question.

Description

Technology Field

[0001] The present invention relates to an online lecture support system, and more specifically, to an online lecture support system based on an integrated reference document for each online classroom. Background Technology

[0002] Currently, online education systems provide educational content through various Learning Management Systems (LMS) and online course platforms. While these systems offer learners the flexibility to freely choose their study times, they have limitations in providing customized learning paths tailored to individual learning styles or needs.

[0003] Furthermore, the absence of a real-time Q&A function, which is essential for improving the quality of online lecture content and learner comprehension, is one of the major factors limiting the effectiveness of online learning.

[0004] Furthermore, while online lectures provide lecture videos and materials, the absence of real-time Q&A features may make it difficult for learners to understand the lecture content. Consequently, it is difficult to receive immediate feedback or answers to questions, which may limit the effectiveness of the learning process.

[0005] In addition, in existing systems, lecture materials, announcements, and reference materials are often provided separately, which makes it difficult for learners to find the materials they need and hinders consistent management of materials.

[0006] Furthermore, assignment feedback is mostly provided manually, and due to the lack of automated feedback systems tailored to learners' progress or learning styles, it takes time to receive feedback. Consequently, if feedback is delayed, learners find it difficult to immediately identify areas for improvement. Prior art literature

[0007] Korean Registered Patent No. 10-2698934 (Published Aug. 23, 2024) (Title of Invention: Online CSAT Lecture and Management System Using AI Education Automation Program) The problem to be solved

[0008] Accordingly, the technical problem of the present invention is based on this point, and the objective of the present invention is to provide an online lecture support system based on an integrated reference document for each online classroom, which combines lecture video text conversion using STT (Speech-to-Text) technology, generation of integrated reference documents for lecture materials and announcements, prompt optimization using a long-term contextual language model, and assignment feedback automation. means of solving the problem

[0009] To realize the purpose of the present invention as described above, an online lecture support system according to one embodiment comprises: a reference document generation module that converts the audio of a lecture video into text, extracts text from a lecture material file containing PDF / PPT materials, and collects lecture-related announcements in text format; a prompt engine that provides a method for efficiently searching by splitting the document into small pieces and converting each chunk into a vector when finding an answer to a specific question in a large document; and an AI response module that generates a natural language response to the user question using a long-term contextual language model as input of a user question and a dynamic prompt generated by the prompt engine, wherein the prompt engine includes: a document chunk splitter that receives the entire text of the reference document, divides it into tokens of a certain size, and outputs a chunk list; a vector embedding converter that converts the document chunks into vector embeddings; an indexing builder that receives a vector list of vectors corresponding to each chunk and builds an index using an IVF (Inverted File) algorithm with a similarity search library; and a dynamic prompt assembler that selects the top 3 chunks most similar to the question when a user question is given. Effects of the invention

[0010] According to this online lecture support system, students can easily access necessary information through the integrated management of lecture materials and announcements, and can be supported in better understanding and reviewing learning content by converting lecture videos into text. Furthermore, learning efficiency can be enhanced by using long-term contextual language models to provide students with more accurate and appropriate responses and feedback. In addition, dynamic processing tailored to classroom characteristics allows for the provision of learning support optimized for the needs of each classroom, thereby improving the personalized learning experience. As a result, students' learning efficiency increases, and the overall quality of the education system improves. Brief explanation of the drawing

[0011] FIG. 1 is a block diagram illustrating an online lecture support system according to one embodiment of the present invention. Figure 2 is a block diagram illustrating the AI ​​learning support unit for each classroom shown in Figure 1. Figure 3 is a block diagram illustrating the reference document generation module shown in Figure 2. Figure 4 is a block diagram illustrating the prompt engine shown in Figure 2. FIG. 5 is a block diagram for explaining the task feedback unit illustrated in FIG. 1. FIGS. 6a and 6b are screens for illustrating a user screen for exemplarily creating a draft of feedback by the task feedback unit shown in FIG. 5. FIG. 7 is a block diagram illustrating a computing environment including a computer device suitable for use in embodiments of the present invention. Specific details for implementing the invention

[0012] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0013] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected" but also cases where they are "electrically connected" with other elements interposed between them. Furthermore, when a part is described as "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but rather allows for the inclusion of additional components; it should be understood that this does not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0014] Terms such as “about,” “substantially,” etc., used throughout the specification, are used to mean at or near the stated value when inherent manufacturing and material tolerances are presented in the stated meaning, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute values ​​are mentioned to aid in understanding the invention. Terms such as “step” or “step of” used throughout the specification of the invention do not mean “step for”.

[0015] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0016] Some of the operations or functions described herein as being performed by a terminal, device, or device may instead be performed by a server connected to said terminal, device, or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal, device, or device connected to said server.

[0017] In this specification, some of the operations or functions described as mapping or matching with a terminal may be interpreted as meaning mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data.

[0018] The present invention will be described in detail below with reference to the attached drawings.

[0019] FIG. 1 is a block diagram illustrating an online lecture support system according to one embodiment of the present invention.

[0020] Referring to FIG. 1, an online lecture support system according to one embodiment of the present invention includes an AI learning support unit (100) and an assignment feedback unit (200) for each classroom, manages lecture content provided by a teacher through a teacher terminal (20), and provides lecture content through a student terminal (30). In this embodiment, the online lecture support system is described as being composed of an AI learning support unit (100) and an assignment feedback unit (200) for each classroom, but this is a logical distinction made for convenience of explanation and is not a hardware distinction.

[0021] The AI ​​learning support unit (100) for each classroom utilizes AI technology to support students' learning by integrating various materials to manage data and using the data to provide quick and accurate answers to user questions. For example, the AI ​​learning support unit (100) for each classroom collects and consistently manages materials (textbooks, lecture notes, assignments, videos, etc.) and learning data for each lecture, structures all materials so that students can easily access them, and enables them to quickly find the necessary materials. In addition, the AI ​​learning support unit (100) for each classroom analyzes students' questions about the lecture or questions that arise during learning and provides quick and accurate answers to them.

[0022] The assignment feedback unit (200) is configured to generate detailed feedback on students' assignments, helping students check and improve what they have learned through the assignments. For example, the assignment feedback unit (200) can thoroughly analyze the assignments submitted by students to determine whether the assignment was performed well, and to identify any misunderstandings or mistakes in certain areas. In other words, not only the content of the assignment but also formal elements (e.g., grammar, logical structure, etc.) can be reviewed.

[0023] FIG. 2 is a block diagram for explaining the AI ​​learning support unit (100) for each classroom shown in FIG. 1.

[0024] Referring to FIGS. 1 and 2, the AI ​​learning support unit (100) for each classroom includes a reference document generation module (120), a prompt engine (130), and an AI response module (140).

[0025] The reference document generation module (120) is configured to manage data by integrating various lecture-related materials into one, including a data processing unit (122), a NoSQL DB (124), and an integrated structuring unit (126).

[0026] The audio of the lecture video is converted into text using speech-to-text (STT) technology, lecture material files such as PDF / PPT are read and text is extracted, and lecture-related announcements are collected in text format. The reference document generation module (120) integrates the input lecture video, lecture material files, and announcements to generate a single structured document. Here, materials of various formats are integrated and structured in an easy-to-understand manner. The integrated document is stored in a NoSQL DB (124) for each classroom. The NoSQL DB (124) contains all reference documents corresponding to each classroom, and a subsequent search and response system operates based on this.

[0027] FIG. 3 is a block diagram for explaining the reference document generation module (120) illustrated in FIG. 2.

[0028] Referring to FIGS. 2 and 3, the reference document generation module (120) includes an STT converter (122a) that extracts text from a lecture video, a PDF / PPT processor (122b) that processes tables and images from materials and converts them into a structured document, a notice processor (122c) that analyzes notices to keyword tag important information and generates metadata in JSON format, and an integrated structuring unit (126) that combines them into a single structured reference document to provide to a user, thereby efficiently processing and structuring various types of content such as lecture materials, PDF / PPT documents, and notices to finally generate a reference document.

[0029] The STT converter (122a) extracts speech from a lecture video using speech recognition (STT) technology and converts it into text. Specifically, the STT converter (122a) extracts speech from the lecture video and converts the speech into text through a speech recognition model. To improve the quality of the text conversion, the STT converter (122a) removes background noise or unnecessary sounds from the lecture video to create a pure text file. The pure text file includes a timestamp to record the creation time. This facilitates subsequent analysis or searching.

[0030] The PDF / PPT processor (122b) receives lecture materials as a PDF file or a PPT file. The PDF file or PPT file may contain not only text but also tables and images. Therefore, the PDF / PPT processor (122b) extracts the tables and images included in the document. If an image is included, the PDF / PPT processor (122b) generates Alt-Text explaining it to provide a text description of the image. For example, if the image describes a "neural network structure," the Alt-Text is converted into a form such as "Figure 1: Neural Network Structure." The extracted information is converted into a structured Markdown document and saved. This document is a well-structured document containing text, tables, image descriptions, etc.

[0031] The notice handler (122c) generally receives notices such as schedules, assignments, and class changes, extracts important keywords, and tags the corresponding keywords. For example, for the keyword "assignment deadline," it can be tagged with the date as in #AssignmentDeadline: 2023-12-31. The tagged information is structured and output in JSON format. JSON is a format that efficiently stores data and can be easily linked with other systems.

[0032] The integrated structuring unit (126) structures the data stored in the NoSQL DB (124), namely the text generated by the STT converter (122a), the Markdown document converted by the PDF / PPT processor (122b), and the JSON metadata generated by the notice processor (122c), into JSON and converts them into a consistent format to finally generate a reference document. The reference document is a comprehensive document containing all related information, such as lecture content, assignment information, and notices, and is provided in a structured form so that the user can access it quickly later.

[0033] Referring again to FIG. 2, the prompt engine (130) is configured as a long-term context prompt engine to respond quickly to user questions by utilizing the data mentioned above, but when finding answers to specific questions in a large number of documents, it provides a method of efficiently searching by breaking the documents into small chunks and converting each chunk into a vector. This allows the most relevant documents to be found quickly in response to the question posed by the user.

[0034] In this embodiment, the long context prompt engine is a system designed to be employed in online education systems, customer support systems, chatbots, and conversational AI, enabling AI language models to efficiently process the context of long conversations or text data and provide consistent answers. Here, long context refers to the flow of multiple sentences or conversations, and becomes important primarily when dealing with various topics or complex conversational contexts. While general language models generate answers based only on current input, the long context engine has the advantage of continuously remembering and utilizing past conversations or contexts to provide more appropriate and accurate responses.

[0035] FIG. 4 is a block diagram for explaining the prompt engine (130) illustrated in FIG. 2.

[0036] Referring to FIGS. 2 and FIGS. 4, the prompt engine (130) includes a document chunk splitter (132), a vector embedding converter (134), an indexing builder (136), and a dynamic prompt assembler (138).

[0037] The document chunk splitter (132) receives the entire text of the reference document, divides it into tokens of a certain size, and outputs a chunk list. Specifically, the document is divided into units of 4K tokens (approximately 4,000 words). At this time, an overlap interval of, for example, 10% is provided so that adjacent chunks partially overlap. This overlap is intended to maintain the context of the document and to prevent important information from being scattered across multiple chunks. The chunk list is output in the form, for example, [Chunk1, Chunk2, ...]. Each chunk is a text fragment and is used as an independent unit during search or vectorization processing.

[0038] The vector embedding converter (134) converts document chunks into vector embeddings. The vector embedding converter (134) may include Sentence-BERT. Sentence-BERT is a model that converts sentences into high-dimensional vectors and can express the meaning of text in numerical form. Each chunk is converted into a 768-dimensional vector. This vector uniquely represents the meaning of the chunk and is subsequently used for similarity search, etc.

[0039] The indexing builder (136) receives a list of vectors corresponding to each chunk and constructs an index using an IVF (Inverted File) algorithm with a similarity search library. In this embodiment, the similarity search library may include the FAISS (Facebook AI Similarity Search) library. FAISS is a tool for optimizing vector search and is useful for efficiently finding similar vectors in a vector space. IVF improves search speed by grouping vectors. An optimized index file is generated by the indexing builder (136), and this can be used to perform fast and efficient search.

[0040] When a user question is given, the dynamic prompt assembler (138) selects the top 3 chunks that are most similar to the question. Here, similarity is calculated through vector search, using the cosine similarity between the vector of each chunk and the user question vector. The dynamic prompt assembler (138) selects the top 3 chunks and provides a result combined to fit the prompt template. This prompt is used as an input to find the optimal answer to the question.

[0041] Referring again to FIG. 2, the AI ​​response module (140) is configured to provide an accurate answer to a question, and generates a natural language response to the user question using a long-term contextual language model as input, a user question and a dynamic prompt generated by the prompt engine (130). The long-term contextual language model understands the context of the question well and provides an appropriate answer by taking into account information given in the past. The AI ​​response module (140) generates a customized response for each classroom, which provides a specific answer tailored to the materials of each classroom, so that different answers may be given depending on which lecture the user is listening to.

[0042] FIG. 5 is a block diagram for explaining the task feedback unit (200) illustrated in FIG. 1.

[0043] Referring to FIGS. 1 and 5, the assignment feedback unit (200) includes a student assignment input unit (210), a similarity analysis unit (220), and a feedback draft generation unit (230). In this embodiment, the assignment feedback unit (200) is described as being composed of a student assignment input unit (210), a similarity analysis unit (220), and a feedback draft generation unit (230), but this is a logical distinction made for convenience of explanation and is not a hardware distinction.

[0044] The student assignment input section (210) receives assignment files or text provided through the student terminal (30). The student assignment input section (210) may provide a separate user interface or user screen to provide easier input through the student terminal (30).

[0045] The similarity analysis unit (220) performs a similarity analysis between the submitted assignment and the educator's example answer as the assignment file or text submitted by the student is provided through the student assignment input unit (210). This process evaluates how closely the student's assignment content matches the example answer provided by the educator, thereby checking the student's understanding or accuracy.

[0046] The feedback draft generation unit (230) generates a draft of feedback for the task, and this feedback generates specific and customized feedback to be provided to the student after comparing the example answer provided by the educator through the teacher terminal (20) with the student's answer through the student terminal (30).

[0047] FIGS. 6a and 6b are screens for illustrating a user screen for exemplarily creating a draft of feedback by the task feedback unit shown in FIG. 5.

[0048] Referring to Fig. 6a, when an assignment titled "Read the book Marketing Architect and write your own marketing strategy" is given in the classroom, the student writes the assignment.

[0049] A window for entering an educator's comment as an educator example feedback input corresponding to the written assignment is displayed as shown in (1). Through the displayed window, the educator can enter a comment and then click the "Use as AI Example" button or the "Register" button. When the "Use as AI Example" button (indicated by (2)) is clicked, the comment corresponding to the student's assignment written by the educator is used as the assignment and feedback example data. When the "Register" button is clicked, the comment corresponding to the student's assignment written by the educator is registered as an evaluation comment corresponding to the assignment.

[0050] Referring to Fig. 6b, prompt assembly is performed as the "AI mode" button (indicated by (3)) is selected to ON.

[0051] When the “AI generate” button (indicated by (4)) is clicked, a draft of feedback is created based on the prompt and displayed in the window for the educator’s comment input (indicated by (5)). In other words, consistent feedback is generated by learning the educator’s example feedback style and patterns. Additionally, the AI ​​generates a new draft of feedback based on the assembled prompt.

[0052] In the window for entering the educator's comment marked (5), a draft of feedback is created, and if necessary, the draft of feedback is modified and registered. That is, the generated AI feedback draft is reviewed, modified if necessary, and the final modified feedback is delivered.

[0053] FIG. 7 is a block diagram illustrating a computing environment including a computer device suitable for use in embodiments of the present invention.

[0054] Referring to FIG. 7, the computing environment includes a computer device (300). In one embodiment, the computer device (300) may be one or more components included in a graph data processing device. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those not described below.

[0055] The computer device (300) includes at least one processor (310), a computer-readable storage medium (320), and a communication bus (330). The processor (310) can enable the computer device (300) to operate according to the exemplary embodiment described above. For example, the processor (310) can execute one or more programs stored in the computer-readable storage medium (320). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to enable the computer device (300) to perform operations according to the exemplary embodiment when executed by the processor (310).

[0056] A computer-readable storage medium (320) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (322) stored in the computer-readable storage medium (320) includes a set of instructions executable by a processor (310). In one embodiment, the computer-readable storage medium (320) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that are accessed by a computer device (300) and capable of storing desired information, or a suitable combination thereof.

[0057] The communication bus (330) interconnects various other components of the computer device (300), including the processor (310) and the computer-readable storage medium (320).

[0058] The computer device (300) may also include one or more input / output interfaces (340) and one or more network communication interfaces (360) that provide interfaces for one or more input / output devices (350). The input / output interfaces (340) and the network communication interfaces (360) are connected to a communication bus (330). The input / output devices (350) may be connected to other components of the computer device (300) through the input / output interfaces (340). Exemplary input / output devices (350) may include input devices such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or output devices such as a printer, a speaker and / or a network card. An exemplary input / output device (350) may be included inside the computer device (300) as a component constituting the computer device (300), or it may be connected to the computer device as a separate device distinct from the computer device (300).

[0059] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0060] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0061] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0062] As described above, according to the present invention, students can easily access necessary information through the integrated management of lecture materials and announcement documents, and can be supported in more easily understanding and reviewing learning content through the text conversion of lecture videos. In addition, learning efficiency can be enhanced by using a long-term contextual language model to provide students with more accurate and appropriate responses and feedback. Furthermore, through dynamic processing tailored to classroom characteristics, learning support optimized for the needs of each classroom can be provided, thereby enhancing the personalized learning experience. As a result, students' learning efficiency increases, and the overall quality of the education system improves.

[0063] Although the invention has been described above with reference to embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Industrial applicability

[0064] This invention can be utilized in various educational environments, such as universities, private academies, corporate training, and online education platforms. It streamlines the management of large-scale educational content as a customized learning support system for each classroom and can also be provided in the form of SaaS (Software as a Service). Explanation of the symbols

[0065] 20 : Teacher's terminal 30 : Student Terminal 100 : AI Learning Support Department by Classroom 120 : Reference document generation module 122 : Data processing unit 124 : NoSQL DB 126 : Integrated Structured Section 122a : STT Converter 122b : PDF / PPT Processor 122c : Notice Handler 130 : Prompt engine 132 : Document Chunk Splitter 134 : Vector Embedding Converter 136 : Indexing Setup Guide 138 : Dynamic Prompt Assembler 140 : AI Response Module 200 : Assignment Feedback Department 210 : Student Assignment Input Section 220 : Similarity Analysis Section 230 : Feedback Draft Generation Section 300 : Computer device 310: Processor 320: Computer-readable storage media 330: Communication bus 340: Input / Output Interface 350: Input / Output Device 360: Network communication interface

Claims

Claim 1 An online lecture support system characterized by comprising: a reference document generation module that converts audio from a lecture video into text, extracts text from lecture material files including PDF / PPT materials, and collects lecture-related announcements in text format; a prompt engine that provides a method for efficiently searching by splitting the document into small chunks and converting each chunk into a vector when finding answers to specific questions in a large document; and an AI response module that generates a natural language response to the user question using a long-term contextual language model as input of a user question and a dynamic prompt generated by the prompt engine, wherein the prompt engine includes: a document chunk splitter that receives the entire text of a reference document, divides it into tokens of a certain size, and outputs a chunk list; a vector embedding converter that converts the document chunks into vector embeddings; an indexing builder that receives a vector list of vectors corresponding to each chunk and builds an index using an IVF (Inverted File) algorithm with a similarity search library; and a dynamic prompt assembler that selects the top 3 chunks most similar to the question when a user question is given.

Images