Artificial intelligence teaching system integrated with multi-mode interactive interface
By integrating multimodal interactive interfaces, the AI teaching system solves the problem of the single interactive form of traditional teaching systems, realizes all-round perception of the teaching process and generation of personalized teaching resources, and improves classroom participation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-25
- Publication Date
- 2026-03-27
Smart Images
Figure CN121747397A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of educational information technology, and in particular to an artificial intelligence teaching system integrating a multimodal interactive interface. Background Technology
[0002] In the wave of digital transformation in education, traditional teaching systems are mainly divided into two subcategories: text-based interaction technology and voice-based interaction technology. Text-based interaction technology facilitates information exchange between teachers and students, and among students themselves, through text input, pop-up Q&A, and forum comments, and is widely used in scenarios such as assignment submission, online Q&A, and course comments. Voice-based interaction technology, relying on basic speech recognition and synthesis algorithms, enables functions such as voice command control and voice Q&A, and is commonly found in modules such as oral practice and voice check-in. However, both suffer from a single-modal interaction method, failing to match diverse teaching scenarios and differentiated learning habits. Furthermore, this single-modal interaction lacks an immersive experience, making it difficult to stimulate students' learning interest, resulting in generally low classroom participation.
[0003] Therefore, how to provide an artificial intelligence teaching system that integrates multimodal interactive interfaces to overcome the difficulties of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0004] In view of this, the present invention provides an artificial intelligence teaching system that integrates multimodal interaction interfaces. By combining multimodal interaction technology, artificial intelligence algorithms and cloud services, it realizes the precise generation of personalized teaching resources and provides intelligent support for students' learning throughout the entire process.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: An artificial intelligence teaching system integrating a multimodal interactive interface includes: a front-end interactive interface module, which is connected to a data processing module through a standardized interface for acquiring user multimodal data; The data processing module communicates and connects with the cloud module to obtain user needs, receive user multimodal data and data captured by the cloud module, and integrate user data. The cloud module includes an open-source project retrieval unit and a knowledge graph construction unit. The open-source project retrieval unit retrieves data corresponding to the user's needs based on the user's requirements, while the knowledge graph construction unit parses the retrieved data and constructs a teaching graph. The terminal module connects to the cloud module and is used to generate teaching materials and practice questions based on the teaching graph.
[0006] Optionally, the front-end interactive interface module includes a micro-teaching recording unit and a data acquisition unit. The microteaching recording unit is used to record the teacher's lecture video and audio data, as well as the students' interaction and feedback during the microteaching process, and generate teaching records; The data acquisition unit is used to collect multimodal data from the education information system, including teaching syllabus, student test scores, and teacher feedback data.
[0007] Optionally, the data processing module includes a data preprocessing unit, a data fusion unit, and a teaching analysis unit; The data preprocessing unit uses robust statistical methods to handle outliers and employs different strategies to handle missing values based on different data types to obtain processed data. The data fusion unit uses a tensor decomposition algorithm to decompose the processed data into tensor form and generate a unified data representation. The instructional analytics unit is used to generate assessment data on student engagement and knowledge acquisition based on data representation, student test scores, and teacher feedback.
[0008] Optionally, the data fusion unit includes: The preprocessed data is converted into tensor form. The preprocessed data includes video data, audio data, and text data. The video data is represented as a three-dimensional tensor, including time, space, and color channels; the audio data is represented as a two-dimensional tensor, including time and frequency; and the text data is represented as a two-dimensional tensor, including word vectors and time series. The high-order singular value decomposition method is used to decompose the original high-dimensional tensor into multiple low-dimensional tensors and a core tensor. Features are extracted from the decomposed low-dimensional tensors through feature selection and clustering algorithms to identify the association patterns between video, audio and text data. The extracted features are integrated into a unified data representation.
[0009] Optionally, the open-source project retrieval unit includes a requirement acquisition module, a multi-platform crawler module, and a teaching suitability ranking module; The demand acquisition module includes acquiring the student's current teaching syllabus and set training objectives, and forming knowledge point keywords; The multi-platform crawler module is used to search public platforms based on knowledge point keywords and obtain data retrieval results; The teaching suitability ranking module is used to obtain the relevance between data retrieval results and knowledge point keywords, and select the high-ranking data as the crawling data.
[0010] Optionally, knowledge graph construction units include: Determine the main body of the teaching syllabus and construct the teaching knowledge points and their corresponding attributes; Based on teaching knowledge points and their corresponding attributes, raw data is extracted from the data representation using a preset knowledge extraction strategy, and then organized and generated into a teaching knowledge graph according to a preset knowledge graph construction strategy.
[0011] Optionally, the graph construction strategy includes a knowledge point attribute mapping strategy: Based on the laws of subject teaching, teaching syllabus and training objectives, a strategy for mapping the attributes of knowledge points is obtained by considering the directionality, reciprocity and transmissibility of knowledge points.
[0012] As can be seen from the above technical solution, compared with the prior art, the present invention provides an artificial intelligence teaching system integrating multimodal interactive interfaces, which has the following beneficial effects: 1) The present invention comprehensively collects multimodal teaching data such as video, audio, and text, breaks through the limitation of single data collection in traditional teaching systems, realizes all-round perception and accurate capture of the teaching process, and provides rich data support for subsequent teaching analysis; 2) The present invention breaks the geographical and platform limitations of traditional teaching resources, provides rich teaching materials for the system, realizes the structured organization and related presentation of knowledge, and meets the learning needs of different students. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0014] Figure 1 This is a block diagram of an artificial intelligence teaching system integrating a multimodal interactive interface disclosed in this invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] Reference Figure 1 As shown, this invention discloses an artificial intelligence teaching system integrating a multimodal interaction interface, including: a front-end interactive interface module, which is connected to a data processing module through a standardized interface for acquiring user multimodal data; The data processing module communicates and connects with the cloud module to obtain user needs, receive user multimodal data and data captured by the cloud module, and integrate user data. The cloud module includes an open-source project retrieval unit and a knowledge graph construction unit. The open-source project retrieval unit retrieves data corresponding to the user's needs based on the user's requirements, while the knowledge graph construction unit parses the retrieved data and constructs a teaching graph. The terminal module connects to the cloud module and is used to generate teaching materials and practice questions based on the teaching graph.
[0017] Furthermore, the front-end interactive interface module includes a micro-teaching recording unit and a data acquisition unit. The microteaching recording unit is used to record the teacher's lecture video and audio data, as well as the students' interaction and feedback during the microteaching process, and generate teaching records; The data acquisition unit is used to collect multimodal data from the education information system, including teaching syllabus, student test scores, and teacher feedback data.
[0018] Specifically, the microteaching recording unit employs a multi-camera synchronous recording solution, which can simultaneously capture multi-dimensional video data from the teacher's lecturing perspective, student interaction perspective, and blackboard or PPT presentation perspective. During the recording process, the system automatically timestamps and synchronizes the video and audio data to ensure audio-visual synchronization.
[0019] The core data collection content of the acquisition unit includes: The syllabus data includes key information such as course objectives, distribution of knowledge points, teaching schedule, and assessment requirements. The data collection unit extracts standardized syllabi for each course from the academic affairs management system, categorizing and organizing them according to dimensions such as subject, grade level, and course type to form a structured syllabus database. Student exam performance data includes scores from various assessments such as quizzes, midterms and finals, and lab evaluations. The data collection unit associates this performance data with student basic information and course information to generate structured data containing fields such as student ID, course ID, assessment type, score, and assessment time. Teacher feedback data includes textual data such as teachers' evaluations of students' classroom performance, homework correction feedback, and periodic learning summaries. The data collection unit uses natural language processing technology to initially process this data, extracting key evaluation information and feedback points to form semi-structured feedback data.
[0020] Furthermore, the data processing module includes a data preprocessing unit, a data fusion unit, and a teaching analysis unit; The data preprocessing unit uses robust statistical methods to handle outliers and employs different strategies to handle missing values based on different data types to obtain processed data. The data fusion unit uses a tensor decomposition algorithm to decompose the processed data into tensor form and generate a unified data representation. The instructional analytics unit is used to generate assessment data on student engagement and knowledge acquisition based on data representation, student test scores, and teacher feedback.
[0021] Furthermore, the data preprocessing unit employs classic outlier detection methods such as box plots and Z-scores for structured data: by calculating the quartiles of the data, a threshold range for outliers is determined, and data exceeding the threshold range is identified as outliers; for the Z-score method, if the absolute value of the Z-score is greater than the threshold, it is identified as an outlier; for unstructured data, a statistical feature-based outlier detection method is used: for video frame data, statistical features such as the mean and variance of pixel grayscale values within the frame are extracted, and abnormal noise frames are identified and removed by comparing them with the statistical features of normal frames; for audio data, burst noise signals are detected and eliminated by calculating the energy and spectral characteristics of the audio signal.
[0022] For random outliers in structured data, methods such as mean replacement and median replacement are used for correction; systematic outliers are marked and retained in the data for reference in subsequent teaching analysis units; outliers in unstructured data are directly removed to avoid affecting the subsequent data fusion effect.
[0023] Furthermore, the data fusion unit includes: The preprocessed data is converted into tensor form. The preprocessed data includes video data, audio data, and text data. The video data is represented as a three-dimensional tensor, including time, space, and color channels; the audio data is represented as a two-dimensional tensor, including time and frequency; and the text data is represented as a two-dimensional tensor, including word vectors and time series. The high-order singular value decomposition method is used to decompose the original high-dimensional tensor into multiple low-dimensional tensors and a core tensor. Features are extracted from the decomposed low-dimensional tensors through feature selection and clustering algorithms to identify the association patterns between video, audio and text data. The extracted features are integrated into a unified data representation.
[0024] Furthermore, the open-source project retrieval unit includes a requirement acquisition module, a multi-platform crawler module, and a teaching suitability ranking module; The demand acquisition module includes acquiring the student's current teaching syllabus and set training objectives, and forming knowledge point keywords; The multi-platform crawler module is used to search public platforms based on knowledge point keywords and obtain data retrieval results; The teaching suitability ranking module is used to obtain the relevance between data retrieval results and knowledge point keywords, and select the high-ranking data as the crawling data.
[0025] Specifically, the core function of the demand acquisition module is to clarify the user's teaching resource needs and generate precise knowledge point keywords. The module obtains the student's current syllabus and set learning objectives through the data processing module, and uses natural language processing technology to perform structured analysis of the syllabus, extracting key information such as the core knowledge points, the relationships between knowledge points, and teaching requirements. Simultaneously, combined with student knowledge mastery assessment data generated by the teaching analysis unit, it identifies the knowledge points that students need to focus on or strengthen. Based on this information, the demand acquisition module uses keyword extraction algorithms to generate a list of knowledge point keywords, with each keyword assigned a weight. These keyword extraction algorithms include TF-IDF and TextRank.
[0026] The multi-platform crawler module supports searching multiple mainstream open-source educational resource platforms, including domestic platforms such as China University MOOC, iCourse, and XuetangX, as well as international platforms such as Coursera and edX. It also supports searching academic paper platforms, open-source textbook platforms, and teaching video platforms.
[0027] The evaluation indicators for the teaching suitability ranking module mainly include: Knowledge point relevance, i.e., the degree of matching between search results and knowledge point keywords, is determined by calculating the cosine similarity between resource titles, abstracts, and keywords; Teaching objective suitability, i.e., whether the searched resources meet the teaching objectives and training requirements set by the teaching syllabus, is determined by comparing resource content with the teaching requirements of the syllabus; Resource quality, the reliability and quality of resources are evaluated through information such as click volume, ratings, user reviews, and author qualifications; Timeliness, prioritizing recently released or updated resources to ensure the timeliness of teaching content; Format suitability, prioritizing suitable resource formats based on students' learning habits and teaching scenario needs.
[0028] Furthermore, the knowledge graph construction units include: Determine the main body of the teaching syllabus and construct the teaching knowledge points and their corresponding attributes; Based on teaching knowledge points and their corresponding attributes, raw data is extracted from the data representation using a preset knowledge extraction strategy, and then organized and generated into a teaching knowledge graph according to a preset knowledge graph construction strategy.
[0029] Furthermore, the knowledge graph construction strategy includes a knowledge point attribute mapping strategy: Based on the laws of subject teaching, teaching syllabus and training objectives, a strategy for mapping the attributes of knowledge points is obtained by considering the directionality, reciprocity and transmissibility of knowledge points.
[0030] Furthermore, the instructional knowledge graph is stored and managed using a graph database. Nodes in the graph represent knowledge point entities, node attributes are various attribute information of the knowledge point, and edges represent the relationships between knowledge points. The instructional knowledge graph supports dynamic updates. When the teaching syllabus is adjusted, new open-source resources are added, or student learning data changes significantly, the knowledge graph construction units will automatically re-extract knowledge data and update the nodes and relationships of the knowledge graph.
[0031] Furthermore, the terminal module will accurately push the generated teaching materials and practice questions to the corresponding terminal devices according to user needs. The pushed content will be directly displayed on the student's learning interface, while also indicating the learning focus and suggested study time.
[0032] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An artificial intelligence teaching system integrating multi-modal interactive interface, characterized in that, Comprise: The front-end interactive interface module is connected with the data processing module through a standardized interface, and is used to obtain user multi-modal data; The data processing module is in communication connection with the cloud module, and is used to obtain user demand, receive user multi-modal data and the data captured by the cloud module, and fuse user data; The cloud module includes an open source project searching unit and a knowledge graph construction unit, which controls the open source project searching unit to search the data captured corresponding to the user demand according to the user demand, and the knowledge graph construction unit analyzes and constructs the teaching graph according to the captured data; The terminal module is connected with the cloud module, and is used to generate teaching materials and exercise questions according to the teaching graph.
2. The artificial intelligence teaching system integrated with the multi-modal interactive interface according to claim 1, wherein the front-end interactive interface module comprises a microteaching recording unit and a collection unit, The microteaching recording unit is used to record the teaching video and audio data of the teacher and the interaction and feedback of the students in the microteaching process, and generate teaching records; The collection unit is used to collect multi-modal data from the education information system, including teaching outline, student test scores and teacher feedback data.
3. The artificial intelligence teaching system integrated with the multi-modal interactive interface according to claim 2, wherein the data processing module comprises a data preprocessing unit, a data fusion unit and a teaching analysis unit; The data preprocessing unit processes abnormal values by using robust statistical methods, processes missing values by using different strategies according to different data types, and obtains processed data; The data fusion unit adopts a tensor decomposition algorithm to decompose the processed data into a tensor form to generate a unified data representation; The teaching analysis unit is used to generate evaluation data on student participation and knowledge mastery according to the data representation, student test scores and teacher feedback data.
4. The artificial intelligence teaching system integrated with the multi-modal interactive interface according to claim 3, wherein the data fusion unit comprises: The preprocessed data is converted into a tensor form, the preprocessed data includes video data, audio data and text data, the video data is represented as a three-dimensional tensor, including time, space and color channel; the audio data is represented as a two-dimensional tensor, including time and frequency, and the text data is represented as a two-dimensional tensor, including word vector and time sequence; A high-order singular value decomposition method is used to decompose the original high-dimensional tensor into a plurality of low-dimensional tensors and a core tensor, features are extracted from the decomposed low-dimensional tensors by feature screening and clustering algorithm, and the association mode between video, audio and text data is identified; The extracted features are integrated into a unified data representation.
5. The artificial intelligence teaching system integrated with the multi-modal interactive interface according to claim 1, wherein the open source project searching unit comprises a demand acquisition module, a multi-platform crawler module and a teaching adaptation degree sorting module; The demand acquisition module includes acquiring the current teaching outline of the students and the set training target to form the key words of knowledge points; The multi-platform crawler module is used to search the public platform according to the key words of knowledge points to obtain data search results; The teaching adaptation degree sorting module is configured to obtain the relevance of the data retrieval result and the knowledge point keyword, and select high-ranking data as the crawling data. 6.The artificial intelligence teaching system integrated with a multi-modal interactive interface according to claim 3, characterized in that, The knowledge graph construction unit comprises: determining a teaching outline subject, constructing a teaching knowledge point and a knowledge point corresponding attribute; extracting the original data from the data representation based on the teaching knowledge point and the knowledge point corresponding attribute through a preset knowledge extraction strategy, and generating a teaching knowledge graph according to a preset knowledge graph construction strategy. 7.The artificial intelligence teaching system integrated with a multi-modal interactive interface according to claim 6, characterized in that, The graph construction strategy comprises a knowledge point attribute mapping strategy: based on the discipline teaching law, the teaching outline and the training target, the knowledge point attribute mapping strategy is obtained in terms of the directionality, interactivity and transferability of the knowledge point.