An intelligent teaching interaction method and system fusing speech transcription and knowledge graph

Abstract

The application relates to the technical field of intelligent teaching, and particularly discloses an intelligent teaching interaction method and system fusing voice transcription and a knowledge graph, which collects audio data of teacher teaching, obtains first teaching knowledge points, analyzes and processes teaching materials, extracts second teaching knowledge points, generates a basic framework of a teaching knowledge graph, carries out interaction analysis according to the first knowledge points and the second knowledge points, respectively evaluates the importance of the knowledge points, determines key knowledge points, and optimizes the knowledge graph. The application realizes automatic structural analysis and key identification of teaching content, can effectively improve the organization and management efficiency of classroom content, can provide students with clearer and more accurate learning paths, and further improves teaching quality and learning effect.

Description

Technical Field

[0001] This invention relates to the field of intelligent teaching technology, and more specifically, to an intelligent teaching interaction method and system that integrates speech transcription and knowledge graph. Background Technology

[0002] With the continuous advancement of educational informatization and smart campus construction, classroom teaching processes are gradually achieving digital recording and platform-based management. Teachers use multimedia courseware to deliver lectures, and the classroom content is synchronously recorded through recording and broadcasting systems. After class, the video resources are uploaded to the teaching platform for students to review. Meanwhile, some platforms have introduced speech-to-text technology to automatically convert classroom recordings into text for retrieval and archiving. However, existing classroom speech-to-text systems typically only generate independent text, failing to achieve precise association between the transcribed text and courseware page numbers or video clips. When students consult the transcribed text, they cannot quickly locate the corresponding courseware page or video clip by clicking on the text; they still need to manually drag and drop among numerous videos, resulting in low retrieval efficiency.

[0003] Traditional knowledge graph construction methods are mostly applied to general knowledge domains, lacking adaptation mechanisms for classroom teaching scenarios. They fail to effectively integrate textbook content, courseware structure, and classroom explanation details, resulting in a disconnect between the knowledge graph and actual teaching content. Furthermore, existing systems typically lack dynamic evolution capabilities; once the course structure changes, manual reorganization and maintenance are required, lacking intelligent update mechanisms and failing to meet the needs of continuous teaching improvement and high-quality resource sharing. Simultaneously, there is a lack of automated association mechanisms between knowledge points and teaching resources. Classroom recordings, self-assessment questions, and other resources often exist independently, failing to be precisely linked to specific knowledge points, making it difficult for students to conduct targeted learning and consolidation around particular knowledge points.

[0004] Therefore, it is necessary to provide an intelligent teaching interaction method and system that integrates speech transcription and knowledge graph to solve the above-mentioned technical problems. In order to solve the above problems, a technical solution is provided. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of existing technologies, this invention provides an intelligent teaching interaction method and system that integrates speech transcription and knowledge graphs. This addresses the problem in existing classroom teaching where there is a lack of effective integration and connection between teacher explanations, textbook structures, and teaching resources, resulting in difficulties in automatically extracting teaching knowledge points, accurately identifying key points, and precisely organizing learning resources according to knowledge structures.

[0006] To achieve the above objectives, the present invention provides the following technical solution: An intelligent teaching interaction method integrating speech transcription and knowledge graph includes the following steps: Audio data of the teacher's lectures is collected through a microphone, preprocessed, and then converted into text lecture data; By associating and matching text-based teaching data with teaching materials, an intelligent navigation association database is constructed. Based on the text-based teaching data, the first teaching knowledge point is extracted, and the hierarchical and relational relationships between the first teaching knowledge points are constructed to generate a visual teaching knowledge mind map. By analyzing and processing the teaching materials, the second teaching knowledge points are extracted. Combined with the chapter structure of the teaching materials, the hierarchical relationship of the second teaching knowledge points is constructed, and the basic framework of the teaching knowledge graph is generated. Based on the first and second knowledge points, an interactive analysis is conducted to assess the importance of each knowledge point. Key knowledge points are determined according to their importance and marked in the basic framework of the teaching knowledge graph. The teaching resources are automatically attached to the corresponding knowledge points, and the teaching knowledge graph is optimized and improved according to teaching needs.

[0007] As a further aspect of the present invention, an intelligent navigation association database is constructed by associating and matching text-based teaching data with teaching courseware. The specific steps are as follows: By connecting with the teaching terminal of the smart classroom, playback data of the teaching materials is collected; A text matching algorithm is used to match text-based teaching data with the content of the teaching materials, thereby establishing a link between the text-based teaching data and the page numbers of the teaching materials. Align the timestamps of the written lecture data with the timeline of the classroom recording video to link the written lecture data with the video clips and build an intelligent navigation association database.

[0008] As a further aspect of the present invention, the first teaching knowledge points are extracted based on the text-based teaching data, the hierarchical and relational relationships between the first teaching knowledge points are constructed, and a visual teaching knowledge map is generated. The specific steps are as follows: An attention-based natural language processing algorithm is used to extract the first teaching knowledge point from the text teaching data; Based on the extracted first teaching knowledge points, the hierarchical and relational relationships between the first teaching knowledge points are constructed to generate a visual teaching knowledge mind map.

[0009] As a further aspect of the present invention, the hierarchical relationship of the first teaching knowledge points is determined: the course name is taken as the first-level knowledge point, the chapter name as the second-level knowledge point, and the extracted core concepts as the third-level knowledge points, thus constructing the hierarchical structure of the first teaching knowledge points; Determine the relationships between the knowledge points in the first lesson: by analyzing the semantic relationships between the knowledge points in the first lesson in the text teaching data.

[0010] As a further aspect of the present invention, by parsing and processing the teaching materials, extracting the second teaching knowledge points, and combining the chapter structure of the teaching materials, constructing the hierarchical relationship of the second teaching knowledge points, a basic framework for the teaching knowledge graph is generated. The specific steps are as follows: The teaching materials uploaded by teachers are analyzed. The teaching materials include teaching materials and teaching slides. OCR technology is used to convert the image content in the teaching materials and teaching slides into text content, and natural language processing algorithms are used to extract the second teaching knowledge points in the teaching materials and teaching slides. Based on the chapter structure of the teaching materials and courseware, the hierarchical relationship of the second teaching knowledge points is constructed, and the basic framework of the teaching knowledge graph is generated.

[0011] As a further aspect of the present invention, interactive analysis is performed based on the first knowledge point and the second knowledge point to assess the importance of each knowledge point. Key knowledge points are then identified based on their importance and marked within the basic framework of the teaching knowledge graph. The specific steps are as follows: Based on the audio data, extract the teacher's voice behavior features for the first knowledge point, and analyze the weight value of classroom explanation based on the teacher's voice behavior features for the first knowledge point. Based on the teaching materials, the teaching text features of the second knowledge point are obtained, and the structural weight value of the textbook is analyzed based on the teaching text features of the second knowledge point. By performing semantic alignment and matching analysis on the first and second knowledge points, a knowledge point interaction relationship network is constructed, and the first and second knowledge points that successfully match are initially selected as candidate key knowledge points. For candidate key knowledge points, a comprehensive importance score is formed by combining the weight values ​​of classroom explanation and textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The key knowledge points are ranked according to their overall importance score, and their order is marked in the basic framework of the teaching knowledge graph.

[0012] As a further aspect of the present invention, by performing semantic alignment and matching analysis on the first knowledge point and the second knowledge point, a knowledge point interaction relationship network is constructed, and the first and second knowledge points that successfully match are initially screened as candidate key knowledge points. The specific steps are as follows: The semantic similarity between the first and second knowledge points is calculated by performing semantic vectorization on the first and second knowledge points respectively and then calculating the semantic similarity between the first and second knowledge points in a unified semantic space. When the semantic similarity is higher than the preset similarity threshold, the first knowledge point and the second knowledge point are successfully matched and form an interactive knowledge point pair; otherwise, the first knowledge point and the second knowledge point fail to match. For the first knowledge point and the second knowledge point that fail to match, the matching is supplemented by synonym expansion and contextual semantic analysis to obtain a complete cross-source knowledge point mapping relationship. The first and second knowledge points that matched successfully were initially selected as candidate key knowledge points.

[0013] As a further aspect of this invention, for candidate key knowledge points, a comprehensive importance score is formed by combining the weight values ​​of classroom explanation and textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The specific steps are as follows: Obtain the first and second knowledge points in each pair of interactive knowledge points. Based on the teacher's voice behavior characteristics of the first knowledge point, normalize the data and then perform weighted calculations to obtain the classroom explanation weight value. Based on the characteristics of the teaching text for the second knowledge point, the weighted calculation is performed after normalization to obtain the weight value of classroom explanation. A comprehensive importance score for knowledge points is generated by weighting and integrating the weight values ​​of classroom explanation and textbook structure. The overall importance score of a knowledge point is compared with a preset importance score threshold. If the overall importance score of a knowledge point is greater than or equal to the preset importance score threshold, the corresponding interactive knowledge point pair is considered a key knowledge point; if the overall importance score of a knowledge point is less than the preset importance score threshold, the corresponding interactive knowledge point pair is considered a non-key knowledge point.

[0014] As a further aspect of this invention, teaching resources are automatically attached to corresponding knowledge points, and the teaching knowledge graph is optimized and improved according to teaching needs. The teaching resources include classroom recording clips and self-assessment questions. The specific steps are as follows: Based on the knowledge points in the basic framework of the teaching knowledge graph, the corresponding text teaching data is extracted. Based on the timestamp of the text teaching data, the time segment corresponding to the classroom recording video is determined. The time segment is cut into several classroom recording video segments, and knowledge point tags are added to each classroom recording video segment. Self-assessment questions are automatically generated based on knowledge points, and teachers can also upload custom questions.

[0015] An intelligent teaching interaction system integrating speech transcription and knowledge graph includes a lecture audio acquisition and transcription module, a voice intelligent navigation module, a lecture material parsing module, a multi-source knowledge point interactive recognition module, and a teaching resource mounting and optimization module; The lecture audio acquisition and transcription module is used to acquire audio data of teachers' lectures through a microphone, and convert it into text lecture data after preprocessing. The voice intelligent navigation module is used to build an intelligent navigation association database by associating and matching text teaching data with teaching courseware, extracting the first teaching knowledge points based on the text teaching data, constructing the hierarchical and association relationships between the first teaching knowledge points, and generating a visual teaching knowledge mind map; The teaching material parsing module is used to analyze and process the teaching materials, extract the second teaching knowledge points, and, in combination with the chapter structure of the teaching materials, construct the hierarchical relationship of the second teaching knowledge points to generate the basic framework of the teaching knowledge graph. The multi-source knowledge point interaction recognition module is used to perform interactive analysis based on the first and second knowledge points, evaluate the importance of each knowledge point, determine the key knowledge points based on their importance, and mark them in the basic framework of the teaching knowledge graph. The teaching resource mounting and optimization module is used to automatically mount teaching resources to corresponding knowledge points and optimize and improve the teaching knowledge graph according to teaching needs.

[0016] The technical effects and advantages of this invention, which integrates speech transcription and knowledge graph, are as follows: This invention integrates and analyzes multi-source teaching data such as classroom lecture audio, courseware content, and textbook materials to achieve automatic extraction, structured organization, and key point identification of classroom knowledge content, thereby constructing a dynamic and visualized teaching knowledge graph and accurately associating teaching resources with knowledge points.

[0017] This invention enables automated structuring and intelligent management of classroom teaching content. It not only unifies and integrates scattered classroom lectures, textbook content, and teaching resources, but also accurately identifies key knowledge points in the course based on data analysis, providing data support for teachers in instructional design and content optimization. Simultaneously, students can engage in targeted learning based on a knowledge graph structure, quickly accessing corresponding explanation videos and practice resources through knowledge point nodes, thus forming a clear learning path and improving learning efficiency. This invention also enables dynamic updating and continuous expansion of teaching resources, ensuring that teaching content is constantly improved as the course progresses, effectively enhancing the intelligence level of teaching management and the efficiency of teaching resource utilization in a smart classroom environment. Attached Figure Description

[0018] Figure 1 A flowchart illustrating an intelligent teaching interaction method integrating speech transcription and knowledge graph, provided as an embodiment of the present invention; Figure 2 This is a system block diagram of an intelligent teaching interaction system that integrates speech transcription and knowledge graph, provided as an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described technical solutions are only a part of this invention, and not all of it. All other technical solutions obtained by those skilled in the art based on the technical solutions of this invention without inventive effort are within the scope of protection of this invention.

[0020] Example 1 like Figure 1 The diagram shown is a flowchart of an intelligent teaching interaction method integrating speech transcription and knowledge graph provided by an embodiment of the present invention. Figure 1 The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. Steps S1 to S5 are detailed as follows: Step S1: Collect audio data of the teacher's lecture through a microphone, and convert it into text lecture data after preprocessing; Step S2: By associating and matching text teaching data with teaching courseware, an intelligent navigation association database is constructed. Based on the text teaching data, the first teaching knowledge point is extracted, the hierarchical relationship and association relationship between the first teaching knowledge points are constructed, and a visual teaching knowledge mind map is generated. Step S3: By parsing and processing the teaching materials, extract the second teaching knowledge points, combine the chapter structure of the teaching materials, construct the hierarchical relationship of the second teaching knowledge points, and generate the basic framework of the teaching knowledge graph. Step S4: Perform interactive analysis based on the first and second knowledge points, assess the importance of each knowledge point, determine the key knowledge points based on their importance, and mark them in the basic framework of the teaching knowledge graph. Step S5: Automatically attach teaching resources to the corresponding knowledge points and optimize and improve the teaching knowledge graph according to teaching needs.

[0021] Preferably, the audio data of the teacher's lecture is collected through a microphone, preprocessed, and then converted into text lecture data. The specific steps are as follows: Audio data of the teacher's lectures is collected via microphones in the smart classroom at a frequency of 44.1kHz to ensure audio clarity. The collected audio data undergoes preprocessing, including noise reduction, silence removal, and speech rate normalization. Noise reduction uses an adaptive filtering algorithm to remove environmental noise and student background noise. Silence removal uses an energy detection algorithm to remove silent segments from the audio. Speech rate normalization uses a time stretching algorithm to adjust the teacher's speaking speed to a standard speed for easier subsequent speech transcription. Speech-to-text transcription is performed using an AI model to obtain written teaching data. The AI ​​model is specifically trained for the characteristics of speech in teaching scenarios, enabling it to accurately recognize professional terminology and colloquial expressions in the teaching field. The open-source large model provides powerful natural language processing capabilities, improving the accuracy of transcription. The specific transcription process is as follows: the pre-processed audio data is input into the self-developed small model for initial transcription, resulting in preliminary transcribed text; the preliminary transcribed text is then input into the open-source large model for optimization and correction, correcting errors and ambiguities in the transcription; timestamps are added to the corrected transcribed text, with each sentence corresponding to a time segment of the classroom recording video.

[0022] In a practical teaching application embodiment of the present invention, when a teacher is giving a lecture in a smart classroom, an array of microphones deployed in the classroom collects audio data in real time during the teacher's explanation. The teacher's speech is continuously collected at a sampling frequency of 44.1kHz to ensure the clarity and integrity of the audio data. Since real classroom environments typically contain interference such as student discussions, page turning, and equipment noise, the collected raw audio data is first preprocessed. Specifically, an adaptive filtering algorithm is used to reduce noise in the audio signal, thereby filtering out environmental noise and student background noise. Then, an energy detection algorithm based on short-time energy thresholds is used to identify and cut out silent segments in the audio, such as silent segments generated during the teacher's blackboard writing or short pauses in class, to reduce invalid data. Furthermore, a time-stretching algorithm is used to normalize the speech rhythm, adjusting the teacher's potentially varying speech rates at different stages of the explanation to a uniform standard speed, thus providing stable input data for subsequent speech recognition.

[0023] After audio preprocessing, the processed audio data is input into the speech-to-text module for speech recognition, employing a "self-developed teaching speech recognition small model + open-source large model collaborative optimization" approach. First, the preprocessed audio data is input into the self-developed small model for preliminary speech recognition. This small model is trained specifically for classroom teaching speech characteristics. In this case, the teacher's lecture content is related to Newton's second law in high school physics. It can recognize subject-specific terms such as "acceleration," "net external force," and "constant mass," converting the teacher's explanation into preliminary transcribed text, such as "When the net external force on an object is not zero, the object will accelerate." Subsequently, this preliminary transcribed text is input into the open-source large model for semantic optimization and error correction. Natural language understanding capabilities are used to correct the recognition results, such as correcting potential homonym misrecognition and completing sentence structures based on context. Finally, timestamps are automatically added to each corrected transcribed text, ensuring that each piece of text-based teaching data corresponds to a specific time segment in the classroom video. For example, "00:12:35—00:12:48: When the net external force acting on an object is not zero, the object will accelerate." Through this process, structured text-based teaching data is obtained, providing a reliable data foundation for the subsequent construction of teaching knowledge maps and knowledge graphs. It also allows students to quickly locate the corresponding classroom explanation video segment after class by clicking on the text, enabling precise recall and learning of classroom content.

[0024] Preferably, an intelligent navigation association database is constructed by associating and matching text-based teaching data with teaching materials. The specific steps are as follows: By connecting with the teaching terminal of the smart classroom, playback data of the teaching materials is collected, including the page number and playback time of the teaching materials; A text matching algorithm is used to match text-based teaching data with the content of the teaching materials, thereby establishing a link between the text-based teaching data and the page numbers of the teaching materials. By aligning the timestamps of the written lecture data with the timeline of the classroom recording video, the association between the written lecture data and the video clips is realized, and an intelligent navigation association database of "transcribed text - courseware page number - video timestamp" is built. When a user clicks on any sentence in the written lecture data, he / she can automatically jump to the corresponding courseware page number and video clip, realizing intelligent navigation.

[0025] In a practical teaching application embodiment of the present invention, a teacher is teaching about the monotonicity of functions in a high school mathematics course. When using a teaching terminal in a smart classroom, the teacher connects with the classroom's teaching control terminal to obtain real-time playback information of the courseware, including the current page number, page switching time, and playback order. For example, when the teacher explains the definition of function monotonicity, if the courseware is playing to page 12, the start time corresponding to that page number is recorded as "00:18:20" in the class video timeline. This courseware playback data is synchronously recorded and stored, providing a temporal and content basis for subsequent text and courseware association.

[0026] After acquiring the courseware playback data, the text teaching data generated from the classroom speech transcription is matched with the courseware text content. Specifically, firstly, the title, key formulas, and core text content of each page are extracted from the courseware, and a courseware text feature vector is constructed. Then, keyword extraction and semantic vectorization are performed on the text teaching data, and the similarity between the text content and the content of each courseware page is calculated using a text matching algorithm. When the semantic similarity between a segment of text teaching data and a certain page of courseware content exceeds a preset threshold, it is determined that the explanation corresponds to that page of courseware, and an association relationship is established between the text teaching data and the courseware page number. For example, when the teacher explains "If a function increases as the independent variable increases within a certain interval, then the function is monotonically increasing within that interval," the content is identified as having a high semantic consistency with the "definition of a monotonically increasing function" on page 12 of the courseware, thus automatically establishing an association between the transcribed text and page 12 of the courseware.

[0027] Simultaneously, the timestamp information recorded in the text-based lecture data is precisely aligned with the timeline of the classroom recording video. Since each transcribed text sentence contains a corresponding time segment, such as "00:18:22—00:18:40," this time period can be mapped to the same time interval in the classroom recording video, thus establishing a correlation between the text content and the video segment. By integrating three types of information—text content, courseware page numbers, and video timelines—a smart navigation database of "transcribed text—courseware page number—video timestamp" is ultimately constructed. When students review after class, if they click on a transcribed text sentence in the learning platform, such as "the function increases as the independent variable increases within a certain interval," they will automatically jump to page 12 of the corresponding courseware and simultaneously play the explanation segment from 00:18:22 to 00:18:40 in the classroom video. This allows students to quickly locate relevant teaching content, achieving intelligent navigation and synchronous learning between classroom explanations, courseware presentations, and video playback, significantly improving the efficiency of teaching resource retrieval and the learning experience.

[0028] Preferably, the first teaching knowledge point is extracted based on the text teaching data, the hierarchical and relational relationships between the first teaching knowledge points are constructed, and a visual teaching knowledge mind map is generated. The specific steps are as follows: An attention-based natural language processing algorithm is used to extract the first teaching knowledge point from the text teaching data. The specific process is as follows: The system performs word segmentation on the transcribed text, dividing continuous text teaching data into individual words and phrases. Part-of-speech tagging is then applied to the segmented text, identifying nouns, verbs, adjectives, etc. The TF-IDF algorithm is used to calculate the weight of each word and phrase; a higher weight indicates greater importance of the word or phrase in the teaching content. Combined with professional dictionaries in the teaching field, specialized terms and core concepts with weights exceeding a preset threshold are selected as the first teaching points. Simultaneously, the system can automatically identify key points and difficulties in the teaching content by analyzing the teacher's emphasis in the transcribed text, such as keywords like "the key point is" and "the difficulty lies in," as well as repetitive expressions, to determine the key points and difficulties in the text teaching data.

[0029] Based on the extracted first teaching knowledge points, the hierarchical and relational relationships between these points are constructed, generating a visual teaching knowledge map. The specific process is as follows: Determine the hierarchical relationship of the first teaching knowledge points: use the course name as the first-level knowledge point, the chapter name as the second-level knowledge point, and the extracted core concepts as the third-level knowledge points to construct the hierarchical structure of the first teaching knowledge points; Determine the relationships between the knowledge points in the first lesson: By analyzing the semantic relationships between the knowledge points in the first lesson in the text teaching data, identify relationships such as causal relationships, parallel relationships, and progressive relationships; Using visualization technology, the hierarchical and relational relationships of the first lesson's knowledge points are displayed graphically, generating a visual knowledge map. This visual knowledge map supports zooming, panning, expanding / collapse, and other operations, making it easy for users to view and use.

[0030] In a practical teaching application embodiment of the present invention, when a teacher teaches about redox reactions in a high school chemistry course in a smart classroom, the complete text teaching data is first obtained through a classroom speech-to-text module. For example, statements made by the teacher during the explanation, such as "the essence of redox reactions is the transfer of electrons" and "identifying oxidizing agents and reducing agents is the focus of this lesson," are transcribed and recorded in real time. This text teaching data is then automatically analyzed and processed. A natural language processing algorithm is used to segment the continuous text into words and phrases such as "redox reaction," "electron transfer," "oxidizing agent," "reducing agent," and "valence change." The segmented text is further tagged with parts of speech to identify semantic units such as noun terms and verb phrases. Then, the TF-IDF algorithm is used to calculate the weight of each word in the overall teaching text, identifying words that frequently appear in the classroom explanation and have high weights. These words are then filtered using a chemistry terminology dictionary to automatically extract core concepts such as "redox reaction," "electron transfer," "oxidizing agent," "reducing agent," "gaining electrons," and "losing electrons," which are then used as the first teaching point of the lesson. At the same time, by analyzing the emphasized statements in the teacher's speech, such as keywords like "this is the focus of this lesson" and "distinguishing between oxidizing agents and reducing agents is a difficult point", as well as the content that the teacher repeats many times during the explanation, the relevant knowledge points are automatically marked as key or difficult knowledge points.

[0031] After extracting the primary knowledge points for the lesson, a hierarchical relationship between these knowledge points is further constructed based on the teaching structure. For example, the course title "Redox Reaction" is set as a primary knowledge point, chapter themes in the courseware or textbook such as "The Concept of Redox Reaction" and "Determination of Oxidizing and Reducing Agents" are set as secondary knowledge points, and specific core concepts such as "Electron Transfer," "Change in Valence," "Electron Gain Reaction," and "Electron Loss Reaction" are set as tertiary knowledge points, thus forming a complete hierarchical structure. Simultaneously, semantic analysis is performed on the textual teaching data to identify the logical relationships between knowledge points. For example, when the teacher explains "Because electron transfer occurs, this reaction belongs to redox reaction," the causal relationship between "electron transfer" and "redox reaction" is identified; when the teacher explains "Oxidizing agents and reducing agents coexist in redox reactions," the parallel relationship between "oxidizing agents" and "reducing agents" is identified; when the teacher further explains from "electron transfer" to "change in valence," a progressive relationship is identified. By establishing these hierarchical and semantic relationships, a complete knowledge structure network is gradually constructed.

[0032] After the knowledge structure is constructed, visualization technology is used to graphically display the aforementioned knowledge relationships, generating a visual knowledge map for instruction. For example, in the teaching platform interface, students can see a knowledge map centered on "redox reactions," which expands to include chapter nodes such as "the nature of the reaction" and "determining oxidizing and reducing agents," and further expands to specific concept nodes such as "electron transfer" and "changes in valence." The nodes are connected by different types of lines to represent causal, parallel, or progressive relationships. This knowledge map supports interactive operations such as zooming, panning, and expanding or collapsing nodes. When reviewing, students can click on a knowledge point to view related explanations or video clips, thus gaining a more intuitive understanding of the course knowledge structure and achieving structured learning and efficient review of classroom content.

[0033] Preferably, by parsing and processing the teaching materials, the second teaching knowledge points are extracted. Combined with the chapter structure of the teaching materials, the hierarchical relationship of the second teaching knowledge points is constructed, generating a basic framework for the teaching knowledge graph. The specific steps are as follows: By analyzing the teaching materials uploaded by teachers, including teaching materials and courseware, OCR technology is used to convert the image content in the teaching materials and courseware into text content, and then natural language processing algorithms are used to extract the second teaching knowledge points from the teaching materials and courseware. Based on the chapter structure of the teaching materials and courseware, the hierarchical relationship of the second teaching knowledge points is constructed, and the basic framework of the teaching knowledge graph is generated.

[0034] The basic framework of the teaching knowledge graph includes basic information such as the name, level, and number of knowledge points, which can be edited and modified by teachers. For example, teachers can add new knowledge points, delete redundant knowledge points, and adjust the hierarchical relationship of knowledge points.

[0035] In a practical teaching application of this invention, teachers upload teaching materials for the lesson before class through a teaching platform, including electronic versions of textbook chapters and PPT slides for classroom explanation. The teaching materials are related to the structure and function of cells in high school biology. First, the teaching materials are analyzed and processed. Since some textbook content and slides contain numerous diagrams, tables, and embedded text images, such as "Diagram of Cell Membrane Structure" and "Comparison Table of Organelle Functions," OCR technology is used to identify and extract the text information from these images, converting the text content, originally in image form, into editable and analyzable text data. This data is then integrated with the main text of the textbook to form a complete set of teaching text data. After integrating the text data, natural language processing algorithms are used to perform semantic analysis on the textbook and slide text. Through methods such as word segmentation, part-of-speech tagging, and keyword weight calculation, core concepts related to the course theme are extracted, such as "cell membrane," "nucleus," "mitochondria," "endoplasmic reticulum," "Golgi apparatus," and "material transport." These core concepts are then identified as the second teaching knowledge point of the course.

[0036] After extracting the second set of teaching knowledge points, the hierarchical relationship between these knowledge points is further constructed by integrating them with the chapter structure of the teaching materials. For example, the course theme "Structure and Function of Cells" is set as a first-level knowledge node, chapter titles in the textbook such as "Structure and Function of Cell Membrane" and "Structure and Function of Organelles" are set as second-level knowledge nodes, and specific concepts such as "Energy Metabolism Function of Mitochondria" and "Material Synthesis Role of Endoplasmic Reticulum" are set as third-level knowledge nodes, thus forming a clear hierarchical knowledge structure. Based on this, a unique number is automatically generated for each knowledge point, and the knowledge point name, its level, and its parent-child node relationships are recorded, thereby constructing a complete teaching knowledge graph framework. This provides a unified data structure support for subsequent teaching content analysis and knowledge structure display.

[0037] Furthermore, this basic framework for the teaching knowledge graph also supports manual editing and optimization by teachers. For example, during actual teaching, teachers may adjust the knowledge structure appropriately based on their teaching experience or students' learning progress. Teachers can manually add new knowledge points in the teaching platform interface, such as supplementing the knowledge node of "active transport and passive transport"; they can also delete redundant knowledge points that are automatically extracted by the system but do not need to be emphasized in actual teaching; at the same time, they can adjust the hierarchical relationship of knowledge points according to the teaching logic, such as promoting "fluid mosaic model of cell membrane" from a third-level node to a more important second-level node. Through this "automatic construction + manual correction" approach, a basic framework for the teaching knowledge graph with a clear structure, complete content, and in line with actual teaching needs is formed, providing a stable data foundation for subsequent functions such as key knowledge identification, teaching resource mounting, and knowledge map display.

[0038] Preferably, interactive analysis is performed based on the first and second knowledge points to assess the importance of each knowledge point. Key knowledge points are then identified based on their importance and marked within the basic framework of the teaching knowledge graph. The specific steps are as follows: Based on the audio data, the teacher's voice behavior features for the first knowledge point are extracted, and the weight value of classroom explanation is analyzed based on the teacher's voice behavior features for the first knowledge point. The teacher's voice behavior features include the frequency of occurrence of the first knowledge point in the audio data, the duration of explanation, the number of repetitions, and the degree of emphasis in the voice. Based on the teaching materials, the teaching text features of the second knowledge point are obtained, and the textbook structure weight value is analyzed according to the teaching text features of the second knowledge point. The textbook structure weight value includes the frequency of the second knowledge point in the teaching materials, the page number coverage ratio of the courseware, and the title level weight. By performing semantic alignment and matching analysis on the first and second knowledge points, a knowledge point interaction relationship network is constructed, and the first and second knowledge points that successfully match are initially selected as candidate key knowledge points. For candidate key knowledge points, a comprehensive importance score is formed by combining the weight values ​​of classroom explanation and textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The key knowledge points are ranked according to their overall importance score, and their order is marked in the basic framework of the teaching knowledge graph.

[0039] Preferably, by performing semantic alignment and matching analysis on the first and second knowledge points, a knowledge point interaction relationship network is constructed, and the first and second knowledge points that successfully match are initially selected as candidate key knowledge points. The specific steps are as follows: The semantic similarity between the first and second knowledge points is calculated by performing semantic vectorization on the first and second knowledge points respectively and then calculating the semantic similarity between the first and second knowledge points in a unified semantic space. When the semantic similarity is higher than the preset similarity threshold, the first knowledge point and the second knowledge point are successfully matched and form an interactive knowledge point pair; otherwise, the first knowledge point and the second knowledge point fail to match. For the first knowledge point and the second knowledge point that fail to match, the matching is supplemented by synonym expansion and contextual semantic analysis to obtain a complete cross-source knowledge point mapping relationship. The first and second knowledge points that matched successfully were initially selected as candidate key knowledge points.

[0040] Preferably, for candidate key knowledge points, a comprehensive importance score is generated by combining the weight values ​​of classroom explanation and textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The specific steps are as follows: Obtain the first and second knowledge points in each pair of interactive knowledge points. Based on the teacher's voice behavior characteristics of the first knowledge point, normalize the data and then perform weighted calculations to obtain the classroom explanation weight value. Based on the characteristics of the teaching text for the second knowledge point, the weighted calculation is performed after normalization to obtain the weight value of classroom explanation. A comprehensive score for the importance of knowledge points is generated by weighting and integrating the weighting values ​​of classroom explanation and textbook structure. The calculation formula is as follows:

[0041] In the formula: Score the overall importance of the knowledge points in the i-th pair of interactive knowledge points. The fusion coefficient for explaining weight values ​​in class. To explain the weight values ​​in class, This is the integration coefficient for the structural weight values ​​of the teaching materials. The weight value for the structure of the teaching materials; The overall importance score of a knowledge point is compared with a preset importance score threshold. If the overall importance score of a knowledge point is greater than or equal to the preset importance score threshold, the corresponding interactive knowledge point pair is considered a key knowledge point; if the overall importance score of a knowledge point is less than the preset importance score threshold, the corresponding interactive knowledge point pair is considered a non-key knowledge point.

[0042] In a real-world teaching scenario embodiment of this invention, the teacher explains data structure knowledge in class, simultaneously recording the lecture and automatically transcribing it into text data, forming the first knowledge point data source. At the same time, the teacher uploads course materials and teaching slides to the teaching platform, and these materials are analyzed to extract the second knowledge point. First, the teacher's vocal behavior characteristics are analyzed from the classroom audio data. For example, when explaining the "binary tree traversal algorithm," the teacher repeats the concept multiple times and provides a detailed explanation over a considerable period, while using emphatic expressions such as "this part is very important" and "exam focus." This allows for the statistical analysis of the frequency of this knowledge point's appearance in class, its explanation duration, repetition count, and degree of vocal emphasis, thereby calculating the corresponding classroom explanation weight value. At the same time, secondary knowledge points such as "binary tree", "preorder traversal", "inorder traversal" and "postorder traversal" are extracted from the teaching materials and courseware, and their chapter positions, frequency of occurrence and page coverage ratio in the textbook and PPT are analyzed. For example, this knowledge point is located in the core chapter of the third chapter of the textbook and appears as a title in multiple pages of courseware. Therefore, a high textbook structure weight value is calculated for it.

[0043] Subsequently, semantic vectorization technology is used to map the first and second knowledge points into a unified semantic space for similarity calculation. For example, semantic matching is performed between "binary tree traversal method" extracted from classroom audio and "binary tree traversal algorithm" from the textbook. When the calculated semantic similarity is higher than a preset threshold, the two are considered to have matched successfully, forming an interactive knowledge point pair. For knowledge points that do not match directly, such as "recursive traversal idea" in classroom explanation and "recursive algorithm idea" in textbook, further supplementary matching is performed through synonym expansion and contextual semantic analysis, thereby constructing a complete cross-source knowledge point mapping relationship network. After obtaining candidate interactive knowledge point pairs, the system normalizes them according to the weight values ​​of classroom explanation and textbook structure, and performs weighted fusion calculation according to preset fusion coefficients to obtain a comprehensive importance score for each knowledge point pair. For example, when the weight value of "binary tree traversal algorithm" in classroom explanation is high, and its weight in textbook structure is also high, its comprehensive importance score will be significantly higher than other knowledge points.

[0044] Finally, the overall importance score is compared with the preset importance score threshold. Knowledge points with higher scores are selected as key knowledge points and sorted according to their scores. These are then highlighted within the basic framework of the course knowledge graph. For example, in the generated course knowledge graph, knowledge points such as "binary tree traversal algorithm," "recursive implementation method," and "time complexity analysis" are marked as key nodes and highlighted with different colors or icons. This facilitates teachers' reinforcement of key knowledge points in subsequent teaching and also helps students quickly identify core course content during review, thereby improving teaching efficiency and learning outcomes.

[0045] Preferably, teaching resources are automatically mounted onto the corresponding knowledge points, and the teaching knowledge graph is optimized and improved according to teaching needs. The teaching resources include classroom transcripts and self-test questions. Based on the knowledge points in the basic framework of the teaching knowledge graph, the corresponding text teaching data is extracted. Based on the timestamps of the text teaching data, the time segments corresponding to the classroom recording videos are determined. The time segments are then divided into several classroom recording video segments, and knowledge point tags are added to each classroom recording video segment. When students click on any knowledge point tag in the teaching knowledge graph, the corresponding classroom recording video segment can be played automatically, which facilitates targeted learning for students. Self-assessment questions are automatically generated based on knowledge points, including multiple-choice, true / false, and other question types. Teachers can also upload custom questions. The generated or uploaded questions are automatically attached to the corresponding knowledge points, allowing students to check their learning progress by completing the self-assessment questions after studying the knowledge points. When teachers upload new classroom recordings or self-assessment questions, the system automatically analyzes the content and associates it with the corresponding knowledge points. For example, after a teacher uploads a new classroom recording, the system automatically performs speech-to-text transcription and extracts knowledge points from the recording, segments the video, and associates them with the corresponding knowledge points, thus achieving dynamic updates and associations of resources.

[0046] In a practical teaching application embodiment of this invention, when a teacher lectures on computer network-related content, the entire lesson is recorded via a recording system and uploaded to the teaching platform. Based on the knowledge points in the teaching knowledge graph framework, such as "TCP protocol," "three-way handshake process," and "congestion control mechanism," the corresponding text teaching data is automatically extracted. By analyzing the timestamp information in the classroom speech-to-text, the specific time period in the classroom recording where the teacher explains a particular knowledge point can be located; for example, the teacher might focus on explaining the "three-way handshake process" from minute 18 to minute 23. The entire classroom video is then automatically segmented into multiple video clips according to the corresponding time periods, and each clip is tagged with a corresponding knowledge point, thus forming teaching video resources corresponding to nodes in the teaching knowledge graph. When students view the teaching knowledge graph on the learning platform, they only need to click on the "three-way handshake process" knowledge point node in the graph to automatically call and play the corresponding video clip, allowing students to directly watch the classroom recording of the teacher explaining that knowledge point. This achieves precise location of learning resources by knowledge point, improving learning efficiency.

[0047] After students complete the learning of a knowledge point, corresponding self-assessment questions will be automatically generated based on that knowledge point. For example, for the knowledge point of "the three-way handshake process," multiple-choice questions such as "What type of message is sent by the client in the second handshake of the TCP three-way handshake?", multiple-answer questions such as "Which of the following are steps in the TCP three-way handshake?", and true / false questions such as "The main purpose of the TCP three-way handshake is to establish a reliable connection" can be automatically generated and attached to the corresponding knowledge point node. After learning the relevant video content, students can directly complete the self-assessment questions under that knowledge point to check their understanding of the knowledge point in a timely manner. At the same time, teachers can also upload custom practice questions according to teaching needs, such as supplementary exercises for key exam points or easily confused concepts. The system automatically identifies keywords in the question content and performs semantic matching with corresponding knowledge points in the knowledge graph, automatically attaching the questions to the corresponding knowledge points, thereby further enriching teaching resources.

[0048] Furthermore, when teachers continuously upload new classroom recordings or add new exercises throughout the semester, the system can automatically analyze and match the new resources with relevant knowledge points. For example, if a teacher adds a classroom video explaining the "TCP congestion control algorithm" in a later course, the system automatically transcribes the video into speech and extracts key knowledge points. Based on identified knowledge points such as "slow start" and "congestion avoidance," the video is then time-sliced, and the corresponding video segments are automatically attached to relevant nodes in the teaching knowledge graph. This enables dynamic updates and automatic association between teaching resources and the knowledge graph. In this way, the teaching knowledge graph not only visually displays the course knowledge structure but also continuously aggregates and updates various types of teaching resources, such as videos and exercises. This allows students to quickly locate learning content and practice resources through the knowledge graph, thus forming an intelligent learning environment centered on knowledge points.

[0049] Example 2 In this embodiment of the invention, teachers deploy a smart classroom teaching platform in a data structure course. First, a microphone array deployed in the classroom collects the teacher's audio data in real time, recording it according to a preset sampling frequency. After preprocessing the collected audio data, including noise reduction, silent segment segmentation, and speech rate standardization, a speech recognition model automatically converts the audio content into text teaching data, forming a complete transcript of the classroom explanation. Subsequently, the text teaching data is correlated and matched with the teaching materials played by the teacher in class. By collecting courseware playback log information, such as page numbers and playback times, a mapping relationship is established between each segment of text content explained by the teacher and the corresponding courseware page. Furthermore, the timestamps in the text teaching data are aligned with the timeline of the classroom recording video, thereby constructing an intelligent navigation association database connecting "transcribed text—courseware page number—video timestamp." When students view the classroom content on the learning platform, they can directly click on a segment of text to automatically jump to the corresponding courseware page and the corresponding time segment of the classroom video, achieving precise navigation and playback.

[0050] Building upon this foundation, semantic analysis is further performed on the text-based lecture data. Natural language processing algorithms are used to extract the core knowledge points covered by the teachers during classroom explanations, such as "linked list structure," "stack and queue," and "binary tree traversal." Based on these primary knowledge points, hierarchical and relational relationships are constructed. For example, the course name is used as a first-level node, chapter topics as second-level nodes, and specific concepts as third-level nodes. Simultaneously, progressive or causal relationships between knowledge points are identified, generating a visual lecture knowledge map. Simultaneously, the lecture materials uploaded by teachers, such as textbooks and courseware, are analyzed and processed. OCR recognition and text analysis technologies are used to extract secondary knowledge points from the textbooks, and a hierarchical knowledge point system is constructed based on the textbook's chapter structure, forming the basic framework of the course lecture knowledge graph. Subsequently, semantic matching and interactive analysis were conducted on the first knowledge point extracted from the classroom explanation and the second knowledge point extracted from the textbook. Combined with information such as the frequency, duration, and emphasis of classroom explanations, as well as the chapter level and frequency of occurrence in the textbook, the importance of each knowledge point was comprehensively evaluated. This allowed the key knowledge points in the course to be selected and marked in the basic framework of the teaching knowledge graph, helping students quickly identify the key learning points.

[0051] In terms of course resource management, teaching resources are automatically linked to corresponding knowledge point nodes. For example, based on the timestamps in the text-based lecture data, classroom recordings are cut into multiple video segments corresponding to knowledge points, and each segment is tagged with a knowledge point. This allows students to directly play the corresponding lecture video when they click on a knowledge point in the knowledge graph. Simultaneously, it can automatically generate corresponding self-assessment questions based on knowledge points, such as single-choice, multiple-answer, or true / false questions, and allows teachers to upload supplementary practice questions, all of which are automatically linked to the corresponding knowledge point nodes. When teachers upload new classroom videos or supplementary questions in subsequent teaching, semantic analysis is automatically performed and resource matching is completed, dynamically linking the new resources to the corresponding knowledge points, thereby continuously improving the teaching knowledge graph structure. Through this approach, the course knowledge structure, classroom lecture content, and teaching resources are organically integrated, enabling students to conduct precise learning and targeted review based on the knowledge graph, thus significantly improving learning efficiency and teaching management level.

[0052] Example 3 An intelligent teaching interaction system integrating speech transcription and knowledge graph includes a lecture audio acquisition and transcription module, a voice intelligent navigation module, a lecture material parsing module, a multi-source knowledge point interactive recognition module, and a teaching resource mounting and optimization module. The lecture audio acquisition and transcription module is connected to the voice intelligent navigation module, the voice intelligent navigation module is connected to the lecture material parsing module, the lecture material parsing module is connected to the multi-source knowledge point interactive recognition module, and the multi-source knowledge point interactive recognition module is connected to the teaching resource mounting and optimization module. The lecture audio acquisition and transcription module is used to acquire audio data of teachers' lectures through a microphone, and convert it into text lecture data after preprocessing. The voice intelligent navigation module is used to build an intelligent navigation association database by associating and matching text teaching data with teaching courseware, extracting the first teaching knowledge points based on the text teaching data, constructing the hierarchical and association relationships between the first teaching knowledge points, and generating a visual teaching knowledge mind map; The teaching material parsing module is used to analyze and process the teaching materials, extract the second teaching knowledge points, and, in combination with the chapter structure of the teaching materials, construct the hierarchical relationship of the second teaching knowledge points to generate the basic framework of the teaching knowledge graph. The multi-source knowledge point interaction recognition module is used to perform interactive analysis based on the first and second knowledge points, evaluate the importance of each knowledge point, determine the key knowledge points based on their importance, and mark them in the basic framework of the teaching knowledge graph. The teaching resource mounting and optimization module is used to automatically mount teaching resources to corresponding knowledge points and optimize and improve the teaching knowledge graph according to teaching needs.

[0053] Through the above embodiments, this invention, by automatically collecting, preprocessing, and transcribing teachers' audio lecture data, can structure traditional classroom oral explanations into searchable and analyzable textual lecture data, achieving digital accumulation and long-term preservation of classroom information. Compared to traditional teaching resource management methods that rely solely on courseware or textbooks, this invention can completely record the teacher's actual lecture process in the classroom, including supplementary explanations, case analyses, and key points emphasized, thus forming a more comprehensive and authentic teaching data foundation and providing a reliable data source for subsequent knowledge extraction and teaching analysis.

[0054] By linking and matching text-based lecture data with lecture slide playback data, an intelligent navigation database is constructed, connecting "transcribed text—lecture slide page number—video timestamp," enabling precise location and rapid retrieval of classroom content. When students or teachers need to review a specific knowledge point, they can simply click on the corresponding text or knowledge node to automatically jump to the relevant lecture slide page and classroom video clip. This significantly improves the efficiency of retrieving teaching resources and the convenience of learning, not only enhancing the traceability of classroom content but also effectively solving the problem of traditional classroom videos struggling to quickly locate key explanations.

[0055] Furthermore, by extracting the first and second lecture knowledge points from classroom lectures and teaching materials respectively, and constructing hierarchical and relational relationships among these knowledge points, a clear and logically complete lecture knowledge graph can be formed. On the one hand, this graph accurately reflects the teacher's focus and knowledge organization methods in the classroom; on the other hand, it maintains consistency with the textbook's chapter structure, making the knowledge structure more standardized and systematic. By performing semantic matching and interactive analysis on the two types of knowledge points, and combining this with a comprehensive evaluation based on the weight of classroom lectures and textbook structure, key knowledge points in the course can be identified more objectively and accurately, thus avoiding the subjective problems caused by relying solely on textbook structure or teacher experience for key point classification.

[0056] Furthermore, by automatically embedding classroom video clips and self-assessment questions into the teaching knowledge graph, scattered teaching content can be deeply integrated with knowledge points, forming a teaching resource organization centered on knowledge points. Students can engage in targeted learning around knowledge graph nodes, watching corresponding classroom explanation videos and completing accompanying exercises for self-assessment, thus achieving an integrated learning model of "knowledge points—explanation content—exercise resources." Simultaneously, when teachers add teaching resources, the system can automatically perform content analysis and knowledge point matching, enabling dynamic updates and continuous improvement of resources, enhancing the intelligence level of teaching resource management.

[0057] In summary, this invention, by integrating classroom voice data analysis, textbook structure analysis, and knowledge graph construction technologies, achieves automated structural organization and key point identification of teaching content. This not only effectively improves the efficiency of classroom content organization and management but also provides students with clearer and more precise learning paths, further enhancing teaching quality and learning outcomes. Furthermore, it provides crucial technical support for applications such as smart classrooms, digital teaching platforms, and teaching data analysis.

[0058] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0059] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent teaching interaction method integrating speech transcription and knowledge graph, characterized in that, Includes the following steps: Audio data of the teacher's lectures is collected through a microphone, preprocessed, and then converted into text lecture data; By associating and matching text-based teaching data with teaching materials, an intelligent navigation association database is constructed. Based on the text-based teaching data, the first teaching knowledge point is extracted, and the hierarchical and relational relationships between the first teaching knowledge points are constructed to generate a visual teaching knowledge mind map. By analyzing and processing the teaching materials, the second teaching knowledge points are extracted. Combined with the chapter structure of the teaching materials, the hierarchical relationship of the second teaching knowledge points is constructed, and the basic framework of the teaching knowledge graph is generated. Based on the first and second knowledge points, an interactive analysis is conducted to assess the importance of each knowledge point. Key knowledge points are determined according to their importance and marked in the basic framework of the teaching knowledge graph. The teaching resources are automatically attached to the corresponding knowledge points, and the teaching knowledge graph is optimized and improved according to teaching needs.

2. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 1, characterized in that, By associating and matching text-based teaching data with teaching materials, an intelligent navigation association database is constructed. The specific steps are as follows: By connecting with the teaching terminal of the smart classroom, playback data of the teaching materials is collected; A text matching algorithm is used to match text-based teaching data with the content of the teaching materials, thereby establishing a link between the text-based teaching data and the page numbers of the teaching materials. Align the timestamps of the written lecture data with the timeline of the classroom recording video to link the written lecture data with the video clips and build an intelligent navigation association database.

3. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 1, characterized in that, The first lecture knowledge points are extracted based on the text-based lecture data. The hierarchical and relational relationships between these knowledge points are constructed, and a visual lecture knowledge map is generated. The specific steps are as follows: An attention-based natural language processing algorithm is used to extract the first teaching knowledge point from the text teaching data; Based on the extracted first teaching knowledge points, the hierarchical and relational relationships between the first teaching knowledge points are constructed to generate a visual teaching knowledge mind map.

4. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 3, characterized in that, Determine the hierarchical relationship of the first teaching knowledge points: use the course name as the first-level knowledge point, the chapter name as the second-level knowledge point, and the extracted core concepts as the third-level knowledge points to construct the hierarchical structure of the first teaching knowledge points; Determine the relationships between the knowledge points in the first lesson: by analyzing the semantic relationships between the knowledge points in the first lesson in the text teaching data.

5. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 1, characterized in that, By analyzing and processing the teaching materials, the second teaching knowledge point is extracted. Combined with the chapter structure of the teaching materials, the hierarchical relationship of the second teaching knowledge point is constructed, generating the basic framework of the teaching knowledge graph. The specific steps are as follows: The teaching materials uploaded by teachers are analyzed. The teaching materials include teaching materials and teaching slides. OCR technology is used to convert the image content in the teaching materials and teaching slides into text content, and natural language processing algorithms are used to extract the second teaching knowledge points in the teaching materials and teaching slides. Based on the chapter structure of the teaching materials and courseware, the hierarchical relationship of the second teaching knowledge points is constructed, and the basic framework of the teaching knowledge graph is generated.

6. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 1, characterized in that, Based on the interactive analysis of the first and second knowledge points, the importance of each knowledge point is assessed, and key knowledge points are identified according to their importance. These key knowledge points are then marked in the basic framework of the teaching knowledge graph. The specific steps are as follows: Based on the audio data, extract the teacher's voice behavior features for the first knowledge point, and analyze the weight value of classroom explanation based on the teacher's voice behavior features for the first knowledge point. Based on the teaching materials, the teaching text features of the second knowledge point are obtained, and the structural weight value of the textbook is analyzed based on the teaching text features of the second knowledge point. By performing semantic alignment and matching analysis on the first and second knowledge points, a knowledge point interaction relationship network is constructed, and the first and second knowledge points that successfully match are initially selected as candidate key knowledge points. For candidate key knowledge points, a comprehensive importance score is formed by combining the weight values ​​of classroom explanation and textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The key knowledge points are ranked according to their overall importance score, and their order is marked in the basic framework of the teaching knowledge graph.

7. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 6, characterized in that, By performing semantic alignment and matching analysis on the first and second knowledge points, a knowledge point interaction relationship network is constructed. The first and second knowledge points that successfully match are initially selected as candidate key knowledge points. The specific steps are as follows: The semantic similarity between the first and second knowledge points is calculated by performing semantic vectorization on the first and second knowledge points respectively and then calculating the semantic similarity between the first and second knowledge points in a unified semantic space. When the semantic similarity is higher than the preset similarity threshold, the first knowledge point and the second knowledge point are successfully matched and an interactive knowledge point pair is formed. Conversely, the first knowledge point and the second knowledge point fail to match. For the first and second knowledge points that fail to match, supplementary matching is performed through synonym expansion and contextual semantic analysis to obtain a complete cross-source knowledge point mapping relationship. The first and second knowledge points that matched successfully were initially selected as candidate key knowledge points.

8. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 7, characterized in that, For candidate key knowledge points, a weighted score is generated by combining the weight of classroom explanation and the weight of textbook structure. Key knowledge points are then selected based on this comprehensive importance score. The specific steps are as follows: Obtain the first and second knowledge points in each pair of interactive knowledge points. Based on the teacher's voice behavior characteristics of the first knowledge point, normalize the data and then perform weighted calculations to obtain the classroom explanation weight value. Based on the characteristics of the teaching text for the second knowledge point, the weighted calculation is performed after normalization to obtain the weight value of classroom explanation. A comprehensive importance score for knowledge points is generated by weighting and integrating the weight values ​​of classroom explanation and textbook structure. The overall importance score of the knowledge point is compared with the preset importance score threshold. If the overall importance score of the knowledge point is greater than or equal to the preset importance score threshold, the corresponding interactive knowledge point pair is a key knowledge point. If the overall importance score of a knowledge point is less than the preset importance score threshold, then the corresponding interactive knowledge point will be used to replace the non-key knowledge point.

9. The intelligent teaching interaction method integrating speech transcription and knowledge graph as described in claim 1, characterized in that, The teaching resources are automatically attached to the corresponding knowledge points, and the teaching knowledge graph is optimized and improved according to teaching needs. The teaching resources include classroom transcripts and self-assessment questions. The specific steps are as follows: Based on the knowledge points in the basic framework of the teaching knowledge graph, the corresponding text teaching data is extracted. Based on the timestamp of the text teaching data, the time segment corresponding to the classroom recording video is determined. The time segment is cut into several classroom recording video segments, and knowledge point tags are added to each classroom recording video segment. Self-assessment questions are automatically generated based on knowledge points, and teachers can also upload custom questions.

10. An intelligent teaching interaction system integrating speech transcription and knowledge graph, applied to the intelligent teaching interaction method integrating speech transcription and knowledge graph as described in any one of claims 1-9, characterized in that, The system includes a lecture audio acquisition and transcription module, a voice intelligent navigation module, a lecture material parsing module, a multi-source knowledge point interactive recognition module, and a teaching resource mounting and optimization module. The lecture audio acquisition and transcription module is used to acquire audio data of teachers' lectures through a microphone, and convert it into text lecture data after preprocessing. The voice intelligent navigation module is used to build an intelligent navigation association database by associating and matching text teaching data with teaching courseware, extracting the first teaching knowledge points based on the text teaching data, constructing the hierarchical and association relationships between the first teaching knowledge points, and generating a visual teaching knowledge mind map; The teaching material parsing module is used to analyze and process the teaching materials, extract the second teaching knowledge points, and, in combination with the chapter structure of the teaching materials, construct the hierarchical relationship of the second teaching knowledge points to generate the basic framework of the teaching knowledge graph. The multi-source knowledge point interaction recognition module is used to perform interactive analysis based on the first and second knowledge points, evaluate the importance of each knowledge point, determine the key knowledge points based on their importance, and mark them in the basic framework of the teaching knowledge graph. The teaching resource mounting and optimization module is used to automatically mount teaching resources to corresponding knowledge points and optimize and improve the teaching knowledge graph according to teaching needs.

Images