Oral medicine teaching system fusing knowledge graph and multi-modal intelligent interaction

Abstract

This invention provides a dental medicine teaching system integrating knowledge graphs and multimodal intelligent interaction, comprising: a dental medicine knowledge graph construction module, used to construct a knowledge graph containing dental medicine teaching entities and semantic relationships based on acquired data; and a multimodal teaching resource semantic association engine, used to semantically annotate teaching resources with knowledge point nodes in the knowledge graph, establishing a closed-loop teaching link of knowledge point → explanation → case → exercise. By constructing a knowledge graph and associating its nodes with teaching resources, a closed-loop teaching link is established, thereby linking related knowledge points in the teaching resources, achieving the goal of associating teaching resource information and building a systematic knowledge network.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to an oral medicine teaching system that integrates knowledge graphs and multimodal intelligent interaction. Background Technology

[0002] Current oral medicine education mainly relies on traditional methods such as textbook lectures, PowerPoint presentations, laboratory classes, and case discussions, which suffer from problems such as fragmented knowledge, insufficient personalized teaching, and weak training in clinical reasoning. In recent years, some studies have attempted to use knowledge graphs for medical information organization or auxiliary diagnosis, but they have not yet been deeply applied to medical education scenarios.

[0003] Existing technologies mainly include:

[0004] Static teaching resource database: It centrally stores courseware, videos, and question banks, but lacks semantic association and intelligent recommendation.

[0005] Single-function simulation systems, such as dental simulation head models or simple VR training systems, only provide an operating environment and lack intelligent feedback and knowledge guidance.

[0006] Basic online learning platforms: support course playback and quizzes, but cannot dynamically adjust content according to students' levels and lack in-depth cognitive support.

[0007] General-purpose AI question-answering tools: use large language models to answer medical questions, but there is a risk of "illusion" and a lack of factual constraints and evidence-based support.

[0008] These solutions generally suffer from problems such as isolated knowledge, weak interactivity, unreliable feedback, and lack of personalization, making it difficult to meet the growing demands of modern oral medicine education for systematic, intelligent, and safe approaches.

[0009] The existing technology has the following significant drawbacks:

[0010] The lack of semantic connections between teaching resources makes it difficult for students to build a systematic knowledge network;

[0011] The operational training system lacks real-time and accurate knowledge feedback, making it impossible to achieve a closed loop of "learning by doing".

[0012] Intelligent tutoring tools lack assurance of medical accuracy and pose a risk of being misleading.

[0013] The learning feedback mechanism is not transparent, and students cannot understand the logical basis for wrong decisions;

[0014] The teaching process lacks personalization and adaptability, making it difficult to teach according to individual needs. Summary of the Invention

[0015] To address the problems existing in the prior art, this invention provides a dental medicine teaching system that integrates knowledge graphs and multimodal intelligent interaction, which at least partially solves the problems of isolated teaching information and lack of correlation in the prior art.

[0016] In a first aspect, embodiments of this disclosure provide a dental medicine teaching system that integrates knowledge graphs and multimodal intelligent interaction, comprising:

[0017] The oral medicine knowledge graph construction module is used to construct a knowledge graph containing oral medicine teaching entities and semantic relationships based on the acquired data.

[0018] A multimodal teaching resource semantic association engine is used to semantically annotate teaching resources with knowledge point nodes in the knowledge graph, and establish a closed-loop teaching link of knowledge point → explanation → case → exercise;

[0019] A learning path generation engine is used to dynamically generate the optimal learning sequence based on the student profile and the semantic relationships in the knowledge graph.

[0020] The operation and real-time feedback module is used to collect student operation parameters in the virtual environment, judge the standardization of operation, and provide feedback on the knowledge points associated in the knowledge graph based on the student operation data.

[0021] The teaching feedback and reasoning path visualization module is used to generate a visualized reasoning path based on the knowledge graph and perform a difference comparison analysis when students diagnose errors.

[0022] The dynamic evaluation and knowledge evolution module is used to optimize teaching effectiveness and knowledge system, track student behavior data to update student knowledge mastery model, and automatically update knowledge graph based on student and teacher feedback to achieve system self-evolution.

[0023] Secondly, this disclosure also provides a method for teaching oral medicine that integrates knowledge graphs and multimodal intelligent interaction, including:

[0024] A knowledge graph containing oral medicine teaching entities and semantic relationships was constructed based on the acquired data;

[0025] The teaching resources are semantically labeled with the knowledge point nodes in the knowledge graph to establish a closed-loop teaching link of knowledge point → explanation → case → exercise.

[0026] The oral medicine teaching system provided by this invention integrates knowledge graphs and multimodal intelligent interaction. By constructing a knowledge graph and associating the nodes of the knowledge graph with teaching resources, a closed-loop teaching link is established, thereby linking related knowledge points in the teaching resources and achieving the goal of associating teaching resource information and building a systematic knowledge network. Attached Figure Description

[0027] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0028] Figure 1 A schematic diagram of the principle of the oral medicine teaching system that integrates knowledge graph and multimodal intelligent interaction provided in the embodiments of this disclosure;

[0029] Figure 2 This is a schematic diagram of VR / AR operation and knowledge feedback interaction provided in an embodiment of this disclosure;

[0030] Figure 3 This is a flowchart illustrating the workflow of an AI virtual teaching assistant provided in this embodiment of the disclosure.

[0031] Figure 4 A schematic diagram of the interpretable feedback and reasoning path visualization interface provided for embodiments of this disclosure;

[0032] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0033] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0034] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0035] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0036] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0037] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0038] For ease of understanding, such as Figure 1 As shown in the figure, this embodiment discloses a dental medicine teaching system that integrates knowledge graphs and multimodal intelligent interaction, including:

[0039] The oral medicine knowledge graph construction module is used to construct a knowledge graph containing oral medicine teaching entities and semantic relationships based on the acquired data.

[0040] A multimodal teaching resource semantic association engine is used to semantically annotate teaching resources with knowledge point nodes in the knowledge graph, establishing a closed-loop teaching link of knowledge point → explanation → case → exercise.

[0041] Oral Medicine Knowledge Graph Construction Module: Used to build a knowledge graph (OralEd-KG) specifically for oral medicine education.

[0042] Specifically, the system extracts entities (such as diseases, anatomical structures, treatment plans, complications, and guideline entries) from real clinical data (electronic medical records, CBCT images, gene reports, and follow-up records), defines semantic relationships (such as "prerequisite knowledge", "contraindications", "recommendation level", and "source of evidence"), and forms a structured knowledge network.

[0043] Multimodal teaching resource semantic association engine: used to perform high-precision semantic annotation of unstructured or semi-structured teaching resources (such as teaching videos, 3D models, typical cases, and exam question banks) with knowledge point nodes in the knowledge graph, and establish a closed-loop teaching link of "knowledge point → explanation → case → practice".

[0044] To ensure the accuracy and traceability of knowledge point annotation, this system adopts a two-stage semantic alignment method that combines a multimodal fusion attention mechanism with knowledge graph path matching verification.

[0045] (1) Multimodal content analysis and preliminary annotation:

[0046] Instructional video: Using a 3D convolutional neural network (C3D) to extract action features, combined with ASR speech recognition text, medical entities (such as "dental preparation" and "shoulder preparation") are extracted using a BiLSTM-CRF model, and vector similarity matching is performed with knowledge graph entities.

[0047] 3D Model: Graph Neural Networks (GNNs) are used to model the topology of the 3D mesh, extract semantic features of the anatomical structure, and perform graph embedding alignment with the "anatomical entities" in the knowledge graph.

[0048] Typical case: Named entity recognition (NER) and relation extraction (RE) models are used to extract the "chief complaint-diagnosis-treatment" triples from the text and map them to the "disease-symptom-treatment" subgraph in the knowledge graph.

[0049] Exam Question Bank: The question text is converted into logical expressions by a semantic parser and matched with the "concept-attribute-relationship" structure in the knowledge graph.

[0050] Knowledge point annotation accuracy assurance mechanism:

[0051] Introducing a semantic consistency scoring function This allows for the quantification of the matching degree between resources and knowledge points, and the setting of thresholds to filter low-confidence annotations.

[0052] The semantic consistency scoring function is:

[0053] ,

[0054] in, For teaching resources (such as a video clip). These are knowledge point nodes in a knowledge graph. For resources Multimodal fusion embedding vectors (from video + speech + text). For nodes Knowledge graph embedding (such as TransE, RotatE encoding). Cosine similarity is used to measure the closeness of vector spaces. From resource-related entities to nodes The probability of the existence of the shortest semantic path in a knowledge graph (calculated based on the PageRank diffusion algorithm). To determine the contextual consistency score, it is judged whether the combination of terms appearing in the resources has reasonable co-occurrence in the knowledge graph (e.g., "root canal treatment" and "working length measurement" should co-occur). , and These are the weighting coefficients (learnable parameters that satisfy...) ).

[0055] Judgment rule, only when Knowledge points will only be included when a preset threshold, such as 0.75, is reached. Labeling resources .

[0056] Construction of a closed-loop teaching system:

[0057] Establish a four-level mapping relationship:

[0058] Knowledge Point → Explanation: Each knowledge point is associated with multiple explanatory resources (videos, PPTs);

[0059] Explanation → Case Study: Each teaching resource links to a typical clinical case (including imaging and medical history);

[0060] Case Studies → Exercises: Each case study is accompanied by generative AI-generated variation exercises or VR operation tasks;

[0061] Practice → Knowledge Point: Practice feedback traces back to the original knowledge point, forming a closed loop;

[0062] This closed loop supports dynamic updates: when a student makes a mistake during practice, it triggers weak knowledge points, pushes relevant resources, and then retests the student, updating their mastery status.

[0063] Learning path generation engine:

[0064] Function: Dynamically generates the optimal learning sequence based on student profiles to meet the learning needs of different stages (undergraduate / standardized residency / continuing education).

[0065] Specific implementation steps:

[0066] Define a personalized learning path as an ordered sequence of knowledge points P={k1,k2,...,k n This satisfies the knowledge dependency constraint and matches the learner's ability.

[0067] Step 1: Construct the student profile vector Vs ;

[0068] V s =[stage,K m ,P f ];

[0069] stage∈{1,2,3}: learning stage (1=undergraduate, 2=residency training, 3=continuing education);

[0070] K m Knowledge mastery vector, dimension equal to the total number of knowledge points, K m (k i The value ∈ [0,1] represents the probability of mastering the knowledge point;

[0071] P f Learning preference vectors include preference type (visual / auditory / hands-on), learning pace (fast / medium / slow), and feedback style (direct / guided).

[0072] Step 2: Extract prerequisite dependencies from the knowledge graph:

[0073] Extract the "prerequisite knowledge" relationships from the knowledge graph to construct a directed acyclic graph G=(K,R), where K is the set of knowledge points. For dependent edges.

[0074] For example: "Root canal treatment" → "Mastering root canal anatomy" → "Mastering tooth anatomy";

[0075] Step 3: Define the learning path and optimize the objective function:

[0076] Minimize total learning cost while maximizing knowledge coverage and interest matching:

[0077] ,

[0078] in: Knowledge transfer distance is defined as the shortest path length (number of hops) between two nodes in the graph.

[0079] s(k): The degree of fit between knowledge point k and the current learner's interest, determined by P. f The degree of matching with resource type determines

[0080] Interest adjustment factor (empirical value 0.3~0.6).

[0081] Step 4: Path generation algorithm (based on A* search):

[0082] Using an improved A* algorithm, search the knowledge graph from the current mastery set;

[0083] To target capability set The shortest path, heuristic function is:

[0084] ,

[0085] in, For nodes To target capability set The shortest path distance, This represents the priority coefficient for weak points. For knowledge mastery vectors, For nodes The probability of mastering it.

[0086] The output path P is the recommended learning sequence.

[0087] Operation and real-time feedback module: implemented based on VR / AR, such as... Figure 2 As shown, the details are as follows:

[0088] Function: Perform operations such as tooth preparation and implantation in a virtual environment. The system monitors operation parameters in real time and determines whether they deviate from the standard, triggering knowledge graph feedback.

[0089] Specific implementation and innovation formula:

[0090] (1) Acquisition of operating parameters

[0091] The following parameters are collected in real time using VR controllers / haptic styluses and head-mounted displays for spatial positioning:

[0092] Angle between the tool axis and the long axis of the tooth (unit: °).

[0093] : Depth of cut (unit: mm)

[0094] p(t)=(x(t), y(t), z(t)): Spatial trajectory of the tool tip.

[0095] Cutting speed (unit: mm / s).

[0096] (2) Formula for determining deviation from specifications:

[0097] Define the local deviation index Used to determine whether a violation has occurred at the current moment:

[0098] ,in, The angle between the tool axis and the long axis of the tooth. For cutting depth, For the spatial trajectory of the tool tip, For cutting speed, , and These are standard reference values ​​for the operating procedures (from an expert database). , and Allowable error range (e.g., angle ±5°, depth ±0.2mm). , and The weighting coefficients are dynamically adjusted based on the operation type, such as the angle being more important in shoulder preparation. This is the local deviation index.

[0099] Triggering condition: If If the value is greater than 1, meaning any parameter exceeds the tolerance range, it is judged as "operational deviation".

[0100] (3) Global trajectory consistency check (preventing false alarms)

[0101] To avoid accidental triggering due to momentary jitter, a sliding window consistency check is introduced:

[0102] ,

[0103] ,

[0104] T: Time window (e.g., 2 seconds), δ: Global threshold (e.g., 0.8).

[0105] Only when D local (t)>1 and D global Knowledge graph feedback is only triggered when the value is greater than δ.

[0106] (4) Feedback and linkage mechanism:

[0107] Once triggered, the system queries the knowledge graph for "normative principles" nodes related to the current operation, for example:

[0108] Operation: "Shoulder angle too steep";

[0109] Triggering knowledge point: "Full crown edge design principle";

[0110] Feedback content: "The shoulder angle should be 90°±5°. Too steep an angle can easily lead to a decrease in marginal fit → Quoted from Prosthodontics, 7th edition."

[0111] like Figure 3 As shown, the AI ​​virtual teaching assistant module adopts a generative AI virtual teaching assistant system (constrained by a knowledge graph):

[0112] Features: Provides intelligent Q&A and personalized guidance, ensuring content safety and accuracy.

[0113] Implementation details: A large language model is deployed as a virtual teaching assistant, but its output is strictly limited to entities and relationships within the knowledge graph. The system only allows it to cite existing evidence from the graph in its answers, with source annotations (such as guidelines and literature) to prevent "illusions." It supports automatic generation of practice questions and simulated doctor-patient dialogues.

[0114] like Figure 4 As shown, the teaching feedback and reasoning path visualization module is interpretable:

[0115] Function: When trainees make incorrect diagnoses or treatment choices, the system provides traceable reasoning paths and evidence support.

[0116] Question: How do we determine if a diagnosis is accurate?

[0117] The determination mechanism is as follows:

[0118] Matching with the standard answer database:

[0119] For structured questions (such as multiple-choice questions), the system has built-in standard answers.

[0120] For open-label cases, the system pre-sets a "gold standard diagnostic pathway" (marked by the expert committee).

[0121] Logical consistency verification based on knowledge graphs:

[0122] Define the diagnostic rationality scoring function R diag :

[0123] ,

[0124] Where: π: the set of knowledge graph edges involved in the student's diagnostic path (e.g., "toothache → caries → deep caries → reversible pulpitis");

[0125] e represents specific diagnostic evidence or diagnostic elements, and e∈π means summing all elements e in the set π.

[0126] confidence(e): The recommendation level of this relationship in clinical guidelines (A / B / C level).

[0127] support(e): The number of supporting documents or the level of evidence for this relationship (e.g., RCT=1.0, case report=0.3).

[0128] If R diag <R threshold If a forbidden edge exists in the path (e.g., "uncontrolled diabetes → direct implantation"), it is considered an error, R. threshold To set a threshold.

[0129] Counterfactual Reasoning:

[0130] The system generates a "correct path" and compares it with the student's path, visualizing the differences:

[0131] Student path: Bleeding gums → Tartar buildup → Recommended teeth cleaning;

[0132] Correct path: Gingival bleeding → attachment loss → bone resorption → periodontitis → systemic treatment required;

[0133] Difference: "Unidentified attachment loss" → Triggers the knowledge point "Early diagnostic criteria for periodontitis".

[0134] Inference path visualization interface:

[0135] The structure is shown below:

[0136] [Symptoms] → [Signs] → [Differential Diagnosis] → [Criteria for Diagnosis] → [Treatment Principles]

[0137] ↘[Exclude diseases] (indicate the reason for exclusion).

[0138] Each step should include the source information: guideline name, document DOI, and level of evidence.

[0139] The Dynamic Assessment and Knowledge Evolution module's function is to continuously optimize teaching effectiveness and the knowledge system.

[0140] Specifically, the system tracks student behavior data (answers, actions, and dwell time) to update their "knowledge mastery model"; teachers can view the class's "knowledge heatmap"; and student and teacher feedback feeds back into the knowledge graph, enabling the system to self-evolve.

[0141] This implementation deploys a large language model as a virtual teaching assistant, but its output is strictly constrained by the OralEd-KG (Oral Medicine Teaching Knowledge Graph): the AI ​​can only respond by referencing entities, relationships, and their evidence sources (such as clinical guidelines and authoritative literature) existing in the graph, and automatically generates citation annotations; the system uses a semantic alignment mechanism to filter or correct "illusionary" responses that exceed the graph's scope, ensuring the medical accuracy, evidence-based approach, and safety of the teaching content. The system supports various teaching scenarios, including intelligent question answering, case analysis, exercise generation, and simulated doctor-patient dialogues.

[0142] When trainees make incorrect diagnoses or inappropriate treatment plans during case decision-making, the system constructs their decision path based on a knowledge graph and uses the diagnostic rationality scoring function R to evaluate the rationality of the diagnosis. diagThe system determines the correctness of the score. If the score is below the threshold or a taboo relationship exists, the system generates visual feedback, showing the difference between the standard reasoning path and the trainee's path, and reveals key error points through counterfactual analysis, thereby improving the transparency and credibility of clinical reasoning training.

[0143] The system extracts content features from multimodal resources such as teaching videos, 3D models, typical cases, and exam question banks, performs semantic matching with knowledge graph nodes, and calculates semantic consistency scores.

[0144] This method supports the construction of a closed-loop teaching chain of "knowledge point → explanation → case → practice".

[0145] Based on student profiles (learning stage, knowledge mastery vector K) m Learning preferences P f Based on the prior dependencies in the knowledge graph, a directed acyclic graph (DAG) is constructed, and an improved A* search algorithm is used to generate the optimal learning sequence. Prioritizing the strengthening of weak knowledge points is encouraged. The optimization objective is to minimize the learning cost.

[0146] To achieve truly individualized and adaptive teaching.

[0147] The overall architecture and workflow of an intelligent oral health teaching system that integrates knowledge graphs, VR / AR, generative AI, and interpretable reasoning.

[0148] The system integrates six modules: knowledge graph construction, multimodal resource association, personalized path recommendation, immersive operation training, AI-assisted question answering, and explainable feedback, forming a complete teaching loop of "knowledge organization—immersive interaction—intelligent service—feedback optimization." Its workflow includes: knowledge graph initialization, resource semantic annotation, student profile modeling, path generation, VR operation and feedback, AI interaction, diagnostic assessment, and system self-evolution, achieving fully digital, intelligent, and traceable teaching support.

[0149] This embodiment includes the design of entity and relation types specifically for teaching purposes:

[0150] Medical entities (such as diseases, anatomical structures, and treatment plans) are extracted from electronic medical records, CBCT images, gene reports, and teaching materials. Teaching-specific semantic relationships are defined, such as "prerequisite knowledge," "easily confused concepts," "common misconceptions," "recommendation levels," "sources of evidence," and "contraindications," forming a knowledge network (OralEd-KG) oriented towards teaching objectives. This atlas not only supports information retrieval but also serves learning path planning, error diagnosis attribution, and the generation of interpretable feedback, forming the core knowledge foundation of the entire system.

[0151] Compared with the prior art, this embodiment has the following significant advantages:

[0152] Systematic learning: By constructing a structured knowledge system through knowledge graphs, students can build a complete cognitive network.

[0153] Immersive training: Combining VR / AR with real-time knowledge feedback significantly improves the efficiency of learning operational skills.

[0154] Intelligent and safe tutoring: Generative AI provides accurate answers under the constraints of knowledge graphs, avoiding misleading questions.

[0155] Explainable feedback: Visualized reasoning paths help trainees understand "why they were wrong" and strengthen their clinical thinking.

[0156] Personalization and Adaptation: Dynamic learning paths and continuous evolution mechanisms cater to individual differences and improve teaching effectiveness.

[0157] Complete teaching loop: Realizes an integrated teaching process of "knowledge organization - immersive interaction - intelligent service - feedback optimization".

[0158] The electronic device according to embodiments of this disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0159] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of this disclosure, the processor is used to run computer-readable instructions stored in the memory, causing the electronic device to perform all or part of the steps of the oral medicine teaching method integrating knowledge graphs and multimodal intelligent interaction described in the foregoing embodiments of this disclosure.

[0160] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0161] like Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 5The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0162] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0163] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to exchange data wirelessly or via wired communication with other devices, such as edge computing devices. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0164] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the oral medicine teaching method integrating knowledge graphs and multimodal intelligent interaction according to embodiments of this disclosure are performed.

[0165] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0166] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the oral medicine teaching method integrating knowledge graphs and multimodal intelligent interaction described in the foregoing embodiments of the present disclosure are performed.

[0167] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0168] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0169] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0170] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0171] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0172] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0173] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0174] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0175] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An oral medicine teaching system fusing a knowledge graph and a multi-modal intelligent interaction, characterized in that, include: The oral medicine knowledge graph construction module is used to construct a knowledge graph containing oral medicine teaching entities and semantic relationships based on the acquired data. A multimodal teaching resource semantic association engine is used to semantically annotate teaching resources with knowledge point nodes in the knowledge graph, and establish a closed-loop teaching link of knowledge point → explanation → case → exercise; A learning path generation engine is used to dynamically generate the optimal learning sequence based on the student profile and the semantic relationships in the knowledge graph. The operation and real-time feedback module is used to collect student operation parameters in the virtual environment, judge the standardization of operation, and provide feedback on the knowledge points associated in the knowledge graph based on the student operation data. The teaching feedback and reasoning path visualization module is used to generate a visualized reasoning path based on the knowledge graph and perform a difference comparison analysis when students diagnose errors. The dynamic evaluation and knowledge evolution module is used to optimize teaching effectiveness and knowledge system, track student behavior data to update student knowledge mastery model, and automatically update knowledge graph based on student and teacher feedback to achieve system self-evolution. The method for semantically annotating teaching resources with knowledge point nodes in the knowledge graph includes: For teaching videos in teaching resources, 3D convolutional neural networks are used to extract operation action features. Combined with speech recognition text, medical entities are extracted through the BiLSTM-CRF model and matched with knowledge graph entities by vector similarity. For 3D models in teaching resources, graph neural networks are used to model the topology of 3D meshes, extract semantic features of anatomical structures, and perform graph embedding alignment with anatomical entities in the knowledge graph. For typical cases in teaching resources, named entity recognition and relation extraction models are used to extract the chief complaint-diagnosis-treatment triple from the text and map it to the disease-symptom-treatment subgraph in the knowledge graph; For the exam question bank in the teaching resources, a semantic parser is used to convert the question text into logical expressions and match them with the concept-attribute-relationship structure in the knowledge graph.

2. The oral medicine teaching system fusing knowledge graph and multi-modal intelligent interaction according to claim 1, characterized in that, Also includes: The AI ​​virtual teaching assistant module is used to provide intelligent question answering and tutoring based on entities and semantic relationships in a knowledge graph.

3. The oral medicine teaching system of claim 1, wherein, The method for semantically annotating teaching resources with knowledge point nodes in the knowledge graph includes: A semantic consistency scoring function is introduced to quantify the matching degree between teaching resources and knowledge points, and a threshold is set to filter low-confidence labels. The semantic consistency scoring function is as follows: ， in, For teaching resources, These are knowledge point nodes in a knowledge graph. For resources The multimodal fusion embedding vector, For nodes Knowledge graph embedding, For cosine similarity, From resource-related entities to nodes The probability of the existence of the shortest semantic path in a knowledge graph. For context consistency score, , and These are the weighting coefficients.

4. The oral medicine teaching system integrating knowledge graph and multimodal intelligent interaction according to claim 2, characterized in that, The method for dynamically generating the optimal learning sequence based on the student profile and the semantic relationships in the knowledge graph includes: Construct student profile vectors; Extract the pre-dependencies of semantic relations in the knowledge graph to construct a directed acyclic graph; The learning path is defined and the objective function is optimized based on student profile vectors and directed acyclic graphs; Based on the learning path optimization objective function and path generation algorithm, the shortest path from the current mastery set to the target ability set is searched on the knowledge graph. Construct the optimal learning sequence based on the shortest path.

5. The oral medicine teaching system integrating knowledge graph and multimodal intelligent interaction according to claim 4, characterized in that, The heuristic function of the path generation algorithm is: ， in, For nodes To the target capability set The shortest path distance, This represents the priority coefficient for weak points. For knowledge mastery vectors, For nodes The probability of mastering it.

6. The oral medicine teaching system integrating knowledge graph and multimodal intelligent interaction according to claim 2, characterized in that, The method for collecting student operation parameters in a virtual environment, determining the standardization of operations, and providing feedback to the knowledge points associated with the knowledge graph based on student operation data includes: Collecting parameters in oral medicine; A local deviation index is defined based on the collected parameters to determine whether a violation has occurred at the current moment. Perform a global trajectory consistency test on the local deviation index; When the test result triggers feedback, query the normative principle nodes in the knowledge graph that are related to the current operation; Feedback on operations is provided based on node-based specification principles.

7. The oral medicine teaching system integrating knowledge graph and multimodal intelligent interaction according to claim 6, characterized in that, The local deviation index defined based on the acquired parameters includes: ， in, The angle between the tool axis and the long axis of the tooth. For cutting depth, For the spatial trajectory of the tool tip, , and These are standard reference values ​​for the operating procedures. , and To allow for the error range, , and These are the weighting coefficients. This is the local deviation index.

8. The oral medicine teaching system integrating knowledge graph and multimodal intelligent interaction according to claim 2, characterized in that, This is used to generate a visual reasoning path based on the knowledge graph and perform a difference comparison analysis when a student diagnoses an error, including: Define a diagnostic rationality scoring function to verify the logical consistency of the knowledge graph, thereby determining whether the student's diagnosis is incorrect; The rationality scoring function is as follows: ， in, This refers to the set of knowledge graph edges involved in the student's diagnostic path. The recommendation level for this relationship in clinical guidelines. The number of supporting documents or the level of evidence for this relationship. For specific diagnostic evidence or diagnostic elements, Score for reasonableness.

9. A teaching method for oral medicine that integrates knowledge graphs and multimodal intelligent interaction, characterized in that, Based on the system according to any one of claims 1 to 8, the method comprises: A knowledge graph containing oral medicine teaching entities and semantic relationships was constructed based on the acquired data; The teaching resources are semantically labeled with the knowledge point nodes in the knowledge graph to establish a closed-loop teaching link of knowledge point → explanation → case → exercise.

Images