Methods and electronic devices for constructing oral health knowledge graphs
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNIV SCHOOL OF STOMATOLOGY
- Filing Date
- 2026-04-21
- Publication Date
- 2026-05-26
Smart Images
Figure CN122088656A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of oral data technology, and in particular relates to a method for constructing an oral health knowledge graph and an electronic device. Background Technology
[0002] Clinical data in oral medicine is highly heterogeneous and multi-sourced, including unstructured handwritten medical records, structured examination data, 3D CBCT and intraoral scans, as well as emerging genetic testing reports and patient wearable device data. Constructing a comprehensive and accurate oral health knowledge graph is of great significance for achieving precision medicine, supporting clinical decision-making, and promoting scientific research.
[0003] Currently, some research has attempted to integrate medical information using knowledge graph technology. Existing technologies typically employ the following approaches: Single-modal information extraction: This mainly targets structured or semi-structured data (such as structured fields in electronic medical records) for information extraction, but it is insufficient for unstructured text, images, and other data.
[0004] Simple data concatenation: This involves simple field associations between data from different sources (such as through patient ID or tooth position code FDI), but it lacks a deep understanding and association mining of multimodal data at the semantic level.
[0005] Static knowledge base: The constructed knowledge graph is mostly static, and updates rely on manual maintenance. It cannot be dynamically optimized and evolved based on new data and expert feedback.
[0006] Lack of deep reasoning: Existing systems mostly remain at the level of entity and relationship identification, lacking the ability to combine domain ontology and multi-source evidence for complex cognitive reasoning.
[0007] These existing technical solutions have obvious drawbacks: they cannot effectively integrate the deep semantic information of multimodal data, resulting in insufficient accuracy and completeness of knowledge extraction; the constructed knowledge graphs have a low degree of "intelligence" and are difficult to discover implicit relationships between data; and the system lacks self-evolution capabilities, making it difficult to continuously improve the quality of knowledge. Summary of the Invention
[0008] To address the problems existing in the prior art, this invention provides a method and electronic device for constructing an oral health knowledge graph, which at least partially solves the problems of low "intelligence" level, lack of self-evolution ability, and difficulty in continuously improving knowledge quality in the existing technologies.
[0009] In a first aspect, embodiments of this disclosure provide a method for constructing an oral health knowledge graph, including: After preprocessing the acquired multimodal heterogeneous data, deep fusion and alignment are performed in a unified semantic space to generate deep semantic representation vectors with cross-modal correlation. Identify entities and relationships from deep semantic representation vectors to generate structured triples; Structured triples that meet the confidence criteria are selected, an initial knowledge graph is constructed based on the selected structured triples, and the confidence of the structured triples is dynamically evaluated based on multi-source feedback, thereby achieving continuous evolution of the knowledge graph.
[0010] Optionally, the step of preprocessing the acquired multimodal heterogeneous data and then performing deep fusion and alignment in a unified semantic space includes: Feature extraction is performed on each mode in the acquired multimodal data to obtain a high-dimensional vector; Project the high-dimensional vectors of all obtained modalities onto a unified dimensional space; Construct a cross-modal attention matrix in a unified dimensional space and calculate the association weights between each pair of modal elements; Based on the obtained correlation weights between each pair of modal elements, a weighted summation method is used to fuse multimodal information and generate a fusion vector; Based on the fusion vector, a multimodal joint embedding vector in a unified semantic space is obtained, thereby obtaining a deep semantic representation vector, and the initial confidence of each modality's contribution to the deep semantic representation vector is calculated.
[0011] Optionally, the weighted summation method includes: The modal importance coefficient is calculated based on a learnable modal importance coefficient, which is automatically adjusted through a gating mechanism.
[0012] Optionally, calculating the association weight between each pair of modal elements includes: The semantic association score between each pair of modal elements is calculated using the following formula: , in, This is the semantic vector of the text segment. The feature vector of a certain ROI in the image. and The projection matrix is learnable. Attention scaling factor This represents the degree of semantic matching from text modality to image modality.
[0013] Optionally, the step of identifying entities and relations from deep semantic representation vectors to generate structured triples includes: Constructing an ontology for the oral cavity domain; Define semantic relation types; The deep semantic representation vector is semantically aligned with the ontology using a cross-modal semantic alignment bridge; Based on semantic alignment results and semantic relationship types, a multi-engine cognitive reasoning engine is used to generate structured knowledge triples.
[0014] Optionally, the step of semantically aligning the deep semantic representation vector with the ontology via a cross-modal semantic alignment bridge includes: Coarse-grained semantic mapping and fine-grained attribute binding; The coarse-grained semantic mapping includes using an ontology-aware semantic similarity matcher to compare the deep semantic representation vector with all candidate entities in the ontology; The fine-grained attribute binding includes checking whether the entity attribute slots can be supported by multimodal data for each candidate entity.
[0015] Optionally, the step of identifying entities and relations from deep semantic representation vectors to generate structured triples includes: A semantic consistency scoring function is constructed to quantify the degree of attribute matching between candidate entities and multimodal data. The semantic consistency scoring function is as follows: , in, and These are the weighting coefficients. For entities The set of attributes, These are the conditions that an attribute must satisfy. The support of multimodal data for attribute a, For entities The semantic similarity between vector representations, This is a deep semantic representation vector.
[0016] Optionally, the dynamic evaluation of the confidence level of the structured triple based on multi-source feedback includes: The confidence level is dynamically evaluated based on a multi-source weighted confidence scoring model. The formula for the scoring model is as follows: , Where M is the set of modes that support the triple. The reliability weights for mode m, Let m be the support strength of mode m for the triplet. For the reliability of the reasoning path, Rate the quality of the data source. and For hyperparameters, For multimodal support, For reasoning and data quality.
[0017] Optionally, the dynamic evaluation of the confidence level of the structured triple based on multi-source feedback includes: The received feedback information is transformed into a reinforcement learning signal based on a loss function, wherein the loss function is: , in, For structured triples For dynamic confidence level, For feedback information.
[0018] Secondly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the oral health knowledge graph construction method described in any of the first aspects.
[0019] The present invention provides a method and electronic device for constructing an oral health knowledge graph. The method for constructing an oral health knowledge graph maps multimodal data to a unified semantic space and dynamically evaluates the confidence of structured triples based on multi-source feedback, thereby enabling the constructed knowledge graph to self-evolve and achieve the purpose of improving the intelligence of the constructed knowledge graph and continuously improving the quality of the knowledge graph. Attached Figure Description
[0020] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.
[0021] Figure 1 A flowchart of the method for constructing an oral health knowledge graph provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of multimodal collaborative sensing provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the cognitive reasoning engine structure provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of a bidirectional feedback closed loop provided in an embodiment of this disclosure; Figure 5 A schematic block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0022] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0023] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0024] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.
[0025] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0026] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.
[0027] like Figure 1 As shown, this embodiment discloses a method for constructing an oral health knowledge graph, including: After preprocessing the acquired multimodal heterogeneous data, deep fusion and alignment are performed in a unified semantic space to generate deep semantic representation vectors with cross-modal correlation. Identify entities and relationships from deep semantic representation vectors to generate structured triples; Structured triples that meet the confidence criteria are selected, an initial knowledge graph is constructed based on the selected structured triples, and the confidence of the structured triples is dynamically evaluated based on multi-source feedback, thereby achieving continuous evolution of the knowledge graph.
[0028] The specific plan is as follows: 1. Multimodal data access and preprocessing: Used to access and preprocess heterogeneous data from different sources.
[0029] Specifically, this involves accessing electronic medical records (EMR), CBCT images, intraoral 3D scan models, gene sequencing data, patient follow-up voice / text, and wearable device data.
[0030] It performs word segmentation and named entity recognition (NER) on text data; automatic segmentation and feature extraction on image data based on deep learning (such as 3D-UNet); bioinformatics analysis on gene data; and point cloud / mesh processing on 3D models.
[0031] 2. Multimodal collaborative perception and feature encoding, such as Figure 2 As shown, this method enables deep fusion and alignment of multimodal clinical data, such as text, images, genes, and 3D models, in a unified semantic space, generating deep semantic representation vectors with cross-modal relevance.
[0032] Implementation details: A customized multimodal pre-training architecture (OralMM-PT) for the oral cavity is employed, with its core being a feature fusion model based on a cross-modal attention mechanism. The input consists of four classes of modal data: Text modality: Such as electronic medical record descriptions; Image modality: Such as CBCT slice sequences; Genetic modality: Such as SNP site expression vectors; 3D structural modes: For example, embedding vectors into point clouds.
[0033] Each modality is encoded by an independent encoder (such as BERT, 3D-ResNet, Transformer-Gene, PointNet++) to obtain the initial feature representation: Then, the cross-modal attention fusion layer is entered, using text as the query and other modalities as the key and value, to perform bidirectional semantic alignment.
[0034] Formula for calculating cross-modal correlation: Define the distance from text t to image region r The semantic relevance score of I is: , in, This is the semantic vector of the text segment. This is the feature vector of a specific ROI (Region of Interest) in the image. and The projection matrix is learnable. Attention scaling factor This represents the degree of semantic matching from text modality to image modality.
[0035] Similarly, the association between genetic traits and clinical symptoms can be defined: , in: : Gene variation embedding vector; Text encoding of clinical symptoms (such as "bleeding gums"); The Sigmoid function outputs a confidence level in the range [0,1]. : Indicates the strength of the potential biological association between a gene mutation and a symptom; , A learnable transformation matrix for mapping genes and symptoms to a shared semantic space; For bias terms; This is a dot product operation, used to convert... The intermediate vector is compressed into a scalar.
[0036] The calculation steps are as follows: Feature extraction is performed on each modality to obtain a high-dimensional vector; Project all modal vectors onto a unified dimensional space; Construct a cross-modal attention matrix and calculate the association weights between each pair of modal elements; Multimodal information is fused using a weighted summation method to generate a fused vector: , in, These are learnable modal importance coefficients, automatically adjusted through a gating mechanism. For this is the first The original feature representation of each modality.
[0037] The final output is a multimodal joint embedding vector z∈RD in the unified semantic space, along with the initial confidence score of each modality contribution. ∈[0,1], used for subsequent knowledge confidence evaluation. The joint embedding vector z is the deep semantic representation vector.
[0038] 3. Oral Cognitive Reasoning and Knowledge Fusion: Based on oral medicine ontology and multi-engine reasoning mechanism, entities and relationships are identified from fused features, structured triples are generated, and semantic accuracy is ensured.
[0039] Specific implementation: (1) Constructing an oral domain ontology, OralCog-Ontology Define the core entity categories: Anatomical structures: FDI (Front Dental Diagnosis), periodontal ligament, dental pulp, etc. Diseases: pulpitis, periodontitis, caries classification, etc.; Symptoms: pain, swelling, and looseness; Treatment: root canal treatment, implant placement, orthodontic treatment plan; Genes: Susceptibility genes such as IL1B, TNF-α, and MMP9; Risk factors: smoking, diabetes, oral hygiene habits.
[0040] (2) Define semantic relation types: The following are some common syntaxes: has_location (located in), caused_by (caused by), treated_with (treated with), associated_with (associated with), and predisposes_to (susceptible to).
[0041] The three reasoning paradigms are integrated, as shown in Table 1. Table 1. Explanation of Reasoning Paradigms The meaning of each parameter in the neural network inference formula in the table is based on existing technology and will not be elaborated here.
[0042] (3) Cross-modal semantic alignment bridge: This embodiment proposes a dual-stage semantic alignment bridging mechanism (DS-SAB): The steps are as follows: Phase 1: Coarse-grained semantic mapping (Concept Matching): Using an ontology-aware semantic similarity matcher, the fused vector z is compared with all candidate entities in OralCog-Ontology: in For the entity The pre-trained embeddings (fine-tuned using BioBERT) are used. The top K candidate entities with the highest similarity are selected as the initial matching results.
[0043] Phase Two: Fine-grained Attribute Binding For each candidate entity Check whether its attribute slots can be supported by multimodal data: For example, if z contains "low-density shadow on the mesial proximal surface of the left mandibular first molar", then verification is required: Does Tooth(36) exist? Does the Caries entity exist? Does it satisfy \text{has_location}(Caries, 36) \land \text{has_aspect}(Caries, \text{mesial})? By constructing a semantic consistency scoring function: , in: and These are weighting coefficients, which can be learned. For entities The set of attributes, These are the conditions that an attribute must satisfy. The support of multimodal data ∈[0,1] for attribute a (output by attention weights or classifier). For entities The semantic similarity between vector representations, This is a deep semantic representation vector.
[0044] Final choice The highest-ranking entity is used as the final mapping result.
[0045] 4. Dynamic Knowledge Graph Construction and Evolution: (1) Initial knowledge graph construction: The high-confidence triples (h, r, t) output by the cognitive reasoning engine are stored in a graph database (such as Neo4j) to form an initial knowledge graph K0. The cognitive reasoning engine, such as... Figure 3 As shown.
[0046] (2) Dynamic assessment of knowledge confidence: Dynamic confidence assessment formula: Design a multi-source confidence scoring model (MSCS): For each triple (h,r,t), its dynamic confidence C(h,r,t) is defined as: , Where M is the set of modes that support the triple. The reliability weights for mode m (which can be learned) The support strength of modality m for the triple (such as attention weight or classification probability). To ensure the reliability of the inference path, the rule chain length or the message passing strength of the GNN is considered. Scoring of data source quality (such as hospital level, equipment accuracy, and labeling consistency). and These are adjustable hyperparameters or learning parameters. For multimodal support, For reasoning and data quality.
[0047] For the first time, the strength of multimodal data, the reliability of inference logic, and the quality of data sources are uniformly quantified to form an interpretable confidence index.
[0048] (3) "Doctor-System" two-way feedback closed loop: like Figure 4 As shown, doctors can review triples through a visual interface and perform the following actions: Confirmed (+1); Correct (modify entities or relationships); Reject (-1); The system transforms feedback into reinforcement learning signals based on a loss function, which is: Where y=1 indicates doctor's confirmation, and y=0 indicates rejection.
[0049] Fine-tuning using this loss function: attention weights in multimodal deep fusion; rule confidence thresholds in cognitive inference engines; and confidence evaluation models. , , Parameters. This enables the continuous evolution of the knowledge graph and model self-optimization.
[0050] Personalized knowledge graphs can be generated based on the above technical solutions.
[0051] Personalized knowledge graph generation and application specifically include: Personalized knowledge graphs primarily address how to identify entities and relationships from fused features, and which relationships to identify.
[0052] Entity recognition process: Input: Patient ID or chief complaint (e.g., "Right lower posterior tooth pain for 3 days"); The system retrieves its multimodal data and generates a fusion vector z through the aforementioned modules; Mapping to ontology entities via the DS-SAB bridge, such as: Tooth(46), Symptom(Pain), Finding(PeriapicalRadiolucency); Relationship identification process: Multi-engine inference using the CRE engine: Rule-based reasoning: If "exposed pulp + pain" →\text{caused_by}(Pain, \text{Irreversible Pulpitis}); GNN inference: If Tooth(46)Tooth(46)andDiabetesDiabetes have an indirect path in the graph →introduce\text{predisposes_to}(Diabetes, \text{Periodontitis}); Steps to generate a personalized subgraph: Input parsing: Receive user requests (such as "Analysis of periodontitis risk factors in patient P00123"); Context extraction: Obtain the patient's multimodal data and generate a fusion representation z; Seed node determination: Core entities (such as Periodontitis, Tooth(34)) are determined through a semantic alignment bridge. 47) Periodontitis, Tooth (34) 47)); Subgraph expansion: Set the extension depth d (default 2 jumps); Perform breadth-first search (BFS) or PageRank-based importance pruning; Retain triples with confidence level C > τ (threshold); Subgraph optimization: Remove redundant paths; Highlight key risk factors (such as smoking, IL1B positivity); Output format: JSON-LD (for API calls), and a visualized force-directed graph (for doctors to view). The specific method of this embodiment is as follows: Acquire and preprocess multi-source heterogeneous oral data (EMR, CBCT, genetics, follow-up, etc.); The multimodal collaborative perception module generates a fused semantic vector z using a cross-modal attention mechanism and calculates the correlation between each modality. Through the cognitive reasoning and knowledge fusion module: Use the DS-SAB bridge to map z to entities in OralCog-Ontology; Call the CRE engine to identify relationships between entities (such as has_location, caused_by, associated_with) and generate candidate triples; Through the dynamic evolution module: Calculate the dynamic confidence level C(h,r,t) for each triple; Construct an initial knowledge graph; Initiate a "doctor-system" feedback loop to collect expert feedback and update model parameters; Based on user requests, personalized knowledge subgraphs are generated to serve scenarios such as clinical decision-making and scientific research analysis.
[0053] Compared with the prior art, this embodiment has the following advantages: Deep integration enhances accuracy: Through multimodal collaborative perception, deep semantic fusion and mutual verification of data such as text, images, and genes are achieved, which significantly improves the accuracy and robustness of knowledge extraction and solves the problem of incomplete or erroneous information in a single modality.
[0054] Intelligent reasoning to uncover tacit knowledge: By introducing multi-engine cognitive reasoning, not only can explicit knowledge be identified, but also complex and implicit relationships between data (such as gene-environment-disease interactions) can be uncovered, thereby improving the "intelligence" level and scientific research value of knowledge graphs.
[0055] Dynamic evolution ensures the freshness of knowledge: The "doctor-system" two-way feedback loop enables the knowledge graph to have the ability to learn and evolve continuously, and the quality of knowledge can be continuously improved, solving the problem of outdated static knowledge bases.
[0056] Quantitative credibility aids decision-making: Dynamic knowledge confidence assessment provides a credibility reference for each knowledge unit, enabling doctors to make decisions based on evidence when using the knowledge graph, thus improving the reliability and usability of the system.
[0057] Flexible applications for precision medicine: It supports the generation of personalized knowledge graphs, which can better meet the diverse and precise needs of clinical practice, scientific research and patient education.
[0058] In addition, the oral health knowledge graph construction system disclosed in this embodiment includes: Multimodal data access and preprocessing module: responsible for accessing and preprocessing heterogeneous data from different sources; Multimodal collaborative perception and feature encoding module: used to achieve deep fusion and alignment of multimodal clinical data such as text, images, genes, and 3D models in a unified semantic space, and generate deep semantic representation vectors with cross-modal correlation.
[0059] Oral Cognitive Reasoning and Knowledge Fusion Module: Based on oral medicine ontology and multi-engine reasoning mechanism, it identifies entities and relationships from fused features, generates structured triples, and ensures accurate semantic mapping.
[0060] Dynamic Knowledge Graph Construction and Evolution Module: Used to achieve continuous evolution of knowledge graphs and model self-optimization.
[0061] The personalized knowledge graph generation and application module is used to generate personalized knowledge graphs based on user requests by calling the multimodal data access and preprocessing module, the multimodal collaborative perception and feature encoding module, the oral cognitive reasoning and knowledge fusion module, and the dynamic knowledge graph construction and evolution module.
[0062] The electronic device disclosed in this embodiment includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.
[0063] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the electronic device to perform all or part of the steps of the oral health knowledge graph construction method of the foregoing embodiments of this disclosure.
[0064] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.
[0065] like Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 5The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0066] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.
[0067] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to exchange data wirelessly or via wired communication with other devices, such as edge computing devices. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0068] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the oral health knowledge graph construction method of embodiments of this disclosure are performed.
[0069] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.
[0070] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the oral health knowledge graph construction method described in the foregoing embodiments of the present disclosure are performed.
[0071] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).
[0072] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.
[0073] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0074] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.
[0075] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.
[0076] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.
[0077] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.
[0078] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.
[0079] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.
Claims
1. An oral health knowledge graph construction method, characterized in that, include: After preprocessing the acquired multimodal heterogeneous data, deep fusion and alignment are performed in a unified semantic space to generate deep semantic representation vectors with cross-modal correlation. Identify entities and relationships from deep semantic representation vectors to generate structured triples; Structured triples that meet the confidence criteria are selected, an initial knowledge graph is constructed based on the selected structured triples, and the confidence of the structured triples is dynamically evaluated based on multi-source feedback, thereby achieving continuous evolution of the knowledge graph.
2. The method of claim 1, wherein, The process of preprocessing the acquired multimodal heterogeneous data and then performing deep fusion and alignment in a unified semantic space includes: Feature extraction is performed on each mode in the acquired multimodal data to obtain a high-dimensional vector; Project the high-dimensional vectors of all obtained modalities onto a unified dimensional space; Construct a cross-modal attention matrix in a unified dimensional space and calculate the association weights between each pair of modal elements; Based on the obtained correlation weights between each pair of modal elements, a weighted summation method is used to fuse multimodal information and generate a fusion vector; Based on the fusion vector, a multimodal joint embedding vector in a unified semantic space is obtained, thereby obtaining a deep semantic representation vector, and the initial confidence of each modality's contribution to the deep semantic representation vector is calculated.
3. The method of claim 2, wherein, The weighted summation method includes: The modal importance coefficient is calculated based on a learnable modal importance coefficient, which is automatically adjusted through a gating mechanism.
4. The method of claim 2, wherein, The calculation of the association weight between each pair of modal elements includes: The semantic association score between each pair of modal elements is calculated using the following formula: ， wherein, is a semantic vector of a text segment, is a feature vector of a ROI in an image, and is a learnable projection matrix, is an attention scaling factor, is a semantic matching degree from a text modality to an image modality, R is a real number set, is a projected image region feature, is a feature vector of the projected image region feature.
5. The method of claim 1, wherein, The process of identifying entities and relations from deep semantic representation vectors to generate structured triples includes: Constructing an ontology for the oral cavity domain; Define semantic relation types; The deep semantic representation vector is semantically aligned with the ontology using a cross-modal semantic alignment bridge; Based on semantic alignment results and semantic relationship types, a multi-engine cognitive reasoning engine is used to generate structured knowledge triples.
6. The method of claim 5, wherein, The step of semantically aligning the deep semantic representation vector with the ontology using a cross-modal semantic alignment bridge includes: Coarse-grained semantic mapping and fine-grained attribute binding; The coarse-grained semantic mapping includes using an ontology-aware semantic similarity matcher to compare the deep semantic representation vector with all candidate entities in the ontology; The fine-grained attribute binding includes checking whether the entity attribute slots can be supported by multimodal data for each candidate entity.
7. The method for constructing an oral health knowledge graph according to claim 6, characterized in that, The process of identifying entities and relations from deep semantic representation vectors to generate structured triples includes: A semantic consistency scoring function is constructed to quantify the degree of attribute matching between candidate entities and multimodal data. The semantic consistency scoring function is as follows: ， in, and These are the weighting coefficients. For entities The set of attributes, These are the conditions that an attribute must satisfy. The support of multimodal data for attribute a, For entities The semantic similarity between vector representations, This is a deep semantic representation vector.
8. The method for constructing an oral health knowledge graph according to claim 7, characterized in that, The dynamic evaluation of the confidence level of structured triples based on multi-source feedback includes: The confidence level is dynamically evaluated based on a multi-source weighted confidence scoring model. The formula for the scoring model is as follows: ， Where M is the set of modes that support the triple. The reliability weights for mode m, Let m be the support strength of mode m for the triplet. For the reliability of the reasoning path, Rate the quality of the data source. and For hyperparameters, For multimodal support, For reasoning and data quality.
9. The method for constructing an oral health knowledge graph according to claim 8, characterized in that, The dynamic evaluation of the confidence level of structured triples based on multi-source feedback includes: The received feedback information is transformed into a reinforcement learning signal based on a loss function, wherein the loss function is: ， in, For structured triples For dynamic confidence level, For feedback information.
10. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the oral health knowledge graph construction method according to any one of claims 1-9.