Multi-modal zygomatic implant operation typing prediction method and device
By using a multimodal zygomatic implant surgery classification prediction method, which combines demographic and imaging information and utilizes deep neural networks for zygomatic implant surgery classification, the problem of high surgical sensitivity and low efficiency caused by reliance on experience in existing technologies is solved, and rapid and accurate classification guidance is achieved.
Patent Information
- Application Number
- CN202511088964.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2025-11-21
AI Technical Summary
In current technology, the classification of zygomatic implant surgery relies on the doctor's experience and lacks a rapid and accurate classification assistance system, resulting in high surgical sensitivity, low standardization, and low clinical efficiency.
A multimodal zygomatic implant surgery classification prediction method was adopted, which combines demographic information, zygomatic implant placement information and imaging information. Graph structure data was generated through deep neural networks, and graph attention networks were used to predict the classification of zygomatic implant surgeries.
It enables rapid and accurate classification of zygomatic bone transplantation surgery, guides doctors in clinical decision-making, and improves clinical diagnosis and treatment efficiency.
Smart Images
Figure CN120995305A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of oral medicine, and in particular to a method and device for multi-modal zygomatic implant surgery classification. BACKGROUND
[0002] The zygomatic implant technique provides a new idea for solving the reconstruction of severe atrophy of the maxilla. The use of zygomatic implant technique can avoid a series of problems such as large bone graft, long treatment cycle, and severe postoperative reaction of patients caused by traditional bone grafting method. Zygomatic implant technique also provides an effective immediate loading scheme for implant restoration of edentulous maxilla. However, due to the long surgical path of zygomatic implant and the proximity of multiple anatomical structures of zygomatic bone, maxilla and maxillary sinus, different surgical classifications correspond to different surgical schemes. For example, in the case of zygomatic implant passing through the maxillary sinus, maxillary sinus lateral wall windowing is essential; for patients with maxillary sinus depression, zygomatic implant socket preparation and zygomatic implant insertion can be directly performed. Early acquisition of zygomatic implant surgery classification is beneficial for the surgeon to choose the appropriate surgical auxiliary method, predict possible postoperative complications and take corresponding preventive measures, that is, to make appropriate surgical decisions.
[0003] At present, the most commonly used zygomatic implant surgery classification method is the ZAGA classification method, which classifies zygomatic implant surgery into 5 types according to the relationship between the zygomatic implant surgery path and the maxillary sinus and maxillary alveolar bone, namely ZAGA0, ZAGA1, ZAGA2, ZAGA3 and ZAGA4. From type 0 to type 4, the zygomatic implant path gradually changes from passing through the maxillary sinus to passing through the maxillary sinus outside. In ZAGA0, the zygomatic implant passes through the maxillary sinus without contacting the lateral wall of the maxillary sinus; in ZAGA1 and ZAGA2, the zygomatic implant passes through the lateral wall of the maxillary sinus; and in ZAGA3 and ZAGA4, the zygomatic implant passes through the maxillary sinus outside and has little relationship with the maxillary sinus.
[0004] At present, the classification of zygomatic implant surgery still highly depends on the experience of clinicians, and there is no system that can provide rapid and accurate classification to assist doctors in making appropriate clinical decisions. Doctors need to judge the degree of recession of the lateral wall of the maxillary sinus and the relationship between the zygomatic implant surgery path according to their own experience. Without computer-aided design system, the surgical decision of the doctor is often made during the operation, and the surgical plan is adjusted according to the intraoperative situation, which greatly increases the technical sensitivity of zygomatic implant surgery and reduces its standardization and generalizability. With the application of computer-aided surgical plan design system, doctors can virtually design zygomatic implants on preoperative CBCT and observe the classification of zygomatic implant surgery, but it still takes a certain amount of time to analyze the specific classification and for some special cases, the classification time will be greatly prolonged, increasing the time of clinical decision-making and causing a decline in clinical efficiency. SUMMARY
[0005] In view of the problems in the prior art, the present application provides a method for predicting the classification of zygomatic implant surgery, which comprises:
[0006] Obtaining text information and imaging information of a patient, wherein the text information comprises demographic information and zygomatic implant implantation information;
[0007] Generating text data encoding and image data encoding according to the text information and the imaging information, respectively;
[0008] Generating graph structure data according to the text data encoding and the image data encoding, sequentially passing the graph structure data through two graph attention networks, and generating a text feature vector and an image feature vector according to the output results, and splicing the text feature vector and the image feature vector to form training input data;
[0009] Training a zygomatic implant surgery classification prediction model using the training input data and labeled zygomatic implant surgery classification labels;
[0010] Performing zygomatic implant surgery classification prediction according to the zygomatic implant surgery classification prediction model.
[0011] Further, the demographic information comprises one or more of gender, age, and whether the maxilla is missing; and the zygomatic implant implantation information comprises one or more of the length of the zygomatic implant, the implanted tooth site, the contact area of the zygomatic implant with the zygomatic bone, the contact area of the zygomatic implant with the maxillary sinus, and the contact area of the zygomatic implant with the maxilla.
[0012] Further, the imaging information comprises an oral panoramic film and a CBCT image.
[0013] Further, the generating of the text data encoding and the image data encoding according to the text information and the imaging information comprises:
[0014] Generating panoramic film data encoding and CBCT data encoding according to the oral panoramic film and the CBCT image, respectively;
[0015] Splicing the panoramic film data encoding and the CBCT data encoding to generate the image data encoding.
[0016] Further, before generating the panoramic film data encoding, the oral panoramic film is set to a preset size, and normalized according to the mean and standard deviation of the three channels of RGB.
[0017] Further, before generating the CBCT data encoding, the CBCT image is set to a preset size, and the grayscale value thereof is normalized.
[0018] Further, the CBCT image adopts a CBCT slice containing bilateral malar bones.
[0019] Further, the malar implant surgery classification label is defined by the ZAGA classification method.
[0020] The application also provides a multi-modal malar implant surgery classification prediction device, comprising:
[0021] a processor; and
[0022] a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the operations of the above method.
[0023] The application also provides a computer readable medium storing instructions that, when executed, cause a system to perform the operations of the above method.
[0024] The application mainly solves the problem of malar implant surgery classification, and proposes a multi-modal malar implant surgery classification prediction method and device, which is based on a deep neural network, integrates patient demographic information, malar implant surgery path contact area, panoramic film and CBCT imaging information, realizes rapid classification of malar implant surgery, is conducive to guiding doctors to make rapid clinical decisions, improves clinical diagnosis and treatment efficiency, has significant social value and application value, and is worth popularization and application. BRIEF DESCRIPTION OF DRAWINGS
[0025] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the following drawings:
[0026] Figure 1 A flowchart of a multi-modal malar implant surgery classification prediction method of one embodiment of the application is shown;
[0027] Figure 2 An adjacency matrix constructed in the multi-modal malar implant surgery classification prediction method of one embodiment of the application is shown.
[0028] Figure 3 Functional modules of an exemplary system that can be used in embodiments of the application are shown.
[0029] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION
[0030] The application will be further described in detail below with reference to the drawings.
[0031] In one typical configuration of the present application, the terminal, the device of the service network and the trusted party each comprise one or more processors (e.g., Central Processing Units, CPUs), input / output interfaces, network interfaces and memories.
[0032] The memory can include non-persistent memory in computer-readable media, Random Access Memory (RAM), and / or non-volatile memory such as Read Only Memory (ROM) or Flash memory. The memory is an example of computer-readable media.
[0033] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), programmable random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device.
[0034] The device referred to in the present application includes but is not limited to a user device, a network device, or a device formed by integrating the user device and the network device through a network. The user device includes but is not limited to any kind of mobile electronic product capable of human-computer interaction with a user (for example, human-computer interaction through a touch panel), such as a smart phone, a tablet computer, etc., which can adopt any operating system, such as an Android operating system, an iOS operating system, etc. The network device includes an electronic device capable of automatically performing numerical calculation and information processing according to a pre-set or stored instruction, and the hardware thereof includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc. The network device includes but is not limited to a computer, a network host, a single network server, a plurality of network servers, or a cloud formed by a plurality of servers; here, the cloud is formed by a large number of computers or network servers based on cloud computing, wherein the cloud computing is a kind of distributed computing, and a virtual supercomputer formed by a group of loosely coupled computer clusters. The network includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network, etc. Preferably, the device can also be a program running on the user device, the network device, or the device formed by integrating the user device and the network device, the network device, a touch terminal, or the device formed by integrating the network device and the touch terminal through a network.
[0035] Of course, those skilled in the art should understand that the above-mentioned device is only an example, and other existing or future devices that can be applicable to the present application should also be included in the protection scope of the present application, and are hereby included by reference.
[0036] In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.
[0037] Figure 1 A flowchart of a method for predicting the classification of a multi-modal zygomatic implant surgery according to an embodiment of the present application is shown, which includes the following steps:
[0038] Step one, obtaining the text information and the imaging information of the patient, wherein the text information includes demographic information and zygomatic implant implantation information.
[0039] In this step, the demographic information includes one or more of the gender, age, and whether the maxilla is missing of the patient, and the zygomatic implant implantation information includes one or more of the length of the zygomatic implant, the implant tooth site, the contact area of the zygomatic implant with the zygomatic bone, the contact area of the zygomatic implant with the maxillary sinus, and the contact area of the zygomatic implant with the maxilla determined according to the patient's condition, and the demographic information and the zygomatic implant implantation information are taken together as text information, wherein the age, the length of the zygomatic implant, the contact area of the zygomatic implant with the zygomatic bone, the contact area of the zygomatic implant with the maxillary sinus, and the contact area of the zygomatic implant with the maxilla are continuous variables, and the gender, whether the maxilla is missing, and the implant tooth site of the zygomatic implant are classification variables.
[0040] For the contact areas of the zygomatic implant with the zygomatic bone, the maxillary sinus, and the maxilla, the calculation method in the prior art can be used, and preferably, the method in the Chinese invention patent application with the application number 202510546748X and the invention name "Method and device for calculating zygomatic implant multi-anatomical structure contact area" of the applicant's prior application can be used to calculate the contact areas of the zygomatic implant with the zygomatic bone, the maxillary sinus, and the maxilla.
[0041] The imaging information includes an oral panoramic film and a CBCT image, the oral panoramic film as two-dimensional imaging information, and the CBCT image as three-dimensional imaging information.
[0042] Step two, generating text data encoding and image data encoding respectively according to the text information and the imaging information.
[0043] In this step, the text information and the imaging information need to be encoded to generate text data encoding and image data encoding respectively, so as to become a form that can be efficiently processed by a computer.
[0044] In this embodiment, for the encoding of the text information, an encoder with 2 fully connected layers is used to generate the text data encoding from the text information.
[0045] For the encoding of the imaging information, panoramic film data encoding and CBCT data encoding are first generated respectively according to the oral panoramic film and the CBCT image, and then the panoramic film data encoding and the CBCT data encoding are spliced to generate the image data encoding.
[0046] Before encoding the oral panoramic film, it is first preprocessed, including setting it to a preset size and normalizing it. In this embodiment, the oral panoramic film is uniformly set to a picture with a size of 512x256, and the above size-adjusted oral panoramic film is normalized according to the mean and standard deviation of the three channels of RGB. For any pixel point in any channel, the calculation formula of the normalized pixel value is as follows:
[0047] Normalized pixel value = (original pixel value - channel mean) / channel standard deviation
[0048] Then the preprocessed panoramic film is encoded, in this embodiment, the preprocessed panoramic film is input into the ResNet50 network, and after passing through 4 layers of residual block groups and 1 1x1 convolution operation, the panoramic film data encoding is obtained.
[0049] Before encoding the CBCT image, it is also preprocessed, including setting the CBCT image to a predetermined size, and normalizing its gray value. In this embodiment, the size of each two-dimensional slice in the CBCT image is uniformly set to 256x256, and the gray value of the resized CBCT image is normalized to the range [600, 1600], the calculation formula is as follows:
[0050]
[0051] where y is the normalized value, x is the gray value of the current pixel point, x min is the minimum value in the original gray value, x max is the maximum value in the original gray value, [y min , y max ] is the target range of normalization, in this embodiment, y min = 600, y max = 1600.
[0052] Preferably, the CBCT image adopts a CBCT slice containing bilateral zygomatic bones, i.e. a CBCT slice containing bilateral zygomatic bones is selected from the source CBCT data, a corresponding classification model can be trained to identify the zygomatic bones in the CBCT slice, so as to select the CBCT slice containing bilateral zygomatic bones, such as the slice classification network based on VGG-16 and training weight adopted in the research result A deep learning-based automatic segmentation of zygomatic bones from cone-beam computed tomography images: A proof of concept of the inventor team of the present application, the CBCT slice is divided into a slice containing bilateral zygomatic bones and a slice not containing zygomatic bones, and the CBCT slice containing bilateral zygomatic bones is taken as the CBCT image. At the same time, considering the number of layers of the CBCT slice containing bilateral zygomatic bones and the computer processing memory requirement, the CBCT slice containing bilateral zygomatic bones is resampled to Bx256x256, where B≤50.
[0053] The pre-processed CBCT image is then encoded. In this embodiment, the pre-processed CBCT image is input into the encoder part of the 3D U-Net, which is used as a three-dimensional imaging information encoder. The encoder of the 3D U-Net comprises five blocks, each of which contains two 3D convolution operations, two ReLU activation functions and two batch normalization operations, i.e. a fixed sequence of convolution-normalization-ReLU-convolution-normalization-ReLU. Finally, the CBCT data encoding is obtained through the encoder of the 3D U-Net.
[0054] Finally, the panoramic film data encoding and the CBCT data encoding are spliced to generate the image data encoding. For example, for a 4-dimensional tensor of the panoramic film data encoding and a 5-dimensional tensor of the CBCT data encoding, the panoramic film data encoding is first expanded into a 5-dimensional tensor, and then spliced with the CBCT data encoding based on the dimensions, thereby obtaining a 5-dimensional tensor of the image data encoding.
[0055] In step three, the graph structure data generated from the text data encoding and the image data encoding is sequentially input into two graph attention networks, and text feature vectors and image feature vectors are generated based on the output results. The text feature vectors and the image feature vectors are spliced to form training input data.
[0056] In this step, M text nodes are determined based on the text data encoding, such as each item in the text information of the patient obtained in step one. N image nodes are determined based on the image data encoding, such as each two-dimensional slice of the CBCT image, thereby generating a node feature matrix and constructing an adjacency matrix. Figure 2 An adjacency matrix constructed in a specific embodiment is shown, which includes 8 text nodes determined by the file data encoding and 36 image nodes determined by the image data encoding.
[0057] The graph structure data composed of the node feature matrix and the adjacency matrix is input into two graph attention networks, i.e. a first graph attention network and a second graph attention network. The input data of the first graph attention network is the graph structure data, and the input data of the second graph attention network is the output result of the first graph attention network.
[0058] Preferably, in the first graph attention network, 8 attention heads are used, and the number of output channels is set to 64. In the second graph attention network, a single attention head is used, the number of input channels is set to 64, and the number of output channels is set to 128.
[0059] The output of the second graph attention network is split into text features and image features, corresponding to the aforementioned, i.e., into M text feature nodes and N image feature nodes. The image features split from the output of the second graph attention network are transposed, then pass through a global average pooling (GAP) layer to obtain an image feature vector, while the text features split from the output of the second graph attention network are a text feature vector, and the text feature vector and the image feature vector are spliced to form the training input data.
[0060] By using the graph attention network, the relationship between the text information and the image information can be comprehensively considered, the information of the two modalities is integrated, and the corresponding scaling is performed according to the relationship of the two modalities for outcome prediction. The encoding features of the nodes are adaptively converted and transmitted between the two modalities through the bidirectional edges and the graph attention mechanism. The text features and the image features jointly define the cross-modal adjacency relationship, and a dynamic adjacency matrix is constructed through a learnable attention weight matrix. Each graph attention layer performs linear transformation on the feature vectors of adjacent nodes through the learned weight matrix, and simultaneously calculates the attention coefficients between the current node and each neighbor, which represent the importance of the neighbor nodes to the current node. The transformed neighbor feature vectors are weighted and summed based on the attention coefficients to obtain the aggregated features. Finally, a LeakyReLU activation function is introduced to introduce nonlinearity, and an updated node representation is generated.
[0061] Step four, using the training input data and the labeled ZAGA classification labels to train a ZAGA classification prediction model.
[0062] In this step, the training input data and the corresponding ZAGA classification labels are used to train a ZAGA classification prediction model. For the ZAGA classification output in the training process, the cross-entropy loss is calculated with the ZAGA classification label, the model parameters are optimized according to the loss, and finally the model training is completed. In this embodiment, the ZAGA classification label is defined by the ZAGA classification method, i.e., the ZAGA classification includes ZAGA0, ZAGA1, ZAGA2, ZAGA3, and ZAGA4.
[0063] Step five, ZAGA classification prediction according to the ZAGA classification prediction model.
[0064] After obtaining the ZAGA classification prediction model, only the preoperative CBCT image and the oral panoramic film of the patient, as well as the demographic information and the zygomatic implant implantation information, need to be input into the ZAGA classification prediction model to obtain the ZAGA classification prediction result, thereby completing the rapid classification of the zygomatic implant surgery.
[0065] The present embodiments also provide a computer-readable storage medium storing computer code that, when executed, causes the method of any preceding embodiment to be performed.
[0066] The present embodiments also provide a computer program product that, when executed by a computer device, causes the method of any preceding embodiment to be performed.
[0067] The present embodiments also provide a computer device comprising:
[0068] one or more processors;
[0069] a memory for storing one or more computer programs;
[0070] when the one or more computer programs are executed by the one or more processors, cause the one or more processors to implement the method of any preceding embodiment.
[0071] Figure 3 An example system that can be used to implement various embodiments described in the present application is shown.
[0072] As Figure 3 shown, in some embodiments, system 1000 can function as a user terminal device in any of the described embodiments. In some embodiments, system 1000 can include one or more computer-readable media (e.g., system memory or NVM / storage 1020) having instructions and one or more processors (e.g., processor(s) 1005) coupled with the one or more computer-readable media and configured to execute the instructions to implement modules to perform the actions described in the present application.
[0073] For one embodiment, system control module 1010 can include any suitable interface controllers to provide for any suitable interface to at least one of the processor(s) 1005 and / or any suitable device or component in communication with system control module 1010.
[0074] System control module 1010 can include a memory controller module 1030 to provide an interface to system memory 1015. Memory controller module 1030 can be a hardware module, a software module, and / or a firmware module.
[0075] System memory 1015 can be used to load and store data and / or instructions, for example, for system 1000. For one embodiment, system memory 1015 can include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 1015 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).
[0076] For one embodiment, system control module 1010 can include one or more input / output (I / O) controllers to provide an interface to NVM / storage 1020 and communication interface(s) 1025.
[0077] For example, NVM / storage 1020 can be used to store data and / or instructions. NVM / storage 1020 can include any suitable non-volatile memory (e.g., flash memory) and / or can include any suitable non-volatile storage device(s) (e.g., one or more Hard Disk, HDDs, one or more Compact Disc (CD) drives, and / or one or more Digital Versatile Disc (DVD) drives).
[0078] NVM / storage 1020 can include a storage resource that is physically part of a device on which system 1000 is installed or that is accessed via the device but is not physically part of the device. For example, NVM / storage 1020 can be accessed via communication interface(s) 1025 through a network.
[0079] Communication interface(s) 1025 can provide an interface for system 1000 to communicate with one or more networks and / or to communicate with any other suitable device. System 1000 can wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols.
[0080] For one embodiment, at least one of processor(s) 1005 can be logically encapsulated with one or more controllers of system control module 1010, such as memory controller module 1030. For one embodiment, at least one of processor(s) 1005 can be logically encapsulated with one or more controllers of system control module 1010 to form a system-in-a-package (SiP). For one embodiment, at least one of processor(s) 1005 can be logically integrated on the same die with one or more controllers of system control module 1010. For one embodiment, at least one of processor(s) 1005 can be logically integrated on the same die with one or more controllers of system control module 1010 to form a system-on-a-chip (SoC).
[0081] In various embodiments, system 1000 can be, but is not limited to, a server, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, system 1000 can have more or fewer components, and / or different architectures. For example, in some embodiments, system 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including touch screen displays), non- volatile memory ports, multiple antennas, graphics chips, application specific integrated circuits (ASICs), and speakers.
[0082] It is noted that the present application can be implemented in software and / or in a combination of software and hardware, e.g., using application specific integrated circuits (ASIC), a general purpose computer or any other similar hardware devices. In one embodiment, the software program of the present application is implemented by the processor so as to perform the steps or functions described above. Also, the software program of the present application (including related data structures) can be stored in a computer readable storage medium, e.g., a RAM memory, a magnetic or optical drive or diskette, and the like. Additionally, some of the steps or functions can be implemented in hardware using, for example, circuitry dedicated to performing the steps or functions.
[0083] In addition, a part of the present application can be applied as a computer program product, such as computer program instructions, when executed by a computer, through the operation of the computer, can invoke or provide a method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source files, executable files, installation package files and the like, and accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.
[0084] Communication media includes wired and wireless media including, for example, computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics changed or set in a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as twisted pair, coaxial, fiber optic, wave guides, and other wired or wired media and wireless media such as acoustic, RF, infrared, other wireless media. Combinations of the above should also be included within the scope of computer readable media.
[0085] By way of example, and not limitation, computer readable storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. For example, computer readable storage media includes, but is not limited to, RAM, such as SRAM, DRAM, or other types of random access memory; and non-volatile memory, such as flash, various types of read-only memories (ROM), electrically programmable (EPROM), electrically erasable (EEPROM), programmable ROM (PROM), programmable read-only memory (PROM), erasable programmable read only memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM), and magnetic and optical storage devices (hard disk, magnetic tape, CD, DVD). Other combinations of the above are also included within the scope of computer readable storage media.
[0086] Herein, according to one embodiment of the present application comprises a device, the device comprises a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, trigger the device to run the method and / or technical solutions based on the foregoing according to the plurality of embodiments of the present application.
[0087] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments, but that the application can be implemented in other concrete forms without deviating from the spirit or the basic characteristics of the application. The embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being defined by the appended claims rather than by the above Description, which is therefore intended merely as explanatory and not as restrictive. No reference signs in the claims should be considered as limiting the scope of the claims in which the reference is made. The word 'comprising' does not exclude other elements or steps, and the singular does not exclude the plural and vice versa, unless the context clearly requires these exclusions. The claim of a device reciting a plurality of means or units can also refer to one single unit or to one single device if the plurality are implemented by one single unit or device. The terms first, second and the like do not denote any ordering, but rather serve as names for naming different components.
Claims
1. A method for predicting the subtype of multimodal zygomatic implant surgery, characterized in that, The method comprises: obtaining text information and imaging information of a patient, the text information comprising demographic information and zygomatic implant implantation information; generating text data encoding and image data encoding according to the text information and the imaging information respectively; generating graph structure data according to the text data encoding and the image data encoding, sequentially passing the graph structure data through two graph attention networks, and generating a text feature vector and an image feature vector according to output results, and forming training input data by splicing the text feature vector and the image feature vector; training a zygomatic implant surgery classification prediction model using the training input data and labeled zygomatic implant surgery classification labels; performing zygomatic implant surgery classification prediction according to the zygomatic implant surgery classification prediction model.
2. The method of claim 1, wherein, The demographic information comprises one or more of gender, age, and whether the maxilla is missing; and the zygomatic implant implantation information comprises one or more of the length of the zygomatic implant, the implanted tooth site, the contact area of the zygomatic implant with the zygomatic bone, the contact area of the zygomatic implant with the maxillary sinus, and the contact area of the zygomatic implant with the maxilla.
3. The method of claim 1, wherein, The imaging information comprises an oral panoramic film and a CBCT image.
4. The method of claim 3, wherein, The method comprises: generating panoramic film data encoding and CBCT data encoding according to the oral panoramic film and the CBCT image respectively; splicing the panoramic film data encoding and the CBCT data encoding to generate the image data encoding.
5. The method of claim 4, wherein, Before generating the panoramic film data encoding, the oral panoramic film is set to a preset size, and normalized according to the mean and standard deviation of the RGB three channels.
6. The method of claim 4, wherein, Before generating the CBCT data encoding, the CBCT image is set to a preset size, and the grayscale value thereof is normalized.
7. The method of claim 6, wherein, The CBCT image uses a CBCT slice containing bilateral zygomatic bones.
8. The method of claim 1, wherein, The zygomatic implant surgery classification labels are defined by the ZAGA classification method.
9. A device for multi-modal zygomatic implant surgery typing prediction, wherein, The device comprises: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform operations of the method according to any one of claims 1 to 8.
10. A computer readable medium storing instructions that, when executed, cause a system to perform operations of the method according to any one of claims 1 to 8.