Oral and maxillofacial surgery image recognition and diagnosis method and system based on deep learning

Abstract

The invention relates to the technical field of medical image processing and artificial intelligence diagnosis, in particular to an oral and maxillofacial surgery image recognition and diagnosis method and system based on deep learning, and the method comprises the following steps: multi-modal image collection and cooperative preprocessing: collecting an oral and maxillofacial surgery CBCT image, a cone beam CT curved surface tomography image, an oral endoscope image and an ultrasonic image, a standardized multi-modal image data set is obtained through inter-modal registration and an adaptive enhancement algorithm; according to the method, a traditional diagnosis framework of'single-mode image + manual film reading 'is broken through, and a three-order diagnosis logic of'multi-mode image cooperative enhancement-cross-scale feature dynamic fusion-focus typing and risk hierarchical linkage' is innovatively provided; and accurate identification, typing and malignant transformation risk prediction of common oral and maxillofacial surgery diseases (such as jaw cyst, wisdom tooth impediment, temporomandibular joint disorder and maxillofacial tumor) are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oral medical image processing and artificial intelligence diagnosis, in particular to an oral and maxillofacial surgery image recognition and diagnosis method and system based on deep learning. BACKGROUND

[0002] The diagnosis of oral and maxillofacial surgery diseases highly depends on image examination, but the traditional diagnosis method has the following significant technical defects:

[0003] Poor adaptability of multi-modal images: CBCT, curved tomography, endoscopy, ultrasound and other multi-modal images are commonly used in clinical practice, but the data formats, resolutions and imaging principles of each modality are quite different (such as CBCT is a three-dimensional tomographic image, and endoscopy is a two-dimensional mucosal surface image), and traditional methods can only analyze single modal images individually, cannot realize collaborative diagnosis between modalities, and lead to missing of lesion information (such as CBCT alone cannot identify early mucosal cancer, and endoscopy alone cannot determine the depth of lesion infiltration).

[0004] Difficulties in recognizing lesion boundaries and features: the oral and maxillofacial region has complex anatomical structures (teeth, jaw bones, nerves and blood vessels are densely distributed), and the gray level difference between part of the lesions (such as early jaw cysts and small tumors) and normal tissues is small, and the edges are blurred, so it is difficult for traditional manual film reading or simple image segmentation algorithms (such as threshold segmentation) to accurately locate the lesions, and misdiagnosis (such as misjudging inflammatory hyperplasia as tumor) or missed diagnosis (such as missing small metastatic lesions) may occur.

[0005] Lack of risk stratification and prediction ability: existing technologies can only determine whether a lesion exists and the type of the lesion, but cannot predict the risk of malignant transformation based on lesion characteristics (such as size, growth rate and boundary definition) and patient history data (such as image follow-up records), which leads to the inability of clinicians to intervene in high-risk patients in advance and miss the best treatment opportunity (such as 3-6 months from early to late stage of maxillofacial tumors, and traditional methods cannot realize early risk warning).

[0006] Low diagnosis efficiency and dependence on experience: manual film reading requires doctors to analyze multi-modal images one by one, which takes an average of 15-20 minutes per patient, and the efficiency is low; and the diagnosis result is greatly influenced by the experience of the doctor (the diagnosis accuracy rate of primary doctors for rare cases is only 50%-60%), and it is difficult to ensure the consistency of diagnosis, therefore, an oral and maxillofacial surgery image recognition and diagnosis method and system based on deep learning are proposed. SUMMARY

[0007] Therefore, the present application provides an oral and maxillofacial surgery image recognition and diagnosis method and system based on deep learning to solve or alleviate the technical problems in the prior art, and at least provides a beneficial choice.

[0008] The technical solution of this invention is implemented as follows: a deep learning-based method for image recognition and diagnosis in oral and maxillofacial surgery, comprising the following steps:

[0009] Step 1: Multimodal image acquisition and collaborative preprocessing: CBCT images, cone-beam computed tomography images, intraoral endoscopic images and ultrasound images of oral and maxillofacial surgery are acquired. A standardized multimodal image dataset is obtained through intermodal registration and adaptive enhancement algorithms.

[0010] Step 2, Multi-scale feature extraction: An improved multi-branch Transformer network is used to extract the micro-features (lesion texture, edge), meso-features (tissue anatomical structure correlation), and macro-features (organ morphology and positional relationship) of each modality of image, generating a multi-scale feature set;

[0011] Step 3, Dynamic Attention Feature Fusion: Construct a dynamic weight allocation model based on lesion type, perform attention enhancement and feature fusion on multi-scale feature sets, and generate a comprehensive diagnostic feature vector;

[0012] Step 4, Lesion Identification and Risk Stratification Diagnosis: Input the comprehensive diagnostic feature vector into the dual-branch deep diagnostic model, and output the lesion type (jawbone cyst / impacted wisdom tooth / temporomandibular joint disorder / maxillofacial tumor) and the malignancy risk level (low risk / intermediate risk / high risk);

[0013] Step 5, Diagnostic Result Validation and Model Iteration: Combining clinical and pathological diagnostic results, incremental learning algorithms are used to optimize model parameters, thereby improving diagnostic accuracy and generalization ability.

[0014] More preferably, in step 1, the "multimodal image collaborative preprocessing" includes:

[0015] Modal registration: A rigid registration algorithm based on mutual information is used to align anatomical landmarks (such as root apex and mandibular angle) between CBCT images and curved tomographic images, with a registration error of <0.1mm;

[0016] Adaptive enhancement: The improved Retinex algorithm is used to suppress metal artifacts in CBCT images, histogram equalization is used to enhance mucosal lesion areas in endoscopic images, and wavelet denoising algorithm is used to improve the signal-to-noise ratio in ultrasound images.

[0017] Standardization: All images are resized to 512×512 pixels, and Z-Score standardization is used to adjust the pixel value distribution. The formula is as follows:

[0018] , where I(x,y) is the original pixel value, μ is the image mean, and σ is the image standard deviation.

[0019] More preferably, in step 2, the "improved multi-branch Transformer network" includes:

[0020] Microscopic feature branch: A 3-layer convolutional neural network (CNN) with a kernel size of 3×3 and a stride of 1 is used to extract features such as lesion edge gradient and texture complexity, and output a 256-dimensional microscopic feature vector.

[0021] Mesoscopic feature branch: A 2-layer graph convolutional network (GCN) is used to construct a topological graph with anatomical structures (such as teeth, jawbone, and temporomandibular joint) as nodes, extract structural correlation features, and output a 128-dimensional mesoscopic feature vector;

[0022] Macro-feature branch: A Transformer encoder (6 layers, 8 attention heads) is used to capture features such as the overall shape and positional offset of the organ, and output a 256-dimensional macro-feature vector.

[0023] Each branch uses Dropout (probability 0.3) to prevent overfitting, and the feature distribution is standardized by LayerNorm.

[0024] More preferably, the weight calculation method of the "dynamic weight allocation model" in step 3 is as follows:

[0025] Define the correlation coefficient α between lesion types t (Jaw cyst α1=0.3, impacted wisdom tooth α2=0.4, temporomandibular joint disorder α3=0.25, maxillofacial tumor α4=0.45);

[0026] Calculate the dynamic weights ω of features at each scale m (Microscopic ω1, Mesoscopic ω2, Macroscopic ω3), the formula is:

[0027] S m The feature significance score (calculated via a self-attention mechanism);

[0028] The formula for calculating the comprehensive diagnostic feature vector F is F = ω1·F micro +ω2·F meso +ω3·F macro It has 512 dimensions.

[0029] More preferably, in step 4, the "dual-branch depth diagnostic model" includes:

[0030] Lesion type branch: Deep belief network (DBN) is used. The input is the comprehensive diagnostic feature vector, the output layer has 4 neurons (corresponding to 4 types of diseases), the activation function is Softmax, and the output is the probability value of each disease.

[0031] Risk stratification branch: Using a long short-term memory network (LSTM) combined with the patient's historical imaging sequence (last 3 years), the probability of malignant transformation risk is output (low risk <30%, intermediate risk 30%-70%, high risk >70%).

[0032] The model loss function is a weighted sum of cross-entropy loss and mean squared error loss, with a weight ratio of 1:0.8.

[0033] A deep learning-based image recognition and diagnostic system for oral and maxillofacial surgery includes:

[0034] Multimodal image acquisition and preprocessing module: including CBCT scanner, curved surface computed tomography system, oral endoscope, ultrasound equipment and preprocessing server, used to acquire and standardize multimodal images;

[0035] Cross-scale feature extraction module: Deploys an improved multi-branch Transformer network for hierarchical extraction of micro, meso, and macro features;

[0036] Dynamic fusion diagnostic module: integrates dynamic weight allocation model and dual-branch deep diagnostic model, outputting lesion type and risk level;

[0037] Intelligent decision feedback module: includes a clinical report generation unit, a risk warning unit, and a model iteration unit, used to output diagnostic reports, trigger risk warnings, and optimize the model.

[0038] Further preferably, the "clinical report generation unit" of the intelligent decision feedback module can automatically mark the location of lesions in the image (marked with a red box) and generate a structured report containing "lesion size, location, type, risk level, and treatment recommendations"; the "risk warning unit" triggers SMS warnings to the attending physician for high-risk patients, with a response time of <10s; the "model iteration unit" fine-tunes the model parameters through the stochastic gradient descent (SGD) algorithm for every 200 clinical data cases, with a learning rate η=η0·exp(-γ·t), where η0=0.01, γ=0.001, and t is the number of iterations.

[0039] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions:

[0040] I. This invention breaks through the traditional diagnostic framework of "single-modal imaging + manual image interpretation" and innovatively proposes a three-order diagnostic logic of "multi-modal image synergistic enhancement - cross-scale feature dynamic fusion - lesion classification and risk stratification linkage". Through an improved multi-branch Transformer network and dynamic attention fusion algorithm, it can achieve accurate identification, classification and malignant risk prediction of common oral and maxillofacial diseases (such as jaw cysts, impacted wisdom teeth, temporomandibular joint disorders and maxillofacial tumors).

[0041] Second, the present invention integrates a multimodal image acquisition and preprocessing module, a cross-scale feature extraction module, a dynamic fusion diagnosis module, and an intelligent decision feedback module, which solves the technical pain points of poor modal adaptability, difficulty in identifying lesion boundaries, and lack of risk prediction ability in the prior art. The diagnostic accuracy rate reaches 94.2%, which is 32% higher than the traditional method, providing efficient and accurate technical support for clinical diagnosis and treatment.

[0042] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Fig. 1 This is a flowchart of the method of the present invention;

[0045] Fig. 2 This is a system module diagram of the present invention. Detailed Implementation

[0046] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0047] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0048] like Figs. 1-2 As shown, this embodiment of the invention provides a deep learning-based method for the identification and diagnosis of oral and maxillofacial surgical images, including the following steps:

[0049] Step 1: Multimodal image acquisition and collaborative preprocessing: CBCT images, cone-beam computed tomography images, intraoral endoscopic images and ultrasound images of oral and maxillofacial surgery are acquired. A standardized multimodal image dataset is obtained through intermodal registration and adaptive enhancement algorithms.

[0050] Step 2, Multi-scale feature extraction: An improved multi-branch Transformer network is used to extract the micro-features (lesion texture, edge), meso-features (tissue anatomical structure correlation), and macro-features (organ morphology and positional relationship) of each modality of image, generating a multi-scale feature set;

[0051] Step 3, Dynamic Attention Feature Fusion: Construct a dynamic weight allocation model based on lesion type, perform attention enhancement and feature fusion on multi-scale feature sets, and generate a comprehensive diagnostic feature vector;

[0052] Step 4, Lesion Identification and Risk Stratification Diagnosis: Input the comprehensive diagnostic feature vector into the dual-branch deep diagnostic model, and output the lesion type (jawbone cyst / impacted wisdom tooth / temporomandibular joint disorder / maxillofacial tumor) and the malignancy risk level (low risk / intermediate risk / high risk);

[0053] Step 5, Diagnostic Result Validation and Model Iteration: Combining clinical and pathological diagnostic results, incremental learning algorithms are used to optimize model parameters, thereby improving diagnostic accuracy and generalization ability.

[0054] In one embodiment, step 1, "multimodal image collaborative preprocessing," includes:

[0055] Modal registration: A rigid registration algorithm based on mutual information is used to align anatomical landmarks (such as root apex and mandibular angle) between CBCT images and curved tomographic images, with a registration error of <0.1mm;

[0056] Adaptive enhancement: The improved Retinex algorithm is used to suppress metal artifacts in CBCT images, histogram equalization is used to enhance mucosal lesion areas in endoscopic images, and wavelet denoising algorithm is used to improve the signal-to-noise ratio in ultrasound images.

[0057] Standardization: All images are resized to 512×512 pixels, and Z-Score standardization is used to adjust the pixel value distribution. The formula is as follows:

[0058] , where I(x,y) is the original pixel value, μ is the image mean, and σ is the image standard deviation.

[0059] In one embodiment, the "improved multi-branch Transformer network" in step 2 includes:

[0060] Microscopic feature branch: A 3-layer convolutional neural network (CNN) with a kernel size of 3×3 and a stride of 1 is used to extract features such as lesion edge gradient and texture complexity, and output a 256-dimensional microscopic feature vector.

[0061] Mesoscopic feature branch: A 2-layer graph convolutional network (GCN) is used to construct a topological graph with anatomical structures (such as teeth, jawbone, and temporomandibular joint) as nodes, extract structural correlation features, and output a 128-dimensional mesoscopic feature vector;

[0062] Macro-feature branch: A Transformer encoder (6 layers, 8 attention heads) is used to capture features such as the overall shape and positional offset of the organ, and output a 256-dimensional macro-feature vector.

[0063] Each branch uses Dropout (probability 0.3) to prevent overfitting, and the feature distribution is standardized by LayerNorm.

[0064] In one embodiment, the weight calculation method of the "dynamic weight allocation model" in step 3 is as follows:

[0065] Define the correlation coefficient α between lesion types t (Jaw cyst α1=0.3, impacted wisdom tooth α2=0.4, temporomandibular joint disorder α3=0.25, maxillofacial tumor α4=0.45);

[0066] Calculate the dynamic weights ω of features at each scale m (Microscopic ω1, Mesoscopic ω2, Macroscopic ω3), the formula is:

[0067] S m The feature significance score (calculated via a self-attention mechanism);

[0068] The formula for calculating the comprehensive diagnostic feature vector F is F = ω1·F micro +ω2·F meso +ω3·F macro It has 512 dimensions.

[0069] In one embodiment, the "dual-branch depth diagnostic model" in step 4 includes:

[0070] Lesion type branch: Deep belief network (DBN) is used. The input is the comprehensive diagnostic feature vector, the output layer has 4 neurons (corresponding to 4 types of diseases), the activation function is Softmax, and the output is the probability value of each disease.

[0071] Risk stratification branch: Using a long short-term memory network (LSTM) combined with the patient's historical imaging sequence (last 3 years), the probability of malignant transformation risk is output (low risk <30%, intermediate risk 30%-70%, high risk >70%).

[0072] The model loss function is a weighted sum of cross-entropy loss and mean squared error loss, with a weight ratio of 1:0.8.

[0073] A deep learning-based image recognition and diagnostic system for oral and maxillofacial surgery includes:

[0074] Multimodal image acquisition and preprocessing module: including CBCT scanner, curved surface computed tomography system, oral endoscope, ultrasound equipment and preprocessing server, used to acquire and standardize multimodal images;

[0075] Cross-scale feature extraction module: Deploys an improved multi-branch Transformer network for hierarchical extraction of micro, meso, and macro features;

[0076] Dynamic fusion diagnostic module: integrates dynamic weight allocation model and dual-branch deep diagnostic model, outputting lesion type and risk level;

[0077] Intelligent decision feedback module: includes a clinical report generation unit, a risk warning unit, and a model iteration unit, used to output diagnostic reports, trigger risk warnings, and optimize the model.

[0078] In one embodiment, the "clinical report generation unit" of the intelligent decision feedback module can automatically annotate the location of lesions in images (marked with red boxes) and generate a structured report containing "lesion size, location, type, risk level, and treatment recommendations"; the "risk warning unit" triggers SMS warnings to the attending physician for high-risk patients, with a response time of <10s; the "model iteration unit" fine-tunes the model parameters through the stochastic gradient descent (SGD) algorithm for every 200 clinical data cases, with a learning rate η=η0·exp(-γ·t), where η0=0.01, γ=0.001, and t is the number of iterations.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based method for image recognition and diagnosis in oral and maxillofacial surgery, characterized by: Includes the following steps: Step 1: Multimodal image acquisition and collaborative preprocessing: CBCT images, cone-beam computed tomography images, intraoral endoscopic images and ultrasound images of oral and maxillofacial surgery are acquired. A standardized multimodal image dataset is obtained through intermodal registration and adaptive enhancement algorithms. Step 2, Multi-scale feature extraction: An improved multi-branch Transformer network is used to extract the micro-features (lesion texture, edge), meso-features (tissue anatomical structure correlation), and macro-features (organ morphology and positional relationship) of each modality of image, generating a multi-scale feature set; Step 3, Dynamic Attention Feature Fusion: Construct a dynamic weight allocation model based on lesion type, perform attention enhancement and feature fusion on multi-scale feature sets, and generate a comprehensive diagnostic feature vector; Step 4, Lesion Identification and Risk Stratification Diagnosis: Input the comprehensive diagnostic feature vector into the dual-branch deep diagnostic model, and output the lesion type (jawbone cyst / impacted wisdom tooth / temporomandibular joint disorder / maxillofacial tumor) and the malignancy risk level (low risk / intermediate risk / high risk); Step 5, Diagnostic Result Validation and Model Iteration: Combining clinical and pathological diagnostic results, incremental learning algorithms are used to optimize model parameters, thereby improving diagnostic accuracy and generalization ability.

2. The deep learning-based oral and maxillofacial surgical image recognition and diagnosis method according to claim 1, characterized in that: In step 1, "multimodal image collaborative preprocessing" includes: Modal registration: A rigid registration algorithm based on mutual information is used to align anatomical landmarks (such as root apex and mandibular angle) between CBCT images and curved tomographic images, with a registration error of <0.1mm; Adaptive enhancement: The improved Retinex algorithm is used to suppress metal artifacts in CBCT images, histogram equalization is used to enhance mucosal lesion areas in endoscopic images, and wavelet denoising algorithm is used to improve the signal-to-noise ratio in ultrasound images. Standardization: All images are resized to 512×512 pixels, and Z-Score standardization is used to adjust the pixel value distribution. The formula is as follows: , where I(x,y) is the original pixel value, μ is the image mean, and σ is the image standard deviation.

3. The deep learning-based oral and maxillofacial surgical image recognition and diagnosis method according to claim 1, characterized in that: In step 2, the "improved multi-branch Transformer network" includes: Microscopic feature branch: A 3-layer convolutional neural network (CNN) with a kernel size of 3×3 and a stride of 1 is used to extract features such as lesion edge gradient and texture complexity, and output a 256-dimensional microscopic feature vector. Mesoscopic feature branch: A 2-layer graph convolutional network (GCN) is used to construct a topological graph with anatomical structures (such as teeth, jawbone, and temporomandibular joint) as nodes, extract structural correlation features, and output a 128-dimensional mesoscopic feature vector; Macro-feature branch: A Transformer encoder (6 layers, 8 attention heads) is used to capture features such as the overall shape and positional offset of the organ, and output a 256-dimensional macro-feature vector. Each branch uses Dropout (probability 0.3) to prevent overfitting, and the feature distribution is standardized by LayerNorm.

4. The deep learning-based oral and maxillofacial surgical image recognition and diagnosis method according to claim 1, characterized in that: In step 3, the weight calculation method of the "dynamic weight allocation model" is as follows: Define the correlation coefficient α between lesion types t (Jaw cyst α1=0.3, impacted wisdom tooth α2=0.4, temporomandibular joint disorder α3=0.25, maxillofacial tumor α4=0.45); Calculate the dynamic weights ω of features at each scale m (Microscopic ω1, Mesoscopic ω2, Macroscopic ω3), the formula is: S m The feature significance score (calculated via a self-attention mechanism); The formula for calculating the comprehensive diagnostic feature vector F is F = ω1·F micro +ω2·F meso +ω3·F macro It has 512 dimensions.

5. The deep learning-based oral and maxillofacial surgical image recognition and diagnosis method according to claim 1, characterized in that: In step 4, the "dual-branch depth diagnostic model" includes: Lesion type branch: Deep belief network (DBN) is used. The input is the comprehensive diagnostic feature vector, the output layer has 4 neurons (corresponding to 4 types of diseases), the activation function is Softmax, and the output is the probability value of each disease. Risk stratification branch: Using a long short-term memory network (LSTM) combined with the patient's historical imaging sequence (last 3 years), the probability of malignant transformation risk is output (low risk <30%, intermediate risk 30%-70%, high risk >70%). The model loss function is a weighted sum of cross-entropy loss and mean squared error loss, with a weight ratio of 1:0.

8.

6. A deep learning-based oral and maxillofacial surgical image recognition and diagnosis system, coupled with the deep learning-based oral and maxillofacial surgical image recognition and diagnosis method as described in any one of claims 1-5, characterized in that: include: Multimodal image acquisition and preprocessing module: including CBCT scanner, curved surface computed tomography system, oral endoscope, ultrasound equipment and preprocessing server, used to acquire and standardize multimodal images; Cross-scale feature extraction module: Deploys an improved multi-branch Transformer network for hierarchical extraction of micro, meso, and macro features; Dynamic fusion diagnostic module: integrates dynamic weight allocation model and dual-branch deep diagnostic model, outputting lesion type and risk level; Intelligent decision feedback module: includes a clinical report generation unit, a risk warning unit, and a model iteration unit, used to output diagnostic reports, trigger risk warnings, and optimize the model.

7. The deep learning-based oral and maxillofacial surgical image recognition and diagnostic system according to claim 6, characterized in that: The intelligent decision feedback module's "clinical report generation unit" can automatically annotate the location of lesions in images (marked with red boxes) and generate a structured report containing "lesion size, location, type, risk level, and treatment recommendations"; the "risk warning unit" triggers SMS warnings to attending physicians for high-risk patients, with a response time of <10s; the "model iteration unit" fine-tunes model parameters using stochastic gradient descent (SGD) algorithm for every 200 clinical data cases, with a learning rate η = η0·exp(-γ·t), where η0 = 0.01, γ = 0.001, and t is the number of iterations.

Images