Intelligent diagnosis and treatment decision-making method and system for jaw osteomyelitis

Abstract

The invention relates to the technical field of jaw disease diagnosis and treatment, in particular to an intelligent diagnosis and treatment decision-making method and system for jaw osteomyelitis. An intelligent diagnosis and treatment decision-making method for jaw osteomyelitis comprises the following steps: acquiring a panoramic radiographic image, and executing panoramic radiographic image pixel-level cutting image cutting and quantitative imageomics feature extraction for a suspicious focus through a SwinTransform model; inputting the input image into a semantic segmentation network based on a UNet architecture for processing; and obtaining multi-modal input data corresponding to the clinical text, and generating diagnosis and treatment decision support information based on a pre-trained multi-modal large language model. According to the system disclosed by the invention, through the multi-dimensional innovation, an intelligent diagnosis and treatment support platform specially aiming at the characteristics of the jaw osteomyelitis disease is constructed. The method not only can assist clinicians in improving diagnosis accuracy and reducing missed diagnosis and misdiagnosis risks, but also can improve clinical diagnosis and treatment quality and efficiency of jaw osteomyelitis on the whole by providing personalized treatment decision reference based on multi-source evidences.

Description

Technical Field

[0001] This invention relates to the field of jawbone disease diagnosis and treatment technology, specifically to an intelligent diagnosis and treatment decision-making method and system for jawbone osteomyelitis. Background Technology

[0002] Osteomyelitis of the jaw is a disease characterized by bone destruction and necrosis, usually caused by drugs, radiation therapy, trauma, or spontaneous factors, often resulting in significant tissue loss. The pathogenesis of osteomyelitis of the jaw is complex, making its diagnosis and treatment challenging. Physicians need a comprehensive and in-depth understanding of the patient's medical history and clinical data to make an accurate diagnosis, which further complicates the development of effective individualized treatment strategies.

[0003] In terms of treatment planning, osteomyelitis of the jaw requires a systematic approach and relies on the mutual support and corroboration of different clinical evidence. Early diagnosis may be achieved through imaging examinations that reveal manifestations such as diffuse sclerosis, highlighting the crucial role of maxillofacial radiography in identifying lesions associated with osteomyelitis of the jaw. Among these, panoramic radiography is usually the preferred initial screening method. Although oral and maxillofacial surgeons have reached a certain consensus on treatment strategies, the initial consultation for patients with osteomyelitis of the jaw is often not made by maxillofacial surgeons, but by physicians from other medical disciplines. Studies on drug-related osteomyelitis of the jaw further indicate that conventional conservative treatment is often ineffective. Therefore, the selection of treatment strategies and the timing of treatment are crucial for the effective management of osteomyelitis of the jaw. Studies have shown that timely and accurate treatment can significantly improve the cure rate, ranging from 36% to 90%. Against this background, research on automated systems for the diagnosis and treatment of osteomyelitis of the jaw has significant clinical implications.

[0004] Currently, the focus of auxiliary diagnosis for osteomyelitis is often on later-stage auxiliary diagnosis. For example, artificial intelligence technology has been applied to assist in the diagnosis of drug-related osteomyelitis of the jaw. These solutions currently mainly focus on the prediction and diagnosis of necrosis risk based on imaging features, such as observing microscopic changes in the trabeculae and cortical bone of the mandible through cone-beam computed tomography. However, most existing AI methods focus on risk control or imaging screening, rarely integrating disease-specific, systematic clinical treatment planning into the solution. In actual clinical practice, such comprehensive treatment planning usually requires experienced oral and maxillofacial surgeons.

[0005] To address the aforementioned diagnostic challenges, especially when patients undergo relevant examinations in other clinics (e.g., when they have foreign bodies in their mouths and require examination and diagnosis in the oral surgery department), intelligent triage and early predictive diagnosis have a significant positive impact on patient efficiency and emotional well-being before their visit. Such intelligent triage and consultation systems often require the use of large language models to provide accurate content output to patients.

[0006] In recent years, with the groundbreaking development of large language models (MLLMs), a new paradigm in natural language processing, significant progress has also been made in multimodal large language models (MLLMs). Therefore, MLLMs have potential applications in achieving efficient diagnosis and generating personalized treatment plans, thereby contributing to improved clinical management of complex diseases. However, while existing general-purpose models perform well in addressing general medical problems, they still have significant limitations in providing reliable, comprehensive, and clinically pathway-compliant recommendations for specific specialty diseases.

[0007] In summary, current research on the application of artificial intelligence in osteomyelitis of the jaw mainly focuses on analyzing its imaging data. However, accurate diagnosis and treatment decisions for osteomyelitis of the jaw require processing both imaging data and clinical case (text) data simultaneously. While existing multimodal large language models have demonstrated powerful multimodal (image and text) understanding and reasoning capabilities, they perform poorly when dealing with this specific disease, primarily because their training data is concentrated in non-medical or general domains. Therefore, this invention proposes a specialized multimodal large language model for the diagnosis and treatment decisions of osteomyelitis of the jaw.

[0008] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the inventors studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention develops an innovative vision-language fusion system, namely, an intelligent diagnostic and treatment decision-making system for osteomyelitis of the jaw, aiming to fill this gap in current clinical practice. This system integrates an image-based deep learning module (covering disease diagnosis, screening, and automated segmentation of the treatment target area) and a professional multimodal large language model module specifically designed for clinical interaction and decision support. This system is expected to be built into a comprehensive clinical decision support platform to assist oral and maxillofacial surgeons in planning the diagnosis and treatment of osteomyelitis of the jaw, ultimately improving the efficiency and accuracy of medical decisions.

[0010] This invention relates to the field of jawbone disease diagnosis and treatment technology, specifically to an intelligent diagnosis and treatment decision-making method and system for jawbone osteomyelitis.

[0011] One objective of this invention is to provide an intelligent diagnostic and treatment decision-making method for osteomyelitis of the jaw, comprising the following steps: Panoramic radiographic images were acquired, and pixel-level image segmentation and quantitative radiomics feature extraction for suspicious lesions were performed using the SwinTransformer model. The SwinTransformer model was pre-trained on a large-scale natural image dataset. All weight parameters except the last layer were frozen, and only the final classification layer was adjusted. Input the input image into a U-based... The semantic segmentation network of the Net architecture is used for processing. The processing includes: extracting multi-scale features through convolution and downsampling of the encoder path, then upsampling and skip connections of the decoder path, fusing the features of the corresponding layer of the encoder to combine semantic and detail information, and finally outputting a segmentation probability map of the same size as the input through a 1×1 convolutional layer. The semantic segmentation network is trained using the Dice loss function to optimize the segmentation accuracy of small target regions with lesions in the image. Multimodal input data corresponding to clinical texts are obtained, and diagnostic decision support information is generated based on a pre-trained multimodal large language model. In this process, a low-rank adaptation method is used, and a professional question-and-answer dataset containing images and diagnostic dialogues is used to adjust the pre-trained multimodal large language model with a cross-entropy loss function.

[0012] According to a preferred embodiment, the SwinTransformer model achieves information interaction and fusion between local features and global dependencies through alternating execution of a window multi-head self-attention mechanism and a shift window multi-head self-attention mechanism. The window multi-head self-attention mechanism is configured as follows: In the window Internal, input features Perform multi-head self-attention:

[0013] in, , , .

[0014] According to a preferred embodiment, the shifted window multi-head self-attention mechanism is configured to establish cross-window connections by performing window displacement operations.

[0015] According to a preferred embodiment, the preprocessing of panoramic radiographic images includes: normalizing the image pixel values ​​to a predetermined range and processing them into an input tensor with three channels. , H and W represent the height and width of the image, respectively, to meet the input requirements of the U-Net segmentation network.

[0016] Preferably, normalization is performed to a predetermined range. or .

[0017] According to a preferred embodiment, the encoder of the U-Net network extracts features step by step through an operation sequence including convolution, ReLU activation function and max pooling downsampling, and the decoder performs upsampling operation and jumps to connect with the corresponding feature map of the encoder to fuse information. Finally, a segmentation probability map is output through a 1×1 convolutional layer.

[0018] According to a preferred embodiment, the multimodal large language model is constructed based on the Qwen2-VL model architecture; during the adjustment process, the pre-trained transformer weights of the model are frozen, and the image and text inputs are processed by word segmentation and embedding, and then the transformer performs multimodal fusion understanding.

[0019] According to a preferred embodiment, a low-rank adaptation method is used to adjust a multimodal large language model, including: adding a low-rank decomposition matrix to the weight matrix of the pre-trained model transformer layer. , Where, B∈ , A∈ , and rank During the adjustment process, only the parameters of the low-rank matrices B and A are updated.

[0020] One of the objectives of this invention is to provide an intelligent diagnostic and treatment decision-making system for osteomyelitis of the jaw, such as... Figure 1 As shown, it includes a jaw osteomyelitis diagnosis module, a jaw osteomyelitis treatment target region segmentation module, and a jaw osteomyelitis treatment support module. The jaw osteomyelitis diagnosis module is used for lesion identification based on panoramic radiographic images. The jaw osteomyelitis treatment target region segmentation module is used to extract and segment quantitative features of the lesion region. The jaw osteomyelitis treatment support module is used to generate key dialogue support by combining disease perception and target area information. Specifically, combining the disease perception seen in the images provided by the jaw osteomyelitis diagnosis module with the target area information obtained from segmentation provided by the jaw osteomyelitis treatment target region segmentation module, the jaw osteomyelitis treatment support module adopts a low-rank adaptation method, utilizes a professional question-and-answer dataset containing images and diagnostic dialogues, adjusts the model with a cross-entropy loss function, and generates diagnostic decision support information based on multimodal input data.

[0021] According to a preferred embodiment, the jaw osteomyelitis diagnostic module is a SwinTransformer model pre-trained on a large-scale natural image dataset, wherein the SwinTransformer model freezes all weight parameters except for the last layer and adjusts only the final classification layer.

[0022] According to a preferred embodiment, the jaw osteomyelitis treatment targeted region segmentation module adopts a semantic segmentation network based on the U-Net architecture. The jaw osteomyelitis treatment targeted region segmentation module is configured as follows: after extracting multi-scale features through convolution and downsampling of the encoder path, it then performs upsampling and skip connections of the decoder path, fuses the corresponding layer features of the encoder to combine semantic and detail information, and finally outputs a segmentation probability map of the same size as the input through a 1×1 convolutional layer. The semantic segmentation network is trained using the Dice loss function to optimize the segmentation accuracy of small target regions of lesions in the image.

[0023] The beneficial effects of this technical solution are: To address the problems of misdiagnosis, missed diagnosis, low diagnostic efficiency, and strong subjectivity in the current clinical diagnosis of osteomyelitis of the jaw, this invention proposes an intelligent diagnostic and treatment decision-making system for osteomyelitis of the jaw. By deeply integrating deep learning and multimodal large language model technology, it achieves collaborative analysis and decision support for image and text data.

[0024] First, at the multimodal information fusion decision-making level, the system of this invention breaks through the current situation where most AI methods are limited to single image analysis. By integrating a multimodal large language model, the system of this invention innovatively realizes the simultaneous interpretation and correlation analysis of patients' panoramic radiographs, CBCT and other imaging data with textual medical records such as chief complaints, medical history, and medication records. This fusion capability enables the system to simulate expert diagnostic thinking, cross-validating morphological evidence in the images (such as the extent of bone destruction and sclerosis characteristics) with clinical evidence in the text (such as drug exposure history and symptom duration), thereby providing more comprehensive and convincing support for diagnosis, directly addressing the core requirement of combining imaging and clinical information for the diagnosis of osteomyelitis of the jaw.

[0025] Secondly, regarding the specialized customization for specific disease characteristics, to overcome the shortcomings of general models in terms of insufficient knowledge in specialized fields, the system of this invention innovatively constructs a specialized question-and-answer dataset covering the classification, differential diagnosis, and treatment selection of osteomyelitis of the jaw during the training process, and performs deep instruction adjustments on the multimodal large language model based on this dataset. This enables the system to not only possess general image and text understanding capabilities, but also internalize specific knowledge of osteomyelitis of the jaw. For example, it can accurately understand the different management strategies for "drug-related osteomyelitis" and "radiation osteomyelitis," or assess the urgency of surgical intervention based on the continuity of cortical bone destruction on imaging combined with medication history. This specialized customization ensures the clinical accuracy and reliability of decision-making recommendations.

[0026] Finally, at the structured workflow and decision support level, the system of this invention constructs a collaborative workflow covering key aspects of diagnosis and treatment. The image-based deep learning module can efficiently complete automatic lesion screening, initial diagnosis, and precise segmentation of treatment target areas (e.g., the extent of necrotic bone), providing objective quantitative evidence. Subsequently, a specialized multimodal large language model module can generate structured diagnostic reasoning and personalized treatment plan suggestions (including drug selection, surgical timing, and follow-up points) based on the above analysis results and complete medical record text, and answer doctors' questions in a natural language interactive manner. This integrated process of "image analysis-data fusion-decision generation" significantly improves the efficiency and standardization of diagnosis and treatment.

[0027] In summary, the system of this invention, through the aforementioned multi-dimensional innovations, constructs an intelligent diagnostic and treatment support platform specifically tailored to the characteristics of osteomyelitis of the jaw. It not only assists clinicians in improving diagnostic accuracy and reducing the risk of missed or misdiagnosed cases, but also optimizes treatment plan selection by providing personalized treatment decision-making references based on multi-source evidence, thereby improving the overall quality and efficiency of clinical diagnosis and treatment for osteomyelitis of the jaw. Attached Figure Description

[0028] Figure 1 This invention describes the construction process and working principle of the intelligent diagnosis and treatment decision-making system for osteomyelitis of the jaw. Figure 2 Demonstration of the AI-based visual question-and-answer (Q&A) intelligent diagnosis and treatment decision-making system for osteomyelitis of the jaw; Figure 3 This is a quantitative comparison of the results of the intelligent diagnosis and treatment decision system for osteomyelitis of the jaw in the diagnosis of osteomyelitis of the jaw according to the present invention; Figure 4 This is a quantitative comparison of the segmentation of the target area for mandibular osteomyelitis treatment using the intelligent diagnosis and treatment decision system for mandibular osteomyelitis of the present invention; Figure 5 This presents the quantitative comparison and ablation experiment results of various visual question-answering models in the intelligent diagnosis and treatment decision-making system for osteomyelitis of the jawbone of the present invention. Detailed Implementation

[0029] In the description of this invention, terminology is used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly defined.

[0030] U Net: A biomedical semantic segmentation network architecture, which is a type of fully convolutional neural network.

[0031] MSA: Multi Head Self Attention is a method that interprets the input sequence from different perspectives and computes multiple attentions simultaneously.

[0032] MLP: Multilayer Perceptron, also known as artificial neural network, is an architecture that contains multiple hidden layers in addition to the input and output layers.

[0033] Example 1 (I) System Construction The jaw osteomyelitis diagnostic module is used in subsequent processes to screen jaw osteomyelitis cases. The diagnostic criteria are based on binary classification of panoramic radiographic images (PR images). This invention employs SwinTransformer to perform the PR image classification task. The advantages of SwinTransformer are: it is built on top of the Transformer architecture, which, with its core attention mechanism, performs excellently in natural image classification tasks, effectively capturing the correlations between different regions of the image; PR images are typically high-resolution, and the sliding window mechanism and hierarchical structure used by SwinTransformer give it excellent performance in high-resolution image feature extraction.

[0034] 1. SwinTransformer processes the input PR image and outputs diagnostic results.

[0035] The specific process is as follows: 1.1 Patch Embedding: Given an input PR image

[0036] Where H and W represent the height and width of the image, respectively.

[0037] Divide it into non-overlapping small blocks, each of size "P×P", then the number of blocks is:

[0038] Each image patch is flattened into a vector and linearly mapped to a C-dimensional embedding space, forming a vector sequence:

[0039] 1.2 SwinTransformer Block Processing: The embedded image block sequence is input into multiple SwinTransformer blocks. Each block contains Window Multi-Head Self-Attention (W-MSA), Shift Window Multi-Head Self-Attention (SW-MSA), Multilayer Perceptron (MLP), Layer Normalization, and Residual Connection.

[0040] 1.3 Windowed Multi-Head Self-Attention (W-MSA): This method divides the feature map into non-overlapping windows and applies multi-head self-attention within each window, thereby improving computational efficiency. Specifically, within window "W", input feature "X" undergoes multi-head self-attention:

[0041] in, , , .

[0042] 1.4 Shift-Window Multi-Head Self-Attention (SW-MSA): Establishing cross-window connections by shifting windows helps capture global dependencies.

[0043] 1.5 Multilayer Perceptron (MLP): Following the attention module, the MLP blocks integrate information from each channel. The training process is aided by normalization and residual connections between modules, as follows: , in, This is the activation function.

[0044] 1.6 Layer Normalization and Residual Connections: For feature x, layer normalization and residual connections are used for updating, as follows: .

[0045] 1.7 Hierarchical Feature Extraction: As features are processed through multiple SwinTransformer stages, the spatial resolution gradually decreases, while the channel dimension gradually increases through the image patch merging process, thereby enabling the learning of multi-scale features.

[0046] 1.8 Final Classification Layer: After passing through the last SwinTransformer block, the global features are pooled into a fixed-length vector. The data is then input into a linear classifier for binary classification (whether it is osteomyelitis of the jawbone), as follows: , in, The output result is used for binary classification.

[0047] Because the available annotated PR image datasets for diagnosing osteomyelitis of the jaw are limited, training a model from scratch may result in insufficient generalization ability. To address this issue, this technical solution introduces a transfer learning strategy to improve the diagnostic performance of osteomyelitis of the jaw. Specifically, a model pre-trained on a large-scale natural image classification dataset (e.g., ImageNet) is used, all parameters except the last layer are frozen, and adjustments are made only to the final classification layer to adapt it to the task of diagnosing osteomyelitis of the jaw.

[0048] To verify the superiority of the jaw osteomyelitis diagnostic module, the system was compared and evaluated with various deep learning models, including methods based on Transformer and Convolutional Neural Networks (CNNs): the Transformer architecture used was the recently proposed CrossViT and ViT; the CNN architecture used were the classic models VGG, ResNet, and Inception. These comparative experiments verified the leading performance of SwinTransformer in the jaw osteomyelitis diagnostic task.

[0049] 2. Training of the targeted region segmentation module for the treatment of osteomyelitis of the jawbone The targeted region segmentation module for jaw osteomyelitis treatment aims to identify the target region for clinical procedures in jaw osteomyelitis, which differs from other experiments. U-Net excels in detecting subtle differences between pixels.

[0050] U-Net is based on an encoder-decoder architecture. The encoder extracts multi-scale features from the input image, while the decoder fuses these features through skip connections for the final prediction.

[0051] Given a panoramic image of size H×W, , First, normalize the image pixels to a certain range (e.g., ...). or Then, the images are stacked into a form with 3 feature channels to meet the input requirements of U-Net:

[0052] The U-Net processing flow for the preprocessed image is as follows: 2.1 Encoder: The image input consists of a series of convolutional layers for feature extraction. Each layer typically contains convolution operations, an activation function (ReLU), and possible pooling operations to reduce spatial dimensionality. In each layer, the spatial resolution of the feature map is downsampled, while the number of feature channels increases, enabling the network to capture increasingly abstract image representations. As the network deepens, higher-level semantic information of the image is extracted. Specifically, the encoder consists of multiple layers of convolution and downsampling operations. The feature extraction process of the "l"th layer can be represented as:

[0053] in, This is the ReLU activation function.

[0054] To gradually reduce spatial resolution and extract high-dimensional features, a downsampling operation is introduced:

[0055] in, This is for max pooling.

[0056] 2.2 Decoder: Feature maps along the encoding path are upsampled (increasing spatial dimension) to gradually restore the original image size. In each upsampling step, the feature map is concatenated with the corresponding feature map along the encoding path through skip connections, thus combining high-level semantic features with detailed spatial information. Subsequently, additional convolutional layers refine the segmentation of the concatenated feature map. Specifically, the decoding process at layer "l" includes upsampling and skip connections:

[0057] in, Indicates upsampling, This indicates a splicing operation.

[0058] The concatenated features are then refined through convolution as follows:

[0059] 2.3 Output Layer: The last layer of the network uses 1×1 convolutions to map each feature vector to the desired number of output channels, corresponding to the number of segmentation categories. The output is a segmentation map of the same size as the input image, where each pixel is assigned a category label based on the learned features. The specific process is as follows: , in, This represents the number of categories.

[0060] The predicted class probability for each pixel is obtained through softmax, as follows: .

[0061] Segmentation tasks face the problem of class imbalance because lesion regions only occupy a small portion of the panoramic image. To address this issue, this invention uses the Dice loss function to train a target region segmentation model, as follows: , in, To predict probabilities, This is a true label.

[0062] This invention compares U-Net with three other deep learning models (SegFormer, FCN, and LR-ASPP) for segmentation of osteomyelitis of the jaw. For a fair comparison, the final prediction results of SegFormer, FCN, and LR-ASPP were upsampled to match the resolution of the input image.

[0063] 3. Supervised adjustments to the treatment support module for osteomyelitis of the jaw. The treatment support module for osteomyelitis of the jaw is a multimodal large language model (MLLM) that provides visual question-answering functionality based on segmented panoramic radiographs (PR) and medical records. This invention employs low-rank adaptation (LoRA) technology to adjust the MLLM, using a modified question-answering dataset expanded from collected clinical documents and image files.

[0064] This invention constructs a treatment support model for osteomyelitis of the jaw based on the Qwen2-VL model, which achieves state-of-the-art performance in various visual question answering tasks. When processing PR images of osteomyelitis of the jaw and corresponding text query prompts, this invention first performs word segmentation on two modalities using linear layers and embedding techniques respectively. The entire image is divided into non-overlapping image blocks, and these image blocks are linearly mapped to feature representations. For text prompts, embedding vectors are used as tokens. Image tokens and text tokens are used together as input to the transformer. To utilize the pre-trained knowledge of Qwen2-VL, this technical solution adopts its pre-trained transformer weights and freezes them during the adjustment process.

[0065] Furthermore, based on the LoRA method, this invention adds learnable weights to the transformer. Specifically, for each weight matrix in the pre-trained model, this invention adds a low-rank decomposition matrix of the form: , in, , rank of a matrix The rank of these matrices is much smaller than the size of the weight matrix of that layer, which means that the number of parameters is greatly reduced.

[0066] To tune the treatment support model for osteomyelitis of the jaw, a question-and-answer dataset was collected based on surgeons' clinical behavior. Each entry contained a PR image and several dialogues about medical records. The treatment decision model was tuned using the cross-entropy loss function.

[0067] in, P Q is the true distribution from the dialogue data, and Q is the predicted distribution from the jaw osteomyelitis treatment support model.

[0068] (II) Functional Verification 1. Diagnostic performance of osteomyelitis of the jawbone In the diagnostic module for osteomyelitis of the jaw, the system uses SwinTransformer as the basic model. The test results compare the diagnostic performance of six deep learning models, including the one used, for osteomyelitis of the jaw: SwinTransformer, CrossViT, ViT, VGG, ResNet, and Inception. The SwinTransformer built in this system performed best, with accuracy, precision, recall, and F1 score of 0.9236 (95% confidence interval, 0.9056–0.9416), 0.9121 (95% CI, 0.8911–0.9331), 0.9595 (95% CI, 0.9315–0.9875), and 0.9352 (95% CI, 0.9142–0.9562), respectively. The evaluation metrics for all models (i.e., accuracy, precision, recall, specificity, and F1 score) are summarized in [the table / data]. Figure 3 In the mean square, SwinTransformer achieved the best performance across all metrics except specificity; while ViT had the highest specificity at 0.8828 (95% confidence interval: 0.8308–0.9348). The accuracy, precision, recall, and F1 score of SwinTransformer were 0.9236 (95% confidence interval: 0.9056–0.9416), 0.9121 (95% confidence interval: 0.8911–0.9331), 0.9595 (95% confidence interval: 0.9315–0.9875), and 0.9352 (95% confidence interval: 0.9142–0.9562), respectively.

[0069] 2. Targeted region segmentation performance for the treatment of osteomyelitis of the jawbone Figure 4This paper presents quantitative comparative results of a smart diagnostic and treatment decision-making system for mandibular osteomyelitis in segmenting the targeted treatment region. In the segmentation module for the targeted treatment region of mandibular osteomyelitis, the system utilizes the U-Net model to segment the targeted treatment area. This invention uses multiple metrics to evaluate model performance, including pixel accuracy (PA), intersection-over-union ratio (IoU), Dice coefficient, precision, recall, specificity, and F1 score. Among all deep learning segmentation models, U-Net achieves the best performance in segmenting the targeted treatment region of mandibular osteomyelitis across all metrics. Specifically, U-Net has an IoU of 0.8682 (95% confidence interval: 0.8459–0.8905) and a Dice coefficient of 0.9295 (95% confidence interval: 0.9165–0.9425). FCN, LR-ASPP, and SegFormer perform well in PA, but their performance in IoU and Dice is less than ideal. This is because target region segmentation for osteomyelitis of the jaw is an imbalanced segmentation task, where the target region is typically much smaller than the normal region. These models tend to predict the normal region, leading to undersegmentation of the target region. In contrast, UNet achieved high scores on PA, IoU, and Dice metrics. Its precision, recall, and specificity were 0.8769 (95% confidence interval: 0.8639–0.8899), 0.9887 (95% confidence interval: 0.9676–0.9999), and 0.9900 (95% confidence interval: 0.9891–0.9909), respectively, demonstrating its superior performance in the target region segmentation task for osteomyelitis of the jaw.

[0070] 3. Supportive properties for the treatment of osteomyelitis of the jawbone Figure 5 Quantitative comparisons and ablation tests of various visual question-answering models in the intelligent diagnostic and treatment decision-making system for osteomyelitis of the jaw are presented. The results show that the system of this invention significantly outperforms previous methods, achieving BLEU-4 scores of 0.5260 (95% confidence interval: 0.4877–0.5644), METEOR scores of 0.8436 (95% CI: 0.8208–0.8664), and ROUGE-L scores of 0.6654 (95% CI: 0.6262–0.7046), respectively. Furthermore, this invention also evaluated the performance of the system in different types of osteomyelitis of the jaw, showing that the system output is stable and balanced.

[0071] Ablation tests were conducted to compare the impact of different modalities on system performance, evaluating performance using only text input (excluding images) or only image input (excluding text). Results showed that removing the text modality resulted in an average performance decrease of 5.00%, while removing the image modality resulted in a performance decrease of 9.67%. The performance of the baseline model and the proposed system was compared using BLEU, ROUGE, and METEOR scores. The results indicate that although LLaVA-Med (an advanced medical multimodal large language model trained on a public dataset) outperforms the untrained Qwen2-VL in the jaw osteomyelitis question-answering task, the proposed system demonstrates superior performance in diagnostic accuracy and content feedback. Figure 2 As shown.

[0072] Example 2 This embodiment discloses an intelligent diagnosis and treatment decision-making system for osteomyelitis of the jaw, such as Figure 1 As shown, it includes a diagnostic module for osteomyelitis of the jaw, a targeted region segmentation module for osteomyelitis of the jaw treatment, and a treatment support module for osteomyelitis of the jaw. The diagnostic module for osteomyelitis of the jaw is used for lesion identification based on panoramic radiographic images. The targeted region segmentation module for osteomyelitis of the jaw treatment is used to extract and segment quantitative features of the lesion area. The treatment support module for osteomyelitis of the jaw is used to generate key dialogue support by combining disease knowledge and target area information.

[0073] Specifically, combining the disease perception seen in the images provided by the jaw osteomyelitis diagnosis module with the target area information obtained from the segmentation provided by the jaw osteomyelitis treatment target area segmentation module, the jaw osteomyelitis treatment assistance module adopts a low-rank adaptation method, utilizes a professional question-and-answer dataset containing images and diagnostic dialogues, adjusts the model with a cross-entropy loss function, and generates diagnostic decision support information based on multimodal input data.

[0074] The functions of each module described above correspond to those in Example 1, and will not be repeated here.

[0075] It should be noted that the specific embodiments described above are exemplary, and those skilled in the art can devise various solutions inspired by the disclosure of this invention. These solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. A smart diagnostic and treatment decision-making method for osteomyelitis of the jaw, characterized in that, Includes the following steps: Panoramic radiographic images are acquired, and pixel-level image segmentation and quantitative radiomics feature extraction for suspicious lesions are performed using the SwinTransformer model. The SwinTransformer model is pre-trained on a large-scale natural image dataset by freezing all weight parameters except the last layer and adjusting only the final classification layer. The input image is input to a U-based system. The semantic segmentation network of the Net architecture is used for processing. The processing includes: extracting multi-scale features through convolution and downsampling of the encoder path, then upsampling and skip connections of the decoder path, fusing the features of the corresponding layer of the encoder to combine semantic and detail information, and finally outputting a segmentation probability map of the same size as the input through a 1×1 convolutional layer. The semantic segmentation network is trained using the Dice loss function to optimize the segmentation accuracy of small target regions with lesions in the image. Multimodal input data corresponding to clinical texts are obtained, and diagnostic decision support information is generated based on a pre-trained multimodal large language model. In this process, a low-rank adaptation method is used, and a professional question-and-answer dataset containing images and diagnostic dialogues is used to adjust the pre-trained multimodal large language model with a cross-entropy loss function.

2. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The SwinTransformer model achieves information interaction and fusion between local features and global dependencies through alternating execution of window multi-head self-attention mechanism and shift window multi-head self-attention mechanism. The window multi-head self-attention mechanism is configured as follows: In the window Internal, input features Perform multi-head self-attention: in, , , .

3. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The shifted window multi-head self-attention mechanism is configured to establish cross-window connections by performing window displacement operations.

4. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The preprocessing of the panoramic radiographic image includes: normalizing the image pixel values ​​to a predetermined range and processing them into an input tensor with three channels. ， H and W represent the height and width of the image, respectively, to meet the input requirements of the U-Net segmentation network.

5. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The encoder of the U-Net network extracts features step by step through an operation sequence that includes convolution, ReLU activation function and max pooling downsampling. The decoder performs upsampling operation and makes skip connections with the corresponding feature maps of the encoder to fuse information. Finally, it outputs a segmentation probability map through a 1×1 convolutional layer.

6. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The multimodal large language model is built on the Qwen2-VL model architecture. During the adjustment process, the pre-trained transformer weights of the model are frozen. After the image and text inputs are processed by word segmentation and embedding, the transformer performs multimodal fusion understanding.

7. The intelligent diagnosis and treatment decision-making method for osteomyelitis of the jawbone according to claim 1, characterized in that, The multimodal large language model is adjusted using a low-rank adaptation method, including adding a low-rank decomposition matrix to the weight matrix of the pre-trained model transformer layer. ， Where, B∈ , A∈ , and rank During the adjustment process, only the parameters of the low-rank matrices B and A are updated.

8. A smart diagnostic and treatment decision-making system for osteomyelitis of the jaw, characterized in that, Include The jawbone osteomyelitis diagnostic module is used for lesion identification based on panoramic radiographic images. The targeted region segmentation module for treating osteomyelitis of the jawbone is used to extract and segment quantitative features of the lesion area. and The jawbone osteomyelitis treatment support module is used to generate key dialogue support by combining disease understanding and target area information. Combining the disease perception seen in the images provided by the jaw osteomyelitis diagnosis module with the target area information obtained from the segmentation provided by the jaw osteomyelitis treatment target area segmentation module, the jaw osteomyelitis treatment assistance module adopts a low-rank adaptation method, utilizes a professional question-and-answer dataset containing images and diagnostic dialogues, adjusts the model with a cross-entropy loss function, and generates diagnostic decision support information based on multimodal input data.

9. The intelligent diagnosis and treatment decision-making system for osteomyelitis of the jawbone according to claim 8, characterized in that, The jaw osteomyelitis diagnostic module is a SwinTransformer model pre-trained on a large-scale natural image dataset. The SwinTransformer model freezes all weight parameters except for the last layer and adjusts only the final classification layer.

10. The intelligent diagnosis and treatment decision-making system for osteomyelitis of the jawbone according to claim 8, characterized in that, The jaw osteomyelitis treatment targeted region segmentation module adopts a semantic segmentation network based on the U-Net architecture. The jaw osteomyelitis treatment targeted region segmentation module is configured as follows: after extracting multi-scale features through convolution and downsampling of the encoder path, it then performs upsampling and skip connections of the decoder path, fuses the corresponding layer features of the encoder to combine semantic and detail information, and finally outputs a segmentation probability map of the same size as the input through a 1×1 convolutional layer. The semantic segmentation network is trained using the Dice loss function to optimize the segmentation accuracy of small target regions of lesions in the image.

Images