Method and system for auxiliary diagnosis of osteoporosis through oral cavity panorama image segmentation based on deep learning

Abstract

The invention provides a deep learning-based oral cavity panorama image segmentation method and system for auxiliary diagnosis of osteoporosis, and the method comprises the steps: carrying out the marking and preprocessing of oral cavity panorama image data, and obtaining a sample data set; combining the segmentation effects of the U-Ne t and Re s-Ne t models, taking the U-Ne t as a backbone, sleeving the Re s-Ne t into a segmentation layer, and constructing an initial improved Re s-U-Ne t deep learning model; training is carried out to obtain an improved Re-U-Ne t deep learning model; and constructing an automatic diagnosis framework of the osteoporosis of the oral medical panorama film based on deep learning to obtain an auxiliary diagnosis result. The deep learning algorithm is combined with the oral panoramic film, so that the oral panoramic film can assist in diagnosing osteoporosis, and manpower and material resources can be saved. On the premise that the segmentation effect of the original network model is not reduced, a new module is added. By improving the structure of the convolutional neural network Re-U-Ne t model, a medical image segmentation algorithm with a readable chart is designed and realized, the loss function pair model is optimized and improved, and the evaluation result is more accurate.

Description

Technical Field

[0001] This application relates to the technical field of image segmentation recognition and analysis, and particularly to a method and system for oral panoramic film image segmentation based on deep learning to assist in the diagnosis of osteoporosis. Background Technique

[0002] The underlying logic of medical image segmentation is to use computer image processing technology to segment medical images into different regions, and based on this, extract the required targets. Then, based on the extraction of corresponding image features, higher-level image analysis and understanding are carried out.

[0003] Traditional medical image segmentation methods mainly rely on manually setting feature extractors for extraction and processing. Usually, a series of classic computer vision and image processing methods are used, including: 1. Threshold segmentation: comparing the pixel values in the image with a predefined threshold to segment the image into different regions; 2. Edge detection: identifying edges and contours by detecting the intensity differences between different regions in the image; 3. Region growing and connectivity analysis: combining a group of predefined pixels into regions with similar attributes to extract the structures in the image; 4. Feature extraction: extracting various features in the image, such as texture, shape, color, etc., to help distinguish different structures and tissues; 5. Filters: applying various filters to enhance specific frequency components of the image or remove noise, including Gaussian filters, median filters, etc.; 6. Model fitting: fitting a mathematical model to describe the structures or lesions in the image, such as using circular fitting to identify circular lesions, but this is often affected by hyperparameters and the effect is generally average.

[0004] Medical image segmentation methods based on deep learning mainly obtain feature information through feature extractors such as convolutional neural networks, and can continuously adjust parameters, with relatively high accuracy. This makes its segmentation performance superior to traditional manual marking. Nowadays, deep learning is widely applied in clinical medical image analysis. Recently, scholars such as Benameur N used deep learning to segment ground-glass opacities and consolidations in CT to diagnose pneumonia. There are also scholars who developed networks trained on skin photos using deep learning models such as Xception, Inception-v3, and Res-Net50, which can analyze 20 skin diseases. Applying deep learning to osteoporosis can save manpower and improve its accuracy, but it is not yet popular due to the high learning cost.

[0005] In the early stage, the diagnostic methods for osteoporosis included X-ray radiography, single photon detector, quantitative ultrasound (QUS), etc., with relatively low accuracy. Currently, the diagnosis of osteoporosis mainly relies on traditional manual methods, usually by further examining the complications of osteoporosis to detect it. This traditional method makes the discovery of osteoporosis relatively passive, with high detection costs and high radiation doses. At the same time, the method for evaluating bone mass in oral medicine is to measure the gray scale or trabecular bone density on cone beam computed tomography (CBCT) images, which has high requirements for basic hardware equipment. The above measurement methods have certain errors caused by human subjectivity and are at risk of missed diagnosis and misdiagnosis.

[0006] Therefore, it is an important research content for those skilled in the art to design a method that combines deep learning algorithms with panoramic oral images, enabling panoramic oral images to assist in the diagnosis of osteoporosis, reducing errors caused by human subjectivity, the risks of missed diagnosis and misdiagnosis, and saving manpower and material resources. Summary of the Invention

[0007] In view of this, it is necessary to provide a method for image segmentation of panoramic oral images based on deep learning to assist in the diagnosis of osteoporosis, including being able to reduce invasiveness, precisely locate under ultrasound guidance, reduce errors and complications, be easy to promote and apply, with relatively simple operation, and suitable for promotion and application in more medical settings.

[0008] The technical solution of the present invention combines deep learning algorithms with panoramic oral images, enabling panoramic oral images to assist in the diagnosis of osteoporosis, realizing a medical image segmentation algorithm with readable graphs, respectively using different deep learning models for testing and comparing performance, and adding new modules on the premise of not reducing the segmentation effect of the original network model. By improving the structure of the convolutional neural network Res-U-Net model to process its feature maps and adding a displayable model training curve graph to make it readable. By optimizing and improving the model through the loss function, measurable indicators that can be read are set, enabling the image segmentation network model and the readable network model to be alternately optimized during training. The obtained evaluation performance is similar to the results of specific detection instruments, and the image segmentation effect is similar to that marked by experts.

[0009] In a first aspect, an embodiment of the present application provides a method for image segmentation of panoramic oral images based on deep learning to assist in the diagnosis of osteoporosis, the method comprising:

[0010] S1: Photograph panoramic oral image data. First, use a calibration tool to perform pixel-level calibration on the panoramic oral image data. After obtaining the calibrated image, perform preprocessing on the panoramic oral image data to obtain a sample data set;

[0011] S2: Combine the segmentation effects of the U-Net and Res-Net models. Using U-Net as the backbone, incorporate Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model.

[0012] S3: Randomly divide the sample data set into a training set, a validation set, and a test set.

[0013] S4: Deeply train the initial improved Res-U-Net deep learning model using the training set. Adjust the model's parameters while the model learns the features and patterns of the training data. Evaluate the model's performance on the validation set to adjust the learning rate and optimizer parameters. Evaluate its performance on unseen data using the test set to obtain an improved Res-U-Net deep learning model.

[0014] S5: Build a deep learning-based automatic osteoporosis diagnosis framework for oral medical panoramic films based on the improved Res-U-Net deep learning model.

[0015] S6: Extract features and perform deep learning image segmentation on the patient's oral panoramic film image data through the deep learning-based automatic osteoporosis diagnosis framework for oral medical panoramic films to obtain high-level semantic information. Based on the high-level semantic information and the judgment criteria, obtain an auxiliary diagnosis result.

[0016] Optionally, in another implementation manner of the first aspect of the present invention, in S1: Take oral panoramic film image data. First, use a calibration tool to perform pixel-level calibration on the oral panoramic film image data. After obtaining the calibrated image, perform preprocessing on the oral panoramic film image data to obtain a sample data set, including:

[0017] Manually mark the mental foramen and the width of the mandibular cortex using the ITK-SNAP medical image labeling software.

[0018] Perform data preprocessing, and the data preprocessing includes:

[0019] S1.1: Data cleaning: Remove salt-and-pepper noise through a median filter, reduce high-frequency noise for a smoothed image through a Gaussian filter, and remove noise points in the image through a mean filter.

[0020] S1.2: Screen and eliminate images. The eliminated images include: images with too low quality, incomplete image content, serious artifacts in the image, and images that do not meet the research objectives.

[0021] S1.3: Resample the image through bilinear interpolation.

[0022] S1.4: Mark labels for specific regions or objects in the image, and perform label format conversion and label resampling.

[0023] S1.5: Perform image enhancement processing on the image through a contrast enhancement method, where the contrast enhancement method includes one or more of linear contrast stretching, non-linear contrast stretching, and histogram equalization, and enhance the edges and details of the structures in the panoramic oral radiograph by sharpening the image, and enhance the texture and shape through a filter;

[0024] S1.6: Rotate the panoramic oral radiograph to simulate different shooting angles, and add random noise to the image to simulate the noise that may occur in the actual acquisition process of the image, increasing the size and diversity of the training set;

[0025] S1.7: Perform brightness standardization and contrast standardization on the panoramic oral radiograph by adjusting the brightness and contrast of the image.

[0026] Optionally, in another implementation manner of the first aspect of the present invention, the resampling of the image by bilinear interpolation includes:

[0027] Find the four adjacent points where the target point is located, and form a rectangular area through the four adjacent points;

[0028] Calculate the position of the target point within the rectangular area, and the position is represented by the horizontal and vertical ratios to obtain a coordinate of (u, v);

[0029] Use the method of linear weighting to weight-average the values of the adjacent points according to the position of the target point in the horizontal and vertical directions, change the size and resolution of the image through the interpolation method, and at the same time maintain the original features of the image;

[0030] The weighted average formula f(P) is:

[0031] f(P) = (1 - u)(1 - v)f(Q11) + u(1 - v)f(Q21) + (1 - v)vf(Q12) + uvf(Q22), where u is the horizontal distance of the coordinate, v is the vertical distance of the coordinate, and Q11, Q21, Q12, and Q22 are the coordinates of the four adjacent points.

[0032] Optionally, in another implementation manner of the first aspect of the present invention, the S2: Combine the segmentation effects of the U-Net and Res-Net models, use the U-Net as the backbone, and incorporate the Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model; including:

[0033] The initial improved Res-U-Net deep learning model includes an encoder and a decoder. Extract the high-level semantic features of the image through the encoder, and restore the high-level semantic features to the resolution of the original image through the decoder, and generate a segmentation result;

[0034] Among them, the Res-U-Net encoder is composed of multiple residual blocks, and each residual block contains two convolutional layers and a skip connection; in each residual block, the input feature map is transformed through two convolutional layers to extract the local features of the image, and then added to the skip connection, which directly passes the input feature map to the output of the residual block to effectively transmit information in the network. By stacking multiple residual blocks, the encoder will gradually reduce the resolution of the feature map and extract more abstract and higher-level semantic features;

[0035] The decoder adopts the U-shaped structure of U-Net and is composed of a series of convolutional and upsampling layers. After receiving the feature map output by the encoder, the decoder gradually restores the resolution through a series of operations and fuses global and local features at different levels. In the decoder, each upsampling layer halves the number of channels of the input feature map and doubles the resolution at the same time, enabling the decoder to gradually restore the detailed information from low-level features to high-resolution. The skip connection is used with the feature map of the same resolution as the corresponding layer of the encoder to retain more context information. The last layer of the decoder is a convolutional layer, and the number of output channels matches the number of segmentation target categories. Each pixel on the output channel is converted into a probability value through a non-linear activation function, indicating the probability that the pixel belongs to the target category.

[0036] Optionally, in another implementation manner of the first aspect of the present invention, in step S4: the initial improved Res-U-Net deep learning model is deeply trained with a training set, and the parameters of the model are adjusted while the model learns the features and patterns of the training data. By evaluating the performance of the model on the validation set, the learning rate and optimizer parameters are adjusted. By evaluating its performance on unseen data through the test set, an improved Res-U-Net deep learning model is obtained, and it further includes:

[0037] Adjust a hyperparameter learning rate in the deep learning model to control the speed at which the model parameters are updated in each iteration and determine the step size of the model's update along the gradient direction in the parameter space;

[0038] Adjust the number of samples used to update the model parameters in each training iteration, so that the model performs forward propagation, calculates the loss based on a batch of samples, and updates the model parameters through backpropagation;

[0039] Normalize each feature in each training batch so that its mean is close to 0 and the standard deviation is close to 1, reduce the internal covariate shift, accelerate convergence and improve the stability of the model;

[0040] The three deep learning models of U-Net, Res-Net, and Res-U-Net are respectively used to train the dataset and test their performance. After obtaining a stable loss function Loss, the performance is evaluated by the accuracy ACC, Dice score, and precision PC index, and a line chart is generated through the visualization module.

[0041] Optionally, in another implementation manner of the first aspect of the present invention, the obtaining of the stable loss function Loss includes:

[0042] Solving and evaluating the model by minimizing the loss function, and the loss function formula is:

[0043]

[0044] where N c and N r respectively represent the normalization coefficients, P i 0 represents the probability that the belonging region belongs to the i-th class, P i * represents the value determined according to the label of the belonging region, P i * ∈{0,1}, λ represents the adjustment weight, t i is the coordinate vector of the predicted rectangular region, is the coordinate vector of the rectangular region.

[0045] Optionally, in another implementation manner of the first aspect of the present invention, the obtaining of the auxiliary diagnosis result according to the high-level semantic information and the judgment criterion includes:

[0046] Measuring the mandibular cortical bone width MCW through the model in the osteoporosis automatic diagnosis framework of the oral panoramic radiograph;

[0047] Giving corresponding results according to the average and median reference criteria of the three bone density categories in MCW.

[0048] In a second aspect, an embodiment of the present application provides a system for oral panoramic radiograph image segmentation based on deep learning to assist in the diagnosis of osteoporosis, which is applied to the method for oral panoramic radiograph image segmentation based on deep learning to assist in the diagnosis of osteoporosis as described in the first aspect, including:

[0049] A data processing module, which is used to photograph oral panoramic radiograph image data. First, the oral panoramic radiograph image data is pixel-level calibrated by using a calibration tool. After obtaining the calibrated image, the oral panoramic radiograph image data is preprocessed to obtain a sample dataset;

[0050] A model construction module, which is used to combine the segmentation effects of the U-Net and Res-Net models, use the U-Net as the backbone, incorporate the Res-Net into the segmentation layer, and construct an initial improved Res-U-Net deep learning model;

[0051] A dataset division module, which is used to randomly divide the sample dataset into a training set, a validation set, and a test set;

[0052] A model training module, which is used to deeply train the initial improved Res-U-Net deep learning model through the training set, adjust the parameters of the model while the model learns the features and patterns of the training data, evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through the test set to obtain an improved Res-U-Net deep learning model;

[0053] A diagnostic framework construction module, which is used to construct an automatic osteoporosis diagnosis framework for oral panoramic films based on deep learning based on the improved Res-U-Net deep learning model;

[0054] An auxiliary diagnosis module, which is used to extract features and perform deep learning image segmentation on the patient's oral panoramic film image data through the automatic osteoporosis diagnosis framework for oral panoramic films based on deep learning to obtain high-level semantic information, and obtain an auxiliary diagnosis result according to the high-level semantic information and judgment criteria.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0056] A processor;

[0057] A memory for storing instructions executable by the processor;

[0058] Wherein, when the processor is configured to execute the instructions, it implements the method for assisting in the diagnosis of osteoporosis by segmenting oral panoramic film images based on deep learning as described in the first aspect.

[0059] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program, and the program instructs the device to execute the method for assisting in the diagnosis of osteoporosis by segmenting oral panoramic film images based on deep learning as described in the first aspect.

[0060] In the technical solution provided by the present invention, a method and system for oral panoramic film image segmentation based on deep learning to assist in the diagnosis of osteoporosis are provided. By capturing oral panoramic film image data, first, a calibration tool is used to perform pixel-level calibration on the oral panoramic film image data. After obtaining the calibrated image, preprocessing of the oral panoramic film image data is performed to obtain a sample data set. Combining the segmentation effects of the U-Net and Res-Net models, with U-Net as the backbone and Res-Net incorporated into the segmentation layer, an initial improved Res-U-Net deep learning model is constructed. The sample data set is randomly divided into a training set, a validation set, and a test set. The initial improved Res-U-Net deep learning model is deeply trained through the training set, and the parameters of the model are adjusted while the model learns the features and patterns of the training data. By evaluating the performance of the model on the validation set, the learning rate and optimizer parameters are adjusted. By evaluating its performance on unseen data through the test set, an improved Res-U-Net deep learning model is obtained. Based on the improved Res-U-Net deep learning model, an automatic diagnosis framework for osteoporosis in oral medical panoramic films based on deep learning is constructed. Feature extraction and deep learning image segmentation are performed on the patient's oral panoramic film image data through the automatic diagnosis framework for osteoporosis in oral medical panoramic films based on deep learning to obtain high-level semantic information. According to the high-level semantic information and judgment criteria, an auxiliary diagnosis result is obtained.

[0061] Beneficial effects:

[0062] (1) Combining deep learning algorithms with oral panoramic films enables oral panoramic films to assist in the diagnosis of osteoporosis, which can save manpower and material resources.

[0063] (2) Different deep learning models are respectively used for testing and comparing the performance. Without reducing the segmentation effect of the original network model, a new module is added. The feature map is processed by improving the structure of the convolutional neural network Res-U-Net model, and a displayable model training curve graph is added to make it readable.

[0064] (3) A medical image segmentation algorithm with readable charts is designed and implemented. The loss function is used to optimize and improve the model, and measurable indicators that can be read are set, enabling the image segmentation network model and the readable network model to be alternately optimized during the training process. The obtained evaluation performance is similar to the results of specific detection instruments, and the image segmentation effect is similar to the expert markings.

[0065] (4) Easy to promote and apply: Since no surgery is required and the operation is relatively simple, it is suitable for promotion and application in more medical settings. Description of the drawings

[0066] Figure 1Schematic diagram of the method for oral panoramic image segmentation based on deep learning to assist in the diagnosis of osteoporosis provided by an embodiment of the present application.

[0067] Figure 2 Image resampling provided by an embodiment of the present application.

[0068] Figure 3 Contrast of the image enhancement method provided by an embodiment of the present application.

[0069] Figure 4 Schematic diagram of the Res-U-Net network architecture provided by an embodiment of the present application.

[0070] Figure 5 Automatic diagnosis framework for osteoporosis in oral panoramic films based on deep learning provided by an embodiment of the present application.

[0071] Figure 6 Deep learning segmentation picture process provided by an embodiment of the present application.

[0072] Figure 7 Deep learning model operation flowchart provided by an embodiment of the present application.

[0073] Figure 8 Schematic diagram of the system module for oral panoramic image segmentation based on deep learning to assist in the diagnosis of osteoporosis provided by an embodiment of the present application.

[0074] Figure 9 Schematic diagram of the electronic device provided by an embodiment of the present application. Detailed implementation manners

[0075] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments.

[0076] It should be noted that "at least one" in the embodiments of the present application refers to one or more, and multiple refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0077] It should be noted that in the embodiments of the present application, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be construed as indicating or implying relative importance, nor can they be construed as indicating or implying an order. Features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to mean as an example, illustration or explanation. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0078] Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0079] Embodiment 1

[0080] The present application provides a method for oral panoramic radiograph image segmentation based on deep learning to assist in the diagnosis of osteoporosis, including: S1: Photograph oral panoramic radiograph image data. First, use a calibration tool to perform pixel-level calibration on the oral panoramic radiograph image data. After obtaining the calibrated image, perform preprocessing on the oral panoramic radiograph image data to obtain a sample data set; S2: Combine the segmentation effects of the U-Net and Res-Net models. Using U-Net as the backbone, insert Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model; S3: Randomly divide the sample data set into a training set, a validation set, and a test set; S4: Deeply train the initial improved Res-U-Net deep learning model through the training set. Adjust the parameters of the model while the model learns the features and patterns of the training data. Evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through the test set to obtain an improved Res-U-Net deep learning model; S5: Build an automatic diagnosis framework for osteoporosis in oral medical panoramic radiographs based on the improved Res-U-Net deep learning model; S6: Extract features and perform deep learning image segmentation on the patient's oral panoramic radiograph image data through the automatic diagnosis framework for osteoporosis in oral medical panoramic radiographs based on deep learning to obtain high-level semantic information, and obtain an auxiliary diagnosis result according to the high-level semantic information and judgment criteria.

[0081] Combining deep learning algorithms with panoramic oral radiographs enables panoramic oral radiographs to assist in the diagnosis of osteoporosis, which can save manpower and material resources. Different deep learning models are used for testing and comparing their performance. Without reducing the segmentation effect of the original network model, new modules are added. By improving the structure of the convolutional neural network Res-U-Net model to process its feature maps and adding a displayable model training curve graph to make it readable. A medical image segmentation algorithm with readable charts is designed and implemented. The loss function is used to optimize and improve the model, and measurable indicators that can be read are set, enabling the image segmentation network model and the readability network model to be alternately optimized during training. The obtained evaluation performance is similar to the results of specific detection instruments, and the image segmentation effect is similar to the expert markings. It is easy to promote and apply: Since no surgery is required and the operation is relatively simple, it is suitable for promotion and application in more medical settings.

[0082] Figure 1 Schematic diagram of the method for panoramic oral radiograph image segmentation based on deep learning to assist in the diagnosis of osteoporosis provided by an embodiment of the present application.

[0083] As Figure 1 shown, a method for panoramic oral radiograph image segmentation based on deep learning to assist in the diagnosis of osteoporosis includes:

[0084] S1: Take panoramic oral radiograph image data. First, use a calibration tool to perform pixel-level calibration on the panoramic oral radiograph image data. After obtaining the calibrated image, perform preprocessing on the panoramic oral radiograph image data to obtain a sample data set.

[0085] It can be understood that in this embodiment, the S1: Take panoramic oral radiograph image data. First, use a calibration tool to perform pixel-level calibration on the panoramic oral radiograph image data. After obtaining the calibrated image, perform preprocessing on the panoramic oral radiograph image data to obtain a sample data set, includes:

[0086] Manually mark the mental foramen and the width of the mandibular cortex through the ITK-SNAP medical image marking software;

[0087] Perform data preprocessing, and the data preprocessing includes:

[0088] S1.1: Data cleaning: Remove salt-and-pepper noise through a median filter, reduce high-frequency noise for a smoothed image through a Gaussian filter, and remove noise points in the image through a mean filter.

[0089] It is understandable that data cleaning is an important step in the image preprocessing stage of panoramic dental radiographs. The purpose of this step is to improve the quality of the data, thereby laying a good foundation for subsequent image segmentation and analysis tasks. Data cleaning includes operations such as removing noise, applying filters, enhancing contrast, and data normalization. Each operation plays a crucial role in improving the model's performance and generalization ability.

[0090] Removing noise is the primary link in data cleaning. Dental X-ray image data may be interfered by noise from the imaging device, the scanning process, or the transmission process. This noise exists in the form of speckle, artifacts, motion artifacts, etc., seriously affecting the quality of the image and the accurate display of oral structures. Therefore, removing noise is a key step in improving the image quality in the data cleaning process. Filters are the most commonly used tools for removing noise. The present invention uses median filters, Gaussian filters, and mean filters. Median filters are used to remove salt-and-pepper noise, Gaussian filters are used to smooth the image and reduce high-frequency noise, and mean filters are used to remove noise points in the image.

[0091] S1.2: Screen and eliminate images. The images to be eliminated include: images with too low quality, incomplete image content, serious artifacts in the images, and images that do not meet the research objectives.

[0092] In the preprocessing of panoramic dental radiograph image data, in addition to removing noise and applying filters, it is also necessary to screen and eliminate images, the purpose of which is to remove images that are not suitable for subsequent analysis and processing. The images that need to be eliminated are as follows: Images with too low quality: If the quality of the image is too low, such as excessive noise, too low contrast, etc., it may seriously affect the subsequent image processing and analysis effects. Such images need to be eliminated. Incomplete image content: If the image content is incomplete, such as incomplete imaging and missing information of important oral structures (such as teeth, dental bones, etc.), such images usually cannot provide sufficient information for subsequent analysis and processing, so they need to be eliminated. Serious artifacts in the images: If there are serious artifacts in the images, such as motion artifacts, scanning artifacts, etc., it may affect the authenticity and accuracy of the image. Such images need to be eliminated. Images that do not meet the research objectives: If the content of the image does not meet the research objectives and requirements, such thesis images need to be eliminated. For example, in a project studying adult oral diseases, children's dental X-ray images need to be eliminated.

[0093] Through such screening and elimination steps, it can be preliminarily ensured that the images used are all of high quality, complete in content, and meet the research objectives, which is of great significance for improving the accuracy and efficiency of subsequent image processing and analysis. Data cleaning is an essential step in the preprocessing of panoramic dental radiographs. Through effective data cleaning, the quality of panoramic dental radiograph data can be significantly improved, providing basic data guarantee for subsequent image segmentation and analysis tasks. This is of great significance for improving the accuracy and stability of dental X-ray data segmentation, as well as enhancing the model performance and generalization ability.

[0094] S1.3: Resample the image by bilinear interpolation.

[0095] Since the acquisition devices, parameters, and resolutions of panoramic dental radiographs may vary, this may result in differences in the size and resolution of the obtained images. Therefore, in order to ensure the consistency and fairness of subsequent processing steps, the image is resampled so that all images have the same size and resolution.

[0096] The basic idea of image resampling is to change the size and resolution of the image through interpolation or sampling methods. Interpolation is a method of predicting unknown pixel values based on known pixel values. Several modern mainstream image resampling methods (all set to reduce by 10%), including bilinear interpolation, nearest neighbor interpolation, bicubic interpolation, and Lanczos interpolation, are as Figure 2 shown.

[0097] Specifically, the resampling of the image by bilinear interpolation includes:

[0098] Find the four adjacent points where the target point is located, and form a rectangular area through the four adjacent points;

[0099] Calculate the position of the target point within this rectangular area, and the position is represented by the horizontal and vertical ratios, obtaining a coordinate of (u, v);

[0100] Use the method of linear weighting to weight and average the values of the adjacent points according to the position of the target point in the horizontal and vertical directions, change the size and resolution of the image through the interpolation method, and at the same time maintain the original features of the image;

[0101] The weighted average formula f(P) is:

[0102] f(P) = (1 - u)(1 - v)f(Q11) + u(1 - v)f(Q21) + (1 - v)vf(Q12) + uvf(Q22), where u is the horizontal distance of the coordinate, v is the vertical distance of the coordinate, and Q11, Q21, Q12, and Q22 are the coordinates of the four adjacent points.

[0103] S1.4: Label the specific regions or objects in the image, and perform label format conversion and label resampling.

[0104] Labels (or markings) are descriptions of specific regions or objects in the image, used to indicate specific structures in the image, such as teeth, dental bone, periodontal tissue, etc. The label processing in this study mainly includes two parts: label generation and label preprocessing.

[0105] The generated labels need to be preprocessed to adapt to subsequent image processing tasks. The label preprocessing in this study includes label format conversion and label resampling. However, label generation is a time-consuming task that requires a basic level of expertise and is sometimes affected by subjective factors. Through effective label processing, high-quality and consistent labels can be generated, laying a good foundation for the training and validation of the model.

[0106] S1.5: Perform image enhancement processing on the oral panoramic radiograph through contrast enhancement methods, where the contrast enhancement methods include one or more of linear contrast stretching, non-linear contrast stretching, and histogram equalization, and enhance the edges and details of the structures in the oral panoramic radiograph through sharpening processing, and enhance the texture and shape through filters.

[0107] Due to the different absorption capabilities of X-rays by the structures in the oral panoramic radiograph, such as teeth, dental bone, periodontal tissue, etc., it may lead to insufficient contrast in the image, thus affecting the clarity and detail performance of the image. By enhancing the contrast, the contrast of these structures in the image can be improved, making their boundaries and details clearer and more visible. The contrast enhancement methods include linear contrast stretching, non-linear contrast stretching, histogram equalization, etc., as Figure 3 shown, the contrast of the image enhancement method.

[0108] Through effective image enhancement, the visual effect and contrast of the oral panoramic radiograph can be significantly improved, highlighting the important features and details in the image, which is beneficial for later observation and analysis.

[0109] S1.6: Rotate the oral panoramic radiograph to simulate different shooting angles, and add random noise to the image to simulate the noise that may occur during the actual acquisition process of the image, increasing the size and diversity of the training set.

[0110] Data augmentation increases the size and diversity of the training set by creating variant forms of the original data, thereby enhancing the generalization ability of the model and reducing the risk of overfitting. The data augmentation method used in the present invention includes geometric transformation and random noise injection. Geometric transformation is a common data augmentation method, including translation, rotation, scaling, and flipping operations. These operations can change the position, orientation, and scale of the image, but do not change the content of the image. For example, rotating an oral panoramic radiograph can simulate different shooting angles and increase the robustness of the model to angle changes. Random noise injection is a more advanced data augmentation method that adds random noise to the image to simulate the noise that may occur during the actual acquisition of the image. However, some data augmentation methods are not fully applicable to oral X-ray images. For example, performing color space transformation on an oral panoramic radiograph is meaningless because oral panoramic radiographs are usually grayscale images.

[0111] S1.7: Standardize the brightness and contrast of the oral panoramic radiograph by adjusting the brightness and contrast of the image.

[0112] In the processing of oral panoramic radiographs, the main objective of data normalization is to adjust the brightness and contrast of the image to make it consistent throughout the dataset, thereby improving the training effect and stability of the model. Data normalization usually includes two steps: brightness normalization and contrast normalization. Brightness normalization adjusts the brightness of the image to a fixed range of 0 - 255 through linear scaling. Contrast normalization adjusts the contrast of the image to a fixed range, that is, adjusts the standard deviation of the pixel values to 1. Data normalization can improve the performance and convergence speed of the built model.

[0113] S2: Combine the segmentation effects of the U-Net and Res-Net models, use U-Net as the backbone, and incorporate Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model.

[0114] It can be understood that in this embodiment, the S2: Combine the segmentation effects of the U-Net and Res-Net models, use U-Net as the backbone, and incorporate Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model, includes:

[0115] The initial improved Res-U-Net deep learning model includes an encoder and a decoder. The encoder extracts the high-level semantic features of the image, and the decoder restores the high-level semantic features to the resolution of the original image and generates a segmentation result;

[0116] Among them, the Res-U-Net encoder consists of multiple residual blocks, and each residual block contains two convolutional layers and a skip connection; in each residual block, the input feature map is transformed through two convolutional layers to extract the local features of the image, and then added to the skip connection, which directly passes the input feature map to the output of the residual block to effectively transmit information in the network. By stacking multiple residual blocks, the encoder will gradually reduce the resolution of the feature map and extract more and more abstract and higher-level semantic features;

[0117] The decoder adopts the U-shaped structure of U-Net and consists of a series of convolutional and upsampling layers. After receiving the feature map output by the encoder, the decoder gradually restores the resolution through a series of operations and fuses global and local features at different levels. In the decoder, each upsampling layer halves the number of channels of the input feature map and doubles the resolution at the same time, enabling the decoder to gradually restore the detailed information from low-level features to high-resolution. The skip connection is used with the feature map of the same resolution as the corresponding layer of the encoder to retain more context information. The last layer of the decoder is a convolutional layer, and the number of output channels matches the number of segmentation target categories. Each pixel on the output channel is converted into a probability value through a non-linear activation function, indicating the probability that the pixel belongs to the target category.

[0118] Specifically, as Figure 4 shown in the Res-U-Net network architecture diagram. The encoder is a key component of the Res-U-Net model and is responsible for extracting the high-level semantic features of the input image. In Res-U-Net, the encoder uses a series of residual blocks to build a deep network structure by stacking multiple residual blocks. The residual block consists of two convolutional layers, and there is a skip connection between each convolutional layer. The skip connection allows the input to directly bypass the convolutional layer, retain the original information, and pass it to the output of the block. This solves the problems of gradient disappearance and gradient explosion in deep networks and makes the network easier to train. The stacking of multiple residual blocks forms the structure of the encoder. The output feature map of each residual block has a smaller resolution but contains higher-level semantic information.

[0119] Encoder model structure: The Res-U-Net encoder consists of multiple residual blocks, and each residual block contains two convolutional layers and a skip connection. In each residual block, the input feature map is transformed through two convolutional layers and then added to the skip connection. The skip connection directly passes the input feature map to the output of the residual block to effectively transmit information in the network. By stacking multiple residual blocks, the encoder will gradually reduce the resolution of the feature map and extract more and more abstract semantic features.

[0120] Residual Block: The residual block is one of the core components in the Res-U-Net model. Its design inspiration comes from Res-Net, aiming to solve the gradient problem in deep neural networks and avoid gradient vanishing or gradient explosion. Each residual block consists of two convolutional layers and a skip connection, with the following functions: Gradient Transmission: The skip connection directly transmits the input feature map to the output of the residual block. This design can bypass the non-linear operations of some convolutional layers, enabling the gradient to be transmitted more easily, which is beneficial to the flow of information and the stability of training. Information Preservation: Through the residual connection, the residual block can preserve the detailed information in the original input feature map. This is very important for image segmentation tasks because the segmentation results require good spatial details and boundary information. Feature Extraction: The convolutional layers in each residual block can effectively extract the local features of the image. The stacking of multiple residual blocks allows the network to extract higher-level semantic features, thus better performing the image segmentation task.

[0121] Function of the Encoder: The function of the encoder in the Res-U-Net model is to transform the input image into a semantically rich feature representation through successive downsampling and feature extraction. Structure of the Decoder Model: The decoder is another important part of the Res-U-Net model, consisting of a series of convolutional and upsampling layers. After receiving the feature map output by the encoder, the decoder gradually restores the resolution through a series of operations and fuses global and local features at different levels. In the decoder, each upsampling layer halves the number of channels of the input feature map while doubling the resolution. Such operations enable the decoder to gradually restore the detailed information from low-level features to high resolution. In addition, the decoder also uses the feature map with the same resolution as the corresponding layer of the encoder for skip connection to retain more context information.

[0122] Function of the Decoder: The main function of the decoder in the Res-U-Net model is to restore the feature map extracted by the encoder to the resolution of the original image and generate the final segmentation result. Resolution Restoration: The decoder achieves the gradual restoration of resolution through upsampling layers. Through multiple upsamplings, the decoder can gradually restore the resolution of the feature map to be close to that of the original image, thus restoring the details and boundaries of the image.

[0123] S3: Randomly divide the sample dataset into a training set, a validation set, and a test set.

[0124] S4: Deeply train the initial improved Res-U-Net deep learning model using the training set, adjust the parameters of the model while the model learns the features and patterns of the training data, evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data using the test set to obtain the improved Res-U-Net deep learning model.

[0125] It is understandable that in this embodiment, step S4: deeply train the initial improved Res-U-Net deep learning model with a training set, adjust the model's parameters while the model learns the features and patterns of the training data, evaluate the model's performance on a validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through a test set to obtain the improved Res-U-Net deep learning model. It further includes:

[0126] Adjust a hyperparameter learning rate in the deep learning model to control the speed at which the model's parameters are updated in each iteration and determine the step size for the model to update along the gradient direction in the parameter space;

[0127] Adjust the number of samples used to update the model's parameters in each training iteration, enabling the model to perform forward propagation, calculate the loss based on a batch of samples, and update the model's parameters through backpropagation;

[0128] Normalize each feature in each training batch so that its mean is close to 0 and the standard deviation is close to 1, reducing internal covariate shift, accelerating convergence, and improving the model's stability;

[0129] Apply three deep learning models, namely U-Net, Res-Net, and Res-U-Net, to train the dataset respectively and test their effectiveness. After obtaining a stable loss function Loss, evaluate the performance using the accuracy ACC, Dice score, and precision PC index, and generate a line chart through a visualization module.

[0130] Specifically, obtaining the stable loss function Loss includes:

[0131] Solve and evaluate the model by minimizing the loss function. The loss function formula is:

[0132]

[0133] where N c and N r represent normalization coefficients respectively, P i 0 represents the probability that the belonging region belongs to the i-th class, P i * represents the value determined according to the label of the belonging region, P i * ∈{0, 1}, λ represents the adjustment weight, t i is the coordinate vector of the predicted rectangular region, is the coordinate vector of the rectangular region.

[0134] S5: Construct an automatic osteoporosis diagnosis framework for oral panoramic films based on deep learning using an improved Res-U-Net deep learning model.

[0135] It can be understood that in this embodiment, as Figure 5 shown, construct an automatic osteoporosis diagnosis framework for oral panoramic films based on deep learning.

[0136] S6: Use the automatic osteoporosis diagnosis framework for oral panoramic films based on deep learning to extract features and perform deep learning image segmentation on the patient's oral panoramic film image data to obtain high-level semantic information. According to the high-level semantic information and the judgment criteria, obtain an auxiliary diagnosis result.

[0137] It can be understood that in this embodiment, three deep learning methods are used for training and testing, and the efficiencies of the three are compared. The process of deep learning image segmentation of oral panoramic films and measuring the width of the mandibular cortical bone is as shown in Figure 6 shown.

[0138] Specifically, the comparison of the three deep learning models is as follows in the table:

[0139] Table 1 Comparison of three deep learning models

[0140]

[0141] The specific comparison experiment results are as shown in the above table. Among them, the bold part in the table represents the best result obtained; the experiment results show that the algorithm proposed in this chapter reaches 51.56% in terms of precision (PC) and 81.68% in terms of the Dice coefficient index, which is better than other comparison algorithms. Generally speaking, the performance of the Res-U-Net algorithm in the prediction of oral osteoporosis and the image segmentation of the condylar foramen and mandibular cortical bone is better than other comparison algorithms.

[0142] Figure 7 This is the flowchart of the operation of the deep learning model provided in an embodiment of the present application, and the image is segmented based on the deep learning model.

[0143] It can be understood that in this embodiment, the obtaining of the auxiliary diagnosis result according to the high-level semantic information and the judgment criteria includes:

[0144] Measure the width of the mandibular cortical bone MCW through the model in the automatic osteoporosis diagnosis framework for oral panoramic films; according to the average and median reference criteria of the three bone density categories in MCW, give the corresponding results.

[0145] Measure the width of the mandibular cortical bone MCW through the model in the automatic osteoporosis diagnosis framework for oral panoramic films; according to the average and median reference criteria of the three bone density categories in MCW, give the corresponding results. As shown in Table 2.

[0146] Table 2 MCW and bone density type values

[0147]

[0148] Example 2

[0149] As Figure 8 shown, this application provides an oral panoramic film image segmentation based on deep learning to assist in diagnosing osteoporosis system, which is applied to the method of oral panoramic film image segmentation based on deep learning to assist in diagnosing osteoporosis as described in Example 1, including: a data processing module 11, a model construction module 12, a data set division module 13, a model training module 14, a diagnostic framework construction module 15, and an auxiliary diagnosis module 16.

[0150] It can be understood that in this embodiment, the data processing module 11 is used to capture oral panoramic film image data. First, use a calibration tool to perform pixel-level calibration on the oral panoramic film image data. After obtaining the calibrated image, then perform preprocessing on the oral panoramic film image data to obtain a sample data set.

[0151] It can be understood that in this embodiment, the model construction module 12 is used to combine the segmentation effects of the U-Net and Res-Net models. With U-Net as the backbone, insert Res-Net into the segmentation layer to construct an initial improved Res-U-Net deep learning model.

[0152] It can be understood that in this embodiment, the data set division module 13 is used to randomly divide the sample data set into a training set, a validation set, and a test set.

[0153] It can be understood that in this embodiment, the model training module 14 is used to perform in-depth training on the initial improved Res-U-Net deep learning model through the training set, adjust the parameters of the model while the model learns the features and patterns of the training data, evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through the test set to obtain an improved Res-U-Net deep learning model.

[0154] It can be understood that in this embodiment, the diagnostic framework construction module 15 is used to construct an automatic diagnosis framework for osteoporosis of oral medical panoramic films based on the improved Res-U-Net deep learning model.

[0155] It can be understood that, in this embodiment, the auxiliary diagnosis module 16 is used to extract features from the patient's oral panoramic image data through the osteoporosis automatic diagnosis framework for oral panoramic films based on deep learning and perform deep learning image segmentation to obtain high-level semantic information, and obtain an auxiliary diagnosis result according to the high-level semantic information and judgment criteria.

[0156] Figure 9 is an electronic device provided by an embodiment of the present application. As Figure 9 shown, the electronic device at least includes the following parts: a processor 101, a memory 100, a communication interface 103, and a bus 102.

[0157] In the embodiment of the present application, the memory 100 is used to store executable instructions of the processor 101, and the processor 101 is configured to implement, when executing the instructions, a device module for assisting in diagnosing osteoporosis by performing various deep learning-based oral panoramic film image segmentations as Figure 8 shown.

[0158] In the embodiment of the present application, a computer-readable storage medium includes instructions for instructing a device to execute the method of the first aspect. For example, the instructions instruct the device to execute an artificial intelligence-based device state prediction and risk assessment method shown in the process steps as Figure 1 shown.

[0159] The program operating in the electronic device involved in an embodiment of the present application can be a program for controlling a central processing unit (CPU) and the like to implement the functions of the above-described embodiments related to a solution of the present invention (a program for making a computer function). Then, the information processed by these devices is temporarily stored in a random access memory (RAM) during its processing, and then stored in various ROMs such as a read-only memory (FlashROM) and a hard disk drive (HDD), and read, corrected, and written by the CPU as needed.

[0160] It should be noted that a part of the electronic device of the above-described embodiment can also be implemented by a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer and executed to implement it.

[0161] Note that the "computer" mentioned here refers to a computer built into an electronic device, which is a computer using hardware including an OS, peripheral devices, etc. In addition, the "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer.

[0162] Moreover, the "computer-readable recording medium" may include: a medium that stores a program dynamically for a short time, such as a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line; a medium that stores a program for a fixed time, such as a volatile memory inside a computer of a server or a client in this case. In addition, the above program may be a program for implementing a part of the above functions, and may also be a program that can implement the above functions by combining with a program already recorded in a computer.

[0163] In addition, the electronic device in the above embodiment can also be implemented as an aggregate (device group) composed of multiple devices. Each device constituting the device group may have some or all of the functions or functional blocks of the electronic device in the above embodiment. As the device group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0164] Those of ordinary skill in the art of this technology should recognize that the above embodiments are only used to illustrate the present application, rather than to limit the present application. As long as it is within the scope of the essential spirit of the present application, appropriate changes and variations made to the above embodiments fall within the scope of protection required by the present application.

Claims

1. A method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis, characterized in that: The method comprises: S1: Take oral panoramic film image data, first use the calibration tool to perform pixel-level calibration on the oral panoramic film image data, obtain the calibrated image, and then pre-process the oral panoramic film image data to obtain a sample data set; S2: Combining the segmentation effects of U-Ne t and Res-Ne t models, using U-Ne t as the backbone, inserting Res-Ne t into the segmentation layer, and constructing the initial improved ResU-Ne t deep learning model; S 3: Randomly divide the sample data set into training set, validation set and test set; S 4: Perform deep training on the initial improved Re sU-Ne t deep learning model through the training set, adjust the parameters of the model in the characteristics and patterns of the model learning training data, evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through the test set to obtain the improved Re sU-Ne t deep learning model; S 5: Construct an automatic diagnosis framework for osteoporosis in dentistry panoramic radiographs based on deep learning based on the improved ResU-Ne t deep learning model; S 6: Perform feature extraction and deep learning image segmentation on the patient's oral panoramic radiograph image data through the deep learning-based oral panoramic radiograph osteoporosis automatic diagnosis framework to obtain high-level semantic information, and obtain auxiliary diagnosis results based on the high-level semantic information and judgment criteria.

2. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 1, characterized in that: S1: photographing oral panoramic film image data, first using a calibration tool to perform pixel-level calibration on the oral panoramic film image data, and after obtaining the calibrated image, preprocessing the oral panoramic film image data to obtain a sample data set, including: The mental foramen and mandibular cortical bone width were manually marked using IT KS NAP medical image marking software; Perform data preprocessing, the data preprocessing comprising: S1.1: Data cleaning: Use a median filter to remove salt and pepper noise, use a Gaussian filter to reduce high-frequency noise for smooth images, and use a mean filter to remove noise points in the image; S1.2: Screen and eliminate images. The eliminated images include: images with low quality, incomplete content, serious artifacts, and images that do not meet the research objectives; S1.3: Resample the image by bilinear interpolation; S1.4: Label specific areas or objects in the image, and perform label format conversion and label resampling; S1.5: Performing image enhancement processing on the image by a contrast enhancement method, wherein the contrast enhancement method includes one or more of linear contrast stretching, nonlinear contrast stretching, and histogram equalization, and enhancing the edges and details of the structures in the oral panoramic film by sharpening the image, and enhancing the texture and shape by a filter; S1.6: Rotate the oral panoramic film to simulate different shooting angles, and add random noise to the image to simulate the noise that may occur during the actual image acquisition process, thereby increasing the size and diversity of the training set; S1.7: Normalize the brightness and contrast of the oral panoramic radiographs by adjusting the brightness and contrast of the images.

3. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 2, characterized in that: The resampling of the image by bilinear interpolation comprises: Find the four adjacent points where the target point is located, and form a rectangular area through the four adjacent points; Calculate the position of the target point within the rectangular area, where the position is expressed using the ratio of the horizontal and vertical directions to obtain a (u, v) coordinate; The linear weighted method is used to perform weighted averaging of the values ​​of adjacent points according to the horizontal and vertical positions of the target point, and the image size and resolution are changed by interpolation method while maintaining the original features of the image. The weighted average formula f(P) is: f(P)=(1-u)(1-v)f(Q11)+u(1-v)f(Q21)+(1-V)vf(Q12)+uvf(Q22), where u is the horizontal distance of the coordinate, v is the vertical distance of the coordinate, and Q11, Q21, Q12, and Q22 are the coordinates of four adjacent points.

4. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 3, characterized in that: S2: Combining the segmentation effects of U-Ne t and Res-Ne t models, taking U-Ne t as the backbone, inserting Res-Ne t into the segmentation layer, and constructing an initial improved ResU-Ne t deep learning model; including: The initial improved ResU-Ne t deep learning model includes an encoder and a decoder, the encoder extracts high-level semantic features of the image, the decoder restores the high-level semantic features to the resolution of the original image, and generates a segmentation result; Among them, the ResU-Ne t encoder consists of multiple residual blocks, each of which contains two convolutional layers and a jump connection; in each residual block, the input feature map is transformed by two convolutional layers to extract the local features of the image, and then added with the jump connection, which passes the input feature map directly to the output of the residual block to effectively transfer information in the network. By stacking multiple residual blocks, the encoder will gradually reduce the resolution of the feature map and extract more and more abstract and higher-level semantic features; The decoder adopts the U-shaped structure of U-Ne t, which consists of a series of convolution and upsampling layers. After receiving the feature map output by the encoder, the decoder gradually restores the resolution through a series of operations and fuses global and local features at different levels. In the decoder, each upsampling layer halves the number of channels of the input feature map and doubles the resolution, so that the decoder can gradually restore the detailed information from low-level features to high-resolution, and uses jump connections with feature maps of the same resolution as the corresponding layer of the encoder to retain more contextual information. The last layer of the decoder is a convolutional layer, and the number of output channels matches the number of segmentation target categories. Each pixel on the output channel is converted into a probability value through a nonlinear activation function, indicating the probability that the pixel belongs to the target category.

5. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 4, characterized in that: S4: performing deep training on the initial improved ResU-Ne t deep learning model through the training set, adjusting the parameters of the model in the features and patterns of the model learning training data, evaluating the performance of the model on the validation set, adjusting the learning rate and optimizer parameters, and evaluating its performance on unseen data through the test set to obtain the improved Res-U-Ne t deep learning model, and also includes: Adjust the learning rate, a hyperparameter in the deep learning model, to control the speed at which the model parameters are updated in each iteration and determine the step size of the model update along the gradient direction in the parameter space; Adjust the number of samples used to update the model parameters in each training iteration so that the model performs forward propagation based on a batch of samples, calculates the loss, and updates the model parameters through backpropagation; Normalize each feature in each training batch so that its mean is close to 0 and its standard deviation is close to 1, reducing internal covariate shift, accelerating convergence and improving model stability; Three deep learning models, U-Ne t, Res-Ne t, and ResU-Ne t, were used to train the data sets and test their effectiveness. After obtaining the stable loss function Loss, the performance was evaluated by accuracy ACC, Dice score, and precision PC index, and a line graph was generated through the visualization module.

6. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 5, characterized in that: The method of obtaining a stable loss function Loss includes: The model is solved and evaluated by minimizing the loss function, which is formulated as: Among them, N c 、N r Respectively represent the normalization coefficient, P i 0 represents the probability that the region belongs to the i-th category, P i * Represents the value determined by the label of the region to which it belongs, P i * ∈{0,1}, λ represents the adjustment weight, t i is the coordinate vector of the predicted rectangular area, is the coordinate vector of the rectangular region.

7. The method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis according to claim 6, characterized in that: The auxiliary diagnosis results are obtained according to the high-level semantic information and judgment criteria, including: The mandibular cortical bone width MCW was determined using a model in the framework of the automatic diagnosis of osteoporosis in panoramic dental radiographs; The corresponding results are given according to the mean and median reference standards of the three bone density categories in MCW.

8. A system for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis, applied to the method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis as claimed in any one of claims 1 to 7, characterized in that: include: The data processing module is used to capture oral panoramic film image data, first use the calibration tool to perform pixel-level calibration on the oral panoramic film image data, obtain the calibrated image, and then perform oral panoramic film image data preprocessing to obtain a sample data set; The model building module is used to combine the segmentation effects of U-Ne t and Res-Ne t models, using U-Ne t as the backbone, inserting Res-Ne t into the segmentation layer, and building the initial improved ResU-Ne t deep learning model; The data set partitioning module is used to randomly partition the sample data set into training set, validation set and test set; A model training module is used to perform deep training on the initial improved Re sU-Ne t deep learning model through the training set, adjust the parameters of the model in the characteristics and patterns of the model learning training data, evaluate the performance of the model on the validation set, adjust the learning rate and optimizer parameters, and evaluate its performance on unseen data through the test set to obtain the improved Re sU-Ne t deep learning model; The diagnostic framework building module is used to build an automatic diagnostic framework for osteoporosis in dentistry panoramic radiographs based on deep learning based on the improved ResU-Ne t deep learning model; The auxiliary diagnosis module is used to perform feature extraction and deep learning image segmentation on the patient's oral panoramic film image data through the deep learning-based oral panoramic film osteoporosis automatic diagnosis framework to obtain high-level semantic information, and obtain auxiliary diagnosis results based on the high-level semantic information and judgment criteria.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis as described in any one of claims 1 to 7 when executing the instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and the program instructs a device to execute the method for deep learning-based oral panoramic radiograph image segmentation to assist in the diagnosis of osteoporosis as described in any one of claims 1 to 7.

Images