Dental maxillofacial image recognition method and system based on deep learning

Through deep learning technology, multi-step preprocessing and feature extraction of dental maxillofacial images are carried out, combined with U-Net and dual-stream convolutional networks, multi-modal feature fusion and task collaborative optimization are realized, and a detailed dental maxillofacial image recognition report is generated , solves the problems of low identification accuracy and simplified reporting in the prior art.

CN119006899APending Publication Date: 2024-11-22GUIZHOU MEDICAL UNIV
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411024671.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing dental and maxillofacial image recognition technology performs poorly when dealing with complex structures and diverse lesions, has low computational efficiency, serious overfitting problems, and lacks an effective feature fusion mechanism. The report generation is simplified and cannot meet clinical needs.

Method used

Deep learning-based approaches are adopted, including multi-step preprocessing and enhancement, U-Net network skeleton structure extraction, dual-stream convolutional graph neural network multi-modal feature extraction, multi-task learning framework classification and segmentation, weighted voting integrated learning and Transformer architecture Natural language description analysis to generate detailed dental and maxillofacial image recognition reports.

Benefits of technology

It significantly improves the accuracy and robustness of dental and maxillofacial image recognition, provides detailed and personalized diagnostic reports, meet clinical needs, and improves the interpretability of identification results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006899A_ABST
    Figure CN119006899A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image processing, and discloses a dental-maxillofacial image recognition method and system based on deep learning, which are used for improving the accuracy of dental-maxillofacial image recognition based on deep learning. The method comprises the following steps: performing multi-modal feature extraction processing on an original dental-maxillofacial image and a dental-maxillofacial skeleton structure diagram to obtain fused multi-modal feature representation data, performing classification and segmentation processing on the fused multi-modal feature representation data, obtaining first dental-maxillofacial anatomical structure classification data and a first dental-maxillofacial segmentation diagram, performing processing on the first dental-maxillofacial anatomical structure classification data and the first dental-maxillofacial segmentation diagram, and obtaining second dental-maxillofacial anatomical structure classification data; carrying out boundary processing on the first dentomaxillofacial segmentation map to obtain a second dentomaxillofacial segmentation map; and performing natural language description analysis on the second tooth and jaw surface anatomical structure classification data and the second tooth and jaw surface segmentation image to obtain a descriptive text, performing feature extraction and visualization processing on the second tooth and jaw surface segmentation image to obtain a visualization image, performing report analysis, and generating a tooth and jaw surface image recognition report.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a dental and maxillofacial image recognition method and system based on deep learning. Background Art

[0002] Existing dentofacial image recognition technologies involve cross-applications in multiple fields, including medical imaging, computer vision, and artificial intelligence. Traditional methods are mainly based on image processing and machine learning algorithms, and usually include the following key steps: First, the original dentofacial images are preprocessed, such as denoising, contrast enhancement, and geometric correction. Secondly, various feature extraction techniques, such as gray-level co-occurrence matrix, Gabor filter, local binary pattern, etc., are used to extract effective features from the preprocessed images. Then, classifiers such as support vector machines, random forests, or artificial neural networks are used to classify the extracted features and identify different dentofacial anatomical structures or lesions. At the same time, image segmentation is performed through algorithms such as edge detection, region growing, or graph cuts to mark the areas of interest. Finally, based on the classification and segmentation results, a rule-based method or a simple template generation technique is used to generate a preliminary diagnosis report.

[0003] However, these traditional methods still face many challenges when processing complex dentofacial images. First, manually designed features are often difficult to fully capture the complexity of dentofacial structures and the diversity of lesions, especially when dealing with highly variable pathological features. Secondly, traditional machine learning algorithms often face problems such as low computational efficiency and overfitting when processing large-scale, high-dimensional medical image data. Furthermore, existing methods usually treat classification and segmentation as independent tasks, ignoring the inherent connection between the two, resulting in limited overall performance. In addition, these methods often lack effective feature fusion mechanisms when processing multimodal medical image data, such as considering 2D images and 3D CT scan results at the same time. Finally, existing report generation methods are too simplistic to provide detailed, accurate and personalized diagnostic recommendations, and cannot meet the needs of clinicians for high-quality auxiliary diagnosis. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and system for dental and maxillofacial image recognition based on deep learning, which are used to improve the accuracy of dental and maxillofacial image recognition based on deep learning.

[0005] The present invention provides a method for recognizing dental and maxillofacial images based on deep learning, comprising: preprocessing and enhancing an original dental and maxillofacial image to obtain a standardized image; performing skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental and maxillofacial skeleton structure diagram; performing multimodal feature extraction processing on the original dental and maxillofacial image and the dental and maxillofacial skeleton structure diagram through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data; performing classification and segmentation processing on the fused multimodal feature representation data through a multi-task learning framework to obtain first dental and maxillofacial anatomical structure classification data and a first dental and maxillofacial segmentation diagram; and performing classification and segmentation processing on the fused multimodal feature representation data through a multi-task learning framework to obtain first dental and maxillofacial anatomical structure classification data and a first dental and maxillofacial segmentation diagram. The first dental and maxillofacial anatomical structure classification data is processed to obtain second dental and maxillofacial anatomical structure classification data, and the first dental and maxillofacial segmentation map is subjected to boundary processing through dense conditional random fields to obtain a second dental and maxillofacial segmentation map; the second dental and maxillofacial anatomical structure classification data and the second dental and maxillofacial segmentation map are subjected to natural language description analysis through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text, the second dental and maxillofacial segmentation map is subjected to feature extraction and visualization processing through a convolutional neural network to obtain a visualized image, and the descriptive text and the visualized image are subjected to report analysis to generate a dental and maxillofacial image recognition report.

[0006] In the present invention, the steps of preprocessing and enhancing the original dental and maxillofacial image to obtain a standardized image include: performing multi-scale retinal enhancement processing on the original dental and maxillofacial image to obtain an illumination-balanced dental and maxillofacial image; performing local contrast enhancement processing on the illumination-balanced dental and maxillofacial image to obtain a detail-enhanced dental and maxillofacial image; performing adaptive nonlinear brightness correction processing on the detail-enhanced dental and maxillofacial image to obtain a corrected dental and maxillofacial image; performing multi-scale denoising processing on the corrected dental and maxillofacial image by a wavelet transform algorithm to obtain a denoised dental and maxillofacial image; and performing super-resolution reconstruction processing on the denoised dental and maxillofacial image by a deep learning algorithm. , obtain a high-resolution maxillofacial image; perform content-aware scaling on the high-resolution maxillofacial image to obtain a scaled maxillofacial image; perform adaptive sharpening on the scaled maxillofacial image to obtain an edge-enhanced maxillofacial image; perform data enhancement on the edge-enhanced maxillofacial image through a cyclic generative adversarial network algorithm to obtain an enhanced maxillofacial image set; perform contrast optimization on the enhanced maxillofacial image set through an adaptive histogram equalization algorithm to obtain a contrast-optimized maxillofacial image set; perform feature enhancement on the contrast-optimized maxillofacial image set through a neural network style transfer algorithm to obtain the standardized image.

[0007] In the present invention, the step of performing skeleton structure extraction processing on the standardized image through the U-Net network to obtain a dental and maxillofacial skeleton structure diagram includes: inputting the standardized image into the encoder part of the U-Net network for multi-scale feature extraction processing to obtain a multi-level image feature map; performing feature compression processing on the multi-level image feature map through a deep separable convolution algorithm to obtain a compressed feature map; performing spatial pyramid pooling processing on the compressed feature map to obtain a multi-scale context feature map; performing feature weighting processing on the multi-scale context feature map through an attention mechanism algorithm to obtain a weighted feature map; The feature map is input into the decoder part of the U-Net network for feature reconstruction processing to obtain an initial skeleton feature map; the initial skeleton feature map is subjected to feature enhancement processing through the residual connection algorithm to obtain an enhanced skeleton feature map; the enhanced skeleton feature map is subjected to multi-scale feature fusion processing to obtain a fused skeleton feature map; the fused skeleton feature map is subjected to boundary refinement processing through the conditional random field algorithm to obtain a refined skeleton feature map; the refined skeleton feature map is subjected to threshold segmentation processing to obtain a binary skeleton map; the binary skeleton map is subjected to skeleton extraction processing through the morphological operation algorithm to obtain the maxillofacial skeleton structure map.

[0008] In the present invention, the step of performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure map through a dual-stream convolutional graph neural network to obtain fused multimodal feature representation data includes: inputting the original maxillofacial image into the EfficientNet-B4 backbone network of the dual-stream convolutional graph neural network for feature extraction processing to obtain a first-level image feature map; inputting the maxillofacial skeleton structure map into the graph convolution backbone network of the dual-stream convolutional graph neural network for feature extraction processing to obtain a first-level skeleton feature map; performing feature compression processing on the first-level image feature map through a deep separable convolution algorithm to obtain a compressed image feature map; performing topological structure encoding processing on the first-level skeleton feature map through a graph convolution algorithm to obtain an encoded image. The invention relates to a method for obtaining a multi-modal feature representation data of a multi-modal image. The method comprises the steps of: inputting the compressed image feature map and the encoded skeleton feature map into the attention layer of the dual-stream convolutional graph neural network for feature weighting processing to obtain a weighted feature map; performing feature enhancement processing on the weighted feature map through a residual connection algorithm to obtain an enhanced feature map; inputting the enhanced feature map into the pooling layer of the dual-stream convolutional graph neural network for feature dimensionality reduction processing to obtain a reduced dimensionality feature map; performing feature reorganization processing on the reduced dimensionality feature map through a channel shuffling algorithm to obtain a reorganized feature map; inputting the reorganized feature map into the fully connected layer of the dual-stream convolutional graph neural network for feature fusion processing to obtain a fused feature vector; normalizing the fused feature vector through an adaptive feature normalization algorithm to obtain the fused multi-modal feature representation data.

[0009] In the present invention, the step of classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map includes: performing feature decomposition processing on the fused multimodal feature representation data to obtain classification feature data and segmentation feature data; performing task weight calculation processing on the classification feature data and the segmentation feature data through a dynamic weight allocation algorithm to obtain a task weight coefficient; performing global average pooling processing on the classification feature data to obtain a compressed classification feature vector; performing nonlinear transformation processing on the compressed classification feature vector through a multilayer perceptron to obtain a transformed classification feature; The transformed classification features are subjected to softmax classification processing to obtain the first dental and maxillofacial anatomical structure classification data; the segmentation feature data are subjected to multi-scale feature extraction processing through a dilated convolution algorithm to obtain a multi-scale segmentation feature map; the multi-scale segmentation feature map is subjected to feature pyramid fusion processing to obtain a fused segmentation feature map; the fused segmentation feature map is subjected to size restoration processing through a bilinear upsampling algorithm to obtain an original-size segmentation feature map; the original-size segmentation feature map is subjected to pixel-level classification processing to obtain an initial segmentation probability map; the initial segmentation probability map is subjected to boundary optimization processing through a conditional random field algorithm to obtain the first dental and maxillofacial segmentation map.

[0010] In the present invention, the first dental and maxillofacial anatomical structure classification data is processed by the integrated learning strategy of weighted voting to obtain the second dental and maxillofacial anatomical structure classification data, and the first dental and maxillofacial segmentation map is subjected to boundary processing by dense conditional random field to obtain the second dental and maxillofacial segmentation map, including: performing classification probability distribution analysis processing on the first dental and maxillofacial anatomical structure classification data to obtain a category probability vector; performing multi-model integration processing on the category probability vector by the Boosting algorithm to obtain an integrated classification result; performing confidence threshold screening processing on the integrated classification result to obtain high-confidence classification data; and performing high-confidence classification processing on the high-confidence classification result by the Bayesian average algorithm. The weighted voting processing is performed on the degree classification data to obtain the second maxillofacial anatomical structure classification data; the edge detection processing is performed on the first maxillofacial segmentation map to obtain the initial boundary map; the initial boundary map is segmented by the graph cut algorithm to obtain the segmentation region map; the segmentation region map is subjected to pixel neighborhood analysis processing to obtain the pixel association matrix; the pixel association matrix is ​​subjected to probability distribution estimation processing by the Gaussian mixture model to obtain the pixel category probability map; the pixel category probability map is subjected to iterative conditional mode update processing to obtain the optimized boundary map; the optimized boundary map is subjected to detail repair processing by the morphological operation algorithm to obtain the second maxillofacial segmentation map.

[0011] In the present invention, the second dental and maxillofacial anatomical structure classification data and the second dental and maxillofacial segmentation map are subjected to natural language description analysis through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text, the second dental and maxillofacial segmentation map is subjected to feature extraction and visualization processing through a convolutional neural network to obtain a visualized image, and the descriptive text and the visualized image are subjected to report analysis to generate a dental and maxillofacial image recognition report step, including: performing feature encoding processing on the second dental and maxillofacial anatomical structure classification data to obtain an encoded classification feature vector; performing regional feature extraction processing on the second dental and maxillofacial segmentation map to obtain a regional feature matrix; performing feature fusion processing on the encoded classification feature vector and the regional feature matrix through an attention mechanism to obtain a fused feature representation; and inputting the fused feature representation into a basis The encoder of the encoder-decoder model based on the Transformer architecture performs context encoding processing to obtain context features; the context features are position encoded to obtain position enhanced features; the position enhanced features are input into the decoder of the encoder-decoder model based on the Transformer architecture for sequence generation processing to obtain an initial text sequence; the initial text sequence is semantically optimized by an optimization search algorithm to obtain the descriptive text; the second dental and maxillofacial segmentation map is multi-scaled feature extracted by the convolutional neural network to obtain a multi-level feature map; the multi-level feature map is feature visualized to obtain the visualized image; the descriptive text and the visualized image are structured and integrated by a template matching algorithm to obtain the dental and maxillofacial image recognition report.

[0012] The present invention also provides a dental and maxillofacial image recognition system based on deep learning, comprising:

[0013] An image enhancement module is used to preprocess and enhance the original dental and maxillofacial images to obtain standardized images;

[0014] A structure extraction module is used to perform skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental and maxillofacial skeleton structure diagram;

[0015] A feature extraction module, used for performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure image through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data;

[0016] A data segmentation module, used for classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map;

[0017] a boundary processing module, configured to process the first dental and maxillofacial anatomical structure classification data by using an integrated learning strategy of weighted voting to obtain second dental and maxillofacial anatomical structure classification data, and to perform boundary processing on the first dental and maxillofacial segmentation map by using a dense conditional random field to obtain a second dental and maxillofacial segmentation map;

[0018] A report analysis module is used to perform natural language description analysis on the second dental and maxillofacial anatomical structure classification data and the second dental and maxillofacial segmentation map through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text, perform feature extraction and visualization processing on the second dental and maxillofacial segmentation map through a convolutional neural network to obtain a visualized image, and perform report analysis on the descriptive text and the visualized image to generate a dental and maxillofacial image recognition report.

[0019] In the technical solution provided by the present invention, the original maxillofacial image is preprocessed and enhanced in multiple steps, including multi-scale retinal enhancement, local contrast enhancement, adaptive nonlinear brightness correction, wavelet transform denoising, deep learning super-resolution reconstruction, content-aware scaling, adaptive sharpening, cyclic generative adversarial network data enhancement, adaptive histogram equalization and neural network style transfer, thereby significantly improving the quality and information richness of the input image, and laying a solid foundation for subsequent recognition tasks; secondly, an improved U-Net network is used for skeleton structure extraction, and combined with deep separable convolution, spatial pyramid pooling, attention mechanism, residual connection, multi-scale feature fusion, conditional random field and morphological operation techniques, the skeleton structure of the maxillofacial face can be accurately extracted, and important structural information is provided for multimodal feature extraction; furthermore, a two-stream convolutional graph neural network is designed, and EfficientNet-B4 and graph convolutional network are used to process the original image and the skeleton structure graph respectively, and combined with deep separable convolution, graph convolution, attention mechanism, residual connection, channel shuffling and adaptive Technologies such as feature normalization have achieved effective fusion of multimodal data and made full use of the visual and structural features of the image. In addition, a multi-task learning framework is used to perform classification and segmentation tasks simultaneously. Through technologies such as dynamic weight allocation, global average pooling, multi-layer perceptron, dilated convolution, feature pyramid fusion and conditional random field, the coordinated optimization of classification and segmentation tasks is achieved, thereby improving the overall recognition accuracy. At the same time, the classification and segmentation results are further optimized through the integrated learning strategy of weighted voting and dense conditional random field. Combined with technologies such as Boosting algorithm, Bayesian averaging, graph cut algorithm, Gaussian mixture model and morphological operation, the accuracy and robustness of the recognition results are significantly improved. Finally, the encoder-decoder model based on the Transformer architecture is combined with the convolutional neural network. Through technologies such as attention mechanism, position encoding, optimized search algorithm and template matching algorithm, the transformation from recognition results to natural language description and visualized images is achieved, and a comprehensive, accurate and easy-to-understand maxillofacial image recognition report is generated, which greatly improves the interpretability of the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of a method for dental and maxillofacial image recognition based on deep learning in an embodiment of the present invention;

[0022] Figure 2 Schematic diagram of a dental and maxillofacial image recognition system based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0025] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , Figure 1 is a flow chart of a method for dental and maxillofacial image recognition based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0027] S101, preprocessing and enhancing the original maxillofacial image to obtain a standardized image;

[0028] S102, performing skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental maxillofacial skeleton structure diagram;

[0029] S103, performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure image through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data;

[0030] S104, classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map;

[0031] S105, processing the first dental maxillofacial anatomical structure classification data by using a weighted voting integrated learning strategy to obtain second dental maxillofacial anatomical structure classification data, and performing boundary processing on the first dental maxillofacial segmentation map by using a dense conditional random field to obtain a second dental maxillofacial segmentation map;

[0032] S106. Perform natural language description analysis on the second maxillofacial anatomical structure classification data and the second maxillofacial segmentation map through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text. Perform feature extraction and visualization processing on the second maxillofacial segmentation map through a convolutional neural network to obtain a visualized image. Perform report analysis on the descriptive text and the visualized image to generate a maxillofacial image recognition report.

[0033] It should be noted that the original maxillofacial image is preprocessed and enhanced to obtain a standardized image. This step includes multiple innovative algorithms, such as multi-scale retinal enhancement processing, which effectively enhances the local contrast of the image by simulating the adaptive ability of the human eye retina to changes in illumination; adaptive nonlinear brightness correction processing, which dynamically adjusts the brightness parameters according to the local features of the image, solving the problem of uneven brightness adjustment in traditional methods; wavelet transform denoising processing, which uses the multi-resolution analysis ability of wavelet transform to effectively remove noise while retaining image details; deep learning super-resolution reconstruction processing, which significantly improves image resolution by training deep neural networks to learn the mapping relationship between high and low resolution images. It can better retain image details, improve image quality, and lay the foundation for subsequent recognition tasks. The skeleton structure of the standardized image is extracted by the improved U-Net network to obtain a maxillofacial skeleton structure diagram. The U-Net network is a classic image segmentation network, which is improved by the present invention by introducing technologies such as deep separable convolution, spatial pyramid pooling, and attention mechanism. Depthwise separable convolution greatly reduces the amount of computation by decomposing the standard convolution into depthwise convolution and pointwise convolution; spatial pyramid pooling enhances the network's ability to recognize objects of different sizes by extracting multi-scale features; the attention mechanism helps the network focus on key areas and improves the accuracy of feature extraction. These improvements make skeleton structure extraction more accurate and efficient.

[0034] The original maxillofacial image and the maxillofacial skeleton structure map are processed by multimodal feature extraction using a two-stream convolutional graph neural network to obtain fused multimodal feature representation data. The two-stream network structure uses EfficientNet-B4 and graph convolutional networks to process the original image and skeleton structure map respectively. EfficientNet-B4 is an efficient convolutional neural network that can extract rich visual features at a low computational cost; the graph convolutional network is specifically used to process non-Euclidean data and can effectively capture the topological information of the skeleton structure. By combining these two networks, the comprehensive extraction and fusion of image visual features and structural features are achieved. The fused multimodal feature representation data is classified and segmented using a multi-task learning framework to obtain the first maxillofacial anatomical structure classification data and the first maxillofacial segmentation map. The multi-task learning framework performs classification and segmentation tasks simultaneously, and achieves mutual promotion between tasks through shared feature representation and dynamic weight allocation. For example, the classification task may identify a certain area as a tooth, and this information can assist the segmentation task in more accurately dividing the tooth boundary.

[0035] The first maxillofacial anatomical structure classification data is processed by the weighted voting ensemble learning strategy to obtain the second maxillofacial anatomical structure classification data, and the first maxillofacial segmentation map is processed by the dense conditional random field to obtain the second maxillofacial segmentation map. The ensemble learning strategy integrates the results of multiple classifiers to improve the accuracy and robustness of classification; the dense conditional random field considers the spatial relationship between pixels, optimizes the segmentation boundary, and makes the segmentation result more accurate. The second maxillofacial anatomical structure classification data and the second maxillofacial segmentation map are analyzed by natural language description through the encoder-decoder model based on the Transformer architecture to obtain descriptive text, and the second maxillofacial segmentation map is feature extracted and visualized by the convolutional neural network to obtain a visualized image, and the descriptive text and the visualized image are reported and analyzed to generate a maxillofacial image recognition report. The Transformer architecture effectively captures long-distance dependencies through the self-attention mechanism, is suitable for processing sequence data, and can generate fluent and accurate descriptive text; the convolutional neural network is good at extracting local features of the image and generating intuitive visualization results.

[0036] For example, for a panoramic X-ray containing multiple teeth, the method first enhances the image contrast and removes noise through a preprocessing step to make the tooth contour clearer. Then, the U-Net network accurately extracts the skeleton structure of each tooth. The two-stream convolutional graph neural network extracts features from the original image and the skeleton image respectively, such as the shape, density, and arrangement of the teeth. The multi-task learning framework simultaneously classifies the teeth (such as judging whether they are incisors, molars, etc.) and segments (accurately divides the boundaries of each tooth). Ensemble learning and dense conditional random fields further optimize the classification and segmentation results, such as correctly identifying an occluded wisdom tooth. Finally, the system generates a detailed report containing information such as the type, location, possible lesions (such as caries, missing teeth) of each tooth, and a clearly labeled visualization image.

[0037] By executing the above steps, the original maxillofacial image is preprocessed and enhanced in multiple steps, including multi-scale retinal enhancement, local contrast enhancement, adaptive nonlinear brightness correction, wavelet transform denoising, deep learning super-resolution reconstruction, content-aware scaling, adaptive sharpening, cyclic generative adversarial network data enhancement, adaptive histogram equalization and neural network style transfer, which significantly improves the quality and information richness of the input image, laying a solid foundation for subsequent recognition tasks; secondly, the improved U-Net network is used for skeleton structure extraction, combined with deep separable convolution, spatial pyramid pooling, attention mechanism, residual connection, multi-scale feature fusion, conditional random field and morphological operation, which can accurately extract the skeleton structure of the maxillofacial face, and provide important structural information for multimodal feature extraction; thirdly, by designing a two-stream convolutional graph neural network, EfficientNet-B4 and graph convolutional network are used to process the original image and skeleton structure graph respectively, and combined with deep separable convolution, graph convolution, attention mechanism, residual connection, channel shuffling and adaptive feature extraction, the skeleton structure of the maxillofacial face is extracted, which provides important structural information for multimodal feature extraction. The proposed method uses techniques such as feature normalization to achieve effective fusion of multimodal data and fully utilize the visual and structural features of the image. In addition, a multi-task learning framework is used to perform classification and segmentation tasks simultaneously. Through technologies such as dynamic weight allocation, global average pooling, multi-layer perceptron, dilated convolution, feature pyramid fusion and conditional random field, the coordinated optimization of classification and segmentation tasks is achieved, thereby improving the overall recognition accuracy. At the same time, the classification and segmentation results are further optimized through the integrated learning strategy of weighted voting and dense conditional random field. The accuracy and robustness of the recognition results are significantly improved by combining technologies such as Boosting algorithm, Bayesian averaging, graph cut algorithm, Gaussian mixture model and morphological operation. Finally, the encoder-decoder model based on the Transformer architecture is combined with the convolutional neural network. Through technologies such as attention mechanism, position encoding, optimization search algorithm and template matching algorithm, the transformation from recognition results to natural language description and visualized images is achieved, and a comprehensive, accurate and easy-to-understand maxillofacial image recognition report is generated, which greatly improves the interpretability of the recognition results.

[0038] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0039] (1) Perform multi-scale retinal enhancement processing on the original maxillofacial image to obtain a light-balanced maxillofacial image;

[0040] (2) Performing local contrast enhancement processing on the illumination-balanced maxillofacial image to obtain a detail-enhanced maxillofacial image;

[0041] (3) performing adaptive nonlinear brightness correction processing on the detail enhanced maxillofacial image to obtain a corrected maxillofacial image;

[0042] (4) Performing multi-scale noise reduction processing on the corrected maxillofacial image using a wavelet transform algorithm to obtain a denoised maxillofacial image;

[0043] (5) Perform super-resolution reconstruction on the denoised maxillofacial image using a deep learning algorithm to obtain a high-resolution maxillofacial image;

[0044] (6) performing content-aware scaling processing on the high-resolution maxillofacial image to obtain a scaled maxillofacial image;

[0045] (7) Adaptively sharpening the scaled maxillofacial image to obtain an edge-enhanced maxillofacial image;

[0046] (8) performing data enhancement processing on the edge-enhanced maxillofacial images using a cyclic generative adversarial network algorithm to obtain a set of enhanced maxillofacial images;

[0047] (9) performing contrast optimization processing on the enhanced maxillofacial image set by using an adaptive histogram equalization algorithm to obtain a contrast optimized maxillofacial image set;

[0048] (10) The contrast-optimized maxillofacial image set is enhanced by using a neural network style transfer algorithm to obtain a standardized image.

[0049] Specifically, the original maxillofacial image is subjected to multi-scale retinal enhancement processing to obtain a maxillofacial image with balanced illumination. Multi-scale retinal enhancement is an algorithm that simulates the adaptive ability of the human retina to illumination changes. It effectively balances the problem of uneven illumination in the image by adjusting the local contrast at different scales. Specifically, the algorithm first decomposes the original image into multiple scales, then calculates the local average brightness and contrast at each scale, and performs adaptive adjustments based on the visual characteristics of the human eye. Finally, the results of each scale are fused to obtain an image with balanced illumination.

[0050] Next, the illumination-balanced maxillofacial image is subjected to local contrast enhancement to obtain a detail-enhanced maxillofacial image. This step adaptively adjusts the contrast of each pixel by analyzing the local statistical characteristics of the image, such as the local mean and standard deviation. In specific implementation, a sliding window is used to calculate local statistics, and then the contrast enhancement parameters are dynamically adjusted based on these statistics, thereby highlighting the subtle structures in the image, such as cracks in teeth or difficult-to-detect lesions such as early caries.

[0051] Subsequently, the detail-enhanced maxillofacial image was subjected to adaptive nonlinear brightness correction to obtain the corrected maxillofacial image. The adaptive nonlinear brightness correction algorithm dynamically adjusts the brightness parameters according to the local features of the image, solving the problem of detail loss in the traditional linear brightness adjustment method. The algorithm first estimates the local brightness distribution of the image, and then designs a nonlinear transformation function to perform differential processing on different brightness areas, thereby brightening the dark areas while avoiding overexposure of the bright areas.

[0052] The corrected maxillofacial image is subjected to multi-scale noise reduction processing by wavelet transform algorithm to obtain the denoised maxillofacial image. The wavelet transform algorithm decomposes the image into different frequency components, removes high-frequency noise by threshold processing, and retains the important structural information of the image. The specific steps include: performing wavelet decomposition on the image to obtain low-frequency approximate coefficients and high-frequency detail coefficients; performing soft threshold or hard threshold processing on the high-frequency coefficients; and finally performing inverse wavelet transform to reconstruct the image.

[0053] After that, the denoised maxillofacial images are super-reconstructed using a deep learning algorithm to obtain high-resolution maxillofacial images. The super-resolution reconstruction algorithm uses a deep convolutional neural network to learn the mapping relationship from low-resolution images to high-resolution images. The network structure usually includes three main stages: feature extraction, nonlinear mapping, and reconstruction. Through training with a large number of low-resolution and high-resolution image pairs, complex image prior knowledge is learned, so that the detailed information in the image can be effectively restored.

[0054] The high-resolution maxillofacial image is subjected to content-aware scaling to obtain a scaled maxillofacial image. The content-aware scaling algorithm considers the semantic information of the image and retains important structural features during the scaling process. The algorithm first calculates the saliency map of the image, and then performs non-uniform deformation on the image grid based on the saliency information to ensure that important areas (such as tooth edges and contours) are retained during the scaling process. The scaled maxillofacial image is subjected to adaptive sharpening to obtain an edge-enhanced maxillofacial image. The adaptive sharpening algorithm dynamically adjusts the sharpening parameters according to the local features of the image to enhance the image edges and details. In the specific implementation, the gradient information of the image is first calculated, and then the sharpening strength is adaptively adjusted according to the gradient strength, so as to enhance the edges while avoiding noise amplification caused by over-sharpening.

[0055] The edge-enhanced maxillofacial images were enhanced by data augmentation using the cyclic generative adversarial network algorithm to obtain a set of enhanced maxillofacial images. The cyclic generative adversarial network consists of a generator and a discriminator, which generates diverse image variants through adversarial training. The generator is responsible for generating maxillofacial images under different transformations, while the discriminator attempts to distinguish between generated images and real images. Through this adversarial process, the network can learn the distribution characteristics of real images and thus generate high-quality enhanced images. The enhanced maxillofacial images were contrast-optimized using the adaptive histogram equalization algorithm to obtain a set of contrast-optimized maxillofacial images. The adaptive histogram equalization algorithm performs histogram equalization in local areas, avoiding the problem of over-enhancement that may be caused by global histogram equalization. In specific implementation, the image is divided into multiple small blocks, and histogram equalization is performed on each small block separately. Then, bilinear interpolation is used to fuse the results of adjacent blocks, thereby achieving local adaptive optimization of contrast.

[0056] The contrast-optimized maxillofacial image set was enhanced by the neural network style transfer algorithm to obtain standardized images. The neural network style transfer algorithm learns the feature representation of a specific style and applies these features to the original image. The algorithm uses a pre-trained deep neural network to extract content and style features, and then fuses the style features into the content image through an optimization process, thereby enhancing certain specific features of the image, such as the texture of the teeth or the edge clarity.

[0057] For example, for an oral X-ray containing multiple teeth, the method first balances the brightness of different areas in the image through multi-scale retinal enhancement processing, so that the root structure of the tooth, which was originally blurred due to uneven exposure, becomes clearly visible. For example, for a 512x512 pixel X-ray image, the algorithm calculates the local contrast at 3 different scales (256x256, 128x128, 64x64) and makes adaptive adjustments. Local contrast enhancement and adaptive nonlinear brightness correction further highlight the fine structures of the teeth, such as the boundary between enamel and dentin. In this process, the algorithm uses a sliding window of 15x15 pixels to calculate local statistics and dynamically adjusts the contrast and brightness of each pixel based on these statistics. Wavelet transform denoising uses 3-level wavelet decomposition to effectively remove the high-frequency granular noise generated during X-ray imaging. Deep learning super-resolution reconstruction increases the original 256x256 resolution image to 1024x1024, greatly increasing the image's detail information. Content-aware scaling and adaptive sharpening ensure that key features of teeth are preserved and enhanced during image processing. For example, the algorithm identifies the edges of teeth as key features and prioritizes these areas during scaling. The recurrent generative adversarial network generates 20 variants of tooth images at different angles and lighting conditions, enriching the dataset. Finally, adaptive histogram equalization uses 8x8 local blocks for processing, and neural network style transfer introduces a pre-defined "clear dental X-ray" style, further optimizing the overall visual effect and feature performance of the image.

[0058] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0059] (1) Input the standardized image into the encoder part of the U-Net network for multi-scale feature extraction to obtain a multi-level image feature map;

[0060] (2) Performing feature compression processing on the multi-level image feature map through a deep separable convolution algorithm to obtain a compressed feature map;

[0061] (3) Perform spatial pyramid pooling on the compressed feature map to obtain a multi-scale context feature map;

[0062] (4) Perform feature weighting processing on the multi-scale context feature map through the attention mechanism algorithm to obtain a weighted feature map;

[0063] (5) Input the weighted feature map into the decoder part of the U-Net network for feature reconstruction to obtain the initial skeleton feature map;

[0064] (6) performing feature enhancement processing on the initial skeleton feature map through a residual connection algorithm to obtain an enhanced skeleton feature map;

[0065] (7) performing multi-scale feature fusion processing on the enhanced skeleton feature map to obtain a fused skeleton feature map;

[0066] (8) performing boundary refinement processing on the fused skeleton feature map by using a conditional random field algorithm to obtain a refined skeleton feature map;

[0067] (9) Performing threshold segmentation processing on the refined skeleton feature map to obtain a binary skeleton map;

[0068] (10) The binary skeleton image is subjected to skeleton extraction processing through a morphological operation algorithm to obtain a dental and maxillofacial skeleton structure image.

[0069] Specifically, the standardized image is input into the encoder part of the U-Net network for multi-scale feature extraction to obtain a multi-level image feature map. U-Net is a classic fully convolutional neural network, whose encoder part consists of multiple convolutional layers and pooling layers, which can extract image features layer by layer. In this process, the network forms a multi-scale feature pyramid from low-level edge and texture features to high-level semantic features. The multi-level image feature map is subjected to feature compression processing by the depthwise separable convolution algorithm to obtain a compressed feature map. Depthwise separable convolution is an efficient convolution operation that decomposes the standard convolution into two steps: depthwise convolution and pointwise convolution. Depthwise convolution performs spatial convolution on each input channel separately, while pointwise convolution performs 1x1 convolution on all channels, which greatly reduces the amount of calculation and the number of parameters while maintaining the effectiveness of feature extraction.

[0070] The compressed feature map is processed by spatial pyramid pooling to obtain a multi-scale context feature map. Spatial pyramid pooling is a multi-scale feature extraction technique that performs pooling operations on feature maps at different scales and concatenates the results. This method can capture context information at different scales and is helpful for processing objects of different sizes. The multi-scale context feature map is subjected to feature weighting processing by the attention mechanism algorithm to obtain a weighted feature map. The attention mechanism simulates the selective attention of the human visual system and can automatically learn the weights of important features. In this process, the algorithm calculates the importance score of each feature position and uses these scores to weight the features so that the network can better focus on key areas.

[0071] The weighted feature map is input into the decoder part of the U-Net network for feature reconstruction to obtain the initial skeleton feature map. The decoder part of U-Net gradually restores the spatial resolution of the feature map through a series of upsampling and convolution operations, and uses the feature map of the corresponding layer of the encoder for jump connection to retain more detail information. The initial skeleton feature map is enhanced by the residual connection algorithm to obtain an enhanced skeleton feature map. The residual connection alleviates the gradient vanishing problem of the deep network by adding the input directly to the output, making it easier for the network to learn the identity mapping, thereby retaining more original information.

[0072] The enhanced skeleton feature map is subjected to multi-scale feature fusion processing to obtain a fused skeleton feature map. This step performs weighted fusion of feature maps of different scales and comprehensively utilizes feature information at different levels to obtain a more comprehensive skeleton representation. The fused skeleton feature map is subjected to boundary refinement processing by the conditional random field algorithm to obtain a refined skeleton feature map. The conditional random field is a probabilistic graph model that can consider the spatial relationship between pixels and refine the segmentation results. It makes the segmentation results smoother and more coherent by minimizing the energy function. The refined skeleton feature map is subjected to threshold segmentation processing to obtain a binary skeleton map. Threshold segmentation is a simple and effective image segmentation method that marks areas with pixel values ​​above a certain threshold as foreground (skeleton) and other areas as background. The binary skeleton map is subjected to skeleton extraction processing by the morphological operation algorithm to obtain a dental and maxillofacial skeleton structure map. Morphological operations include erosion, dilation, opening operation, closing operation, etc., which can effectively extract and optimize the skeleton structure.

[0073] For example, for a 512x512 pixel maxillofacial X-ray image, the encoder of the U-Net network first generates multi-scale feature maps ranging from 32x32 to 256x256 through 5 layers of convolution and pooling operations. Depthwise separable convolution replaces each 3x3 standard convolution with a 3x3 depthwise convolution and a 1x1 pointwise convolution, reducing the amount of computation by about 9 times. Spatial pyramid pooling performs pooling at four scales of 1x1, 2x2, 4x4, and 8x8, and concatenates the results into a multi-scale feature map. The attention mechanism calculates weights for each feature channel and spatial position, highlighting the features of key areas such as tooth edges and root apex. The decoder of U-Net gradually restores the feature map to its original resolution by upsampling 4 times. The residual connection adds a short-circuit connection in each decoder block, so that the network depth can reach 20 layers and can still be effectively trained. Multi-scale feature fusion adjusts the number of channels of feature maps of different scales through 1x1 convolution and then performs weighted summation. The conditional random field uses a 5x5 neighborhood system and optimizes the boundary through 10 iterations. The threshold segmentation uses the Otsu method to adaptively select the threshold. Finally, the morphological operation uses a 3x3 structural element for refinement and pruning to obtain a skeleton structure with a single pixel width. This series of processing transforms the originally blurred X-ray image into a clear dental and maxillofacial skeleton structure, laying the foundation for subsequent tooth recognition and lesion detection.

[0074] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0075] (1) The original maxillofacial image is input into the EfficientNet-B4 backbone network of the two-stream convolutional graph neural network for feature extraction to obtain the first-level image feature map;

[0076] (2) Inputting the dentofacial skeleton structure graph into the graph convolution backbone network of the two-stream convolutional graph neural network for feature extraction processing to obtain the first-level skeleton feature graph;

[0077] (3) Performing feature compression processing on the first-level image feature map through a depth-separable convolution algorithm to obtain a compressed image feature map;

[0078] (4) topological structure encoding processing is performed on the first-level skeleton feature map through a graph convolution algorithm to obtain a coded skeleton feature map;

[0079] (5) Inputting the compressed image feature map and the encoded skeleton feature map into the attention layer of the two-stream convolutional graph neural network for feature weighting processing to obtain a weighted feature map;

[0080] (6) Performing feature enhancement processing on the weighted feature map through a residual connection algorithm to obtain an enhanced feature map;

[0081] (7) Inputting the enhanced feature map into the pooling layer of the two-stream convolutional graph neural network for feature dimensionality reduction processing to obtain a reduced dimensionality feature map;

[0082] (8) performing feature reorganization processing on the dimension reduction feature map by using a channel shuffling algorithm to obtain a reorganized feature map;

[0083] (9) Inputting the reorganized feature map into the fully connected layer of the two-stream convolutional graph neural network for feature fusion processing to obtain a fused feature vector;

[0084] (10) The fused feature vector is normalized by an adaptive feature normalization algorithm to obtain fused multimodal feature representation data.

[0085] Specifically, the original maxillofacial image is input into the EfficientNet-B4 backbone network of the two-stream convolutional graph neural network for feature extraction, and the first-level image feature map is obtained. EfficientNet-B4 is an efficient convolutional neural network architecture that balances network depth, width and resolution through a compound scaling method, and can extract rich visual features at a low computational cost. At the same time, the maxillofacial skeleton structure map is input into the graph convolution backbone network of the two-stream convolutional graph neural network for feature extraction, and the first-level skeleton feature map is obtained. The graph convolution network is a neural network specially designed to process graph structure data, which can effectively capture the topological relationship and geometric features in the skeleton structure. The first-level image feature map is subjected to feature compression processing by the depthwise separable convolution algorithm to obtain a compressed image feature map. The depthwise separable convolution decomposes the standard convolution into two steps: depthwise convolution and pointwise convolution, which greatly reduces the amount of calculation and the number of parameters, while maintaining the effectiveness of feature extraction. At the same time, the first-level skeleton feature map is subjected to topological structure encoding processing by the graph convolution algorithm to obtain an encoded skeleton feature map. The graph convolution algorithm updates the features of the central node by aggregating the information of neighboring nodes, thereby effectively encoding the local and global topological information of the skeleton structure.

[0086] The compressed image feature map and the encoded skeleton feature map are input into the attention layer of the two-stream convolutional graph neural network for feature weighting processing to obtain a weighted feature map. The attention mechanism simulates the selective attention of the human visual system and can automatically learn the weights of important features, so that the network pays more attention to key areas. The weighted feature map is feature enhanced by the residual connection algorithm to obtain an enhanced feature map. The residual connection alleviates the gradient vanishing problem of the deep network by adding the input directly to the output, making it easier for the network to learn the identity mapping, thereby retaining more original information. The enhanced feature map is input into the pooling layer of the two-stream convolutional graph neural network for feature dimensionality reduction processing to obtain a reduced dimensionality feature map. The pooling operation reduces the amount of calculation by reducing the spatial resolution of the feature map, while increasing the translation invariance of the feature. Then, the reduced dimensionality feature map is feature reorganized by the channel shuffling algorithm to obtain a reorganized feature map. Channel shuffling is a lightweight feature reorganization method that randomly groups and mixes the channels of the feature map, enhances the information exchange between different channels, and improves the expressiveness of the features. The reorganized feature map is input into the fully connected layer of the two-stream convolutional graph neural network for feature fusion processing to obtain a fused feature vector. The fully connected layer can capture the high-order correlation between features and achieve deep fusion of features of different modalities. Finally, the fused feature vector is standardized by the adaptive feature normalization algorithm to obtain fused multimodal feature representation data. Adaptive feature normalization can dynamically adjust the mean and variance of the feature, making the feature distribution of different modalities more consistent, which is beneficial to subsequent classification and segmentation tasks.

[0087] For example, for a 512x512 pixel maxillofacial X-ray image and its corresponding skeleton structure map, EfficientNet-B4 first uses 8 moving inverted bottleneck (MBConv) blocks to gradually reduce the resolution of the image feature map to 32x32, while increasing the number of channels to 448. The graph convolution network encodes the skeleton map into a 32x32x80 feature map through 5 layers of graph convolution operations, using 16 convolution kernels per layer. Depthwise separable convolution uses 3x3 depthwise convolution and 1x1 pointwise convolution to compress the 448-channel feature map to 224 channels. The graph convolution algorithm uses a 2-hop neighborhood to aggregate information and encode the local structure of the skeleton. The attention mechanism calculates weights for each feature channel and spatial position, highlighting the features of key areas such as tooth edges and root tips. The residual connection adds short-circuit connections in each processing block, so that the network depth can reach 50 layers and can still be effectively trained. Global average pooling reduces the dimension of the feature map to 1x1xC. Channel shuffling randomly divides 224 channels into 16 groups, and permutes channels within each group. The fully connected layer maps the shuffled features to a 1024-dimensional fused feature vector. Adaptive feature normalization dynamically adjusts the mean and variance of the features according to the batch statistics to obtain the final fused multimodal feature representation.

[0088] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0089] (1) Performing feature decomposition processing on the fused multimodal feature representation data to obtain classification feature data and segmentation feature data;

[0090] (2) Performing task weight calculation on the classification feature data and segmentation feature data through a dynamic weight allocation algorithm to obtain a task weight coefficient;

[0091] (3) Perform global average pooling on the classification feature data to obtain a compressed classification feature vector;

[0092] (4) Performing nonlinear transformation on the compressed classification feature vector through a multi-layer perceptron to obtain transformed classification features;

[0093] (5) performing softmax classification processing on the transformed classification features to obtain the first dental and maxillofacial anatomical structure classification data;

[0094] (6) performing multi-scale feature extraction processing on the segmentation feature data through a dilated convolution algorithm to obtain a multi-scale segmentation feature map;

[0095] (7) performing feature pyramid fusion processing on the multi-scale segmentation feature map to obtain a fused segmentation feature map;

[0096] (8) Performing size restoration processing on the fused segmentation feature map through a bilinear upsampling algorithm to obtain the original size segmentation feature map;

[0097] (9) Perform pixel-level classification processing on the original-size segmentation feature map to obtain an initial segmentation probability map;

[0098] (10) The initial segmentation probability map is optimized by the conditional random field algorithm to obtain the first maxillofacial segmentation map.

[0099] Specifically, the fused multimodal feature representation data is subjected to feature decomposition processing to obtain classification feature data and segmentation feature data. Feature decomposition is the process of projecting a high-dimensional feature space into two subspaces, one suitable for classification tasks and the other suitable for segmentation tasks. This decomposition is achieved by learning two projection matrices, making the decomposed features more task-specific. The classification feature data and segmentation feature data are processed by the dynamic weight allocation algorithm to obtain the task weight coefficient. The dynamic weight allocation algorithm dynamically adjusts the importance of each task according to the difficulty of the task and the current performance. Specifically, the algorithm calculates the gradient norm of each task and assigns weights according to the relative size of the gradient norm, so that the network can automatically balance the learning of different tasks during the training process.

[0100] The classification feature data is processed by global average pooling to obtain a compressed classification feature vector. Global average pooling averages the spatial dimensions of each feature map, greatly reducing the number of parameters while retaining the global information of each feature map. The compressed classification feature vector is processed by nonlinear transformation through a multi-layer perceptron to obtain a transformed classification feature. The multi-layer perceptron consists of multiple fully connected layers, each of which is followed by a nonlinear activation function (such as ReLU), which can learn complex nonlinear mapping relationships. The transformed classification features are processed by softmax classification to obtain the first maxillofacial anatomical structure classification data. The Softmax function converts the original category score into a probability distribution, so that the sum of the probabilities of all categories is 1, which is convenient for subsequent classification decisions.

[0101] The segmentation feature data is processed for multi-scale feature extraction through the dilated convolution algorithm to obtain a multi-scale segmentation feature map. The dilated convolution increases the receptive field by inserting "holes" in the convolution kernel without increasing the number of parameters. By using dilated convolutions with different dilation rates, contextual information can be captured at different scales, which is conducive to processing objects of different sizes. The multi-scale segmentation feature map is processed by feature pyramid fusion to obtain a fused segmentation feature map. Feature pyramid fusion fuses feature maps of different scales through top-down paths and lateral connections, so that the final feature map contains both high-level semantic information and low-level detail information.

[0102] The fused segmentation feature map is resized by bilinear upsampling to obtain the original-size segmentation feature map. Bilinear upsampling is an interpolation method that considers four pixels in a 2x2 neighborhood and calculates the value of the interpolation point by weighted average, which can obtain smoother results than the nearest neighbor interpolation. The original-size segmentation feature map is subjected to pixel-level classification to obtain the initial segmentation probability map. Pixel-level classification achieves fine semantic segmentation by applying a small fully connected network to each pixel to map the feature vector to the category probability space.

[0103] The initial segmentation probability map is processed by boundary optimization through the conditional random field algorithm to obtain the first maxillofacial segmentation map. The conditional random field considers the spatial relationship between pixels and optimizes the segmentation result by minimizing the energy function. It can make the segmentation boundary smoother and more accurate, especially near the target boundary.

[0104] For example, for a 512x512 pixel maxillofacial X-ray image containing multiple teeth, the dimension of the fused multimodal features to represent the data is 1024. Feature decomposition projects it into two 512-dimensional subspaces, which are used for classification and segmentation tasks respectively. The dynamic weight allocation algorithm calculates the weight of the classification task as 0.4 and the weight of the segmentation task as 0.6. The classification features are globally average pooled to obtain a 512-dimensional vector, which is then transformed nonlinearly using a three-layer multi-layer perceptron (512-256-128 layers) using the ReLU activation function. The transformed 128-dimensional features are converted into a probability distribution of 32 types of tooth types through the softmax function. For the segmentation task, 3x3 dilated convolutions with three different dilation rates (1, 6, 12) are used to extract multi-scale features, resulting in three 64x64x256 feature maps. Feature pyramid fusion combines these three feature maps with the original high-resolution feature map (128x128x128) to obtain a 128x128x256 fused feature map. The feature map is restored to 512x512x256 by bilinear upsampling, and then a 1x1 convolution is used for pixel-level classification to obtain a 32-channel initial segmentation probability map. Finally, a conditional random field of a 5x5 neighborhood system is used to optimize the segmentation boundary through 10 iterations to obtain the final maxillofacial segmentation map.

[0105] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0106] (1) Performing classification probability distribution analysis on the first dental and maxillofacial anatomical structure classification data to obtain a category probability vector;

[0107] (2) Perform multi-model integration processing on the category probability vector through the Boosting algorithm to obtain the integrated classification result;

[0108] (3) Perform confidence threshold screening on the integrated classification results to obtain high-confidence classification data;

[0109] (4) Using the Bayesian average algorithm to perform weighted voting on the high confidence classification data, the second maxillofacial anatomical structure classification data is obtained;

[0110] (5) performing edge detection processing on the first maxillofacial segmentation image to obtain an initial boundary image;

[0111] (6) Performing region segmentation processing on the initial boundary graph by using a graph cut algorithm to obtain a segmented region graph;

[0112] (7) Performing pixel neighborhood analysis on the segmented region map to obtain a pixel correlation matrix;

[0113] (8) Probability distribution estimation is performed on the pixel association matrix through a Gaussian mixture model to obtain a pixel category probability map;

[0114] (9) performing iterative conditional pattern update processing on the pixel category probability map to obtain an optimized boundary map;

[0115] (10) The optimized boundary map is processed for detail restoration using a morphological operation algorithm to obtain a second maxillofacial segmentation map.

[0116] Specifically, the classification probability distribution analysis and processing of the first maxillofacial anatomical structure classification data is performed to obtain a category probability vector. This step converts the original classification results into probability distributions for each category, providing a basis for subsequent ensemble learning. Specifically, for each sample, the probability of it belonging to each category is calculated to form a probability vector. The category probability vector is subjected to multi-model ensemble processing through the Boosting algorithm to obtain an integrated classification result. Boosting is a powerful ensemble learning method that iteratively trains multiple weak classifiers and combines them into a strong classifier. In this process, each iteration focuses on samples that were misclassified in the previous iteration, thereby gradually improving the overall classification performance. Commonly used Boosting algorithms include AdaBoost and Gradient Boosting.

[0117] The integrated classification results are screened by confidence threshold to obtain high-confidence classification data. This step sets a confidence threshold to screen out highly confident results of the classifier to improve the reliability of the final classification. The specific operation is to compare the highest category probability of each sample with the preset threshold, and only retain samples that exceed the threshold. The high-confidence classification data is weighted voted by the Bayesian averaging algorithm to obtain the second maxillofacial anatomical structure classification data. The Bayesian average takes into account the prior probability and observed data, and can provide more robust estimates in the case of finite samples. In specific implementation, the posterior probability is calculated using the prior probability of each category and the observed classification results, and weighted voting is performed based on the posterior probability.

[0118] The first maxillofacial segmentation map is processed by edge detection to obtain the initial boundary map. Edge detection is a key step in segmentation optimization. Commonly used methods include Sobel operator, Canny edge detection, etc. These methods calculate the gradient of the image and identify areas with drastic brightness changes to obtain the contour of the object. The initial boundary map is processed by region segmentation using the graph cut algorithm to obtain a segmentation region map. The graph cut algorithm converts the image segmentation problem into the minimum cut problem in graph theory, which can effectively segment the image into multiple semantically consistent regions. In specific implementation, a graph model is constructed in which each pixel is regarded as a node, and the weight of the edge represents the similarity of adjacent pixels. Then the minimum cut is solved to obtain the optimal segmentation.

[0119] The segmented region map is processed by pixel neighborhood analysis to obtain a pixel association matrix. This step analyzes the relationship between each pixel and its neighboring pixels and constructs a matrix that represents the degree of association between pixels. Usually, a fixed-size sliding window (such as 3x3 or 5x5) is used to define the neighborhood, and the similarity or difference between the central pixel and the neighboring pixels is calculated. The pixel association matrix is ​​processed by probability distribution estimation using a Gaussian mixture model to obtain a pixel category probability map. The Gaussian mixture model assumes that the data consists of multiple Gaussian distributions and estimates the model parameters using the EM algorithm. Here, it is used to model the probability distribution of pixel associations to obtain the probability that each pixel belongs to each category.

[0120] The pixel category probability map is updated by iterative conditional mode to obtain the optimized boundary map. Iterative conditional mode is an algorithm for optimizing the label field. It iteratively updates the label of each pixel to minimize the overall energy function. In each iteration, the labels of other pixels are fixed and only the label of the current pixel is updated until convergence or the maximum number of iterations is reached. The optimized boundary map is repaired in detail by the morphological operation algorithm to obtain the second maxillofacial segmentation map. Morphological operations include erosion, dilation, opening operation, closing operation, etc., which can effectively remove noise, fill small holes, smooth boundaries, etc., thereby optimizing the details of the segmentation results.

[0121] For example, for a 512x512 pixel maxillofacial X-ray image containing multiple teeth, the first maxillofacial anatomical structure classification data contains a probability distribution of 32 categories. The boosting algorithm uses AdaBoost and trains 50 decision trees as weak classifiers. The confidence threshold is set to 0.85 to filter out high confidence samples. Bayesian averaging uses the frequency of each category in the training set as the prior probability. Edge detection uses the Canny algorithm with thresholds set to 50 and 150. The graph cut algorithm constructs a 512x512 graph, and the edge weights are based on pixel intensity differences. The pixel neighborhood analysis uses a 5x5 sliding window. The Gaussian mixture model assumes 5 Gaussian distribution components. Iterative conditional pattern update is performed for 100 iterations. Morphological operations use a 3x3 structural element for opening and closing operations.

[0122] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0123] (1) performing feature coding processing on the second dental and maxillofacial anatomical structure classification data to obtain a coded classification feature vector;

[0124] (2) performing regional feature extraction processing on the second maxillofacial segmentation image to obtain a regional feature matrix;

[0125] (3) The encoding classification feature vector and the regional feature matrix are fused through the attention mechanism to obtain the fused feature representation;

[0126] (4) Inputting the fused feature representation into the encoder of the encoder-decoder model based on the Transformer architecture for context encoding processing to obtain context features;

[0127] (5) Perform position encoding processing on the context features to obtain position enhanced features;

[0128] (6) Inputting the position enhancement features into the decoder of the encoder-decoder model based on the Transformer architecture for sequence generation processing to obtain an initial text sequence;

[0129] (7) semantically optimizing the initial text sequence through an optimization search algorithm to obtain a descriptive text;

[0130] (8) performing multi-scale feature extraction processing on the second maxillofacial segmentation image through a convolutional neural network to obtain a multi-level feature map;

[0131] (9) Performing feature visualization processing on the multi-level feature map to obtain a visualized image;

[0132] (10) The descriptive text and visual images are structured and integrated through the template matching algorithm to obtain a dental and maxillofacial image recognition report.

[0133] It should be noted that this solution uses innovative deep learning technology to achieve the generation of natural language descriptions and visual reports from the results of maxillofacial image recognition. First, the second maxillofacial anatomical structure classification data is feature encoded to obtain the encoded classification feature vector. Feature encoding is the process of converting discrete classification data into continuous vector representation. Common methods include one-hot encoding and word embedding. Word embedding technology is used here to map each anatomical structure category to a high-dimensional vector space, so that categories with similar semantics are close in this space. The second maxillofacial segmentation map is processed for regional feature extraction to obtain a regional feature matrix. Regional feature extraction aims to capture the visual features and spatial relationships of each region in the segmentation map. In specific implementation, a sliding window or a region proposal network (RPN) is used to locate the region of interest, and then a convolutional neural network is applied to each region to extract features to form a two-dimensional matrix in which each row represents the feature vector of a region.

[0134] The coded classification feature vector and the regional feature matrix are fused through the attention mechanism to obtain a fused feature representation. The attention mechanism allows the model to dynamically focus on different features and regions, achieving effective fusion of classification information and visual information. Specifically, a multi-head attention mechanism is used to calculate the correlation score between the classification feature and each regional feature, and then the regional features are weighted summed based on these scores. The fused feature representation is input into the encoder of the encoder-decoder model based on the Transformer architecture for context encoding to obtain context features. The Transformer encoder consists of multiple layers of self-attention and feedforward neural networks, which can effectively capture long-distance dependencies between features. Each layer of self-attention mechanism allows features to interact with each other to form a rich context representation.

[0135] The context features are positionally encoded to obtain position-enhanced features. Position encoding is a key component of the Transformer architecture. It adds a unique vector to each position, allowing the model to distinguish features at different positions. Common position encoding methods include sine-cosine functions and learnable embedding vectors. The position-enhanced features are input into the decoder of the encoder-decoder model based on the Transformer architecture for sequence generation to obtain the initial text sequence. When generating each word, the Transformer decoder not only pays attention to the output of the encoder, but also uses the generated words as context. Through multiple layers of self-attention and encoder-decoder attention mechanisms, descriptive text is generated word by word.

[0136] The initial text sequence is semantically optimized through an optimization search algorithm to obtain a descriptive text. Common optimization search algorithms include beam search and nucleus sampling. These algorithms retain multiple candidate sequences during the decoding process and select the best output based on the probability distribution to balance the quality and diversity of the generated text.

[0137] At the same time, the second maxillofacial segmentation map is processed with multi-scale feature extraction through a convolutional neural network to obtain a multi-level feature map. Multi-scale feature extraction uses convolution kernels of different sizes or dilated convolutions to capture local details and global structures of the image. In specific implementation, a structure similar to FPN (feature pyramid network) can be used to fuse features at different levels.

[0138] Perform feature visualization on multi-level feature maps to obtain a visualized image. Feature visualization aims to convert abstract feature maps into images that can be understood by humans. Common methods include deconvolution, guided backpropagation, and class activation mapping (CAM). Here, an improved Grad-CAM method is used to generate a heat map to highlight the important areas that the model focuses on.

[0139] Finally, the descriptive text and visual images are structured and integrated through the template matching algorithm to obtain the maxillofacial image recognition report. The template matching algorithm inserts the generated text and images into appropriate locations based on the predefined report template. In specific implementation, regular expressions or natural language processing technology are used to identify key information in the text, and then this information is matched with the placeholders in the template.

[0140] For example, for a 512x512 pixel maxillofacial X-ray image containing multiple teeth, the second maxillofacial anatomical structure classification data contains the probability distribution of 32 categories. Feature encoding maps each category to a 128-dimensional vector space. Regional feature extraction uses ResNet-50 as the backbone to extract 64 128x128 regions of interest, each represented by a 2048-dimensional vector. Attention fusion uses an 8-head attention mechanism to fuse the 128-dimensional classification features with the 64x2048 regional feature matrix into a 64x256 representation. The Transformer encoder contains 6 layers, each with 8 attention heads and a hidden layer dimension of 512. Position encoding uses sine-cosine functions to generate a 512-dimensional position vector. The Transformer decoder also contains 6 layers and uses a beam search algorithm (beam width of 4) to generate a description of up to 100 words. The convolutional neural network uses ResNet-101 for feature extraction and obtains 5 feature maps of different scales. The Grad-CAM method generates a 512x512 heat map that highlights the tooth margins and lesion areas. Finally, the generated descriptive text and heat map visualization results are organized into a structured report using a predefined HTML template.

[0141] The embodiment of the present invention also provides a dental and maxillofacial image recognition system based on deep learning, such as Figure 2 As shown, the deep learning-based dental and maxillofacial image recognition system specifically includes:

[0142] An image enhancement module 201 is used to pre-process and enhance the original dental and maxillofacial image to obtain a standardized image;

[0143] The structure extraction module 202 is used to perform skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental and maxillofacial skeleton structure diagram;

[0144] The feature extraction module 203 is used to perform multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure image through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data;

[0145] A data segmentation module 204 is used to classify and segment the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map;

[0146] The boundary processing module 205 is used to process the first dental and maxillofacial anatomical structure classification data through an integrated learning strategy of weighted voting to obtain second dental and maxillofacial anatomical structure classification data, and perform boundary processing on the first dental and maxillofacial segmentation map through a dense conditional random field to obtain a second dental and maxillofacial segmentation map;

[0147] The report analysis module 206 is used to perform natural language description analysis on the second dental and maxillofacial anatomical structure classification data and the second dental and maxillofacial segmentation map through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text, perform feature extraction and visualization processing on the second dental and maxillofacial segmentation map through a convolutional neural network to obtain a visualized image, and perform report analysis on the descriptive text and the visualized image to generate a dental and maxillofacial image recognition report.

[0148] Through the collaborative work of the above modules, the original maxillofacial images are preprocessed and enhanced in multiple steps, including multi-scale retinal enhancement, local contrast enhancement, adaptive nonlinear brightness correction, wavelet transform denoising, deep learning super-resolution reconstruction, content-aware scaling, adaptive sharpening, cyclic generative adversarial network data enhancement, adaptive histogram equalization and neural network style transfer, which significantly improves the quality and information richness of the input images and lays a solid foundation for subsequent recognition tasks. Secondly, the improved U-Net network is used for skeleton structure extraction, combined with deep separable convolution, spatial pyramid pooling, attention mechanism, residual connection, multi-scale feature fusion, conditional random field and morphological operation, which can accurately extract the skeleton structure of the maxillofacial face and provide important structural information for multimodal feature extraction. Furthermore, by designing a two-stream convolutional graph neural network, EfficientNet-B4 and graph convolutional network are used to process the original image and skeleton structure graph respectively, and combined with deep separable convolution, graph convolution, attention mechanism, residual connection, channel shuffling and adaptive The feature normalization and other technologies are used to realize the effective fusion of multimodal data and make full use of the visual and structural features of the image. In addition, the multi-task learning framework is used to perform classification and segmentation tasks simultaneously. Through dynamic weight allocation, global average pooling, multi-layer perceptron, dilated convolution, feature pyramid fusion and conditional random field and other technologies, the coordinated optimization of classification and segmentation tasks is realized, and the overall recognition accuracy is improved. At the same time, the classification and segmentation results are further optimized through the integrated learning strategy of weighted voting and dense conditional random field. Combined with the Boosting algorithm, Bayesian averaging, graph cut algorithm, Gaussian mixture model and morphological operation and other technologies, the accuracy and robustness of the recognition results are significantly improved. Finally, the encoder-decoder model based on the Transformer architecture is combined with the convolutional neural network. Through the attention mechanism, position encoding, optimization search algorithm and template matching algorithm and other technologies, the transformation from recognition results to natural language description and visual images is realized, and a comprehensive, accurate and easy-to-understand maxillofacial image recognition report is generated, which greatly improves the interpretability of the recognition results.

[0149] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for dental and maxillofacial image recognition based on deep learning, characterized in that: include: Preprocess and enhance the original maxillofacial image to obtain a standardized image; Performing skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental and maxillofacial skeleton structure diagram; Performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure diagram through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data; Classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map; The first dental and maxillofacial anatomical structure classification data is processed by an integrated learning strategy of weighted voting to obtain second dental and maxillofacial anatomical structure classification data, and the first dental and maxillofacial segmentation map is processed by a dense conditional random field to obtain a second dental and maxillofacial segmentation map; The second dento-maxillofacial anatomical structure classification data and the second dento-maxillofacial segmentation map are subjected to natural language description analysis through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text. The second dento-maxillofacial segmentation map is subjected to feature extraction and visualization processing through a convolutional neural network to obtain a visualized image. The descriptive text and the visualized image are subjected to report analysis to generate a dento-maxillofacial image recognition report.

2. The method for dental and maxillofacial image recognition based on deep learning according to claim 1, characterized in that: The step of preprocessing and enhancing the original maxillofacial image to obtain a standardized image includes: Performing multi-scale retinal enhancement processing on the original maxillofacial image to obtain a light-balanced maxillofacial image; Performing local contrast enhancement processing on the illumination-balanced maxillofacial image to obtain a detail-enhanced maxillofacial image; Performing adaptive nonlinear brightness correction processing on the detail-enhanced maxillofacial image to obtain a corrected maxillofacial image; Performing multi-scale noise reduction processing on the corrected maxillofacial image by wavelet transform algorithm to obtain a denoised maxillofacial image; Performing super-resolution reconstruction processing on the denoised maxillofacial image by a deep learning algorithm to obtain a high-resolution maxillofacial image; Performing content-aware scaling on the high-resolution maxillofacial image to obtain a scaled maxillofacial image; Adaptively sharpening the scaled maxillofacial image to obtain an edge-enhanced maxillofacial image; Performing data enhancement processing on the edge-enhanced maxillofacial images by using a cyclic generative adversarial network algorithm to obtain an enhanced maxillofacial image set; Performing contrast optimization processing on the enhanced maxillofacial image set by using an adaptive histogram equalization algorithm to obtain a contrast optimized maxillofacial image set; The contrast-optimized maxillofacial image set is subjected to feature enhancement processing by a neural network style transfer algorithm to obtain the standardized image.

3. The method for dental and maxillofacial image recognition based on deep learning according to claim 1, characterized in that: The step of performing skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental maxillofacial skeleton structure diagram includes: Inputting the standardized image into the encoder part of the U-Net network for multi-scale feature extraction processing to obtain a multi-level image feature map; Performing feature compression processing on the multi-level image feature map by a depth-separable convolution algorithm to obtain a compressed feature map; Performing spatial pyramid pooling processing on the compressed feature map to obtain a multi-scale context feature map; Performing feature weighting processing on the multi-scale context feature map through an attention mechanism algorithm to obtain a weighted feature map; Inputting the weighted feature map into the decoder part of the U-Net network for feature reconstruction processing to obtain an initial skeleton feature map; Performing feature enhancement processing on the initial skeleton feature map by a residual connection algorithm to obtain an enhanced skeleton feature map; Performing multi-scale feature fusion processing on the enhanced skeleton feature map to obtain a fused skeleton feature map; Performing boundary refinement processing on the fused skeleton feature map by a conditional random field algorithm to obtain a refined skeleton feature map; Performing threshold segmentation processing on the refined skeleton feature map to obtain a binary skeleton map; The binary skeleton image is subjected to skeleton extraction processing by a morphological operation algorithm to obtain the dental and maxillofacial skeleton structure image.

4. The method for dental and maxillofacial image recognition based on deep learning according to claim 1, characterized in that: The step of performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure diagram through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data includes: Inputting the original maxillofacial image into the EfficientNet-B4 backbone network of the dual-stream convolutional graph neural network for feature extraction processing to obtain a first-level image feature map; Inputting the dental maxillofacial skeleton structure graph into the graph convolution backbone network of the dual-stream convolutional graph neural network for feature extraction processing to obtain a first-level skeleton feature graph; Performing feature compression processing on the first-level image feature map by a depth-separable convolution algorithm to obtain a compressed image feature map; Performing topological structure encoding processing on the first-level skeleton feature map by using a graph convolution algorithm to obtain an encoded skeleton feature map; Inputting the compressed image feature map and the encoded skeleton feature map into the attention layer of the two-stream convolutional graph neural network for feature weighting processing to obtain a weighted feature map; Performing feature enhancement processing on the weighted feature map by a residual connection algorithm to obtain an enhanced feature map; Inputting the enhanced feature map into the pooling layer of the two-stream convolutional graph neural network to perform feature dimensionality reduction processing to obtain a reduced dimensionality feature map; Performing feature reorganization processing on the dimension reduction feature map by using a channel shuffling algorithm to obtain a reorganized feature map; Inputting the reorganized feature map into the fully connected layer of the two-stream convolutional graph neural network for feature fusion processing to obtain a fused feature vector; The fused feature vector is normalized by an adaptive feature normalization algorithm to obtain the fused multimodal feature representation data.

5. The method for dental and maxillofacial image recognition based on deep learning according to claim 1, characterized in that: The step of classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map includes: Performing feature decomposition processing on the fused multimodal feature representation data to obtain classification feature data and segmentation feature data; Performing task weight calculation processing on the classification feature data and the segmentation feature data by using a dynamic weight allocation algorithm to obtain a task weight coefficient; Performing global average pooling processing on the classification feature data to obtain a compressed classification feature vector; Performing nonlinear transformation processing on the compressed classification feature vector by a multi-layer perceptron to obtain a transformed classification feature; Performing softmax classification processing on the transformed classification features to obtain the first dental and maxillofacial anatomical structure classification data; Performing multi-scale feature extraction processing on the segmentation feature data by using a dilated convolution algorithm to obtain a multi-scale segmentation feature map; Performing feature pyramid fusion processing on the multi-scale segmentation feature map to obtain a fused segmentation feature map; Performing size restoration processing on the fused segmentation feature map by a bilinear upsampling algorithm to obtain an original-size segmentation feature map; Performing pixel-level classification processing on the original-size segmentation feature map to obtain an initial segmentation probability map; The initial segmentation probability map is subjected to boundary optimization processing by a conditional random field algorithm to obtain the first dental and maxillofacial segmentation map.

6. The method for dental and maxillofacial image recognition based on deep learning according to claim 1, characterized in that: The steps of processing the first dental and maxillofacial anatomical structure classification data by using the weighted voting integrated learning strategy to obtain the second dental and maxillofacial anatomical structure classification data, and performing boundary processing on the first dental and maxillofacial segmentation map by using a dense conditional random field to obtain the second dental and maxillofacial segmentation map include: Performing classification probability distribution analysis on the first dental and maxillofacial anatomical structure classification data to obtain a category probability vector; Performing multi-model integration processing on the category probability vector by using a Boosting algorithm to obtain an integrated classification result; Performing confidence threshold screening processing on the integrated classification results to obtain high-confidence classification data; Performing weighted voting processing on the high confidence classification data by using a Bayesian average algorithm to obtain the second maxillofacial anatomical structure classification data; Performing edge detection processing on the first dental and maxillofacial segmentation map to obtain an initial boundary map; Performing region segmentation processing on the initial boundary graph by using a graph cut algorithm to obtain a segmented region graph; Performing pixel neighborhood analysis on the segmented region map to obtain a pixel association matrix; Performing probability distribution estimation processing on the pixel association matrix through a Gaussian mixture model to obtain a pixel category probability map; Performing iterative conditional mode update processing on the pixel category probability map to obtain an optimized boundary map; The optimized boundary map is processed for detail restoration using a morphological operation algorithm to obtain the second maxillofacial segmentation map.

7. The method for dental and maxillofacial image recognition based on deep learning according to claim 6, characterized in that: The steps of performing natural language description analysis on the second maxillofacial anatomical structure classification data and the second maxillofacial segmentation map by using an encoder-decoder model based on a Transformer architecture to obtain a descriptive text, performing feature extraction and visualization processing on the second maxillofacial segmentation map by using a convolutional neural network to obtain a visualized image, and performing report analysis on the descriptive text and the visualized image to generate a maxillofacial image recognition report include: Performing feature coding processing on the second dental and maxillofacial anatomical structure classification data to obtain a coded classification feature vector; Performing regional feature extraction processing on the second maxillofacial segmentation image to obtain a regional feature matrix; Performing feature fusion processing on the encoding classification feature vector and the regional feature matrix through an attention mechanism to obtain a fused feature representation; Inputting the fused feature representation into an encoder of an encoder-decoder model based on a Transformer architecture for context encoding processing to obtain context features; Performing position encoding processing on the context feature to obtain a position enhancement feature; Inputting the position enhancement feature into the decoder of the encoder-decoder model based on the Transformer architecture for sequence generation processing to obtain an initial text sequence; Performing semantic optimization processing on the initial text sequence by using an optimization search algorithm to obtain the descriptive text; Performing multi-scale feature extraction processing on the second maxillofacial segmentation map through the convolutional neural network to obtain a multi-level feature map; Performing feature visualization processing on the multi-level feature map to obtain the visualization image; The descriptive text and the visual image are structured and integrated by a template matching algorithm to obtain the dental and maxillofacial image recognition report.

8. A deep learning-based dental and maxillofacial image recognition system, used to execute the deep learning-based dental and maxillofacial image recognition method according to any one of claims 1 to 7, characterized in that: include: An image enhancement module is used to preprocess and enhance the original dental and maxillofacial images to obtain standardized images; A structure extraction module is used to perform skeleton structure extraction processing on the standardized image through a U-Net network to obtain a dental and maxillofacial skeleton structure diagram; A feature extraction module, used for performing multimodal feature extraction processing on the original maxillofacial image and the maxillofacial skeleton structure image through a two-stream convolutional graph neural network to obtain fused multimodal feature representation data; A data segmentation module, used for classifying and segmenting the fused multimodal feature representation data through a multi-task learning framework to obtain first maxillofacial anatomical structure classification data and a first maxillofacial segmentation map; a boundary processing module, configured to process the first dental and maxillofacial anatomical structure classification data by using an integrated learning strategy of weighted voting to obtain second dental and maxillofacial anatomical structure classification data, and to perform boundary processing on the first dental and maxillofacial segmentation map by using a dense conditional random field to obtain a second dental and maxillofacial segmentation map; A report analysis module is used to perform natural language description analysis on the second dental and maxillofacial anatomical structure classification data and the second dental and maxillofacial segmentation map through an encoder-decoder model based on the Transformer architecture to obtain a descriptive text, perform feature extraction and visualization processing on the second dental and maxillofacial segmentation map through a convolutional neural network to obtain a visualized image, and perform report analysis on the descriptive text and the visualized image to generate a dental and maxillofacial image recognition report.

Citation Information

Cited By

  • Lightweight video semantic communication method

    CN119767030A

  • Tooth image intelligent analysis system and method for oral surgery

    CN120107254A

  • Infantile lymphoma ultrasonic auxiliary diagnosis system combined with artificial intelligence

    CN120727259A

  • An ultrasonic auxiliary diagnosis system for pediatric lymphoma combined with artificial intelligence

    CN120727259B

  • Intelligent orthopedic CT image processing system

    CN120912611A