A tool for quantitative analysis of fine anatomical structures in nanometer-level images based on fine-grained networks

By constructing a fine-grained network-based image fine anatomical structure analysis tool, the problems of large errors and time-consuming and labor-intensive quantitative analysis of image fine anatomical structures have been solved, realizing efficient and automated image analysis, which is suitable for mobile-assisted diagnosis and promotes the equalization of medical resources.

CN116246755BActive Publication Date: 2026-11-17HOSPITAL OF STOMATOLOGY SUN YAT SEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310041744.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-28
Publication Date
2026-11-17
Estimated Expiration
2043-01-28

AI Technical Summary

Technical Problem

In existing technologies, the quantitative analysis of anatomical structures in medical images at the millimeter-micrometer level suffers from problems such as large errors in manual analysis, time-consuming and labor-intensive processes, difficulty in standardization, and low accuracy. Furthermore, existing artificial intelligence tools are unable to effectively locate and extract fine anatomical structural features from images, and data annotation costs are high.

Method used

We construct a fine-grained network-based image-based anatomical structure analysis tool. Through a standardized image database, a fine-grained image recognition network, and weakly supervised learning, we achieve efficient quantitative analysis of fine-grained anatomical structures in images, including data input, feature extraction, fusion, and quantitative analysis, adapting to the needs of different specialties.

Benefits of technology

It enables efficient and automated quantitative analysis of detailed anatomical structures in images, reduces physician error, lowers labor costs, improves medical efficiency, is suitable for mobile-assisted diagnosis, and promotes the balanced development of medical resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116246755B_ABST
    Figure CN116246755B_ABST
Patent Text Reader

Abstract

This invention provides a millimeter-micron level image fine anatomical structure analysis tool based on fine-grained networks. It constructs a standardized image fine anatomical structure database, develops an artificial intelligence analysis mode for millimeter-micron level fine anatomical structures adapted to different specialties based on a fine-grained image quantification analysis network, and integrates these into a medical imaging millimeter-micron level fine image structure intelligent analysis system. This invention utilizes deep learning technology, eliminating the need for extensive human and time-consuming dataset annotation, and enables efficient, automated, and quantitative analysis of image fine anatomical structures. It solves the problems of artificial intelligence's difficulty in locating fine anatomical structures in imaging, fully extracting image features from small regions, and quantifying image features, effectively reducing the workload of clinicians and improving medical efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field This patent relates to the field of medical image analysis based on deep learning networks, specifically a tool for analyzing fine anatomical structures in millimeter-micron level images based on fine-grained networks. Background Technology Medical imaging provides indispensable information for clinical diagnosis, disease screening, treatment planning, and prognosis. With the advent of precision medicine, the clinical significance of numerous millimeter- to micrometer-level fine anatomical structures in medical imaging is being increasingly explored. Examples include the size of aneurysms, the number of small bronchi, the size of pulmonary nodules, and the classification of the labial bone walls of the upper anterior teeth. While the dimensions of these specialized fine anatomical structures are often less than 10 mm, their quantitative and semi-quantitative characteristics, such as number, size, classification, morphology, and location, have significant clinical guiding value. Clinically, the probability of benign or malignant aneurysms can be predicted based on whether their diameter exceeds 6 mm in CT images. The size of cerebral aneurysms in angiography determines the appropriate intervention method; aneurysms with a diameter less than 7 mm have a lower risk of rupture and tend to be treated conservatively. The number of small bronchi with a diameter less than 2 mm in lung CT scans is related to physiological lung function loss, idiopathic pulmonary fibrosis, and other physiological and pathological changes. Early detection and treatment of subtle brain metastases from malignant tumors will improve patient survival and quality of life. The thickness classification of the labial bone walls of the upper anterior teeth is closely related to immediate implant decisions and prognosis. Therefore, quantitative and semi-quantitative analysis of detailed anatomical structures is increasingly becoming a focus of attention for various clinical specialties.

[0001] Currently, the quantitative analysis of millimeter-micron level anatomical structures in imaging is mainly conducted manually by physicians, which is influenced by numerous factors, including physician factors (experience in reading images, professional background knowledge, visual resolution, etc.), imaging equipment factors (image resolution and contrast, artifacts and noise, image preprocessing, etc.), and patient factors (variations or omissions in anatomical structures, imaging position, etc.). Compared with the quantitative analysis of large-scale anatomical structures, the quantitative analysis of fine anatomical structures remains a challenging problem in clinical practice. First, the quantitative analysis of millimeter-micron level anatomical structures in imaging is affected by the aforementioned factors, and manual analysis is subject to subjective errors, making it difficult to establish standardized analysis procedures and reproducible results, thus limiting the further optimization and development of precision medicine. Second, there are significant individual differences in millimeter-micron level anatomical structures in imaging, and manual quantitative analysis consumes a lot of time and manpower, which is not conducive to the full utilization of medical resources and improving medical efficiency. In addition, fine anatomical structures occupy a relatively small area in the image, and are affected by factors such as image resolution and artifacts. Inexperienced physicians often find it difficult to filter out redundant interference information and accurately locate and analyze target structures, leading to a high rate of misdiagnosis and missed diagnosis.

[0002] With the development of deep learning, some studies have attempted to use artificial intelligence methods to achieve intelligent quantitative analysis of fine anatomical structures. For example, U-net is used to segment and measure the size of cerebral aneurysms on 2D MRI images; ResNet is used to predict the thickness of the retinal nerve fiber layer on SD-OCT images to determine if a patient has glaucoma. However, these AI analysis tools still have many shortcomings: First, the fine anatomical structures in micrometer-millimeter level images appear as pixel-to-subpixel features in medical images, making it difficult for models to accurately extract and quantify these features; second, these tools require manual segmentation of regions of interest for machine training, resulting in high human and time costs for data annotation; furthermore, the diagnostic accuracy of these tools varies, with most remaining unsatisfactory. Currently, there is a lack of a widely applicable paradigm for the quantitative analysis of fine anatomical structures using AI, which is inconsistent with the trend towards digital and precision medicine.

[0003] Fine-grained networks are neural networks designed for detailed object analysis in two-dimensional images. Compared to conventional deep learning models that utilize global information, fine-grained networks focus more on discriminative local information, thus exhibiting significantly superior region localization and feature extraction capabilities, demonstrating great potential in the analysis and discrimination of fine anatomical structures. Fine-grained image recognition, based on the differentiation of basic categories, achieves more refined subclassification. Essentially, it locates discriminative local regions of the target, extracts features from these regions, and trains the classifier. However, current fine-grained image recognition methods are only used for the identification and classification of fine objects, and methods for the quantitative analysis of fine anatomical structures are still lacking. Summary of the Invention To alleviate the burden on clinicians in analyzing detailed anatomical structures in images, reduce errors from manual measurement and analysis, improve the fine feature extraction capabilities and accuracy of deep learning models, and achieve quantitative analysis of detailed anatomical structures in images, this invention utilizes a millimeter-micrometer level image detailed anatomical structure analysis tool based on fine-grained networks, aiming to solve at least one of the technical problems existing in the prior art: 1. By constructing a standardized database of detailed anatomical structures in images, the problem of errors between image samples caused by differences in imaging equipment can be solved; 2. By constructing and training a fine-grained image analysis network tool, we can solve the problems that artificial intelligence struggles to locate fine anatomical structures in imaging, fully extract image features from small regions, and quantify and analyze image features. 3. Weakly supervised learning mode is adopted to solve the problem that the previous artificial intelligence image detailed anatomical structure analysis required a lot of manpower and time for data annotation; 4. Based on fine-grained image analysis networks, develop artificial intelligence analysis modes for millimeter-micron level fine anatomical structures that are adapted to different specialties, and integrate them into a medical imaging millimeter-micron level fine image structure intelligent analysis system.

[0004] Therefore, the technical solution adopted by the present invention is as follows: A tool for analyzing fine anatomical structures in millimeter-micron level images based on fine-grained networks includes the following steps: S1. Construct a standardized database of detailed anatomical structures from images. S1.1 Standardized Collection of Medical Images Collect imaging data required by various specialties in the diagnosis and treatment of diseases. During the collection of imaging data, professional personnel need to conduct quality control of the images, screen out blurry images and images containing artifacts, and ensure that the collected imaging data can accurately reflect the quantitative and semi-quantitative characteristics of the corresponding fine anatomical structures. S1.2, Construct a standardized and detailed anatomical structure image database: Based on the source and type of the imaging data, the standardized imaging data collected in step S1.1 are classified and stored. According to the analysis needs of different specialties for the corresponding fine anatomical structures, the most favorable cross-section for diagnosis and feature observation is selected, and the image is cropped to obtain standard two-dimensional images of fine anatomical structures. S1.3, Module for Extraction, Measurement, and Annotation of Fine Anatomical Structures: The standard two-dimensional image of the fine anatomical structure obtained in step S1.2 is imported into the data quantification analysis software. The indicators are extracted, measured and labeled by systematically trained personnel to obtain quantitative data of the fine anatomical structure. The size of the fine anatomical structure is usually less than 10 mm. According to the quantitative data and clinical semantics, the fine anatomical structure is assigned a corresponding semi-quantitative category label. The extracted quantitative data and / or semi-quantitative labels are saved in tabular or text format in a specified path. S1.4 Construct a standardized database of detailed anatomical structures from images: The standard two-dimensional images of fine anatomical structures obtained in step S1.2 are matched with the quantitative data and semi-quantitative labels obtained in step S1.3. The patient information from which the images originate is used as the primary key to associate the standard two-dimensional images, quantitative data and semi-quantitative labels belonging to that patient, thereby constructing a standardized image database of fine anatomical structures. S2. Construct a module for quantitative analysis of fine anatomical structures in images equipped with a fine-grained image recognition network. This document describes a module for the detailed anatomical structure quantification analysis of fine-grained image recognition network models. These networks primarily consist of: a data input unit, a feature extraction unit, a feature fusion unit, and a data quantification analysis unit. Fine-grained image recognition network models include BiCNN, RCNN, RACNN, MA-CNN, and PA-CNN. Taking BiCNN (Bilinear Convolutional Neural Networks, a fine-grained image recognition network based on high-order feature complementarity) as an example, the BiCNN model... B Specifically, it includes: feature extraction unit f Feature fusion unit P and data quantitative analysis unit C As shown below: ; S2.1 Constructing the data input unit Python statements are used to automatically read the two-dimensional images and corresponding data in the standardized image fine anatomical structure database constructed in step S1.4. The data input unit converts the two-dimensional images in the standardized fine anatomical structure database into matrix information and inputs it into the feature extraction unit. S2.2 Constructing Feature Extraction Units The convolutional neural network backbone is used as the feature extraction unit. In addition, a feature extraction enhancement module, including APN and / or RPN, can be added. The matrix information in the data input unit described in step S2.1 is input into the convolutional neural network backbone to obtain a feature map containing quantitative information of fine anatomical structure. For example, if two parallel CNN network backbones are used as feature extraction units, the matrix information in the data input unit described in step S2.1 is input into the two parallel CNN network backbones to obtain two feature maps containing quantitative information of fine anatomical structure. S2.3 Constructing Feature Fusion Units The feature fusion unit includes a bilinear pooling layer, a gradient boosting loss function, and / or a fully connected layer. The detailed anatomical structure quantification information feature map obtained in step S2.2 is input into the feature fusion unit for feature fusion and dimensionality reduction operations, ensuring that important features are not lost. Taking a bilinear pooling layer and a fully connected layer as an example, the detailed anatomical structure quantification information feature map obtained in step S2.2 is input into the feature fusion unit to obtain the same image. I The same spatial location in the same position of two feature maps l The outer product of corresponding features captures the pairwise relationships between feature channels, enhancing the network's discriminative ability, as shown below: ; Then all positions l The feature outer product matrices are added together to obtain the fused feature map. (I) As shown below: ; Then fuse the features (I) Perform vector expansion and L 2. Regularization operation to ensure that important features are not lost, as shown below:

[0005]

[0006] ; S2.4 Constructing a data quantification analysis unit The softmax layer is used as the data quantification analysis unit: the fusion features obtained in step S2.3 are input into the softmax layer, the softmax layer maps the output of the above unit to the (0,1) interval, calculates the selected loss function, obtains the prediction probability of different categories, and obtains the quantification analysis results of multiple fine anatomical structures by giving a threshold T, including quantitative data and semi-quantitative data. S3. Training a fine-grained image recognition module capable of quantitative analysis of detailed anatomical structures in images. S3.1 Constructing a Quantitative Analysis Module for Detailed Anatomical Structures The training sample set is fed into the image fine anatomical structure quantification analysis module equipped with a fine-grained image recognition network as described in step S2 for training and verification, thereby constructing the image fine anatomical structure quantification analysis module. S3.2 Performance Evaluation of the Quantitative Analysis Module for Detailed Anatomical Structures Input the test dataset into the image fine anatomical structure quantitative analysis module described in step S3.1, and evaluate the output quantitative analysis results; S4. Construct an intelligent analysis system for fine image structures at the millimeter-micrometer level. Based on the quantitative analysis module of fine anatomical structure of the image obtained in step S3, an intelligent analysis system for fine image structure at the millimeter-micrometer level is constructed.

[0007] In a further improvement, in step S1.1, the imaging data must accurately reflect the quantitative and semi-quantitative characteristics of the quantity, size, classification, morphology, and location of the corresponding fine anatomical structures.

[0008] In a further improvement, in step S1.1, the sources of imaging data include CT, CBCT, PET-CT, MRI, and / or ultrasound images of the US type.

[0009] In a further improvement, in step S1.2, before classifying and storing the image data, the image data is preprocessed, including image cropping, rotation, and contrast adjustment.

[0010] In a further improved scheme, the quantitative data in step S1.3 includes the size, number, and area of ​​fine anatomical structures; the semi-quantitative category labels include the classification, subclass, morphology, and location of fine anatomical structures.

[0011] In a further improved scheme, the convolutional neural network backbone used in step S2.2 includes LeNet-5, Vgg16, ResNet, DenseNet, or GoogleNet convolutional neural network models; the parallel convolutional neural network backbone in step S2.2 includes multiple convolutional layers and max pooling layers; after inputting the matrix information in the data input unit described in step S2.1 into the convolutional neural network backbone, the convolutional neural network backbone performs feature extraction and downsampling through convolutional layers and max pooling layers to obtain feature maps containing quantitative information of fine anatomical structures. For example, after inputting the matrix information in the data input unit described in step S2.1 into two parallel CNN network backbones, the two CNN network backbones perform feature extraction and downsampling through convolutional layers and max pooling layers to obtain two feature maps containing quantitative information of fine anatomical structures.

[0012] In a further improved scheme, in step S2.4, the quantitative analysis results of the fine anatomical structure include quantitative data containing size, area, and quantity, as well as semi-quantitative data containing classification, subclass, location, and morphology.

[0013] In a further improvement, in step S3.1, the training sample set is fed into the image fine anatomical structure quantification analysis module equipped with the fine-grained image recognition network described in step S2 for training and verification. Data augmentation, resampling, K-fold cross-validation and / or model ensemble techniques are used to improve the generalization performance and prediction performance of the model.

[0014] In a further improved scheme, in step S3.2, the performance evaluation of the quantitative analysis results includes the following performance evaluation indicators: mean absolute error, root mean square error, accuracy, precision, sensitivity, specificity, positive predictive value, negative predictive value, F1-score, AUROC, and AUPRC.

[0015] In a further improved scheme, the specific method of step S4 is as follows: Input a standard two-dimensional image of fine anatomical structure that meets the requirements of step S1.2. The fine-grained image recognition module trained in step S3, which has the ability to quantitatively analyze fine anatomical structures in images, outputs the corresponding quantitative analysis results. Finally, the analysis tool is applied to a mobile device to assist clinicians in completing intelligent analysis of fine anatomical structures in images applicable to different specialties, such as disease screening, diagnosis, and disease assessment.

[0016] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention uses deep learning technology to annotate datasets without consuming a lot of manpower and time, and can achieve efficient, automated and quantitative analysis of fine anatomical structures in images, effectively reducing the workload of clinicians and improving medical work efficiency. (2) This invention uses fine-grained networks as the basic framework and adopts a weakly supervised learning mode, which has a stronger ability to accurately locate and extract fine structural features than other artificial intelligence analysis tools; (3) This invention can effectively reduce the analytical errors caused by doctors’ lack of experience in the quantitative analysis of fine anatomical structures in images, and help the development of precision medicine and high-level medical care. (4) This invention can be applied to assist doctors in performing quantitative analysis of fine anatomical structures on mobile devices. It can break the limitations of time and space and provide expert-level analysis and diagnosis results and timely referral suggestions for patients in primary hospitals and community hospitals where high-quality medical resources are scarce. This is conducive to promoting the balanced development of medical resources.

[0017] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments: Attached Figure Description Figure 1 This is a schematic diagram of the overall workflow of the present invention; Figure 2 This is a schematic diagram illustrating the workflow for constructing a standardized image-based fine anatomical structure database in this invention. Figure 3 This is a schematic diagram of the image fine anatomical structure quantitative analysis system equipped with a fine-grained image recognition network constructed in this invention; Figure 4 This is a schematic diagram illustrating the workflow of the fine-grained image recognition module in this invention, which is capable of quantitative analysis of detailed anatomical structures in images. Detailed Implementation To better illustrate the purpose, technical solution, and advantages of the present invention, the present invention will now be described in further detail with reference to the embodiments.

[0018] Various embodiments and aspects of this application will be described with reference to the details discussed below, and the accompanying drawings will illustrate these various embodiments. The following description and drawings are illustrative of this application and should not be construed as limiting it. Numerous specific details are described to provide a comprehensive understanding of the various embodiments of this application. However, in some cases, well-known or conventional details have not been described to provide a concise discussion of the embodiments of this application.

[0019] The reference to "one embodiment" or "implementation" in this specification means that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this application. The phrase "in one embodiment" appearing in various places in this specification does not necessarily refer to the same embodiment.

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of this disclosure.

[0021] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will also be understood that, unless expressly stated herein, terms such as those defined in common dictionaries shall be interpreted as having the same meaning as they have in the relevant technical field and / or in the context of this specification, and shall not be interpreted in an idealized or overly formal sense.

[0022] In addition, it should be noted that: (1) some steps may be performed selectively; (2) the steps may not be limited to the specific order stated herein; (3) some steps may be performed in a different order; and (4) some steps may be performed simultaneously.

[0023] I. Example 1 To analyze the fine anatomical structure of the labial bone wall of the upper anterior teeth, a millimeter-micron level image-based fine anatomical structure analysis tool based on fine-grained networks was developed, such as... Figure 1 As shown, it includes the following steps: S1. Construct a standardized database of detailed anatomical structures from images, such as Figure 2 As shown, S1.1 Standardized data collection module for oral CBCT Acquire CBCT medical images taken at the same center, using the same machine, and with the same technical parameters. Collect CBCT medical images taken using the NewTom VG dental CBCT system within the same hospital. After excluding CBCT medical image data of poor quality such as blurriness and artifacts, export CBCT files through software to obtain image data containing the patient's upper and lower jaws and teeth. The acquired data is saved in DICOM format and according to the standard naming convention in the specified path. S1.2, Construct a standardized database of anterior maxillary dental imaging data. Import the standardized CBCT file obtained in step S1.1 into the implant simulation software coDaignostix. Based on the clinical analysis pattern of the labial bone wall of the maxillary anterior teeth, a standardized screenshot process is designed and implemented as follows: ① Use the occlusal plane and the human midline as the horizontal and vertical reference planes of the 3D reconstruction model, respectively; ② On the cross section of the 3D reconstruction model, use the cementoenamel junction of each maxillary anterior tooth as the horizontal plane, and draw the dental arch curve according to the center of the pulp cavity of each tooth; ③ Select the tooth position to be analyzed, and cut the maximum sagittal plane of the tooth as the standard section. The section includes the labial bone wall and related measurement reference marks; ④ Convert the selected section into TIFF format, save it in the specified path according to the standard naming method, and take screenshots according to this standard process to obtain the standardized imaging database of the maxillary anterior teeth. S1.3, Extraction, Measurement, and Labeling Module of the Labial Bone Wall of the Upper Anterior Teeth Personnel with systematic training imported the standardized two-dimensional images of the maxillary anterior teeth obtained in step S1.2 into Adobe Illustrator. Using the standard ruler in the two-dimensional image as a reference, they adjusted the image size and used the line tool to measure the thickness of the labial bone wall at the 2mm apical level of the cementoenamel junction, thus obtaining quantitative data of the labial bone wall of the maxillary anterior teeth. Based on the different thicknesses of the labial bone wall of the maxillary anterior teeth, the labial bone wall was divided into two types: thin (<1mm) and thick (≥1mm). Each image was assigned a corresponding bone mass grade label, thus obtaining a semi-quantitative category label for the labial bone wall of the maxillary anterior teeth. S1.4 Construct a standardized database of labial bone walls of the upper anterior teeth. The standard two-dimensional image of the labial bone wall of the maxillary anterior teeth obtained in step S1.2 is matched with the labial bone wall thickness 2mm below the cementoenamel junction and the bone mass grade label obtained in step S1.3. The patient information from which the image originates is used as the primary key to associate the standard two-dimensional image, quantitative indicators and clinical semantic labels belonging to that patient, and a standardized database of the labial bone wall of the maxillary anterior teeth is constructed. S2. Construct a semi-quantitative analysis module for the labial bone wall of the upper anterior teeth equipped with a fine-grained image recognition network, such as... Figure 3 As shown, A semi-quantitative analysis module for the labial bone wall of the upper anterior teeth based on the BiCNN model was constructed. B Specifically, it includes: feature extraction unit f Feature fusion unit P and data quantitative analysis unit C As shown below:

[0024] S2.1 Constructing a data input unit for the labial bone wall of the upper anterior teeth Python statements are used to automatically read data from a standardized database of labial bone walls of the maxillary anterior teeth. The data input unit converts the standard two-dimensional image of the labial bone wall of the maxillary anterior teeth into matrix information and inputs it into the BiCNN network. S2.2 Constructing a feature extraction unit for the labial bone wall of the maxillary anterior teeth A two-parameter parallel VGG16 backbone network is used as the feature extraction unit. Specifically, a standard two-dimensional image of the labial bone wall of the upper anterior teeth is input into the feature extraction unit, which can output two feature maps. f A , f B In this embodiment, the VGG16 backbone network is selected, retaining only the convolutional layer and the max pooling layer, in order to fully extract the image features of the labial bone wall of the upper anterior teeth; S2.3 Constructing a fusion unit of labial bone wall features of the maxillary anterior teeth A bilinear pooling layer was used as the feature fusion unit for the labial bone wall of the maxillary anterior teeth. Specifically, the bilinear pooling layer obtained the same image from the feature extraction unit of the labial bone wall of the maxillary anterior teeth in step S2.2. I The same spatial location in the same position of two feature maps l The outer product of corresponding features captures the pairwise relationships between feature channels, enhancing the network's discriminative ability, as shown below:

[0025] Then all positions l The feature outer product matrices are added together to obtain the fused feature map. (I) As shown below:

[0026] Then fuse the features (I) Perform vector expansion and L 2. Regularization operation to ensure that important features are not lost, as shown below:

[0027]

[0028]

[0029] Next, the fused feature map is input into the fully connected layer, which maps the distributed features to the sample label space, greatly reducing the impact of feature location on the classification of the labial bone wall. S2.4 Constructing semi-quantitative classification units of the labial bone wall of the maxillary anterior teeth The softmax layer is used as a semi-quantitative classification unit for the labial bone wall of the upper anterior teeth. The fusion features obtained in step S2.3 are input into the softmax layer. The softmax layer maps the output of the above unit to the (0,1) interval to obtain the predicted probability of the thickness classification of the labial bone wall of the upper anterior teeth. The threshold T is set to 0.5 to obtain the thickness classification result of the labial bone wall of the upper anterior teeth. S3. Train a fine-grained image recognition module capable of semi-quantitative analysis of the labial bone wall of the anterior teeth in the oral cavity, such as... Figure 4 As shown, S3.1 Constructing a Quantitative Analysis Module for Detailed Anatomical Structures The standardized maxillary anterior tooth labial bone wall database obtained in step S1.4 is divided into training and test sets at a 9:1 ratio. The training set is input into the BiCNN model to continue training the analysis module. Based on the characteristics of the model and data, the following methods are used for model training and optimization: ① Data augmentation: Rotation, translation, flipping, and cropping of the training set images are performed to expand the training set and enhance the robustness of the model; ② Resampling: Resampling is used to improve prediction performance in response to the imbalance of anterior tooth image data; ③ 5-fold cross-validation: The training set is divided into 5 mutually exclusive subsets. Four subsets are selected as the training set, and the remaining subset is used as the validation set. The training is repeated 5 times to reduce the risk of overfitting; ⑤ Model ensemble: The averaging method is used to integrate the prediction structures of the sub-models to improve the classification accuracy of the module. Finally, a semi-quantitative analysis module for the maxillary anterior tooth labial bone wall with high accuracy, high robustness, and high feasibility is constructed. S3.2 Performance Evaluation of the Semi-Quantitative Analysis Module for the Labial Bone Wall of the Upper Anterior Teeth The test set is input into the performance indicators of the semi-quantitative analysis module of the labial bone wall of the upper anterior teeth described in step S3.1. Multiple performance evaluation indicators are used, including accuracy, sensitivity, specificity, F1-score, AUROC, AUPRC, etc., to comprehensively evaluate the performance of the semi-quantitative analysis network of the labial bone wall of the upper anterior teeth, and to evaluate the performance of the semi-quantitative analysis module of the labial bone wall of the upper anterior teeth by comprehensively evaluating multiple indicators.

[0030] S4. Construct a semi-quantitative intelligent analysis system for the labial bone wall of the upper anterior teeth at the millimeter-micrometer level. Based on the semi-quantitative analysis module of the labial bone wall of the upper anterior teeth obtained in step S3, an intelligent analysis system for the labial bone wall of the upper anterior teeth is constructed. Specifically, by inputting a standard two-dimensional image of the labial bone wall of the upper anterior teeth that meets the requirements of step S1.2, the thickness classification of the labial bone wall can be obtained. Applying the code to the mobile terminal can assist dentists in completing the preoperative assessment and surgical plan decision for upper anterior tooth implantation.

[0031] II. Example 2 To address the intricate anatomical structure of the ophthalmic choroid, a millimeter- to micrometer-level image-based tool for analyzing fine anatomical structures is developed, using fine-grained networks. Figure 1 As shown, it includes the following steps: S1. Construct a standardized database of detailed anatomical structures from images, such as Figure 2 As shown, S1.1 Constructing a standardized retinal choroid database We acquired fundus choroid OCT images from the same center, using the same machine and with the same scanning parameters. Specifically, we collected fundus OCT images taken by the Heidelberg SPECTRALIS optical coherence tomography scanner (Spectralis OCT) from the same hospital. After excluding data with poor eye positioning, noise, or poor quality, we obtained the imaging data of fundus choroid OCT. S1.2, Construct a standardized database of fundus choroidal imaging data. The OCT image of the fundus choroid obtained in step S1.1 is scanned in the horizontal and vertical directions of the fovea of ​​the macula. After image cropping, a standard two-dimensional image of the fundus choroid is obtained. The standard two-dimensional image of the fundus choroid is saved in the specified path according to the standard naming method. S1.3, Module for extraction, measurement, and labeling of choroid. Professionally trained staff import the standard two-dimensional image of the fundus choroid obtained in step S1.2 into the measurement software to measure the thickness of the choroid at different sites. Specifically, the thickness of the fundus choroid is defined as the vertical distance from the outer edge of the high reflectivity line of the retinal pigment epithelium to the inner surface of the sclera. The measurement sites are the center of the fovea, above the fovea, below the fovea, on the nasal side, and on the temporal side at a distance of 750 micrometers from the center of the fovea. S1.4 Constructing a standardized choroid database The standard two-dimensional image data of the fundus choroid obtained in step S1.2 is matched with the choroid thickness at each measurement site obtained in step S1.3. Specifically, the patient information from which the image originates is used as the primary key to associate the standard two-dimensional image and quantitative indicators belonging to that patient to construct a standardized fundus choroid database. S2. Construct a module for quantitative analysis of fundus choroid thickness equipped with a fine-grained image recognition network, such as... Figure 3 As shown, A module for quantitative analysis of fundus choroid thickness based on the BiCNN model was constructed. In the BiCNN model... B Specifically, it includes: feature extraction unit f Feature fusion unit P and data quantitative analysis unit C As shown below: ; S2.1 Constructing a fundus image choroidal data input unit Python statements are used to automatically read data from a standardized fundus image venous database. The data input unit converts the standard two-dimensional images of the standardized fundus images into matrix information and inputs it into the BiCNN network. S2.2 Constructing a choroid feature extraction unit for fundus images A two-parameter parallel VGG16 backbone network is used as the feature extraction unit. Specifically, a standard two-dimensional image of the fundus choroid is input into the feature extraction unit, which can output two feature maps. f A , f B In this embodiment, the VGG16 backbone network is selected to retain only the convolutional layer and the max pooling layer in order to fully extract the image features of the standard two-dimensional image of the fundus choroid. S2.3 Constructing a feature fusion unit for the retinal choroid A bilinear pooling layer is used as the standard two-dimensional image feature fusion unit for the fundus choroid. Specifically, the bilinear pooling layer obtains the same image from the fundus image choroid feature extraction unit in step S2.2. I The same spatial location in the same position of two feature maps l The outer product of corresponding features captures the pairwise relationships between feature channels, enhancing the network's discriminative ability, as shown below: ; Then all positions l The feature outer product matrices are added together to obtain the fused feature map. (I) As shown below: ; Then fuse the features (I) Perform vector expansion and L 2. Regularization operation to ensure that important features are not lost, as shown below:

[0032]

[0033] ; Next, the fused feature map is input into the fully connected layer, and the distributed features are mapped to the sample label space, which can greatly reduce the impact of feature location on the quantitative analysis of the choroid in fundus images. S2.4 Constructing a data quantification analysis unit The softmax layer is used as the data quantification analysis unit: the fusion features obtained in step S2.3 are input into the softmax layer, the softmax layer maps the output of the above unit to the (0,1) interval, calculates the selected loss function, obtains the prediction probability of different categories, and obtains the choroidal quantification thickness result of the analysis site by giving a threshold T. S3. Train a fine-grained image recognition module with the ability to quantitatively analyze detailed anatomical structures in images, such as... Figure 4 As shown, S3.1 Training a fine-grained image analysis network The training sample set is fed into the image fine anatomical structure quantification analysis module equipped with a fine-grained image recognition network as described in step S2 for training and verification, thereby constructing the image fine anatomical structure quantification analysis module. S3.2 Performance Evaluation of the Quantitative Analysis Module for Detailed Anatomical Structures Input the test dataset into the image fine anatomical structure quantitative analysis module described in step S3.1, and evaluate the output quantitative analysis results. Specific analysis indicators include: mean absolute error, root mean square error, etc. S4. Construct an intelligent analysis system for fine image structures at the millimeter-micrometer level. Based on the quantitative analysis module of fine anatomical structure of images obtained in step S3, a fine image structure intelligent analysis system at the millimeter-micron level is constructed. The analysis tools are applied to mobile devices to assist clinicians in completing intelligent analysis of the fine anatomical structure of the ophthalmic choroid.

[0034] III. Example 3 To analyze the fine anatomical structure of regenerative nodules in liver cirrhosis, a millimeter-micron level imaging fine anatomical structure analysis tool based on fine-grained networks was developed, such as... Figure 1 As shown, it includes the following steps: S1. Construct a standardized database of detailed anatomical structures from images, such as Figure 2 As shown, S1.1 Constructing a standardized database of fine anatomical structures for regenerative nodules in liver cirrhosis. Acquire liver ultrasound contrast images taken at the same center, using the same machine, and with the same technical parameters; specifically, collect liver ultrasound contrast images taken using the Philips-iU22 system within the same hospital, and obtain liver ultrasound imaging data after excluding liver ultrasound contrast data of poor quality such as artifacts and blurring. S1.2 Constructing a standardized image database of detailed anatomical structures of regenerative nodules in liver cirrhosis. Eight standard liver ultrasound sections were selected from the standard two-dimensional image of the cirrhotic regenerative nodules obtained in step S1.1, including the subcostal oblique section showing the first hepatic hilum, the subcostal oblique section showing the second hepatic hilum, the subxiphoid transverse section showing the left lateral angle, the subxiphoid sagittal section showing the left interlobar fissure, the subxiphoid sagittal section showing the long axis section of the left lobe across the abdominal aorta, the right subcostal sagittal section showing the median fissure, the subcostal oblique section showing the diaphragm roof, and the right intercostal oblique section showing the right interlobar fissure. These images were then cropped to obtain a standard two-dimensional image of liver ultrasound contrast imaging. S1.3, Extraction, Measurement, and Labeling Module for Regenerative Nodules in Liver Cirrhosis The standardized two-dimensional images of liver ultrasound obtained in step S1.2 are imported into ultrasound image analysis software by systematically trained personnel. The software tools are used to measure the number and size of regenerating nodules under each ultrasound section. Specifically, the length and width of the regenerating nodules seen in each ultrasound section are measured in millimeters, and the number of regenerating nodules seen in each ultrasound interface is counted. The measured quantitative indicators are stored in a specified path. S1.4 Constructing a standardized ultrasound contrast-enhanced imaging database for regenerative nodules in liver cirrhosis. The standard two-dimensional images of liver ultrasound contrast obtained in step S1.2 are matched with the regenerating nodule sizes under each ultrasound section obtained in step S1.3. The patient information from which the images originate is used as the primary key to associate the standard two-dimensional images and quantitative indicators belonging to that patient, thereby constructing a standardized ultrasound database of regenerating nodules in liver cirrhosis. S2. Construct a quantitative analysis module for regenerative nodules in liver cirrhosis equipped with a fine-grained image recognition network, such as... Figure 3 As shown, A quantitative analysis module for regenerative nodules in liver cirrhosis based on the BiCNN model was constructed. The BiCNN model... B Specifically, it includes: feature extraction unit f Feature fusion unit P and data quantitative analysis unit C As shown below: ; S2.1 Constructing an ultrasound contrast-enhanced imaging data input unit for regenerating nodules in liver cirrhosis. Python statements are used to automatically read data from the standardized liver cirrhosis regenerative nodule ultrasound contrast imaging database. The data input unit converts the standard two-dimensional images of liver ultrasound contrast imaging into matrix information and inputs it into the BiCNN network. S2.2 Constructing a feature extraction unit for regenerative nodules in liver cirrhosis A two-parameter parallel VGG16 backbone network is used as the feature extraction unit. Specifically, a standard two-dimensional image of liver ultrasound contrast imaging is input into the feature extraction unit, which can output two feature maps. f A , f B In this embodiment, the VGG16 backbone network is selected to retain only the convolutional layer and the max pooling layer in order to fully extract the image features of liver ultrasound contrast imaging. S2.3 Constructing a feature fusion unit for regenerative nodules in liver cirrhosis A bilinear pooling layer was used as the feature fusion unit for regenerative nodules in liver cirrhosis. Specifically, the bilinear pooling layer obtained the same image from the feature extraction unit for regenerative nodules in liver cirrhosis in step S2.2. I The same spatial location in the same position of two feature maps l The outer product of corresponding features captures the pairwise relationships between feature channels, enhancing the network's discriminative ability, as shown below: ; Then all positions l The feature outer product matrices are added together to obtain the fused feature map. (I) As shown below: ; Then fuse the features ( I Perform vector expansion and L 2. Regularization operation to ensure that important features are not lost, as shown below:

[0035]

[0036] ; Next, the fused feature map is input into the fully connected layer, and the distributed features are mapped to the sample label space, which can greatly reduce the impact of feature location on the identification of cirrhotic regenerative nodules in ultrasound contrast images. S2.4 Constructing a data quantification analysis unit The softmax layer is used as the data quantification analysis unit: the fusion features obtained in step S2.3 are input into the softmax layer, the softmax layer maps the output of the above unit to the (0,1) interval, calculates the selected loss function, obtains the prediction probability of different categories, and outputs the number and size of regenerating nodules in the specified liver ultrasound contrast image by giving a threshold T. S3. Train a fine-grained image recognition module with the ability to quantitatively analyze detailed anatomical structures in images, such as... Figure 4 As shown, S3.1 Training a fine-grained image analysis network The training sample set is fed into the image fine anatomical structure quantification analysis module equipped with a fine-grained image recognition network as described in step S2 for training and verification, thereby constructing the image fine anatomical structure quantification analysis module. S3.2 Performance Evaluation of the Quantitative Analysis Module for Detailed Anatomical Structures Input the test dataset into the image fine anatomical structure quantitative analysis module described in step S3.1, evaluate the output quantitative analysis results, and evaluate the indicators including mean absolute error, root mean square error, accuracy, precision, sensitivity, etc. S4. Construct an intelligent analysis system for fine image structures at the millimeter-micrometer level. Based on the quantitative analysis module of fine anatomical structure of images obtained in step S3, an intelligent analysis system for fine image structure at the millimeter-micron level is constructed. The analysis tool is applied to mobile devices to assist clinicians in completing the intelligent quantitative analysis of fine anatomical structure of regenerating nodules in liver cirrhosis, which helps to provide a rapid and accurate diagnosis in clinical practice.

[0037] Although the present invention has been described in detail with reference to the above embodiments, it will be apparent to those skilled in the art that various changes or modifications can be made to the invention without departing from the principles and spirit of the invention as defined by the claims. Therefore, the detailed description of the embodiments in this disclosure is for illustrative purposes only and is not intended to limit the invention; rather, the scope of protection is defined by the content of the claims.

Claims

1. A tool for analyzing fine anatomical structures in millimeter-micron level images based on fine-grained networks, characterized in that, Includes the following steps: S1. Construct a standardized database of detailed anatomical structures from images. S1.1 Standardized Collection of Medical Images Collect imaging data required by various specialties in the diagnosis and treatment of diseases. During the collection of imaging data, professional personnel need to conduct quality control of the images, screen out blurry images and images containing artifacts, and ensure that the collected imaging data can accurately reflect the quantitative and semi-quantitative characteristics of the corresponding fine anatomical structures. S1.2, Construct a standardized and detailed anatomical structure image database Based on the source and type of the imaging data, the standardized imaging data collected in step S1.1 are classified and stored. According to the analysis needs of different specialties for the corresponding fine anatomical structures, the most favorable cross-section for diagnosis and feature observation is selected, and the image is cropped to obtain standard two-dimensional images of fine anatomical structures. S1.3, Module for Extraction, Measurement, and Annotation of Fine Anatomical Structures The standard two-dimensional images of the fine anatomical structures obtained in step S1.2 are imported into data quantification analysis software, where systematically trained personnel extract, measure, and label the indicators to obtain quantitative data of the fine anatomical structures. Based on the quantitative data and clinical semantics, the fine anatomical structures are assigned corresponding semi-quantitative category labels. S1.4 Construct a standardized image database of fine anatomical structures The standard two-dimensional images of fine anatomical structures obtained in step S1.2 are matched with the quantitative data and semi-quantitative labels obtained in step S1.

3. The patient information from which the images originate is used as the primary key to associate the standard two-dimensional images, quantitative data and semi-quantitative labels belonging to that patient, and a standardized image fine anatomical structure database is constructed. S2. Construct a module for quantitative analysis of fine anatomical structures in images equipped with a fine-grained image recognition network. A fine-grained image recognition network model is constructed using a fine-anatomical structure quantification analysis module. The fine-grained image recognition network mainly consists of the following parts: a data input unit, a feature extraction unit, a feature fusion unit, and a data quantification analysis unit. S2.1 Constructing the data input unit The computer program automatically reads the two-dimensional images and corresponding data from the standardized image fine anatomical structure database constructed in step S1.

4. The data input unit converts the two-dimensional images from the standardized fine anatomical structure database into matrix information and inputs it into the feature extraction unit. S2.2 Constructing Feature Extraction Units Using the backbone of a convolutional neural network as the feature extraction unit, the matrix information in the data input unit described in step S2.1 is input into the backbone of the convolutional neural network to obtain a feature map containing quantitative information of fine anatomical structure. S2.3 Constructing Feature Fusion Units The feature fusion unit includes a bilinear pooling layer, a gradient boosting loss function, and / or a fully connected layer. The fine anatomical structure quantification information feature map obtained in step S2.2 is input into the feature fusion unit for feature fusion and dimensionality reduction operations to ensure that important features are not lost. S2.4 Constructing a data quantification analysis unit The softmax layer is used as the data quantification analysis unit: the fusion features obtained in step S2.3 are input into the softmax layer, the softmax layer maps the output of the above unit to the (0,1) interval, calculates the selected loss function, obtains the prediction probability of different categories, and obtains the quantification analysis results of multiple fine anatomical structures by giving a threshold T, including quantitative data and semi-quantitative data. S3. Training a fine-grained image recognition module capable of quantitative analysis of detailed anatomical structures in images. S3.1 Constructing a Quantitative Analysis Module for Detailed Anatomical Structures The training sample set is fed into the image fine anatomical structure quantification analysis module equipped with a fine-grained image recognition network as described in step S2 for training and verification, thereby constructing the image fine anatomical structure quantification analysis module. S3.2 Performance Evaluation of the Quantitative Analysis Module for Detailed Anatomical Structures Input the test dataset into the image fine anatomical structure quantitative analysis module described in step S3.1, and evaluate the output quantitative analysis results; S4. Construct an intelligent analysis system for fine image structures at the millimeter-micrometer level. Based on the quantitative analysis module of fine anatomical structure of the image obtained in step S3, an intelligent analysis system for fine image structure at the millimeter-micrometer level is constructed.

2. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S1.1, the imaging data must accurately reflect the quantitative and semi-quantitative characteristics of the quantity, size, classification, morphology, and location of the corresponding fine anatomical structures.

3. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S1.1, the sources of imaging data include CT, CBCT, PET-CT, MRI and / or US types.

4. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S1.2, before classifying and storing the image data, the image data is preprocessed, including image cropping, rotation, and contrast adjustment.

5. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S1.3, the quantitative data includes the size, number, and area of ​​fine anatomical structures; the semi-quantitative category labels include the classification, subclass, morphology, and location of fine anatomical structures. In step S2.4, the quantitative analysis results of the detailed anatomical structure include quantitative data containing size, area, and quantity, as well as semi-quantitative data containing classification, subclass, location, and morphology.

6. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, The convolutional neural network backbone used in step S2.2 includes LeNet-5, Vgg16, ResNet, DenseNet, or GoogleNet convolutional neural network models; the parallel convolutional neural network backbone in step S2.2 includes multiple convolutional layers and max pooling layers; an additional feature extraction enhancement module is provided in step S2.2, including APN and / or RPN; after the matrix information in the data input unit described in step S2.1 is input into the convolutional neural network backbone, the convolutional neural network backbone performs feature extraction and downsampling through convolutional layers and max pooling layers to obtain a feature map containing quantitative information of fine anatomical structure.

7. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, Fine-grained image recognition network models include BiCNN, RCNN, RACNN, MA-CNN, or PA-CNN.

8. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S3.1, the training sample set is fed into the image fine anatomical structure quantification analysis module equipped with fine-grained image recognition network described in step S2 for training and verification. Data augmentation, resampling, K-fold cross-validation and / or model ensemble techniques are used to improve the generalization performance and prediction performance of the model.

9. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, In step S3.2, the performance evaluation of the quantitative analysis results includes the following performance evaluation indicators: mean absolute error, root mean square error, accuracy, precision, sensitivity, specificity, positive predictive value, negative predictive value, F1-score, AUROC, and AUPRC.

10. The fine-grained network-based image-based fine anatomical structure analysis tool according to claim 1, characterized in that, The specific method for step S4 is as follows: Input a standard two-dimensional image of fine anatomical structure from an imaging source that meets the requirements of step S1.

2. The fine-grained image recognition module, trained in step S3, which has the ability to quantitatively analyze fine anatomical structures from imaging sources, outputs the corresponding quantitative analysis results. Finally, the analysis tool is applied to a mobile device to assist clinicians in completing intelligent analysis of fine anatomical structures from imaging sources applicable to different specialties.