Thyroid tumor diagnosis auxiliary system based on artificial intelligence
By designing an auxiliary system based on artificial intelligence in thyroid tumor diagnosis, using multi-scale convolutional neural network and self-supervised learning modules, combined with distributed data sharing, the problems of inconsistent diagnostic results and weak generalization capabilities in the existing technology are solved. , improves the accuracy and response speed of thyroid tumor diagnosis.
Patent Information
- Application Number
- CN202411260109.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing ultrasound imaging diagnosis methods rely on the experience of doctors, which affects the accuracy and consistency of diagnostic results. Artificial intelligence systems have problems such as insufficient data labeling and weak generalization capabilities in the diagnosis of thyroid tumors, which are difficult to meet clinical needs.
A thyroid tumor diagnosis assist system based on artificial intelligence is designed, including ultrasonic image acquisition module, image processing module, self-supervised learning module, joint learning module and diagnostic assist module. The features are extracted and fusionized through multi-scale convolutional neural network to generate Pseudo-labels are pre-trained by model, combined with distributed data sharing and model joint training to realize the generation and application of diagnostic assisted models.
It improves the accuracy of thyroid tumor diagnosis and generalization ability of the model, reduces the dependence on large-scale annotation data, enhances the robustness and response speed of the model, and is suitable for rapid diagnostic scenarios.
Smart Images

Figure CN119067957A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of diagnostic assistance technology for thyroid diseases, and in particular to an artificial intelligence-based thyroid tumor diagnostic assistance system. Background Art
[0002] Thyroid tumors are one of the major diseases that affect global health. Early detection and accurate diagnosis of thyroid nodules play a vital role in improving the cure rate of patients. At present, the commonly used clinical diagnosis methods for thyroid tumors mainly include imaging examinations, fine needle aspiration cytology (FNAC), and laboratory tests, such as thyroid hormone level testing. Among them, ultrasound imaging has become the main means of thyroid nodule detection due to its non-invasiveness and high efficiency.
[0003] However, existing ultrasound imaging diagnosis mainly relies on the doctor's experience. The interpretation of images and the judgment of nodules may vary depending on the doctor's experience and skill level, which affects the accuracy and consistency of the diagnosis results. Although artificial intelligence technology has made significant progress in medical image analysis, in the diagnosis of thyroid tumors, the existing AI system still has problems such as insufficient data annotation and weak model generalization ability, which makes it difficult to meet clinical needs. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a thyroid tumor diagnosis auxiliary system based on artificial intelligence to solve the problems mentioned in the background technology and improve the accuracy of thyroid tumor diagnosis.
[0005] The present invention discloses a thyroid tumor diagnosis auxiliary system based on artificial intelligence, which includes an ultrasonic image acquisition module, an image processing module, a self-supervised learning module, a joint learning module and a diagnosis auxiliary module; wherein:
[0006] The ultrasound image acquisition module is used to obtain the patient's thyroid ultrasound image;
[0007] The image processing module is used to perform feature extraction and feature fusion operations on thyroid ultrasound images through a multi-scale convolutional neural network;
[0008] The self-supervised learning module is used to generate pseudo labels for unlabeled thyroid ultrasound images, and pre-train the model by combining the unlabeled data for generating the pseudo labels with the labeled data to obtain a first pre-trained model;
[0009] The joint learning module is used to perform distributed data sharing and joint model training among multiple medical institutions to obtain a diagnosis-assisted model;
[0010] The diagnostic assistance module is used to analyze the patient's thyroid ultrasound image through a diagnostic assistance model to obtain the patient's thyroid analysis results, and to assist medical staff in diagnosing the patient's thyroid tumor based on the thyroid analysis results.
[0011] Furthermore, the image processing module includes a multi-layer convolutional neural network and a feature fusion device;
[0012] The multi-layer convolutional neural network is used to extract multi-scale features of thyroid ultrasound images;
[0013] The feature fuser is used to fuse features of different scales to generate a comprehensive feature vector for thyroid classification.
[0014] Furthermore, the training process of the first training model includes:
[0015] Receiving multi-scale features output by the image processing module;
[0016] Performing image enhancement operations on the received features;
[0017] Pair the enhanced thyroid ultrasound image features with the original features to generate pseudo-label pairs;
[0018] A self-supervised learning model was constructed based on a convolutional neural network. The enhanced thyroid ultrasound image features were used as the input of the self-supervised learning model, and the original features were used as the target output. The self-supervised learning model was trained based on an unlabeled dataset with pseudo labels, and the self-supervised learning model was fine-tuned using labeled data.
[0019] The fine-tuned self-supervised learning model is used as the first pre-training model.
[0020] Furthermore, the model joint training of the joint learning module includes performing model joint training based on the first training model to obtain a diagnosis auxiliary model.
[0021] Furthermore, the training process of the diagnosis auxiliary model includes:
[0022] Deploy the federated learning framework in multiple medical institutions, and each medical institution initializes the parameters of the first pre-trained model;
[0023] Each medical institution trains the model based on local thyroid ultrasound imaging data;
[0024] Aggregate the trained models of each medical institution to generate a new global model;
[0025] Each medical institution continues to train the new global model based on local thyroid ultrasound imaging data;
[0026] The process of local model training and global model updating is repeated until the training convergence condition is reached, and the final global model is used as a diagnosis auxiliary model.
[0027] Furthermore, the system also includes an edge computing module, which is used to process ultrasound image data in real time based on an edge server.
[0028] Furthermore, after acquiring the ultrasound image, the edge computing module performs image preprocessing and preliminary classification operations.
[0029] Furthermore, the preliminary classification operation performed by the edge computing module includes:
[0030] Extract the basic features of ultrasound images through lightweight convolutional neural networks;
[0031] Based on the extracted basic features, a preliminary classification operation is performed through a classification model to obtain normal ultrasound images and incompletely normal ultrasound images;
[0032] The incompletely normal ultrasound image data is transmitted to the image processing module.
[0033] Furthermore, the process in which the edge computing module transmits the not completely normal ultrasound image data to the image processing module includes compressing the not completely normal ultrasound image data before transmitting it to the image processing module.
[0034] Furthermore, the system also includes a dynamic load balancing module, and the dynamic load balancing module is used to dynamically adjust the allocation of computing resources according to the amount of diagnosis requests.
[0035] Compared with the prior art, the present invention has the following beneficial effects:
[0036] 1. Through the multi-scale convolutional neural network in the image processing module, the system can extract and fuse image features from different scales, making the model more accurate in identifying thyroid nodules. The fused comprehensive feature vector can better capture subtle changes in the image, thereby improving the classification accuracy of benign and malignant nodules;
[0037] 2. The self-supervised learning module generates pseudo labels and pre-trains the model based on unlabeled and labeled data, which can effectively reduce the dependence on large-scale labeled data and improve the generalization ability of the model, so that the model can still maintain high performance when processing complex and small amounts of labeled data;
[0038] 3. The joint learning module conducts distributed data sharing and joint model training among multiple medical institutions. The generated diagnostic auxiliary model integrates the diverse data of multiple institutions, improves the generalization ability and robustness of the model, and also ensures the privacy and security of the data, so that the system can operate stably in different scenarios;
[0039] 4. By performing real-time processing close to the data source through the edge computing module, the system can perform preliminary classification and preprocessing at the moment the data is generated, reducing the delay in data transmission and improving the response speed of the system, making it suitable for clinical scenarios that require rapid diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the economic application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0041] Figure 1 This is a schematic diagram of the structure of an artificial intelligence-based thyroid tumor diagnosis assistance system disclosed in Example 1 of the present invention. DETAILED DESCRIPTION
[0042] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments.
[0043] Embodiment 1
[0044] The present invention discloses a thyroid tumor diagnosis auxiliary system based on artificial intelligence, please refer to Figure 1 , Figure 1 is a structural diagram of an artificial intelligence-based thyroid tumor diagnosis auxiliary system disclosed in an embodiment of the present invention, the system includes an ultrasonic image acquisition module, an image processing module, a self-supervised learning module, a joint learning module and a diagnosis auxiliary module; wherein,
[0045] The ultrasound image acquisition module is used to obtain the patient's thyroid ultrasound image;
[0046] The image processing module is used to perform feature extraction and feature fusion operations on thyroid ultrasound images through a multi-scale convolutional neural network;
[0047] The self-supervised learning module is used to generate pseudo labels for unlabeled thyroid ultrasound images, and pre-train the model by combining the unlabeled data for generating the pseudo labels with the labeled data to obtain a first pre-trained model;
[0048] The joint learning module is used to perform distributed data sharing and joint model training among multiple medical institutions to obtain a diagnosis-assisted model;
[0049] The diagnostic assistance module is used to analyze the patient's thyroid ultrasound image through a diagnostic assistance model to obtain the patient's thyroid analysis results, and to assist medical staff in diagnosing the patient's thyroid tumor based on the thyroid analysis results.
[0050] Furthermore, the image processing module includes a multi-layer convolutional neural network and a feature fuser, wherein the multi-layer convolutional neural network is used to extract multi-scale features of thyroid ultrasound images, and the feature fuser is used to fuse features of different scales to generate a comprehensive feature vector for thyroid classification.
[0051] In the analysis of thyroid ultrasound images, the size, shape, boundary, internal echo and other features of the nodules may affect the diagnosis results. Thyroid nodules may appear as very small areas with complex boundaries or texture changes, and these features will appear at different scales. Multi-scale features of thyroid ultrasound images are extracted through multi-layer convolutional neural networks, such as local features such as edges, corners, and subtle textures are extracted through low-level convolutional layers, medium-scale features such as the overall shape and echo pattern of nodules are extracted through middle-level convolutional layers, and global features such as the position of nodules in the image and the relationship with surrounding tissues are extracted through high-level convolutional layers. The features extracted by different convolutional layers have different focuses. The low-level features are detailed but have a small coverage range, while the high-level features have a large coverage range but fewer details. Relying on the features of a certain layer alone may cause the model to ignore some key information. For example, if it only relies on high-level features, the model may miss the subtle boundaries of the nodules; if it only relies on low-level features, the model may not understand the overall morphology of the nodules. Extracting multi-scale features of thyroid ultrasound images through multi-layer convolutional neural networks can avoid the above problems.
[0052] In the embodiment of the present invention, the function of the feature fuser is to fuse the features extracted by different convolutional layers to form a comprehensive feature vector. The comprehensive feature vector can more accurately reflect the complexity and diversity of nodules in the image, so in the subsequent classification task, the model can better distinguish between benign and malignant nodules. For example, a nodule with fuzzy boundaries and uneven internal echoes may be classified as malignant, while a nodule with clear boundaries and uniform internal echoes may be classified as benign.
[0053] By using a multi-layer convolutional neural network to extract multi-scale features and fusing these features together through a feature fuser, the system can fully capture the key features in thyroid ultrasound images at multiple scales. This fusion of multi-scale features enables the model to more accurately extract all information related to diagnosis when faced with complex thyroid ultrasound images, thereby improving the accuracy and reliability of diagnosis.
[0054] Furthermore, the training process of the first training model includes:
[0055] Receiving multi-scale features output by the image processing module;
[0056] Performing image enhancement operations on the received features;
[0057] Pair the enhanced thyroid ultrasound image features with the original features to generate pseudo-label pairs;
[0058] A self-supervised learning model was constructed based on a convolutional neural network. The enhanced thyroid ultrasound image features were used as the input of the self-supervised learning model, and the original features were used as the target output. The self-supervised learning model was trained based on an unlabeled dataset with pseudo labels, and the self-supervised learning model was fine-tuned using labeled data.
[0059] The fine-tuned self-supervised learning model is used as the first pre-training model.
[0060] It can be understood that the multi-scale features output by the image processing module received by the self-supervised learning module include different features of the image, such as detailed texture, edge information, overall shape, etc. By performing various enhancement operations on these features, such as contrast adjustment, rotation, flipping, scaling, etc., it is possible to simulate the changes that may occur in the image under different shooting conditions and generate multiple versions of feature representations.
[0061] By generating pseudo-label pairs, self-supervised learning is performed without real labels. The goal of the first pre-training model of this application is to recover or approach the original features from the enhanced features, minimize the difference between input and output, and learn stable and useful feature representations. In addition, the purpose of training the self-supervised learning model on a large amount of unlabeled data and fine-tuning the model with a small amount of labeled data is to extract universal feature representations from a large amount of unlabeled data, and the fine-tuning process combines these universal features with specific tasks to improve the performance of the model in subsequent specific diagnostic analysis tasks. This combination greatly reduces the need for a large amount of labeled data while improving the performance of the model.
[0062] Furthermore, the model joint training of the joint learning module includes performing model joint training based on the first training model to obtain a diagnosis auxiliary model.
[0063] Furthermore, the training process of the diagnosis-assisted model includes:
[0064] Deploy the federated learning framework in multiple medical institutions, and each medical institution initializes the parameters of the first pre-trained model;
[0065] Each medical institution trains the model based on local thyroid ultrasound imaging data;
[0066] Aggregate the trained models of each medical institution to generate a new global model;
[0067] Each medical institution continues to train the new global model based on local thyroid ultrasound imaging data;
[0068] The process of local model training and global model updating is repeated until the training convergence condition is reached, and the final global model is used as a diagnosis auxiliary model.
[0069] It is understandable that the first pre-trained model in the embodiment of the present invention is trained based on a large amount of unlabeled data and a small amount of labeled data, with the purpose of having good feature extraction performance. Using the first pre-trained model as the training starting point for the diagnostic replication model can speed up the model training process and ensure that the model of each institution has a certain initial performance, while also facilitating subsequent model aggregation operations.
[0070] Specifically, let the parameters of the first pre-trained model be θ 0 , each medical institution i (where i∈
[0071] 1,2,...,N}) have local data sets in, represents the j-th thyroid ultrasound image data, as a preferred implementation of this embodiment, express The corresponding label is benign nodule or malignant nodule.
[0072] Each medical institution i initializes the parameter θ 0 Based on this, local training is performed to optimize its loss function L i (θ):
[0073]
[0074] in, is the predicted output of the model, and L(·) is the cross entropy loss function.
[0075] After a round of local training, each medical institution will get a new set of model parameters θ i , these model parameters are aggregated through the server to generate new global model parameters:
[0076]
[0077] Among them, M is the total number of data of all medical institutions, is the weight of the data of medical institution i in the total data.
[0078] The new global model parameters θ (t+1) Distributed to each medical institution again as the starting point for the next round of local training, each medical institution uses the new global model parameter θ (t+1)For further training:
[0079]
[0080] After the new local training, each institution submits the updated model parameters again To the server, the server aggregates the new model.
[0081] After multiple iterations, until the global model parameters converge, set them as θ * , that is, θ * is the parameter of the diagnostic auxiliary model. When the diagnostic auxiliary model receives new thyroid ultrasound image data x new After that, the output diagnosis result is:
[0082]
[0083] in, is the prediction result of the model for the input ultrasound image data. As a preferred implementation of this embodiment, the prediction result is whether it is a malignant tumor.
[0084] Embodiment 2
[0085] Based on the first embodiment, the second embodiment also includes the following contents.
[0086] Furthermore, the system also includes an edge computing module, which is used to process ultrasound image data in real time based on an edge server.
[0087] It is understandable that the edge computing module in the embodiment of the present invention can be deployed on the ultrasound device or the local server cluster. When the ultrasound image data is generated, it is immediately transmitted to the nearby edge computing module for processing.
[0088] Furthermore, after acquiring the ultrasound image, the edge computing module performs image preprocessing and preliminary classification operations.
[0089] Furthermore, the preliminary classification operations performed by the edge computing module include:
[0090] Extract the basic features of ultrasound images through lightweight convolutional neural networks;
[0091] Based on the extracted basic features, a preliminary classification operation is performed through a classification model to obtain normal ultrasound images and incompletely normal ultrasound images;
[0092] The incompletely normal ultrasound image data is transmitted to the image processing module.
[0093] Furthermore, the process in which the edge computing module transmits the not completely normal ultrasound image data to the image processing module includes compressing the not completely normal ultrasound image data before transmitting it to the image processing module.
[0094] In the embodiment of the present invention, through rapid classification, images with possible problems can be screened out at the earliest stage, so that more computing resources can be concentrated on images that require in-depth analysis, which can improve diagnostic efficiency while reducing the workload of doctors.
[0095] Incompletely normal ultrasound image data refers to ultrasound image data other than 100% normal ultrasound image data. For image data classified as "incompletely normal", the edge computing module will compress it and transmit it to the image processing module. The compression operation includes but is not limited to image encoding, reducing resolution, etc. to reduce the amount of data. The compressed data is transmitted to the central server through the hospital's internal network or the cloud for further in-depth analysis.
[0096] Compressing data helps reduce network bandwidth usage and transmission time, especially when the amount of image data is large. In this way, the system can quickly transmit important data to the backend for in-depth processing while maintaining data quality.
[0097] Furthermore, the system also includes a dynamic load balancing module, which is used to dynamically adjust the allocation of computing resources according to the amount of diagnosis requests.
[0098] The dynamic load balancing module monitors the load of each processing node and the number of diagnostic requests in the system in real time. When the number of diagnostic requests increases, the system automatically allocates more computing resources to the edge computing module or image processing module to ensure that the system can continue to operate efficiently. For example, during peak hours of the day (such as 9 am to 11 am), the number of diagnostic requests increases significantly, and the system speeds up processing by allocating more CPU / GPU resources to the edge computing module.
[0099] Finally, it should be noted that the artificial intelligence-based thyroid tumor diagnosis auxiliary system disclosed in the embodiment of the present invention is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An artificial intelligence-based thyroid tumor diagnosis auxiliary system, characterized in that: The system includes an ultrasound image acquisition module, an image processing module, a self-supervised learning module, a joint learning module and a diagnosis assistance module; wherein, The ultrasound image acquisition module is used to obtain the patient's thyroid ultrasound image; The image processing module is used to perform feature extraction and feature fusion operations on thyroid ultrasound images through a multi-scale convolutional neural network; The self-supervised learning module is used to generate pseudo labels for unlabeled thyroid ultrasound images, and pre-train the model by combining the unlabeled data for generating the pseudo labels with the labeled data to obtain a first pre-trained model; The joint learning module is used to perform distributed data sharing and joint model training among multiple medical institutions to obtain a diagnosis-assisted model; The diagnostic assistance module is used to analyze the patient's thyroid ultrasound image through a diagnostic assistance model to obtain the patient's thyroid analysis results, and to assist medical staff in diagnosing the patient's thyroid tumor based on the thyroid analysis results.
2. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 1 is characterized in that: The image processing module includes a multi-layer convolutional neural network and a feature fusion device; The multi-layer convolutional neural network is used to extract multi-scale features of thyroid ultrasound images; The feature fuser is used to fuse features of different scales to generate a comprehensive feature vector for thyroid classification.
3. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 2 is characterized in that: The training process of the first training model includes: Receiving multi-scale features output by the image processing module; Performing image enhancement operations on the received features; Pair the enhanced thyroid ultrasound image features with the original features to generate pseudo-label pairs; A self-supervised learning model was constructed based on a convolutional neural network. The enhanced thyroid ultrasound image features were used as the input of the self-supervised learning model, and the original features were used as the target output. The self-supervised learning model was trained based on an unlabeled dataset with pseudo labels, and the self-supervised learning model was fine-tuned using labeled data. The fine-tuned self-supervised learning model is used as the first pre-training model.
4. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 3 is characterized in that: The model joint training of the joint learning module includes performing model joint training based on the first training model to obtain a diagnosis auxiliary model.
5. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 4 is characterized in that: The training process of the diagnosis auxiliary model includes: Deploy the federated learning framework in multiple medical institutions, and each medical institution initializes the parameters of the first pre-trained model; Each medical institution trains the model based on local thyroid ultrasound imaging data; Aggregate the trained models of each medical institution to generate a new global model; Each medical institution continues to train the new global model based on local thyroid ultrasound imaging data; The process of local model training and global model updating is repeated until the training convergence condition is reached, and the final global model is used as a diagnosis auxiliary model.
6. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to any one of claims 1 to 5, characterized in that: The system also includes an edge computing module, which is used to process ultrasound image data in real time based on an edge server.
7. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 6 is characterized in that: After acquiring the ultrasound image, the edge computing module performs image preprocessing and preliminary classification operations.
8. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 7 is characterized in that: The preliminary classification operations performed by the edge computing module include: Extract the basic features of ultrasound images through lightweight convolutional neural networks; Based on the extracted basic features, a preliminary classification operation is performed through a classification model to obtain normal ultrasound images and incompletely normal ultrasound images; The incompletely normal ultrasound image data is transmitted to the image processing module.
9. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 8, characterized in that: The process in which the edge computing module transmits the not completely normal ultrasound image data to the image processing module includes compressing the not completely normal ultrasound image data before transmitting it to the image processing module.
10. The artificial intelligence-based thyroid tumor diagnosis auxiliary system according to claim 9, characterized in that: The system also includes a dynamic load balancing module, which is used to dynamically adjust the allocation of computing resources according to the amount of diagnosis requests.
Citation Information
Patent Citations
Tumor malignant risk stratification auxiliary diagnosis system of artificial intelligence medical image
CN109166105A
High-speed rail bearing fault diagnosis method and system based on deep learning and cloud edge cooperation
CN116821794A
Pulmonary nodule intelligent grading method and system based on multi-modal feature fusion
CN116883768A
Magnetic resonance image detection method based on multi-center prototype and pseudo tag
CN117809081A
Federal unsupervised model training and classification method based on prototype consistency and self-marking
CN117975132A
