Intelligent image-based thyroid cancer invasiveness evaluation model and system

Abstract

The invention discloses a thyroid cancer invasiveness evaluation model based on an intelligent image. The thyroid cancer invasiveness evaluation model comprises the following steps: acquiring thyroid nodule ultrasonic image data and clinical text data; performing feature extraction on the thyroid nodule ultrasonic image data to obtain an image feature map; performing feature extraction on the clinical text data to obtain a text feature map; fusing the image feature map and the text feature map, and further performing feature extraction and classification to obtain an evaluation result; wherein the thyroid nodule ultrasonic image data comprises a cross section image of a thyroid nodule, a longitudinal section image of the thyroid nodule, an elastic image of the thyroid nodule and a Doppler ultrasonic image of the thyroid nodule; the clinical text data comprises basic information of a patient and ultrasonic feature information of the thyroid nodule. According to the evaluation model, comprehensive and accurate evaluation of the invasiveness of the thyroid nodules is realized, a powerful auxiliary tool can be provided for clinical diagnosis, and the accuracy and efficiency of diagnosis are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of ultrasonic medical diagnosis and artificial intelligence. More specifically, the present invention relates to a thyroid cancer invasiveness assessment model and system based on intelligent imaging. Background Art

[0002] Papillary thyroid carcinoma (PTC) generally accounts for the main category of thyroid malignancies, specifically accounting for an astonishing 85-90%. Although PTC usually shows a relatively slow course and a relatively good prognosis, once a patient has cervical lymph node metastasis, the risk of recurrence and distant metastasis will increase significantly, thus further threatening the patient's survival. In terms of the number of cervical lymph node metastases, it has been regarded as a core indicator for evaluating the recurrence risk of thyroid cancer.

[0003] The current ultrasonic technology for predicting cervical lymph node metastasis in PTC can be mainly divided into two major principles: one is to focus on examining the image data of the original lesion; the other is to concentrate on analyzing the imaging characteristics of the metastatic lymph nodes. Although the invasiveness prediction based on the characteristics of metastatic lymph nodes seems more intuitive, due to the anatomical location of the thyroid and the inherent limitations of imaging technology, preoperative cervical lymph node ultrasound examination still faces major challenges in efficiently detecting suspicious lymph nodes. Existing research tends to start from the original lesion and further explore various factors of cervical lymph node metastasis. For example, the invention patent "A method for predicting the invasiveness of thyroid nodules based on object detection" with the application publication number CN112927217A and the invention patent "A method for predicting the invasiveness of thyroid nodules based on a deep learning segmentation network" with the application publication number CN112950615A both obtain the prediction results of thyroid nodule invasiveness by processing thyroid ultrasound images, but neither of them considers the influence of clinical data on the prediction of thyroid nodule invasiveness, and the accuracy of the prediction results remains to be improved. Summary of the Invention

[0004] An object of the present invention is to provide a thyroid cancer invasiveness assessment model and system based on intelligent imaging to at least solve the above problems.

[0005] To achieve the objects and other advantages of the present invention, a thyroid cancer invasiveness assessment model based on intelligent imaging is provided, including:

[0006] Obtaining thyroid nodule ultrasound image data and clinical text data;

[0007] Performing feature extraction on the thyroid nodule ultrasound image data to obtain an image feature map;

[0008] Performing feature extraction on the clinical text data to obtain a text feature map;

[0009] Fuse the image feature map and the text feature map, and further perform feature extraction and classification to obtain an evaluation result; where

[0010] The thyroid nodule ultrasound image data includes the cross-sectional image of the thyroid nodule, the longitudinal-sectional image of the thyroid nodule, the elastogram of the thyroid nodule, and the Doppler ultrasound image of the thyroid nodule; the clinical text data includes patient basic information and thyroid nodule ultrasound feature information.

[0011] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment model, an image feature extraction module based on Swin-Transformer is used to extract features from the thyroid nodule ultrasound image data to obtain the image feature map.

[0012] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment model, the clinical text data is first normalized through the LN layer, then a one-dimensional convolutional module is used to mine local features and patterns in the data, and then through the ReLU activation function layer and the linear transformation layer, the text feature map is obtained.

[0013] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment model, the image feature map and the text feature map are concatenated, and after being normalized, through a linear connection layer with a ReLU activation function, features are further extracted and transformed, and then input into the fully connected layer for classification, and through the mapping of the Softmax function, the evaluation result is obtained.

[0014] The present invention also provides an intelligent imaging-based thyroid cancer invasiveness assessment system, including

[0015] An acquisition module, which is used to acquire thyroid nodule ultrasound image data and clinical text data;

[0016] An image feature extraction module, which is used to extract features from the thyroid nodule ultrasound image data to obtain an image feature map;

[0017] A text feature extraction module, which is used to extract features from the clinical text data to obtain a text feature map;

[0018] An evaluation module, which is used to fuse the image feature map and the text feature map, and further perform feature extraction and classification to obtain an evaluation result; where

[0019] The thyroid nodule ultrasound image data includes the cross-sectional image of the thyroid nodule, the longitudinal-sectional image of the thyroid nodule, the elastogram of the thyroid nodule, and the Doppler ultrasound image of the thyroid nodule; the clinical text data includes patient basic information and thyroid nodule ultrasound feature information.

[0020] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment system, an image feature extraction module based on Swin-Transformer is used to extract features from the ultrasonic image data of the thyroid nodules to obtain the image feature map.

[0021] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment system, the clinical text data is first normalized through the LN layer, then a one-dimensional convolutional module is used to mine local features and patterns in the data, and then through the ReLU activation function layer and the linear transformation layer, the text feature map is obtained.

[0022] Preferably, for the intelligent imaging-based thyroid cancer invasiveness assessment system, the image feature map and the text feature map are concatenated, and after being normalized, through a linear connection layer with a ReLU activation function, features are further extracted and transformed, and then input into the fully connected layer for classification, and through the mapping of the Softmax function, the evaluation result is obtained.

[0023] The present invention also provides a storage medium in which a computer program is stored. When the computer program is loaded and executed by a processor, the above-mentioned intelligent imaging-based thyroid cancer invasiveness assessment model is implemented.

[0024] The present invention also provides an electronic device, including: a processor and a memory; wherein, the memory is used to store a computer program; the processor is used to load and execute the computer program so that the electronic device executes the above-mentioned intelligent imaging-based thyroid cancer invasiveness assessment model.

[0025] The present invention at least includes the following beneficial effects:

[0026] By efficiently fusing and analyzing the image features of different types of ultrasonic image data and the text features of clinical text data, the present invention realizes a comprehensive and accurate assessment of the invasiveness of thyroid nodules, can provide a powerful auxiliary tool for clinical diagnosis, and effectively improves the accuracy and efficiency of diagnosis.

[0027] Other advantages, objectives and features of the present invention will be partially reflected by the following description, and partially will also be understood by those skilled in the art through the research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of the overall process of the intelligent imaging-based thyroid cancer invasiveness assessment model according to an embodiment of the present invention;

[0029] Figure 2It is the overall block diagram of the thyroid cancer invasiveness assessment model based on intelligent imaging in an embodiment of the present invention;

[0030] Figure 3 It is the block diagram for extracting characteristics of clinical text data in an embodiment of the present invention;

[0031] Figure 4 It is the block diagram for feature fusion and classification in an embodiment of the present invention;

[0032] Figure 5 It is the evaluation result of the evaluation model of the present invention applied to the test set;

[0033] Figure 6 It is the evaluation result of the thyroid nodule ultrasound image data on the test set using the existing Swin-transformer algorithm;

[0034] Figure 7 It is the evaluation result of the clinical text data on the test set using the existing MLP algorithm;

[0035] Figure 8 It is the overall structural schematic diagram of the thyroid cancer invasiveness assessment system based on intelligent imaging in an embodiment of the present invention. Detailed implementation manners

[0036] The following further elaborates on the present invention in conjunction with embodiments and drawings, so that those skilled in the art can implement it with reference to the text of the specification.

[0037] It should be understood that terms such as "having", "including", and "comprising" as used herein do not exclude the presence or addition of one or more other elements or their combinations.

[0038] It should be noted that the experimental methods described in the following implementation schemes are all conventional methods unless otherwise specified, and the reagents and materials can be obtained from commercial channels unless otherwise specified.

[0039] As Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 shown, an embodiment of the present invention provides a thyroid cancer invasiveness assessment model based on intelligent imaging, including:

[0040] S1. Obtain thyroid nodule ultrasound image data and clinical text data.

[0041] Obtaining of image data: Rich and multi-angle image information is obtained by collecting four types of thyroid nodule images, including:

[0042] (1) Cross-sectional images: Capture the morphology and size of the thyroid nodule in the transverse plane.

[0043] (2) Longitudinal section image: shows the shape and size of the thyroid nodule in the longitudinal plane.

[0044] (3) Doppler ultrasound images: provide information about the blood flow distribution and velocity of thyroid nodules.

[0045] (4) Elasticity image: shows the hardness and elasticity information of thyroid nodules.

[0046] These four types of image data reflect the basic characteristics and possible changes of thyroid nodules from different angles.

[0047] Acquisition of clinical text data: Clinical text data includes basic patient information and ultrasound characteristic information of thyroid nodules. Basic patient information is obtained through inquiries, including demographic information, such as height, weight, age and other basic information, to provide a data basis for the correlation analysis between nodule invasiveness and individual characteristics. Ultrasound characteristic information of thyroid nodules is obtained from the physician's examination, including echo conditions, overall size and blood flow of the gland, location, size, composition, echo characteristics, morphology, margins, the presence or absence of calcification and blood flow conditions, etc.

[0048] S2. Extract features from the thyroid nodule ultrasound image data to obtain an image feature map.

[0049] Specifically, the image feature extraction module based on Swin-Transformer is used to identify and analyze the above-mentioned thyroid nodule ultrasound image data. Through multiple stages of image processing, the features in the image are deeply extracted and the invasiveness of the thyroid nodules is preliminarily classified.

[0050] S3. Extract features from the clinical text data to obtain a text feature graph.

[0051] Specifically, the clinical text data is first standardized through the LN layer (Layer Normalization). After that, the data is passed through a one-dimensional convolution (conv1d) module, and the one-dimensional convolution kernel is used to deeply mine the local features and patterns in the data. Subsequently, the ReLU activation function layer is used to enhance the expressiveness of the model using nonlinear mapping to capture and understand the complex features and information in the data. Finally, the data is processed through a linear transformation layer to generate a refined and efficient text feature map for subsequent model prediction analysis.

[0052] S4: Fusing the image feature map and the text feature map, and further performing feature extraction and classification to obtain an evaluation result.

[0053] Specifically, first, the image feature map and the text feature map are concatenated (Concat), and after normalization processing, they pass through a linear connection layer with ReLU activation function to further extract and transform features. Finally, the data stream enters the next fully connected layer, and through the mapping of the Softmax function, the model will output the evaluation of the invasiveness of the thyroid nodule, that is, the evaluation result of the invasiveness of the thyroid nodule is obtained.

[0054] Through the above steps, the present invention can efficiently fuse and analyze the image features of different types of ultrasound image data and the text features of clinical text data, and achieve a comprehensive and accurate evaluation of the invasiveness of thyroid nodules. As Figure 5 、 Figure 6 and Figure 7 shown, through verification, the evaluation model of the present invention shows a high AUC value (0.924) on the test set, demonstrating high accuracy (96.3%), sensitivity and specificity, which indicates that the model has good predictive ability in non-invasive evaluation of the degree of PTC lymph node invasion and is a predictive method with clinical application value; among them, Figure 5 、 Figure 6 and Figure 7 in (a) is the loss curve during the training process, (b) is the accuracy curve during the training process, and (c) is the AUC curve. As Figure 8 shown, the present invention also provides a thyroid cancer invasiveness evaluation system based on intelligent imaging, which is based on the same inventive concept as the above evaluation model, and includes:

[0055] An acquisition module, which is used to acquire thyroid nodule ultrasound image data and clinical text data.

[0056] Acquisition of image data: By collecting four types of thyroid nodule images, rich and multi-angle image information is obtained, including:

[0057] (1) Cross-sectional image: Capturing the morphology and size of the thyroid nodule in the transverse plane.

[0058] (2) Longitudinal-sectional image: Showing the morphology and size of the thyroid nodule in the longitudinal plane.

[0059] (3) Doppler ultrasound image: Providing information about the blood flow distribution and velocity of the thyroid nodule.

[0060] (4) Elastogram: Showing the hardness and elasticity information of the thyroid nodule.

[0061] These four types of image data reflect the basic characteristics and possible changes of the thyroid nodule from different angles.

[0062] Obtaining clinical text data: Clinical text data includes patients' basic information and ultrasonic feature information of thyroid nodules. The patients' basic information is obtained through inquiry, including demographic information such as basic information like height, weight, age, etc., providing a data basis for the correlation analysis between nodule invasiveness and individual characteristics. The ultrasonic feature information of thyroid nodules is obtained from doctors' examinations, including echo conditions, overall size and blood flow status of the gland, location, size, composition, echo characteristics, shape, margin, presence or absence of calcification and its blood flow situation, etc.

[0063] An image feature extraction module, which is used to extract features from the ultrasonic image data of the thyroid nodules to obtain an image feature map.

[0064] Specifically, an image feature extraction module based on Swin-Transformer is adopted to identify and analyze the above ultrasonic image data of thyroid nodules. Through multiple stages of image processing, features in the image are deeply extracted and the invasiveness of thyroid nodules is preliminarily classified.

[0065] A text feature extraction module, which is used to extract features from the clinical text data to obtain a text feature map.

[0066] Specifically, first, the above clinical text data is normalized through the LN layer (Layer Normalization). Then, the data passes through a one-dimensional convolution (conv1d) module, and the one-dimensional convolution kernel is used to deeply mine local features and patterns in the data. Subsequently, through the ReLU activation function layer, the non-linear mapping is used to enhance the expression ability of the model, capture and understand complex features and information in the data. Finally, the data undergoes the processing of a linear transformation layer, that is, a refined and efficient text feature map for subsequent model prediction and analysis is generated.

[0067] An evaluation module, which is used to fuse the image feature map and the text feature map, and further extract features and classify to obtain an evaluation result.

[0068] Specifically, first, the image feature map and the text feature map are concatenated (Concat), and after normalization processing, then through a linear connection layer with a ReLU activation function, features are further extracted and transformed. Finally, the data stream enters the next fully connected layer, and through the mapping of the Softmax function, the model will output an evaluation of the invasiveness of thyroid nodules, that is, an evaluation result of the invasiveness of thyroid nodules is obtained.

[0069] The intelligent imaging-based thyroid cancer invasiveness evaluation system of the present invention can efficiently fuse and analyze the image features of different types of ultrasonic image data and the text features of clinical text data, and realizes a comprehensive and accurate evaluation of the invasiveness of thyroid nodules.

[0070] The present invention also provides a storage medium, which includes various media capable of storing program codes, such as ROM, RAM, magnetic disk or optical disc, etc., and stores a computer program which, when loaded and executed by a processor, implements all or part of the steps of the above-mentioned intelligent image-based thyroid cancer invasiveness assessment model.

[0071] The present invention also provides an electronic device, which is a device including a processor (CPU / MCU / SOC) and a memory (ROM / RAM), such as a desktop computer, a portable computer, a smart phone, etc. Specifically, the memory stores a computer program which, when loaded and executed by the processor, implements all or part of the steps of the above-mentioned intelligent image-based thyroid cancer invasiveness assessment model.

[0072] The number of devices and the scale of processing described here are used to simplify the description of the present invention. The application, modification and variation of the intelligent image-based thyroid cancer invasiveness assessment model and system of the present invention will be obvious to those skilled in the art.

[0073] Although the embodiments of the present invention have been disclosed as above, they are not limited to only the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those skilled in the art, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the illustrated examples here.

Claims

1. An intelligent image-based thyroid cancer invasion assessment model, characterized in that, Comprising: Obtaining thyroid nodule ultrasound image data and clinical text data; Performing feature extraction on the thyroid nodule ultrasound image data to obtain an image feature map; Performing feature extraction on the clinical text data to obtain a text feature map; Fusing the image feature map and the text feature map, and further performing feature extraction and classification to obtain an evaluation result; wherein, The thyroid nodule ultrasound image data includes a cross-sectional image of the thyroid nodule, a longitudinal-sectional image of the thyroid nodule, an elastic image of the thyroid nodule, and a Doppler ultrasound image of the thyroid nodule; the clinical text data includes patient basic information and thyroid nodule ultrasound feature information.

2. The thyroid cancer invasiveness assessment model based on intelligent imaging according to claim 1, characterized in that Performing feature extraction on the thyroid nodule ultrasound image data by using an image feature extraction module based on Swin-Transformer to obtain the image feature map.

3. The thyroid cancer invasion assessment model based on intelligent imaging according to claim 2, wherein First, performing normalization processing on the clinical text data through an LN layer, then using a one-dimensional convolutional module to mine local features and patterns in the data, and then passing through a ReLU activation function layer and a linear transformation layer to obtain the text feature map.

4. The thyroid cancer invasiveness assessment model based on intelligent imaging according to claim 3, wherein Concatenating the image feature map and the text feature map, after normalization processing, further extracting and transforming features through a linear connection layer with a ReLU activation function, and then inputting them into a fully connected layer for classification, and obtaining the evaluation result through the mapping of the Softmax function.

5. An intelligent image-based thyroid cancer invasiveness assessment system, characterized in that, Including, An acquisition module for obtaining thyroid nodule ultrasound image data and clinical text data; An image feature extraction module for performing feature extraction on the thyroid nodule ultrasound image data to obtain an image feature map; A text feature extraction module for performing feature extraction on the clinical text data to obtain a text feature map; An evaluation module for fusing the image feature map and the text feature map, and further performing feature extraction and classification to obtain an evaluation result; wherein, The thyroid nodule ultrasound image data includes a cross-sectional image of the thyroid nodule, a longitudinal-sectional image of the thyroid nodule, an elastic image of the thyroid nodule, and a Doppler ultrasound image of the thyroid nodule; the clinical text data includes patient basic information and thyroid nodule ultrasound feature information.

6. The thyroid cancer invasiveness assessment system based on intelligent imaging according to claim 5, characterized in that, Performing feature extraction on the thyroid nodule ultrasound image data by using an image feature extraction module based on Swin-Transformer to obtain the image feature map.

7. The thyroid cancer invasiveness assessment system based on intelligent imaging according to claim 6, characterized in that First, performing normalization processing on the clinical text data through an LN layer, then using a one-dimensional convolutional module to mine local features and patterns in the data, and then passing through a ReLU activation function layer and a linear transformation layer to obtain the text feature map.

8. The thyroid cancer invasiveness assessment system based on intelligent imaging according to claim 7, characterized in that Concatenating the image feature map and the text feature map, after normalization processing, further extracting and transforming features through a linear connection layer with a ReLU activation function, and then inputting them into a fully connected layer for classification, and obtaining the evaluation result through the mapping of the Softmax function.

9. A storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by a processor, it implements the intelligent imaging-based thyroid cancer invasiveness assessment model according to any one of claims 1-4.

10. An electronic device, characterized in that, Including: A processor and a memory; wherein, the memory is used for storing a computer program; the processor is used for loading and executing the computer program so that the electronic device executes the intelligent image-based thyroid cancer invasiveness assessment model according to any one of claims 1-4.

Images