Thyroid follicle lesion segmentation model training method, segmentation method and device

Abstract

The application provides a thyroid follicle lesion segmentation model training method, a segmentation method and equipment, relates to the field of image data processing, and the training method comprises the following steps: performing contrast enhancement processing based on contrast limitation on each image block of a thyroid follicular gray-scale ultrasound image; merging and performing edge smoothing transition processing on each contrast enhancement block to obtain an enhanced image sample; and training a target Unet model using the enhanced image sample to obtain a thyroid follicle lesion segmentation model. The application can effectively improve the definition and application reliability of the data sample used to train the thyroid follicle lesion segmentation model, can reduce the risk of overall distortion of the data sample, reduce noise and maintain the overall visual consistency of the data sample, can effectively improve the training effectiveness and reliability of the thyroid follicle lesion segmentation model, and can further improve the accuracy of the sampling high thyroid follicle lesion segmentation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, in particular to a thyroid follicular lesion segmentation model training method, a segmentation method and equipment. BACKGROUND

[0002] Thyroid follicular tumors are a class of tumors that occur in the thyroid, which are derived from follicular cells in the thyroid. Lesion segmentation, as a machine learning task, can automatically extract key information such as morphology, size, boundary and structure of the lesion through accurate segmentation of thyroid lesions, providing a strong basis for disease benign and malignant classification. In particular, in the differentiation of follicular thyroid cancer and follicular adenoma, which are very similar in pathological structure, the importance of segmentation technology is particularly prominent.

[0003] At present, in the existing thyroid follicular lesion segmentation technology, most algorithms only focus on optimizing the network structure, and the exploration of image preprocessing and image enhancement technology is insufficient, thereby ignoring the low quality problem of the ultrasound image itself, resulting in poor robustness of the model to noise and artifacts, and it is difficult to achieve efficient and stable segmentation effect in real clinical environment. SUMMARY

[0004] In view of this, the embodiments of the present application provide a thyroid follicular lesion segmentation model training method, a segmentation method and equipment to eliminate or improve one or more defects in the prior art.

[0005] The first aspect of the present application provides a thyroid follicular lesion segmentation model training method, comprising:

[0006] Performing contrast enhancement processing based on contrast restriction on each image block corresponding to each thyroid follicular gray-scale ultrasound image respectively to obtain a contrast-enhanced block corresponding to each image block;

[0007] Merging and performing edge smoothing transition processing on each contrast-enhanced block belonging to one thyroid follicular gray-scale ultrasound image to obtain an enhanced image sample corresponding to each thyroid follicular gray-scale ultrasound image;

[0008] training the target Unet model as a thyroid follicle lesion segmentation model for outputting a thyroid follicle lesion region segmented from the enhanced image sample and a classification label of the thyroid follicle lesion region; wherein the label set comprises a lesion mask label and a classification label of the thyroid follicular gray-scale ultrasound image corresponding to the enhanced image sample; the lesion mask label is used to display a thyroid follicle lesion region; and the classification label comprises a benign label and a malignant label.

[0009] In some embodiments of the present application, the contrast enhancement processing based on the contrast limit is respectively performed on each image block corresponding to each thyroid follicular gray-scale ultrasound image to obtain a contrast enhanced block corresponding to each image block, comprising:

[0010] each thyroid follicular gray-scale ultrasound image is divided into a plurality of image blocks;

[0011] the cumulative distribution value of the histogram corresponding to each image block is calculated respectively;

[0012] if the cumulative distribution value of the histogram corresponding to the image block is greater than the contrast limit threshold, the target cumulative distribution value corresponding to the image block is set as the contrast limit threshold; if the cumulative distribution value of the histogram corresponding to the image block is less than or equal to the contrast limit threshold, the target cumulative distribution value corresponding to the image block is set as the cumulative distribution value of the histogram corresponding to the image block;

[0013] each image block is subjected to contrast enhancement processing according to the target cumulative distribution value corresponding to each image block to obtain a contrast enhanced block corresponding to each image block.

[0014] In some embodiments of the present application, the contrast enhanced blocks belonging to one thyroid follicular gray-scale ultrasound image are merged and subjected to edge smoothing transition processing to obtain an enhanced image sample corresponding to each thyroid follicular gray-scale ultrasound image, comprising:

[0015] the contrast enhanced blocks belonging to one thyroid follicular gray-scale ultrasound image are merged to obtain a merged image corresponding to each thyroid follicular gray-scale ultrasound image;

[0016] the edge smoothing transition processing is performed between the contrast enhanced blocks belonging to one merged image in a bilinear interpolation manner to obtain an enhanced image sample corresponding to each thyroid follicular gray-scale ultrasound image.

[0017] In some embodiments of the present application, a cross-fusion feature context Transformer module is arranged between the output end of the encoder and the input end of the decoder in the target Unet model.

[0018] The cross-fusion feature context Transformer module is configured to sequentially perform feature aggregation, multi-scale feature extraction, grouped convolution, and feature interaction processing based on a multi-head attention mechanism on the down-sampling feature data corresponding to the enhanced image sample output by the encoder, to obtain a target feature map corresponding to the enhanced image sample, and input the target feature map into the decoder to enable the decoder to output thyroid follicular lesion segmentation result data corresponding to the enhanced image sample according to the target feature map; wherein the thyroid follicular lesion segmentation result data is used to represent a thyroid follicular lesion region segmented from the enhanced image sample and a classification label corresponding to the thyroid follicular lesion region.

[0019] In some embodiments of the present application, the cross-fusion feature context Transformer module includes a feature aggregation layer, a multi-scale feature extraction layer, a grouped convolution layer based on depth separable convolution, a Transformer module based on a multi-head attention mechanism, and an output layer.

[0020] The feature aggregation layer is configured to perform initial feature aggregation processing on the down-sampling feature data corresponding to the enhanced image sample output by the encoder using a convolution kernel, and perform layer normalization processing on the down-sampling feature data after the initial feature aggregation processing, to obtain aggregated feature data corresponding to the enhanced image sample.

[0021] The multi-scale feature extraction layer is configured to perform convolution feature extraction on the aggregated feature data using convolution kernels of different sizes respectively, to obtain extracted feature maps of different scales corresponding to the aggregated feature data, and fuse the extracted feature maps to obtain convolution extracted feature data corresponding to the enhanced image sample.

[0022] The grouped convolution layer based on depth separable convolution is configured to divide the convolution extracted feature data into feature image blocks of a fixed size; perform a flattening operation on each feature image block, and map each feature image block into an embedding vector through linear projection; perform depth-wise convolution and point-wise convolution on each embedding vector using a depth separable convolution manner, to obtain injection feature maps of each channel of the convolution extracted feature data, and then aggregate the injection feature maps of each channel using a convolution kernel to obtain a dynamic feature vector corresponding to the enhanced image sample.

[0023] The Transformer module based on the multi-head attention mechanism is configured to perform feature interaction on the dynamic feature vector based on the multi-head attention mechanism to obtain convolution injection feature data corresponding to the enhanced image sample.

[0024] The output layer is configured to fuse the convolution extraction feature data and the convolution injection feature data corresponding to the enhanced image sample to obtain a target feature map corresponding to the enhanced image sample.

[0025] In some embodiments of the present application, a region attention module is arranged in an encoder in the target Unet model.

[0026] The region attention module is configured to sequentially adopt a spatial attention mechanism and a channel attention mechanism to perform feature extraction on input data to obtain region attention feature data corresponding to the enhanced image sample, and input the region attention feature data into a downsampling layer in the encoder of the region attention module to make the downsampling layer output downsampling feature data corresponding to the region attention feature data.

[0027] If the encoder in which the region attention module is arranged is the first encoder in the target Unet model, the input data is the enhanced image sample; if the encoder in which the region attention module is arranged is a non-first encoder in the target Unet model, the input data is downsampling feature data output by another encoder connected to the input side of the encoder.

[0028] In some embodiments of the present application, the region attention module comprises a spatial attention layer and a channel attention layer.

[0029] The spatial attention layer is configured to adopt a convolution layer and a ReLU activation function connected in sequence to obtain a spatial attention map corresponding to the input data, and multiply the spatial attention map and the input data element by element to obtain spatial attention feature data corresponding to the enhanced image sample.

[0030] The channel attention layer is configured to adopt a global pooling layer, a 1-dimensional convolution layer and a ReLU activation function connected in sequence to obtain a spatial and channel attention map corresponding to the spatial attention feature data, and multiply the spatial and channel attention map and the input data element by element to obtain region attention feature data corresponding to the enhanced image sample.

[0031] The second aspect of the present application provides a thyroid follicular lesion segmentation method, comprising:

[0032] perform contrast enhancement processing based on contrast restriction on each image block corresponding to the target thyroid follicular gray-scale ultrasound image to obtain a contrast-enhanced block of each of the image blocks;

[0033] merge and perform edge smoothing transition processing on each of the contrast-enhanced blocks to obtain an enhanced image sample corresponding to the target thyroid follicular gray-scale ultrasound image;

[0034] input the enhanced image sample into a thyroid follicular lesion segmentation model to enable the thyroid follicular lesion segmentation model to output a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region; wherein the thyroid follicular lesion segmentation model is obtained by pre-training based on the thyroid follicular lesion segmentation model training method of the first aspect.

[0035] A third aspect of the present application provides a thyroid follicular lesion segmentation model training device, comprising:

[0036] a first contrast enhancement module configured to perform contrast enhancement processing based on contrast restriction on each image block corresponding to each of the thyroid follicular gray-scale ultrasound images to obtain a contrast-enhanced block of each of the image blocks;

[0037] a first smoothing transition module configured to merge and perform edge smoothing transition processing on each of the contrast-enhanced blocks belonging to one of the thyroid follicular gray-scale ultrasound images to obtain an enhanced image sample corresponding to each of the thyroid follicular gray-scale ultrasound images;

[0038] a model training module configured to train a target Unet model using each of the enhanced image samples and a label group corresponding to each of the enhanced image samples to train the target Unet model into a thyroid follicular lesion segmentation model configured to output a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region; wherein the label group comprises a lesion mask label and a classification label of the thyroid follicular gray-scale ultrasound image corresponding to the enhanced image sample; the lesion mask label is configured to display a thyroid follicular lesion region; and the classification label comprises a benign label and a malignant label.

[0039] A fourth aspect of the present application provides a thyroid follicular lesion segmentation device, comprising:

[0040] a second contrast enhancement module configured to perform contrast enhancement processing based on contrast restriction on each image block corresponding to the target thyroid follicular gray-scale ultrasound image to obtain a contrast-enhanced block of each of the image blocks;

[0041] The second smooth transition module is configured to merge and perform edge smooth transition processing on each of the contrast enhancement blocks to obtain an enhanced image sample corresponding to the target thyroid follicular grayscale ultrasound image.

[0042] The model segmentation module is configured to input the enhanced image sample into a thyroid follicle lesion segmentation model to output a thyroid follicle lesion region segmented from the enhanced image sample and a classification label of the thyroid follicle lesion region by the thyroid follicle lesion segmentation model.

[0043] A fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the thyroid follicle lesion segmentation model training method according to the first aspect, and / or implement the thyroid follicle lesion segmentation method according to the second aspect.

[0044] A sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the thyroid follicle lesion segmentation model training method according to the first aspect, and / or implement the thyroid follicle lesion segmentation method according to the second aspect.

[0045] A seventh aspect of the present application provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the thyroid follicle lesion segmentation model training method according to the first aspect, and / or implement the thyroid follicle lesion segmentation method according to the second aspect.

[0046] The thyroid follicle lesion segmentation model training method provided in the application performs contrast enhancement processing based on contrast limitation on each image block corresponding to each thyroid follicular gray-scale ultrasound image respectively to obtain a contrast enhancement block corresponding to each image block; each contrast enhancement block belonging to one thyroid follicular gray-scale ultrasound image is merged and edge-smooth transition processed to obtain an enhanced image sample corresponding to each thyroid follicular gray-scale ultrasound image; a target Unet model is trained using each enhanced image sample and a label group corresponding to each enhanced image sample to train the target Unet model into a thyroid follicle lesion segmentation model for outputting a thyroid follicle lesion region segmented from the enhanced image sample and a classification label of the thyroid follicle lesion region; wherein the label group includes a lesion mask label and a classification label of the thyroid follicular gray-scale ultrasound image corresponding to the enhanced image sample; the lesion mask label is used to display a thyroid follicle lesion region; and the classification label includes a benign label and a malignant label. That is, by independently performing contrast enhancement on each image block, the application can selectively adjust the contrast for different image regions, thereby reducing the risk of overall distortion; by performing contrast enhancement processing based on contrast limitation, the application can prevent excessive enhancement of noise and artifacts, thereby improving image clarity while effectively reducing noise; by merging and edge-smooth transition processing each contrast enhancement block belonging to one thyroid follicular gray-scale ultrasound image, the application can eliminate the gray difference between adjacent image blocks to maintain the overall visual consistency of the data sample; thereby the application reliability of the data sample used to train the thyroid follicle lesion segmentation model can be improved, the training effectiveness and reliability of the thyroid follicle lesion segmentation model can be effectively improved, and the accuracy of the sampling thyroid follicle lesion segmentation result can be improved.

[0047] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following figures and detailed description thereof or can be learned by practice of the application. The advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0048] It will be understood by those skilled in the art that the objectives and advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. For purposes of clarity and a consistent approach, portions of the drawings may have been exaggerated from the actual scale, and are intended as an aid for describing and understanding the embodiments of this application and are intended not to limit the present application. In the drawings:

[0050] Figure 1 The first flowchart of the thyroid follicular lesion segmentation model training method in an embodiment of the present application.

[0051] Figure 2 The second flowchart of the thyroid follicular lesion segmentation model training method in an embodiment of the present application.

[0052] Figure 3 The total architecture of the target Unet model in an embodiment of the present application.

[0053] Figure 4 The architecture of the cross-fusion feature context Transformer module in an embodiment of the present application.

[0054] Figure 5 The architecture of the CFCT module in an application example of the present application.

[0055] Figure 6 The specific architecture of the target Unet model in an embodiment of the present application.

[0056] Figure 7 The architecture of the region attention module in an embodiment of the present application.

[0057] Figure 8 The flowchart of the thyroid follicular lesion segmentation method in an embodiment of the present application.

[0058] FIG. 9(a) is a comparison result schematic diagram of the original and the CLAHE preprocessed thyroid follicular carcinoma (FTC) image in an application example of the present application.

[0059] FIG. 9(b) is a comparison result schematic diagram of the original and the CLAHE preprocessed thyroid follicular adenoma (FA) image in an application example of the present application.

[0060] Figure 10 The data set construction flowchart in an application example of the present application.

[0061] FIG. 11(a) is a schematic diagram of a comparison of the thyroid follicular lesion segmentation model in an application example of the present application and other models for thyroid follicular carcinoma (FTC) image segmentation results.

[0062] FIG. 11(b) is a schematic diagram of a comparison of the thyroid follicular lesion segmentation model in an application example of the present application and other models for thyroid adenoma (FA) image segmentation results.

[0063] Figure 12 FIG. 10 is a structural schematic diagram of a thyroid follicular lesion segmentation model training device in an embodiment of the present application.

[0064] Figure 13 FIG. 9 is a structural schematic diagram of a thyroid follicular lesion segmentation device in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed description will be given to the present application in combination with embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but not as a limitation to the present application.

[0066] It should be noted that, in order to avoid obscuring the present application due to unnecessary details, only structures and / or processing steps closely related to the scheme according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0067] It should be emphasized that the term “comprises / comprising” is used herein to indicate the presence of a feature, element, step or component, but not to exclude the presence or addition of one or more other features, elements, steps or components.

[0068] It should be noted that, if not specifically stated, the term “connected” herein can not only mean direct connection, but also indirect connection with an intermediate.

[0069] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0070] Artificial intelligence technology is increasingly closely integrated with medical images. Using artificial intelligence to process medical images has become a major research hotspot. In the medical process, doctors spend most of their time manually segmenting the area of interest to calculate organ function indicators. Applying artificial intelligence to image segmentation not only greatly reduces time costs and speeds up the medical diagnosis process, but also serves as a prerequisite for tasks such as auxiliary diagnosis. Due to its low resolution, high signal-to-noise ratio, and other characteristics, ultrasound image segmentation has now become one of the research hotspots.

[0071] The thyroid gland is the largest endocrine gland in the human body, located below the thyroid cartilage in the neck, alongside the trachea. It is an important organ that controls the rate at which the body uses energy, produces proteins, and regulates sensitivity to other endocrine hormones.

[0072] Thyroid cancer is a malignant tumor that originates from thyroid follicular epithelial or parafollicular epithelial cells, and is also the most common malignant tumor of the head and neck. According to the origin and differentiation of the tumor, thyroid cancer is divided into papillary thyroid carcinoma (PTC), follicular thyroid carcinoma (FTC), and other categories. Follicular thyroid carcinoma is the second most common pathological type of thyroid cancer, accounting for 10-15% of all thyroid cancers. Follicular carcinoma usually has a capsule, invasive growth, and is prone to hematogenous metastasis to the lungs and bone tissue, with a prognosis generally worse than that of papillary carcinoma. Thyroid follicular tumors are divided into benign follicular adenomas and malignant follicular carcinomas, with about 10% of follicular tumors prone to malignant transformation. The cell morphology of these two conditions is similar, and preoperative cytological examination cannot determine the presence of a capsule and vascular invasion, so preoperative diagnosis of thyroid follicular carcinoma and thyroid follicular adenoma is relatively difficult.

[0073] With the continuous development of science and technology, techniques such as nuclear magnetic resonance imaging, computed tomography, and ultrasonography have been widely applied in medical examinations. Due to its economic, convenient, non-invasive, and real-time advantages, ultrasonography has become a commonly used diagnostic method for thyroid nodules. It has achieved good results in terms of locating the size and number of nodules, diagnosing malignant tumors in combination with fine needle aspiration cytology (FNAC), distinguishing solid from cystic lesions, and improving the performance of ultrasonography in thyroid nodule diagnosis using the ultrasonography risk stratification system (US-RSS). However, ultrasonography has certain limitations in differentiating thyroid follicular carcinoma from thyroid follicular adenoma: thyroid follicular carcinoma and thyroid follicular adenoma often exhibit similar characteristics on ultrasonography, making it difficult to distinguish between the two; the presence of some thyroid follicular carcinoma uncertainties and benign lesions with atypical features increases the difficulty of ultrasonography differentiation; capsule invasion and vascular invasion are key diagnostic "gold standards" for thyroid follicular carcinoma, but these features are difficult to accurately observe on ultrasonography. Preoperative diagnosis of follicular carcinoma and benign adenoma is relatively difficult, and only postoperative pathological examination can effectively distinguish between the two. However, postoperative pathological diagnosis can affect the normal function of the thyroid gland.

[0074] In the current era of big data, artificial intelligence has become a hot research topic in the field of the intersection of medicine and engineering, and its application is rapidly driving the development of medical diagnosis and disease management. At present, the application of artificial intelligence technology in disease diagnosis has covered multiple fields, such as data analysis and real-time warning of critical illnesses, accurate lesion positioning and prediction based on imaging examination results, auxiliary differential diagnosis of diseases, risk prediction of benign and malignant lesions, and evaluation of the effectiveness of treatment plans, etc. However, there are still obvious deficiencies in the field of artificial intelligence research on thyroid follicular carcinoma. Most current research focuses on the screening and risk prediction of thyroid papillary carcinoma, while research on thyroid follicular carcinoma is relatively scarce. Due to the similarity between follicular carcinoma and follicular adenoma in pathological and imaging manifestations, the difficulty of its diagnosis is significantly increased. In addition, existing research mostly relies on gray-scale ultrasound, a single modality of image data, while the performance of follicular carcinoma in gray-scale ultrasound images is often highly similar to that of adenoma, making it challenging to distinguish between the two using a single modality.

[0075] Further, gray-scale ultrasound has some inherent limitations in medical imaging. Compared with other imaging techniques such as computed tomography (CT), gray-scale ultrasound images generally have lower signal-to-noise ratio, lower resolution, and smaller contrast, resulting in insufficient image detail information. In addition, the acquisition of ultrasound images depends on the experience of the operator and the performance of the equipment, and this human factor and equipment difference can cause significant fluctuations in image quality. Under the combined effect of these unfavorable conditions, the accuracy and robustness of artificial intelligence models based on ultrasound images in classifying and diagnosing thyroid follicular tumors are severely constrained.

[0076] Therefore, it is of great significance to develop an intelligent screening and malignant risk prediction method for thyroid follicular tumors that can break through the existing technical bottlenecks. This method should make full use of artificial intelligence technology to achieve efficient fusion and deep learning of multi-modal data, automatically complete the segmentation and feature extraction of ultrasound images, and overcome the limitations of traditional gray-scale imaging. In addition, by combining ultrasound data with data from other modalities, the accuracy and stability of diagnosis can be effectively improved, thereby providing new technical support for the early screening and accurate diagnosis of thyroid follicular tumors.

[0077] Ultrasound is the first-line examination for thyroid nodules, and has high sensitivity and specificity in distinguishing benign and malignant nodules. However, existing studies on ultrasound in thyroid cancer are based on papillary thyroid cancer. Because thyroid follicular carcinoma and thyroid follicular adenoma often show similar features on ultrasound imaging, it is still controversial whether ultrasound can be directly used for thyroid follicular carcinoma. In medicine, the World Health Organization has proposed a diagnostic decision tree for follicular thyroid cancer, which requires a 7-step decision-making process to make a definitive diagnosis of thyroid follicular carcinoma. Moreover, fine needle aspiration biopsy (FNAB) cannot distinguish between thyroid follicular carcinoma and adenoma. Therefore, the preoperative diagnosis of thyroid follicular carcinoma is relatively difficult in clinical practice.

[0078] Currently, deep learning has been widely used in medical diagnosis, and artificial intelligence-based medical diagnosis methods have gradually emerged in disease risk prediction and difficult disease classification. In the classification and diagnosis of thyroid nodules, computer-aided diagnosis (CAD) and deep learning methods have been tried and have achieved good results. Some scholars have applied artificial intelligence technology to thyroid cell pathology, which helps to distinguish between papillary carcinoma and benign lesions, distinguish between follicular adenoma and carcinoma, and identify non-invasive follicular thyroid tumors with papillary nuclear features. Some scholars use deep separable convolutional neural network Xception as the main classification network to extract feature information through multi-scale convolution of ultrasound images, and predict and classify various thyroid nodular lesions including thyroid cancer. In the comparison experiment of multiple networks, the best result was obtained, with a classification accuracy of 97.2%. Some scholars use a two-stage convolutional neural network (CNN) to use tissue section staining results to predict the benign or malignant results of follicular nodules, with a sensitivity of 92.0% and a specificity of 90.5%. Some scholars use residual network ResNet or improved networks based on ResNet and classic convolutional neural network VGG network model for multi-classification of thyroid nodular lesions, and also achieved remarkable results, getting diagnostic results not inferior to professional physicians.

[0079] However, the existing research has the following problems: first, the use of ultrasonic thyroid nodule images may not fully represent all the characteristics of the lesion due to its low resolution, high signal-to-noise ratio and other characteristics, and the potential information loss problem may significantly affect the performance of the deep learning model; in addition, since the ultrasonic image needs to be manually intercepted, there is no standard for image interception, and the accuracy of data labeling is also a major reason for the performance of the model; most of the existing researches are aimed at the large category of diseases (such as inflammation, cancer, etc.), without detailed research, especially in the research of thyroid follicular tumors, there is still a big gap; it is difficult to distinguish thyroid follicular carcinoma and thyroid adenoma from ultrasonic images alone.

[0080] Thanks to the increased computing power, algorithm optimization, and inherent flexibility of GPUs, deep learning methods have surpassed traditional machine learning approaches in object classification, localization, and semantic segmentation. Based on the U-net model, some studies have achieved segmentation of nasopharyngeal tumor magnetic resonance imaging (MRI) images. This approach utilizes a contracting path to acquire surrounding information and, based on this, expands the path to achieve precise localization. Some researchers have combined the results of the previous dilated convolution algorithm with those of traditional convolution. The stitched image is then transferred to the next dilated convolutional layer, and a dense atrous spatial pyramid pooling (DenseASPP) model is proposed. Some scholars have combined tracking three-dimensional ultrasound with CNN segmentation, significantly reducing the inter-observer variability of thyroid volume measurement and improving measurement accuracy through shorter acquisition times. Some scholars have proposed a new hybrid Transformer-UNet (H-TUNet) to segment the thyroid gland in ultrasound sequences. The proposed method outperforms other state-of-the-art methods in the ultrasound dataset TSUD and the medical imaging dataset TG3k. Transformer-UNet is a deep learning architecture that combines the Transformer, a deep learning model based on an attention mechanism, with the U-shaped fully convolutional neural network (UNet), primarily used in fields such as medical image segmentation. Its core idea is to combine the global context processing capabilities of the Transformer with the local feature extraction advantages of the UNet to improve segmentation accuracy. Based on the densely connected convolutional network DenseNet-121 network structure model, combined with the core module ASPP (Atrous Spatial Pyramid Pooling) for dense prediction tasks (especially semantic segmentation), some scholars proposed a new thyroid nodule ultrasound image segmentation model, which greatly improved the segmentation effect of thyroid nodule ultrasound images. Some scholars also proposed a thyroid region prior guided feature enhancement network TRFE-Net for thyroid nodule segmentation. This framework uses inferred thyroid region priors to enhance the feature representation of thyroid nodule segmentation. Other scholars proposed an FCN architecture called IVUS-Net, followed by a post-processing contour extraction step, to automatically segment the internal (lumen) and external (media and adventitia) regions of human arteries.

[0081] However, existing research has the following problems: First, due to the large amount of noise, fuzzy boundaries and complex background problems in ultrasound images, the traditional segmentation model has low segmentation accuracy in ultrasound images; in order to overcome the shortcomings of ultrasound images, the complexity of the segmentation model increases, which is not ideal for accelerating training and reducing optimization difficulty; due to the heterogeneity of the appearance of thyroid nodules and the possibility that thyroid tissue may be confused with the edge effects of other tissues under ultrasound, how to accurately segment and classify them is a challenge.

[0082] Follicular thyroid tumors can be divided into benign follicular adenomas (FA) and malignant follicular thyroid carcinomas (FTC). Benign adenomas usually have a capsule and do not invade, while positive follicular thyroid cancer can invade blood vessels and potentially metastasize, with common metastatic sites including bone or lung. Because adenomas and follicular carcinomas exhibit similar cytological characteristics, it is difficult to reliably distinguish between the two by fine needle biopsy, and it remains a great challenge to distinguish follicular adenomas from thyroid follicular carcinomas by ultrasound due to the lack of specific ultrasound features.

[0083] In recent years, deep learning-based methods have been widely used in the classification of thyroid nodule lesions due to their ability to analyze complex features in medical images and pathological data. Lesion segmentation, as a machine learning task, can automatically extract key information such as morphology, size, boundary, and structure of the lesion by accurately segmenting the thyroid lesion, providing a strong basis for the benign and malignant classification of the disease. In particular, when distinguishing between pathological structures such as follicular thyroid carcinoma and follicular adenoma, which are very similar, the importance of segmentation technology is particularly prominent. Compared to manual image analysis, automated segmentation using artificial intelligence not only speeds up the process, but also reduces subjective errors caused by differences in physician experience, thus better guiding the staging of thyroid cancer, planning of surgical range, and calculation of radiotherapy dose, thereby optimizing treatment strategies. However, some existing segmentation models often focus only on optimizing network structure, and insufficient exploration of image preprocessing and image enhancement techniques, thus ignoring the low quality of ultrasound images themselves, resulting in poor robustness of the model to noise and artifacts, making it difficult to achieve efficient and stable segmentation results in real clinical environments.

[0084] Therefore, in order to improve the clarity and application reliability of data samples for training thyroid follicular lesion segmentation models, the embodiments of the present application provide a thyroid follicular lesion segmentation model training method, a thyroid follicular lesion segmentation model training device for executing the thyroid follicular lesion segmentation model training method, a thyroid follicular lesion segmentation method, a thyroid follicular lesion segmentation device, an entity device, a computer readable storage medium, and a computer program product, which can reduce the risk of overall distortion of data samples, reduce noise, and maintain the overall visual consistency of data samples, effectively improve the training effectiveness and reliability of thyroid follicular lesion segmentation models, and further improve the accuracy of sampling thyroid follicular lesion segmentation results.

[0085] The embodiments are specifically described as follows.

[0086] Based on this, the application provides a thyroid follicular lesion segmentation model training method which can be implemented by a thyroid follicular lesion segmentation model training device, referring to Figure 1 The thyroid follicular lesion segmentation model training method specifically includes the following contents:

[0087] Step 100: Perform contrast enhancement processing based on contrast restriction on each image block corresponding to each thyroid follicular gray-scale ultrasound image respectively to obtain a contrast-enhanced block corresponding to each image block.

[0088] In one or more embodiments of the application, the thyroid follicular gray-scale ultrasound image can be referred to as a thyroid ultrasound image or an ultrasound image, both of which refer to an ultrasound image showing thyroid follicular adenoma or thyroid follicular carcinoma. In the thyroid follicular gray-scale ultrasound image

[0089] The image block refers to each sub-block obtained by cutting or dividing a thyroid follicular gray-scale ultrasound image, which can also be referred to as a window or a grid region. In an example, the thyroid follicular gray-scale ultrasound image can be divided into a plurality of image blocks of the same size.

[0090] In step 100, contrast enhancement processing is independently performed on each image block. This localized method can selectively adjust the contrast of different image regions, thereby reducing the risk of overall distortion of the thyroid follicular gray-scale ultrasound image. In order to reduce the risk of overall distortion of the data sample and reduce noise, in step 100, contrast enhancement processing based on contrast restriction is independently performed on each image block. By setting a contrast restriction, the over-enhancement of image block noise and artifacts in the contrast enhancement process is prevented.

[0091] In an example, the contrast restriction can be implemented by a contrast restriction threshold E, which is specifically described in subsequent embodiments.

[0092] Step 200: Merge and perform edge smoothing transition processing on each contrast-enhanced block belonging to one of the thyroid follicular gray-scale ultrasound images to obtain an enhanced image sample corresponding to each of the thyroid follicular gray-scale ultrasound images.

[0093] In step 200, for each thyroid follicular gray-scale ultrasound image, each contrast-enhanced block corresponding to the thyroid follicular gray-scale ultrasound image is merged and subjected to edge smoothing transition processing to obtain an enhanced image sample corresponding to the thyroid follicular gray-scale ultrasound image. By smoothing the transition between adjacent sub-blocks, the gray-scale difference between the edges of adjacent image blocks is eliminated to maintain the overall visual consistency of the data sample.

[0094] Specifically, the principle of ultrasound imaging is based on the propagation, reflection and scattering characteristics of high-frequency sound waves. The ultrasound probe is both a transmitting and receiving side, and the internal signal processing system calculates the depth and echo intensity of the reflecting interface to generate an ultrasound image. At this time, if the acoustic impedance of two adjacent tissues, that is, the difference in the ability of sound waves to pass through the medium, is small, the reflected acoustic signal will be weak, resulting in poor image contrast. When ultrasound waves propagate in human tissues, scattering and absorption phenomena occur, which are particularly pronounced in soft tissues. Scattering causes the distribution of reflected signals to be chaotic, and at this time the receiver receives weaker echo signals, and the contrast of the image is thus poor. And common artifacts in ultrasound images, such as acoustic shadows, enhancement effects, mirror artifacts, etc. also affect the contrast. Therefore, the problem of poor contrast of ultrasound images is the result of the combined action of multiple factors, which brings great challenges to the accurate identification of lesions.

[0095] Therefore, in order to improve the quality of thyroid ultrasound images before training, the technical means provided by the above steps 100 and 200 in the embodiments of the present application can be referred to as a preprocessing method based on contrast limited adaptive histogram equalization (CLAHE), which can be referred to as CLAHE. CLAHE is an advanced adaptive histogram equalization technique designed to enhance the contrast of local regions of an image while avoiding excessive amplification of noise and artifacts. It maintains visual consistency by applying contrast enhancement in local regions and smoothing the transition between adjacent regions.

[0096] In addition, the thyroid follicular lesion segmentation model training apparatus also receives a label group corresponding to each of the thyroid follicular gray-scale ultrasound images at the same time as the thyroid follicular gray-scale ultrasound images are obtained or before step 300 is performed. The label group includes a lesion mask label and a classification label of the thyroid follicular gray-scale ultrasound image corresponding to the enhanced image sample; the lesion mask label is used to display the thyroid follicular lesion area; and the classification label includes a benign label and a malignant label. The lesion mask label and the classification label can be extracted from the results of pathological examination of the slices of the thyroid follicular gray-scale ultrasound images.

[0097] The label group of the thyroid follicular gray-scale ultrasound image is then used as the label group of the enhanced image sample corresponding to the thyroid follicular gray-scale ultrasound image to perform the following step 300. It should be noted that the thyroid follicular gray-scale ultrasound images mentioned in the present application for training the target Unet model are all ultrasound images authorized by the patient for use.

[0098] Step 300: training a target Unet model by using each of the enhanced image samples and a label set corresponding to each of the enhanced image samples, to train the target Unet model into a thyroid follicular lesion segmentation model for outputting a thyroid follicular lesion region segmented from the enhanced image samples and a classification label of the thyroid follicular lesion region; wherein the label set comprises a lesion mask label and a classification label of the thyroid follicular gray-scale ultrasound image corresponding to the enhanced image sample; the lesion mask label is used to display a thyroid follicular lesion region; and the classification label comprises a benign label and a malignant label.

[0099] In step 300, the target Unet model refers to the Unet model currently as a target model framework, which can adopt a general Unet model architecture, or can be further improved for improving the accuracy and robustness of thyroid ultrasound image segmentation, which will be described in detail in subsequent embodiments.

[0100] The benign label is used to represent a probability value of the thyroid follicle corresponding to the enhanced image sample being a benign thyroid follicular adenoma; and the malignant label is used to represent a probability value of the thyroid follicular adenoma corresponding to the enhanced image sample being a malignant thyroid follicular carcinoma.

[0101] As can be seen from the above description, the thyroid follicular lesion segmentation model training method provided by the embodiments of the present application can selectively adjust the contrast for different image regions by independently performing contrast enhancement for each image block, thereby reducing the risk of overall distortion; by performing contrast enhancement processing based on contrast limitation, it can prevent excessive enhancement of noise and artifacts, thereby improving image clarity while effectively reducing noise; by merging and edge smoothing transition processing of each contrast enhancement block belonging to one thyroid follicular gray-scale ultrasound image, it can eliminate the gray difference between adjacent image blocks to maintain the overall visual consistency of the data sample; and further improve the application reliability of the data sample for training the thyroid follicular lesion segmentation model, effectively improve the training effectiveness and reliability of the thyroid follicular lesion segmentation model, and further improve the accuracy of the sampling thyroid follicular lesion segmentation result.

[0102] In order to further improve the clarity and application reliability of the data sample for training the thyroid follicular lesion segmentation model, and reduce the risk of overall distortion of the data sample and reduce noise, in the thyroid follicular lesion segmentation model training method provided by the embodiments of the present application, referring to Figure 2 , the step 100 in the thyroid follicular lesion segmentation model training method specifically contains the following contents:

[0103] Step 110: Divide each thyroid follicular grayscale ultrasound image into a plurality of image blocks;

[0104] Step 120: Calculating the cumulative distribution value of the histogram corresponding to each of the image blocks respectively;

[0105] Step 130: If the cumulative distribution value of the histogram corresponding to the image block is greater than the contrast limit threshold, setting the target cumulative distribution value corresponding to the image block as the contrast limit threshold; if the cumulative distribution value of the histogram corresponding to the image block is less than or equal to the contrast limit threshold, setting the target cumulative distribution value corresponding to the image block as the cumulative distribution value of the histogram corresponding to the image block;

[0106] In step 130, the target cumulative distribution value of the image block can be calculated using the following formula:

[0107]

[0108] in, Grayscale ultrasound image of thyroid follicles The target cumulative distribution value of the image patch in ; Indicates the size (dimensions) of the image block; represents the cumulative distribution value of the histogram corresponding to the image block; is the contrast limit threshold; Indicates that if the cumulative distribution value of the histogram corresponding to the image block is less than or equal to the ratio limit threshold ; Indicates that if the cumulative distribution value of the histogram corresponding to the image block is greater than the ratio limit threshold .

[0109] Step 140: performing contrast enhancement processing on each of the image blocks according to the target cumulative distribution value corresponding to each of the image blocks, so as to obtain a contrast enhanced block for each of the image blocks.

[0110] In order to further improve the clarity and application reliability of the data samples used to train the thyroid follicular lesion segmentation model and maintain the overall visual consistency of the data samples, in a thyroid follicular lesion segmentation model training method provided in an embodiment of the present application, see Figure 2 Step 200 in the thyroid follicular lesion segmentation model training method specifically includes the following contents:

[0111] Step 210: Merge the contrast enhancement blocks belonging to the same thyroid follicular grayscale ultrasound image to obtain merged images corresponding to the respective thyroid follicular grayscale ultrasound images.

[0112] Step 220: performing edge smoothing transition processing between the contrast enhancement blocks belonging to the same merged image in a bilinear interpolation manner to obtain enhanced image samples corresponding to the respective thyroid follicular grayscale ultrasound images.

[0113] Specifically, since each image block is processed independently of its grayscale histogram, in order to avoid discontinuity at the boundaries between image blocks, an interpolation method can be used to smooth the transition between adjacent image blocks. The output grayscale value of each pixel is calculated as follows:

[0114]

[0115] in, is the position of the input thyroid follicle in the grayscale ultrasound image The pixel value of is the position of the input thyroid follicle in the grayscale ultrasound image The corresponding output pixel value at and Represent the minimum and maximum values ​​of the gray level respectively; Indicates the location of thyroid follicles in grayscale ultrasound images The target cumulative distribution value of the image block corresponding to .

[0116] Subsequently, bilinear interpolation is applied to smooth the transition between image patches.

[0117] In solving the above-mentioned existing thyroid follicular lesion segmentation technology, most algorithms only focus on optimizing the network structure, and insufficiently explore image preprocessing and image enhancement technologies, thereby ignoring the low quality of ultrasound images themselves. In addition, the existing technology also has the following technical problems:

[0118] Thyroid cancers vary greatly in morphology and size, and some tumors have irregular shapes. In the actual clinical diagnosis of thyroid cancer, physicians must not only identify long-range features such as the anatomical structure of the thyroid gland and surrounding tissues and conduct a comprehensive analysis of the overall morphology of the image, but also analyze short-range features such as smaller lesion details or tissue boundaries. Long-range features can help the model capture global anatomical structures and long-range dependencies, while short-range features play a key role in distinguishing local details and lesion boundaries. However, most models only focus on the extraction of local information and ignore the necessity of coordinated processing of global and local information. This results in ordinary models not being able to segment tumors with complex morphology or lesions with blurred boundaries in an ideal way.

[0119] Based on the above, based on the embodiment of the thyroid follicular lesion segmentation model training method, for the shape and size difference of thyroid cancer, and some tumors have irregular shape problem, in the thyroid follicular lesion segmentation model training method provided in the embodiment of the present application, see Figure 3 , the output end of the encoder in the target Unet model in the thyroid follicular lesion segmentation model training method is provided with a cross-fusion feature context Transformer module between the input end of the decoder;

[0120] The cross-fusion feature context Transformer module is used for sequentially performing feature aggregation, multi-scale feature extraction, grouped convolution and feature interaction processing based on a multi-head attention mechanism on the down-sampling feature data corresponding to the enhanced image sample output by the encoder, to obtain a target feature map corresponding to the enhanced image sample, and input the target feature map into the decoder to make the decoder output thyroid follicular lesion segmentation result data corresponding to the enhanced image sample according to the target feature map; wherein the thyroid follicular lesion segmentation result data is used to represent the thyroid follicular lesion region segmented from the enhanced image sample and the classification label corresponding to the thyroid follicular lesion region.

[0121] Wherein, the Transformer module (or model) is a deep learning model architecture, mainly used for processing sequence to sequence tasks, especially in the field of natural language processing (NLP). The core of the Transformer model is its attention mechanism, which realizes the transformation processing of input data by calculating the relationship between each vector in the matrix. The purpose is to remove invalid information and strengthen effective information, so as to better solve the final mathematical problem and calculate the optimal solution.

[0122] Wherein, the cross-fusion feature context Transformer module can be referred to as CFCT (Cross-fusion Feature Contextual Transformer) module.

[0123] In order to further solve the problems of Unet in capturing global context information, ignoring the fusion of global features, and not enough extraction of fine-grained short-distance features, in the thyroid follicular lesion segmentation model training method provided in the embodiment of the present application, see Figure 4 , the cross-fusion feature context Transformer module in the thyroid follicular lesion segmentation model training method includes: a feature aggregation layer, a multi-scale feature extraction layer, a grouped convolution layer based on depth separable convolution, a Transformer module based on multi-head attention mechanism and an output layer;

[0124] The feature aggregation layer is configured to perform initial feature aggregation processing on down-sampled feature data corresponding to the enhanced image sample output by the encoder using a convolution kernel, and perform layer normalization processing on the down-sampled feature data after the initial feature aggregation processing, to obtain aggregated feature data corresponding to the enhanced image sample. Figure 5 The convolution kernel in the feature aggregation layer can be a 3 × 3 convolution kernel.

[0125] The multi-scale feature extraction layer is configured to perform convolution feature extraction on the aggregated feature data using convolution kernels of different sizes, to obtain extracted feature maps of different scales corresponding to the aggregated feature data, and fuse the extracted feature maps to obtain convolution extracted feature data corresponding to the enhanced image sample. Figure 5 The convolution kernels of different sizes in the multi-scale feature extraction layer can be a 1 × 1 convolution kernel and a 3 × 3 convolution kernel.

[0126] The group convolution layer based on deep separable convolution is configured to divide the convolution extracted feature data into feature image blocks of a fixed size; perform a flattening operation on each feature image block, and map each feature image block into an embedding vector through linear projection; perform depth-wise convolution and point-wise convolution on each embedding vector using a deep separable convolution method, to obtain injection feature maps of each channel of the convolution extracted feature data, and then aggregate the injection feature maps of each channel using a convolution kernel to obtain a dynamic feature vector corresponding to the enhanced image sample.

[0127] The Transformer module based on the multi-head attention mechanism is configured to perform feature interaction on the dynamic feature vector based on the multi-head attention mechanism, to obtain convolution injection feature data corresponding to the enhanced image sample. Figure 5 The dynamic feature vector input into the Transformer module based on the multi-head attention mechanism contains Q (Query), K (Key), and V (Value), which are core components of self-attention (Self-Attention), and realize context modeling by dynamically calculating the relevance of different positions in the input sequence. Q represents a query vector, which is used to match Key vectors of other positions; K represents a key vector, which provides semantic information to be queried; and V represents a value vector, which carries the features actually transmitted.

[0128] The output layer is configured to fuse the convolution extracted feature data and the convolution injection feature data corresponding to the enhanced image sample, to obtain a target feature map corresponding to the enhanced image sample.

[0129] In solving the above existing thyroid follicular lesion segmentation technology, most algorithms only focus on optimizing the network structure, and the exploration of image preprocessing and image enhancement technology is insufficient, thereby ignoring the low quality problem of the ultrasound image itself, and most models only focus on the extraction of local information, ignoring the necessity of cooperative processing of global and local information, which leads to the technical problem that the ordinary model does not have an ideal segmentation effect on tumors with complex morphology or lesions with fuzzy boundaries. Based on the above technical problems of the existing thyroid follicular lesion segmentation model training method, there are still the following technical problems:

[0130] In the thyroid ultrasound image, not only the thyroid gland and the tumor itself are usually included, but also many irrelevant anatomical structures in the image, such as muscle tissue, trachea, artery and vein, which are not helpful for diagnosis and even interfere with the content. The existence of these irrelevant contents increases the complexity of the algorithm in extracting effective information related to the lesion, for example, the high-contrast echo of the trachea and blood vessels may mask the signal returned by the thyroid nodule, thereby causing the model to misjudge or making the model unable to accurately capture the boundary of the lesion. The existing algorithm lacks focus on the target area when extracting features. This not only reduces the segmentation accuracy of the algorithm, but also may cause the model to perform unstably in real clinical scenarios.

[0131] Based on the above embodiment of the thyroid follicular lesion segmentation model training method, the thyroid ultrasound image may include many irrelevant anatomical structures, such as muscle tissue, trachea, artery and vein. These irrelevant contents, which are not helpful for diagnosis and even interfere with the content, increase the complexity and effectiveness of the algorithm in extracting effective information related to the lesion. In the thyroid follicular lesion segmentation model training method provided in the embodiment of the present application, referring to Figure 6 , the encoder in the target Unet model is provided with a region attention module;

[0132] The region attention module is used for sequentially adopting a spatial attention mechanism and a channel attention mechanism to extract features from the input data to obtain region attention feature data corresponding to the enhanced image sample; and inputting the region attention feature data into a downsampling layer in the encoder to make the downsampling layer output downsampling feature data corresponding to the region attention feature data.

[0133] Wherein, if the encoder where the region attention module is located is the first encoder in the target Unet model, the input data is the enhanced image sample; if the encoder where the region attention module is located is not the first encoder in the target Unet model, the input data is the downsampling feature data output by another encoder connected to the input side of the encoder.

[0134] The target Unet model comprises four encoders and four decoders, each decoder corresponds to each encoder, each encoder comprises a regional attention module and a down-sampling layer, and each decoder comprises an up-sampling layer. The input (output) and the one-to-one corresponding four groups of decoders and encoders form five stages, and the channel numbers of each stage from top to bottom are 16, 32, 64, 128 and 160.

[0135] The regional attention module can be abbreviated as RA (Regional Attention) module.

[0136] In order to accurately extract the features related to the target region while effectively suppressing the information of irrelevant regions, in an embodiment of the thyroid follicle lesion segmentation model training method provided in the present application, referring to Figure 7 , the regional attention module in the thyroid follicle lesion segmentation model training method comprises a spatial attention layer and a channel attention layer.

[0137] The spatial attention layer is used to obtain a spatial attention map corresponding to the input data by using a convolution layer and a ReLU activation function connected in sequence, and to obtain spatial attention feature data corresponding to the enhanced image sample by multiplying the spatial attention map and the input data element by element.

[0138] The channel attention layer is used to obtain a spatial and channel attention map corresponding to the spatial attention feature data by using a global pooling layer, a 1-dimensional convolution layer and a ReLU activation function connected in sequence, and to obtain regional attention feature data corresponding to the enhanced image sample by multiplying the spatial and channel attention map and the input data element by element.

[0139] Based on the above embodiment of the thyroid follicle lesion segmentation model training method, the present application further provides an embodiment of a thyroid follicle lesion segmentation method, referring to Figure 8 , the thyroid follicle lesion segmentation method specifically comprises the following contents:

[0140] Step 400: Perform contrast enhancement processing based on contrast restriction on each image block corresponding to the target thyroid follicular gray-scale ultrasound image to obtain a contrast-enhanced block for each image block.

[0141] Step 500: Merge and perform edge smoothing transition processing on each contrast-enhanced block to obtain an enhanced image sample corresponding to the target thyroid follicular gray-scale ultrasound image.

[0142] Step 600: inputting the enhanced image sample into the thyroid follicular lesion segmentation model, so that the thyroid follicular lesion segmentation model outputs a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region; wherein the thyroid follicular lesion segmentation model is obtained by pre-training based on the thyroid follicular lesion segmentation model training method.

[0143] In the thyroid follicular lesion segmentation method, step 600 adopts the thyroid follicular lesion segmentation model training method mentioned in the foregoing embodiments, and specific steps refer to the thyroid follicular lesion segmentation model training method mentioned in the foregoing embodiments, which will not be described herein again.

[0144] As can be seen from the foregoing description, the thyroid follicular lesion segmentation method provided by the embodiments of the present application can improve the accuracy and application reliability of the sampling thyroid follicular lesion segmentation result, and provides more accurate and effective auxiliary diagnosis data for clinical use.

[0145] In order to further illustrate the embodiments of the thyroid follicular lesion segmentation model training method and the thyroid follicular lesion segmentation method, the present application further provides a specific application example of the thyroid follicular lesion segmentation model training method, i.e. a thyroid follicular lesion segmentation algorithm based on Transformer. By deploying an image enhancement and segmentation integrated pipeline, and designing a segmentation model based on Unet model, Transformer model, channel attention and spatial attention, the above problems can be solved, and the quality of thyroid follicular lesion segmentation can be effectively improved to achieve the purpose of assisting doctors in diagnosis and treatment. The application example of the present application is specifically described as follows:

[0146] (I) CLAHE-based image preprocessing method

[0147] For the CLAHE algorithm, it mainly includes three steps:

[0148] (1) Image block division: the CLAHE algorithm divides the ultrasound image into smaller image blocks or grid regions, for example, the ultrasound image can be divided into a total of 8x8=64 sub-blocks, called windows, and the contrast enhancement is independently performed in each window. This localized method can selectively adjust the contrast of different image regions, thereby reducing the risk of overall distortion.

[0149] (2) Contrast limited: Within each image block, the CLAHE algorithm limits the magnitude of contrast enhancement. By setting a limit threshold, it constrains the values of some gray level histograms to prevent excessive enhancement of noise and artifacts. Excess histogram values beyond the threshold are redistributed to other gray levels, effectively reducing noise while improving image clarity.

[0150] (3) Gray mapping: After applying contrast limited histogram equalization to each image block, CLAHE combines these results into a whole image and eliminates the gray level difference between image blocks and image block edges by smoothing the transition between adjacent image blocks.

[0151] Specifically, if now for a thyroid follicular gray-scale ultrasound image , let be the number of times the gray level appears, be the total number of pixels. The probability of the gray level appearing in the thyroid follicular gray-scale ultrasound image is represented in the form of equation (3-1):

[0152] (3-1)

[0153] The cumulative distribution function (CDF) of the histogram is a function of the image gray level, which describes how many pixels in the image are not greater than the gray level. The definition of the cumulative distribution function of the histogram is given by equation (3-2):

[0154] (3-2)

[0155] In the CLAHE algorithm, each image is divided into image blocks of size . In the histogram equalization process of each image block, the CLAHE limits the contrast limit threshold of the cumulative distribution function, which reduces the risk of amplifying noise by preventing excessive enhancement of high-contrast areas, where E=40. The target cumulative distribution value of each image block can be calculated by equation (3-3):

[0156] (3-3)

[0157] Since each image patch in CLAHE is processed independently for its histogram, to avoid the discontinuity between image patches, CLAHE uses an interpolation method to smooth the transition between adjacent image patches. The output gray value of each pixel is calculated as shown in equation (3-4). Wherein, is the pixel value at position in the input thyroid follicular gray-scale ultrasound image, is the corresponding output pixel value at position in the input thyroid follicular gray-scale ultrasound image; and represent the minimum and maximum values of the gray scale, respectively; represents the target cumulative distribution value of the image patch corresponding to the position in the thyroid follicular gray-scale ultrasound image. Subsequently, bilinear interpolation is applied to smooth the boundary transition between image patches.

[0158] (3-4)

[0159] Figure 9 (a) shows the comparison results of the original and the CLAHE preprocessed thyroid follicular carcinoma (FTC) image; Figure 9 (b) shows the comparison results of the original and the CLAHE preprocessed thyroid follicular adenoma (FA) image. Compared with the original thyroid follicular carcinoma and thyroid follicular adenoma images, the method of the present application enhances the local details and reduces the noise effect. At the same time, the method improves the problem of uneven brightness, but the computational complexity is not significantly increased. Therefore, this method is suitable for real-time diagnosis applications, and can provide clearer visibility of subtle structures, thereby improving the diagnosis quality in poor visibility ultrasound images.

[0160] (II) Thyroid follicular carcinoma segmentation lesion network

[0161] 1. Network architecture

[0162] The design of segmentation models based on thyroid tumors has become a popular research direction. Initially, it was to solve the segmentation task at the cellular level, and now Unet has become a widely used benchmark model in medical image segmentation tasks. Despite its wide application, UNet has limitations in modeling long-range dependencies, which limits its ability to fully capture global context information. Since the skip connection in Unet directly transmits local features, it often ignores the fusion of global features. At the same time, high-resolution spatial and channel information will also be partially lost in the multiple downsampling process, thereby hindering the extraction of fine features.

[0163] To solve the above limitations, the application application instance proposes to integrate a novel Cross-fusion Feature Contextual Transformer (CFCT) module and a Regional Attention (RA) module into a two-dimensional Unet model. The introduction of the two modules is to solve the problems of long and short distance fusion and target region focusing mentioned above.

[0164] The architecture of the target UNet model is shown in Figure 6 The encoder of the benchmark two-dimensional UNet model contains five stages, and the number of channels of stages 1 to 5 is 16, 32, 64, 128, and 160, respectively. The size of the input and output ultrasound images is set to 224x224.

[0165] (1) Cross-fusion Feature Contextual Transformer module

[0166] In the style of the thyroid gray ultrasound image, not only long-distance anatomical features need to be focused on, but also short-distance features such as gland-like textures need to be focused on. In view of the problems of Unet in capturing global contextual information, ignoring the fusion of global features, and not enough extraction of short-distance features, the application application instance proposes CFCT to solve the above problems.

[0167] The architecture of the CFCT module is shown in Figure 5 The CFCT module starts from the initial aggregation of features. The input feature map is first aggregated by a 3x3 convolution kernel. This step not only extracts surface features, but also smooths the input features, effectively suppressing noise and irrelevant details. Then, layer normalization is applied to stabilize the training process by normalizing the feature distribution and enhance the generalization ability of the model. The process of layer normalization is shown in Table 1.

[0168] Table 1

[0169]

[0170] Then, multi-scale feature extraction is performed. Multi-scale features are extracted by combining 1x1 and 3x3 convolution layers. The 1x1 convolution layer focuses on extracting fine-grained local features while reducing channel redundancy. While the 3x3 convolution layer captures more rich contextual information with its larger receptive field, supplementing local operations to model long-range dependencies. This multi-scale strategy can effectively extract short-range and long-range features to effectively fuse local details and global context, and then generate comprehensive feature representations.

[0171] In the following, the application introduces a convolution-injected operation. In this stage, the input image is divided into fixed-size feature image blocks, which are then flattened and mapped into embedding vectors through linear projection. However, this operation destroys the spatial relationship in the image, which may lead to the loss of local details and poor multi-scale feature modeling. To alleviate these problems, the application introduces depth-wise separable convolutions in the Transformer module. Depth-wise separable convolutions are composed of depth-wise convolutions and point-wise convolutions. The role of depth-wise convolutions is to extract spatial features, while the role of point-wise convolutions is to extract channel features. It can be understood that depth-wise separable convolutions group convolution in feature dimension, and each channel is independently convolved by depth-wise convolution, and all channels are aggregated using a 3x3 convolution before output.

[0172] The advantage of using depth-wise separable convolutions is that depth-wise separable convolutions naturally embed location information through local calculation in spatial dimension, thus eliminating the need for explicit position encoding in the original Transformer. Moreover, using depth-wise separable convolutions instead of the original linear mapping allows input features to directly interact with spatial context information through convolution operations, achieving efficient modeling of local and global features. The scalability of depth-wise separable convolution operations also allows the application to adapt to different scale feature extraction requirements by adjusting the size of the convolution kernel.

[0173] If the encoding feature data set obtained after information extraction in the encoder of Unet is , if the operation of the entire CFCT module can be described as , then the output set of the CFCT module designed by the application is . Here, the input enhanced image sample is , the target feature map corresponding to the output enhanced image sample after processing is , and and represent the convolution extraction and convolution injection Transformer processing, respectively. The overall output can be represented as equation (3-5):

[0174] (3-5)

[0175] wherein, represents; represents the CFCT operation on the enhanced image sample; representing the convolutional extracted feature data corresponding to the enhanced image sample; representing the convolutional injected feature data.

[0176] In summary, the design of CFCT ensures that the scale of input and output feature maps remains unchanged, enabling deep feature re-extraction and multi-level feature fusion. By dynamically capturing global context and local details, the CFCT module significantly enhances the ability to model long-range dependencies and integrate complex features, improving its performance in various medical imaging tasks.

[0177] (2) Regional attention mechanism

[0178] Since the thyroid ultrasound image may contain many irrelevant anatomical structures, such as muscle tissue, trachea, arteries and veins, etc. These information is not helpful for diagnosis and even irrelevant content that interferes with the algorithm in extracting effective information related to the lesion increases the complexity and effectiveness. In order to solve the existence of this problem, the application instance proposes a regional attention (RA) module specially designed for ultrasound image segmentation, and the architecture of the regional attention (RA) module is as shown in Figure 7 In the design of the application instance, the spatial attention mechanism (SAM) can dynamically highlight the regions corresponding to follicular carcinoma and adenoma in the image, and by enhancing the features of these spatial regions, the spatial attention mechanism can reduce the interference of irrelevant structures and enhance the spatial positioning of the target. The channel attention mechanism (CAM) assigns different weights to the feature channels, emphasizing the features related to the target region, while also suppressing redundant features caused by irrelevant structures. This ensures that the model focuses on valuable information for diagnosis, improving segmentation accuracy. The combination of spatial attention and channel attention enables accurate extraction of features related to the target region while effectively suppressing information from irrelevant regions.

[0179] The spatial attention mechanism reduces the dimension of the input feature map to 1 through convolution operation, thereby realizing spatial weighting. This operation can generate a spatial attention map, which can be used to highlight relevant parts such as lesion regions, while effectively suppressing the interference of anatomical structures such as trachea, arteries, etc. In this way, the spatial attention module can help the model focus on more important regions, reducing the impact of irrelevant structures on segmentation. Especially in thyroid ultrasound images, there are many complex anatomical structures, and the spatial weighting mechanism of spatial attention can effectively improve the spatial positioning accuracy of the target region, thereby improving the segmentation effect.

[0180] The channel attention mechanism adopts a lightweight architecture in the network model ECANe for learning the context-related vector representation of nodes, but replaces the fully connected layer with a 1x1 convolution operation. The main purpose of this design is to improve computational efficiency by reducing the parameter amount of the model, while retaining sufficient expression ability of the model. The channel attention module calculates the weight of each channel through the operation of global average pooling, and adjusts each channel of the input feature map according to the channel weight. In this way, the channel attention not only enhances the features related to the target region, but also suppresses the redundant features irrelevant to the target. This mechanism enables the model to focus on key diagnostic information, improving segmentation accuracy, especially when dealing with complex thyroid structures.

[0181] In addition, the output part of each attention module in the application examples of the present application also adds a point-by-point jump connection, which allows the features of different attention modules to be directly fused, helping to maintain multi-scale information and avoid the loss of detailed information during feature extraction. Especially when dealing with multi-scale thyroid regions, the application examples of the present application can effectively avoid oversimplification or loss of details, thereby improving the robustness of the model. The final output is processed through a ReLU activation function. The use of ReLU as an attention gating mechanism can further enhance the nonlinear representation of features, not only helping the model to add nonlinear relationships, but also filtering useless feature information, so that the model can better adapt to complex medical images in the case of various lesion morphologies and complex structures. Through the combination of the above series of mechanisms, the spatial attention and channel attention work together to significantly improve the accuracy and robustness of thyroid ultrasound image segmentation.

[0182] (Three) Construction of data set

[0183] The data set used by the application examples of the present application contains thyroid gray-scale ultrasound images collected from clinical practice, covering two different types of thyroid tumors: benign follicular adenoma (FA) and malignant thyroid follicular carcinoma (FTC). Each case is pathologically diagnosed by subsequent histopathological examination, further enhancing the reliability of the data set. The overall data set construction process is shown in Figure 10

[0184] ​After that, image enhancement and mask annotation of the collected thyroid ultrasound images are also needed. In the process of image enhancement, all images are enhanced by the introduced CLAHE algorithm, and further data enhancement is performed by random inversion and random cropping. In the process of the experiment, a single NVIDIA RTX A6000 graphics card is used, which is equipped with 48 GB of video memory. The high-performance GPU provides sufficient computing power support for intensive training and testing tasks. The entire experiment is implemented based on the open-source deep learning framework Pytorch for machine learning and deep learning. In terms of training configuration, the batch size is set to 16, balancing the memory occupation and training efficiency. The initial learning rate for controlling the model weight update step is set to 10 (-4) , and a smaller initial value ensures stable convergence in the training process. The model has undergone 300 complete training cycles, and cross-entropy is selected as the loss function. The cross-entropy function calculates the difference between the predicted probability distribution and the true label, and has a significant advantage in classification tasks. In terms of parameter optimization, the Adam optimizer widely used in the field of deep learning is adopted. This algorithm combines the advantages of the adaptive gradient algorithm AdaGrad and the adaptive learning rate optimization algorithm RMSProp, and can realize the adaptive and efficient update of the model weight. To further improve the training effect, the application also introduces a cosine annealing learning rate decay strategy: as the training process advances, the learning rate is dynamically adjusted according to the cosine curve. Such a design can not only achieve more fine-tuning of parameters in the later training stage, but also effectively suppress the overfitting phenomenon in the training process.

[0185] The effect comparison chart of the thyroid follicle lesion segmentation model used by the application example and other models is shown in FIG. 11(a) and FIG. 11(b). FIG. 11(a) includes a thyroid follicular carcinoma original image (FTC Original), a label (Ground Truth), a U-shaped convolutional network (Unet), an attention gate mechanism-based U-shaped convolutional network (AttentionUnet), a U-shaped convolutional network++ (Unet++), a U-shaped convolutional network based on a transformer architecture (TransUnet), a U-shaped convolutional network+contrast limited adaptive histogram equalization (Unet+CLAHE), a U-shaped convolutional network+cross fusion feature context Transformer module (Unet+CFCT), a U-shaped convolutional network+region attention (Unet+RA), and the segmentation model provided by the application. FIG. 11(b) includes a thyroid follicular adenoma original image (FA Original), a label (Ground Truth), a U-shaped convolutional network (Unet), an attention gate mechanism-based U-shaped convolutional network (AttentionUnet), a U-shaped convolutional network++ (Unet++), a U-shaped convolutional network based on a transformer architecture (TransUnet), a U-shaped convolutional network+contrast limited adaptive histogram equalization (Unet+CLAHE), a U-shaped convolutional network+cross fusion feature context Transformer module (Unet+CFCT), a U-shaped convolutional network+region attention (Unet+RA), and the segmentation model provided by the application.

[0186] From the software level, the application also provides a thyroid follicle lesion segmentation model training device for executing all or part of the thyroid follicle lesion segmentation model training method, which is shown in Figure 12 , and the thyroid follicle lesion segmentation model training device specifically includes the following contents:

[0187] The first contrast enhancement module 10 is configured to perform contrast enhancement processing based on contrast limitation on each image block corresponding to each thyroid follicular gray-scale ultrasound image, respectively, to obtain a contrast enhancement block corresponding to each image block.

[0188] The first smooth transition module 20 is configured to merge and perform edge smooth transition processing on each contrast enhancement block belonging to one of the thyroid follicular gray-scale ultrasound images, to obtain an enhanced image sample corresponding to each of the thyroid follicular gray-scale ultrasound images.

[0189] The model training module 30 is configured to train a target Unet model by using each of the enhanced image samples and a label set corresponding to each of the enhanced image samples, so as to train the target Unet model into a thyroid follicular lesion segmentation model for outputting a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region. The label set includes a lesion mask label and a classification label of the thyroid follicular grayscale ultrasound image corresponding to the enhanced image sample. The lesion mask label is used to display the thyroid follicular lesion region. The classification label includes a benign label and a malignant label.

[0190] The embodiment of the thyroid follicular lesion segmentation model training apparatus provided in the present application can be used to execute the processing procedure of the embodiment of the thyroid follicular lesion segmentation model training method described above, and the functions thereof will not be repeated here. Please refer to the detailed description of the embodiment of the thyroid follicular lesion segmentation model training method described above.

[0191] The part of the thyroid follicular lesion segmentation model training apparatus for performing the thyroid follicular lesion segmentation model training can be completed in a client device. Specifically, the selection can be made according to the processing capability of the client device and the limitation of the user's use scenario, etc. The present application does not make any limitation in this regard. If all the operations are completed in the client device, the client device can further include a processor for performing the specific processing of the thyroid follicular lesion segmentation model training.

[0192] The client device described above can have a communication module (i.e., a communication unit) and can be communicatively connected with a remote server to realize the data transmission with the server. The server can include a server of a task scheduling center side, and can also include a server of an intermediate platform in other implementation scenarios, such as a server of a third-party server platform communicatively connected with the server of the task scheduling center. The server can include a single computer device, or can include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0193] The server and the client device can use any suitable network protocol to communicate with each other, including a network protocol that has not been developed at the filing date of the present application. The network protocol can include, for example, a TCP / IP protocol, a UDP / IP protocol, an HTTP protocol, an HTTPS protocol, etc. Of course, the network protocol can also include, for example, a RPC protocol (Remote Procedure Call Protocol) used on the basis of the above-mentioned protocols, a REST protocol (Representational State Transfer), etc.

[0194] As can be known from the above description, the thyroid follicular lesion segmentation model training device provided by the embodiment of the present application can independently perform contrast enhancement on each image block, selectively adjust the contrast for different image regions, and thus reduce the risk of overall distortion; by performing contrast enhancement processing based on contrast limitation, the over-enhancement of noise and artifacts can be prevented, so that the image clarity is improved while effectively reducing noise; by merging and performing edge smoothing transition processing on each contrast enhancement block belonging to the same thyroid follicular adenoma grayscale ultrasound image, the grayscale difference between adjacent image blocks can be eliminated to maintain the overall visual consistency of the data sample; and thus the application reliability of the data sample for training the thyroid follicular carcinoma lesion segmentation model can be improved, the training effectiveness and reliability of the thyroid follicular carcinoma lesion segmentation model can be effectively improved, and thus the accuracy of the sampling thyroid follicular carcinoma lesion segmentation result can be improved.

[0195] From the software aspect, the present application also provides a thyroid follicular lesion segmentation device for executing all or part of the thyroid follicular lesion segmentation method, which is described below with reference to Figure 13 , and the thyroid follicular lesion segmentation device specifically includes the following contents:

[0196] The second contrast enhancement module 40 is configured to perform contrast enhancement processing based on contrast limitation on each image block corresponding to the target thyroid follicular grayscale ultrasound image, respectively, to obtain a contrast enhancement block for each image block.

[0197] The second smoothing transition module 50 is configured to merge and perform edge smoothing transition processing on each contrast enhancement block to obtain an enhanced image sample corresponding to the target thyroid follicular grayscale ultrasound image.

[0198] The model segmentation module 60 is configured to input the enhanced image sample into a thyroid follicular lesion segmentation model, so that the thyroid follicular lesion segmentation model outputs a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region; and the thyroid follicular lesion segmentation model is trained based on the thyroid follicular lesion segmentation model training method of the first aspect.

[0199] The embodiment of the thyroid follicular lesion segmentation device provided by the present application can be specifically used to execute the processing flow of the embodiment of the thyroid follicular lesion segmentation method in the above embodiment, and the functions thereof will not be described here again, and can be referred to the detailed description of the above embodiment of the thyroid follicular lesion segmentation method.

[0200] The electronic device can include a processor, a memory, a receiver, and a transmitter. The processor is configured to perform the thyroid follicular lesion segmentation model training method mentioned in the above embodiments and / or the thyroid follicular lesion segmentation method provided in the second aspect. The processor and the memory can be connected by a bus or other means. The receiver can be connected to the processor and the memory by wired or wireless means.

[0201] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof.

[0202] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the thyroid follicular lesion segmentation model training method in the embodiments of the present application and / or the thyroid follicular lesion segmentation method provided in the second aspect. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the thyroid follicular lesion segmentation model training method in the above method embodiments and / or the thyroid follicular lesion segmentation method provided in the second aspect.

[0203] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0204] The one or more modules are stored in the memory and, when executed by the processor, perform the thyroid follicular lesion segmentation model training method in the embodiments and / or the thyroid follicular lesion segmentation method provided by the second aspect.

[0205] In some embodiments of the present application, the user equipment can include a processor, a memory, and a transceiver which can include a receiver and a transmitter, the processor, the memory, the receiver and the transmitter can be connected through a bus system, the memory is configured to store computer instructions, and the processor is configured to execute the computer instructions stored in the memory to control the transceiver to transceive signals.

[0206] As an implementation manner, the functions of the receiver and the transmitter in the present application can be implemented by a transceiver circuit or a dedicated transceiver chip, and the processor can be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.

[0207] As another implementation manner, the server provided by the embodiments of the present application can be implemented by using a general-purpose computer. That is, program codes for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general-purpose processor implements the functions of the processor, the receiver and the transmitter by executing the codes in the memory.

[0208] The embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the thyroid follicular lesion segmentation model training method and / or the thyroid follicular lesion segmentation method provided by the second aspect. The computer readable storage medium can be a tangible storage medium such as a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0209] The embodiments of the present application further provide a computer program product including a computer program, and the computer program is executed by a processor to implement the steps of the thyroid follicular lesion segmentation model training method and / or the thyroid follicular lesion segmentation method provided by the second aspect.

[0210] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application-Specific Integrated Circuit (ASIC), appropriate firmware, a plug-in card, a function card, etc. When implemented in software, the elements of the present application are the program or code segments to perform a particular task. The program or code segments can be stored in a machine-readable medium, or carried by a data signal in a carrier wave through a transmission medium or a communication link.

[0211] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the figures. For simplicity, detailed descriptions of known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0212] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0213] The above description is merely illustrative of the application, and is not intended to limit the application. The embodiments of the application can be modified and changed in various ways by those skilled in the art, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A thyroid follicular lesion segmentation model training method, characterized in that: include: performing contrast enhancement processing based on contrast limitation on each image block corresponding to each thyroid follicular grayscale ultrasound image, so as to obtain a contrast enhanced block for each image block; Merging the contrast enhancement blocks belonging to the same thyroid follicular grayscale ultrasound image and performing edge smoothing to obtain enhanced image samples corresponding to the respective thyroid follicular grayscale ultrasound images; A target Unet model is trained using each of the enhanced image samples and the label group corresponding to each of the enhanced image samples, so as to train the target Unet model into a thyroid follicular lesion segmentation model for outputting a thyroid follicular lesion region segmented from the enhanced image sample and a classification label of the thyroid follicular lesion region; wherein the label group includes a lesion mask label and a classification label of the thyroid follicular grayscale ultrasound image corresponding to the enhanced image sample; the lesion mask label is used to display the thyroid follicular lesion region; and the classification label includes a benign label and a malignant label; The step of performing contrast enhancement processing based on contrast limitation on each image block corresponding to each thyroid follicular grayscale ultrasound image to obtain a contrast enhanced block for each image block includes: Divide each thyroid follicular grayscale ultrasound image into multiple image blocks; Calculating the cumulative distribution value of the histogram corresponding to each of the image blocks respectively; If the cumulative distribution value of the histogram corresponding to the image block is greater than a contrast limit threshold, the target cumulative distribution value corresponding to the image block is set as the contrast limit threshold; if the cumulative distribution value of the histogram corresponding to the image block is less than or equal to the contrast limit threshold, the target cumulative distribution value corresponding to the image block is set as the cumulative distribution value of the histogram corresponding to the image block; Contrast enhancement processing is performed on each of the image blocks according to the target cumulative distribution value corresponding to each of the image blocks, so as to obtain a contrast enhanced block for each of the image blocks.

2. The thyroid follicular lesion segmentation model training method according to claim 1, characterized in that: The merging and edge smoothing processing of the contrast enhancement blocks belonging to the same thyroid follicular grayscale ultrasound image to obtain enhanced image samples corresponding to the respective thyroid follicular grayscale ultrasound images includes: Merging the contrast enhancement blocks belonging to the same thyroid follicular grayscale ultrasound image to obtain merged images corresponding to the respective thyroid follicular grayscale ultrasound images; In a bilinear interpolation manner, edge smoothing transition processing is performed between the contrast enhancement blocks belonging to the same merged image to obtain enhanced image samples corresponding to the respective thyroid follicular grayscale ultrasound images.

3. The thyroid follicular lesion segmentation model training method according to claim 1, characterized in that: A cross-fusion feature context Transformer module is provided between the output end of the encoder and the input end of the decoder in the target Unet model; The cross-fusion feature context Transformer module is used to perform feature aggregation, multi-scale feature extraction, group convolution and feature interaction processing based on a multi-head attention mechanism on the downsampled feature data corresponding to the enhanced image sample output by the encoder in sequence to obtain a target feature map corresponding to the enhanced image sample, and input the target feature map into the decoder so that the decoder outputs the thyroid follicular lesion segmentation result data corresponding to the enhanced image sample according to the target feature map; wherein the thyroid follicular lesion segmentation result data is used to represent the thyroid follicular lesion area segmented from the enhanced image sample and the classification label corresponding to the thyroid follicular lesion area.

4. The thyroid follicular lesion segmentation model training method according to claim 3, characterized in that: The cross-fusion feature context Transformer module includes: a feature aggregation layer, a multi-scale feature extraction layer, a grouped convolution layer based on depthwise separable convolution, a Transformer module based on a multi-head attention mechanism, and an output layer; The feature aggregation layer is used to perform initial feature aggregation processing on the downsampled feature data corresponding to the enhanced image samples output by the encoder using a convolution kernel, and perform layer normalization processing on the downsampled feature data after the initial feature aggregation processing to obtain aggregated feature data corresponding to the enhanced image samples; The multi-scale feature extraction layer is used to perform convolution feature extraction on the aggregated feature data using convolution kernels of different sizes to obtain extracted feature maps of different scales corresponding to the aggregated feature data, and to fuse the extracted feature maps to obtain convolution feature data corresponding to the enhanced image sample; The grouped convolution layer based on depthwise separable convolution is used to divide the convolution-extracted feature data into feature image blocks of fixed size; flatten each feature image block and map each feature image block into an embedding vector by linear projection; perform depthwise convolution and pointwise convolution on each embedding vector using depthwise separable convolution to obtain an injection feature map of each channel corresponding to the convolution-extracted feature data, and then use a convolution kernel to aggregate the injection feature maps of each channel to obtain a dynamic feature vector corresponding to the enhanced image sample; The Transformer module based on the multi-head attention mechanism is used to perform feature interaction on the dynamic feature vector based on the multi-head attention mechanism to obtain convolution injection feature data corresponding to the enhanced image sample; The output layer is used to fuse the convolution extraction feature data and the convolution injection feature data corresponding to the enhanced image sample to obtain a target feature map corresponding to the enhanced image sample.

5. The thyroid follicular lesion segmentation model training method according to claim 1, characterized in that: The encoder in the target Unet model is provided with a regional attention module; The regional attention module is used to extract features from the input data using a spatial attention mechanism and a channel attention mechanism in sequence to obtain regional attention feature data corresponding to the enhanced image sample; And input the regional attention feature data into the downsampling layer of the encoder itself, so that the downsampling layer outputs the downsampling feature data corresponding to the regional attention feature data; Among them, if the encoder where the regional attention module is located is the first encoder in the target Unet model, the input data is the enhanced image sample; if the encoder where the regional attention module is located is not the first encoder in the target Unet model, the input data is the downsampled feature data output by another encoder connected to the input side of the encoder.

6. The thyroid follicular lesion segmentation model training method according to claim 5, characterized in that: The regional attention module includes: a spatial attention layer and a channel attention layer; The spatial attention layer is used to obtain a spatial attention map corresponding to the input data using sequentially connected convolutional layers and ReLU activation functions, and multiply the spatial attention map by the input data element-by-element to obtain spatial attention feature data corresponding to the enhanced image sample; The channel attention layer is used to use a global pooling layer, a one-dimensional convolution layer and a ReLU activation function connected in sequence to obtain the spatial and channel attention maps corresponding to the spatial attention feature data, and multiply the spatial and channel attention maps with the input data element by element to obtain the regional attention feature data corresponding to the enhanced image sample.

7. A thyroid follicular lesion segmentation method, characterized in that: include: performing contrast enhancement processing based on contrast limitation on each image block corresponding to the target thyroid follicular grayscale ultrasound image to obtain a contrast enhanced block for each image block; Merging the contrast enhancement blocks and performing edge smoothing to obtain an enhanced image sample corresponding to the target thyroid follicular grayscale ultrasound image; The enhanced image sample is input into a thyroid follicular lesion segmentation model so that the thyroid follicular lesion segmentation model outputs a thyroid follicular lesion area segmented from the enhanced image sample and a classification label of the thyroid follicular lesion area; wherein the thyroid follicular lesion segmentation model is pre-trained based on the thyroid follicular lesion segmentation model training method according to any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the thyroid follicular lesion segmentation model training method according to any one of claims 1 to 6, and / or implements the thyroid follicular lesion segmentation method according to claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the thyroid follicular lesion segmentation model training method according to any one of claims 1 to 6, and / or implements the thyroid follicular lesion segmentation method according to claim 7.

Images