Thyroid lesion detection method and system based on ultrasonic image

Abstract

The invention provides a thyroid lesion detection method and system based on an ultrasonic image, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining original thyroid ultrasonic image data of a patient end, and carrying out the preprocessing to generate target thyroid ultrasonic image data; constructing a thyroid region segmentation model to automatically segment the target thyroid ultrasound image data to generate thyroid region data; constructing a thyroid lesion area detection model to perform lesion detection on the thyroid area data to generate thyroid lesion area data; performing feature extraction on the thyroid lesion area data to generate thyroid lesion features; inputting the thyroid lesion features into a thyroid lesion classification model to generate a thyroid lesion diagnosis result; and based on the user interaction interface, the thyroid lesion diagnosis result is sent to the doctor end, so that automatic segmentation and lesion detection of the thyroid region can be realized, the accuracy and generalization ability of lesion detection are improved, and doctors are assisted in thyroid lesion diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for detecting thyroid lesions based on ultrasonic images. Background Art

[0002] Thyroid lesions are common thyroid diseases, especially thyroid nodules, and their incidence rate is increasing year by year. Ultrasound imaging is one of the main means of diagnosing thyroid lesions, but its diagnostic results often rely on the doctor's experience, and there are problems such as strong subjectivity and high misdiagnosis rate. In addition, the existing computer-aided diagnosis system still has problems such as insufficient detection accuracy and poor generalization ability in thyroid lesion detection.

[0003] Therefore, it is necessary to provide a thyroid lesion detection method and system based on ultrasound imaging to solve the above technical problems. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a thyroid lesion detection method and system based on ultrasound imaging to solve the problems that the diagnostic results of ultrasound imaging often rely on the doctor's experience, are highly subjective and have a high misdiagnosis rate, and the existing computer-aided diagnosis system has the problems of insufficient detection accuracy and poor generalization ability in thyroid lesion detection.

[0005] The present invention provides a method for detecting thyroid lesions based on ultrasound imaging, the method comprising:

[0006] Acquiring original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data;

[0007] Constructing a thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data;

[0008] Constructing a thyroid lesion area detection model, performing lesion detection on the thyroid area data, and generating thyroid lesion area data;

[0009] Extracting features from the thyroid lesion area data to generate thyroid lesion features;

[0010] Constructing a thyroid lesion classification model, inputting the thyroid lesion characteristics into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result;

[0011] Based on the user interaction interface, the thyroid lesion diagnosis result is sent to the physician end.

[0012] Preferably, the obtaining of original thyroid ultrasound image data from the patient and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data specifically includes:

[0013] The original thyroid ultrasound image data of the patient is obtained, and noise in the original thyroid ultrasound image data is removed based on a non-local means filtering algorithm to generate corresponding first thyroid ultrasound image data. The calculation formula of the non-local means filtering algorithm is as follows:

[0014]

[0015] In the formula, I 1 (x) represents the gray value of pixel x in the first thyroid ultrasound image data; C(x) represents the normalization coefficient; Ω represents the search window centered on pixel x; ω(x,y) represents the weight between pixel x and y; I(x) represents the gray value of pixel x in the original thyroid ultrasound image data;

[0016] The calculation formula of the weight ω(x,y) between the pixel points x and y is as follows:

[0017]

[0018] Where ω(x,y) represents the weight between pixel points x and y; N x represents the local neighborhood centered on pixel x; N y represents the local neighborhood centered on pixel y; I(N x ) represents the grayscale value of the local neighborhood centered on the pixel point x; I(N y ) represents the grayscale value of the local neighborhood centered on pixel y; Represents the local neighborhood N x and N y The Euclidean distance between them; h represents the weight decay control parameter; exp() represents the exponential function with e as the base;

[0019] The calculation formula of the normalization coefficient C(x) is as follows:

[0020]

[0021] Where C(x) represents the normalization coefficient; Ω represents the search window centered on pixel x; ω(x,y) represents the weight between pixel x and y;

[0022] Based on the histogram equalization technology, performing local contrast enhancement processing on the first thyroid ultrasound image data to generate corresponding second thyroid ultrasound image data;

[0023] The second thyroid ultrasound image data is standardized to generate the corresponding target thyroid ultrasound image data.

[0024] Preferably, the step of constructing a thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data specifically includes:

[0025] The target thyroid ultrasound image features corresponding to the target thyroid ultrasound image data are extracted, and the target thyroid ultrasound image features are input into the thyroid region segmentation model to obtain the output image features of the lth layer. The corresponding calculation formula is as follows:

[0026] A l =σ(ω l *A l-1 +b l )

[0027] In the formula, A l represents the output image feature of the lth layer; σ() represents the activation function; ω l A represents the convolution kernel weight of the lth layer; l-1 represents the output image features of the l-1th layer; b l represents the bias term of the lth layer;

[0028] The output image features of the lth layer are subjected to maximum pooling processing to obtain the pooled image features of the lth layer. The corresponding calculation formula is as follows:

[0029]

[0030] In the formula, A represents the pooled image features of the lth layer; l represents the output image features of the lth layer; p and q represent the pixel indexes within the pooling window Φ; max represents the maximum value operation;

[0031] The pooled image features of the lth layer are deconvolved to obtain the deconvolution image features of the lth layer. The corresponding calculation formula is as follows:

[0032]

[0033] In the formula, Represents the deconvolution image features of the lth layer; Represents the transposed convolution kernel weight of layer l; Represents the pooled image features of the lth layer.

[0034] Preferably, the deconvolution image features and the pooling image features of the lth layer are subjected to skip connection processing to obtain the spliced ​​image features of the lth layer, and the corresponding calculation formula is as follows:

[0035]

[0036] In the formula, Represents the spliced ​​image features of the lth layer; concat() represents the feature map channel splicing function; Represents the deconvolution image features of the lth layer; Represents the pooled image features of the lth layer;

[0037] Determine the thyroid region probability corresponding to each pixel point in the target thyroid ultrasound image data according to the stitched image feature;

[0038] If the thyroid region probability meets the preset segmentation condition, the corresponding pixel point is determined to be a thyroid region pixel point, and the information of all the thyroid region pixels is summarized to generate the thyroid region data.

[0039] Preferably, the step of constructing a thyroid lesion region detection model, performing lesion detection on the thyroid region data, and generating thyroid lesion region data specifically includes:

[0040] Extracting thyroid region features corresponding to the thyroid region data, and determining a preset anchor point at each position of the thyroid region features;

[0041] For each preset anchor point, based on the region candidate network of the thyroid lesion region detection model, the probability that the preset anchor point belongs to the foreground is predicted, and the corresponding calculation formula is as follows:

[0042] P 1 =sigmoid(ω cls *B 1 +b cls )

[0043] Where P 1 Indicates the probability that the preset anchor point belongs to the foreground; sigmoid() indicates the activation function; ω cls represents the weight of the classification layer in the region candidate network; B 1 represents the feature vector at the preset anchor point; b cls Represents the bias term of the classification layer in the region proposal network;

[0044] Predict the bounding box coordinate offset of the preset anchor point, and the corresponding calculation formula is as follows:

[0045] t 1 =ω reg *B 1 +b reg

[0046] Where, t 1Represents the bounding box coordinate offset of the preset anchor point; ω reg represents the weight of the regression layer in the region candidate network; B 1 represents the feature vector at the preset anchor point; b reg Represents the bias term of the regression layer in the region proposal network;

[0047] The candidate thyroid lesion region corresponding to the thyroid region data is determined based on the predicted probability that the preset anchor point belongs to the foreground and the bounding box coordinate offset of the preset anchor point.

[0048] Preferably, based on the region of interest pooling layer of the thyroid lesion region detection model, the thyroid lesion candidate region is pooled to obtain a corresponding pooled candidate region;

[0049] For each pooled candidate region, the prediction probability of thyroid lesions is determined based on the classification regression network of the thyroid lesion region detection model. The corresponding calculation formula is as follows:

[0050] P 2 =softmax(ω fc *B 2 +b fc )

[0051] Where P 2 represents the prediction probability of thyroid lesions; softmax() represents the activation function; ω fc Represents the weight of the fully connected layer in the classification regression network; B 2 represents the feature vector corresponding to the pooled candidate area; b fc Represents the bias term of the fully connected layer in the classification regression network;

[0052] Predict the bounding box coordinate offset of the pooled candidate area, and the corresponding calculation formula is as follows:

[0053] t 2 =ω bbox *B 2 +b bbox

[0054] Where, t 2 Represents the bounding box coordinate offset of the pooled candidate area; ω bbox represents the weight of the bounding box regression layer in the classification regression network; B 2 represents the feature vector corresponding to the pooled candidate area; b bbox Represents the bias term of the bounding box regression layer in the classification regression network;

[0055] If the predicted probability of the thyroid lesion meets the preset lesion detection condition, the pooled candidate region belongs to the thyroid lesion region, and the corresponding thyroid lesion region data is determined.

[0056] Preferably, the feature extraction of the thyroid lesion area data to generate thyroid lesion features specifically includes:

[0057] Extracting the size, shape and boundary clarity data corresponding to the thyroid lesion area data to generate corresponding morphological lesion features;

[0058] Extracting global texture features corresponding to the thyroid lesion area data based on a gray level co-occurrence matrix, extracting local texture features corresponding to the thyroid lesion area data based on a local binary pattern, and summarizing the global texture features and the local texture features to generate corresponding texture lesion features;

[0059] Based on the deep learning model, high-level features corresponding to the thyroid lesion area data are extracted to generate corresponding deep learning lesion features;

[0060] The morphological lesion features, texture lesion features and deep learning lesion features are fused to generate the corresponding thyroid lesion features, and the principal component analysis technology is used to reduce the dimension of the thyroid lesion features.

[0061] Preferably, the step of constructing a thyroid lesion classification model, inputting the thyroid lesion features into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result specifically includes:

[0062] Constructing the thyroid lesion classification model, and inputting the thyroid lesion features into the thyroid lesion classification model to generate preliminary lesion classification results, wherein the preliminary lesion classification results include benign thyroid lesion results and malignant thyroid lesion results;

[0063] The clinical data of the patient is obtained, and the thyroid lesion diagnosis result is generated according to the preliminary lesion classification result and the clinical data.

[0064] A thyroid lesion detection system based on ultrasonic imaging, the detection system comprising:

[0065] An acquisition module, used for acquiring original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data;

[0066] A segmentation module is used to construct a thyroid region segmentation model, automatically segment the target thyroid ultrasound image data, and generate thyroid region data;

[0067] A detection module is used to construct a thyroid lesion area detection model, perform lesion detection on the thyroid area data, and generate thyroid lesion area data;

[0068] An extraction module, used for extracting features from the thyroid lesion area data to generate thyroid lesion features;

[0069] A diagnosis module, used for constructing a thyroid lesion classification model, inputting the thyroid lesion characteristics into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result;

[0070] The sending module is used to send the thyroid disease diagnosis result to the physician end based on the user interaction interface.

[0071] Compared with the related art, the method and system for detecting thyroid lesions based on ultrasound imaging provided by the present invention have the following beneficial effects:

[0072] The present invention obtains original thyroid ultrasound image data from the patient end, and pre-processes the original thyroid ultrasound image data to generate target thyroid ultrasound image data; constructs a thyroid region segmentation model, automatically segments the target thyroid ultrasound image data, and generates thyroid region data; constructs a thyroid lesion region detection model, performs lesion detection on the thyroid region data, and generates thyroid lesion region data; extracts features from the thyroid lesion region data to generate thyroid lesion features; constructs a thyroid lesion classification model, inputs the thyroid lesion features into the thyroid lesion classification model, and generates thyroid lesion diagnosis results; based on a user interaction interface, the thyroid lesion diagnosis results are sent to the physician end, thereby realizing automatic segmentation and lesion detection of the thyroid region, improving the accuracy and generalization ability of lesion detection, and assisting doctors in diagnosing thyroid lesions.

[0073] The present invention can automatically segment the thyroid ultrasound image data through a thyroid region segmentation model to determine the thyroid region, and can perform lesion detection on the thyroid region data through a thyroid lesion region detection model to determine the thyroid lesion region, thereby reducing human errors and ensuring the accuracy of thyroid lesion region identification; the present invention can extract features from thyroid lesion region data by combining multi-scale feature fusion and advanced feature extraction technology to obtain thyroid lesion features, and determine thyroid lesion diagnosis results based on thyroid lesion features through a thyroid lesion classification model, thereby improving the accuracy and generalization ability of lesion detection; and the present invention can provide a visual user interaction interface to send the thyroid lesion diagnosis results to doctors in a timely manner, so that doctors can quickly understand the patient's condition. Therefore, the system of the present invention can be widely used in various scenarios to assist doctors in diagnosing thyroid lesions and subsequent treatments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of a method for detecting thyroid lesions based on ultrasound imaging of the present invention;

[0075] Figure 2 is a flow chart of the preprocessing process of the original thyroid ultrasound image data of the present invention;

[0076] Figure 3 The system block diagram of a thyroid lesion detection system based on ultrasonic imaging of the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0078] Embodiment 1

[0079] like Figure 1 As shown, a method for detecting thyroid lesions based on ultrasound imaging, the detection method comprising:

[0080] S1, obtaining original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data;

[0081] S2, constructing a thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data;

[0082] S3, constructing a thyroid lesion area detection model, performing lesion detection on the thyroid area data, and generating thyroid lesion area data;

[0083] S4, extracting features from the thyroid lesion area data to generate thyroid lesion features;

[0084] S5, constructing a thyroid lesion classification model, inputting the thyroid lesion features into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result;

[0085] S6, based on the user interaction interface, sending the thyroid disease diagnosis result to the physician end.

[0086] It should be noted that the original thyroid ultrasound image data is usually stored in the DICOM (Digital Imaging and Communications in Medicine) format, which contains the grayscale information of the patient's thyroid region and its surrounding tissues. Since the original thyroid ultrasound images often have problems such as speckle noise, artifacts, and low contrast, the original thyroid ultrasound image data needs to be preprocessed.

[0087] Specifically, the preprocessing process includes image denoising, contrast enhancement and standardization. The denoising operation can use the non-local mean filtering algorithm to eliminate speckle noise in the image data while retaining the detail information in the image. The contrast enhancement process uses the histogram equalization technology to improve the distinction between the thyroid region and the surrounding tissue. The standardization process includes image size normalization and gray value normalization, which can ensure the input consistency of the subsequent model.

[0088] It is understandable that based on the thyroid region segmentation model, the boundaries of the thyroid tissue can be accurately segmented and extracted from the ultrasound image through the encoder and decoder structure, effectively capturing the morphological characteristics of the thyroid region. And after the segmentation is completed, the segmentation results can be post-processed through morphological operations such as opening and closing operations to remove noise and irrelevant areas to ensure segmentation accuracy.

[0089] Next, a thyroid lesion area detection model can be constructed to perform lesion detection on the thyroid region data. Candidate regions can be generated through the region candidate network, and the candidate regions can be accurately located and classified in combination with the classification regression network to identify suspicious lesions in the thyroid region, such as nodules, cysts, etc.

[0090] In addition, in order to improve the detection accuracy of small lesion areas, a multi-scale feature fusion mechanism, such as feature pyramid network (FPN), can be introduced into the thyroid lesion area detection model to capture lesion features of different scales. After the detection is completed, the overlapping candidate areas can be removed by non-maximum suppression (NMS) technology to generate the final lesion area data.

[0091] Furthermore, discriminative features can be extracted from the lesion area. These features include morphological lesion features, texture lesion features, and deep learning lesion features. Specifically, morphological lesion features include geometric parameters such as size, shape, boundary clarity, such as area, perimeter, aspect ratio, etc. of the lesion area; texture lesion features are extracted through gray-level co-occurrence matrix (GLCM) and local binary pattern (LBP), which can reflect the grayscale distribution and local texture information of the lesion area; deep learning lesion features are extracted through pre-trained deep learning models, such as ResNet, DenseNet, etc., which can capture high-level semantic information of the lesion area. Finally, morphological lesion features, texture lesion features, and deep learning lesion features can be fused to generate comprehensive thyroid lesion features.

[0092] In practical applications, based on the thyroid lesion classification model, the lesion area can be classified according to the extracted thyroid lesion characteristics to generate preliminary classification results of the lesion area, including benign and malignant lesion results, and combined with clinical data for comprehensive diagnosis. These clinical data include patient age, gender, medical history, etc., to generate thyroid lesion diagnosis results, thereby improving the accuracy of the diagnosis results.

[0093] Finally, the diagnosis results of thyroid lesions can be sent to the physician based on the user interface. The user interface can support the display of thyroid ultrasound images, the annotation of lesion areas, the viewing and modification of diagnosis results, etc. The diagnosis results are presented in a structured form, including the location, size, classification results and diagnosis suggestions of the lesion area. The physician can further analyze the diagnosis results through the user interface and make corresponding adjustments based on clinical experience.

[0094] Through the above method, the automated detection and diagnosis of thyroid lesions can be realized, and the accuracy and generalization ability of lesion detection can be improved, effectively assisting physicians in the diagnosis of thyroid lesions and improving diagnostic efficiency and accuracy.

[0095] In the specific implementation process, Figure 2 As shown, the method of obtaining the original thyroid ultrasound image data of the patient and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data specifically includes:

[0096] The original thyroid ultrasound image data of the patient is obtained, and noise in the original thyroid ultrasound image data is removed based on a non-local means filtering algorithm to generate corresponding first thyroid ultrasound image data. The calculation formula of the non-local means filtering algorithm is as follows:

[0097]

[0098] In the formula, I 1 (x) represents the gray value of pixel x in the first thyroid ultrasound image data; C(x) represents the normalization coefficient; Ω represents the search window centered on pixel x; ω(x,y) represents the weight between pixel x and y; I(x) represents the gray value of pixel x in the original thyroid ultrasound image data;

[0099] The calculation formula of the weight ω(x,y) between the pixel points x and y is as follows:

[0100]

[0101] Where ω(x,y) represents the weight between pixel points x and y; N x represents the local neighborhood centered on pixel x; N yrepresents the local neighborhood centered on pixel y; I(N x ) represents the grayscale value of the local neighborhood centered on the pixel point x; I(N y ) represents the grayscale value of the local neighborhood centered on pixel y; Represents the local neighborhood N x and N y The Euclidean distance between them; h represents the weight decay control parameter; exp() represents the exponential function with e as the base;

[0102] The calculation formula of the normalization coefficient C(x) is as follows:

[0103]

[0104] Where C(x) represents the normalization coefficient; Ω represents the search window centered on pixel x; ω(x,y) represents the weight between pixel x and y;

[0105] Based on the histogram equalization technology, performing local contrast enhancement processing on the first thyroid ultrasound image data to generate corresponding second thyroid ultrasound image data;

[0106] The second thyroid ultrasound image data is standardized to generate the corresponding target thyroid ultrasound image data.

[0107] Among them, first, the original thyroid ultrasound image data can be obtained from the patient side, which is usually derived from the DICOM format medical image output by the digital ultrasound imaging device. In view of the inherent speckle noise characteristics of ultrasound images, the non-local mean filtering algorithm can be used to reduce the noise and generate the first thyroid ultrasound image data. And the global self-similarity characteristics of the image can be used to dynamically adjust the weight coefficient by calculating the structural similarity between pixel neighborhoods. Specifically, for the target pixel point, all candidate pixel points in its search window can be traversed, and the weight matrix can be constructed by comparing the difference in the local neighborhood grayscale distribution of the two, in which the weight value is negatively exponentially related to the Euclidean distance of the neighborhood grayscale distribution, and the physical validity of the pixel grayscale value is ensured by the normalization coefficient. This process effectively suppresses high-frequency speckle noise while fully retaining the edge characteristics of the thyroid tissue.

[0108] Furthermore, the first thyroid ultrasound image after noise reduction can be subjected to local contrast enhancement processing, and the adaptive histogram equalization technology can be used to nonlinearly expand the grayscale dynamic range of the region of interest while maintaining the overall grayscale distribution. This method divides the image into sub-regions and calculates the local grayscale cumulative distribution function. For different tissue density regions, a differentiated contrast enhancement strategy is used to significantly improve the grayscale boundary clarity between the thyroid parenchyma and the lesion area, and finally generate the second thyroid ultrasound image data, thereby optimizing the display effect of small nodules under low contrast conditions.

[0109] In the image standardization processing stage, a bimodal normalization operation can be performed on the second thyroid ultrasound image data. Specifically, first, the second thyroid ultrasound image data can be normalized in spatial dimension, and the image resolution can be unified to a preset standard size using a bicubic interpolation algorithm to eliminate the scale deviation caused by differences in imaging parameters between devices. Secondly, the second thyroid ultrasound image data can be normalized in grayscale. Based on the window width and window position adjustment technology, the original grayscale value can be linearly mapped to the standard grayscale space, and the light intensity difference can be eliminated through a standardized method, thereby ensuring that the generated target thyroid ultrasound image data has uniform geometric features and grayscale distribution characteristics.

[0110] The constructing of the thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data specifically includes:

[0111] The target thyroid ultrasound image features corresponding to the target thyroid ultrasound image data are extracted, and the target thyroid ultrasound image features are input into the thyroid region segmentation model to obtain the output image features of the lth layer. The corresponding calculation formula is as follows:

[0112] A l =σ(ω l *A l-1 +b l )

[0113] In the formula, A l represents the output image feature of the lth layer; σ() represents the activation function; ω l A represents the convolution kernel weight of the lth layer; l-1 represents the output image features of the l-1th layer; b l represents the bias term of the lth layer;

[0114] The output image features of the lth layer are subjected to maximum pooling processing to obtain the pooled image features of the lth layer. The corresponding calculation formula is as follows:

[0115]

[0116] In the formula, A represents the pooled image features of the lth layer; l represents the output image features of the lth layer; p and q represent the pixel indexes within the pooling window Φ; max represents the maximum value operation;

[0117] The pooled image features of the lth layer are deconvolved to obtain the deconvolution image features of the lth layer. The corresponding calculation formula is as follows:

[0118]

[0119] In the formula, Represents the deconvolution image features of the lth layer; Represents the transposed convolution kernel weight of layer l; Represents the pooled image features of the lth layer.

[0120] It can be understood that the key features in the target thyroid ultrasound image data, namely the target thyroid ultrasound image features, can be first extracted and input into the constructed thyroid region segmentation model. Each layer in the model will output the corresponding image features, and the obtained output image features can be subjected to maximum pooling processing to extract the main features and reduce the amount of data to generate pooled image features. Finally, the pooled image features can be subjected to deconvolution processing to restore the resolution of the image features and generate deconvolution image features.

[0121] The deconvolution image features and pooling image features of the lth layer are skipped and connected to obtain the spliced ​​image features of the lth layer. The corresponding calculation formula is as follows:

[0122]

[0123] In the formula, Represents the spliced ​​image features of the lth layer; concat() represents the feature map channel splicing function; Represents the deconvolution image features of the lth layer; Represents the pooled image features of the lth layer;

[0124] Determine the thyroid region probability corresponding to each pixel point in the target thyroid ultrasound image data according to the stitched image feature;

[0125] If the thyroid region probability meets the preset segmentation condition, the corresponding pixel point is determined to be a thyroid region pixel point, and the information of all the thyroid region pixels is summarized to generate the thyroid region data.

[0126] It is understandable that the deconvolution image features and the pooling image features can be skip-connected to generate a spliced ​​image feature. Then, the probability of the thyroid region corresponding to each pixel in the target thyroid ultrasound image data can be determined based on the obtained spliced ​​image feature.

[0127] Furthermore, if the thyroid region probability of a certain pixel satisfies the preset segmentation condition, that is, the thyroid region probability is greater than or equal to the critical segmentation probability, the pixel is determined to be a thyroid region pixel, and the critical segmentation probability refers to the critical probability used to determine whether the pixel is a thyroid region pixel. Finally, the information of all thyroid region pixels can be summarized to generate the final thyroid region data.

[0128] The step of constructing a thyroid lesion region detection model, performing lesion detection on the thyroid region data, and generating thyroid lesion region data specifically includes:

[0129] Extracting thyroid region features corresponding to the thyroid region data, and determining a preset anchor point at each position of the thyroid region features;

[0130] For each preset anchor point, based on the region candidate network of the thyroid lesion region detection model, the probability that the preset anchor point belongs to the foreground is predicted, and the corresponding calculation formula is as follows:

[0131] P 1 =sigmoid(ω cls *B 1 +b cls )

[0132] Where P 1 Indicates the probability that the preset anchor point belongs to the foreground; sigmoid() indicates the activation function; ω cls represents the weight of the classification layer in the region candidate network; B 1 represents the feature vector at the preset anchor point; b cls Represents the bias term of the classification layer in the region proposal network;

[0133] Predict the bounding box coordinate offset of the preset anchor point, and the corresponding calculation formula is as follows:

[0134] t 1 =ω reg *B 1 +b reg

[0135] Where, t 1 Indicates the bounding box coordinate offset of the preset anchor point; ω reg represents the weight of the regression layer in the region candidate network; B 1 represents the feature vector at the preset anchor point; breg Represents the bias term of the regression layer in the region proposal network;

[0136] The candidate thyroid lesion region corresponding to the thyroid region data is determined based on the predicted probability that the preset anchor point belongs to the foreground and the bounding box coordinate offset of the preset anchor point.

[0137] Based on the region of interest pooling layer of the thyroid lesion region detection model, the thyroid lesion candidate region is pooled to obtain a corresponding pooled candidate region;

[0138] For each pooled candidate region, the prediction probability of thyroid lesions is determined based on the classification regression network of the thyroid lesion region detection model. The corresponding calculation formula is as follows:

[0139] P 2 =softmax(ω fc *B 2 +b fc )

[0140] Where P 2 represents the prediction probability of thyroid lesions; softmax() represents the activation function; ω fc Represents the weight of the fully connected layer in the classification regression network; B 2 represents the feature vector corresponding to the pooled candidate area; b fc Represents the bias term of the fully connected layer in the classification regression network;

[0141] Predict the bounding box coordinate offset of the pooled candidate area, and the corresponding calculation formula is as follows:

[0142] t 2 =ω bbox *B 2 +b bbox

[0143] Where, t 2 Represents the bounding box coordinate offset of the pooled candidate area; ω bbox represents the weight of the bounding box regression layer in the classification regression network; B 2 represents the feature vector corresponding to the pooled candidate area; b bbox Represents the bias term of the bounding box regression layer in the classification regression network;

[0144] If the predicted probability of the thyroid lesion meets the preset lesion detection condition, the pooled candidate region belongs to the thyroid lesion region, and the corresponding thyroid lesion region data is determined.

[0145] Among them, firstly, feature information in the thyroid region data can be extracted, and a preset anchor point can be set at each key position of the thyroid region feature.

[0146] For each preset anchor point, the region candidate network in the thyroid lesion region detection model can be used to predict the probability that the anchor point belongs to the foreground, where the foreground is the lesion region. At the same time, the bounding box coordinate offset of the preset anchor point can be predicted to determine the specific location of the lesion region. Based on the predicted foreground probability and bounding box coordinate offset, the thyroid lesion candidate region corresponding to the thyroid region data can be determined.

[0147] Then, the region of interest pooling layer in the thyroid lesion region detection model can be used to pool these candidate regions to obtain pooled candidate regions. For each pooled candidate region, the classification regression network can be further used to determine the corresponding thyroid lesion prediction probability and predict the offset of the bounding box coordinates.

[0148] If the thyroid lesion prediction probability meets the preset lesion detection condition, that is, the thyroid lesion prediction probability is greater than or equal to the critical lesion detection probability, then the pooled candidate region is determined to belong to the thyroid lesion region, and the corresponding thyroid lesion region data is determined. And the critical lesion detection probability refers to the critical probability used to determine whether the pooled candidate region belongs to the thyroid lesion region.

[0149] The feature extraction of the thyroid lesion area data to generate thyroid lesion features specifically includes:

[0150] Extracting the size, shape and boundary clarity data corresponding to the thyroid lesion area data to generate corresponding morphological lesion features;

[0151] Extracting global texture features corresponding to the thyroid lesion area data based on a gray level co-occurrence matrix, extracting local texture features corresponding to the thyroid lesion area data based on a local binary pattern, and summarizing the global texture features and the local texture features to generate corresponding texture lesion features;

[0152] Based on the deep learning model, high-level features corresponding to the thyroid lesion area data are extracted to generate corresponding deep learning lesion features;

[0153] The morphological lesion features, texture lesion features and deep learning lesion features are fused to generate the corresponding thyroid lesion features, and the principal component analysis technology is used to reduce the dimension of the thyroid lesion features.

[0154] It should be noted that, first, the size, shape and boundary clarity information of the thyroid lesion area can be extracted to construct the morphological lesion features. Secondly, the gray level co-occurrence matrix technology can be used to analyze the global texture features of the lesion area, and the local binary pattern method can be combined to extract the local texture features, and the two can be summarized to form a comprehensive texture lesion feature. Then, based on the deep learning model, the high-level features of the lesion area can be further analyzed to form a deep learning lesion feature. Finally, the morphological lesion features, texture lesion features and deep learning lesion features can be deeply fused to construct a comprehensive thyroid lesion feature, and the principal component analysis technology can be used to reduce the dimensionality of the thyroid lesion features.

[0155] The step of constructing a thyroid lesion classification model, inputting the thyroid lesion features into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result specifically includes:

[0156] Constructing the thyroid lesion classification model, and inputting the thyroid lesion features into the thyroid lesion classification model to generate preliminary lesion classification results, wherein the preliminary lesion classification results include benign thyroid lesion results and malignant thyroid lesion results;

[0157] The clinical data of the patient is obtained, and the thyroid lesion diagnosis result is generated according to the preliminary lesion classification result and the clinical data.

[0158] In practical applications, in order to accurately classify and diagnose thyroid lesions, a thyroid lesion classification model can be constructed. After receiving the thyroid lesion features, the model can perform preliminary lesion classification processing, and the classification results include benign and malignant thyroid lesions.

[0159] In order to ensure the accuracy and comprehensiveness of the diagnosis results, clinical data from the patient can be further obtained, including medical history, physical signs, laboratory tests, etc. Then, the preliminary lesion classification results can be comprehensively analyzed with the clinical data to generate the final thyroid lesion diagnosis results, thereby improving the accuracy of lesion diagnosis and ensuring the reliability and practicality of the diagnosis results.

[0160] Embodiment 2

[0161] like Figure 3 As shown, a thyroid lesion detection system based on ultrasound imaging, the detection system comprises:

[0162] An acquisition module, used for acquiring original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data;

[0163] A segmentation module is used to construct a thyroid region segmentation model, automatically segment the target thyroid ultrasound image data, and generate thyroid region data;

[0164] A detection module is used to construct a thyroid lesion area detection model, perform lesion detection on the thyroid area data, and generate thyroid lesion area data;

[0165] An extraction module, used for extracting features from the thyroid lesion area data to generate thyroid lesion features;

[0166] A diagnosis module, used for constructing a thyroid lesion classification model, inputting the thyroid lesion characteristics into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result;

[0167] The sending module is used to send the thyroid disease diagnosis result to the physician end based on the user interaction interface.

[0168] Through the introduction of the above embodiments, the present invention uses a thyroid lesion detection method and system based on ultrasound imaging, which obtains the original thyroid ultrasound image data from the patient end, and pre-processes the original thyroid ultrasound image data to generate target thyroid ultrasound image data; constructs a thyroid region segmentation model, automatically segments the target thyroid ultrasound image data, and generates thyroid region data; constructs a thyroid lesion region detection model, performs lesion detection on the thyroid region data, and generates thyroid lesion region data; extracts features from the thyroid lesion region data to generate thyroid lesion features; constructs a thyroid lesion classification model, inputs the thyroid lesion features into the thyroid lesion classification model, and generates thyroid lesion diagnosis results; based on the user interaction interface, the thyroid lesion diagnosis results are sent to the physician end, thereby realizing automatic segmentation and lesion detection of the thyroid region, improving the accuracy and generalization ability of lesion detection, and assisting doctors in diagnosing thyroid lesions.

[0169] The present invention can automatically segment the thyroid ultrasound image data through a thyroid region segmentation model to determine the thyroid region, and perform lesion detection on the thyroid region data through a thyroid lesion region detection model to determine the thyroid lesion region, thereby reducing human errors and ensuring the accuracy of thyroid lesion region identification; the present invention combines multi-scale feature fusion and advanced feature extraction technology to extract features from thyroid lesion region data to obtain thyroid lesion features, and determines thyroid lesion diagnosis results based on thyroid lesion features through a thyroid lesion classification model, thereby improving the accuracy and generalization ability of lesion detection; and the present invention provides a visual user interaction interface, which can send the thyroid lesion diagnosis results to doctors in a timely manner, so that doctors can quickly understand the patient's condition. Therefore, the system of the present invention can be widely used in various scenarios to assist doctors in diagnosing thyroid lesions and subsequent treatment.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0172] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A method for detecting thyroid lesions based on ultrasound imaging, characterized in that: The detection method comprises: Acquiring original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data; Constructing a thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data; Constructing a thyroid lesion area detection model, performing lesion detection on the thyroid area data, and generating thyroid lesion area data; Extracting features from the thyroid lesion area data to generate thyroid lesion features; Constructing a thyroid lesion classification model, inputting the thyroid lesion characteristics into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result; Based on the user interaction interface, the thyroid lesion diagnosis result is sent to the physician end.

2. The method for detecting thyroid lesions based on ultrasound imaging according to claim 1, characterized in that: The step of obtaining original thyroid ultrasound image data from a patient and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data specifically includes: The original thyroid ultrasound image data of the patient is obtained, and noise in the original thyroid ultrasound image data is removed based on a non-local means filtering algorithm to generate corresponding first thyroid ultrasound image data. The calculation formula of the non-local means filtering algorithm is as follows: Wherein, I1(x) represents the gray value of pixel x in the first thyroid ultrasound image data; C(x) represents the normalization coefficient; Ω represents the search window centered on pixel x; ω(x,y) represents the weight between pixel x and y; I(x) represents the gray value of pixel x in the original thyroid ultrasound image data; The calculation formula of the weight ω(x,y) between the pixel points x and y is as follows: Where ω(x,y) represents the weight between pixel points x and y; N x represents the local neighborhood centered on pixel x; N y represents the local neighborhood centered on pixel y; I(N x ) represents the grayscale value of the local neighborhood centered on the pixel point x; I(N y ) represents the grayscale value of the local neighborhood centered on pixel y; Represents the local neighborhood N x and N y The Euclidean distance between them; h represents the weight decay control parameter; exp() represents the exponential function with e as the base; The calculation formula of the normalization coefficient C(x) is as follows: Where C(x) represents the normalization coefficient; Ω represents the search window centered at pixel x; ω(x,y) represents the weight between pixel x and y; Based on the histogram equalization technology, performing local contrast enhancement processing on the first thyroid ultrasound image data to generate corresponding second thyroid ultrasound image data; The second thyroid ultrasound image data is standardized to generate the corresponding target thyroid ultrasound image data.

3. The method for detecting thyroid lesions based on ultrasound imaging according to claim 1, characterized in that: The constructing of the thyroid region segmentation model, automatically segmenting the target thyroid ultrasound image data, and generating thyroid region data specifically includes: The target thyroid ultrasound image features corresponding to the target thyroid ultrasound image data are extracted, and the target thyroid ultrasound image features are input into the thyroid region segmentation model to obtain the output image features of the lth layer. The corresponding calculation formula is as follows: A l =σ(ω l *A l-1 +b l ) In the formula, A l represents the output image feature of the lth layer; σ() represents the activation function; ω l A represents the convolution kernel weight of the lth layer; l-1 represents the output image features of the l-1th layer; b l represents the bias term of the lth layer; The output image features of the lth layer are subjected to maximum pooling processing to obtain the pooled image features of the lth layer. The corresponding calculation formula is as follows: In the formula, A represents the pooled image features of the lth layer; l represents the output image features of the lth layer; p and q represent the pixel indexes within the pooling window Φ; max represents the maximum value operation; The pooled image features of the lth layer are deconvolved to obtain the deconvolution image features of the lth layer. The corresponding calculation formula is as follows: In the formula, Represents the deconvolution image features of the lth layer; Represents the transposed convolution kernel weight of the lth layer; Represents the pooled image features of the lth layer.

4. The method for detecting thyroid lesions based on ultrasound imaging according to claim 3, characterized in that: The deconvolution image features and pooling image features of the lth layer are skipped and connected to obtain the spliced ​​image features of the lth layer. The corresponding calculation formula is as follows: In the formula, Represents the spliced ​​image features of the lth layer; concat() represents the feature map channel concatenation function; Represents the deconvolution image features of the lth layer; Represents the pooled image features of the lth layer; Determine the thyroid region probability corresponding to each pixel point in the target thyroid ultrasound image data according to the stitched image feature; If the thyroid region probability meets the preset segmentation condition, the corresponding pixel point is determined to be a thyroid region pixel point, and the information of all the thyroid region pixels is summarized to generate the thyroid region data.

5. The method for detecting thyroid lesions based on ultrasound imaging according to claim 1, characterized in that: The step of constructing a thyroid lesion region detection model, performing lesion detection on the thyroid region data, and generating thyroid lesion region data specifically includes: Extracting thyroid region features corresponding to the thyroid region data, and determining a preset anchor point at each position of the thyroid region features; For each preset anchor point, based on the region candidate network of the thyroid lesion region detection model, the probability that the preset anchor point belongs to the foreground is predicted, and the corresponding calculation formula is as follows: P1=sigmoid(ω cls *B1+b cls ) Where P1 represents the probability that the preset anchor point belongs to the foreground; sigmoid() represents the activation function; ω cls represents the weight of the classification layer in the region candidate network; B1 represents the feature vector at the preset anchor point; b cls Represents the bias term of the classification layer in the region proposal network; Predict the bounding box coordinate offset of the preset anchor point, and the corresponding calculation formula is as follows: t1=ω reg *B1+b reg Where t1 represents the bounding box coordinate offset of the preset anchor point; ω reg represents the weight of the regression layer in the region candidate network; B1 represents the feature vector at the preset anchor point; b reg Represents the bias term of the regression layer in the region proposal network; The candidate thyroid lesion region corresponding to the thyroid region data is determined based on the predicted probability that the preset anchor point belongs to the foreground and the bounding box coordinate offset of the preset anchor point.

6. The method for detecting thyroid lesions based on ultrasound imaging according to claim 5, characterized in that: Based on the region of interest pooling layer of the thyroid lesion region detection model, the thyroid lesion candidate region is pooled to obtain a corresponding pooled candidate region; For each pooled candidate region, the prediction probability of thyroid lesions is determined based on the classification regression network of the thyroid lesion region detection model. The corresponding calculation formula is as follows: P2=softmax(ω fc *B2+b fc ) Where P2 represents the predicted probability of thyroid lesions; softmax() represents the activation function; ω fc represents the weight of the fully connected layer in the classification regression network; B2 represents the feature vector corresponding to the pooled candidate region; b fc Represents the bias term of the fully connected layer in the classification regression network; Predict the bounding box coordinate offset of the pooled candidate area, and the corresponding calculation formula is as follows: t2=ω bbox *B2+b bbox Where t2 represents the bounding box coordinate offset of the pooled candidate area; ω bbox represents the weight of the bounding box regression layer in the classification regression network; B2 represents the feature vector corresponding to the pooled candidate region; b bbox Represents the bias term of the bounding box regression layer in the classification regression network; If the predicted probability of the thyroid lesion meets the preset lesion detection condition, the pooled candidate region belongs to the thyroid lesion region, and the corresponding thyroid lesion region data is determined.

7. The method for detecting thyroid lesions based on ultrasound imaging according to claim 1, characterized in that: The feature extraction of the thyroid lesion area data to generate thyroid lesion features specifically includes: Extracting the size, shape and boundary clarity data corresponding to the thyroid lesion area data to generate corresponding morphological lesion features; Extracting global texture features corresponding to the thyroid lesion area data based on a gray level co-occurrence matrix, extracting local texture features corresponding to the thyroid lesion area data based on a local binary pattern, and summarizing the global texture features and the local texture features to generate corresponding texture lesion features; Based on the deep learning model, high-level features corresponding to the thyroid lesion area data are extracted to generate corresponding deep learning lesion features; The morphological lesion features, texture lesion features and deep learning lesion features are fused to generate the corresponding thyroid lesion features, and the principal component analysis technology is used to reduce the dimension of the thyroid lesion features.

8. The method for detecting thyroid lesions based on ultrasound imaging according to claim 1, characterized in that: The step of constructing a thyroid lesion classification model, inputting the thyroid lesion features into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result specifically includes: Constructing the thyroid lesion classification model, and inputting the thyroid lesion features into the thyroid lesion classification model to generate preliminary lesion classification results, wherein the preliminary lesion classification results include benign thyroid lesion results and malignant thyroid lesion results; The clinical data of the patient is obtained, and the thyroid lesion diagnosis result is generated according to the preliminary lesion classification result and the clinical data.

9. A thyroid lesion detection system based on ultrasound imaging, applied to a thyroid lesion detection method based on ultrasound imaging as claimed in any one of claims 1 to 8, the detection system comprising: An acquisition module, used for acquiring original thyroid ultrasound image data from a patient, and preprocessing the original thyroid ultrasound image data to generate target thyroid ultrasound image data; A segmentation module is used to construct a thyroid region segmentation model, automatically segment the target thyroid ultrasound image data, and generate thyroid region data; A detection module is used to construct a thyroid lesion area detection model, perform lesion detection on the thyroid area data, and generate thyroid lesion area data; An extraction module, used for extracting features from the thyroid lesion area data to generate thyroid lesion features; A diagnosis module, used for constructing a thyroid lesion classification model, inputting the thyroid lesion characteristics into the thyroid lesion classification model, and generating a thyroid lesion diagnosis result; The sending module is used to send the thyroid disease diagnosis result to the physician end based on the user interaction interface.

Images