A real-time segmentation system for thyroid gland based on ultrasound video data

Abstract

The present invention discloses a real-time thyroid gland segmentation system based on ultrasonic video data, which relates to the technical field of artificial intelligence and ultrasonic image analysis. A thyroid gland ultrasonic video data set that meets image quality requirements is collected, video frames are annotated, and the data set is divided into a training set and a test set; a thyroid gland segmentation neural network model is constructed using a deep learning framework; hyperparameters of the neural network model are set and the constructed neural network model is trained using the training set video data, and the model is saved after the training is completed; the trained model is deployed on a segmentation device, and the thyroid ultrasound video data collected by an acquisition card can be segmented in real time using the device. The model obtained by the method has the advantages of high precision and good generalization performance, and can achieve fast and accurate segmentation of thyroid ultrasound video data.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and ultrasonic image analysis, and in particular to a real-time thyroid gland segmentation system based on ultrasonic video data. Background Art

[0002] The thyroid gland is an important endocrine organ in the human body. Its main function is to secrete thyroid hormones that regulate human metabolism. The amount of thyroid hormone secretion affects the growth and development of children and the metabolism of adults. Thyroid-related diseases (such as hyperthyroidism, Hashimoto's thyroiditis, etc.) mostly involve changes in the shape and volume of the thyroid gland. Therefore, changes in the thyroid gland are an important pathological sign and are of great significance for the diagnosis of thyroid-related diseases.

[0003] In clinical imaging diagnosis related to the thyroid gland, ultrasound examination is the main detection method. It has the advantages of being non-invasive, radiation-free, economical and practical, and real-time imaging, and is widely used in clinical diagnosis. In addition, in clinical diagnosis, ultrasound doctors operate ultrasound probes to image the patient's thyroid region in real time. Since the thyroid gland is smaller than other soft tissue organs, its primary task is to find a suitable thyroid imaging position. Therefore, it is of great significance to accurately segment the thyroid gland region from the thyroid ultrasound video in real time, whether for related tasks (ultrasound image Hashimoto's thyroiditis classification, ultrasound image thyroid nodule segmentation, ultrasound image thyroid nodule benign and malignant classification, etc.), or to guide ultrasound doctors to find a suitable thyroid imaging position.

[0004] At present, the image quality of ultrasound imaging is relatively low and there are problems such as strong noise and multiple spots, which makes the thyroid gland area blurred and the edge is not clearly contrasted with other non-glandular areas. This makes it impossible for many traditional image segmentation algorithms to accurately segment the thyroid gland area. In recent years, deep learning, especially convolutional neural networks (CNN), has been widely used in the field of image segmentation and has achieved great success. However, the segmentation network models based on CNN are generally more complex and computationally intensive, and cannot be directly used in real-time ultrasound imaging. For the case of real-time video segmentation, there are also some CNN segmentation networks specifically used in this case, but basically they reduce the complexity of the model at the expense of segmentation accuracy, thereby improving the computing speed and meeting the requirements of real-time computing. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a real-time segmentation system of thyroid gland based on ultrasound video data, comprising:

[0006] The video data collection module is used to collect thyroid ultrasound video data, screen all video data using two image quality evaluation methods: image blur and image noise, and annotate the screened data;

[0007] A neural network model building module is used to build a thyroid gland segmentation neural network model using a deep learning framework. The thyroid gland segmentation neural network model includes a previous video frame feature extraction network model and a current frame segmentation network model. The previous video frame feature extraction network model is used to fuse features of three consecutive ultrasound image frames before the current frame, and output a predicted current frame ultrasound image and an intermediate feature map; the current frame segmentation network model is used to output a segmentation map;

[0008] A neural network model training module is used to set the hyperparameters of the thyroid gland segmentation neural network model and train the neural network model to determine the parameters of the optimal model and save the optimal model after the training is completed;

[0009] The segmentation result output module is used to input the thyroid ultrasound video data into the optimal model, output the thyroid gland segmentation result, and display the position outline of the thyroid gland in real time on the ultrasound video according to the segmentation result.

[0010] The technical solution further defined in the present invention is:

[0011] Furthermore, the method for collecting and marking the video data collection module includes the following steps:

[0012] A1. Collect thyroid ultrasound video data from the ultrasound machine through a video acquisition card;

[0013] A2. All video data are screened using two image quality evaluation methods: image blur and image noise, to select video frame data that meets the image quality requirements. Only when the imaging quality of three consecutive image frames meets the evaluation requirements is it judged to meet the image quality requirements;

[0014] A3. The ultrasound doctor shall annotate the ultrasound image that meets the image quality requirements, that is, draw the outline of the thyroid gland on the ultrasound image;

[0015] A4. Convert the labeled data into a binary grayscale image, where the grayscale value of the thyroid gland area is set to 255, and the grayscale values ​​of other areas are set to 0;

[0016] A5. Divide the video dataset into a training set and a test set in a ratio of 8:2.

[0017] As described above, in the real-time segmentation system for thyroid gland based on ultrasound video data, in step A2, the grayscale variance algorithm is used to quantify the blur degree of the ultrasound image, as shown in the following formula:

[0018]

[0019] Where f(x, y) is the grayscale value at each pixel; N x and N y are the pixel sizes of the image in width and height respectively; s is the quantified value of the blur degree of the image, which is used to indicate the average degree of grayscale change of the image;

[0020] The noise of ultrasound images is quantified using the signal-to-noise ratio noise evaluation method based on filtering, as shown in the following formula:

[0021]

[0022] Among them, US represents the original ultrasound image, and Filter(US) represents the image after the median filtering operation is performed on the original ultrasound image.

[0023] As described above, in the real-time segmentation system of thyroid gland based on ultrasound video data, in the neural network model construction module, the method of fusing the features of the three consecutive ultrasound image frames before the current frame with the feature extraction network model of the previous video frame includes the following steps

[0024] B1. Through three convolutional neural networks CNN 1 ,CNN 2 and CNN 3 Feature extraction is performed on the three consecutive ultrasound image frames before the current frame, and the output features of the three convolutional neural networks are spliced ​​by channel to obtain the feature vector feature;

[0025] B2, input the feature vector feature into the generative network model Generator, and output the predicted ultrasound image image′ of the current frame;

[0026] B3, input the predicted image image′ of the current frame and the real image image of the current frame into the discriminator network model respectively, and the output of the discriminator network model is the probability value of judging whether the input image is true;

[0027] B4. Train the generator network model and the discriminator network model respectively;

[0028] B5. Input the feature vector feature into the convolutional neural network CNN 5 , outputs a feature map that integrates the features of the three previous consecutive ultrasound frames of the current frame, convolutional neural network CNN 5 It consists of two convolutions connected in series, and the stride of both convolutions is set to 1.

[0029] As described above, in the real-time segmentation system of thyroid gland based on ultrasound video data, in step B1, three convolutional neural networks (CNNs) 1 ,CNN 2 and CNN 3 They are composed of three operations: convolution, batch normalization, and activation function, and downsampling is achieved through convolution with a step size of 2.

[0030] As described above, in a real-time thyroid gland segmentation system based on ultrasound video data, in step B2, a network model Generator is generated, which is composed of a convolution layer, an upsampling layer, a batch normalization layer, and an activation function, wherein the resolution of the feature map is enlarged one by one through the upsampling layer, and finally reaches the resolution of the input ultrasound image, and linear interpolation with size value clipping is used in the upsampling process to achieve a smooth transition of the feature map; except for the Tanh activation function after the last convolution layer, the activation function uses the LeakyRelu activation function after all other convolutions;

[0031] In step B3, the input of the discriminator network model Discriminator is the real image image of the current frame or the predicted image image′ of the current frame generated by the generator network model Generator, and the output is the probability value of judging whether the input image is true or not; the discriminator network model Discriminator is composed of a convolution layer, a normalization layer, a fully connected layer and an activation function, wherein the resolution of the feature map is gradually reduced by a convolution with a step size of 2; the activation function uses the LeakyRelu activation function after all the convolutions except the last convolution layer which uses the Sigmoid function.

[0032] In the above-mentioned real-time thyroid gland segmentation system based on ultrasound video data, in step B4, when training the discriminator network model Discriminator, the real image label of the current frame is set to 1, and the generated predicted image label of the current frame is set to 0; when training the generator network model Generator, the predicted image label of the current frame generated by the generator network model Generator is set to 1;

[0033] In the discriminator network model, the loss function uses cross entropy loss. The specific formula is as follows:

[0034] S=-(ylogy′+(1-y)log(1-y′))

[0035] Among them, y is the label value and y′ is the output value of the identification network model.

[0036] As described above, in the real-time segmentation system for thyroid gland based on ultrasound video data, in the neural network model building module, the method for outputting the segmentation map of the current frame segmentation network model includes the following steps

[0037] C1. Through Convolutional Neural Network (CNN) 4 Downsample the current frame, convolutional neural network CNN 4 It consists of three operations: convolution, batch normalization, and activation function. Downsampling is achieved through convolution with a step size of 2.

[0038] C2. Combine the downsampled feature map with the convolutional neural network CNN 5 The output feature maps are spliced;

[0039] C3. Through Convolutional Neural Network (CNN) 6 Perform feature extraction and upsampling on the concatenated feature map in step C2 to obtain a segmentation map, a convolutional neural network CNN 6 It consists of convolution, batch normalization, activation function, and bilinear interpolation function.

[0040] As described above, in a real-time thyroid gland segmentation system based on ultrasound video data, in step C3, upsampling uses a bilinear interpolation algorithm, and finally obtains a thyroid position characterization feature map with a channel number of 2 and a size that is the same as the size of the input image frame; then, the two element values ​​of the thyroid position characterization feature map are compared by channel, and when the element value of the first channel is greater than or equal to the second channel, the element value of the first channel is set to 0; when the element value of the first channel is less than the second channel, the element value of the first channel is set to 1; finally, each pixel value is multiplied by 255 and converted into a grayscale image to obtain a segmentation map.

[0041] As described above, in the real-time thyroid gland segmentation system based on ultrasound video data, in the neural network model training module, the training set data is used to train the thyroid gland segmentation neural network model. The Adam optimizer is used in the training process. The formula of the Adam optimizer is as follows:

[0042] v t =β 2 v t-1 +(1-β 2 ) t 2

[0043] m t =β 1 m t-1 +(1-β 1 ) t

[0044]

[0045] Among them, g t represents the gradient at time t; m t Represents the exponential moving average of the gradient at time t, with an initial value of m 0 =0;v t Represents the square exponential moving average of the gradient at time t, with an initial value of v 0 =0; η represents the learning rate; ε = 10 -8 , used to avoid division by 0; θ t represents the model parameters at time t, θ t-1 represents the model parameters at time t-1;

[0046] The model generated during the training process is evaluated using the test set data, and the parameters of the optimal model are determined by comprehensively considering the loss value of the model training and the accuracy on the test set.

[0047] The beneficial effects of the present invention are:

[0048] (1) In the present invention, a real-time thyroid gland segmentation technical solution based on video data is proposed, as there is currently no relevant real-time thyroid gland segmentation technical solution that can be used for thyroid ultrasound. For the real-time video segmentation network based on deep learning, the balance between segmentation accuracy and model complexity is a technical difficulty. The present invention uses an imaging quality evaluation strategy to not perform segmentation processing on some data that does not meet the image quality requirements, thereby maintaining a high segmentation accuracy while also meeting the computational requirements of real-time segmentation under a lower hardware configuration.

[0049] (2) In the present invention, a lightweight feature extraction network is used to extract features from the previous frame and the features are transferred to the current frame, which can reduce the number of parameters of the feature extraction network of the current frame and improve the speed and accuracy of the segmentation of the current frame. A generative adversarial network is used to guide the learning of the feature extraction network of the previous frame, which can improve the efficiency of the feature extraction network. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the process of image quality evaluation in an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of the structure of the thyroid gland segmentation neural network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] This embodiment provides a real-time thyroid gland segmentation system based on ultrasound video data, including a video data collection module, a neural network model construction module, a neural network model training module and a segmentation result output module.

[0053] The video data collection module is used to collect thyroid ultrasound video data, screen all video data using two image quality evaluation methods: image blur and image noise, and annotate the screened data. The specific steps include the following steps:

[0054] A1. Build a real-time segmentation device for the thyroid gland. The device is mainly composed of a video capture card, a computer host, a monitor and other equipment. The video capture card is used to collect the thyroid ultrasound video data on the ultrasound machine, and the data is transmitted to the computer host and stored on the computer host.

[0055] A2. Figure 1 As shown, all video data are screened by two image quality evaluation methods, image blur and image noise, to select video frame data that meet the image quality requirements. Only when the imaging quality of three consecutive image frames meets the evaluation requirements can it be judged to meet the image quality requirements. That is, the thyroid gland segmentation operation is performed only when the imaging quality of three consecutive image frames meets the relevant evaluation requirements.

[0056] Image blur is an important factor that degrades image quality. There are many factors that cause image blur during the process of image acquisition, compression, transmission and storage. For ultrasound imaging, the main source of image blur is motion blur caused by the movement of the ultrasound probe.

[0057] Image blur evaluation refers to objectively measuring the blur degree of an image by applying specific quantitative indicators. In this embodiment, a grayscale variance algorithm is used to quantify the blur degree of an image. The algorithm uses the grayscale average value of all pixels in the image as a reference, calculates the difference of the grayscale value of each pixel, and then squares it. Finally, the grayscale value is normalized by the total number of pixels. The specific formula is as follows:

[0058]

[0059] Where f(x, y) is the grayscale value at each pixel; N x and N y are the pixel sizes of the image in width and height respectively; s is the quantified value of the blur degree of the image, which is used to indicate the average degree of grayscale change of the image. The smaller the average degree of grayscale change, the blurrier the image, and vice versa.

[0060] Image noise evaluation refers to quantifying the noise in the image using relevant quantitative indicators. For ultrasound images, due to their imaging principles, the main noise is multiplicative speckle noise. Therefore, this embodiment uses a SNR (Signal-Noise Ratio) noise evaluation method based on a filtering method to quantify the noise of the image. The specific formula is as follows:

[0061]

[0062] Among them, US represents the original ultrasound image, and Filter(US) represents the image after the median filtering operation is performed on the original ultrasound image.

[0063] A3. The ultrasound doctor will annotate the ultrasound image that meets the image quality requirements, that is, draw the outline of the thyroid gland on the ultrasound image.

[0064] A4. Convert the labeled data into a binary grayscale image by writing a program, where the grayscale value of the thyroid gland area is set to 255 and the grayscale values ​​of other areas are set to 0.

[0065] A5. Divide the video dataset into a training set and a test set in a ratio of 8:2.

[0066] Neural network model building module for building thyroid gland segmentation neural network models using deep learning frameworks such as Figure 2 As shown, the thyroid gland segmentation neural network model includes a previous video frame feature extraction network model and a current frame segmentation network model. The previous video frame feature extraction network model is used to fuse the features of the three consecutive ultrasound image frames before the current frame, and output the predicted current frame ultrasound image and the intermediate feature map; the current frame segmentation network model is used to output the segmentation map.

[0067] The previous video frame feature extraction network model is a feature extraction network guided by generative adversarial networks. The input of the network model is the three consecutive ultrasound image frames before the current frame, and the output is the predicted ultrasound image of the current frame and the intermediate feature map. The specific process includes the following steps:

[0068] B1. Through three convolutional neural networks CNN 1 ,CNN 2 and CNN 3 Feature extraction is performed on the three consecutive ultrasound image frames before the current frame respectively. The three convolutional neural networks are composed of a series of convolution, batch normalization and activation function operations. Downsampling is achieved through convolution with a step size of 2. Then the output features of the three convolutional neural networks are spliced ​​by channel to obtain the feature vector feature.

[0069] B2. Input the feature vector feature into the generative network model Generator, and the output of the generative network model is the prediction of the ultrasound image of the current frame; the generative network model Generator consists of a series of convolutional layers, upsampling layers, batch normalization layers and activation functions, in which the resolution of the feature map is enlarged one by one through the upsampling layer, and finally reaches the resolution of the input ultrasound image, and linear interpolation with size value clipping is used in the upsampling process to achieve a smooth transition of the feature map; except for the Tanh activation function after the last convolution layer, the LeakyRelu activation function is used after all other convolutions.

[0070] B3. The predicted image image′ of the current frame and the real image image of the current frame are respectively input into the discriminator network model Discriminator. The discriminator network model Discriminator consists of a series of convolutional layers, normalization layers, fully connected layers and activation functions, wherein the resolution of the feature map is gradually reduced by convolution with a step size of 2; the input of the discriminator network model is a real ultrasound image or an ultrasound image generated by the generative network model, and the output is a probability value for judging whether the input image is true. Except for the Sigmoid function after the last convolution layer, the activation function after the remaining convolutions all uses the LeakyRelu activation function.

[0071] B4. Train the generator network model and the discriminator network model respectively. When training the discriminator network model, the real ultrasound image label is set to 1, and the generated ultrasound image label is set to 0, that is, the discriminator network model can recognize that the real ultrasound image is true and the predicted image generated by the generator network model is false through learning.

[0072] When training the generative network model Generator, the ultrasound image label generated by the generative network model Generator is set to 1, that is, the generative network model Generator can generate a prediction image that is realistic enough through learning.

[0073] In the discriminator network model, the loss function uses cross entropy loss. The specific formula is as follows:

[0074] S=-(ylogy′+(1-y)log(1-y′))

[0075] Among them, y is the label value, and y′ is the output value of the identification network model. During the training process, the identification network model’s ability to identify the input image becomes higher and higher, while the generation network model’s ability to forge images is also constantly improving.

[0076] B5. Three Convolutional Neural Networks (CNNs) 1 ,CNN 2 and CNN 3 The output feature concatenation vector feature is passed through the convolutional neural network CNN 5 The feature map obtained after processing integrates the features of the three consecutive ultrasound image frames before the current frame, and the feature map is input into the current frame segmentation network model to optimize the segmentation result of the current frame. Convolutional neural network CNN 5 It consists of two convolutions connected in series, and the stride of both convolutions is set to 1 to ensure that the size of the output feature map is consistent with the input feature map.

[0077] In the current frame segmentation network model, the convolutional neural network CNN is first used 4 Downsample the current frame and combine the downsampled feature map with the convolutional neural network CNN 5 The output feature maps are spliced ​​and finally the convolutional neural network CNN 6 The spliced ​​feature map is subjected to feature extraction and upsampling. The upsampling uses a bilinear interpolation algorithm to obtain a thyroid position characterization feature map with 2 channels and the same size as the input image frame. The two element values ​​of the thyroid position characterization feature map are then compared by channel. When the element value of the first channel is greater than or equal to the second channel, the element value of the first channel is set to 0. When the element value of the first channel is less than the second channel, the element value of the first channel is set to 1. Finally, each pixel value is multiplied by 255 and converted into a grayscale image to obtain a segmentation map.

[0078] Convolutional Neural Network (CNN) 4 It consists of a series of three operations: convolution, batch normalization, and activation function. Downsampling is achieved through convolution with a step size of 2; Convolutional Neural Network CNN 6 It consists of convolution, batch normalization, activation function, and bilinear interpolation function.

[0079] The loss function for training the current frame segmentation network model uses DiceLoss plus cross entropy loss function. The specific formula of DiceLoss is as follows:

[0080]

[0081] Among them, y′ represents the predicted value and y represents the label value.

[0082] The neural network model training module is used to set the hyperparameters of the thyroid gland segmentation neural network model and train the neural network model using the training set data to determine the parameters of the optimal model. After the training is completed, the optimal model is saved. The Adam optimizer is used during the training process. The formula of the Adam optimizer is as follows:

[0083] v t =β 2 v t-1 +(1-β 2 ) t 2

[0084] m t =β 1 m t-1 +(1-β 1 ) t

[0085]

[0086]

[0087] Among them, g t represents the gradient at time t; m t Represents the exponential moving average of the gradient at time t, with an initial value of m 0 =0;v t Represents the square exponential moving average of the gradient at time t, with an initial value of v 0 =0; η represents the learning rate; ε = 10 -8 , used to avoid division by 0; θ t represents the model parameters at time t, θ t-1 Represents the model parameters at time t-1.

[0088] The model generated during the training process is evaluated using the test set data, and the parameters of the optimal model are determined by comprehensively considering the loss value of the model training and the accuracy on the test set.

[0089] Segmentation result output module: After the thyroid gland segmentation neural network model training is completed, the optimal model obtained is the final thyroid gland real-time segmentation model. The final thyroid gland real-time segmentation model is deployed on the segmentation device, and the thyroid gland video data is collected through the video acquisition card and input into the final thyroid gland real-time segmentation model. The model can output the thyroid gland segmentation result and display the position outline of the thyroid gland in real time on the ultrasound video according to the segmentation result.

[0090] In view of the problem that traditional image segmentation algorithms cannot accurately segment the thyroid gland area on thyroid ultrasound images, this embodiment uses artificial intelligence technology to achieve fast and accurate segmentation and recognition of the thyroid gland area on thyroid ultrasound images; compared with existing segmentation and recognition methods, the model obtained by this method has the advantages of high accuracy and good generalization performance, and can achieve fast and accurate segmentation of thyroid ultrasound video data.

[0091] This embodiment uses an imaging quality evaluation strategy and does not perform segmentation processing on some data that does not meet the image quality requirements, thereby maintaining a high segmentation accuracy while also meeting the computational requirements for real-time segmentation with a lower hardware configuration.

[0092] This embodiment uses a lightweight feature extraction network to extract features from the previous frame and transfers the features to the current frame, which can reduce the number of parameters of the feature extraction network of the current frame and improve the speed and accuracy of the segmentation of the current frame; using a generative adversarial network to guide the learning of the feature extraction network of the previous frame can extract the efficiency of the feature extraction network.

[0093] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.

Claims

1. A real-time segmentation system for thyroid gland based on ultrasound video data, characterized in that: include The video data collection module is used to collect thyroid ultrasound video data, screen all video data using two image quality evaluation methods: image blur and image noise, and annotate the screened data; A neural network model building module is used to build a thyroid gland segmentation neural network model using a deep learning framework. The thyroid gland segmentation neural network model includes a previous video frame feature extraction network model and a current frame segmentation network model. The previous video frame feature extraction network model is used to fuse features of three consecutive ultrasound image frames before the current frame, and output a predicted current frame ultrasound image and an intermediate feature map; the current frame segmentation network model is used to output a segmentation map; A neural network model training module is used to set the hyperparameters of the thyroid gland segmentation neural network model and train the neural network model to determine the parameters of the optimal model and save the optimal model after the training is completed; The segmentation result output module is used to input the thyroid ultrasound video data into the optimal model, output the thyroid gland segmentation result, and display the position outline of the thyroid gland in real time on the ultrasound video according to the segmentation result; In the neural network model building module, the method of fusing the features of the three consecutive ultrasound image frames before the current frame with the previous video frame feature extraction network model includes the following steps: B1. Use three convolutional neural networks CNN1, CNN2 and CNN3 to extract features from the three consecutive ultrasound image frames before the current frame, and concatenate the output features of the three convolutional neural networks by channel to obtain a feature vector feature; B2, input the feature vector feature into the generative network model Generator, and output the predicted ultrasound image image′ of the current frame; B3, input the predicted image image′ of the current frame and the real image image of the current frame into the discriminator network model respectively, and the output of the discriminator network model is the probability value of judging whether the input image is true; B4. Train the generator network model and the discriminator network model respectively; B5, input the feature vector feature into the convolutional neural network CNN5, and output a feature map that integrates the features of the three consecutive ultrasound image frames before the current frame. The convolutional neural network CNN5 consists of two convolutions in series, and the step sizes of the two convolutions are both set to 1; In step B2, the network model Generator is composed of a convolution layer, an upsampling layer, a batch normalization layer, and an activation function. The upsampling layer is used to enlarge the resolution of the feature map one by one, and finally reaches the resolution of the input ultrasound image. In the upsampling process, linear interpolation with size value clipping is used to achieve a smooth transition of the feature map. Except for the Tanh activation function after the last convolution layer, the activation function uses the LeakyRelu activation function after all other convolutions. In step B3, the input of the discriminator network model is the real image image of the current frame or the predicted image image′ of the current frame generated by the generator network model, and the output is the probability value of judging whether the input image is true or not; The discriminator network model consists of a convolution layer, a normalization layer, a fully connected layer, and an activation function. The resolution of the feature map is gradually reduced by a convolution with a step size of 2. The activation function uses the LeakyRelu activation function after all convolutions except the last convolution layer, which uses the Sigmoid function. In the neural network model training module, the training set data is used to train the thyroid gland segmentation neural network model. The Adam optimizer is used in the training process. The formula of the Adam optimizer is as follows: v t =β2v t-1 +(1-β2)g t 2 m t =β1m t-1 +(1-β1)g t Among them, gt represents the gradient at time t; m t represents the exponential moving average of the gradient at time t, with an initial value of m0=0; v t represents the exponential moving average of the squared gradient at time t, with an initial value of v0 = 0; η represents the learning rate; ε = 10 -8 , used to avoid division by 0; θ t represents the model parameters at time t, θ t-1 represents the model parameters at time t-1; The model generated during the training process is evaluated using the test set data, and the parameters of the optimal model are determined by comprehensively considering the loss value of the model training and the accuracy on the test set.

2. A real-time thyroid gland segmentation system based on ultrasonic video data according to claim 1, characterized in that: The method for collecting and marking by the video data collection module comprises the following steps A1. Collect thyroid ultrasound video data from the ultrasound machine through a video acquisition card; A2. All video data are screened using two image quality evaluation methods: image blur and image noise, to select video frame data that meets the image quality requirements. Only when the imaging quality of three consecutive image frames meets the evaluation requirements is it judged to meet the image quality requirements; A3. The ultrasound doctor shall annotate the ultrasound image that meets the image quality requirements, that is, draw the outline of the thyroid gland on the ultrasound image; A4. Convert the labeled data into a binary grayscale image, where the grayscale value of the thyroid gland area is set to 255, and the grayscale values ​​of other areas are set to 0; A5. Divide the video dataset into training set and test set in a ratio of 8:

2.

3. The real-time segmentation system of thyroid gland based on ultrasonic video data according to claim 2, characterized in that: In step A2, the grayscale variance algorithm is used to quantify the blur degree of the ultrasound image, as shown in the following formula: Where f(x, y) is the grayscale value at each pixel; N x and N y are the pixel sizes of the image in width and height respectively; s is the quantified value of the blur degree of the image, which is used to indicate the average degree of grayscale change of the image; The noise of ultrasound images is quantified using the signal-to-noise ratio noise evaluation method based on filtering, as shown in the following formula: Among them, US represents the original ultrasound image, and Filter(US) represents the image after the median filtering operation is performed on the original ultrasound image.

4. The real-time segmentation system of thyroid gland based on ultrasound video data according to claim 1, characterized in that: In step B1, the three convolutional neural networks CNN1, CNN2 and CNN3 are composed of three operations: convolution, batch normalization and activation function, and downsampling is achieved through convolution with a step size of 2.

5. The real-time segmentation system of thyroid gland based on ultrasound video data according to claim 1, characterized in that: In the step B4, when training the discriminator network model, the real image label of the current frame is set to 1, and the generated predicted image label of the current frame is set to 0; when training the generator network model, the predicted image label of the current frame generated by the generator network model is set to 1; In the discriminator network model, the loss function uses cross entropy loss. The specific formula is as follows: S=-(ylogy′+(1-y)log(1-y′)) Among them, y is the label value and y′ is the output value of the identification network model.

6. The real-time segmentation system of thyroid gland based on ultrasound video data according to claim 1, characterized in that: In the neural network model construction module, the method for outputting a segmentation map of the current frame segmentation network model comprises the following steps c1. Downsample the current frame through the convolutional neural network CNN4. The convolutional neural network CNN4 consists of three operations: convolution, batch normalization, and activation function. It achieves downsampling through convolution with a step size of 2; c2. Concatenate the downsampled feature map with the feature map output by the convolutional neural network CNN5; c3. Perform feature extraction and upsampling on the feature map spliced ​​in step c2 through the convolutional neural network CNN6 to obtain a segmentation map. The convolutional neural network CNN6 consists of convolution, batch normalization, activation function and bilinear interpolation function.

7. The real-time segmentation system of thyroid gland based on ultrasound video data according to claim 6, characterized in that: In the step c3, upsampling uses a bilinear interpolation algorithm, and finally obtains a thyroid position characterization feature map with a channel number of 2 and a size that is the same as the size of the input image frame; then, two element values ​​of the thyroid position characterization feature map are compared by channel, and when the element value of the first channel is greater than or equal to the second channel, the element value of the first channel is set to 0; when the element value of the first channel is less than the second channel, the element value of the first channel is set to 1; Finally, multiply each pixel value by 255 and convert it to a grayscale image to get the segmentation map.

Images