Dataset intelligent augmentation method for thyroid ultrasound image segmentation

Abstract

The application discloses a kind of data set intelligent enhancement methods for thyroid ultrasound image segmentation, by original thyroid ultrasound data set and expert labeled label Intelligent data enhancement is carried out, and a complete initial data set is constructed;Then determine all data enhancement algorithms, automatically randomly select two kinds from data enhancement algorithm Composition 1 pair combination method, and then automatically randomly select 4 pairs of combination method to form a plan, use all the planning to form planning domain, automatically select an enhanced data set from the planning domain Planning, and train the sub-segmentation network to obtain the average intersection-over-union ratio for verification, use the average intersection-over-union ratio as the incentive to update the initialized LSTM controller, until the controller is trained;After training, the optimal 5 planning is selected, and all 20 methods in the 5 planning are combined to form a final planning scheme, and the intelligent enhanced data set is obtained using the final planning scheme.

Description

Technical Field

[0001] This invention relates to the field of ultrasound medical image recognition and processing, specifically to image enhancement technology at the intersection of deep learning and medical images, and more specifically to a dataset intelligent enhancement method for thyroid ultrasound image segmentation. Background Technology

[0002] With the continuous development of medical equipment, ultrasound imaging instruments have become one of the most widely used medical tools in clinical practice due to their advantages such as non-invasiveness, real-time imaging, ease of operation, and low cost. The thyroid gland's impact on human health is increasing, and ultrasound technology is recognized as a suitable technique for examination. Thyroid ultrasound imaging is a common medical imaging technique in clinical practice, and effective segmentation of thyroid ultrasound images can provide doctors with a wealth of useful information for clinical diagnosis.

[0003] Deep learning is a method that uses deep neural network models to learn patterns from massive amounts of data. It typically requires training a deep network with millions of parameters on a massive dataset to obtain a model that better reflects the mapping relationship between samples in the dataset and the training target. In the combination of medical imaging technology and deep learning, achieving accurate segmentation of medical images is of great practical significance. Accurate segmentation can provide clinicians with a wealth of precise shape information about target regions, assisting them in making diagnostic decisions.

[0004] The emergence of deep learning has made it feasible to segment thyroid ultrasound images without the heavy workload of professional physicians. However, the acquisition and annotation of thyroid ultrasound images still heavily relies on the skills and experience of doctors, and is time-consuming and labor-intensive, which contradicts the massive amounts of data required for training deep learning networks. In practical applications, deep learning segmentation networks trained with limited thyroid ultrasound samples often perform poorly. Integrating or improving common deep learning and traditional machine learning methods to address the problem of thyroid segmentation using limited ultrasound images has yielded some research results both domestically and internationally, but there is still significant room for research on intelligent image enhancement to fundamentally improve the performance of segmentation networks. In the field of deep learning image segmentation, dataset augmentation techniques are image processing techniques tailored to the specific application scenario. Based on the characteristics of the images in the dataset, they obtain datasets with more abundant data, clearer images, and richer samples to improve the generalization ability of deep learning. Dataset augmentation techniques typically include methods such as cropping, flipping, filtering and denoising, random cropping, brightness adjustment, and contrast adjustment. Current dataset augmentation techniques are mainly designed manually. The design process involves selecting and combining some dataset augmentation methods based on engineering experience, making it difficult to maximize the richness and sufficiency of the dataset.

[0005] Therefore, this paper proposes an intelligent augmentation method for thyroid ultrasound image segmentation. This method intelligently augments a limited thyroid ultrasound image dataset to increase the amount and diversity of samples in the original dataset, thereby fundamentally improving the quality of the dataset, enhancing the performance of intelligent thyroid ultrasound image segmentation, and facilitating the use of ultrasound images by subsequent users. This is a technical problem that urgently needs to be solved in the current market. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to provide an intelligent dataset augmentation method for thyroid ultrasound image segmentation. Based on a thyroid ultrasound dataset, this method intelligently plans and augments the dataset, comprehensively expanding it and fully utilizing the samples to effectively enhance sample diversity and sufficiency, thereby significantly improving the accuracy of thyroid ultrasound image recognition and segmentation. The intelligently augmented thyroid ultrasound dataset is more advantageous for training deep learning networks and possesses transferability to other medical ultrasound image datasets.

[0007] Because combining deep learning with thyroid ultrasound images for segmentation often encounters the problem of small sample sizes, dataset augmentation methods are particularly important in this field. Traditional dataset augmentation methods not only heavily rely on human design, making the augmentation overly dependent on the designer's subjective will, but also offer limited improvement in diversity after expansion, and usually require the designer to invest a significant amount of effort in debugging and validation. Furthermore, simply using all augmentation methods to improve the dataset can lead to dataset redundancy, resulting in excessively long training times and images negatively impacting training results.

[0008] To address the shortcomings of existing technologies, the present invention aims to provide an intelligent dataset enhancement method for thyroid ultrasound image segmentation. This method overcomes the aforementioned problems of traditional dataset enhancement by performing intelligent dataset enhancement, thereby maximizing the diversity and volume of dataset samples while ensuring that the dataset is not redundant. This provides a complete foundation for improving the subsequent thyroid ultrasound image segmentation results.

[0009] To achieve the above objectives, this invention provides a dataset intelligent enhancement method for thyroid ultrasound image segmentation, comprising the following steps:

[0010] Step S1: Automatically construct the original dataset;

[0011] Step S2: Determine the dataset augmentation method to be used, and automatically construct a planning domain containing multiple dataset augmentation plans based on the determined dataset augmentation method;

[0012] Step S3: Enhance the planning using the various datasets in the constructed planning domain, train and update the LSTM controller;

[0013] Step S4: Automatically select the optimal plan, concatenate the optimal plans, and obtain the enhanced dataset for thyroid ultrasound image segmentation.

[0014] In one embodiment of the present invention, step S1 includes the following sub-steps:

[0015] Step S11: Obtain the original thyroid ultrasound image and obtain the labels annotated by medical experts, wherein the type of the labels annotated by medical experts is a binary image;

[0016] Step S12: Automatically match the original thyroid ultrasound image with the labels annotated by the medical expert;

[0017] Step S13: Create the original dataset and automatically divide the original dataset into multiple data subsets according to a preset ratio, including: training set, test set and validation set.

[0018] In one embodiment of the present invention, in step S13, the specific rules for dividing the original dataset based on the acquired original thyroid ultrasound images and labels annotated by medical experts include:

[0019] Ensure that multiple raw thyroid ultrasound images of any subject from different scanning angles during the same examination, along with their corresponding matching labels annotated by medical experts, are in the same data subset; and

[0020] Ensure that multiple raw thyroid ultrasound images obtained from multiple examinations of the same subject and their corresponding matching labels annotated by medical experts are in the same data subset.

[0021] In one embodiment of the present invention, step S2 includes the following sub-steps:

[0022] Step S21: Determine the set of dataset augmentation methods to be used, including: determining dataset augmentation methods that are general in the field of deep learning and dataset augmentation methods that are specific to the field of ultrasound image segmentation;

[0023] Step S22: For each dataset augmentation method, two additional parameters are automatically assigned: augmentation probability and augmentation strength. Each parameter has a maximum of nine preset discrete values, where:

[0024] If any dataset augmentation method does not require augmentation probability and / or augmentation strength parameters, then the corresponding parameter of the dataset augmentation method is marked as False;

[0025] If any dataset augmentation method requires augmentation probability and / or augmentation strength parameters, then the corresponding parameters for the dataset augmentation method are marked as True;

[0026] When a parameter is marked as False, the corresponding parameter has no value. When a parameter is marked as True, the corresponding parameter takes multiple discrete values. The different values ​​of the two parameters of each dataset augmentation method are combined, and a method parameter is set for the combined method itself to obtain a set of dataset augmentation methods. Each method in the set contains at least 3 parameters.

[0027] Step S23: Each time, two data augmentation methods are automatically and randomly selected from the set of data augmentation methods to form a pair of combined methods, resulting in a set of data augmentation combined methods, where each pair of combined methods has 2×3=6 parameters;

[0028] Step S24: From the set of constructed dataset augmentation combination methods, automatically and randomly select 4 pairs of combination methods each time to obtain a dataset augmentation plan. Each dataset augmentation plan contains 6 × 4 = 24 parameters.

[0029] Step S25: Enhance the planning domain by combining all the obtained datasets.

[0030] In one embodiment of the present invention, the dataset augmentation method commonly used in the field of deep learning includes: X-axis projection, Y-axis projection, rotation / rotation + flip, X-axis translation, Y-axis translation, contrast adjustment, sharpening, brightness adjustment, exposure adjustment, random cropping, phase inversion and / or histogram equalization; the dataset augmentation method specifically used in the field of ultrasound image segmentation includes: linear grayscale stretching, median filtering and / or Sobel operator edge enhancement.

[0031] In one embodiment of the present invention, step S3 includes the following sub-steps:

[0032] Step S31: Automatically and randomly select one planning method from the constructed planning domain. S t The training set in step S13 is augmented to obtain an augmented dataset to be trained.

[0033] Step S32: Train a segmentation sub-network using the obtained augmented dataset to be trained, and validate the trained segmentation sub-network using the validation set from step S13 to obtain the validation average intersection-union ratio. A t ;

[0034] Step S33: Build the LSTM controller and primitive the LSTM controller, that is, primitive all values ​​to the range [-0.1, 0.1];

[0035] Step S34: Calculate the verification average crossover and union ratio obtained in step S32. A t As part of this incentive, and based on this incentive... A t And the correlation between this planning and the previous planning, using the survival of the fittest allocation rule, to train and update the LSTM controller;

[0036] Repeat steps S31-S34 until the controller training is complete.

[0037] In one embodiment of the present invention, the segmentation sub-network is a U-Net network, the LSTM controller includes a layer of LSTM neural network with 100 hidden units and a Softmax classifier with 24 output predictions, and the LSTM neural network describes the probability distribution of the next operation based on the operation of the previous step, and finally estimates the distribution of each operation.

[0038] In one embodiment of the present invention, in step S34, the product of the relevance probabilities predicted by the 24 outputs of the Softmax classifier is used as the joint probability, and the relevance is measured by a ratio, thereby obtaining the following loss function:

[0039] ;

[0040] In the formula, This represents the expected value of the probability distribution during planning. Indicating in training planning S t The joint probability after that, Indicating in training planning S t The joint probability before, Indicating in planning S t The intersection-union ratio (IoU) of the model with respect to the validation set is the mean. O The label represents the output of the segmentation subnetwork. M This represents the corresponding true label in the dataset. w Indicates the penalty weight. Information divergence is used to reflect the current planning. S t The gap between the current plan and the previous plan.

[0041] In one embodiment of the present invention, step S4 includes the following sub-steps:

[0042] Step S41: Using the trained LSTM controller, evaluate each plan with a loss function, and automatically select the top 5 optimal plans from the planning domain to obtain a total of 20 pairs of combined methods, which constitute the optimal dataset augmentation scheme.

[0043] Step S42: After sorting and concatenating the 20 pairs of combination methods in descending order of quality, the training set is augmented to obtain the final intelligently augmented thyroid segmentation dataset.

[0044] Compared with existing technologies, this invention considers two types of dataset augmentation methods and uses an LSTM controller to automatically and intelligently select the 20 optimal combination methods from a large number of combination method pairs to achieve intelligent augmentation of thyroid ultrasound image datasets. This frees the datasets from the heavy workload of manual data augmentation and the influence of subjective human factors, while better improving the diversity and volume of the datasets. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of an embodiment of the intelligent dataset enhancement method for thyroid ultrasound image segmentation according to the present invention;

[0047] Figure 2 This is a dataset augmentation example under one planning method in one embodiment of the present invention;

[0048] Figure 3 This is a detailed diagram illustrating the training of an LSTM controller to filter plans in one embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Figure 1 This is a flowchart of an embodiment of the intelligent dataset enhancement method for thyroid ultrasound image segmentation according to the present invention, as shown below. Figure 1As shown, this invention performs intelligent data augmentation on the original thyroid ultrasound dataset and expert-annotated labels. Before augmentation, the images are first divided into training, testing, and validation sets according to a scientific ratio to construct a complete initial dataset. Then, all data augmentation algorithms are determined, including two parts: commonly used deep learning data augmentation and methods specifically for thyroid ultrasound segmentation. A sufficient number of methods suitable for this field can improve the performance of the augmented dataset. Next, two methods are automatically and randomly selected from the data augmentation algorithms to form a combination method, and then four combinations are automatically and randomly selected to form a plan. All plans are used to form a planning domain. From the planning domain, a plan is automatically selected to augment the dataset, and a sub-segmentation network is trained to obtain the validation average intersection-union ratio (IU). The obtained IU is used as the stimulus to update the constructed and initialized LSTM controller until the controller training is complete. After training, the five optimal plans are selected, and all 20 method pairs contained in the five plans are used to form the final planning scheme. The final planning scheme is used to obtain the intelligent augmented dataset. The specific steps of the intelligent augmentation method for thyroid ultrasound image segmentation in this embodiment include:

[0051] Step S1: Automatically construct the original dataset;

[0052] In this embodiment, step S1 may include the following sub-steps:

[0053] Step S11: Obtain the original thyroid ultrasound image and obtain the labels annotated by medical experts, wherein the type of the labels annotated by medical experts is a binary image; in this embodiment, the original thyroid ultrasound image and the labels annotated by medical experts can be collected and annotated by professional physicians.

[0054] Step S12: Automatically match the original thyroid ultrasound image with the labels marked by the medical expert; In this embodiment, the matching of the original thyroid ultrasound image and the labels marked by the medical expert can be done by automatic renaming. For example, if the name of the original thyroid ultrasound image is "xxx.bmp", the name of the corresponding label marked by the medical expert is "xxx_mask.bmp", where "xxx" can be the serial number of the person being tested. The original thyroid ultrasound image usually marks the image source with the serial number of the person being tested.

[0055] Step S13: Create the original dataset and automatically divide the original dataset into multiple data subsets according to a preset ratio, including: training set, test set and validation set.

[0056] In this embodiment, in step S13, the original dataset is divided according to rigorous scientific rules based on the acquired original thyroid ultrasound images and labels annotated by medical experts. These rules include:

[0057] Ensure that multiple raw thyroid ultrasound images of any subject from different scanning angles during the same examination, along with their corresponding matching labels annotated by medical experts, are in the same data subset; and

[0058] Ensure that multiple raw thyroid ultrasound images obtained from multiple examinations of the same subject and their corresponding matching labels annotated by medical experts are in the same data subset.

[0059] In this embodiment, when dividing the training, validation, and test sets, images from folders with the same index can be automatically grouped into the same dataset, and the original dataset can be automatically divided into training, validation, and test sets according to a preset ratio of 0.75:0.15:0.15. In other embodiments, the division ratio of multiple data subsets can also be adjusted as needed, and this invention does not limit it.

[0060] Step S2: Determine the dataset augmentation method to be used, and automatically construct a planning domain containing multiple dataset augmentation plans based on the determined dataset augmentation method;

[0061] In this embodiment, step S2 may include the following sub-steps:

[0062] Step S21: Determine the set of dataset augmentation methods to be used, including: determining dataset augmentation methods that are general in the field of deep learning and dataset augmentation methods that are specific to the field of ultrasound image segmentation;

[0063] Dataset augmentation methods commonly used in the field of deep learning can include projection (X / Y axis), rotation / rotation + flip, translation (X / Y axis), contrast adjustment, sharpening, brightness adjustment, exposure adjustment, random cropping, phase inversion, histogram equalization, etc. Dataset augmentation methods specific to the field of ultrasound image segmentation can include linear grayscale stretching, median filtering, Sobel operator edge enhancement, etc.

[0064] Step S22: For each dataset augmentation method, two additional parameters are automatically assigned: augmentation probability and augmentation strength. Each parameter has a maximum of nine preset discrete values, where:

[0065] If any dataset augmentation method does not require augmentation probability and / or augmentation strength parameters, then the corresponding parameter of the dataset augmentation method is marked as False;

[0066] If any dataset augmentation method requires augmentation probability and / or augmentation strength parameters, then the corresponding parameters for the dataset augmentation method are marked as True;

[0067] When a parameter is marked as False, the corresponding parameter has no value, which is equivalent to the data augmentation method having only one augmentation probability and / or augmentation strength. When a parameter is marked as True, the corresponding parameter takes multiple discrete values. The different values ​​of the two parameters of each dataset augmentation method are combined, and the combined method itself sets a method parameter, resulting in a set of dataset augmentation methods, and each method in the set contains at least 3 parameters.

[0068] For example, Tables 1 and 2 below are lists of ultrasound image segmentation-specific dataset enhancement methods and general dataset enhancement methods used in one embodiment of the present invention:

[0069] ;

[0070] ;

[0071] The augmentation probability of each augmentation method is discretized into nine values ​​evenly distributed within the range of [0.1-0.9]. If an augmentation method does not require an augmentation probability parameter, the corresponding column in Table 1 or Table 2 is marked as False; if an augmentation method requires an augmentation probability parameter, the corresponding column in Table 1 or Table 2 is marked as True. In this embodiment, the augmentation probability represents the probability of planning to use the corresponding dataset augmentation method in subsequent steps. For example, in Table 1 and Table 2, the augmentation probability column is all True, indicating that each method requires the augmentation probability parameter, and this parameter takes nine discrete values: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9.

[0072] Similarly, the augmentation intensity of each dataset augmentation method can also be divided into 9 levels, specifically, 9 levels evenly distributed within their respective value ranges. For example, when the augmentation intensity value of median filtering is in the range of [3-19], the 9 levels of the median filtering augmentation intensity threshold can be 3, 5, 7, 9, 11, 13, 15, 17, and 19, respectively. If a dataset augmentation method does not require setting an augmentation intensity parameter, it is marked as False in Table 1 or Table 2. The augmentation intensity of the rotation / rotation + flip method is divided into eight levels, with thresholds for 0°, 90°, 180°, 270°, and flips at these four degrees. The Sobel operator edge augmentation has nine levels of augmentation intensity parameters: no stretching, stretching to [112, 143], stretching to [96, 159], stretching to [80, 175], stretching to [64, 191], stretching to [48, 207], stretching to [32, 223], stretching to [16, 239], and stretching to [0, 255]. In other words, there are eight levels plus no stretching within the range [0, 255], totaling nine levels.

[0073] By combining the different values ​​of the two parameters of each dataset augmentation method in Tables 1 and 2, a set of dataset augmentation methods is obtained. Among them, the augmentation strength parameter of inversion and histogram equalization is marked as False, so its value is 1. The augmentation strength parameter of rotation / rotation + flip has 8 values. The two parameters of other dataset augmentation methods all have 9 values. Therefore, by combining the 3 types + 12 types of dataset augmentation methods in Tables 1 and 2, there are a total of 12 × 9 × 9 + 2 × 9 + 1 × 9 × 8 = 1062 candidate methods in the set of dataset augmentation methods (among which 2 methods do not contain augmentation strength parameters, and one method contains only 8 augmentation strength parameters). Each method includes 3 parameters: one method parameter, one augmentation strength parameter, and one augmentation probability parameter.

[0074] Step S23: Each time, two data augmentation methods are automatically and randomly selected from the set of data augmentation methods to form a pair of combined methods, resulting in a set of data augmentation combined methods, where each pair of combined methods has 2×3=6 parameters; among them, two data augmentation methods are selected from all the methods in Table 1 and Table 2 in order to form a pair of combined methods, so 1062 ×1061 = 1126782 pairs of combined methods can be obtained.

[0075] Step S24: From the constructed set of dataset augmentation combination methods, automatically and randomly select 4 pairs of combination methods each time to obtain a dataset augmentation program. Each dataset augmentation program contains 6 × 4 = 24 parameters. Among them, randomly selecting 4 pairs of combination methods from the entire set of combination methods, regardless of the order, to form one program, can obtain approximately Such a plan.

[0076] Step S25: Enhance the planning domain by combining all the obtained datasets.

[0077] Figure 2 This is an example of dataset augmentation under one planning method in one embodiment of the present invention, such as... Figure 2As shown, the leftmost image is an original thyroid ultrasound image. The images in the first row are obtained by data augmentation using the first method of any pair of combination methods in step 23, and the images in the second row are obtained after data augmentation using the same pair of combination methods. The four pairs of combination methods used in this planning example are as follows: the first pair of combination methods rotates the image by 270° with a probability of 0.8, and then adjusts the brightness by 1.6 with a probability of 0.6; the second pair of combination methods sharpens the image with a probability of 0.8 with a probability of 0.9, and then performs Sobel operator enhancement without stretching with a probability of 0.4; the third pair of combination methods adjusts the contrast with a probability of 0.4 with a probability of 0.3, and then randomly crops the image by 8 pixels with a probability of 0.7; the fourth pair of combination methods flips the image with a probability of 0.6, and then performs median filtering with a window size of 0.3 with a probability of 0.5.

[0078] After the dataset is augmented using the dataset augmentation method of this invention, the diversity of samples in the dataset can be enriched and the amount of data in the dataset can be increased.

[0079] Step S3: Enhance the planning by utilizing the various datasets in the constructed planning domain, train and update the LSTM (Long Short-Term Memory) controller to obtain an LSTM controller that can evaluate each plan.

[0080] Figure 3 In one embodiment of the present invention, training an LSTM controller to filter detailed planning graphs is shown, such as... Figure 3 As shown, in this embodiment, the process of training the LSTM controller is as follows: First, the constructed LSTM controller is initialized. Then, any planning algorithm in the planning domain is used to augment the original dataset, and the U-Net segmentation sub-network is trained using the augmented dataset. After obtaining the trained segmentation model, the model is validated on the validation set, and the validation average intersection-union ratio is used as the stimulus to update the LSTM controller. After the LSTM controller update is completed, one training cycle is completed. Then, the aforementioned process using planning is repeated until training is complete. Step S3 may include the following sub-steps:

[0081] Step S31: Automatically and randomly select one planning method from the constructed planning domain. S t The training set in step S13 is augmented to obtain an augmented dataset to be trained.

[0082] Step S32: Train a segmentation sub-network using the obtained augmented dataset to be trained, and validate the trained segmentation sub-network using the validation set from step S13 to obtain the validation average intersection-union ratio. A t In subsequent steps, the verification average crossover ratio obtained in this step will be used as the basis for calculation. At As an indicator for evaluating various plans.

[0083] Step S33: Build the LSTM controller and primitive the LSTM controller, that is, primitive all values ​​to the range [-0.1, 0.1];

[0084] Step S34: Calculate the verification average crossover and union ratio obtained in step S32. A t As part of this incentive, and based on this incentive... A t And the correlation between this planning and the previous planning, using the survival of the fittest allocation rule, to train and update the LSTM controller;

[0085] Repeat steps S31-S34 until the controller training is complete.

[0086] In this embodiment, the segmentation sub-network is a U-Net network (an algorithm for semantic segmentation using fully convolutional networks). In this embodiment, the LSTM controller comprises a single-layer LSTM neural network with 100 hidden units and a Softmax classifier with 24 output predictions. The principle of the LSTM neural network (Long Short-Term Memory network controller) is: based on the operation of the previous step, it describes the probability distribution of the next operation, and finally, the distribution of each operation can be estimated.

[0087] In this embodiment, in step S34, the product of the relevance probabilities predicted by the 24 outputs of the Softmax classifier is used as the joint probability, and the relevance is measured by the ratio, thereby obtaining the following loss function, which is used to estimate the degree of inconsistency between the model's predicted values ​​and the true values:

[0088] ;

[0089] Its main body is:

[0090] ;

[0091] In the formula, This represents the expected value of the probability distribution during planning. Indicating in training planning S t The joint probability after that, Indicating in training planning S t The joint probability before, Indicating in planning S t The intersection-union ratio (IoU) of the model with respect to the validation set is the mean. OThis represents the label output by the segmentation subnetwork (U-Net segmentation subnetwork). M This represents the corresponding true label in the dataset;

[0092] In addition, to encourage controller updates and the experimentation of more planning methods, the rules penalize the loss function, and the penalty part is as follows:

[0093] ;

[0094] In the formula, w Indicates the penalty weight. Information divergence is used to reflect the current planning. S t The gap between the current plan and the previous plan.

[0095] Among them, the penalty weights in the loss function w The value can be designed to be 0.00001. During the LSTM training of the controller, as many plans as possible will be traversed, for example, up to 10,000 plans, to ensure that the loss function does not converge too quickly. Training can stop once the loss function converges. The formula for calculating the loss function ensures that when the current result is highly correlated with the previous result, ... L It will be very small, on the contrary L It will be very large. (Use) L The LSTM controller, which serves as the loss function, can assign lower scores to methods that perform well so that they are retained, while assigning higher scores to methods that perform poorly so that they are eliminated.

[0096] Step S4: Automatically select the optimal plan, concatenate the optimal plans, and obtain the enhanced dataset for thyroid ultrasound image segmentation.

[0097] In this embodiment, step S4 may include the following sub-steps:

[0098] Step S41: Using the trained LSTM controller, evaluate each plan using the loss function, and automatically select the top 5 optimal plans from the planning domain (each plan has 4 pairs of combined methods), resulting in a total of 20 pairs of combined methods, forming the optimal dataset augmentation scheme. The optimal plan is the one with the smallest loss function value; the top 5 optimal plans are the 5 plans with the smallest loss function values.

[0099] Typically, these 20 pairs of combination methods are of different types or have different parameters because: firstly, a single planning algorithm will not have two pairs of combination methods with exactly the same parameters; secondly, the parameters of each pair of combination methods are optimal within their respective planning algorithms, and it is difficult for them to be completely identical after training. Even if some of the final 20 pairs of combination methods have the same parameters, the processing method is the same. Just process them sequentially according to the selected 20 methods; if they exist twice, they can be processed twice. For example, the dataset might first be flipped 90 degrees with a 20% probability, with some other augmentations performed in between, and then flipped 90 degrees again with a 20% probability; therefore, the combination methods do not conflict.

[0100] Step S42: After sorting and concatenating the 20 pairs of combination methods in descending order of quality, the training set is augmented to obtain the final intelligently augmented thyroid segmentation dataset.

[0101] The above describes a specific implementation of the present invention for intelligent augmentation of a thyroid ultrasound image segmentation dataset available in an existing laboratory. By considering two types of dataset augmentation methods, the present invention utilizes an LSTM controller to automatically and intelligently select the 20 optimal combination methods from 1,126,782 combinations to achieve intelligent augmentation of the thyroid ultrasound image dataset. This approach eliminates the heavy workload of manual data augmentation and the influence of subjective human factors, while simultaneously improving the diversity and volume of the dataset.

[0102] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0103] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dataset intelligent augmentation method for thyroid ultrasound image segmentation, characterized in that, Includes the following steps: Step S1: Automatically construct the original dataset; Step S2: Determine the dataset augmentation method to be used, and automatically construct a planning domain containing multiple dataset augmentation plans based on the determined dataset augmentation method, specifically including: Step S21: Determine the set of dataset augmentation methods to be used, including: determining dataset augmentation methods general to the deep learning field and dataset augmentation methods specific to the ultrasound image segmentation field. The dataset augmentation methods general to the deep learning field include: X-axis projection, Y-axis projection, rotation / rotation + flip, X-axis translation, Y-axis translation, contrast adjustment, sharpening, brightness adjustment, exposure adjustment, random cropping, phase inversion, and / or histogram equalization. The dataset augmentation methods specific to the ultrasound image segmentation field include: linear grayscale stretching, median filtering, and / or Sobel operator edge enhancement. Step S22: For each dataset augmentation method, two additional parameters are automatically assigned: augmentation probability and augmentation strength. Each parameter has a maximum of nine preset discrete values, where: If any dataset augmentation method does not require augmentation probability and / or augmentation strength parameters, then the corresponding parameter of the dataset augmentation method is marked as False; If any dataset augmentation method requires augmentation probability and / or augmentation strength parameters, then the corresponding parameters for the dataset augmentation method are marked as True; When a parameter is marked as False, the corresponding parameter has no value. When a parameter is marked as True, the corresponding parameter takes multiple discrete values. The different values ​​of the two parameters of each dataset augmentation method are combined, and a method parameter is set for the combined method itself to obtain a set of dataset augmentation methods. Each method in the set contains at least 3 parameters. Step S23: Each time, two data augmentation methods are automatically and randomly selected from the set of data augmentation methods to form a pair of combined methods, resulting in a set of data augmentation combined methods, where each pair of combined methods has 2×3=6 parameters; Step S24: From the set of constructed dataset augmentation combination methods, automatically and randomly select 4 pairs of combination methods each time to obtain a dataset augmentation plan. Each dataset augmentation plan contains 6 × 4 = 24 parameters. Step S25: Combine the obtained datasets into a planning domain using augmented planning. Step S3: Enhance the planning using the various datasets in the constructed planning domain, train and update the LSTM controller; Step S4: Automatically select the optimal plan, concatenate the optimal plans, and obtain the enhanced dataset for thyroid ultrasound image segmentation.

2. The method according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Obtain the original thyroid ultrasound image and obtain the labels annotated by medical experts, wherein the type of the labels annotated by medical experts is a binary image; Step S12: Automatically match the original thyroid ultrasound image with the labels annotated by the medical expert; Step S13: Create the original dataset and automatically divide the original dataset into multiple data subsets according to a preset ratio, including: training set, test set and validation set.

3. The method according to claim 2, characterized in that, In step S13, the specific rules for dividing the original dataset based on the acquired raw thyroid ultrasound images and labels annotated by medical experts include: Ensure that multiple raw thyroid ultrasound images of any subject from different scanning angles during the same examination, along with their corresponding matching labels annotated by medical experts, are in the same data subset; and Ensure that multiple raw thyroid ultrasound images obtained from multiple examinations of the same subject and their corresponding matching labels annotated by medical experts are in the same data subset.

4. The method according to claim 2, characterized in that, Step S3 includes the following sub-steps: Step S31: Automatically and randomly select one planning method from the constructed planning domain. S t The training set in step S13 is augmented to obtain an augmented dataset to be trained. Step S32: Train a segmentation sub-network using the obtained augmented dataset to be trained, and validate the trained segmentation sub-network using the validation set from step S13 to obtain the validation average intersection-union ratio. A t ; Step S33: Build the LSTM controller and primitive the LSTM controller, that is, primitive all values ​​to the range [-0.1, 0.1]; Step S34: Calculate the verification average crossover and union ratio obtained in step S32. A t As part of this incentive, and based on this incentive... A t And the correlation between this planning and the previous planning, using the survival of the fittest allocation rule, to train and update the LSTM controller; Repeat steps S31-S34 until the controller training is complete.

5. The method according to claim 4, characterized in that, The segmentation sub-network is a U-Net network. The LSTM controller includes a single-layer LSTM neural network with 100 hidden units and a Softmax classifier with 24 output predictions. The LSTM neural network describes the probability distribution of the next operation based on the operation of the previous step, and finally estimates the distribution of each operation.

6. The method according to claim 5, characterized in that, In step S34, the product of the relevance probabilities predicted by the 24 outputs of the Softmax classifier is used as the joint probability, and the relevance is measured by the ratio, thus obtaining the following loss function: ； In the formula, This represents the expected value of the probability distribution during planning. Indicating in training planning S t The joint probability after that, Indicating in training planning S t The joint probability before, Indicating in planning S t The intersection-union ratio (IoU) of the model with respect to the validation set is the mean. O The label represents the output of the segmentation subnetwork. M This represents the corresponding true label in the dataset. w Indicates the penalty weight. Information divergence is used to reflect the current planning. S t The gap between the current plan and the previous plan.

7. The method according to claim 1, characterized in that, Step S4 includes the following sub-steps: Step S41: Using the trained LSTM controller, evaluate each plan with a loss function, and automatically select the top 5 optimal plans from the planning domain to obtain a total of 20 pairs of combined methods, which constitute the optimal dataset augmentation scheme. Step S42: After sorting and concatenating the 20 pairs of combination methods in descending order of quality, the training set is augmented to obtain the final intelligently augmented thyroid segmentation dataset.

Images