A malignant cell image data augmentation method based on gradient features

Abstract

The present application relates to a kind of malignant cell image data expansion method based on gradient feature, belong to medical image malignant cell identification field.First, a certain target detection model is trained using malignant cell dataset until convergence.Then using the model to load the model of training completion, all samples in training set are calculated using the weight of the image in the gradient of back propagation one by one.Then, the gradient value is sorted, the N th gradient value is taken out and is recorded by constructing threshold matrix with its corresponding two-dimensional coordinates.Second, these coordinate information is clustered using Kmeans algorithm, the maximum and minimum value of horizontal and vertical coordinates in each class is taken as the coordinates of the upper left corner and the lower right corner of the selected region of expansion image, and the selected region is expanded to save the context information of target, then the original image is cropped, and the cropped image is saved.Finally, the saved data is mixed with the original data, and the mixed data is used to train the weight of new model.The present application can effectively expand the real malignant cell image data, and save its target context information.

Description

Technical Field

[0001] This invention belongs to the field of malignant cell identification in medical images, and relates to a method for augmenting malignant cell image data based on gradient features. Background Technology

[0002] Data augmentation aims to address the problem of insufficient data by generating new sample data using existing sample data. Data augmentation is a crucial step in object detection, improving the generalization ability of object detectors and mitigating overfitting during training. Due to the unique characteristics of the medical field, such as patient privacy, ethical concerns, and sampling costs, available sample data is scarce. Therefore, solving the data problem in the medical image field is urgent. Medical image data processing methods can be categorized into two types: data augmentation based on traditional image processing methods and data augmentation based on deep learning methods. Traditional image processing-based data augmentation mainly involves geometric transformations and color space transformations, which may damage key structures of the target in the image, misleading the model to learn the correct features. Deep learning-based data augmentation primarily uses GAN networks to synthesize target images. While this method can increase sample data and alleviate uneven sample distribution, the generated images may deviate from real physiological data, thus requiring strict control over the quality of images generated by the GAN network. To augment more realistic medical image data, this patent proposes a gradient feature-based method for malignant cell image data augmentation. First, a target detection model is trained on a malignant cell dataset until convergence. Then, the trained model is loaded, and all samples in the training set are predicted using the weights, with the gradient of each image calculated during backpropagation. Next, the gradient values ​​are sorted, and the N% gradient value is extracted and its corresponding 2D coordinates are recorded. Then, these coordinates are clustered using the K-means algorithm, and the maximum and minimum values ​​of the x and y coordinates in each cluster are used as the coordinates of the top-left and bottom-right corners of the augmented image to crop the original image. The cropped image is then saved. Finally, the saved data is mixed with the original data, and the mixed data is used to train the weights of a new model. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method for augmenting malignant cell image data based on image gradient features for localization.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A method for augmenting malignant cell image data based on image gradient features includes the following steps: S1: Train the target detection model using the original malignant cell dataset until it converges; S2: Using the model weights trained in S1, predict the data in the training set one by one and calculate the image gradient; S3: Sort the image gradient information obtained in S2, take the top N% of gradients and record the corresponding two-dimensional coordinate information, cluster the coordinate information, and use each clustered region as a cropping region. Crop the selected region and save it. S4: Mix the newly saved data with the original data, and use the mixed data to train the weights of the new model. Furthermore, step S1 specifically includes the following steps: S11: Divide the original data into training set, validation set and test set in a ratio of 8:1:1; S12: The SGD optimizer was used for a total of 50,000 training steps. The formula for calculating the total number of rounds is as follows: Where Batch_Size is the number of training batches, Step is the set total step size of 50000, and Epoch is the total number of training rounds.

[0005] S13: Load the data into the dataset loader according to the Batch_Size size, and perform Mosaic and MixUp data augmentation. That is, there is a 50% probability of performing Mosaic data augmentation, and after Mosaic data augmentation, there is a 50% probability of performing MixUp data augmentation. S14: Train the model until it converges; Furthermore, step S2 specifically includes the following steps: S21: Load the images in the dataset one by one without performing any data processing operations; S22: Use the model trained in S14 to predict the image in S21, and calculate the image gradient information through backpropagation; Furthermore, step S3 specifically includes the following steps: S31: Flatten the gradient information calculated in step S22 and sort it in descending order; S32: Set parameter N, select the N% gradient value and record the two-dimensional coordinate information of each feature point; S33: Set parameter C to cluster the selected two-dimensional coordinate information into C classes using the K-means algorithm; S34: Select the maximum and minimum values ​​of the horizontal and vertical coordinates in each type of two-dimensional coordinate; S35: Use the minimum x-coordinate and minimum y-coordinate as the coordinates of the top left corner of the selected area, and use the maximum x-coordinate and maximum y-coordinate as the coordinates of the bottom right corner of the selected area; S36: Set the dilation factor EP, dilate the selected area one by one according to the coordinates of the selected area to EP times the selected area, crop the original image, and save the image of the selected area and the corresponding label; Furthermore, step S4 specifically includes the following steps: S41: Divide the cropped data into training and validation sets in an 8:2 ratio; S42: Add the data from the new training set and the new validation set to the original training set and the original validation set respectively for mixing; S43: Train the mixed data according to the steps in S1, and test it using the original test set.

[0006] The beneficial effects of this invention are as follows: the data augmentation method proposed in this patent can greatly increase the amount of sample data, which are real samples, and the contextual information of the target can be preserved through the inflation factor. By training the new data with the original data, the generalization ability of the model can be improved.

[0007] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0008] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the method for augmenting malignant cell image data based on gradient features according to the present invention. Figure 2 This is a schematic diagram of image gradient acquisition. Figure 3 This is a schematic diagram illustrating the conversion from gradient information to two-dimensional coordinates of a selected region. Detailed Implementation

[0009] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0010] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0011] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0012] Please see Figures 1-3 This describes a method for augmenting malignant cell image data based on gradient features, comprising the following steps: The specific steps for obtaining image gradients are as follows: The training images are loaded one by one using the dataset loader, and the Variable() function in the autograd module of torch is used to encapsulate the images so that they have the ability to automatically calculate derivatives. The image is then forward-propagated using the weights of the model that has been trained to convergence. The result of the forward propagation is transmitted to the loss function to calculate the loss. The loss is backpropagated to obtain the image gradient.

[0013] The method for converting gradient information into two-dimensional coordinate information of a selected region includes the following steps: The absolute value of the gradient information of the image is first taken, then the gradients of all channels are summed and flattened. Sort the flattened gradient information from largest to smallest; Take the N% gradient value and set it as the threshold to create a threshold map where all values ​​are set to the threshold.

[0014] The threshold map is compared pixel by pixel with the image gradient information, and the coordinates of each image gradient information point that is greater than the threshold map are recorded. Set the number of clusters C, and use the K-means algorithm to cluster the coordinates of all records from the previous step; For each clustered category, calculate the maximum and minimum values ​​of its x and y coordinates, and use the minimum x and y coordinates as the coordinates of the top left corner of the selected area, and the maximum x and y coordinates as the coordinates of the bottom right corner of the selected area. Set the expansion factor EP to expand the length and width of the selected area by EP times; Find the intersection of all labels in the original image with the selected region. If a label intersects with a region, then assign the label to that region. This label assignment has a one-to-many relationship. Save the selected area and labels to the corresponding directory.

[0015] The specific implementation details are as follows: 1. When calculating image gradients using pre-trained weights, resizing the image via letterboxing may result in black / gray borders. Newly generated data may crop and save these borders, potentially shifting the image distribution of the training data to the left / center during mixed training, thus affecting the mean and variance of each batch normalization. To address this, techniques such as Mosaic should be employed during mixed training to reduce the impact of data distribution on batch normalization. 2. When augmenting training data, the training and validation sets can be expanded and added to the original training and test sets, respectively. The test set should not be expanded and retrained, as this would cause the model to learn the features of the test set prematurely. 3. When augmenting data, 3-5 clusters are generally optimal. Too many clusters will result in excessively small expanded image sizes, further exacerbating image feature shift. When dilating a selected region, the dilation coefficient should be at least doubled to preserve the target's contextual information.

[0016] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for augmenting malignant cell image data based on gradient features, characterized in that: Includes the following steps: S1: Train the target detection model using the original malignant cell dataset until it converges; S2: Using the model weights trained in S1, predict the data in the training set one by one and calculate the image gradient; S3: Sort the image gradient information obtained in S2, take the N% gradient value and record the corresponding two-dimensional coordinate information, cluster the coordinate information, and use each clustered region as a cropping region. Crop the selected region and save it. S4: Mix the newly saved data with the original data, and use the mixed data to train the weights of the new model.

2. The training method according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Divide the original data into training set, validation set and test set in a ratio of 8:1:1; S12: The SGD optimizer was used for a total of 50,000 training steps. The formula for calculating the total number of rounds is as follows: Where Batch_Size is the number of training samples per batch, Step is the set total step size of 50,000, and Epoch is the total number of training rounds. S13: Load the data into the dataset loader according to the size of Batch_Size, and perform Mosaic and MixUp data augmentation. That is, there is a 50% probability of performing Mosaic data augmentation, and after Mosaic data augmentation, there is a 50% probability of performing MixUp data augmentation. S14: Train the model until it converges.

3. The image gradient calculation method according to claim 2, characterized in that: Step S2 specifically includes the following steps: S21: Load the images in the dataset one by one without performing any data processing operations; S22: Use the model trained in S14 to predict the image in S21 and calculate the image gradient information through backpropagation.

4. The data augmentation method according to claims 3, 4 and 5, characterized in that: Step S3 specifically includes the following steps: S31: Flatten the gradient information calculated in step S22 and sort it in descending order; S32: Set parameter N, select the top N% of gradients and record the two-dimensional coordinate information of each feature point; S33: Set parameter C to cluster the selected two-dimensional coordinate information into C classes using the K-means algorithm; S34: Select the maximum and minimum values ​​of the horizontal and vertical coordinates in each type of two-dimensional coordinate; S35: Use the minimum x-coordinate and minimum y-coordinate as the coordinates of the top left corner of the selected area, and use the maximum x-coordinate and maximum y-coordinate as the coordinates of the bottom right corner of the selected area; S36: Set the dilation factor EP, dilate the selected area one by one according to the coordinates of the selected area to EP times the selected area, crop the original image, and save the image of the selected area and the corresponding label.

5. The data fusion training method according to claim 6, characterized in that: Step S4 specifically includes the following steps: S41: Divide the cropped data into training and validation sets in an 8:2 ratio; S42: Add the data from the new training set and the new validation set to the original training set and the original validation set respectively for mixing; S43: Train the mixed data according to the steps in S1, and test it using the original test set.

Images