Chromosome classification method

Abstract

The invention discloses a chromosome classification method, which belongs to the field of image processing, and comprises the following steps: correcting angles of all chromosomes, enabling the chromosomes to be vertical in images, unifying the sizes of all chromosome images after angle correction, dividing all chromosome images into a training set and a verification set, and constructing a chromosome polarity identification model. Identifying the chromosome polarity and correcting the chromosome polarity; constructing a chromosome category identification model to identify chromosome types on the chromosome image and corresponding confidence; according to the method, chromosomes which may be wrongly identified are detected, two sets including a candidate set and a deletion set are constructed, the candidate set is used for temporarily storing chromosomes which may be wrongly classified into other categories, the deletion set is used for recording categories with insufficient chromosomes, the categories of the chromosomes in the candidate set are corrected, and the chromosomes in the candidate set can be accurately identified within a short time. Therefore, the efficiency and the accuracy of existing karyotype analysis are greatly improved, and the workload of doctors is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a chromosome classification method. Background Art

[0002] In traditional chromosome karyotype analysis, doctors need to judge the type of chromosome based on the length of the chromosome, the ratio of the long and short arms, the location of the centromere, the color and number of chromosome bands. However, the number of chromosomes in a normal human is 46, and it takes a lot of time to manually identify them one by one. In addition, the accuracy of manual identification is easily affected by factors such as noise, image clarity, and the quality of chromosome specimens, and it is highly dependent on the personal experience of clinicians.

[0003] There are methods for automatically classifying chromosomes in the prior art, but the accuracy is not high. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, one of the purposes of the present invention is to provide a chromosome classification method that can complete the identification of a large number of chromosomes with high accuracy in a short time, greatly improve the efficiency and accuracy of existing karyotype analysis, and reduce the workload of doctors.

[0005] One of the purposes of the present invention is achieved by the following technical solution:

[0006] A chromosome classification method comprises the following steps:

[0007] S1: A large number of individual chromosome images are obtained from the chromosome metaphase image, and each image contains only a single chromosome;

[0008] S2: Preprocess the images containing only a single chromosome, correct the angles of all chromosomes so that the chromosomes appear vertical in the images, unify the sizes of all chromosome images after angle correction, divide all chromosome images into training sets and validation sets, and normalize all training set and validation set images;

[0009] S3: Construct a chromosome polarity recognition model to identify chromosome polarity and correct chromosome polarity;

[0010] S4: Construct a chromosome category recognition model to identify the chromosome type and the corresponding confidence on the chromosome image;

[0011] S5: Detect chromosomes that may be misidentified, and construct two sets: candidate set and missing set. The candidate set is used to temporarily store chromosomes that may be mistakenly classified into other categories, and the missing set is used to record categories with insufficient chromosome numbers and correct the categories of chromosomes in the candidate set.

[0012] Furthermore, in step S2, correcting the angle of the chromosome specifically includes the following steps:

[0013] S21: converting the original image I(x, y) into a binary image B(x, y) by using a threshold segmentation method, that is, setting pixels in the image that are greater than a preset threshold t to 1, and setting pixels that are less than or equal to t to zero;

[0014] S22: Calculate the four moments M of the binary image B(x,y) 00 =∑ x ∑ y x 0 y 0 B(x,y),M 20 =∑ x ∑ y x 2 y 0 B(x,y),M 02 =∑ x ∑ y x 0 y 2 B(x,y),M 11 =∑ x ∑ y x 1 y 1 B(x,y);

[0015] S23: Calculate the tilt angle θ of the non-zero region in the binary image B(x,y), Where:

[0016] S24: Rotate B(x, y) by θ and θ+90, and calculate the projection areas of the two in the perpendicular direction respectively, and use the angle corresponding to the smaller projection area as the angle θ′ used for correction;

[0017] S25: Rotate I(x,y) by θ′ to make the chromosomes in the image vertical.

[0018] Furthermore, in step S2, the sizes of all chromosome images after angle correction are unified as follows: the size of the chromosome image is expanded to 350×350 using a pixel value of 255. If the length of the chromosome is greater than 350, the chromosome is downsampled to 350, and all images are resampled to 256×256.

[0019] Furthermore, in step S2, all training set and validation set images are normalized by calculating the mean and variance of the chromosome region in the training set, and normalizing the current image based on the mean and variance obtained from the training set.

[0020] Furthermore, in step S3, the chromosome polarity is identified and corrected as follows: the image is input into the chromosome polarity recognition model to identify the polarity. If the prediction result is 1, it means that the long arm of the current chromosome is on top, and it is rotated 180 degrees so that the short arm is on top; if the prediction result is 0, it means that the short arm of the current chromosome is on top and no processing is performed.

[0021] Furthermore, in step S3, the chromosome polarity recognition model includes 13 3×3 convolutional layers, 5 maximum pooling layers and 3 fully connected layers, where the first convolutional layer and the last fully connected layer are the input and output layers of the model.

[0022] Further, in step S4, the chromosome category recognition model contains multiple parallel branch structures, which are used to process the original resolution The intermediate layer features of the image are obtained and the features of each resolution are fused multiple times.

[0023] Furthermore, in step S4, it also includes using the training set to optimize the weights of the convolutional layer and the fully connected layer in the chromosome polarity recognition model and the chromosome category recognition model. The optimizer is a stochastic gradient descent algorithm, the number of batches is 128, the initial learning rate is 0.0001, the momentum coefficient is 0.9, the weight decay coefficient is 0.00001, the loss function is cross entropy, and the number of training times is 100.

[0024] Further, in step S5, two sets of candidate set and missing set are constructed specifically as follows: when the number of a certain type of chromosomes in the recognition result is greater than 2, the chromosomes with a confidence less than a threshold value P are placed in the candidate set, and X and Y are regarded as the same type of chromosomes, that is, the 23rd type; when the number of a certain type of autosomes in the recognition result is less than 2, the number of missing chromosomes in this type is recorded in the missing set.

[0025] Furthermore, in step S5, the category of the chromosome in the candidate set is corrected, which specifically includes the following steps:

[0026] S51: When the category C′ corresponding to the second largest confidence of a chromosome appears in the missing set, the original recognition result C is replaced by C′; if there are multiple chromosomes in the candidate set whose categories are all C′, and the number is greater than the number of missing chromosomes n of this category, the confidences are sorted in descending order, and the first n chromosomes are selected to modify their categories, and the categories of the remaining chromosomes remain unchanged; repeat the above process until all chromosomes in the candidate set are processed;

[0027] S52: After executing step S51, if the candidate set is still not an empty set, and there is a case in which only one autosome of category C is missing in the missing set, and the autosome is not X and Y, the difference in length between the non-missing chromosome and each chromosome in the candidate set is calculated, and the category of the chromosome with the smallest difference is changed to C;

[0028] S53: When the candidate set is still not an empty set after step S52 is implemented, all possible arrangements of chromosomes in the candidate set are listed according to the remaining categories in the missing set, and the one with the largest confidence is selected;

[0029] S54: after executing step S53, if the candidate set is still not an empty set and the missing set is empty, the category of the chromosome in the candidate set is not modified;

[0030] S55: Compare the lengths of chromosomes in class 23. If one chromosome is no longer than the other, If the chromosome is shorter, the category is changed to Y; otherwise, it is considered X.

[0031] Compared with the prior art, the chromosome classification method of the present invention corrects the angles of all chromosomes so that the chromosomes are vertical in the image, unifies the sizes of all chromosome images after angle correction, divides all chromosome images into training sets and verification sets, and constructs a chromosome polarity recognition model to identify and correct chromosome polarity; constructs a chromosome category recognition model to identify the chromosome type and the corresponding confidence on the chromosome image; detects chromosomes that may be misidentified, and constructs two sets, a candidate set and a missing set. The candidate set is used to temporarily store chromosomes that may be mistakenly classified into other categories, and the missing set is used to record categories with insufficient chromosome numbers. The categories of chromosomes in the candidate set are corrected, and the recognition of large quantities of chromosomes can be completed in a short time with high accuracy, which greatly improves the efficiency and accuracy of existing karyotype analysis and reduces the workload of doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the chromosome classification method of the present invention;

[0033] Figure 2 are multiple original chromosome images;

[0034] Figure 3 These are multiple chromosome images after angle correction;

[0035] Figure 4 The classification result of the chromosome classification method of the present invention for female XX chromosomes;

[0036] Figure 5 This is the classification result of male XY chromosomes using the chromosome classification method of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may be another intermediate component through which it is fixed. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may be another intermediate component at the same time. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be another intermediate component at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0040] Figure 1 The present invention is a flow chart of the chromosome classification method, which comprises the following steps:

[0041] A chromosome classification method, characterized in that it comprises the following steps:

[0042] S1: A large number of individual chromosome images are obtained from the chromosome metaphase image, and each image contains only a single chromosome;

[0043] S2: Preprocess the images containing only a single chromosome, correct the angles of all chromosomes so that the chromosomes appear vertical in the images, unify the sizes of all chromosome images after angle correction, divide all chromosome images into training sets and validation sets, and normalize all training set and validation set images;

[0044] S3: Construct a chromosome polarity recognition model to identify chromosome polarity and correct chromosome polarity;

[0045] S4: Construct a chromosome category recognition model to identify the chromosome type and the corresponding confidence on the chromosome image;

[0046] S5: Detect chromosomes that may be misidentified, and construct two sets: candidate set and missing set. The candidate set is used to temporarily store chromosomes that may be mistakenly classified into other categories, and the missing set is used to record categories with insufficient chromosome numbers and correct the categories of chromosomes in the candidate set.

[0047] Specifically, in step S2, the purpose of correcting the angle of the chromosome so that the chromosome is vertical in the image is to facilitate the processing of the subsequent algorithm. Let a single chromosome image be I(x,y), where x and y are the horizontal and vertical coordinates of the image.

[0048] The specific steps include:

[0049] S21: converting the original image I(x, y) into a binary image B(x, y) by using a threshold segmentation method, that is, setting pixels in the image that are greater than a preset threshold t to 1, and setting pixels that are less than or equal to t to zero;

[0050] S22: Calculate the four moments M of the binary image B(x,y) 00 =∑ x ∑ y x 0 y 0 B(x,y),M 20 =∑ x ∑ y x 2 y 0 B(x,y),M 02 =∑ x ∑ y x 0 y 2 B(x,y),M 11 =∑ x ∑ y x 1 y 1 B(x,y);

[0051] S23: Calculate the tilt angle θ of the non-zero region in the binary image B(x,y), Where:

[0052] S24: Rotate B(x, y) by θ and θ+90, and calculate the projection areas of the two in the perpendicular direction respectively, and use the angle corresponding to the smaller projection area as the angle θ′ used for correction;

[0053] S25: Rotate I(x,y) by θ′ to make the chromosomes in the image vertical.

[0054] like Figure 2 As shown in the figure, there are multiple original chromosome images, in which the angles of chromosomes are randomly distributed; after correction, multiple chromosome images are as follows Figure 3 As shown, the chromosomes appear vertical in the image.

[0055] In step S2, the sizes of all chromosome images after angle correction are unified as follows: the size of the chromosome image is expanded to 350×350 using a pixel value of 255. If the length of the chromosome is greater than 350, the chromosome is downsampled to 350, and all images are resampled to 256×256.

[0056] All chromosome images are divided into training set and validation set. The training set is used to train the chromosome polarity recognition model and the chromosome category recognition model, and the validation set is used to test the performance of the chromosome polarity recognition model and the chromosome category recognition model. The mean and variance of the chromosome region in the training set are calculated, and all training set and validation set images are normalized.

[0057] In step S3, a chromosome polarity recognition model based on the classic deep learning model VGG16 is constructed, that is, after inputting a chromosome image, the model can identify whether the chromosome is short-armed or long-armed. The VGG16 model contains 13 3×3 convolutional layers, 5 maximum pooling layers, and 3 fully connected layers, of which the first convolutional layer and the last fully connected layer are the input and output layers of the model. The image is input into the chromosome polarity recognition model to identify the polarity. If the prediction result is 1, it means that the long arm of the current chromosome is on top, and it is rotated 180 degrees to make its short arm on top; if the prediction result is 0, it means that the short arm of the current chromosome is on top and no processing is performed.

[0058] In step S4, a chromosome category recognition model based on the deep learning model HRNet is constructed. The model is characterized by having multiple parallel branch structures, which are used to process the original resolution. The advantages of this structure are that it can alleviate the loss of intermediate layer details caused by excessive downsampling without increasing the computational complexity of the model too much, and will not affect the receptive field of the deep convolutional layer. It is very suitable for processing targets such as chromosomes with band details as the main features. In this application, the model has a total of 24 outputs, corresponding to 24 types of chromosomes, 1-22, X, and Y.

[0059] The training set is used to optimize the weights of the convolutional layer and the fully connected layer in the chromosome polarity recognition model and the chromosome category recognition model. The optimizer is the stochastic gradient descent algorithm, the batch number is 128, the initial learning rate is 0.0001, the momentum coefficient is 0.9, the weight decay coefficient is 0.00001, the loss function is cross entropy, and the number of training times is 100. The initial weights of the model are all obtained based on imagenet training.

[0060] In step S5, two sets, candidate set and missing set, are constructed as follows: when the number of a certain type of chromosome in the recognition result is greater than 2, the chromosomes with a confidence level less than a threshold value P are placed in the candidate set, and X and Y are regarded as the same type of chromosome, i.e., the 23rd type; when the number of a certain type of autosome in the recognition result is less than 2, the number of missing chromosomes in this type is recorded in the missing set.

[0061] In step S5, the category of the chromosome in the candidate set is modified, which specifically includes the following steps:

[0062] S51: When the category C′ corresponding to the second largest confidence of a chromosome appears in the missing set, the original recognition result C is replaced by C′; if there are multiple chromosomes in the candidate set whose categories are all C′, and the number is greater than the number of missing chromosomes n of this category, the confidences are sorted in descending order, and the first n chromosomes are selected to modify their categories, and the categories of the remaining chromosomes remain unchanged; repeat the above process until all chromosomes in the candidate set are processed;

[0063] S52: After executing step S51, if the candidate set is still not an empty set, and there is a case in which only one autosome of category C is missing in the missing set, and the autosome is not X and Y, the difference in length between the non-missing chromosome and each chromosome in the candidate set is calculated, and the category of the chromosome with the smallest difference is changed to C;

[0064] S53: When the candidate set is still not an empty set after step S52 is implemented, all possible arrangements of chromosomes in the candidate set are listed according to the remaining categories in the missing set, and the one with the largest confidence is selected;

[0065] S54: after executing step S53, if the candidate set is still not an empty set and the missing set is empty, the category of the chromosome in the candidate set is not modified;

[0066] S55: Compare the lengths of chromosomes in class 23. If one chromosome is no longer than the other, If the chromosome is shorter, the category is changed to Y; otherwise, it is considered X.

[0067] At this point, all post-processing steps are completed. After testing, the chromosome recognition accuracy of this application is 99.3%. Figure 4 as well as Figure 5 shown.

[0068] The present application provides a chromosome classification method, which can complete the identification of a large number of chromosomes with high accuracy in a short time, greatly improve the efficiency and accuracy of existing karyotype analysis, and avoid tedious and complicated manual analysis work.

[0069] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which are equivalent modifications and improvements made to the above embodiments based on the essential technology of the present invention, and all of them belong to the protection scope of the present invention.

Claims

1. A chromosome classification method, characterized in that: The following steps are involved: S1: A large number of individual chromosome images are obtained from the chromosome metaphase image, and each image contains only a single chromosome; S2: Preprocess the images containing only a single chromosome, correct the angles of all chromosomes so that the chromosomes appear vertical in the images, unify the sizes of all chromosome images after angle correction, divide all chromosome images into training sets and validation sets, and normalize all training set and validation set images; S3: Construct a chromosome polarity recognition model to identify chromosome polarity and correct chromosome polarity; S4: Construct a chromosome category recognition model to identify the chromosome type and the corresponding confidence on the chromosome image; S5: Detect chromosomes that may be misidentified, and construct two sets: candidate set and missing set. The candidate set is used to temporarily store chromosomes that may be mistakenly classified into other categories, and the missing set is used to record categories with insufficient chromosome numbers and correct the categories of chromosomes in the candidate set.

2. The chromosome classification method according to claim 1, characterized in that: In step S2, correcting the angle of the chromosome specifically includes the following steps: S21: converting the original image I(x, y) into a binary image B(x, y) by using a threshold segmentation method, that is, setting pixels in the image that are greater than a preset threshold t to 1, and setting pixels that are less than or equal to t to zero; S22: Calculate the four moments M of the binary image B(x,y) 00 =∑ x ∑ y x 0 y 0 B(x,y),M 20 =∑ x ∑ y x 2 y 0 B(x,y),M 02 =∑ x ∑ y x 0 y 2 B(x,y),M 11 =∑ x ∑ y x 1 y 1 B(x,y); S23: Calculate the tilt angle θ of the non-zero region in the binary image B(x,y), Where: S24: Rotate B(x, y) by θ and θ+90, and calculate the projection areas of the two in the perpendicular direction respectively, and use the angle corresponding to the smaller projection area as the angle θ′ used for correction; S25: Rotate I(x,y) by θ′ to make the chromosomes in the image vertical.

3. The chromosome classification method according to claim 1, characterized in that: In step S2, the sizes of all chromosome images after angle correction are unified as follows: the size of the chromosome image is expanded to 350×350 using a pixel value of 255. If the length of the chromosome is greater than 350, the chromosome is downsampled to 350, and all images are resampled to 256×256.

4. The chromosome classification method according to claim 1, characterized in that: In step S2, all training set and validation set images are normalized by calculating the mean and variance of the chromosome region in the training set, and normalizing the current image based on the mean and variance obtained from the training set.

5. The chromosome classification method according to claim 1, characterized in that: In step S3, the chromosome polarity is identified and corrected as follows: the image is input into the chromosome polarity recognition model to identify the polarity. If the prediction result is 1, it means that the long arm of the current chromosome is on top, and it is rotated 180 degrees so that the short arm is on top; if the prediction result is 0, it means that the short arm of the current chromosome is on top and no processing is performed.

6. The chromosome classification method according to claim 1, characterized in that: In step S3, the chromosome polarity recognition model contains 13 3×3 convolutional layers, 5 maximum pooling layers and 3 fully connected layers, where the first convolutional layer and the last fully connected layer are the input and output layers of the model.

7. The chromosome classification method according to claim 1, characterized in that: In step S4, the chromosome category recognition model contains multiple parallel branch structures, which are used to process the original resolution The intermediate layer features of the image are obtained and the features of each resolution are fused multiple times.

8. The chromosome classification method according to claim 1, characterized in that: In step S4, it also includes using the training set to optimize the weights of the convolutional layer and the fully connected layer in the chromosome polarity recognition model and the chromosome category recognition model. The optimizer is the stochastic gradient descent algorithm, the batch number is 128, the initial learning rate is 0.0001, the momentum coefficient is 0.9, the weight decay coefficient is 0.00001, the loss function is cross entropy, and the number of training times is 100.

9. The chromosome classification method according to claim 1, characterized in that: In step S5, two sets, candidate set and missing set, are constructed as follows: when the number of a certain type of chromosome in the recognition result is greater than 2, the chromosomes with a confidence level less than a threshold value P are placed in the candidate set, and X and Y are regarded as the same type of chromosome, i.e., the 23rd type; when the number of a certain type of autosome in the recognition result is less than 2, the number of missing chromosomes in this type is recorded in the missing set.

10. The chromosome classification method according to claim 1, characterized in that: In step S5, the category of the chromosome in the candidate set is modified, which specifically includes the following steps: S51: When the category C′ corresponding to the second largest confidence of a chromosome appears in the missing set, the original recognition result C is replaced by C′; if there are multiple chromosomes in the candidate set whose categories are all C′, and the number is greater than the number of missing chromosomes n of this category, the confidences are sorted in descending order, and the first n chromosomes are selected to modify their categories, and the categories of the remaining chromosomes remain unchanged; repeat the above process until all chromosomes in the candidate set are processed; S52: After executing step S51, if the candidate set is still not an empty set, and there is a case in which only one autosome of category C is missing in the missing set, and the autosome is not X and Y, the difference in length between the non-missing chromosome and each chromosome in the candidate set is calculated, and the category of the chromosome with the smallest difference is changed to C; S53: When the candidate set is still not an empty set after step S52 is implemented, all possible arrangements of chromosomes in the candidate set are listed according to the remaining categories in the missing set, and the one with the largest confidence is selected; S54: after executing step S53, if the candidate set is still not an empty set and the missing set is empty, the category of the chromosome in the candidate set is not modified; S55: Compare the lengths of chromosomes in class 23. If one chromosome is no longer than the other, If the chromosome is shorter, the category is changed to Y; otherwise, it is considered X.

Images