R-banded chromosome karyotype image recognition method and system based on grouping information assistance

Abstract

The present invention provides an R-banded chromosome karyotype image recognition method based on grouping information assistance, comprising: obtaining a chromosome image; normalizing the chromosome image; constructing a staining recognition model, wherein the chromosome recognition model uses chromosome length grouping and centromere position grouping as auxiliary tasks and chromosome recognition as a main task; training the chromosome recognition model; and using the trained chromosome recognition model to predict categories. The present invention effectively improves the analysis efficiency of chromosome karyotypes, accurately completes the automatic classification and sorting of chromosomes, has a simple process, and has great application prospects.

Description

Technical Field

[0001] The present invention relates to the field of chromosome karyotype image processing, and in particular to a method and system for recognizing R-banded chromosome karyotype images based on grouping information assistance. Background Art

[0002] Chromosome karyotype is an important tool for diagnosing various hematological malignancies and genetic diseases. Human somatic cells have a total of 23 pairs of chromosomes, including 22 pairs of autosomes (1-22) and a pair of sex chromosomes (XX for females, XY for males), totaling 24 types. The most significant features used by cytogeneticists to distinguish different types of chromosomes are the length of the chromosome, the location of the centromere, and the order of dark and light bands. Commonly used chromosome banding techniques include Giemsa (G)-, Reverse (R)-, and centromere (C)-banding. Compared with G-banding, R-banding chromosomes have poor morphology. However, some abnormalities occurring at the ends of chromosomes are of great significance for the diagnosis of leukemia and can be identified by the R-banding pattern. Therefore, finding an effective method for identifying R-banding chromosomes is of great significance and challenge.

[0003] There are few existing research works and patents on image recognition of R-banded chromosomes. Compared with G-banded chromosome images, R-banded chromosome images are more ambiguous and are prone to misjudgment on chromosomes of similar size and banding pattern, especially chromosomes 16-19, 5-12, and 18-Y. However, there is a hierarchical structure in the naming of the 24 categories of chromosomes (1-22, X, Y): they can be organized into 7 groups (ag) in descending order of length; from the perspective of centromere position, chromosomes can be divided into three subtypes (metacentromere, submetacentromere, and telocentromere). Summary of the invention

[0004] In view of the defects in the prior art, the object of the present invention is to provide a method and system for R-banded chromosome karyotype image recognition based on grouping information assistance.

[0005] According to one aspect of the present invention, a method for R-banded chromosome karyotype image recognition based on grouping information assistance is provided, comprising:

[0006] Obtain chromosome images;

[0007] Performing normalization processing on the chromosome image;

[0008] Constructing a chromosome recognition model, wherein the chromosome recognition model uses chromosome length grouping and centromere position grouping as auxiliary tasks and chromosome recognition as a main task;

[0009] Training the chromosome recognition model;

[0010] Use the trained chromosome recognition model to predict the category.

[0011] Preferably, the normalization process includes:

[0012] Calculating the length of each chromosome in the chromosome image, taking the length of the long side of its smallest enclosing rectangle as the chromosome length;

[0013] Taking the longest chromosome length among the 46 chromosomes as the standard length, a blank rectangle with the standard length as the side length is established;

[0014] Each chromosome is placed vertically in the center of the rectangle.

[0015] Preferably, the dye recognition model includes a backbone network, two grouping auxiliary networks and a feature fusion network.

[0016] Preferably, the backbone network uses a residual network structure, including 4 residual modules for image feature extraction

[0017] Preferably, the two auxiliary modules are respectively a chromosome length grouping task auxiliary module and a centromere position grouping auxiliary module;

[0018] Both auxiliary modules adopt an encoder-decoder structure, combined with a fully connected layer and a leap-layer connection; the third residual block output feature map X3 of the backbone network is introduced into the centromere position grouping auxiliary module; the fourth residual block output feature map X4 of the backbone network is introduced into the length grouping task auxiliary module.

[0019] Preferably, the feature fusion module fuses the output features of the backbone network and the two auxiliary modules for chromosome classification;

[0020] The fused features are expressed as:

[0021]

[0022] Dec L and Enc L Denote the decoder and encoder of the length grouping auxiliary module respectively, Dec P and Enc P Represent the decoder and encoder of the centromere position grouping auxiliary module, respectively.

[0023] Preferably, the loss function of the group-assisted dye recognition model includes a cross entropy loss function and a consistency loss function; which is expressed as: L = L recog +λ1(L groupL +L groupP )+λ2(L consistL +L consistP )

[0024] Among them, λ1,λ2 are hyper parameters, L recog Set as classification loss function; L groupL ,L groupP Set as the group loss function, L consistL , L consistP Set as consistency loss function;

[0025] The L recog The mathematical expression is

[0026] The L consistL and L consistP Using cosine similarity calculation, the cosine similarity of two feature vectors a and b is expressed as:

[0027]

[0028] Preferably, the number of feature channels of the two auxiliary modules after encoding is reduced to 256, the parameters of the corresponding fully connected layer are reduced, and feature learning is performed inside the encoder and decoder.

[0029] Preferably, the chromosomes are paired and arranged according to the identification results to obtain a final chromosome arrangement map.

[0030] According to a second aspect of the present invention, there is provided an R-banded chromosome karyotype image recognition system based on grouping information assistance, comprising:

[0031] The backbone network module extracts the basic features of chromosome images;

[0032] Auxiliary module for chromosome length grouping task, which extracts the length features of chromosomes;

[0033] A centromere position grouping auxiliary module, which extracts the centromere position features of chromosomes;

[0034] A feature fusion module fuses the basic features, length features and centromere position features, and outputs fused features for chromosome classification.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] A chromosome identification method based on grouping information assistance in an embodiment of the present invention effectively improves the analysis efficiency of chromosome karyotype, accurately completes the automatic classification and sorting of chromosomes, has a simple process, and has great application prospects;

[0037] Considering the relationship between chromosome identification and grouping, domain knowledge information is integrated into the end-to-end feature extraction process to guide the model to extract more critical feature information;

[0038] In addition, the chromosome length and centromere position grouping auxiliary module adopts an encoder-decoder model structure, which reduces the number of parameters of the fully connected layer used for auxiliary task classification;

[0039] Finally, the addition of the consistency loss function constrains the consistency of the grouping and recognition results, increasing the reliability of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0041] Figure 1 A flow chart of an R-banded chromosome karyotype image recognition method based on grouping information assistance in one embodiment of the present invention;

[0042] Figure 2 This is a diagram of the R-banded chromosome identification result of an application example of the present invention;

[0043] Figure 3 This is a microscopic image of the metaphase of an R-banded chromosome and a final karyotype diagram in an application example of the present invention. DETAILED DESCRIPTION

[0044] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0045] In view of the characteristics of R-banded chromosome karyotype images and the problems existing in existing recognition methods, the present invention provides an embodiment of an R-banded chromosome karyotype image recognition method based on grouping information assistance. This embodiment uses two auxiliary tasks, chromosome length grouping and centromere position grouping, to assist the feature extraction process of the main task of chromosome recognition to improve the performance of the final chromosome recognition. This embodiment can be used to assist the artificial chromosome karyotype analysis process to improve the efficiency of chromosome karyotype analysis and the stability of the results. Specifically, it includes:

[0046] S1, usually collect R-banded chromosome karyotype images of normal patients from clinical samples to obtain independent chromosome images;

[0047] S2, preprocessing: chromosome cutting, cleaning, labeling, chromosome size and position normalization, construction of training set and test set samples;

[0048] S3, establish a chromosome recognition model based on grouping assistance, build the network backbone, grouping assistance module, feature fusion module, and design the model loss function;

[0049] S4, randomly initialize model parameters and use training set samples for model training;

[0050] S5, use the trained model to test the samples in the test set to obtain the label information of each chromosome;

[0051] S6, pairing and arranging the chromosomes according to the recognition results to obtain the final arrangement diagram.

[0052] Since the size characteristics of chromosomes have an important influence on the chromosome recognition results, in order to retain the size information of chromosomes as completely as possible, the present invention provides a preferred embodiment for image normalization processing before recognition. Specifically, it includes:

[0053] S201, calculating the length of each chromosome in a case image, and taking the length of the long side of its smallest enclosing rectangle as the chromosome length;

[0054] S202, taking the longest chromosome length among the 46 chromosomes as the standard length, and establishing a blank rectangle with the standard length as the side length, so as to better preserve the chromosome length information;

[0055] S203, place each chromosome vertically in the center of the rectangle. Assume that the width and height of a single chromosome are w and h respectively, the length of the longest chromosome in this case and the side length of the normalized image are R, and the starting coordinates of the chromosome in the normalized image are:

[0056]

[0057] In order to improve the accuracy of chromosome recognition, the present invention provides a preferred embodiment for constructing a chromosome recognition model. The chromosome recognition model consists of a backbone network and two auxiliary modules. The backbone network uses a residual network structure and consists of 4 residual modules for extracting basic image features. The chromosome length grouping task auxiliary module and the centromere position grouping auxiliary module are used to extract chromosome length and centromere position-specific features, which correspond to the length groups and the specific types of the three subtypes, respectively, to assist the main task of chromosome recognition.

[0058] Furthermore, both auxiliary modules are encoder-decoder structures, combined with fully connected layers, for feature extraction and auxiliary task classification. Based on the different features of the chromosome length grouping task and the centromere position grouping task; the third residual block output feature map X3 of the backbone network is introduced to the centromere position grouping auxiliary module, and the last residual block output feature map X4 is introduced to the chromosome length grouping task auxiliary module. The two auxiliary modules adopt the encoder-decoder structure and add a jump-layer connection. The jump-layer connection is a structure inside the two auxiliary modules, which is used for feature extraction of auxiliary tasks.

[0059] Finally, the output features of the backbone network and the two auxiliary modules are fused for chromosome classification. The fused features can be expressed as:

[0060]

[0061] Dec L and Enc L Denote the decoder and encoder of the length grouping auxiliary module respectively, Dec P and Enc P Represent the decoder and encoder of the centromere position grouping auxiliary module, respectively.

[0062] In order to increase the reliability of the prediction results, the present invention provides a preferred embodiment. In this embodiment, the loss function of the group-assisted chromosome recognition model is composed of a group cross entropy loss function and a consistency loss function. The specific composition of the loss function is as follows:

[0063] L=L recog +λ1(L groupL +L groupP )+λ2(L consistL +L consistP )

[0064] Among them, λ1, λ2 are hyperparameters, and the classification loss function L recog ,L groupL ,L groupP Set to the cross entropy loss function, with the loss function L recog For example, its mathematical expression is:

[0065]

[0066] Among them, x is the predicted category distribution and y is the true category label;

[0067] Consistency Assessment consistL and L consistP Using cosine similarity calculation, the cosine similarity of two feature vectors a and b can be expressed as:

[0068]

[0069] Unlike multi-task learning methods, this network only focuses on the performance of the chromosome recognition task (the main task). During training, the model parameters are optimized by the chromosome recognition main task and two grouping auxiliary tasks. Two auxiliary task modules are used to extract task-specific features. In the auxiliary module, the number of feature channels after encoding is reduced to 256, thereby reducing the parameters of the auxiliary task fully connected layer, forcing feature learning to be performed inside the encoding and decoding modules, maximizing the effectiveness of the auxiliary task on the main task.

[0070] In order to verify the effect of the above embodiment, the present invention provides an application embodiment.

[0071] 1. Data collection: This embodiment uses 1735 clinically collected R-banded chromosome karyotype images of metaphase cell division, including 714 patients' data (423 males and 291 females). The experiment randomly selects 1312 samples for training and verification, and the remaining 423 samples for testing. All data have been annotated and checked by professional doctors.

[0072] 2. Verification content:

[0073] This embodiment verifies the R-banded chromosome karyotype image recognition method based on grouping information assistance in the above embodiment from both qualitative and quantitative perspectives.

[0074] 2.1 Qualitative Results

[0075] This embodiment is tested on the collected R-banded chromosome recognition test set. Although the R-banded chromosome bands are relatively fuzzy and some types of chromosomes are highly similar in size and band characteristics, the method in the above embodiment can still achieve relatively ideal results. Figure 2 As shown, the R-banded chromosome identification result diagram for qualitative verification in this embodiment.

[0076] 2.2 Quantitative analysis

[0077] After the model in this embodiment is trained on the training set, the model performance is evaluated on the test set, and precision and recall are selected as evaluation indicators. Overall, the model can achieve an accuracy of 96.87%. Further, the experimental results of each chromosome are statistically analyzed. As shown in Table 1, the accuracy of chromosomes 1-12 and X is higher, and incorrect recognition occurs more on smaller chromosomes.

[0078] Table 1. Recognition results statistics on 24 types of chromosomes 1-22, X, Y, precision (P), recall (R)

[0079]

[0080]

[0081] In order to verify the effectiveness of the R-banded chromosome karyotype image recognition method (GRC-Net) assisted by grouping information in the embodiment, the experimental results of this method and some mainstream classification methods were compared. In particular, the accuracy of chromosome recognition (Rec), the length grouping (Rec-L) and the centromere position grouping (Rec-P) results inferred from the recognition results were verified and compared. As shown in Table 2 below. From the results, the method described can achieve better performance in all results. Figure 3 Shown are the metaphase microscopic images of the R-banded chromosomes and the final karyotype diagram of this embodiment.

[0082] Table 2. Performance of different recognition methods on the R-banded chromosome recognition test set, where the best experimental results are in bold

[0083]

[0084] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A method for R-banded chromosome karyotype image recognition based on grouping information, characterized in that: include: Obtain chromosome images; Performing normalization processing on the chromosome image; Constructing a chromosome recognition model, wherein the chromosome recognition model uses chromosome length grouping and centromere position grouping as auxiliary tasks and chromosome recognition as a main task; Training the chromosome recognition model; Use the trained chromosome recognition model to predict the category; The dye recognition model includes a backbone network, two grouping auxiliary networks and a feature fusion network; The backbone network includes 4 residual modules for extracting image features; The two grouping auxiliary networks are respectively a chromosome length grouping task auxiliary module and a centromere position grouping auxiliary module; Both of the packet-assisted networks include an encoder, a decoder, a fully connected layer, and a jump-layer connection, wherein: The fully connected layer is used to classify the extracted features; The jump-layer connection is used to fuse low-layer and high-layer features; The third residual block output feature map X3 of the backbone network is introduced into the centromere position grouping auxiliary module; the fourth residual block output feature map X4 of the backbone network is introduced into the length grouping task auxiliary module; The feature fusion network fuses the output features of the backbone network and the two grouping auxiliary networks for chromosome classification; The fused features It is expressed as: Among them, Dec L and Enc L Denote the decoder and encoder of the length grouping auxiliary module respectively, Dec P and Enc P denote the decoder and encoder of the centromere position grouping auxiliary module, respectively; The loss function L of the dye recognition model includes a cross entropy loss function and a consistency loss function. L=L recog +λ1(L groupL +L groupP )+λ2(L consistL +L consistP ), Among them, λ1,λ2 are hyper parameters, L recog is the classification loss function; L groupL ,L groupP is the grouping loss function, L consistL , L consistP is the consistency loss function; in, x is the predicted category distribution, y is the true category label; The L consistL and L consistP Using cosine similarity calculation, the cosine similarity of two feature vectors a and b is expressed as:

2. The R-banded chromosome karyotype image recognition method based on grouping information assistance according to claim 1 is characterized in that: The normalization process comprises: Calculating the length of each chromosome in the chromosome image, taking the length of the long side of its smallest enclosing rectangle as the chromosome length; Taking the longest chromosome length among the 46 chromosomes as the standard length, a blank rectangle with the standard length as the side length is established; Each chromosome is placed vertically in the center of the rectangle.

3. The R-banded chromosome karyotype image recognition method based on grouping information assistance according to claim 1 is characterized in that: The number of feature channels of the two group-assisted networks after encoding is reduced to 256, the parameters of the corresponding fully connected layer are reduced, and feature learning is performed inside the encoder and decoder.

4. The R-banded chromosome karyotype image recognition method based on grouping information assistance according to any one of claims 1 to 3, characterized in that: The chromosomes are paired and arranged according to the identification results to obtain the final chromosome arrangement map.

5. A grouping information-assisted R-banded chromosome karyotype image recognition system, which implements the grouping information-assisted R-banded chromosome karyotype image recognition method according to claims 1-4, characterized in that: include: A backbone network module that extracts image features of chromosome images; Auxiliary module for chromosome length grouping task, which extracts the length features of chromosomes; A centromere position grouping auxiliary module, which extracts the centromere position features of chromosomes; A feature fusion module fuses the image features, length features and centromere position features, outputs fusion features for chromosome classification, and obtains recognition results.

Images