Chromosome image anomaly identification method based on deep learning

Abstract

The invention discloses a deep learning-based chromosome image anomaly recognition method. The method comprises the following steps of chromosome data acquisition, image preprocessing, data enhancement, data expansion, chromosome detection network construction and training, chromosome classification network construction and training, and model reasoning. According to the method, the accuracy and robustness of chromosome detection and anomaly recognition can be effectively improved, the problem of class imbalance is relieved, the auxiliary diagnosis efficiency and reliability are improved, meanwhile, the complexity of each stage is reduced, the detection precision is improved, dependence on computing resources is reduced, and the overall performance is optimized.

Description

Technical Field

[0001] This invention pertains to image processing methods in the field of computer vision, specifically relating to a method for identifying chromosome image anomalies based on deep learning. Background Technology

[0002] In clinical genetics, the detection of abnormal chromosomes is a crucial and complex task. Traditional chromosome analysis methods typically rely on manual observation and image analysis, which requires a high level of expertise and is susceptible to human error, leading to inefficiency and the risk of misdiagnosis. However, image processing techniques, especially those utilizing artificial intelligence (AI), can significantly improve the efficiency and accuracy of chromosomal abnormality detection. Current AI technologies, particularly deep learning, are widely applied in image analysis, especially in chromosomal abnormality detection. AI models can automatically analyze chromosome images, quickly identify abnormal chromosomes, and assist doctors in making accurate diagnoses. This automated detection method not only reduces the workload of manual analysis but also significantly lowers the misdiagnosis rate and improves diagnostic accuracy.

[0003] However, existing technologies often encounter the problem of "imbalanced data" when processing chromosome images. Normal chromosome samples are typically far more numerous than abnormal chromosome samples. This leads to overfitting of object detection models to normal chromosomes, resulting in poor performance in detecting abnormal chromosomes. Furthermore, object detection methods struggle to maintain good generalization ability when faced with imbalanced data distributions. Therefore, relying solely on traditional object detection methods is often insufficient to effectively address this issue. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a deep learning-based method for identifying chromosome image anomalies, comprising the following steps: S1. Collect chromosome data and construct datasets: Collect an image of a complete chromosome set within a cell, label and crop individual chromosomes, and construct detection and classification datasets respectively; S2. Preprocessing: Perform uniform preprocessing operations on the chromosome set and the extracted single chromosome images respectively, adjust the image resolution to the standard size required for model training, and normalize the pixel values ​​to eliminate image differences caused by different acquisition conditions. S3. Data Augmentation: During the detection model training phase, mosaic enhancement, random rotation enhancement, and occlusion enhancement are applied to chromosome images to enable the model to better learn the spatial distribution features of chromosomes under complex backgrounds and dense arrangement conditions; during the classification model training phase, scaling enhancement, rotation enhancement, and occlusion enhancement are applied to cropped single chromosome images to improve the model's ability to distinguish chromosome morphology under different scales, poses, and local missing conditions. S4. Data augmentation: For scarce classes with a small number of samples, low-rank adaptation is used to fine-tune the diffusion generation model and generate synthetic samples that are highly consistent with the real chromosome images in terms of structure and texture, thereby expanding the scale of minority class data and mitigating the adverse effects of class imbalance on the training of classification models. S5. Construct and train a chromosome detection network: Use a target detection model to train the preprocessed chromosome image so that it can extract and detect chromosome features: The backbone network encodes the original image in multiple levels to generate feature maps with different scales. Then, the feature interaction structure fuses the multi-scale information, and the detection head completes the position regression and category prediction of the chromosome target to realize the localization of the coarse chromosome region. S6. Construct and train a chromosome classification network: Use the classification model to train the preprocessed chromosome classification dataset so that it can further extract features and classify chromosomes. Extract high-level semantic features through the backbone network to determine whether they are abnormal. S7. Model inference: After feature extraction from two networks, the inference result is obtained.

[0005] Optionally, step S1 specifically includes the following steps: S11. Obtain the unlabeled dataset, and manually label the data using a self-built labeling platform. The labeling content includes the coordinates of the top left corner and bottom right corner of each chromosome, as well as the label of each chromosome, to construct the detection dataset; S12. Extract the images within the coordinate region using the labeled information, then cluster them according to different categories, and sample based on chromosome characteristics to construct a classification dataset.

[0006] Optionally, step S2 specifically includes the following steps: S21. For the detection dataset used by the target detection model, perform unified preprocessing on complete microscopic images containing multiple chromosomal targets; S22. For the classification dataset used by the classification model, perform independent preprocessing on the single chromosome sub-images obtained by cropping the output results of the detection model.

[0007] Optionally, step S3 specifically includes the following steps: S31. During the training of the detection model, the entire chromosome microscopic image is used as input, and multiple data augmentation methods are randomly selected and combined for processing in each training iteration; S32. During the classification model training phase, the cropped single chromosome image is used as the input sample, and one or more image enhancement operations are randomly applied to each sample during the training process.

[0008] Optionally, step S4 specifically includes the following steps: S41. Compression and noise addition: The image in the pixel space is compressed into a low-dimensional latent representation, which is then entered into the latent space. In the latent space, noise is gradually added to the latent representation according to a predetermined rule. S42. Decoding and Denoising: The highly noisy latent representation is fed into the denoising network to process the noisy latent representation. The denoised latent representation is then fed into the decoder to finally generate the image. S43. Low-rank adaptive fine-tuning: Freeze most of the parameters of the original denoising network, update only the parameters of some layers, and learn the low-rank matrix to adjust the parameters of the relevant modules of the denoising network.

[0009] Optionally, step S5 specifically includes the following steps: S51. Construct an optimized image processing backbone network, perform layer-by-layer convolutional coding on the input image, and extract multi-level features including edge information, morphological structure and texture distribution; S52. In the feature fusion stage, a feature pyramid network structure is used to fuse feature maps from different levels through top-down and horizontal connections, so as to effectively integrate high-level semantic information with low-level detailed information. S53. In the target localization and category discrimination stage, a decoupled detection head structure is adopted to model and optimize the bounding box regression task and the positive and negative sample classification task separately, so as to reduce mutual interference between tasks and improve training stability and convergence efficiency.

[0010] Optionally, step S6 specifically includes the following steps: S61. A multi-stage feature extraction structure is adopted, with different stages corresponding to feature representations at different resolutions; S62. A segmented feature embedding method is adopted, which divides the input chromosome image into multiple local image blocks through small-sized convolutional kernels, and maps each image block to a corresponding feature vector, which serves as the input of subsequent network modules; S63. Introduce depthwise separable convolutional structures to separate spatial convolution from channel convolution, reducing the number of parameters and computational cost while maintaining sensitivity to fine-grained structural features; S64. Introduce a larger convolutional kernel structure to expand the effective receptive field and achieve more accurate fine-grained classification.

[0011] Optionally, step S7 specifically includes the following steps: S71. Extract chromosome features from the image using an object detection network, obtain the coordinates of a single chromosome using the features, and then crop it. S72. Extract individual chromosome features using a classification network, and determine the chromosome type based on these features; S73. Organize the results and return them.

[0012] Beneficial Effects: This application employs data augmentation techniques, particularly by using a diffusion model to augment a small number of abnormal chromosome samples, thereby improving data distribution and achieving a balanced dataset. The diffusion model, by generating realistic synthetic images, effectively expands a small number of abnormal chromosome samples, enhancing the model's training data and enabling the model to better learn the characteristics of abnormal chromosomes, thus improving its detection accuracy. Secondly, this patent constructs a dataset suitable for chromosome classification based on the morphological characteristics of chromosomes, improving the model's accuracy. Furthermore, this patent employs a two-stage detection method: first, chromosome images are detected and cropped; then, the cropped chromosomes are fed into a classification network for anomaly identification. This strategy decomposes the abnormal chromosome detection task into two stages, reducing the complexity of each stage, improving detection accuracy, while simultaneously reducing reliance on computational resources and optimizing overall performance. Attached Figure Description

[0013] Figure 1 This is a flowchart of the present invention.

[0014] Figure 2 This is a flowchart of the reasoning process.

[0015] Figure 3 To detect the heatmap of the image after network processing.

[0016] Figure 4 These are image feature maps at different stages after processing by the classification network. Detailed Implementation

[0017] like Figures 1-2 As shown, this invention proposes a method for identifying chromosome image anomalies based on deep learning, comprising the following steps: S1. Collect chromosome data and construct datasets: Collect an image of a complete chromosome set within a cell, label and crop individual chromosomes, and construct detection and classification datasets respectively; S2. Preprocessing: Perform uniform preprocessing operations on the chromosome set and the extracted single chromosome images respectively, adjust the image resolution to the standard size required for model training, and normalize the pixel values ​​to eliminate image differences caused by different acquisition conditions. S3. Data Augmentation: During the detection model training phase, mosaic enhancement, random rotation enhancement, and occlusion enhancement are applied to chromosome images to enable the model to better learn the spatial distribution features of chromosomes under complex backgrounds and dense arrangement conditions; during the classification model training phase, scaling enhancement, rotation enhancement, and occlusion enhancement are applied to cropped single chromosome images to improve the model's ability to distinguish chromosome morphology under different scales, poses, and local missing conditions. S4. Data augmentation: For scarce classes with a small number of samples, low-rank adaptation is used to fine-tune the diffusion generation model and generate synthetic samples that are highly consistent with the real chromosome images in terms of structure and texture, thereby expanding the scale of minority class data and mitigating the adverse effects of class imbalance on the training of classification models. S5. Construct and train a chromosome detection network: Use a target detection model to train the preprocessed chromosome image so that it can extract and detect chromosome features: The backbone network encodes the original image in multiple levels to generate feature maps with different scales. Then, the feature interaction structure fuses the multi-scale information, and the detection head completes the position regression and category prediction of the chromosome target to realize the localization of the coarse chromosome region. S6. Construct and train a chromosome classification network: Use the classification model to train the preprocessed chromosome classification dataset so that it can further extract features and classify chromosomes. Extract high-level semantic features through the backbone network to determine whether they are abnormal. S7. Model inference: After feature extraction from two networks, the inference result is obtained.

[0018] Optionally, step S1 specifically includes the following steps: S11. Obtain the unlabeled dataset, and manually label the data using a self-built labeling platform. The labeling content includes the coordinates of the top left corner and bottom right corner of each chromosome, as well as the label of each chromosome, to construct the detection dataset; S12. Extract the images within the coordinate region using the labeled information, then cluster them according to different categories, and sample based on chromosome characteristics to construct a classification dataset.

[0019] Optionally, step S2 specifically includes the following steps: S21. For the detection dataset used by the target detection model, perform unified preprocessing on complete microscopic images containing multiple chromosomal targets; S22. For the classification dataset used by the classification model, perform independent preprocessing on the single chromosome sub-images obtained by cropping the output results of the detection model.

[0020] Optionally, step S3 specifically includes the following steps: S31. During the training of the detection model, the entire chromosome microscopic image is used as input, and multiple data augmentation methods are randomly selected and combined for processing in each training iteration; S32. During the classification model training phase, the cropped single chromosome image is used as the input sample, and one or more image enhancement operations are randomly applied to each sample during the training process.

[0021] Optionally, step S4 specifically includes the following steps: S41. Compression and noise addition: The image in the pixel space is compressed into a low-dimensional latent representation, which is then entered into the latent space. In the latent space, noise is gradually added to the latent representation according to a predetermined rule. S42. Decoding and Denoising: The highly noisy latent representation is fed into the denoising network to process the noisy latent representation. The denoised latent representation is then fed into the decoder to finally generate the image. S43. Low-rank adaptive fine-tuning: Freeze most of the parameters of the original denoising network, update only the parameters of some layers, and learn the low-rank matrix to adjust the parameters of the relevant modules of the denoising network.

[0022] Optionally, step S5 specifically includes the following steps: S51. Construct an optimized image processing backbone network, perform layer-by-layer convolutional coding on the input image, and extract multi-level features including edge information, morphological structure and texture distribution; S52. In the feature fusion stage, a feature pyramid network structure is used to fuse feature maps from different levels through top-down and horizontal connections, so as to effectively integrate high-level semantic information with low-level detailed information. S53. In the target localization and category discrimination stage, a decoupled detection head structure is adopted to model and optimize the bounding box regression task and the positive and negative sample classification task separately, so as to reduce mutual interference between tasks and improve training stability and convergence efficiency.

[0023] Optionally, step S6 specifically includes the following steps: S61. A multi-stage feature extraction structure is adopted, with different stages corresponding to feature representations at different resolutions; S62. A segmented feature embedding method is adopted, which divides the input chromosome image into multiple local image blocks through small-sized convolutional kernels, and maps each image block to a corresponding feature vector, which serves as the input of subsequent network modules; S63. Introduce depthwise separable convolutional structures to separate spatial convolution from channel convolution, reducing the number of parameters and computational cost while maintaining sensitivity to fine-grained structural features; S64. Introduce a larger convolutional kernel structure to expand the effective receptive field and achieve more accurate fine-grained classification.

[0024] Optionally, step S7 specifically includes the following steps: S71. Extract chromosome features from the image using an object detection network, obtain the coordinates of a single chromosome using the features, and then crop it. S72. Extract individual chromosome features using a classification network, and determine the chromosome type based on these features; S73. Organize the results and return them.

[0025] Image heatmap after processing by the trained detection network, as shown below Figure 3 As shown: the darker the color, the more the model focuses on that area. It can be seen that the detection model, after training, can effectively locate the regions where chromosomes appear.

[0026] Image feature maps at different stages after processing by the trained classification network are as follows: Figure 4 As shown: the left side represents positive sample chromosomes, and the right side represents negative sample chromosomes. The vertical axis represents image features at different stages, and the horizontal axis represents image features in different channels. As the model progresses through more stages, the chromosome features become increasingly abstract, and the model focuses more on features that are difficult to detect with the naked eye.

[0027] Example 1 During the training phase, complete chromosome images were first acquired using a hospital's scanning system, and the positional coordinates and category labels of each chromosome were manually labeled to construct a chromosome detection dataset. Subsequently, for the classification dataset, chromosomes were cropped and sampled to ensure the quality and representativeness of each sample. In the data augmentation process, to improve the model's robustness, during the training phase of the detection model, random rotation, mosaic enhancement, and occlusion enhancement were applied to the chromosome images, enabling the model to learn the spatial distribution characteristics of chromosomes in complex backgrounds and dense arrangements. During the training phase of the classification model, scaling and rotation enhancement operations were performed on individual chromosome images to improve the model's ability to discriminate chromosome morphology under different scales, poses, and local missing data. To address the problem of imbalanced data samples, a low-rank adaptive fine-tuning method was used to fine-tune the diffusion generation model and generate high-quality synthetic samples, thereby expanding the minority class data and mitigating the negative impact of class imbalance on the training of the classification model. Next, the image data undergoes unified preprocessing. For training the detection model, the image resolution is adjusted and pixel values ​​are normalized to eliminate image differences caused by different acquisition conditions. For the classification dataset, the cropped individual chromosome images are preprocessed independently to ensure image consistency.

[0028] In the inference phase, the trained chromosome detection and classification model performs actual anomaly identification. First, the object detection network extracts chromosome features from the image, locates the coordinates of each chromosome, and extracts individual chromosome images. Next, these cropped chromosome images are input into the classification network. The model extracts image features to determine whether each chromosome has an abnormality. Specifically, the object detection network accurately regresses the location of the chromosome target and predicts its category through multi-level feature extraction and multi-scale information fusion; the classification network performs high-level semantic feature analysis on each individual chromosome, outputting whether it is an abnormal chromosome. Finally, the detection and classification results are processed, returning the coordinates, category, and presence of an abnormality for each chromosome, assisting clinicians in making accurate diagnoses. Through this complete detection and inference process, the model achieves efficient and accurate chromosome anomaly identification, providing important support for subsequent medical applications.

[0029] The above embodiments are merely preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. Those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A method for identifying chromosome image anomalies based on deep learning, characterized in that, Includes the following steps: S1. Collect chromosome data and construct datasets: Collect an image of a complete chromosome set within a cell, label and crop individual chromosomes, and construct detection and classification datasets respectively; S2. Preprocessing: Perform uniform preprocessing operations on the chromosome set and the extracted single chromosome images respectively, adjust the image resolution to the standard size required for model training, and normalize the pixel values ​​to eliminate image differences caused by different acquisition conditions. S3. Data Augmentation: During the detection model training phase, mosaic enhancement, random rotation enhancement, and occlusion enhancement are applied to chromosome images to enable the model to better learn the spatial distribution features of chromosomes under complex backgrounds and dense arrangement conditions; during the classification model training phase, scaling enhancement, rotation enhancement, and occlusion enhancement are applied to cropped single chromosome images to improve the model's ability to distinguish chromosome morphology under different scales, poses, and local missing conditions. S4. Data augmentation: For scarce classes with a small number of samples, low-rank adaptation is used to fine-tune the diffusion generation model and generate synthetic samples that are highly consistent with the real chromosome images in terms of structure and texture, thereby expanding the scale of minority class data and mitigating the adverse effects of class imbalance on the training of classification models. S5. Construct and train a chromosome detection network: Use a target detection model to train the preprocessed chromosome image so that it can extract and detect chromosome features: The backbone network encodes the original image in multiple levels to generate feature maps with different scales. Then, the feature interaction structure fuses the multi-scale information, and the detection head completes the position regression and category prediction of the chromosome target to realize the localization of the coarse chromosome region. S6. Construct and train a chromosome classification network: Use the classification model to train the preprocessed chromosome classification dataset so that it can further extract features and classify chromosomes. Extract high-level semantic features through the backbone network to determine whether they are abnormal. S7. Model inference: After feature extraction from two networks, the inference result is obtained.

2. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Obtain the unlabeled dataset, and manually label the data using a self-built labeling platform. The labeling content includes the coordinates of the top left corner and bottom right corner of each chromosome, as well as the label of each chromosome, to construct the detection dataset; S12. Extract the images within the coordinate region using the labeled information, then cluster them according to different categories, and sample based on chromosome characteristics to construct a classification dataset.

3. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. For the detection dataset used by the target detection model, perform unified preprocessing on complete microscopic images containing multiple chromosomal targets; S22. For the classification dataset used by the classification model, perform independent preprocessing on the single chromosome sub-images obtained by cropping the output results of the detection model.

4. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31. During the training of the detection model, the entire chromosome microscopic image is used as input, and multiple data augmentation methods are randomly selected and combined for processing in each training iteration; S32. During the classification model training phase, the cropped single chromosome image is used as the input sample, and one or more image enhancement operations are randomly applied to each sample during the training process.

5. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S4 specifically includes the following steps: S41. Compression and noise addition: The image in the pixel space is compressed into a low-dimensional latent representation, which is then entered into the latent space. In the latent space, noise is gradually added to the latent representation according to a predetermined rule. S42. Decoding and Denoising: The highly noisy latent representation is fed into the denoising network to process the noisy latent representation. The denoised latent representation is then fed into the decoder to finally generate the image. S43. Low-rank adaptive fine-tuning: Freeze most of the parameters of the original denoising network, update only the parameters of some layers, and learn the low-rank matrix to adjust the parameters of the relevant modules of the denoising network.

6. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: S51. Construct an optimized image processing backbone network, perform layer-by-layer convolutional coding on the input image, and extract multi-level features including edge information, morphological structure and texture distribution; S52. In the feature fusion stage, a feature pyramid network structure is used to fuse feature maps from different levels through top-down and horizontal connections, so as to effectively integrate high-level semantic information with low-level detailed information. S53. In the target localization and category discrimination stage, a decoupled detection head structure is adopted to model and optimize the bounding box regression task and the positive and negative sample classification task separately, so as to reduce mutual interference between tasks and improve training stability and convergence efficiency.

7. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S6 specifically includes the following steps: S61. A multi-stage feature extraction structure is adopted, with different stages corresponding to feature representations at different resolutions; S62. A segmented feature embedding method is adopted, which divides the input chromosome image into multiple local image blocks through small-sized convolutional kernels, and maps each image block to a corresponding feature vector, which serves as the input of subsequent network modules; S63. Introduce depthwise separable convolutional structures to separate spatial convolution from channel convolution, reducing the number of parameters and computational cost while maintaining sensitivity to fine-grained structural features; S64. Introduce a larger convolutional kernel structure to expand the effective receptive field and achieve more accurate fine-grained classification.

8. The chromosome image anomaly recognition method based on deep learning according to claim 1, characterized in that: Step S7 specifically includes the following steps: S71. Extract chromosome features from the image using an object detection network, obtain the coordinates of a single chromosome using the features, and then crop it. S72. Extract individual chromosome features using a classification network, and determine the chromosome type based on these features; S73. Organize the results and return them.

Images