Chromosome image segmentation and recognition method and system based on deep learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGKEYIHE INTELLIGENT MEDICAL TECH (BEIJING) CO LTD
- Filing Date
- 2025-07-07
- Publication Date
- 2026-07-24
Smart Images
Figure CN120783337B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, and in particular to a chromosome image segmentation and recognition method and system based on deep learning. Background Technology
[0002] The U-Net network architecture is a convolutional neural network commonly used in medical image segmentation design. Its core features are a symmetrical encoder-decoder structure and skip connections. The encoder extracts high-level semantic features step by step through convolution and downsampling, while the decoder restores spatial resolution through upsampling and deconvolution. Skip connections directly concatenate the feature data output by each layer of the encoder with the corresponding layer of the decoder. However, because chromosome images often exhibit complex structures, irregular distributions, overlapping, and variable morphologies, these characteristics increase the difficulty of chromosome image segmentation. The standard U-Net network does not perform ideally for chromosome image segmentation, and the quality of segmentation directly affects the accuracy of chromosome analysis and recognition. Summary of the Invention
[0003] The purpose of this invention is to provide a chromosome image segmentation and recognition method and system based on deep learning, which has good segmentation and recognition effects on chromosome images.
[0004] To achieve the above objectives, in a first aspect, the present invention provides a chromosome image segmentation and recognition method based on deep learning, the method comprising:
[0005] The original chromosome image data is processed for target detection, and the output is chromosome image data with chromosome clusters marked.
[0006] By using dynamic convolution kernels, multi-level convolution and downsampling processing is performed on the output chromosome image data to obtain intermediate layer downsampling feature data and final layer downsampling feature data.
[0007] By combining a parameter-free attention mechanism, depthwise separable convolution and pixel recombination are performed on the final layer downsampled feature data to obtain deep feature optimized data.
[0008] The deep feature optimization data and the intermediate layer downsampled feature data are deconvolutionally processed and spliced with skip connections to output chromosome segmentation image data; wherein, the chromosome segmentation image data contains the segmented image data of a single chromosome.
[0009] The individual chromosome image data in the chromosome segmentation image data are classified and arranged to obtain the chromosome karyotype analysis results.
[0010] Optionally, the step of combining a parameter-free attention mechanism to perform depthwise separable convolution and pixel recombination on the final layer downsampled feature data to obtain deep feature optimized data includes:
[0011] The final layer downsampled feature data is processed by depth convolution and point-by-point convolution to obtain depth combined feature data.
[0012] The energy function based on the parameterless attention mechanism is used to optimize the deep combined feature data to obtain the optimized deep combined feature data.
[0013] The optimized deep feature data is subjected to pixel recombination processing to obtain deep feature optimization data.
[0014] Optionally, the final layer downsampled feature data is subjected to depthwise convolution and pointwise convolution to obtain depthwise combined feature data, including:
[0015] Batch normalization is performed on the final layer downsampled feature data to obtain batch normalized data;
[0016] The batch normalized data is processed using a non-linear activation function to obtain activation feature map data;
[0017] The activation feature map data is processed by depthwise convolution and pointwise convolution to obtain convolutional feature data.
[0018] Batch normalization is performed on the convolutional feature data to obtain deep combined feature data.
[0019] Optionally, the method for constructing the dynamic convolutional kernel includes:
[0020] Features of the input chromosome image data are extracted using standard convolution kernels;
[0021] The attention weights for each convolutional kernel are calculated using an attention mechanism;
[0022] Adjust the weights and size of the convolutional kernels based on attention weights;
[0023] The adjusted convolutional kernel is replaced with the standard convolutional kernel to obtain the dynamic convolutional kernel.
[0024] Secondly, the present invention provides a chromosome image segmentation and recognition system based on deep learning, the system comprising a chromosome detection model, a chromosome segmentation model, and a chromosome classification model, wherein:
[0025] The chromosome detection model is used to perform target detection processing on raw chromosome image data and output chromosome image data with chromosomal clusters labeled.
[0026] A chromosome segmentation model is used to perform multi-level convolution and downsampling processing on the output chromosome image data using dynamic convolution kernels to obtain intermediate layer downsampling feature data and final layer downsampling feature data; and, based on a parameter-free attention mechanism, to perform depthwise separable convolution processing and pixel recombination processing on the final layer downsampling feature data to obtain deep feature optimization data; and, to perform deconvolution processing and skip connection concatenation processing on the deep feature optimization data and intermediate layer downsampling feature data to output chromosome segmentation image data; wherein, the chromosome segmentation image data contains segmented individual chromosome image data;
[0027] The chromosome classification model is used to classify and arrange individual chromosome image data in chromosome segmentation image data to obtain chromosome karyotype analysis results.
[0028] Optionally, the chromosome segmentation model uses a U-Net network structure. The model includes a symmetrically designed encoder and decoder module, with a feature dimension enhancement module positioned between them. This module performs depthwise separable convolution and pixel reconstruction on the final layer downsampled feature data using a parameter-free attention mechanism to obtain optimized deep feature data. The feature dimension enhancement module comprises a depthwise separable convolution unit module, a parameter-free attention unit module, and a pixel reconstruction unit module, wherein:
[0029] The depthwise separable convolutional unit module is used to perform depthwise convolution and pointwise convolution on the downsampled feature data of the final layer to obtain depthwise combined feature data.
[0030] The parameterless attention unit module is used to optimize the deep combined feature data based on the energy function of the parameterless attention mechanism to obtain the optimized deep combined feature data.
[0031] The pixel recombination unit module is used to perform pixel recombination processing on the optimized depth combination feature data to obtain deep feature optimization data.
[0032] Optionally, the feature dimension enhancement module further includes a first normalization unit module, a nonlinear activation layer, and a second normalization unit module, wherein:
[0033] The first normalization unit module is used to perform batch normalization processing on the final layer downsampled feature data to obtain batch normalized data;
[0034] Nonlinear activation layers are used to process batch normalized data using nonlinear activation functions to obtain activation feature map data.
[0035] The depthwise separable convolutional unit module is used to perform depthwise convolution and pointwise convolution on activation feature map data to obtain convolutional feature data.
[0036] The second normalization unit module is used to perform batch normalization on the convolutional feature data to obtain deep combined feature data.
[0037] Optionally, the chromosome segmentation model also includes a convolution kernel adjustment module;
[0038] The convolution kernel adjustment module is used to extract features from the input chromosome image data using standard convolution kernels; calculate the attention weight of each convolution kernel using an attention mechanism; adjust the weight of the convolution kernels according to the attention weights; and replace the standard convolution kernels with the adjusted convolution kernels to obtain dynamic convolution kernels.
[0039] Optionally, the chromosome detection model is a Fast R-CNN network structure; and / or, the chromosome classification model is a ConvNeXt network structure.
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] Compared with existing technologies, the chromosome image segmentation and recognition method and system based on deep learning of the present invention improves the standard U-Net network by using dynamic convolution kernels instead of standard convolution kernels. The weights of the convolution kernels are dynamically adjusted according to the distribution of input features, thereby improving the segmentation network's ability to process details of complex chromosome images. At the same time, depthwise separable convolution processing is added to the bottom layer of the U-Net network architecture. Through the combination of depthwise separable convolution and parameter-free attention mechanism, the improved U-Net network's ability to capture complex chromosome morphological features is enhanced, enabling the system to adaptively focus on important feature regions, effectively handle chromosome overlap and morphological variations, and enhance spatial resolution through pixel recombination technology to ensure the extraction of fine structural information and banding features of chromosomes. In addition, the present invention performs target detection first, then image segmentation, and finally classification. This staged processing method helps to improve the segmentation accuracy and recognition accuracy of chromosome images. Attached Figure Description
[0042] Figure 1 The diagram shown is a schematic representation of the structure of a chromosome image segmentation and recognition system according to an embodiment of the present invention;
[0043] Figure 2 The diagram shown is a schematic representation of the structure of a chromosome image segmentation model according to an embodiment of the present invention.
[0044] Figure 3 The diagram shown is a structural schematic of a feature dimension enhancement module according to an embodiment of the present invention;
[0045] Figure 4 The diagram shown is a flowchart of a chromosome image segmentation and recognition method according to an embodiment of the present invention;
[0046] Figure 5 The diagram shown is a schematic flowchart of the execution method of the feature dimension enhancement module according to an embodiment of the present invention;
[0047] Figure 6 The image shown is a schematic diagram of a chromosome with labeled chromosome clusters according to an embodiment of the present invention;
[0048] Figure 7 The image shown is a schematic diagram of chromosome segmentation according to an embodiment of the present invention;
[0049] Figure 8 The diagram shown is a chromosome karyotype analysis diagram according to an embodiment of the present invention. Detailed Implementation
[0050] Various exemplary embodiments, features, and aspects of the present invention will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.
[0051] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.
[0052] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.
[0053] It should be understood that the terms "first," "second," and "third," etc., in the claims, specification, and drawings of this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.
[0054] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.
[0055] like Figure 1 The diagram illustrates the structure of the deep learning-based chromosome image segmentation and recognition system provided in this embodiment. The system includes a chromosome detection model 1, a chromosome segmentation model 2, and a chromosome classification model 3. The chromosome detection model 1 uses a Fast R-CNN network (Region-based Convolutional Network) to detect clusters in the original chromosome image. The chromosome segmentation model 2 uses an improved U-Net (Convolutional Networks for Biomedical Image Segmentation) network for chromosome segmentation. The chromosome classification model 3 uses a ConvNeXt network to classify each segmented individual chromosome region. In short, the chromosome image segmentation and recognition system in this embodiment first uses the chromosome detection model 1 for target detection, then uses the chromosome segmentation model 2 for image segmentation, and finally uses the chromosome classification model 3 for classification, ultimately obtaining the chromosome karyotype analysis results. This staged processing method helps improve the segmentation accuracy and recognition accuracy of chromosome images. Figure 2 As shown, the chromosome segmentation model 2 in this embodiment includes a symmetrically designed encoder module 21 and decoder module 22. The encoder module 21 extracts high-level semantic features stepwise through convolution and downsampling to form a feature map that shrinks layer by layer. The decoder module 22 restores spatial resolution through upsampling and convolution to generate an accurate segmentation mask. Skip connections directly concatenate the feature data output from each layer of the encoder module 21 with the corresponding layer of the decoder module 22. Figure 2 As shown, in this embodiment, the standard convolution is replaced by a dynamic convolution kernel 23. The weights and size of the convolution kernel are adjusted by the convolution kernel adjustment module to realize the convolution operation on the input features using the dynamic convolution kernel 23. A feature dimension enhancement module 24 is set between the encoder module 21 and the decoder module 22. The feature dimension enhancement module 24 is set at the bottom layer of the U-Net network architecture, that is, the bottleneck layer, and its function is to optimize deep features.
[0056] The chromosome image segmentation and recognition system utilizes three deep learning models to process image data in stages. The chromosome image segmentation and recognition method in this embodiment includes a target detection stage (step S100), a chromosome segmentation stage (steps S200-S400), and a chromosome recognition stage (step S500). Chromosome detection model 1 performs the target detection stage steps, chromosome segmentation model 2 performs the chromosome segmentation stage steps, and chromosome classification model 3 performs the chromosome recognition stage steps. The following combines... Figure 4 The chromosome image segmentation and recognition method of this embodiment will be described.
[0057] Step S100: Perform target detection processing on the original chromosome image data and output chromosome image data with chromosomal clusters marked.
[0058] In the target detection stage, the chromosome detection model 1 in this embodiment uses the Fast R-CNN model to detect chromosome clusters in the chromosome image data and generate regions of interest (ROIs). The input raw chromosome image data is the raw chromosome microscope image. The Fast R-CNN model outputs multiple detection boxes, such as... Figure 6 As shown, this is chromosome image data that identifies regions containing chromosome clusters.
[0059] In step S200, a dynamic convolution kernel is used to perform multi-level convolution and downsampling processing on the output chromosome image data to obtain intermediate layer downsampled feature data and final layer downsampled feature data.
[0060] In the chromosome segmentation stage, the chromosome segmentation model 2 of this embodiment (e.g.) Figure 2 As shown, in the first, second and third convolutional layers, the standard convolutional kernel is replaced with a dynamic convolutional kernel 23. The convolutional kernel adjustment module executes the construction method of the dynamic convolutional kernel, which includes the following steps S210~S240.
[0061] Step S210: Use standard convolution kernels to extract features from the input chromosome image data;
[0062] Step S220: Calculate the attention weights for each convolutional kernel using the attention mechanism;
[0063] Step S230: Adjust the weights and size of the convolution kernels according to the attention weights;
[0064] Step S240: Replace the standard convolution kernel with the adjusted convolution kernel to obtain the dynamic convolution kernel.
[0065] Understandably, the weights of the convolutional kernels are dynamically adjusted according to the distribution of input features to adapt to different shapes and boundary characteristics. In other words, the dynamic convolutional kernels are added to the U-Net network to integrate a multi-scale feature fusion mechanism. By combining local features and global information, the segmentation network's ability to process the details of complex chromosomes can be improved.
[0066] In step S300, combined with a parameterless attention mechanism, the final layer downsampled feature data is subjected to depthwise separable convolution processing and pixel recombination processing to obtain deep feature optimization data.
[0067] It is understood that the chromosome segmentation model 2 in this embodiment adds a feature dimension enhancement module 24 to the last downsampling layer (i.e., the bottleneck layer). The feature dimension enhancement module 24 is used to perform the combined parameterless attention mechanism to perform depthwise separable convolution processing and pixel recombination processing on the downsampled feature data of the final layer to obtain deep feature optimization data. The feature dimension enhancement module 24 in this embodiment is mainly designed for the bottleneck layer of the U-Net network. In the network structure design of U-Net, upsampling begins after downsampling to the bottleneck layer. The feature dimension enhancement module 24 is integrated at the bottleneck layer, replacing the upsampling operation of the first layer. With the corresponding parameter transformation, the feature map resolution can be improved without introducing additional learnable parameters. The depthwise separable convolution in this embodiment consists of two parts: depthwise convolution and pointwise convolution (i.e., 1×1 convolution). The above step S300 includes the following steps S310~S330:
[0068] Step S310: Perform depthwise convolution and pointwise convolution on the final layer downsampled feature data to obtain depthwise combined feature data.
[0069] Step S320: Optimize the deep combined feature data using an energy function based on a parameterless attention mechanism to obtain optimized deep combined feature data.
[0070] Step S330: Perform pixel recombination processing on the optimized depth combination feature data to obtain deep feature optimization data.
[0071] Since the banded structure of chromosomes has strong spatial correlation, the depthwise separable convolution in this embodiment can extract spatial features, but it is limited by the information between image channels. The parameterless attention mechanism effectively makes up for this limitation by adaptively emphasizing important feature regions, further enhancing the ability to identify key chromosome structures. The depthwise convolution process independently captures spatial features on each input channel without being interfered with by information between channels. The features extracted by the depthwise convolution are integrated after pointwise convolution to generate new feature representations, namely depthwise combined feature data. Pixel reconstruction processing plays the role of feature enhancement and spatial restoration. After the depthwise separable convolution and attention mechanism extract image features, the pixel reconstruction processing retains the information of the channel dimension and converts it into spatial details, ensuring that this key information is not lost during the upsampling process, maintaining the amount of information while increasing the spatial resolution of the feature map.
[0072] Furthermore, in step S310 above, because chromosome images have obvious channel-specific characteristics, different input channels capture different types of morphological information. Therefore, batch normalization can standardize the feature distribution of each input channel, effectively ensuring the channel specificity in the chromosome image. Moreover, batch normalization can complement the parameter-free attention mechanism. Batch normalization provides a standardized feature distribution, while the parameter-free attention mechanism adaptively adjusts feature importance. The combination of the two enhances the ability to focus on key chromosome regions. Specifically, in this embodiment, step S310 includes the following steps S311 to S314:
[0073] Step S311: Perform batch normalization on the final layer downsampled feature data to obtain batch normalized data.
[0074] Step S312: Process the batch normalized data using a non-linear activation function to obtain activation feature map data.
[0075] Step S313: Perform depthwise convolution and pointwise convolution on the activation feature map data to obtain convolutional feature data.
[0076] Step S314: Batch normalization is performed on the convolutional feature data to obtain depth-combined feature data.
[0077] Optionally, this embodiment employs Batch Normalization (BatchNorm) for batch normalization. The basic principle of BatchNorm is to normalize small batches of data along the channel dimension. The parameterless attention module uses the SimAM module. The SimAM (Simple Attention Module) quantifies the importance of each neuron in the feature map by optimizing an energy function and assigns attention weights accordingly, thus achieving a parameterless attention mechanism. Figure 3 As shown, the feature dimension enhancement module includes a first normalization unit module 241, a nonlinear activation layer 242, a second normalization unit module 243, a depthwise separable convolutional unit module 244, a SimAM module 245, and a pixel recombination unit module 246. The first normalization unit module 241 performs step S311, the nonlinear activation layer performs step S312, the second normalization unit module performs step S313, the depthwise separable convolutional unit module performs step S314, the SimAM module performs step S320, and the pixel recombination unit module performs step S330. Furthermore, applying batch normalization after the pixel recombination operation ensures the consistency of feature quality during the upsampling process, ultimately yielding deep feature optimization data. See also... Figure 5 In the feature dimension enhancement module, from an information flow perspective, the input feature data undergoes a series of processes: a 1×1 convolutional layer, batch normalization, a ReLU layer, a depthwise separable convolutional layer, batch normalization, a SimAM module, pixel shuffle, another 1×1 convolutional layer, and then batch normalization again, ultimately outputting optimized deep feature data. As mentioned earlier, chromosome images exhibit significant channel-specific characteristics; different channels capture different types of morphological information. BatchNorm normalization, by independently processing the statistical characteristics of each channel, preserves these key differences. Experiments have shown that it performs better than other normalization methods in terms of adaptability to the feature dimension enhancement module designed in this embodiment. The mathematical principle of SimAM in this embodiment can be understood through the design and optimization of the energy function. For the input feature map... SimAM defines an energy function to quantify the importance of each location. For each location... eigenvalues at The energy function can be expressed as:
[0078]
[0079] in, and Representing channels The mean and variance, It is a small constant used to prevent division by zero and ensure numerical stability. This energy function is based on the intuition that the greater the difference between an eigenvalue and the channel mean, the more information the eigenvalue may contain; at the same time, the greater the channel variance, the more dispersed the eigenvalue distribution, and the more important the individual eigenvalue may be.
[0080] By differentiating the energy function and setting it to zero, a closed-form solution for the energy minimum can be obtained. The reciprocal of the energy has been proven to be an effective attention weight.
[0081]
[0082] in, This represents the square of the difference between the feature and the channel mean. It is the mean across the channel dimension. It is the channel variance, which can be expressed as Ultimately, SimAM limits the range of attention weights using the sigmoid function and optimizes features using a multiplication mechanism.
[0083]
[0084] Therefore, it not only conforms to neuroscience theory, but is also computationally efficient and requires no additional parameters.
[0085] It is important to emphasize that in this embodiment, the SimAM module is placed after depthwise separable convolution in the feature dimension enhancement module design. This design is based on the fact that SimAM highlights important features and suppresses unimportant features. Therefore, placing SimAM after depthwise separable convolution extracts image features can highlight important features through an attention mechanism. Specifically, when faced with problems such as blurred chromosome boundaries and complex shapes, depthwise separable convolution can effectively capture these complex shapes, while SimAM helps the model focus on key regions, such as chromosome boundaries and banding structures, thereby improving segmentation accuracy. In addition, one of the design features of the feature dimension enhancement module is its small number of parameters and high efficiency. Compared with lengthy and complex attention mechanisms, the zero-parameter design of SimAM perfectly matches this goal.
[0086] Step S400: Perform deconvolution processing and skip connection concatenation processing on the deep feature optimization data and the intermediate layer downsampled feature data to output chromosome segmentation image data; wherein, the chromosome segmentation image data includes segmented single chromosome image data.
[0087] Understandably, in combination Figure 2As shown, in this embodiment, the decoder module 22 is used to execute step S400, upsampling the data output by the feature dimension enhancement module 24 and the data from the skip connections of each convolutional layer of the encoder module 21, and finally outputting the segmented single chromosome image data, such as... Figure 7 As shown. The analysis focuses on three aspects: information flow, feature representation capability, and computational resource allocation. Adding a feature dimension enhancement module to the bottleneck layer is crucial. First, from the perspective of information flow, the bottleneck layer is the connection point between the downsampling and upsampling processes. This layer not only represents the key point of information extraction and compression in downsampling but also serves as the starting point for feature extraction and information reconstruction in subsequent upsampling. Therefore, improving the bottleneck layer can significantly impact the entire network. Second, regarding feature representation capability, the standard U-Net network architecture faces significant limitations at the bottleneck layer because the image information has already been drastically compressed in the spatial dimension during downsampling, especially the information at the lowest level, which contains almost no detail. In the process of downsampling information transmission, details such as chromosome edge texture may be lost. Furthermore, the bottleneck layer suffers from low feature channel utilization efficiency and uneven feature weight distribution. To address these issues, a feature dimension enhancement module is added to the bottleneck layer. Through depthwise separable convolution and attention mechanisms, the utilization efficiency of feature channels is optimized, achieving adaptive weight allocation for key features. From a computational resource allocation perspective, the bottleneck layer has the smallest feature map space size. Therefore, increasing module complexity here has a relatively small impact on overall computational cost. Although the feature dimension enhancement module increases the complexity of feature processing, the additional computational cost is manageable because it operates on a small feature map. The improvement to the bottleneck layer can influence the weight optimization of the entire encoder module 21 through backpropagation, and simultaneously affect the feature reconstruction of the decoder module 22 through forward propagation, resulting in network-wide performance improvements with limited investment.
[0088] Step S500: Classify and arrange the individual chromosome image data in the chromosome segmentation image data to obtain chromosome karyotype analysis results.
[0089] In the chromosome identification stage, the chromosome classification model 3 in this embodiment uses the ConvNeXt network to classify each segmented individual chromosome region. The ConvNeXt network is designed based on modern convolutional models, employing deep feature learning to improve classification accuracy. A global context-aware mechanism is added to the input layer to ensure accurate classification. For example... Figure 8 As shown, the final output is the category number of each chromosome, which facilitates subsequent genetic analysis.
[0090] In summary, compared with existing technologies, the chromosome image segmentation and recognition method and system based on deep learning of this invention improves the standard U-Net network by using dynamic convolutional kernels instead of standard convolutional kernels. The weights of the convolutional kernels are dynamically adjusted according to the distribution of input features, improving the segmentation network's ability to process details of complex chromosome images. Simultaneously, depthwise separable convolution processing is added to the bottom layer of the U-Net network architecture. The combination of depthwise separable convolution and a parameter-free attention mechanism enhances the improved U-Net network's ability to capture complex chromosome morphological features, enabling the system to adaptively focus on important feature regions and effectively handle chromosome overlap and morphological variations. Pixel recombination technology enhances spatial resolution, ensuring the extraction of fine structural information and banding features of chromosomes. Furthermore, this invention performs target detection first, then image segmentation, and finally classification, employing a staged processing method that helps improve the segmentation accuracy and recognition accuracy of chromosome images.
[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions created by the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions created by the present invention without departing from the essence and scope of the technical solutions created by the present invention.
Claims
1. A chromosome image segmentation and recognition method based on deep learning, characterized in that, The method includes: Based on the Fast R-CNN network structure model, target detection processing is performed on the original chromosome image data, and chromosome image data with labeled chromosome clusters is output. Based on the U-Net network architecture model, the encoder of the U-Net network architecture model utilizes dynamic convolution kernels to perform multi-level convolution and downsampling processing on the output chromosome image data, obtaining intermediate layer downsampling feature data and final layer downsampling feature data; and, In the bottleneck layer of the U-Net network architecture model, batch normalization is performed on the downsampled feature data of the final layer to obtain batch-normalized data; a non-linear activation function is used to process the batch-normalized data to obtain activation feature map data; depthwise convolution and pointwise convolution are performed on the activation feature map data to obtain convolutional feature data; batch normalization is then performed on the convolutional feature data to obtain deep combined feature data; an energy function based on a parameterless attention mechanism is used to optimize the deep combined feature data to obtain optimized deep combined feature data; pixel recombination is then performed on the optimized deep combined feature data to obtain deep feature optimized data; and so on. In the decoder of the U-Net network structure model, deconvolution processing and skip connection concatenation processing are performed on the deep feature optimization data and the intermediate layer downsampled feature data to output chromosome segmentation image data; wherein, the chromosome segmentation image data contains segmented individual chromosome image data; Based on the ConvNeXt network structure model, the individual chromosome image data in the chromosome segmentation image data are classified and arranged to obtain the chromosome karyotype analysis results.
2. The method according to claim 1, characterized in that, The method for constructing the dynamic convolutional kernel includes: Features of the input chromosome image data are extracted using standard convolution kernels; The attention weights for each convolutional kernel are calculated using an attention mechanism; Adjust the weights and size of the convolutional kernels based on attention weights; The adjusted convolutional kernel is replaced with the standard convolutional kernel to obtain the dynamic convolutional kernel.
3. A chromosome image segmentation and recognition system based on deep learning, characterized in that, The system includes a chromosome detection model, a chromosome segmentation model, and a chromosome classification model, wherein: A chromosome detection model is used to perform target detection processing on raw chromosome image data and output chromosome image data with labeled chromosome clusters; the chromosome detection model is a Fast R-CNN network structure. The chromosome segmentation model includes a symmetrically designed encoder and decoder module. A feature dimension enhancement module is positioned between the encoder and decoder modules. This module comprises a first normalization unit module, a non-linear activation layer, a second normalization unit module, a depthwise separable convolutional unit module, a parameter-free attention unit module, and a pixel recombination unit module. The chromosome segmentation model uses a U-Net network structure, and the feature dimension enhancement module is located in the bottleneck layer of the U-Net network structure. The encoder module utilizes dynamic convolution kernels to perform multi-level convolution and downsampling processing on the output chromosome image data, obtaining intermediate layer downsampled feature data and final layer downsampled feature data; and, The first normalization unit module performs batch normalization on the downsampled feature data of the final layer to obtain batch normalized data; the nonlinear activation layer processes the batch normalized data using a nonlinear activation function to obtain activation feature map data; the depthwise separable convolution unit module performs depthwise convolution and pointwise convolution on the activation feature map data to obtain convolutional feature data; the second normalization unit module performs batch normalization on the convolutional feature data to obtain depthwise combined feature data; the parameterless attention unit module optimizes the depthwise combined feature data based on the energy function of the parameterless attention mechanism to obtain optimized depthwise combined feature data; the pixel recombination unit module performs pixel recombination on the optimized depthwise combined feature data to obtain deep feature optimized data; and so on. The decoder module is used to perform deconvolution processing and skip connection concatenation processing on deep feature optimization data and intermediate layer downsampled feature data, and output chromosome segmentation image data; wherein, the chromosome segmentation image data contains segmented single chromosome image data; The chromosome classification model is used to classify and arrange individual chromosome image data in chromosome segmentation image data to obtain chromosome karyotype analysis results; the chromosome classification model is a ConvNeXt network structure.
4. The system according to claim 3, characterized in that, The chromosome segmentation model also includes a convolution kernel adjustment module; The convolution kernel adjustment module is used to extract features from the input chromosome image data using standard convolution kernels; calculate the attention weight of each convolution kernel using an attention mechanism; and adjust the weight of the convolution kernels based on the attention weights. The adjusted convolutional kernel is replaced with the standard convolutional kernel to obtain the dynamic convolutional kernel.