Deep learning-based chromosome image segmentation identification method and system

Abstract

A chromosome image segmentation identification method and system based on deep learning relates to the technical field of medical image processing, and comprises the following steps: performing target detection processing on original chromosome image data, and outputting chromosome image data calibrated with chromosome clusters; performing multi-level convolution and down-sampling processing on the chromosome image data by using a dynamic convolution kernel to obtain middle-layer down-sampling feature data and final-layer down-sampling feature data; in combination with a parameter-free attention mechanism, performing depth separable convolution processing and pixel recombination processing on the sampling feature data under the final layer to obtain deep feature optimization data; performing deconvolution processing and jump connection splicing processing on the deep feature optimization data and the intermediate layer down-sampling feature data, and outputting chromosome segmentation image data; classifying and arranging the chromosome segmentation image data to obtain a chromosome karyotype analysis result; the chromosome image segmentation and identification method and system have good segmentation and identification effects on the chromosome image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a chromosome image segmentation and recognition method and system based on deep learning. BACKGROUND

[0002] The U-Net network architecture is often used in the design of convolutional neural networks for medical image segmentation, and its core features are the symmetrical encoder-decoder structure and the skip connection. The encoder gradually extracts high-level semantic features through convolution and down-sampling, the decoder restores the spatial resolution through up-sampling and deconvolution, and the skip connection directly splices the feature data output by each layer of the encoder with the corresponding layer of the decoder. However, due to the complex structure, irregular distribution, overlapping and variable morphology of chromosome images, the standard U-Net network does not have an ideal effect on chromosome image segmentation, and the segmentation quality directly affects the accuracy of chromosome analysis and recognition. SUMMARY

[0003] The present application relates to the technical field of medical image processing, and particularly relates to a chromosome image segmentation and recognition method and system based on deep learning.

[0004] To achieve the above-mentioned purpose, in a first aspect, the present application provides a chromosome image segmentation and recognition method based on deep learning, which comprises: performing target detection processing on original chromosome image data to output chromosome image data labeled with chromosome clusters; using a dynamic convolution kernel and performing multi-level convolution and down-sampling processing on the output chromosome image data to obtain intermediate layer down-sampling feature data and final layer down-sampling feature data; combining a parameter-free attention mechanism to perform deep separable convolution processing and pixel reorganization processing on the final layer down-sampling feature data to obtain deep feature optimization data; performing deconvolution processing and skip connection splicing processing on the deep feature optimization data and the intermediate layer down-sampling feature data to output chromosome segmentation image data; wherein the chromosome segmentation image data comprises single chromosome image data after segmentation; performing classification and arrangement processing on the single chromosome image data in the chromosome segmentation image data to obtain a chromosome karyotype analysis result.

[0005] Optionally, the deep separable convolution processing and pixel reorganization processing on the final layer down-sampling feature data to obtain deep feature optimization data in combination with the parameter-free attention mechanism comprises: performing deep convolution processing and point-by-point convolution processing on the final layer down-sampling feature data to obtain deep combination feature data; The energy function based on the parameter-free attention mechanism is used to optimize the deep combined feature data, and the optimized deep combined feature data is obtained. The pixel reorganization processing is performed on the optimized deep combined feature data, and deep feature optimization data is obtained.

[0006] Optionally, the final layer down-sampling feature data is subjected to deep convolution processing and point-by-point convolution processing to obtain deep combined feature data, including: The batch normalization processing is performed on the final layer down-sampling feature data to obtain batch normalization data. The batch normalization data is processed by using a nonlinear activation function to obtain activation feature map data. The activation feature map data is subjected to deep convolution processing and point-by-point convolution processing to obtain convolution feature data. The batch normalization processing is performed on the convolution feature data to obtain deep combined feature data.

[0007] Optionally, the construction method of the dynamic convolution kernel includes: The standard convolution kernel is used to extract features of the input chromosome image data. The attention mechanism is used to calculate the attention weight of each convolution kernel. The weight and size of the convolution kernel are adjusted according to the attention weight. The adjusted convolution kernel is used to replace the standard convolution kernel to obtain the dynamic convolution kernel.

[0008] In a second aspect, the application provides a chromosome image segmentation and recognition system based on deep learning, which includes a chromosome detection model, a chromosome segmentation model and a chromosome classification model, wherein: The chromosome detection model is used to perform target detection processing on the original chromosome image data, and output chromosome image data labeled with chromosome clusters. The chromosome segmentation model is used to use the dynamic convolution kernel, perform multi-level convolution and down-sampling processing on the output chromosome image data, obtain intermediate layer down-sampling feature data and final layer down-sampling feature data, perform deep separable convolution processing and pixel reorganization processing on the final layer down-sampling feature data based on the parameter-free attention mechanism, obtain deep feature optimization data, perform deconvolution processing and jump connection splicing processing on the deep feature optimization data and the intermediate layer down-sampling feature data, and output chromosome segmentation image data; wherein the chromosome segmentation image data includes segmented single chromosome image data. The chromosome classification model is used to perform classification and arrangement processing on the single chromosome image data in the chromosome segmentation image data, and obtain chromosome karyotype analysis results.

[0009] Optionally, the chromosome segmentation model is a U-Net network structure, the chromosome segmentation model comprises a symmetrically designed encoder module and a decoder module, a feature dimension enhancement module is arranged between the encoder module and the decoder module, the feature dimension enhancement module is used for performing deep separable convolution processing and pixel recombination processing on the final layer down-sampling feature data by combining a parameter-free attention mechanism to obtain deep feature optimization data; the feature dimension enhancement module comprises a deep separable convolution unit module, a parameter-free attention unit module and a pixel recombination unit module, wherein: The deep separable convolution unit module is used for performing deep convolution processing and point-by-point convolution processing on the final layer down-sampling feature data to obtain deep combination feature data. The parameter-free attention unit module is used for performing optimization processing on the deep combination feature data based on an energy function of the parameter-free attention mechanism to obtain optimized deep combination feature data. The pixel recombination unit module is used for performing pixel recombination processing on the optimized deep combination feature data to obtain deep feature optimization data.

[0010] Optionally, the feature dimension enhancement module further comprises a first normalization unit module, a nonlinear activation layer and a second normalization unit module, wherein: The first normalization unit module is used for performing batch normalization processing on the final layer down-sampling feature data to obtain batch normalization data. The nonlinear activation layer is used for processing the batch normalization data by using a nonlinear activation function to obtain activation feature map data. The deep separable convolution unit module is used for performing deep convolution processing and point-by-point convolution processing on the activation feature map data to obtain convolution feature data. The second normalization unit module is used for performing batch normalization processing on the convolution feature data to obtain deep combination feature data.

[0011] Optionally, the chromosome segmentation model further comprises a convolution kernel adjustment module. The convolution kernel adjustment module is used for extracting features of the input chromosome image data by using a standard convolution kernel, calculating attention weights of each convolution kernel by using an attention mechanism, adjusting weights of the convolution kernel according to the attention weights, and replacing the standard convolution kernel with the adjusted convolution kernel to obtain a dynamic convolution kernel.

[0012] Optionally, the chromosome detection model is a Fast R-CNN network structure; and / or, the chromosome classification model is a ConvNeXt network structure.

[0013] Compared with the prior art, the present application has the following beneficial effects: Compared with the prior art, the chromosome image segmentation and recognition method and system based on deep learning of the application improves the standard U-Net network, uses a dynamic convolution kernel to replace the standard convolution kernel, dynamically adjusts the weight of the convolution kernel according to the distribution of input features, and improves the detail processing capability of the segmentation network for complex chromosome images; at the same time, a deep separable convolution processing is added to the bottom layer of the U-Net network architecture, the combination of the deep separable convolution and the parameter-free attention mechanism enhances the ability of the improved U-Net network to capture complex chromosome morphological features, so that the system can adaptively focus on important feature regions, effectively process chromosome overlapping and morphological variation, and through the pixel recombination technology, the spatial resolution is enhanced, the fine structure information and band feature information of the chromosome are extracted; in addition, the application first performs target detection, then performs image segmentation, and finally performs classification, and the method of processing in stages is beneficial to improving the segmentation accuracy and recognition accuracy of the chromosome image. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 Fig. 1 shows a structural schematic diagram of a chromosome image segmentation and recognition system according to an embodiment of the application; Figure 2 Fig. 2 shows a structural schematic diagram of a chromosome image segmentation model according to an embodiment of the application; Figure 3 Fig. 3 shows a structural schematic diagram of a feature dimension enhancement module according to an embodiment of the application; Figure 4 Fig. 4 shows a flowchart of a chromosome image segmentation and recognition method according to an embodiment of the application; Figure 5 Fig. 5 shows a flowchart of an execution method of a feature dimension enhancement module according to an embodiment of the application; Figure 6 Fig. 6 shows a schematic diagram of a chromosome image labeled with chromosome clusters according to an embodiment of the application; Figure 7 Fig. 7 shows a schematic diagram of a chromosome segmentation image according to an embodiment of the application; Figure 8 Fig. 8 shows a karyotype analysis diagram according to an embodiment of the application. DETAILED DESCRIPTION

[0015] Various exemplary embodiments, features and aspects of the application will be described in detail below with reference to the accompanying drawings. The same reference signs in the drawings represent functionally identical or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0016] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0017] The term "and / or", used in the context of describing a combination of entities, refers to a full or partial combination of any of the entities listed in the combination, for example, A and / or B can refer to only A, to only B, or to both A and B. Further, the term "at least one of' used in the context of describing a combination of entities refers to any of the individual entities in the combination or any combination of two or more of the entities in the combination, for example, at least one of A, B, and C can refer to only A, only B, only C, AB, AC, BC, or ABC.

[0018] It should be understood that the terms "first", "second", and "third" and the like used in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. The terms "comprises", "comprising", "includes", "including" and the like used in the present specification

[0019] In addition, for the purpose of clarity and a concise description, numerous specific details are set forth in the following detailed description. It should be understood, however, that the application can be practiced in other embodiments that do not include all the specific details described. In some instances, well-known methods, structures, and techniques have not been described in detail so as not to obscure the

[0020] As Figure 1As shown, the structure schematic diagram of the chromosome image segmentation and recognition system based on deep learning provided by the embodiment is shown. The chromosome image segmentation and recognition system comprises a chromosome detection model 1, a chromosome segmentation model 2 and a chromosome classification model 3. The chromosome detection model 1 uses a Fast R-CNN network (Region-based Convolutional Network) to detect clusters in the original chromosome image. The chromosome segmentation model 2 uses an improved U-Net (Convolutional Networks for Biomedical Image Segmentation) network to perform chromosome segmentation. The chromosome classification model 3 uses a ConvNeXt network to classify each single chromosome region obtained by segmentation. In short, the chromosome image segmentation and recognition system of the embodiment first uses the chromosome detection model 1 to perform target detection, then uses the chromosome segmentation model 2 to perform image segmentation, and finally uses the chromosome classification model 3 to perform classification, so as to obtain the chromosome karyotype analysis result. The method of processing in stages can improve the segmentation accuracy and recognition accuracy of the chromosome image. Figure 2 As shown, the chromosome segmentation model 2 of the embodiment comprises a symmetrically designed encoder module 21 and a decoder module 22. The encoder module 21 gradually extracts high-level semantic features through convolution and down-sampling, forms a feature map that is gradually contracted layer by layer, and the decoder module 22 restores the spatial resolution through up-sampling and convolution to generate an accurate segmentation mask. The jump connection directly splices the feature data output by each layer of the encoder module 21 with the corresponding layer of the decoder module 22. Figure 2 As shown, the standard convolution of the embodiment is replaced by a dynamic convolution kernel 23. The weight and size of the convolution kernel are adjusted by a convolution kernel adjustment module to realize convolution operation on the input features by using the dynamic convolution kernel 23. A feature dimension enhancement module 24 is arranged between the encoder module 21 and the decoder module 22. The feature dimension enhancement module 24 is arranged at the bottom layer of the U-Net network architecture, that is, the bottleneck layer, and functions to optimize deep features.

[0021] The chromosome image segmentation and recognition system uses three deep learning models to process image data in stages. The chromosome image segmentation and recognition method of the embodiment comprises a target detection stage (step S100 below), a chromosome segmentation stage (steps S200-S400 below) and a chromosome recognition stage (step S500 below). The chromosome detection model 1 performs the steps of the target detection stage, the chromosome segmentation model 2 performs the steps of the chromosome segmentation stage, and the chromosome classification model 3 performs the steps of the chromosome recognition stage. The chromosome image segmentation and recognition method of the embodiment will be described below in combination with Figure 4 The chromosome image segmentation and recognition method of the embodiment will be described below in combination with

[0022] Step S100 , performing target detection processing on the original chromosome image data, and outputting chromosome image data marked with chromosome clusters.

[0023] In the target detection stage, the chromosome detection model 1 of this embodiment uses the Fast R-CNN model to detect chromosome clusters in the chromosome image data and generate a region of interest (ROI). The input original chromosome image data is the original chromosome microscope image, and the Fast R-CNN model outputs multiple detection frames, such as Figure 6 As shown, the chromosome image data containing chromosome cluster regions are marked.

[0024] In step S200 , a dynamic convolution kernel is used to perform multi-level convolution and downsampling processing on the output chromosome image data to obtain intermediate layer downsampling feature data and final layer downsampling feature data.

[0025] In the chromosome segmentation stage, the chromosome segmentation model 2 of this embodiment (such as Figure 2 As shown in FIG, the standard convolution kernel is replaced by the dynamic convolution kernel 23 in the first, second and third convolution layers, and the convolution kernel adjustment module executes the method for constructing the dynamic convolution kernel, which includes the following steps S210 to S240.

[0026] Step S210, extracting features of the input chromosome image data using a standard convolution kernel; Step S220, using the attention mechanism to calculate the attention weight of each convolution kernel; Step S230, adjusting the weight and size of the convolution kernel according to the attention weight; Step S240: Replace the standard convolution kernel with the adjusted convolution kernel to obtain a dynamic convolution kernel.

[0027] It is understandable that the weights of the convolution kernel are dynamically adjusted according to the distribution of the input features to adapt to different shapes and boundary characteristics, that is, the dynamic convolution kernel is added to the U-Net network, and the multi-scale feature fusion mechanism is integrated. By combining local features and global information, the segmentation network's ability to process the details of complex chromosomes can be improved.

[0028] In step S300, the parameter-free attention mechanism is combined to perform depth-separable convolution processing and pixel reassembly processing on the final layer downsampled feature data to obtain deep feature optimization data.

[0029] It can be understood that the chromosome segmentation model 2 of the embodiment adds a feature dimension enhancement module 24 at the last layer of downsampling (i.e. the bottleneck layer), which is used to perform the combination of the parameter-free attention mechanism, and perform deep separable convolution processing and pixel recombination processing on the final layer down-sampling feature data to obtain deep feature optimization data. The feature dimension enhancement module 24 of the embodiment is mainly designed for the bottleneck layer of the U-Net network. In the network structure design of the U-Net, the up-sampling is started after the down-sampling to the bottleneck layer, and the feature dimension enhancement module 24 is integrated at the bottleneck layer, replacing the up-sampling operation of the first layer, corresponding to the parameter transformation, without introducing additional learnable parameters to realize the improvement of the feature map resolution. The deep separable convolution of the embodiment is composed of two parts: depthwise convolution (Depthwise Convolution) and pointwise convolution (Pointwise Convolution, i.e. 1x1 convolution). The step S300 includes the following steps S310-S330: Step S310, performing deep convolution processing and pointwise convolution processing on the final layer down-sampling feature data to obtain deep combination feature data.

[0030] Step S320, performing optimization processing on the deep combination feature data based on the energy function of the parameter-free attention mechanism to obtain optimized deep combination feature data.

[0031] Step S330, performing pixel recombination processing on the optimized deep combination feature data to obtain deep feature optimization data.

[0032] Since the band structure of the chromosome has strong spatial correlation, the deep separable convolution in the embodiment can extract spatial features, but will be limited by the related information between image channels. The parameter-free attention mechanism effectively makes up for this limitation by adaptively emphasizing important feature regions, further enhancing the recognition ability of the key structure of the chromosome. Deep convolution processing independently captures spatial features on each input channel without being disturbed by the information between channels. The features extracted by the deep convolution are integrated after the pointwise convolution operation to generate new feature representations, i.e. deep combination feature data. Pixel recombination processing plays a role in feature enhancement and spatial recovery. After the image features are extracted by the deep separable convolution and the attention mechanism, the pixel recombination processing retains the channel dimension information and converts it into spatial details, ensuring that these key information will not be lost in the up-sampling process, maintaining the amount of information while increasing the spatial resolution of the feature map.

[0033] Further, in the step S310, the chromosome image has obvious channel-specific characteristics, different input channels capture different types of morphological information, and therefore applying batch normalization processing can standardize the feature distribution of each input channel, and well guarantee the specificity between channels in the chromosome image. Moreover, batch normalization processing can form a complementary effect with the parameter-free attention mechanism, batch normalization processing provides standardized feature distribution, and the parameter-free attention mechanism adaptively adjusts feature importance, and the combination of the two enhances the attention ability to the key regions of the chromosome. Specifically, in the embodiment, the step S310 includes the following steps S311-S314: In the step S311, the batch normalization processing is performed on the final layer down-sampling feature data to obtain batch normalization data.

[0034] In the step S312, the nonlinear activation function is used to process the batch normalization data to obtain activation feature map data.

[0035] In the step S313, the depth convolution processing and the point-by-point convolution processing are performed on the activation feature map data to obtain convolution feature data.

[0036] In the step S314, the batch normalization processing is performed on the convolution feature data to obtain deep combination feature data.

[0037] Optionally, the batch normalization processing of the embodiment adopts the Batch Normalization (BatchNorm) normalization method, the basic principle of BatchNorm is to perform normalization processing on small batch data in the channel dimension; the parameter-free attention module adopts the SimAm module, the SimAM (Simple Attention Module) module quantifies the importance of each neuron in the feature map by optimizing an energy function, and accordingly allocates attention weights, realizing a parameter-free attention mechanism. Combined with Figure 3 As shown, the feature dimension enhancement module includes a first normalization unit module 241, a nonlinear activation layer 242, a second normalization unit module 243, a depth separable convolution unit module 244, a SimAM module 245, and a pixel reorganization unit module 246, the first normalization unit module 241 is used to execute the step S311, the nonlinear activation layer is used to execute the step S312, the second normalization unit module is used to execute the step S313, the depth separable convolution unit module is used to execute the step S314, the SimAM module is used to execute the step S320, and the pixel reorganization unit module is used to execute the step S330. In addition, after the pixel rearrangement operation, the batch normalization processing can guarantee the consistency of the feature quality in the up-sampling process, and finally obtain the deep feature optimization data. Referring to Figure 5, the feature dimension enhancement module, when performing the above steps, from the perspective of information flow, the input feature data is subjected to 1x1 convolution layer, normalization processing (Batch Normal), Relu layer, depth separable convolution processing, normalization processing (Batch Normal), SimAm module, pixel reorganization processing (PixelShuffle), 1x1 convolution layer, and then normalization processing (Batch Normal), and finally output deep feature optimization data. As mentioned above, the chromosome image has obvious channel-specific features, and different channels capture different types of morphological information, and the BatchNorm normalization processing retains these key differences by independently processing the statistical properties of each channel. For the adaptability of the feature dimension enhancement module designed in this embodiment, compared with other normalization processing methods, the effect is better. The mathematical principle of SimAM in this embodiment can be understood through the design and optimization of the energy function. For the input feature map , SimAM defines an energy function to quantify the importance of each position. For the feature value at position , the energy function can be expressed as: where and represent the mean and variance of channel , respectively, is a small constant to prevent division by zero and ensure numerical stability. This energy function is based on the following intuition: the greater the difference between the feature value and the channel mean, the more information the feature may contain; at the same time, the greater the channel variance, the more dispersed the feature distribution, and the relative importance of individual feature values may be reduced.

[0038] By taking the derivative of the energy function and setting it equal to zero, the closed-form solution for the minimum energy can be obtained. The reciprocal of the energy is proven to be an effective attention weight: where represents the square of the difference between the feature and the channel mean, is the mean in the channel dimension, is the channel variance, which can be expressed as . Finally, SimAM limits the range of attention weights through a sigmoid function and optimizes the features using a multiplication mechanism: Therefore, it not only conforms to the theory of neuroscience, but also is computationally efficient without additional parameters.

[0039] It should be emphasized that in the feature dimension enhancement module design of the present embodiment, the SimAM module is placed after the deep separable convolution after analysis and selection. This design is because SimAM has the function of highlighting important features and suppressing unimportant features, so placing SimAM after the deep separable convolution to extract image features can highlight important features through the attention mechanism. Specifically, when facing the problems of blurred chromosome image boundaries and complex morphology, the deep separable convolution can effectively capture these complex morphologies, and SimAM helps the model focus on key areas such as chromosome boundaries and band structure, thereby improving segmentation accuracy. In addition, one of the design features of the feature dimension enhancement module is that it has a small number of parameters and is efficient. Compared with the long and complex attention mechanism, the zero-parameter design of SimAM perfectly meets this goal.

[0040] In step S400, the deep feature optimization data and the intermediate layer down-sampling feature data are subjected to deconvolution processing and jump connection splicing processing, and chromosome segmentation image data is output; wherein the chromosome segmentation image data comprises single chromosome image data after segmentation.

[0041] It can be understood that, in combination with Figure 2 It can be understood that, in combination with Figure 7As shown in the figure. From the three aspects of information flow, feature representation ability and computing resource allocation, a feature dimension enhancement module is added at the bottleneck layer: first, from the perspective of information flow, the bottleneck layer is the connection junction of the downsampling process and the upsampling process. This layer not only represents the key point of information extraction and compression of the downsampling process, but also is the starting point of feature extraction and information reconstruction of the subsequent upsampling process. Therefore, improving the bottleneck layer can have a great impact on the entire network; second, from the perspective of feature representation ability, the standard U-Net network architecture faces obvious limitations at the bottleneck layer. Because the information of the image has been greatly compressed in the spatial dimension during the downsampling process, especially the information at the bottom layer, which almost contains no detailed information. In the information transmission of downsampling, these detailed information such as chromosome edge texture may also be lost. Moreover, the bottleneck layer has low utilization efficiency of feature channels and uneven distribution of feature weights. According to the above factors, a feature dimension enhancement module is added at the bottleneck layer. Through depth separable convolution and attention mechanism, the utilization efficiency of feature channels is optimized, and adaptive weight distribution of key features is realized; from the perspective of computing resource allocation, the bottleneck layer has the smallest feature map space size. Adding a module at this layer has relatively small impact on the overall computing cost. Although the feature dimension enhancement module increases the complexity of feature processing, the additional computing cost is controllable because it acts on a small size feature map. The improvement of the bottleneck layer can affect the weight optimization of the entire encoder module 21 through back propagation, and at the same time affect the feature reconstruction of the decoder module 22 through forward propagation, so that limited investment produces network-wide performance improvement.

[0042] In step S500, the single chromosome image data in the chromosome segmentation image data is classified and arranged to obtain a chromosome karyotype analysis result.

[0043] In the chromosome recognition stage, the chromosome classification model 3 of the present embodiment uses the ConvNeXt network to classify each single chromosome region obtained by segmentation. The design of the ConvNeXt network is based on modern convolutional models, which uses deep feature learning to improve classification accuracy. The global context perception mechanism is added at the input layer to ensure the accuracy of classification. As shown in the figure, the final output is the class number of each chromosome, which is convenient for subsequent genetic analysis. Figure 8

[0044] ​Compared with the prior art, the chromosome image segmentation and recognition method and system based on deep learning improves the standard U-Net network, uses a dynamic convolution kernel to replace the standard convolution kernel, dynamically adjusts the weight of the convolution kernel according to the distribution of input features, and improves the detail processing capability of the segmentation network for complex chromosome images. At the same time, a deep separable convolution processing is added to the bottom layer of the U-Net network architecture. Through the combination of deep separable convolution and parameter-free attention mechanism, the improved U-Net network can capture the complex chromosome morphological features, so that the system can adaptively focus on important feature areas, effectively process chromosome overlapping and morphological variation, and through pixel recombination technology, the spatial resolution is enhanced, the fine structure information and band feature information of the chromosome are ensured to be extracted. In addition, the present application first detects the target, then segments the image, and finally classifies, and the phased processing method is beneficial to improve the segmentation accuracy and recognition accuracy of the chromosome image.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the scope of protection of the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the essence and scope of the technical solutions of the present application.

Claims

1. A chromosome image segmentation and recognition method based on deep learning, characterized in that: The method comprises: Performing target detection processing on the original chromosome image data and outputting chromosome image data marked with chromosome clusters; Using dynamic convolution kernel, the output chromosome image data is subjected to multi-level convolution and downsampling processing to obtain the intermediate layer downsampling feature data and the final layer downsampling feature data; Combined with the parameter-free attention mechanism, the final layer downsampled feature data is subjected to depth-separable convolution and pixel reassembly to obtain deep feature optimization data; Performing deconvolution processing and jump connection splicing processing on the deep feature optimization data and the intermediate layer downsampled feature data to output chromosome segmentation image data; wherein the chromosome segmentation image data includes the segmented single chromosome image data; The single chromosome image data in the chromosome segmentation image data is classified and arranged to obtain the chromosome karyotype analysis result.

2. The method according to claim 1, characterized in that The parameter-free attention mechanism is combined to perform deep separable convolution processing and pixel reassembly processing on the final layer downsampled feature data to obtain deep feature optimization data, including: Perform depth convolution and point-by-point convolution on the final layer downsampled feature data to obtain deep combined feature data; The energy function based on the parameter-free attention mechanism is used to optimize the deep combination feature data to obtain the optimized deep combination feature data; The optimized deep combination feature data is subjected to pixel reorganization processing to obtain deep feature optimization data.

3. The method according to claim 2, characterized in that Perform depth convolution and point-by-point convolution on the final layer downsampled feature data to obtain deep combined feature data, including: Perform batch normalization on the downsampled feature data of the final layer to obtain batch normalized data; The batch normalization data is processed using a nonlinear activation function to obtain activation feature map data; Performing depth convolution and point-by-point convolution on the activated feature map data to obtain convolution feature data; The convolution feature data is batch normalized to obtain deep combined feature data.

4. The method according to claim 1, wherein The method for constructing the dynamic convolution kernel includes: Use standard convolution kernel to extract features of input chromosome image data; Use the attention mechanism to calculate the attention weight of each convolution kernel; Adjust the weight and size of the convolution kernel according to the attention weight; The adjusted convolution kernel replaces the standard convolution kernel to obtain a dynamic convolution kernel.

5. Chromosome image segmentation and recognition system based on deep learning, characterized by: The system includes a chromosome detection model, a chromosome segmentation model and a chromosome classification model, wherein: The chromosome detection model is used to perform target detection processing on the original chromosome image data and output chromosome image data marked with chromosome clusters; A chromosome segmentation model is configured to utilize a dynamic convolution kernel and perform multi-level convolution and downsampling processing on the output chromosome image data to obtain intermediate layer downsampled feature data and final layer downsampled feature data; and to perform depthwise separable convolution processing and pixel reassembly processing on the final layer downsampled feature data based on a parameter-free attention mechanism to obtain deep feature optimization data; and to perform deconvolution processing and jump connection splicing processing on the deep feature optimization data and the intermediate layer downsampled feature data to output chromosome segmentation image data; wherein the chromosome segmentation image data includes segmented single chromosome image data; The chromosome classification model is used to classify and arrange the single chromosome image data in the chromosome segmentation image data to obtain the chromosome karyotype analysis results.

6. The system according to claim 5, characterized in that The chromosome segmentation model adopts a U-Net network structure. It includes a symmetrically designed encoder module and a decoder module. A feature dimension enhancement module is set between the encoder module and the decoder module. The feature dimension enhancement module is used to perform a parameter-free attention mechanism, perform deep separable convolution processing and pixel reassembly processing on the final layer downsampled feature data, and obtain deep feature optimization data. The feature dimension enhancement module includes a depth-separable convolution unit module, a parameter-free attention unit module, and a pixel reassembly unit module, where: The depth-wise separable convolution unit module is used to perform depth-wise convolution processing and point-by-point convolution processing on the final layer downsampled feature data to obtain deep combined feature data; The parameter-free attention unit module is used to optimize the deep combination feature data based on the energy function of the parameter-free attention mechanism to obtain the optimized deep combination feature data; The pixel reorganization unit module is used to perform pixel reorganization processing on the optimized deep combination feature data to obtain deep feature optimization data.

7. The system according to claim 6, characterized in that The feature dimension enhancement module further includes a first normalization unit module, a nonlinear activation layer and a second normalization unit module, wherein: The first normalization unit module is used to perform batch normalization processing on the final layer downsampled feature data to obtain batch normalized data; The nonlinear activation layer is used to process the batch normalized data using a nonlinear activation function to obtain activation feature map data; The depth-wise separable convolution unit module is used to perform depth-wise convolution processing and point-by-point convolution processing on the activation feature map data to obtain convolution feature data; The second normalization unit module is used to perform batch normalization processing on the convolution feature data to obtain deep combined feature data.

8. The system according to claim 5, wherein: The chromosome segmentation model also includes a convolution kernel adjustment module; The convolution kernel adjustment module is used to extract the features of the input chromosome image data using the standard convolution kernel; calculate the attention weight of each convolution kernel using the attention mechanism; adjust the weight of the convolution kernel according to the attention weight; and replace the standard convolution kernel with the adjusted convolution kernel to obtain a dynamic convolution kernel.

9. The system according to claim 5, characterized in that The chromosome detection model is a Fast R-CNN network structure; and / or the chromosome classification model is a ConvNeXt network structure.

Images