Chromosome recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses a chromosome recognition method and device, electronic equipment and a storage medium. Performing first detection on the chromosome metaphase image based on a preset target detection model to obtain a first bounding box; cutting out a sub-chromosome mid-term image corresponding to the chromosome mid-term image based on the first bounding box, performing primary detection based on the sub-chromosome mid-term image to obtain a second bounding box, obtaining a single chromosome from the chromosome mid-term image based on the second bounding box, and performing first attitude standardization on the single chromosome; inputting the image corresponding to the single chromosome after the first posture standardization into a preset anomaly classification model, and obtaining an identification result of each chromosome in the chromosome metaphase image; wherein when the preset target detection model is trained, medical priori knowledge is fused, and morphological feature constraint conditions are added. The method can improve the efficiency and accuracy of chromosome recognition.

Description

Technical Field

[0001] This application relates to the field of biomedical image processing technology, specifically to a chromosome identification method, device, electronic device, and storage medium. Background Technology

[0002] Chromosome identification is a crucial technique in clinical genetic diagnosis and research, widely used in fields such as genetic disease diagnosis, prenatal screening, and tumor genetics. By observing and analyzing images of chromosome metaphases, the number, structure, and abnormalities of chromosomes can be determined, thereby identifying chromosomal abnormalities.

[0003] Traditional chromosome classification mainly relies on manual microscopic observation and manual classification. However, manual analysis is time-consuming, depends on the operator's experience, and is easily affected by subjective judgment, resulting in poor consistency of results, especially when processing high-throughput samples. Furthermore, chromosome metaphase images often contain complex situations such as overlap, blurring, background impurities, and small-sized targets, which traditional image processing methods find difficult to accurately segment and identify. Summary of the Invention

[0004] In view of the above, this application provides a chromosome identification method, apparatus, electronic device, and storage medium that can efficiently and accurately identify chromosomes from metaphase images.

[0005] To solve the above-mentioned technical problems, the technical solution of this application is implemented as follows:

[0006] In one embodiment, a chromosome identification method is provided, the method comprising:

[0007] Obtain images of chromosome metaphase;

[0008] The coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image are obtained based on a preset target detection model; each first bounding box is a rectangular box of a single chromosome in the chromosome metaphase image; the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training.

[0009] A sub-chromosome metaphase image is cropped from the chromosome metaphase image based on a plurality of first bounding boxes; wherein, the sub-chromosome metaphase image is the image corresponding to the smallest rectangle that includes all the first bounding boxes;

[0010] Based on the preset target detection model, multiple second bounding boxes corresponding to the metaphase image of the subchromosome are obtained; each second bounding box is a rectangular box of a single chromosome in the metaphase image of the subchromosome.

[0011] Based on the second bounding box, obtain the segmentation mask of all chromosomes corresponding to the chromosome metaphase image; based on the segmentation mask, crop out the corresponding single chromosome from the chromosome metaphase image, and perform a first pose normalization on the single chromosome;

[0012] The image corresponding to the single chromosome after the first posture standardization is input into the preset abnormality classification model to obtain the identification result of each chromosome in the chromosome metaphase image; the identification result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

[0013] The acquisition of the preset target detection model includes:

[0014] An initial target detection model is established; wherein, the initial target detection model includes: a feature extraction network, a feature pyramid, and a detection head;

[0015] Obtain a training sample set; the training sample set includes: chromosome metaphase images and medical prior knowledge;

[0016] The initial target detection model is trained using the training sample set. Training ends when the value of the preset loss function is less than the preset loss value, and the preset target detection model is obtained.

[0017] The medical prior knowledge mentioned includes: centromere coordinates, long arm length, short arm length, long arm-to-short arm ratio, and satellite markers;

[0018] The training sample set also includes labels, which include: the category of each chromosome, the center coordinates and width and height of the bounding rectangle of each chromosome, the absolute coordinates of each centromere in the image; the length of the long arm and short arm of each chromosome; the ratio of the long arm to the short arm of each chromosome; and whether satellites are present on each chromosome.

[0019] The feature extraction network embeds a state-space model or a large-kernel selective convolution module.

[0020] For each feature map output by the feature pyramid, the detection head constructs three parallel convolutional branches: a first branch, a second branch, and a third branch. The first branch is used to predict bounding box information; the second branch is used to predict chromosome category confidence; and the third branch is used to predict the relative normalized coordinates of the centromere, the normalized lengths of the long and short arms, the ratio of long to short arms, and the probability of satellite marker presence.

[0021] The preset loss function is a weighted sum of bounding box regression loss, classification loss, distribution focus loss, and morphological loss; wherein, the morphological loss is calculated based on the label and the predicted chromosome category, centromere position, long arm length, short arm length, long-short arm ratio, and whether it is a satellite.

[0022] The method further includes:

[0023] Obtain the segmentation mask, category, anomaly classification, key point coordinates, and polarity direction of each chromosome in the metaphase image of the chromosome;

[0024] The segmentation mask is used to segment the corresponding single chromosome;

[0025] The second pose standardization of the single chromosome is performed using the polarity direction corresponding to the chromosome and the coordinates of the key points;

[0026] Chromosomes are grouped according to their categories, homologous chromosomes are paired, and karyotype arrangement diagrams are generated based on chromosomes standardized in the second pose, with the categories marked; if any chromosome has an abnormal classification, it is marked as an abnormal chromosome.

[0027] Based on the type and number of chromosomes, a karyotype expression is generated.

[0028] This includes obtaining the coordinates and polarity of key points on the chromosome, including:

[0029] Obtain the set of individual chromosomes corresponding to the metaphase image of the chromosome; the set of individual chromosomes includes a first type of chromosome and a second type of chromosome; the first type of chromosomes are chromosomes that have successfully undergone first pose standardization, and the second type of chromosomes are chromosomes that have failed first pose standardization before the first pose standardization was performed;

[0030] Keypoint detection is performed on the first type of chromosomes to obtain keypoint coordinates and a first confidence level; polarity classification is performed on the first type of chromosomes to obtain polarity direction and a second confidence level; for the first type of chromosome, the obtained corresponding keypoint coordinates and first confidence level, as well as polarity direction and second confidence level are fused to obtain updated keypoint coordinates;

[0031] Keypoint detection is performed on the second type of chromosome to obtain keypoint coordinates; and the corresponding polarity direction is determined for the second type of chromosome based on the obtained keypoint coordinates.

[0032] The method further includes:

[0033] The chromosome category, chromosome category confidence score, and segmentation mask for each chromosome corresponding to the metaphase image of the chromosome are obtained; the chromosome category confidence score is obtained based on the preset target detection model when obtaining multiple second bounding boxes corresponding to the metaphase image of the subchromosome.

[0034] Select the chromosome with the highest density of a predetermined number from the chromosomes corresponding to the metaphase images of the chromosomes;

[0035] Based on the selected chromosome category and segmentation mask, a quantity configuration score, global dispersion score, overlap score, sharpness score, and skeleton length score are determined. The quantity configuration score measures chromosome integrity and karyotype representativeness; the global dispersion score measures the spatial layout quality of the chromosomes; the overlap score measures chromosome separability and occlusion; the sharpness score measures contrast, detail richness, edge sharpness, and noise level; and the skeleton length score measures the degree of chromosome morphological unfolding and image scale.

[0036] The quantity configuration score, the global dispersion score, the overlap score, the clarity score, and the skeleton length score are weighted and summed using preset weight values;

[0037] Calculate the average confidence level of the chromosome category confidence level;

[0038] The value obtained by multiplying the average confidence value and the weighted sum is used as the quality assessment value corresponding to the chromosome metaphase image; the higher the quality assessment value, the higher the quality of the chromosome metaphase image.

[0039] The step of selecting the most densely packed chromosomes of a predetermined number from the chromosomes corresponding to the metaphase image includes:

[0040] Calculate the centroid coordinates of each chromosome.

[0041] Construct the Euclidean distance matrix for the centroid coordinates of each pair of chromosomes;

[0042] Calculate the total distance from each chromosome to all other chromosomes based on the Euclidean distance matrix;

[0043] Sort the chromosomes by total distance from smallest to largest, and select the first preset number of chromosomes.

[0044] In another embodiment, a chromosome identification device is provided, the device comprising:

[0045] Acquisition unit, used to acquire images of chromosome metaphase;

[0046] The first detection unit is used to obtain the coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image based on a preset target detection model; each first bounding box is a rectangular box of a single chromosome in the chromosome metaphase image; the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training.

[0047] A cropping unit is configured to crop a sub-chromosome metaphase image from the chromosome metaphase image based on a plurality of first bounding boxes; wherein the sub-chromosome metaphase image is an image corresponding to the smallest rectangle that includes all the first bounding boxes;

[0048] The second detection unit is used to obtain multiple second bounding boxes corresponding to the metaphase image of the subchromosome based on the preset target detection model; each second bounding box is a rectangular box of a single chromosome in the metaphase image of the subchromosome.

[0049] The segmentation unit is used to obtain the segmentation mask of all chromosomes corresponding to the chromosome metaphase image based on the second bounding box; to crop out the corresponding single chromosome from the chromosome metaphase image based on the segmentation mask; and to perform a first pose normalization on the single chromosome.

[0050] The recognition unit is used to input the image corresponding to the single chromosome after the first posture standardization into a preset abnormality classification model to obtain the recognition result of each chromosome in the chromosome metaphase image; the recognition result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

[0051] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a chromosome identification method.

[0052] In another embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a chromosome identification method.

[0053] As can be seen from the above technical solution, in the above embodiments, a first bounding box is obtained by performing a first detection on the chromosome metaphase image based on a preset target detection model; a sub-chromosome metaphase image corresponding to the chromosome metaphase image is cropped based on the first bounding box, and a second bounding box is obtained by performing a second detection on the sub-chromosome metaphase image; a single chromosome is obtained from the chromosome metaphase image based on the second bounding box, and the single chromosome is normalized in a first pose; the image corresponding to the single chromosome after the first pose normalization is input into a preset anomaly classification model to obtain the recognition result of each chromosome in the chromosome metaphase image; wherein, the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training. This detection strategy, which incorporates biological characteristics and performs two detections, effectively reduces background interference and false negative rate by focusing on local regions, thereby improving the accuracy of chromosome localization; chromosome recognition based on this processing can improve the efficiency and accuracy of chromosome recognition. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the process for obtaining the preset target detection model in the embodiments of this application;

[0055] Figure 2 This is a schematic diagram of the initial target detection model structure in the embodiments of this application;

[0056] Figure 3 This is a schematic diagram of the chromosome identification process in an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the smallest rectangle corresponding to the metaphase image of the sub-chromosome in the embodiments of this application;

[0058] Figure 5 This is a schematic diagram illustrating the generation process of the core permutation diagram and kernel type expression in the embodiments of this application;

[0059] Figure 6 This is a flowchart illustrating the process of obtaining the coordinates and polarity of key points of chromosomes in an embodiment of this application.

[0060] Figure 7 This is a schematic diagram of the karyotype arrangement in an embodiment of this application;

[0061] Figure 8 This is a schematic diagram of the quality assessment process for chromosome metaphase images in the embodiments of this application;

[0062] Figure 9 This is a schematic diagram of the chromosome recognition device in the embodiments of this application;

[0063] Figure 10 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe the order or sequence of objects. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0066] The technical solution of the present invention will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0067] Traditional chromosome classification mainly relies on manual microscopic observation and manual classification. However, manual analysis is time-consuming, depends on the operator's experience, and is easily affected by subjective judgment, resulting in poor consistency of results, especially when processing high-throughput samples. Furthermore, chromosome metaphase images often contain complex situations such as overlap, blurring, background impurities, and small-sized targets, which traditional image processing methods find difficult to accurately segment and identify.

[0068] To address the aforementioned technical issues, this application provides a chromosome identification method. First, a pre-defined target detection model is trained by integrating prior medical knowledge and adding morphological feature constraints. Then, this model performs two detections: after the initial target detection, a metaphase image of the sub-chromosomes is obtained based on bounding box cropping and subjected to a second, refined detection. This detection strategy, incorporating biological characteristics and performing two detections, effectively reduces background interference and false negatives by focusing on local regions, thus improving the accuracy of chromosome localization. Based on the chromosome segmentation mask obtained from the pre-defined detection target, chromosome images of each chromosome are acquired, and finally, the results are input into a pre-defined anomaly classification model to obtain classification results. This method improves the efficiency and accuracy of chromosome identification.

[0069] The chromosome identification process in the embodiments of this application is described in detail below with reference to the accompanying drawings.

[0070] In this embodiment of the application, a preset target detection model needs to be trained and obtained before chromosome identification. The specific training process is given below:

[0071] See Figure 1 , Figure 1 This is a schematic diagram illustrating the process of acquiring the preset target detection model in an embodiment of this application. The specific steps are as follows:

[0072] Step 101: Establish an initial target detection model; wherein, the initial target detection model includes: a feature extraction network, a feature pyramid, and a detection head.

[0073] See Figure 2 , Figure 2 This is a schematic diagram of the initial target detection model structure in an embodiment of this application. For non-rigid targets such as chromosomes, the receptive field of traditional 3×3 convolution is limited. Figure 2 The feature extraction network embeds a state-space model or a large-kernel selective convolution module; specifically, a Cross-Stage Partial (CSP) structure can be used as the basic unit. It includes four stages: Feature Extraction CNN-Stage 1, Feature Extraction CNN-Stage 2, Feature Extraction CNN-Stage 3, and Feature Extraction CNN-Stage 4. In the deep feature layer, i.e., Feature Extraction CNN-Stage 4, a continuous state-space sequence model is introduced, utilizing its linearly complex global attention mechanism to capture the complete skeletal features of chromosomes from telomeres to centromeres, reducing feature fragmentation caused by chromosome breakage or overlap. Feature Extraction CNN-Stage 3 outputs high-resolution, low-semantic features, while Feature Extraction CNN-Stage 4 outputs low-resolution, high-semantic features.

[0074] The outputs of feature extraction CNN-Stage 2, feature extraction CNN-Stage 3, and feature extraction CNN-Stage 4 are all input into the feature pyramid, and the feature pyramid outputs a hybrid multi-scale feature.

[0075] For each layer of the mixed multi-scale feature map output by the feature pyramid, the detection head constructs three parallel convolutional branches: the first branch, the second branch, and the third branch. The first branch predicts bounding box information; the second branch predicts chromosome class confidence; and the third branch predicts chromosome morphological information, including: the relative normalized coordinates of the centromere, the normalized long and short arm lengths, the long and short arm ratio, and the probability of satellite marker presence. Unlike related implementations, a third branch is added. For branch one: it outputs 4×Reg_max channels for predicting bounding box information; branch two: it outputs Nclass channels for predicting chromosome class confidence; branch three: it consists of stacked convolutional layers, outputting 6 channels; channels 1-2 predict the relative normalized coordinates of the centromere; channels 3-4 predict the normalized long and short arm lengths; channel 5 predicts the long and short arm ratio; and channel 6 predicts the probability of satellite marker presence.

[0076] Step 102: Obtain the training sample set; the training sample set includes: chromosome metaphase images and medical prior knowledge.

[0077] In addition to chromosome metaphase images, the training sample set here also includes medical prior knowledge, which includes centromere coordinates, long arm length, short arm length, long arm-to-short arm ratio, and satellite markers.

[0078] The training sample set also includes labels, which are used to calculate the loss function during model training.

[0079] The labels include: the category of each chromosome, the center coordinates and dimensions of the bounding rectangle of each chromosome, the absolute coordinates of each centromere in the image, the length of the long arm and short arm of each chromosome, the ratio of the long arm to the short arm of each chromosome, and whether satellites are present on each chromosome.

[0080] Step 103: Train the initial object detection model using the training sample set. End the training when the function value of the preset loss function is less than the preset loss value, and obtain the preset object detection model.

[0081] In this embodiment, the preset loss function is a weighted sum of bounding box regression loss, classification loss, distribution focus loss, and morphological loss; wherein, the morphological loss is calculated based on the label and the predicted chromosome category, centromere position, long arm length, short arm length, long-short arm length ratio, and whether it is a satellite.

[0082] A label can be represented as: [C, B] x, B y B w B h , P cx , P cy , L long , L short , R arm, R sat ];

[0083] in:

[0084] C: The category of each chromosome, such as autosomes 1-22 and X and Y sex chromosomes;

[0085] B x B y B w B h : The center coordinates, width, and height of the bounding rectangle of each chromosome (normalized to 0-1);

[0086] P cx , P cy : The absolute coordinates of each centromere in the image;

[0087] L long , L short, : The lengths of the long and short arms of each chromosome, where the lengths refer to their physical lengths;

[0088] R arm The ratio of the long arm to the short arm of each chromosome, i.e., the ratio of the length of the long arm to the length of the short arm, is calculated using the formula L. long / L short ;

[0089] R sat Does each chromosome contain satellites?

[0090] Based on the above letter representation, the specific representation of the preset loss function is given as follows:

[0091] ;

[0092] in, Indicates the preset loss function. This represents the bounding box regression loss. Represents classification loss, Indicates the distribution focus loss. Indicates morphological loss;

[0093] The weights for the bounding box regression loss, classification loss, distribution focus loss, and morphological loss are as follows: , , , .

[0094] In this application, no specific restrictions are placed on the specific implementation functions of the bounding box regression loss and the classification loss. The CIoU loss can be used to calculate the bounding box regression loss, and the Focal loss can be used to calculate the classification loss.

[0095] The distribution focus loss can be, but is not limited to, expressed as follows:

[0096] ;

[0097] in, Represents the normalized continuous coordinate values ​​of the true bounding box; and The closest The two discrete integer points on the left and right (i.e. ); and This formula predicts the probability that the network boundary lies between these two points. It utilizes the cross-entropy form, expressed as the distance difference. and As linear weighting coefficients. When the true value The closer to the left integer ,coefficient The larger the network, the more it forces the network to improve. The predicted probability is calculated by the sum of the probability distributions of two discrete points, and vice versa. This weighting mechanism enables the network to unbiasedly reconstruct continuous floating-point coordinates from the probability distributions of two discrete points, thereby achieving super-resolution boundary localization.

[0098] Morphological loss term ( The calculation is performed only on positive samples, i.e., anchor points where the IoU between the predicted bounding box and the ground truth bounding box is greater than a threshold, and the mean squared error (MSE) is used as the basic function, specifically expressed as:

[0099] ;

[0100] in, These are the predicted and actual vectors of the centromere position, respectively.

[0101] It includes both predicted and actual values ​​for the long arm length;

[0102] It includes both predicted and actual values ​​for the short arm length;

[0103] These represent the confidence level in predicting the presence of a satellite and the true value of the satellite label, respectively.

[0104] The true value here corresponds to the label in the training sample.

[0105] These are the weights for each sub-item, used to balance differences in magnitude.

[0106] This application embodiment introduces... During backpropagation, the gradient update of the network is affected not only by the bounding box position but also by the accuracy of the centromere localization. This forces the network to focus on the internal texture of chromosomes during the feature extraction stage, thereby significantly improving the classification accuracy of morphologically similar chromosomes.

[0107] To reduce false positives for impurities, this embodiment optimizes the data augmentation strategy during the target detection model training process. The multi-scale augmentation range is set to 0.8-1.2 to focus on a range closer to the actual chromosome size, reducing sensitivity to background impurities. Training settings include: 200 training epochs, a batch size of 48, a label smoothing strategy to alleviate overfitting, and an HSV color enhancement parameter v set to 0.6 to enhance image robustness. To further optimize the model's adaptability to complex scenes, mosaic data augmentation is disabled for the first 15 epochs to avoid excessive interference from complex backgrounds, while other hyperparameters remain at their default values ​​to ensure model stability. Training uses the Adam optimizer with an initial learning rate of 0.001, combined with a learning rate decay strategy to ensure model convergence and generalization ability.

[0108] At this point, the training of the initial object detection model is complete, and the preset object detection model is obtained.

[0109] When the preset target detection model is used for target detection, it no longer inputs prior chromosome knowledge. The third branch of the output is not used in this embodiment of the application. It can be set not to output, or output but not used. It can also record the output content for use in other places.

[0110] See Figure 3 , Figure 3 This is a schematic diagram of the chromosome identification process in an embodiment of this application. The specific steps are as follows:

[0111] Step 301: Obtain a metaphase image of the chromosome.

[0112] Step 302: Obtain the coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image based on the preset target detection model; each first bounding box is a rectangle of a single chromosome in the chromosome metaphase image.

[0113] When performing object detection for the first time, only the coordinates of the output bounding box need to be obtained.

[0114] Step 303: Cropping a sub-chromosome metaphase image from the chromosome metaphase image based on multiple first bounding boxes; wherein, the sub-chromosome metaphase image is the image corresponding to the smallest rectangle that includes all the first bounding boxes.

[0115] In the specific implementation, the smallest rectangle corresponding to all the first bounding boxes is used as the cropping box to crop the sub-chromosome metaphase image from the original chromosome metaphase image. Then, before performing the second target detection, the sub-chromosome metaphase image is enlarged by a specified factor and the boundary is expanded by a specified number of pixels.

[0116] For this preset target detection model, a confidence threshold of 0.45 and a non-maximum suppression IoU threshold of 0.7 can be set during inference. There are no restrictions on the specific implementation.

[0117] See Figure 4 , Figure 4 This is a schematic diagram of the smallest rectangle corresponding to the metaphase image of the subchromosome in the embodiments of this application. Figure 4 As a type of chromosome metaphase image, after the first detection by the target detection model, the bounding rectangle of each chromosome is the first bounding box output by the preset target detection model. The large box outside all the first bounding boxes is the smallest rectangle that includes all the first bounding boxes. Using this smallest rectangle to crop the chromosome metaphase image, the sub-chromosome metaphase image can be obtained.

[0118] Step 304: Obtain multiple second bounding boxes corresponding to the metaphase images of subchromosomes based on a preset target detection model; each second bounding box is a rectangle of a single chromosome in the metaphase image of subchromosomes.

[0119] The implementation of target detection based on subchromosome metaphase images focuses on local regions, reduces global background interference, and outputs refined chromosome bounding box coordinates and categories.

[0120] Step 305: Obtain the segmentation mask of all chromosomes corresponding to the metaphase image of the chromosome based on the second bounding box; based on the segmentation mask, cut out the corresponding single chromosome from the metaphase image of the chromosome, and perform first pose normalization on the single chromosome.

[0121] Based on the second bounding box, segment each chromosome into a single chromosome image from the metaphase image; input the single chromosome image into the preset segmentation model to obtain the segmentation mask of the chromosome.

[0122] The segmentation mask here can be obtained based on a preset segmentation model. This application does not limit the specific acquisition and implementation structure of the preset segmentation model. For example, it can be based on an advanced instance segmentation deep learning framework, combined with efficient feature extraction and multi-scale information fusion techniques for improvement. The feature extraction backbone network adopts a deep convolutional network with multi-scale feature extraction capabilities to enhance the ability to capture features of complex chromosome morphologies. The segmentation head adopts a structure that integrates multi-scale features, further strengthening the integration capability of multi-scale feature information through top-down and bottom-up feature fusion paths. This is merely an example of a segmentation model structure, and the specific implementation is not limited to this.

[0123] The training settings for this segmentation model can be as follows: random center cropping scale of 0.8, random rotation probability of 0.5, random flipping settings including horizontal flipping, vertical flipping, and diagonal flipping, and random filling to a square. To adapt to the characteristics of biomedical images, random Gamma transformation, random Gaussian noise, and Gaussian blur are set, while other hyperparameters are kept at their default values ​​to optimize model performance. These settings are only examples and are not limited in this embodiment.

[0124] The pose normalization of a single chromosome involves cropping the main chromosome region from the single chromosome image, determining the rotation angle through contour extraction and minimum bounding rectangle calculation, applying affine transformation to normalize the chromosome orientation, expanding the cropped region to preserve the complete chromosome, and outputting the rotated and cropped image along with a mask. This normalization process may succeed or fail. If it fails, the unnormalized chromosome image is used for subsequent processing; if successful, the normalized chromosome image is used for subsequent processing.

[0125] Step 306: Input the image corresponding to a single chromosome after the first pose standardization into the preset abnormality classification model to obtain the recognition result of each chromosome in the chromosome metaphase image; the recognition result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

[0126] The anomaly classification model used in this embodiment can be a relevant implementation model, or a preset anomaly classification model built and trained based on the image classification deep learning framework in this embodiment. The preset anomaly classification model in this embodiment is optimized for the characteristics of various chromosomal anomaly types. The algorithm uses an efficient visual transformer network as the backbone feature extractor, extracting deep semantic features of chromosome images through multi-layer transformer blocks and hierarchical feature aggregation mechanisms. The network architecture is designed as a 12-layer small variant to achieve a balance between computational efficiency and classification accuracy. DropPath is set to 0.2 to enhance the model's generalization ability. Preprocessing includes normalization to adapt to the dynamic range of biomedical images. To improve classification robustness, a hybrid enhancement strategy is also introduced, generating soft targets through linear interpolation samples and labels to reduce overfitting and improve boundary decisions.

[0127] Mixed precision can be used to accelerate training, reducing memory consumption and speeding up computation. Training settings include: batch size 192, AdamW variant optimizer, initial learning rate 1e-3, default weight decay of 0.05, and support for gradient clipping. The learning rate scheduler uses cosine annealing. Training data augmentation strategies include random scaling and clipping, horizontal flipping, color dithering, and random erasure. This is a pre-defined anomaly classification model; specific implementations are not limited to this.

[0128] In this embodiment, a pre-defined target detection model is first trained by integrating prior medical knowledge and adding morphological feature constraints for two-stage detection. After the initial target detection, the metaphase image of the chromosome is cropped based on the bounding box and a second refinement detection is performed. This detection strategy, which incorporates biological characteristics and performs two-stage detection, effectively reduces background interference and false negative rate by focusing on local regions, thus improving the accuracy of chromosome localization. Based on the chromosome segmentation mask obtained from the pre-defined detection target, chromosome images of each chromosome are acquired, and finally, the images are input into a pre-defined anomaly classification model to obtain the classification results. This method can improve the efficiency and accuracy of chromosome identification.

[0129] This application also provides a method for generating karyotype arrangement maps based on chromosome metaphase images, and the specific implementation process is given below.

[0130] See Figure 5 , Figure 5 This is a schematic diagram illustrating the generation process of the core permutation diagram and kernel type expression in an embodiment of this application. The specific steps are as follows:

[0131] Step 501: Obtain the segmentation mask, category, anomaly classification, key point coordinates, and polarity direction of each chromosome in the metaphase image.

[0132] The segmentation mask, category, and abnormal classification of each chromosome corresponding to the obtained metaphase chromosome image in this step can be obtained based on the method in steps 301 to 305.

[0133] The coordinates and polarity of key points on a chromosome can be obtained in the following way:

[0134] See Figure 6 , Figure 6 This is a flowchart illustrating the process of obtaining the coordinates and polarity of key points on chromosomes in an embodiment of this application. The specific steps are as follows:

[0135] Step 601: Obtain the set of single chromosomes corresponding to the metaphase chromosome image; the set of single chromosomes includes a first type of chromosome and a second type of chromosome; the first type of chromosomes are chromosomes that have been successfully standardized in the first pose, and the second type of chromosomes are chromosomes that have failed to be standardized in the first pose before the first pose standardization was performed.

[0136] The key point coordinates and polarity directions of the first and second types of chromosomes are obtained through different methods in the following steps.

[0137] Step 602: Perform keypoint detection on the first type of chromosomes to obtain keypoint coordinates and first confidence level; and perform polarity classification on the first type of chromosomes to obtain polarity direction and second confidence level.

[0138] For the first type of chromosome, it is necessary to obtain the polarity direction of the chromosome in order to adjust the coordinates of the key points of the chromosome. Specifically, this can be achieved as follows:

[0139] The keypoint detection model is used to determine the keypoint coordinates and first confidence level of the first type of chromosome, that is, to locate the three keypoints at the end of the short arm, the centromere, and the end of the long arm of the chromosome and their corresponding first confidence levels.

[0140] The polarity direction and confidence level of the first type of chromosome are obtained through a polarity detection model.

[0141] Step 603: For the first type of chromosome, fuse the obtained keypoint coordinates, first confidence score, polarity direction, and second confidence score to obtain updated keypoint coordinates. End this process.

[0142] In this embodiment, the key point coordinates and polarity directions of all first-type chromosomes are not forcibly fused, but the following prerequisites must be met:

[0143] The second confidence level is greater than the preset confidence threshold, where the preset confidence threshold is set according to the actual application scenario, such as 0.7; but not limited to this.

[0144] The fusion function is enabled.

[0145] If both of the above conditions are met, it is determined that the fusion correction process needs to be initiated; otherwise, the kernel arrangement diagram is generated using the directly determined key point coordinates and polarity direction.

[0146] Based on the polarity direction and the coordinates of key points corrected by the segmentation mask, ensuring chromosome polarity consistency, the fusion correction process begins, specifically as follows:

[0147] Calculate the current pose angle: Extract the coordinates of the short arm end and long arm end from the keypoints. Calculate the direction vector from the short arm to the long arm through vector operations, and convert it into a normalized angle of 0-360 degrees, which serves as the chromosome's current predicted "pose angle".

[0148] Calculate the angle difference: Compare the predicted "attitude angle" with the reference polarity angle, and calculate the minimum angle difference between the two, i.e., the absolute difference. Considering the periodicity of the angle, when the difference exceeds 180 degrees, perform angle compensation (360 - difference) to obtain the minimum rotation angle. Here, the reference polarity angle is the polarity direction obtained through the polarity detection model.

[0149] Trigger judgment: Set an angle tolerance threshold, such as 15 degrees.

[0150] When the absolute difference is not greater than the preset angle tolerance threshold, the key point prediction is considered accurate, and the original key points are directly output without correction.

[0151] When the absolute difference is greater than the preset angle tolerance threshold, it is considered that there may be a deviation in the prediction of key points, and the subsequent correction process is triggered.

[0152] Once the correction is triggered, perform the following steps to fix the critical points:

[0153] Determine the desired direction: Construct a unit direction vector using trigonometric functions based on the reference polarity angle.

[0154] Search for the extreme points of the contour:

[0155] First, obtain the segmentation mask outline point set of the chromosome;

[0156] Then calculate the vector of each contour point relative to the chromosome centroid.

[0157] Then, calculate the projection length (dot product) of all vectors in the "desired direction". The value of this dot product represents the projection distance of points on the contour onto the polarity axis.

[0158] Finally, among all projected values, identify the contour points with the largest (farthest point in the positive direction) and the smallest (farthest point in the negative direction). The contour points corresponding to these two indices represent the points at both ends of the target on the polarity axis.

[0159] Keypoint remapping: Based on the salience of the projected values, compare the absolute values ​​of the maximum and minimum projected values ​​of the two found contour extrema. If the maximum projected value (positive direction) is more significant, mark the positive direction extrema as the new head keypoint and the negative direction extrema as the new tail keypoint. Otherwise, swap their assignments; that is, mark the negative direction extrema as the new head keypoint and the positive direction extrema as the new tail keypoint. Update the short arm end keypoint and long arm end keypoint with the contour extrema found above, while keeping the centromere keypoint as the centroid coordinate.

[0160] Coordinate mapping: The updated keypoint coordinates in the mask coordinate system are finally mapped back to the coordinates corresponding to the original chromosome metaphase image.

[0161] The first confidence level primarily serves as a reliability reference for the keypoint coordinates. When no correction is triggered, it supports direct output of the original results; during the correction process of finding contour extrema, it implicitly plays a role by relying on a high-precision segmentation mask.

[0162] The second confidence level is a prerequisite for deciding whether to proceed to the subsequent judgment and correction process. It ensures that the polarity direction prediction is only used to correct or verify the key point coordinates when it is sufficiently reliable, thus avoiding errors introduced by low-quality predictions.

[0163] Step 604: Perform keypoint detection on the second type of chromosome to obtain keypoint coordinates.

[0164] The keypoint detection model is used to determine the coordinates of keypoints on the second type of chromosome, namely, to locate the three keypoints at the end of the short arm, the centromere, and the end of the long arm of the chromosome.

[0165] Step 605: For the second type of chromosome, determine the corresponding polarity direction based on the obtained key point coordinates.

[0166] The process of determining the polarity direction based on the coordinates of key points can be summarized as follows:

[0167] Construct the direction vector: Calculate the vector from the end of the short arm to the end of the long arm based on the coordinates of the key points.

[0168] Calculate the angle: Use the arctangent function, such as atan2, to calculate the angle of the above vector in the two-dimensional coordinate system of the image. This angle reflects the direction of the chromosome's long axis in the image plane.

[0169] Angle normalization: The calculated angle value is converted and normalized to a standard angle space of 0-360 degrees. The normalized angle is the determined polarity direction.

[0170] Step 502: Use a segmentation mask to segment out the corresponding single chromosome.

[0171] Step 503: Using the polarity direction and key point coordinates of the chromosome, perform second pose standardization on a single chromosome.

[0172] The second posture standardization process is as follows: the profile is rotated using the polarity direction so that the short arm faces upward and the long arm faces downward to achieve posture standardization.

[0173] Step 504: Group chromosomes according to their categories, pair homologous chromosomes, generate a karyotype arrangement map based on the chromosomes after second pose standardization, and label the categories; if there are abnormal classifications of chromosomes, then mark the chromosomes as abnormal chromosomes.

[0174] See Figure 7 , Figure 7 This is a schematic diagram of the karyotype arrangement diagram in an embodiment of this application. The karyotype arrangement diagram is grouped by category, with chromosome pairs 1 to 22 arranged sequentially, each pair horizontally side-by-side. Then, the X and Y chromosomes are arranged vertically using a grid layout to ensure uniform spacing, generating the karyotype arrangement diagram. The standardized outline of each chromosome is placed in its corresponding position on the canvas, and a category label is added to each chromosome. Each chromosome is labeled using the abnormality classification results. If classified as a certain abnormal type, it is marked using a specific symbol in the arrangement diagram, such as... Figure 7 The red arrows are used to indicate exceptions, and specific exception types can also be marked in the implementation.

[0175] Step 505: Generate a karyotype expression based on the type and number of chromosomes.

[0176] For example, in humans, a normal male is "46, XY" and a female is "46, XX". If the numerical value is abnormal, add a description, such as "47, XY, +21" to indicate trisomy 21; if the structure is abnormal, add a description based on the abnormality classification result, such as "46, XY, t(14; 21)".

[0177] In this embodiment, high-quality chromosomes are obtained from metaphase chromosome images based on secondary target detection, and the coordinates of chromosome key points are accurately determined by key point detection and polarity classification fusion, which significantly improves the robustness and accuracy of polarity classification. Furthermore, the karyotype arrangement diagram and karyotype expression are automatically generated, and abnormal chromosomes are displayed. Compared with the traditional time-consuming and inefficient karyotype analysis process, this method can improve the accuracy of karyotype arrangement diagram and karyotype expression generation while saving costs.

[0178] This application also provides a method for quality assessment of chromosome metaphase images, the specific process of which is as follows:

[0179] See Figure 8 , Figure 8 This is a schematic diagram of the quality assessment process for chromosome metaphase images in an embodiment of this application. The specific steps are as follows:

[0180] Step 801: Obtain the chromosome category, chromosome category confidence score, and segmentation mask for each chromosome corresponding to the metaphase image of the chromosome; the chromosome category confidence score is obtained when multiple second bounding boxes corresponding to the metaphase image of the sub-chromosome are obtained based on the preset target detection model.

[0181] The chromosome category and chromosome category confidence level of each chromosome corresponding to the chromosome image obtained here can be obtained through two target detections in this application embodiment; or it can be obtained by other methods, and this application embodiment does not impose any restrictions on this.

[0182] Step 802: Select the chromosome with the highest density of a preset number of chromosomes from the chromosomes corresponding to the metaphase image of the chromosome.

[0183] In practice, the preset number of chromosomes can be set to 46. The specific implementation of selecting the chromosome with the highest preset number of chromosomes from the chromosomes corresponding to the metaphase image in this step is as follows:

[0184] First, calculate the centroid coordinates of each chromosome;

[0185] Then construct the Euclidean distance matrix of the centroid coordinates of each pair of chromosomes;

[0186] Then, the total distance from each chromosome to all other chromosomes is calculated based on the Euclidean distance matrix;

[0187] Finally, sort the chromosomes by their total distance from smallest to largest, and select the first preset number of chromosomes.

[0188] Step 803: Determine the quantity configuration score, global dispersion score, overlap score, sharpness score, and skeleton length score based on the selected chromosome category and segmentation mask. Among them, the quantity configuration score is used to measure the chromosome integrity and karyotype representativeness of the image; the global dispersion score is used to measure the spatial layout quality of the chromosomes; the overlap score is used to measure the separability and occlusion of the chromosomes; the sharpness score is used to measure contrast, detail richness, edge sharpness, and noise level; and the skeleton length score is used to measure the morphological unfolding of the chromosomes and the image scale.

[0189] The specific method for obtaining the quantity configuration score is as follows:

[0190] The number configuration dimension is based on 46 chromosomes. If the total number is exactly 46, the base score is 1.0. The score decreases linearly to 0.7-0.9 for 42 to 50 chromosomes, to 0.4-0.8 for 30 to 42 chromosomes, to 0.2-0.4 for 20 to 30 chromosomes, and to 0.1 for the rest.

[0191] Additional bonus: A maximum of 0.3 points are awarded for each type of autosome with 2 chromosomes, and 0.2 points are awarded for XX or XY sex chromosome combinations;

[0192] If the types and numbers of autosomes and sex chromosomes follow the standard rules, that is, when the total number of chromosomes is exactly 46, the number of all autosome types is 2, and the combination of sex chromosomes is XX or XY, an extra 0.3 points are awarded.

[0193] The total score for quantity configuration is limited to 1.5 points. If the score exceeds 1.5 points, it will still be counted as 1.5 points.

[0194] The global dispersion score is obtained as follows:

[0195] First, calculate three sub-indicators: distance ratio, distance coefficient of variation, and centroid coverage; and set corresponding weights for each, such as 0.4, 0.3, and 0.3, but not limited to these.

[0196] The distance ratio is calculated as follows: the centroid distance between each chromosome in the image and its nearest neighbor chromosome is calculated; the minimum of all the above distances is denoted as D_min; the ideal distance is calculated as: D_ideal = sqrt(total image area / 46). 0.8.

[0197] The distance ratio is = D_min / D_ideal, and the closer the distance ratio is to 1, the better.

[0198] Coefficient of variation: The ratio of the standard deviation to the mean of the centroid distances between all pairs of chromosomes. The smaller the value, the more uniform the distribution.

[0199] Centroid coverage: Project the centroids of all chromosomes onto the X-axis (width direction) and Y-axis (height direction) of the image, respectively.

[0200] Calculate the proportion of the pixel range covered by the projection point in the X-axis direction to the total width of the image, denoted as C_x.

[0201] Calculate the proportion of the pixel range covered by the projection point in the Y-axis direction to the total height of the image, denoted as C_y.

[0202] Centroid coverage = (min(C_x,0.8) / 0.8 + min(C_y,0.8) / 0.8) / 2. 0.8 is the target coverage set.

[0203] Global dispersion score = (distance ratio × 0.4) + ((1 - distance coefficient of variation) × 0.3) + (centroid coverage × 0.3).

[0204] Overlap score is used to assess the severity of occlusion between chromosomes. The overlap score is obtained as follows:

[0205] The degree of overlap is based on the ratio of the mask intersection area to the smaller area. The overlap ratio, average overlap intensity, and severe overlap (greater than 30%) ratio are statistically analyzed and can be set with weight penalties of 0.4, 0.4, and 0.2 respectively, but are not limited to these.

[0206] Calculate overlapping pairs: Traverse all chromosome pairs (i, j) and calculate the area of ​​intersection of their segmentation masks, Area_intersect.

[0207] Calculate three penalty sub-indices:

[0208] Overlapping pair ratio (weight 0.4): Number of overlapping chromosome pairs / Total number of chromosome pairs.

[0209] Average overlap intensity (weight 0.4): For all chromosome pairs that overlap, calculate the overlap intensity = Area_intersect / min(Area_i, Area_j), and then calculate the average of all overlap intensities.

[0210] Severe overlap ratio (weight 0.2): Calculate the proportion of chromosome pairs with an overlap intensity > 30% to the total number of chromosome pairs.

[0211] Overlap score = 1.0 - (overlap ratio) 0.4+ average overlap strength 0.4+ severe overlap ratio 0.2);

[0212] A higher score indicates less overlap. This is a penalty metric, deducted from the maximum score of 1.0.

[0213] The specific process for obtaining image sharpness scores is as follows:

[0214] Calculate four feature sub-indicators:

[0215] Texture contrast (weight 0.15): Calculate the contrast of the gray-level co-occurrence matrix (GLCM) in the four directions of 0°, 45°, 90°, and 135° within the union region, take the average value, and then divide by 100 for normalization.

[0216] High-frequency energy (weight 0.15): Perform db1 wavelet first-level decomposition within the union region, calculate the sum of the energy of the high-frequency subbands (HL, LH, HH), and then normalize by dividing by 1000.

[0217] Edge intensity standard deviation (weight 0.35): Calculate the edge intensity image after Sobel operator processing within the union region, and then calculate the standard deviation of pixel intensity within that region as a sub-index value.

[0218] Black and white contrast ratio (weight 0.35):

[0219] Calculate the average gray level avg_gray within the union region.

[0220] Calculate the standard deviation of gray levels std_gray within the union region.

[0221] Black and white contrast ratio = 0.6 (1 / avg_gray)+0.4 (std_gray / 128), assuming the image is 8-bit grayscale, 255 is the whitest.

[0222] Image sharpness score = (texture contrast) 0.15) + (High-frequency energy) 0.15) + (Standard deviation of edge strength) 0.35) + (Black and white contrast) 0.35);

[0223] The skeleton length score is used to assess whether the chromosome morphology is sufficiently extended. It is calculated based on its key points, and the specific acquisition process is as follows:

[0224] Calculate the total length: For each chromosome:

[0225] Calculate the pixel distance L_short from the end of the short arm to the centromere.

[0226] Calculate the pixel distance L_long from the centromere to the end of the long arm.

[0227] The skeletal length of this chromosome = L_short + L_long.

[0228] Total skeleton length = sum of the skeleton lengths of all chromosomes.

[0229] Score mapping:

[0230] Set length thresholds: L_low = 1000 pixels, L_high = 5000 pixels.

[0231] If the total length is less than or equal to L_low, the score is 1.0.

[0232] If the total length >= L_high, the score = 1.5.

[0233] If L_low < total length < L_high, the score is linearly interpolated between 1.0 and 1.5:

[0234] Skeleton length score = 1.0 + 0.5 (total length - L_low) / (L_high - L_low).

[0235] Step 804, use the preset weight values to weight and sum the quantity configuration score, global dispersion score, overlap degree score, clarity score, and skeleton length score.

[0236] In specific implementation, the weights of each index are pre-configured, and the specific values are not limited. For example, the weights set for the quantity configuration score, global dispersion score, overlap degree score, clarity score, and skeleton length score are: 0.23, 0.17, 0.20, 0.06, and 0.35 respectively.

[0237] Step 805, calculate the average confidence value of the chromosome category confidence.

[0238] The chromosome category confidence is obtained during the second target detection. The average value of the chromosome category confidences corresponding to all chromosomes is used as the adjustment factor.

[0239] Step 806, use the value obtained by multiplying the average confidence value by the weighted sum value as the quality evaluation value corresponding to the chromosome metaphase image; the higher the quality evaluation value, the higher the quality of the chromosome metaphase image.

[0240] The final quality evaluation value is obtained by multiplying the average confidence value by the weighted sum value. In specific implementation, the quality evaluation value can also be scaled by 100 times and then truncated to 0 - 150 points to achieve an objective evaluation of the image integrity, distribution uniformity, clarity, and morphological integrity, providing a reliable basis for image screening and priority ranking for high-throughput clinical karyotype analysis.

[0241] In the embodiments of this application, it aims to integrate object detection, instance segmentation, pose normalization, key point detection, polarity classification, anomaly classification, and image quality evaluation that incorporate prior medical knowledge through a multi-stage cascade processing framework to achieve fully automatic, high-precision, and high-efficiency chromosome karyotype analysis. The system solves problems such as chromosome overlap, inconsistent directions, and difficult anomaly recognition through optimized algorithm design and multi-modal fusion;

[0242] Furthermore, by employing technologies such as secondary refinement detection and key point and polarity fusion, the accuracy of localization, segmentation and classification is significantly improved, the false detection and false negative rates are effectively reduced, and multi-dimensional high-quality image sorting ensures that the analysis prioritizes clear, evenly distributed and non-overlapping images, thereby improving diagnostic reliability.

[0243] Simultaneously, a comprehensive quality assessment mechanism is introduced to screen high-quality images to improve the reliability of the analysis. This application is applicable to clinical genetic diagnosis, with significant advantages in automation and standardization, providing an efficient solution for high-throughput sample analysis and genetic disease diagnosis.

[0244] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0245] Based on the same inventive concept, this application also provides a chromosome identification device. See also Figure 9 , Figure 9 This is a schematic diagram of the chromosome recognition device in an embodiment of this application. The device includes:

[0246] Acquisition unit 901 is used to acquire images of chromosome metaphase;

[0247] The first detection unit 902 is used to obtain the coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image based on a preset target detection model; each first bounding box is a rectangular box of a single chromosome in the chromosome metaphase image; the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training.

[0248] The cropping unit 903 is used to crop out a sub-chromosome metaphase image from a chromosome metaphase image based on multiple first bounding boxes; wherein, the sub-chromosome metaphase image is the image corresponding to the smallest rectangle that includes all the first bounding boxes;

[0249] The second detection unit 904 is used to obtain multiple second bounding boxes corresponding to the metaphase image of the subchromosome based on a preset target detection model; each second bounding box is a rectangular box of a single chromosome in the metaphase image of the subchromosome.

[0250] The segmentation unit 905 is used to obtain the segmentation mask of all chromosomes corresponding to the metaphase image of the chromosome based on the second bounding box; to crop out the corresponding single chromosome from the metaphase image of the chromosome based on the segmentation mask; and to perform a first pose normalization on the single chromosome.

[0251] The recognition unit 906 is used to input the image corresponding to a single chromosome after the first posture standardization into a preset abnormality classification model to obtain the recognition result of each chromosome in the chromosome metaphase image; the recognition result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

[0252] In another example,

[0253] The acquisition of the preset target detection model includes:

[0254] An initial target detection model is established; the initial target detection model includes: a feature extraction network, a feature pyramid, and a detection head;

[0255] Obtain the training sample set; the training sample set includes: chromosome metaphase images and medical prior knowledge;

[0256] The initial object detection model is trained using the training sample set. Training ends when the value of the preset loss function is less than the preset loss value, and the preset object detection model is obtained.

[0257] In another example,

[0258] Prior medical knowledge includes: centromere coordinates, long arm length, short arm length, long arm-to-short arm ratio, and satellite markers;

[0259] The training sample set also includes labels, which include: the category of each chromosome, the center coordinates and width and height of the bounding rectangle of each chromosome, the absolute coordinates of each centromere in the image, the length of the long arm and short arm of each chromosome, the ratio of the long arm to the short arm of each chromosome, and whether satellites are present on each chromosome.

[0260] Embedding state-space models or large-kernel selective convolutional modules in feature extraction networks;

[0261] For each feature map output by the feature pyramid, the detection head constructs three parallel convolutional branches: the first branch, the second branch, and the third branch. The first branch is used to predict bounding box information; the second branch is used to predict chromosome category confidence; and the third branch is used to predict the relative normalized coordinates of the centromere, the normalized lengths of the long and short arms, the ratio of long to short arms, and the probability of satellite marker presence.

[0262] The preset loss function is a weighted sum of bounding box regression loss, classification loss, distribution focus loss, and morphological loss; among which, the morphological loss is calculated based on the label and the predicted chromosome category, centromere position, long arm length, short arm length, long-short arm ratio, and whether it is a satellite.

[0263] In another example, the device further includes:

[0264] The generation unit 907 is used to acquire the segmentation mask, category, abnormality classification, key point coordinates, and polarity direction of each chromosome in the metaphase image of chromosomes; segment the corresponding single chromosomes using the segmentation mask; perform second pose normalization on the single chromosomes using the corresponding polarity direction and key point coordinates; group chromosomes according to their categories, pair homologous chromosomes, generate a karyotype arrangement map based on the chromosomes after second pose normalization, and label the categories; if there are abnormal classifications of chromosomes, the chromosomes are marked as abnormal chromosomes; and generate a karyotype expression based on the category and number of chromosomes.

[0265] In another example,

[0266] The generation unit 907 is specifically used to acquire a set of single chromosomes corresponding to the metaphase image of chromosomes when obtaining the keypoint coordinates and polarity direction of chromosomes. The set of single chromosomes includes a first type of chromosome and a second type of chromosome. The first type of chromosomes are chromosomes that have successfully undergone first-position standardization, and the second type of chromosomes are chromosomes that failed first-position standardization before undergoing first-position standardization. Keypoint detection is performed on the first type of chromosomes to obtain keypoint coordinates and a first confidence score. Polarity classification processing is then performed on the first type of chromosomes to obtain the polarity direction and a second confidence score. For the first type of chromosomes, the obtained corresponding keypoint coordinates, first confidence score, polarity direction, and second confidence score are fused to obtain updated keypoint coordinates. Keypoint detection is performed on the second type of chromosomes to obtain keypoint coordinates. The corresponding polarity direction is determined for the second type of chromosome based on the obtained keypoint coordinates.

[0267] In another example, the device further includes:

[0268] Evaluation unit 908 is used to acquire the chromosome category, chromosome category confidence score, and segmentation mask for each chromosome corresponding to the metaphase image of chromosomes. The chromosome category confidence score is obtained when multiple second bounding boxes corresponding to the metaphase images of subchromosomes are acquired based on a preset target detection model. A preset number of the densest chromosomes are selected from the chromosomes corresponding to the metaphase images. Based on the selected chromosome category and segmentation mask, a quantity configuration score, global dispersion score, overlap score, sharpness score, and skeleton length score are determined. The quantity configuration score measures the chromosome integrity and karyotype representativeness of the image; the global dispersion score measures the chromosome... Spatial layout quality; overlap score is used to measure chromosome separability and occlusion; sharpness score is used to measure contrast, detail richness, edge sharpness, and noise level; skeleton length score is used to measure chromosome morphological unfolding and image scale; the quantity configuration score, global dispersion score, overlap score, sharpness score, and skeleton length score are weighted and summed using preset weight values; the average confidence value of chromosome category confidence is calculated; the average confidence value is multiplied by the weighted sum to obtain the quality assessment value corresponding to the chromosome metaphase image; the higher the quality assessment value, the higher the quality of the chromosome metaphase image.

[0269] In another example,

[0270] The evaluation unit 908 is specifically used to select the most densely packed chromosomes of a preset number from the chromosomes corresponding to the metaphase images of chromosomes, calculate the centroid coordinates of each chromosome; construct the Euclidean distance matrix of the centroid coordinates of each pair of chromosomes; calculate the total distance from each chromosome to all other chromosomes based on the Euclidean distance matrix; sort the chromosomes according to the total distances corresponding to each chromosome from smallest to largest, and select the first preset number of chromosomes.

[0271] The units in the above embodiments can be integrated into one unit or deployed separately; they can be merged into one unit or further divided into multiple sub-units.

[0272] In another embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a chromosome identification method.

[0273] In another embodiment, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, implement a chromosome identification method.

[0274] Figure 10 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Figure 10As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute the following methods:

[0275] Obtain images of chromosome metaphase;

[0276] The coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image are obtained based on the preset target detection model; each first bounding box is a rectangle of a single chromosome in the chromosome metaphase image; during training, the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints.

[0277] Sub-chromosome metaphase images are cropped from chromosome metaphase images based on multiple first bounding boxes; wherein, the sub-chromosome metaphase image is the image corresponding to the smallest rectangle that includes all the first bounding boxes;

[0278] Multiple second bounding boxes corresponding to the metaphase images of subchromosomes are obtained based on a preset target detection model; each second bounding box is a rectangle of a single chromosome in the metaphase image of subchromosomes.

[0279] Based on the second bounding box, obtain the segmentation mask of all chromosomes corresponding to the metaphase image of the chromosome; based on the segmentation mask, crop out the corresponding single chromosome from the metaphase image of the chromosome, and perform first pose normalization on the single chromosome;

[0280] The image corresponding to a single chromosome after the first posture standardization is input into a preset abnormality classification model to obtain the identification result of each chromosome in the chromosome metaphase image; the identification result is normal or abnormal, and if it is abnormal, the abnormality type is also included.

[0281] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0282] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0283] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0284] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments disclosed in this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings. For example, two blocks shown connectedly may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0285] Those skilled in the art will understand that the features described in the various embodiments and / or claims disclosed in this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope of this application.

[0286] This document uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of these embodiments are merely illustrative of the method and core concepts of the present invention and are not intended to limit this application. Those skilled in the art can make changes to the specific implementation methods and application scope based on the ideas, spirit, and principles of the present invention. Any modifications, equivalent substitutions, or improvements made should be included within the scope of protection of this application.

Claims

1. A chromosome identification method, characterized in that, The method includes: Obtain images of chromosome metaphase; The coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image are obtained based on a preset target detection model; each first bounding box is a rectangular box of a single chromosome in the chromosome metaphase image; the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training. A sub-chromosome metaphase image is cropped from the chromosome metaphase image based on a plurality of first bounding boxes; wherein, the sub-chromosome metaphase image is the image corresponding to the smallest rectangle that includes all the first bounding boxes; Based on the preset target detection model, multiple second bounding boxes corresponding to the metaphase image of the subchromosome are obtained; each second bounding box is a rectangular box of a single chromosome in the metaphase image of the subchromosome. Based on the second bounding box, obtain the segmentation mask of all chromosomes corresponding to the chromosome metaphase image; based on the segmentation mask, crop out the corresponding single chromosome from the chromosome metaphase image, and perform a first pose normalization on the single chromosome; The image corresponding to the single chromosome after the first posture standardization is input into the preset abnormality classification model to obtain the identification result of each chromosome in the chromosome metaphase image; the identification result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

2. The method according to claim 1, characterized in that, The acquisition of the preset target detection model includes: An initial target detection model is established; wherein, the initial target detection model includes: a feature extraction network, a feature pyramid, and a detection head; Obtain a training sample set; the training sample set includes: chromosome metaphase images and medical prior knowledge; The initial target detection model is trained using the training sample set. Training ends when the value of the preset loss function is less than the preset loss value, and the preset target detection model is obtained.

3. The method according to claim 2, characterized in that, The medical prior knowledge includes: centromere coordinates, long arm length, short arm length, long arm-to-short arm ratio, and satellite markers; The training sample set also includes labels, which include: the category of each chromosome, the center coordinates and width and height of the bounding rectangle of each chromosome, the absolute coordinates of each centromere in the image; the length of the long arm and short arm of each chromosome; the ratio of the long arm to the short arm of each chromosome; and whether satellites are present on each chromosome. The feature extraction network embeds a state-space model or a large-kernel selective convolution module. For each feature map output by the feature pyramid, the detection head constructs three parallel convolutional branches: a first branch, a second branch, and a third branch. The first branch is used to predict bounding box information; the second branch is used to predict chromosome category confidence; and the third branch is used to predict the relative normalized coordinates of the centromere, the normalized lengths of the long and short arms, the ratio of long to short arms, and the probability of satellite marker presence. The preset loss function is a weighted sum of bounding box regression loss, classification loss, distribution focus loss, and morphological loss; wherein, the morphological loss is calculated based on the label and the predicted chromosome category, centromere position, long arm length, short arm length, long-short arm ratio, and whether it is a satellite.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the segmentation mask, category, anomaly classification, key point coordinates, and polarity direction of each chromosome in the metaphase image of the chromosome; The segmentation mask is used to segment the corresponding single chromosome; The second pose standardization of the single chromosome is performed using the polarity direction corresponding to the chromosome and the coordinates of the key points; Chromosomes are grouped according to their categories, homologous chromosomes are paired, and karyotype arrangement diagrams are generated based on chromosomes standardized in the second pose, with the categories marked; if any chromosome has an abnormal classification, it is marked as an abnormal chromosome. Based on the type and number of chromosomes, a karyotype expression is generated.

5. The method according to claim 4, characterized in that, Obtain the coordinates and polarity of key points on the chromosome, including: Obtain the set of individual chromosomes corresponding to the metaphase image of the chromosome; the set of individual chromosomes includes a first type of chromosome and a second type of chromosome; the first type of chromosomes are chromosomes that have successfully undergone first pose standardization, and the second type of chromosomes are chromosomes that have failed first pose standardization before the first pose standardization was performed; Keypoint detection is performed on the first type of chromosomes to obtain keypoint coordinates and a first confidence level; polarity classification is performed on the first type of chromosomes to obtain polarity direction and a second confidence level; for the first type of chromosome, the obtained corresponding keypoint coordinates and first confidence level, as well as polarity direction and second confidence level are fused to obtain updated keypoint coordinates; Keypoint detection is performed on the second type of chromosome to obtain keypoint coordinates; and the corresponding polarity direction is determined for the second type of chromosome based on the obtained keypoint coordinates.

6. The method according to claim 1, characterized in that, The method further includes: The chromosome category, chromosome category confidence score, and segmentation mask for each chromosome corresponding to the metaphase image of the chromosome are obtained; the chromosome category confidence score is obtained based on the preset target detection model when obtaining multiple second bounding boxes corresponding to the metaphase image of the subchromosome. Select the chromosome with the highest density of a predetermined number from the chromosomes corresponding to the metaphase images of the chromosomes; Based on the selected chromosome category and segmentation mask, a quantity configuration score, global dispersion score, overlap score, sharpness score, and skeleton length score are determined. The quantity configuration score measures chromosome integrity and karyotype representativeness; the global dispersion score measures the spatial layout quality of the chromosomes; the overlap score measures chromosome separability and occlusion; the sharpness score measures contrast, detail richness, edge sharpness, and noise level; and the skeleton length score measures the degree of chromosome morphological unfolding and image scale. The quantity configuration score, the global dispersion score, the overlap score, the clarity score, and the skeleton length score are weighted and summed using preset weight values; Calculate the average confidence level of the chromosome category confidence level; The value obtained by multiplying the average confidence value and the weighted sum is used as the quality assessment value corresponding to the chromosome metaphase image; the higher the quality assessment value, the higher the quality of the chromosome metaphase image.

7. The method according to claim 6, characterized in that, The step of selecting a predetermined number of the most densely packed chromosomes from the chromosomes corresponding to the metaphase image of the chromosome includes: Calculate the centroid coordinates of each chromosome. Construct the Euclidean distance matrix for the centroid coordinates of each pair of chromosomes; Calculate the total distance from each chromosome to all other chromosomes based on the Euclidean distance matrix; Sort the chromosomes by total distance from smallest to largest, and select the first preset number of chromosomes.

8. A chromosome identification device, characterized in that, The device includes: Acquisition unit, used to acquire images of chromosome metaphase; The first detection unit is used to obtain the coordinates of multiple first bounding boxes corresponding to the chromosome metaphase image based on a preset target detection model; each first bounding box is a rectangular box of a single chromosome in the chromosome metaphase image; the preset target detection model incorporates medical prior knowledge and adds morphological feature constraints during training. A cropping unit is configured to crop a sub-chromosome metaphase image from the chromosome metaphase image based on a plurality of first bounding boxes; wherein the sub-chromosome metaphase image is an image corresponding to the smallest rectangle that includes all the first bounding boxes; The second detection unit is used to obtain multiple second bounding boxes corresponding to the metaphase image of the subchromosome based on the preset target detection model; each second bounding box is a rectangular box of a single chromosome in the metaphase image of the subchromosome. The segmentation unit is used to obtain the segmentation mask of all chromosomes corresponding to the chromosome metaphase image based on the second bounding box; to crop out the corresponding single chromosome from the chromosome metaphase image based on the segmentation mask; and to perform a first pose normalization on the single chromosome. The recognition unit is used to input the image corresponding to the single chromosome after the first posture standardization into a preset abnormality classification model to obtain the recognition result of each chromosome in the chromosome metaphase image; the recognition result is normal or abnormal, and if it is abnormal, it also includes the abnormality type.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1-7.

Images