Method, device and medium for detecting abnormal chromosomes in bone marrow based on template images

Abstract

The application provides a template image-based bone marrow abnormal chromosome detection method, device and medium. The method comprises: acquiring a bone marrow chromosome image to be detected; determining a target category of chromosomes in the bone marrow chromosome image through a trained and tested classification model; selecting a template image corresponding to the target category from a template library, wherein the template library stores a plurality of images representing normal bone marrow chromosomes; inputting the template image and the bone marrow chromosome image into a trained and tested difference analysis model to obtain a detection result, wherein the detection result comprises identification information representing whether the chromosomes in the bone marrow chromosome image are normal or abnormal, and an abnormal type corresponding to the abnormality when the chromosomes are abnormal. In this way, the accuracy of chromosome abnormality detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of biomedical image processing technology, and more specifically, to a method, device, and medium for detecting abnormal chromosomes in bone marrow based on template images. Background Technology

[0002] Chromosomal abnormalities in bone marrow are a core pathogenic factor in many hematologic disorders, such as leukemia and myelodysplastic syndromes. Accurate detection of chromosomal abnormalities (including deletions, duplications, inversions, and other structural abnormalities) is crucial for disease diagnosis, treatment planning, and prognostic assessment. Traditional methods of bone marrow chromosome testing require experienced cytogeneticists to observe chromosome banding patterns under a microscope, relying on manual judgment of the presence and type of abnormalities. This method is susceptible to errors and missed diagnoses due to factors such as physician experience and fatigue. Existing image-based detection models often employ simple convolutional neural networks (CNNs), primarily extracting local texture features. These models cannot effectively capture long-range dependencies in chromosome banding patterns and lack the ability to identify complex structural abnormalities such as inversions and translocations, leading to frequent false positives or false negatives. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, device and medium for detecting abnormal chromosomes in bone marrow based on template images, which can improve the problem of false detection or missed detection that easily occurs in the process of detecting abnormal chromosomes in bone marrow, resulting in inaccurate detection.

[0004] To achieve the above technical objectives, the technical solution adopted in this application is as follows:

[0005] In a first aspect, embodiments of this application provide a method for detecting abnormal chromosomes in bone marrow based on template images, the method comprising:

[0006] Obtain images of the chromosomes in the bone marrow to be tested;

[0007] The target category of chromosomes in the bone marrow chromosome image is determined using a trained and tested classification model.

[0008] From the template library, a template image corresponding to the target category is selected, wherein the template library stores multiple types of images representing normal bone marrow chromosomes;

[0009] The template image and the bone marrow chromosome image are input into a trained and tested differential analysis model to obtain detection results. The detection results include identification information indicating whether the chromosomes in the bone marrow chromosome image are normal or abnormal, and the corresponding abnormality type when abnormal. The differential analysis model includes a dual-branch feature extraction module, a feature fusion module, a differential mapping module, and a classification module. Each branch feature extraction module in the dual-branch feature extraction module includes a backbone network, an attention module, and a Transformer encoder.

[0010] Secondly, embodiments of this application also provide an electronic device, which includes a processor and a memory coupled to each other. The memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the above-described method.

[0011] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the above-described method.

[0012] The invention employing the above technical solution has the following advantages:

[0013] The technical solution provided in this application achieves automated detection of abnormal chromosomes in bone marrow by combining a template library and a differential analysis model. Chromosomal category identification and abnormality judgment can be completed without manual intervention, reducing the workload of physicians and improving detection efficiency. Furthermore, the dual-branch feature extraction module is beneficial for capturing multi-dimensional information such as detailed texture, spatial structure, global semantics, and long-range dependencies in chromosome images. Through attention mechanisms and differential calculations, it facilitates precise differential localization, reducing false positives or false negatives, thereby improving the accuracy of chromosome abnormality detection. Attached Figure Description

[0014] This application can be further illustrated by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate some embodiments of this application and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained from these drawings without any inventive effort.

[0015] Figure 1 This is a schematic flowchart of a template image-based method for detecting abnormal chromosomes in bone marrow, provided in an embodiment of this application.

[0016] Figure 2 A schematic diagram of the network structure of the difference analysis model provided in the embodiments of this application. Detailed Implementation

[0017] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] Please refer to the reference. Figure 1 and Figure 2 This application provides a method for detecting abnormal chromosomes in bone marrow based on template images. This method can be applied to an electronic device, which executes the steps of the method. The electronic device can be, but is not limited to, a personal computer, a server, etc., and is not specifically limited here.

[0019] Please refer to Figure 1 Methods for detecting abnormal chromosomes in bone marrow may include the following steps:

[0020] Step S10: Obtain the image of the bone marrow chromosome to be tested;

[0021] Step S20: Determine the target category of chromosomes in the bone marrow chromosome image using a trained and tested classification model;

[0022] Step S30: Select a template image corresponding to the target category from the template library, wherein the template library stores multiple types of images representing normal bone marrow chromosomes;

[0023] Step S40: Input the template image and the bone marrow chromosome image into the trained and tested differential analysis model to obtain the detection result. The detection result includes identification information indicating whether the chromosomes in the bone marrow chromosome image are normal or abnormal, and the corresponding abnormality type when abnormal. The differential analysis model includes a dual-branch feature extraction module, a feature fusion module, a differential mapping module, and a classification module. Each branch feature extraction module in the dual-branch feature extraction module includes a backbone network, an attention module, and a Transformer encoder.

[0024] The following is a detailed explanation of each step in the bone marrow abnormal chromosome detection method:

[0025] Please refer to Figure 2 In this embodiment, the difference analysis model is a Siamese-Transformer hybrid architecture, which includes four functional modules. The specific functions of each module are as follows:

[0026] The dual-branch feature extraction module includes a first-branch feature extraction module and a second-branch feature extraction module. Each branch feature extraction module contains a "backbone network + attention module + Transformer encoder". The two branches share parameters. For example, during the training phase, the shared parameters are iteratively updated with the total loss, and the feature extraction capabilities of the two branches are optimized synchronously. During the model inference and application phase, the template image (normal baseline) and the test image (bone marrow chromosome image) use the same set of parameters to extract features, avoiding "feature space inconsistency" caused by differences in branch parameters and ensuring the fairness of difference analysis.

[0027] In the dual-branch feature extraction module, the backbone network can use ResNet50 (with the fully connected layers removed from a regular ResNet50), consisting of 5 convolutional stages (Conv1-Conv5), with an output feature dimension of 2048×32×8 (H×W×C), used to extract local chromosome banding texture features. The attention module can use CBAM (Convolutional Block Attention Module), including channel attention units and spatial attention units. Channel attention units are used to enhance key channel features; spatial attention units are used to enhance the spatial structural features of chromosome banding.

[0028] The calculation formula for the channel attention unit is:

[0029] (1)

[0030] in, For channel attention weights, It is the Sigmoid activation function. It is a two-layer perceptron (hidden layer dimension 2048 / 16=128). It is a global average / max pooling function; These are the output features of the backbone network, such as image features output by the backbone network based on template images or sample chromosome images.

[0031] The calculation formula for the spatial attention unit is:

[0032] (2)

[0033] in, Spatial attention weights, It is a 7×7 convolution kernel (output channel 1). Dimensional splicing.

[0034] The Transformer encoder can employ a 6-layer encoder with the following parameters: input dimension 2048 (sequence length 256 = 32 × 8), 8-head self-attention (256 dimensions per head), feedforward network (2048→8192→2048), and GELU activation function, used to capture long-range dependencies in chromosome banding patterns (such as changes in banding order in inversion anomalies). In the feedforward network, "2048→8192→2048" represents the weights (2048 × 8192) and biases (8192 dimensions) of linear layer 1, and the weights (8192 × 2048) and biases (2048 dimensions) of linear layer 2. By "upgrading dimensions to mine fine-grained features and reducing dimensions to ensure dimensionality compatibility," the model's ability to capture nonlinear associations and subtle anomalies in chromosome banding patterns is enhanced without compromising link dimensionality matching. Ultimately, this improves the accuracy of the differential analysis model in identifying complex structural anomalies (inversions, translocations, and minor deletions).

[0035] During training, the first branch feature extraction module can output the first image features (template image features) based on the input template image. The second branch feature extraction module can output second image features (sample chromosome image features) based on the input sample chromosome image. ).

[0036] The feature fusion module can perform similarity calculation, weight allocation, and feature fusion on the first image features and the second image features, and output a similarity matrix. Similarity matrix Based on the following cosine similarity constitute.

[0037] Specifically, similarity calculation: first, for , The formula for calculating cosine similarity is:

[0038] (3)

[0039] In the formula, express and The cosine similarity; T represents the matrix transpose operation; It is an L2 norm.

[0040] Weight allocation: The weight of each branch is obtained by taking the row mean of the similarity matrix. The formula is:

[0041] (4)

[0042] In the formula, For the first Mean line similarity This represents the maximum value of the similarity matrix.

[0043] Feature fusion:

[0044] (5)

[0045] in, For the first Fusion features of each sequence position; Indicates the first Template image features at each sequence position; Indicates the first The feature fusion module outputs fused features from sample chromosome images at each sequence location. .

[0046] The difference mapping module can process fused features Perform feature inverse flattening, difference calculation and heat map generation.

[0047] Specifically, feature inverse flattening: Restored to spatial dimensions .

[0048] Difference calculation: Calculate the L1 distance between the template and the branch feature to be tested pixel by pixel, using the following formula:

[0049] (6)

[0050] in, For spatial coordinates (32×8). Output differential features for channel dimension (2048). .

[0051] Heatmap generation: The differential feature D is reduced to 1 channel by 1×1 convolution, and then upsampled to 1024×256 (consistent with the size of the bone marrow chromosome image) to obtain the differential heatmap. .

[0052] The classification module can include classification units and regression units. The classification unit can include two fully connected layers, which can... Features after global average pooling (dimension 1) and The flattened global features (2048 dimensions) are concatenated into a feature vector of 2049 dimensions, activated using Softmax, and output 7 probabilities (normal + 6 abnormal).

[0053] In this embodiment, the six types of anomalies can be missing, duplicate, inversion, translocation, isoarm, and ring-shaped anomalies.

[0054] Missing: Chromosomal segments are lost, which is manifested as the disappearance of banding in the corresponding region of the bone marrow chromosome image, and the region shows a high difference value (e.g., difference value ≥ 0.15) in the difference heatmap.

[0055] Duplication: Chromosomal segment duplication is manifested as repeated banding patterns in the corresponding region of bone marrow chromosome images, and the region shows continuous high difference values ​​in the differential heatmap.

[0056] Inversion: Chromosomal segments rotated 180°, resulting in the banding order of the bone marrow chromosome image being reversed compared to the template image. The Transformer encoder captures long-range dependency anomalies.

[0057] Translocation: The exchange of segments between two different chromosomes, manifested as a mismatch between some banding patterns in the bone marrow chromosome image and the template image, and the difference region involves non-continuous banding patterns;

[0058] Isochromosomal: Chromosomes with abnormal long and short arm lengths (e.g., only long or short arms), which are manifested as significant differences in the overall structure of bone marrow chromosome images compared to template images.

[0059] Ring chromosomes: The two ends of a chromosome are joined to form a ring, which is manifested as the disappearance of the terminal bands in bone marrow chromosome images, with a closed ring structure in the middle region.

[0060] The regression unit can be a three-layer fully connected layer, which can be based on... For regions with high disparity features (e.g., regions with a disparity value ≥ 0.1 after threshold filtering), output the coordinates of the bounding box of the abnormal region. The structures of the classification and regression units are conventional and will not be described in detail here.

[0061] Prior to step S10, the method may further include a step of training a differential analysis model for testing. That is, before obtaining the bone marrow chromosome image to be tested in step S10, the method may further include:

[0062] Obtain an image set for model training and testing. The image set includes a training set and a test set. The image set includes multiple first-class image groups and second-class image groups. Each first-class image group includes normal chromosome images and template images of bone marrow of the same chromosome category. Each second-class image group includes abnormal chromosome images of bone marrow of the same chromosome category, template images, and labels of abnormality types.

[0063] The difference analysis model is trained using the training set and a preset loss function, wherein the preset loss function is:

[0064] (7)

[0065] (8)

[0066] (9)

[0067] In the formula, Total loss; Classification loss; Refers to regression loss; N refers to the regression weight; M refers to the number of samples representing the corresponding image group within the batch; M refers to the number of abnormality types, which can be 7 (i.e., 1 normal type + 6 abnormal types). It refers to the probability that the i-th sample is predicted to be of the k-th class; Refers to the one-hot encoded label of the i-th sample in the k-th class; Log-likelihood is the natural logarithm used to predict probabilities. For the first The first sample The actual coordinates of each bounding box; For the first The first sample Predicted coordinate values ​​for each bounding box; For example, a piecewise loss function. γ is the focusing coefficient, γ=2 (used to reduce the weight of normal samples and solve data imbalance).

[0068] The trained differential analysis model is tested using the test set, and the trained differential analysis model is optimized based on the test results to obtain a trained and tested differential analysis model.

[0069] In this embodiment, the image set for model training and testing is obtained through a legitimate and conventional acquisition method. As an example, normal chromosome images are from 2000 normal samples collected clinically, and abnormal chromosome images are from 1000 abnormal samples collected clinically (≥200 cases of each type of abnormality).

[0070] The first type of image group construction: For each type of chromosome (1-22, X / Y), the normal chromosome image is paired with the corresponding template image to form a "normal chromosome image-template image" pair.

[0071] The second type of image group construction: For each type of chromosome, the abnormal chromosome image, the corresponding template image, and the label (including normal / abnormal identifier, abnormal type, and abnormal region coordinates) are paired to form an "abnormal chromosome image-template image-label" pair.

[0072] Dataset partitioning: The two image groups are divided into a training set (70%) and a test set (30%) in a 7:3 ratio; in other implementations, the image set may also include a validation set, which can be partitioned into training, test, and validation sets in a 7:2:1 ratio. Ensure that each type of anomaly is distributed across the training, test, or validation sets.

[0073] The images in the image set are preprocessed images. The preprocessing process may include performing alignment (rigid registration), tensor normalization (normalizing pixel values ​​to [0,1]) and label encoding on all images (one-hot encoding of classification labels and normalization of regression labels to [0,1]). The preprocessing method is conventional and will not be described in detail here.

[0074] Model training can be performed iteratively according to steps A10-A80 below until a specified number of iterations or a preset early stopping condition is reached. The specified number of iterations can be flexibly determined according to the actual situation, and no specific restrictions are imposed here. The preset early stopping condition can be: after each training epoch (referring to the process of the entire training set being completely traversed once), the model is used for inference on the validation set / test set, and the validation set F1 score is calculated. If the F1 score on the test set (or validation set) does not improve for three consecutive epochs, then training is stopped.

[0075] The training of the difference analysis model using the training set and a preset loss function may include:

[0076] A10, from the training set, take a batch of images and their corresponding labels each time, wherein the labels are used as the true values;

[0077] A20, input the template image and sample chromosome image in the image group into the dual-branch feature extraction module to obtain the first image feature corresponding to the template image and the second image feature corresponding to the sample chromosome image, wherein the sample chromosome image is a normal bone marrow chromosome image or an abnormal bone marrow chromosome image;

[0078] A30, the feature fusion module fuses the first image features and the second image features to obtain fused image features;

[0079] A40, through the difference mapping module, the fused image features are differentially mapped to obtain the abnormal classification probability of the chromosome, and the type with the highest classification probability is used as the prediction value;

[0080] A50, based on the predicted value and the actual value, calculate the total loss value using the preset loss function;

[0081] A60, based on the total loss value, performs backpropagation to determine the gradient values ​​of the trainable parameters in the difference analysis model;

[0082] A70, update the trainable parameters in the difference analysis model based on the gradient value;

[0083] A80, based on the differential analysis model after updating the trainable parameters, perform iterative training to repeat steps A10 to A70 until the number of iterations reaches a specified number or the preset early stopping condition is met, so as to obtain the trained differential analysis model.

[0084] As an example, step A10 can load the training set using PyTorch's DataLoader, and enable multi-threading (num_workers=8) to speed up the loading process.

[0085] In this embodiment, step A20 may include:

[0086] A21, the template image is input into the first branch feature extraction module in the dual-branch feature extraction module, and features are extracted through the backbone network in the first branch feature extraction module to obtain the first initial image features. The first initial image features are then enhanced by the attention module, and the enhanced first initial image features are encoded by the Transformer encoder to obtain the first image features.

[0087] A22, the sample chromosome image is input into the second branch feature extraction module of the dual-branch feature extraction module, and features are extracted through the backbone network in the second branch feature extraction module to obtain the second initial image features. The second initial image features are then enhanced by the attention module, and the enhanced second initial image features are encoded by the Transformer encoder to obtain the second image features.

[0088] Specifically, the first image feature extraction process is as follows:

[0089] Backbone network extraction: The template image is passed through the Conv1-Conv5 convolutional layers of ResNet50 to output the first initial image features (2048×32×8).

[0090] Attention enhancement: The initial features are enhanced through the CBAM module, which strengthens key channels and spatial features;

[0091] Transformer encoding: The enhanced features are flattened into a sequence (256×2048), which is then processed by a 6-layer Transformer encoder to capture long-range dependencies and output the first image features. .

[0092] The second image feature extraction process is consistent with the first branch. The sample chromosome image is processed by a ResNet50+CBAM+Transformer encoder to output the second image features. (The two branches share parameters to ensure consistent feature extraction standards).

[0093] In step A30, the feature fusion module needs to perform similarity calculation, weight allocation and feature fusion during the fusion of the first image features and the second image features, and finally obtain the batch-processed fused image features. The specific implementation process can be referred to the function of the feature fusion module mentioned above, and will not be repeated here.

[0094] In step A40, the fused image features are derived from step A30. The specific implementation process of the difference mapping can be referred to the function of the difference mapping module mentioned above, and will not be repeated here.

[0095] In step A50, the classification loss is calculated by substituting the predicted classification probability and the true classification label from the one-hot encoding into the classification loss. The calculation formula is the above formula (8), which is used to calculate the average classification loss of the batch processing.

[0096] Regression loss calculation: Substitute the predicted coordinates of the outlier region and the normalized true coordinates into the regression loss. The calculation formula is the above formula (9), which is used to calculate the batch average regression loss;

[0097] Total loss calculation: based on By combining the two types of losses, the total batch processing loss value is obtained.

[0098] Total loss value The backpropagation process leading to step A60 uses the classification and regression loss values ​​as training logs.

[0099] In step A60, the backpropagation process may include:

[0100] Gradient clearing: Call the optimizer's zero_grad() function to clear the gradient values ​​left over from the previous batch of training (to avoid gradient accumulation leading to abnormal parameter updates);

[0101] Backpropagation calculation: Calling the `backward()` function of the total loss value, each trainable parameter pair is derived backward from the total loss value based on the chain rule. The partial derivatives (i.e., gradient values) (where w is a trainable parameter, such as weights, biases, etc. updated by the optimizer (AdamW)).

[0102] Gradient clipping: Perform L2 regularization clipping (max_norm=1.0) on the gradients of all parameters to prevent gradient explosion from causing model training instability.

[0103] The gradient value corresponding to each trainable parameter can be obtained through backpropagation, and the gradient value flows to the parameter update process in step A70.

[0104] In step A70, the update process can be as follows:

[0105] Optimizer Update: The optimizer updates the parameters according to the gradient values ​​and a preset learning rate. The update formula is as follows: ,in For the updated parameters, For parameters before the update, This is the current learning rate;

[0106] Parameter synchronization: Updated parameters are synchronized to the difference analysis model in real time, replacing the old parameters.

[0107] The updated differential analysis model is used for the next batch of training (return to step A10) and is also cached on the training device (such as a server).

[0108] In step A80, a model checkpoint (.pth format) is saved every 5 epochs. After training is terminated, the model with the highest F1 score on the validation set is selected as the model for differential analysis after training.

[0109] As an example, the model testing optimization process can be as follows:

[0110] During testing, model performance is evaluated by calculating the overall accuracy on the test set, F1 scores for each anomaly class, and mean intersection-over-union ratio (mIOU). The preset target thresholds are: overall accuracy ≥ 98.5%, F1 score for each anomaly class ≥ 97%, and mIOU ≥ 0.85. If these thresholds are not met, the following optimization strategies are adopted:

[0111] At the sample level: supplement the training set with outlier samples (≥100 cases) corresponding to the low F1 class;

[0112] Parameter-wise: Adjust the γ value (γ=3 for low F1 classes), and reduce the initial learning rate to 5e. -5 Retrain for 10 epochs;

[0113] At the architecture level: If the average intersection-union ratio mIOU < 0.85, add one 3×3 convolutional layer (512 channels) before the regression head.

[0114] Repeat testing: After optimization, re-execute the test steps until the model performance meets the requirements.

[0115] In this embodiment, the parameter-sharing design of the dual-branch feature extraction module ensures that the feature extraction standards of the template image and the sample image are consistent, improving the accuracy of difference comparison; operations such as gradient zeroing and gradient clipping avoid gradient accumulation and gradient explosion problems during training, improving the stability of model training; combined with epoch loops and early stopping strategies, overtraining is avoided while ensuring model performance, shortening the training time.

[0116] Prior to step S10, the method may further include a step of constructing a template library. That is, before obtaining the bone marrow chromosome image to be tested in step S10, the method further includes:

[0117] S01. For each type of chromosome, acquire multiple normal bone marrow chromosome images of the same type and perform preprocessing to obtain preprocessed normal bone marrow chromosome images. The preprocessing includes image alignment and grayscale conversion.

[0118] S02, for each category of chromosome, perform pixel-level weighted summation on the preprocessed bone marrow normal chromosome image to obtain the template image corresponding to each category of chromosome;

[0119] S03, a template library is constructed based on the template image corresponding to each category of chromosome.

[0120] Specifically, in step S01, the preprocessing is performed using conventional methods. For example, the preprocessing procedure may be:

[0121] Image filtering: Remove blurry, unevenly colored, or overlapping original images, and retain ≥100 valid images;

[0122] Image alignment: For chromosome images of the same type, the image labeled as "most typical banding" by the physician is used as the benchmark. Rigid registration (rotation, translation, scaling) is performed based on maximizing mutual information to ensure that the banding positions are consistent.

[0123] Grayscale conversion: Converting an image to a grayscale image;

[0124] Size uniformity: normalized to 1024×256 pixels to ensure that the geometric dimensions of all preprocessed images are consistent.

[0125] In this embodiment, step S02 involves performing pixel-level weighted summation on the preprocessed normal chromosome images of the bone marrow for each chromosome category to obtain a template image corresponding to each chromosome category, including:

[0126] For each type of chromosome, the grayscale value of each pixel at the same location in the preprocessed bone marrow normal chromosome image is determined by the following formula, expressed as:

[0127] (10)

[0128] (11)

[0129] In the formula, This represents the grayscale value after fusion, where 'c' represents the chromosome category. This represents the pixel coordinates in the normal chromosome image of the bone marrow, where I represents the total number of normal chromosome images of the same category, and i is an integer ranging from 1 to I. The entropy weight of the i-th normal chromosome image from the bone marrow. The coordinates of the i-th normal chromosome in the bone marrow image are: The grayscale value of the pixel;

[0130] Based on the grayscale value of each pixel after fusion, a fused image of normal chromosomes in the bone marrow is obtained and used as a template image.

[0131] Specifically, the implementation process of step S02 can be as follows:

[0132] Entropy weight calculation: For each preprocessed image of the same category, calculate the gray-level entropy (the smaller the entropy value, the more regular the band shape, the more stable the feature, and the higher the weight), the formula is as follows: ,in, Let be the grayscale entropy of the i-th image; For the first The grayscale value of the image is Pixel percentage;

[0133] Weight normalization: Calculate the normalized weights for each image. ,in The total number of images of the same category. The weights of low-entropy images are reinforced by an exponential function, ensuring that samples with stable band shapes contribute more to the template, and that the sum of the weights of all images is 1.

[0134] Pixel-level weighted summation: For all images of the same category with the same coordinates, calculate the weighted sum according to the weights, as shown in the aforementioned formula (10), where For category In the template, in coordinates grayscale value at that location For the first In the coordinates of the image The grayscale value at that location;

[0135] Gray-scale mapping: Map the weighted sum of gray values ​​back to the [0,255] range to generate the fused template image. Values ​​outside the range are truncated (0 for values ​​less than 0, 255 for values ​​greater than 255).

[0136] In step S03, metadata (such as the number of fused samples, generation time, and baseline image ID) can be associated with each template image and stored as a JSON file; a local template library folder can be built to store template images and metadata hierarchically according to chromosome category, supporting fast retrieval.

[0137] During template library construction, preprocessing operations (image alignment, grayscale conversion) ensure the feature consistency of normal chromosome images of the same category, providing a standardized data foundation for subsequent fusion. Weighted fusion based on multiple normal samples avoids the limitations of traditional single templates, resulting in template images with stronger group representativeness and adaptability to individual differences within the normal population. Weight allocation based on grayscale entropy assigns higher weights to normal samples with stable banding patterns, improving the feature representativeness and reliability of the template images. The fusion process is standardized and quantifiable, avoiding the subjectivity of manually constructed templates and ensuring consistent template quality. Pixel-level weighted summation achieves feature fusion of samples of the same category, enabling template images to reflect the common features of chromosomes in the normal population while also considering individual differences. Furthermore, the template library contains only normal chromosome features, providing a stable and reliable benchmark for anomaly detection and improving the accuracy of anomaly identification.

[0138] In step S10, the method for acquiring the bone marrow chromosome image to be tested is conventional. The bone marrow chromosome image to be tested can be a pre-processed image; if it has not been pre-processed, pre-processing steps can be performed. The pre-processing process is conventional. For example, the pre-processing process can be as follows: for the clinical bone marrow smear sample and the original image output by the image acquisition device, adaptive median filtering is used to eliminate acquisition noise for denoising; then, geometric correction is performed, for example, extracting chromosome contours using the Canny operator, and performing rotation (correcting tilt angle), scaling (e.g., normalizing to 1024×256 pixels), and translation (aligning to the central axis) using affine transformation; after geometric correction, grayscale normalization is then performed; next, single chromosome segmentation is performed, for example, using the U-Net++ model to extract the ROI (Region of Interest) of a single chromosome, removing the background and adhered regions. The pre-processed image of the ROI of the single chromosome to be tested is obtained as the bone marrow chromosome image to be tested. The Canny operator, affine transformation, and U-Net++ model are all existing implementations and will not be elaborated further here.

[0139] In step S20, the bone marrow chromosome image of a single chromosome is input into the classification model. This model outputs the probability distribution of 24 human chromosome classes (1-22, X / Y). The classification model selects the class corresponding to the highest probability as the target class, such as "chromosome 1" or "chromosome X". The classification model performs chromosome classification in a conventional manner; for example, it can be a ResNet34 model.

[0140] In step S30, the template library pre-stores images representing various types of normal bone marrow chromosomes. After obtaining the target category of the chromosome in the bone marrow chromosome image, the electronic device can search the template library for a template image of the chromosome with the same target category. For example, if the target category is chromosome 1, the template image labeled "1" is retrieved. This template image serves as a reference benchmark for subsequent chromosome abnormality detection, helping to improve the accuracy of the detection.

[0141] In step S40, the template image and the image to be tested are input into the differential analysis model, which are a set of images after image alignment. If the images have not been aligned, the image alignment process needs to be performed. The image alignment method is conventional, for example, performing rigid registration (based on maximizing mutual information) on the template image and the image to be tested to ensure accurate alignment of chromosome banding positions.

[0142] By inputting two aligned images into a difference analysis model, the model performs bi-branch feature extraction, feature fusion, difference mapping, and classification regression to output detection results. These results may include normal / abnormal identifiers, abnormality types, and coordinates of the abnormal region. The results can be reviewed by physicians to help them quickly locate abnormalities and improve diagnostic accuracy.

[0143] In this embodiment, the method may further include:

[0144] When the detection result indicates a chromosomal abnormality in the bone marrow chromosome image, a prompt message is issued, and the corresponding abnormal area is marked in the bone marrow chromosome image.

[0145] As an example, an anomaly label=1 indicates a chromosomal abnormality; an anomaly label=0 indicates a normal chromosomal chromosome. During the anomaly identification process, the anomaly label in the detection results is read. If label=1, a prompt message is generated. The prompt message can be constructed by generating a targeted prompt based on the anomaly type, for example, in the format "Warning: An anomaly of [abnormality type] was detected, confidence level: [confidence score], please have a physician review," where the confidence score is the anomaly probability (0-1) output by the model. The prompt message is output to the physician through pop-ups, sounds, etc., within the clinical diagnostic system.

[0146] The methods for marking the corresponding abnormal regions can be as follows:

[0147] Coordinate analysis: Extracting the coordinates of the bounding boxes of abnormal regions from the detection results. ;

[0148] Region labeling: In the ROI image to be tested, select the abnormal region with a red rectangle (3 pixels wide) and label the abnormality type (such as "missing") next to the rectangle;

[0149] Image saving: Save the labeled image, naming it as "sample ID_anomaly type_labeled.png".

[0150] In this embodiment, the abnormality alerts are intuitive and accurate, quickly reminding physicians to pay attention to abnormal samples and avoid missed detections. The abnormal region marking transforms the abstract test results into a visual image, helping physicians quickly locate the abnormality and improve review efficiency. The marking information includes the abnormality type and confidence level, providing physicians with decision-making references and reducing diagnostic difficulty. The marking of the corresponding abnormal regions in the bone marrow chromosome image, combined with the visual output, conforms to the working habits of clinicians and improves the practicality and ease of use of the method.

[0151] Based on the above design, the template library of multiple normal samples is highly representative, and the dual-branch feature extraction module can accurately capture local texture and long-distance structural features, which is beneficial to improving the accuracy of anomaly detection. This method can realize an end-to-end automated process from image preprocessing, category recognition, template matching to anomaly detection, reducing the workload of physicians and improving detection efficiency. The detection results include anomaly identification, anomaly type, and anomaly region coordinates, and support visual labeling and prompts, which conforms to the working habits of clinicians and can be used to assist in diagnosis.

[0152] This application provides an electronic device that may include a processing module and a storage module. The storage module stores a computer program, which, when executed by the processing module, enables the electronic device to perform the corresponding steps in the aforementioned bone marrow abnormal chromosome detection method.

[0153] In this embodiment, the processing module can be an integrated circuit chip with signal processing capabilities. The processing module can be a general-purpose processor. For example, the processor can be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0154] The storage module can be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. In this embodiment, the storage module can be used to store template libraries, classification models, difference analysis models, etc. Of course, the storage module can also be used to store programs, which the processing module executes after receiving execution instructions.

[0155] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the electronic device described above can be referred to the corresponding steps in the aforementioned method, and will not be elaborated further here.

[0156] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the bone marrow abnormal chromosome detection method as described in the above embodiments.

[0157] Computer-readable storage media may be magnetic disks, optical disks, read-only memory, random access memory, flash memory, USB flash drives, hard disks, or solid-state drives, etc., and may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implement the methods shown in the above embodiments.

[0158] Based on the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, electronic device, or network device, etc.) to execute the methods described in the various implementation scenarios of this application.

[0159] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0160] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting abnormal chromosomes in bone marrow based on template images, characterized in that, The method includes: Obtain images of the chromosomes in the bone marrow to be tested; The target category of chromosomes in the bone marrow chromosome image is determined using a trained and tested classification model. From the template library, a template image corresponding to the target category is selected, wherein the template library stores multiple types of images representing normal bone marrow chromosomes; The template image and the bone marrow chromosome image are input into a trained and tested differential analysis model to obtain detection results. The detection results include identification information indicating whether the chromosomes in the bone marrow chromosome image are normal or abnormal, and the corresponding abnormality type when abnormal. The differential analysis model includes a dual-branch feature extraction module, a feature fusion module, a differential mapping module, and a classification module. Each branch feature extraction module in the dual-branch feature extraction module includes a backbone network, an attention module, and a Transformer encoder.

2. The method according to claim 1, characterized in that, Before acquiring the bone marrow chromosome image to be tested, the method further includes: Obtain an image set for model training and testing. The image set includes a training set and a test set. The image set includes multiple first-class image groups and second-class image groups. Each first-class image group includes normal chromosome images and template images of bone marrow of the same chromosome category. Each second-class image group includes abnormal chromosome images of bone marrow of the same chromosome category, template images, and labels of abnormality types. The difference analysis model is trained using the training set and a preset loss function, wherein the preset loss function is: ； ； ； In the formula, Total loss; Classification loss; Refers to regression loss; N refers to the regression weights; M refers to the number of samples representing the corresponding image group within the batch; M refers to the number of anomaly types. It refers to the probability that the i-th sample is predicted to be of the k-th class; Refers to the one-hot encoded label of the i-th sample in the k-th class; γ is the focusing coefficient; For the first The first sample The actual coordinates of each bounding box; For the first The first sample Predicted coordinate values ​​for each bounding box; For piecewise loss functions; The trained differential analysis model is tested using the test set, and the trained differential analysis model is optimized based on the test results to obtain a trained and tested differential analysis model.

3. The method according to claim 2, characterized in that, The differential analysis model is trained using the training set and a preset loss function, including: A10, from the training set, take a batch of images and their corresponding labels each time, wherein the labels are used as the true values; A20, input the template image and sample chromosome image in the image group into the dual-branch feature extraction module to obtain the first image feature corresponding to the template image and the second image feature corresponding to the sample chromosome image, wherein the sample chromosome image is a normal bone marrow chromosome image or an abnormal bone marrow chromosome image; A30, the feature fusion module fuses the first image features and the second image features to obtain fused image features; A40, through the difference mapping module, the fused image features are differentially mapped to obtain the abnormal classification probability of the chromosome, and the type with the highest classification probability is used as the prediction value; A50, based on the predicted value and the actual value, calculate the total loss value using the preset loss function; A60, based on the total loss value, performs backpropagation to determine the gradient values ​​of the trainable parameters in the difference analysis model; A70, update the trainable parameters in the difference analysis model based on the gradient value; A80, based on the differential analysis model after updating the trainable parameters, perform iterative training to repeat steps A10 to A70 until the number of iterations reaches a specified number or the preset early stopping condition is met, so as to obtain the trained differential analysis model.

4. The method according to claim 3, characterized in that, Step A20 includes: A21, the template image is input into the first branch feature extraction module in the dual-branch feature extraction module, and features are extracted through the backbone network in the first branch feature extraction module to obtain the first initial image features. The first initial image features are then enhanced by the attention module, and the enhanced first initial image features are encoded by the Transformer encoder to obtain the first image features. A22, the sample chromosome image is input into the second branch feature extraction module of the dual-branch feature extraction module, and features are extracted through the backbone network in the second branch feature extraction module to obtain the second initial image features. The second initial image features are then enhanced by the attention module, and the enhanced second initial image features are encoded by the Transformer encoder to obtain the second image features.

5. The method according to claim 1, characterized in that, Before acquiring the bone marrow chromosome image to be tested, the method further includes: For each type of chromosome, multiple normal bone marrow chromosome images of the same type are acquired and preprocessed to obtain preprocessed normal bone marrow chromosome images. The preprocessing includes image alignment and grayscale conversion. For each type of chromosome, pixel-level weighted summation is performed on the preprocessed bone marrow normal chromosome image to obtain the template image corresponding to each type of chromosome. A template library is constructed based on the template image corresponding to each category of chromosome.

6. The method according to claim 5, characterized in that, For each chromosome category, pixel-level weighted summation is performed on the preprocessed bone marrow normal chromosome images to obtain the template image corresponding to each chromosome category, including: For each type of chromosome, the grayscale value of each pixel at the same location in the preprocessed bone marrow normal chromosome image is determined by the following formula, expressed as: ； ； In the formula, This represents the grayscale value after fusion, where 'c' represents the chromosome category. This represents the pixel coordinates in the normal chromosome image of the bone marrow, where I represents the total number of normal chromosome images of the same category, and i is an integer ranging from 1 to I. The entropy weight of the i-th normal chromosome image from the bone marrow. The coordinates of the i-th normal chromosome in the bone marrow image are: The grayscale value of the pixel; Based on the grayscale value of each pixel after fusion, a fused image of normal chromosomes in the bone marrow is obtained and used as a template image.

7. The method according to claim 1, characterized in that, The method further includes: When the detection result indicates a chromosomal abnormality in the bone marrow chromosome image, a prompt message is issued, and the corresponding abnormal area is marked in the bone marrow chromosome image.

8. The method according to any one of claims 1 to 7, characterized in that, The abnormality types include missing, duplicate, inversion, translocation, isoarm, and ring.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory coupled together, the memory storing a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

Images