Automatic cell detection method and device based on fluorescence in-situ hybridization image and readable storage medium thereof

Abstract

The invention provides an automatic cell detection method and device of a fluorescence in-situ hybridization image and a readable storage medium of the automatic cell detection method and device. According to the method, the fluorescence signal classification detection capability of Yolov12 and the fine boundary segmentation advantage of Cellpose are fused; through data preparation and labeling, double-model targeted training, segmentation prediction, post-processing optimization and mask logic and operation, collaborative decision is realized, and the problems of difficult cell overlapping segmentation, fluorescence signal attenuation interference, strong subjectivity of manual interpretation and the like in existing FISH image analysis are solved. The method can significantly improve the segmentation precision and the edge recognition capability of the overlapped cells, achieves full-automatic efficient processing, provides an objective and reliable basis for the diagnosis of related diseases, and has high precision, high efficiency and strong objectivity.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to an automated cell detection method and device involving fluorescence in situ hybridization images and a readable storage medium thereof. Background Art

[0002] Fluorescence in situ hybridization (FISH) technology is an important means of molecular pathology diagnosis. It uses fluorescently labeled nucleic acid probes to specifically hybridize with target DNA sequences in the cell nucleus and observe the distribution of fluorescent signals under a fluorescence microscope. It can accurately determine gene amplification or deletion (such as HER2 gene amplification), providing important basis for the classification and treatment guidance of related cancers.

[0003] However, the current analysis of FISH images mainly relies on manual interpretation by professionals: technicians need to identify cells one by one under a microscope and count fluorescent signals. This is not only a huge workload and inefficient, but also difficult to meet the needs of rapid processing of large-scale clinical samples; manual interpretation is also affected by factors such as subjective experience and fatigue status. The interpretation results of different observers or even the same observer at different times may vary significantly, resulting in poor diagnostic consistency.

[0004] Furthermore, FISH imaging itself presents numerous technical challenges. Firstly, cells often form overlapping or adherent clusters due to their dense distribution, making it difficult to accurately delineate their boundaries using traditional segmentation methods. Secondly, fluorescence signals are susceptible to attenuation and unevenness due to imaging conditions, further interfering with cell identification and signal statistics. These challenges collectively limit the automation and accuracy of FISH diagnostics, necessitating an efficient, precise, and objective automated cell detection method. Summary of the Invention

[0005] The embodiments of the present invention provide an automated cell detection method, device, and readable storage medium for fluorescence in situ hybridization images. These methods address the problems of existing FISH image analysis, which relies on manual interpretation and suffers from heavy workload, strong subjectivity, and low efficiency. Furthermore, traditional automated methods suffer from insufficient accuracy due to difficulties in segmenting overlapping cells and interference from fluorescence signal attenuation.

[0006] The core technology of this invention is to build a dual-model collaborative decision-making mechanism by integrating the fluorescence signal detection capability of the Yolov12 model with the fine boundary segmentation advantage of the Cellpose model, and use mask logic and operations to realize automatic cell classification and segmentation of FISH images.

[0007] In a first aspect, the present invention provides an automated cell detection method based on fluorescence in situ hybridization images, the method comprising the following steps: S1. Data preparation: Acquire and annotate FISH images, and divide the cells in the images into the first category (cells requiring fluorescence point counting) and the second category (cells not yet analyzed). The first category of cells is complete, single cells in the field of view, with uniform size and consistent morphology. The second category of cells is cell clusters with incomplete boundaries or adhesions. S2. Generate dataset: Convert the annotation results into the standard format required by the first model. The conversion includes coordinate normalization, category coding and quality control. S3. First model training: Load the pre-trained parameters of the first model and fine-tune the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process uses a combination of multiple loss functions, combined with data augmentation and regularization strategies. S4. Second model training: Load the pre-trained parameters of the second model and adapt the input channels to suit the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data augmentation and dynamic training strategies. S5. Dual-model segmentation prediction: Use the trained first model and second model to segment the FISH image, output the segmentation mask, category identifier and confidence of the first model, and the segmentation mask of the second model; S6. Optimizing the segmentation results of the first model: performing post-processing on the segmentation results of the first model, including confidence ranking, mask scaling, morphological operations, and overlapping area removal; S7. Dual-model collaborative decision-making: Perform a logical AND operation on the segmentation mask of the second model and the optimized segmentation mask of the first model, determine the cell category based on the number and distribution of connected domains in the operation result, and generate the final detection result.

[0008] Furthermore, in step S1, the annotation is performed by a two-person annotation verification method, and the ambiguous samples are finally interpreted by a third-party expert, and the annotation tool is an image annotation tool.

[0009] Furthermore, in step S2, quality control includes filtering invalid annotations, checking polygon closure, and verifying vertex coordinate ranges.

[0010] Furthermore, in step S3, the first model is the Yolov12 model, the combined loss function includes a weighted combination of Dice loss, Focal loss, Jaccard loss, mutual exclusion loss and soft sampling weighted loss, data enhancement includes rotation, flipping and color space adjustment, and the regularization strategy includes label smoothing, weight decay and random dropout.

[0011] Furthermore, in step S4, the second model is the Cellpose model, the mask loss is the joint loss of Dice loss and cross entropy loss, the boundary loss is the multi-scale matching loss, the regularization loss is the morphological consistency loss, and the targeted data enhancement includes elastic deformation, fluorescence channel separation and incorporation of adversarial samples.

[0012] Furthermore, in step S4, the dynamic training strategy includes a learning rate warm-up, a cosine annealing decay and restart mechanism, and an early stopping mechanism based on a boundary evaluation indicator.

[0013] Furthermore, in step S6, the morphological operation includes performing a dilation operation and a closing operation on the binary mask, and the overlapping area removal is achieved by a logical AND operation.

[0014] Furthermore, in step S7, when the result of the logical AND operation contains a single connected domain, the category identifier predicted by the first model is directly used; when there are multiple connected domains, the category identifier of the first model corresponding to the connected domain with the largest number of pixels is used as the result; when there is no connected domain, the segmentation mask of the second model is expanded until it overlaps with the segmentation mask of the first model, and the category identifier of the first model corresponding to the overlapping area is used as the result.

[0015] Furthermore, in step S3, the weight of the combined loss function is determined by genetic algorithm optimization, and the average intersection-over-union ratio of the validation set is used as the fitness indicator.

[0016] Furthermore, in step S6, the mask scaling uses an interpolation method to adjust the mask image to the size of the source image, and the interpolation method includes quadratic spline interpolation.

[0017] In a second aspect, the present invention provides an automated cell detection device for fluorescence in situ hybridization images, comprising: The data preparation module acquires and annotates FISH images, dividing the cells in the images into the first category of cells for which fluorescence point counting is required, and the second category of cells that are not yet analyzed. The first category of cells are complete and single cells in the field of view with uniform size and consistent morphology, while the second category of cells are cell clusters with incomplete boundaries or adhesions. The dataset module converts the annotation results into the standard format required by the first model. The conversion includes coordinate normalization, category coding and quality control. The first model module loads the pre-trained parameters of the first model and fine-tunes the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process uses a combination of multiple loss functions and combines data enhancement and regularization strategies. The second model module loads the pre-trained parameters of the second model and adapts the input channels to suit the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data augmentation and dynamic training strategies; The dual-model segmentation prediction module segments the FISH image using the trained first model and the second model, and outputs the segmentation mask, category identifier, and confidence score of the first model, as well as the segmentation mask of the second model. The result optimization module performs post-processing on the segmentation results of the first model, including confidence-based sorting, mask scaling, morphological operations, and overlapping area removal; The collaborative decision-making module performs a logical AND operation on the segmentation mask of the second model and the optimized segmentation mask of the first model, determines the cell category based on the number and distribution of connected domains in the operation result, and generates the final detection result.

[0018] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned automated cell detection method of fluorescence in situ hybridization images.

[0019] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes an automated cell detection method based on the above-mentioned fluorescence in situ hybridization image.

[0020] The main contributions and innovations of the present invention are as follows: 1. Significantly improved classification and segmentation accuracy: Through dual-model collaboration (Yolov12's classification and detection capabilities and Cellpose's fine segmentation advantages) and mask logic and operations, it effectively solves the problems of difficult overlapping cell segmentation and inaccurate boundary delineation, improves edge signal recognition capabilities, and especially outperforms traditional methods in processing overlapping cells and cells with weak fluorescence signals.

[0021] 2. Significantly improved processing efficiency: A fully automated processing flow is built, and dual models collaborate to achieve high-speed parallel computing, significantly shortening analysis time while ensuring accuracy, meeting clinical needs for efficient processing of large-scale samples.

[0022] 3. Enhanced diagnostic objectivity and consistency: This approach avoids the subjectivity of manual interpretation and outputs stable and repeatable analysis results through standardized algorithms, providing an objective and reliable basis for the diagnosis of related cancers and improving diagnostic consistency across different institutions and individuals.

[0023] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of an automated cell detection method based on fluorescence in situ hybridization images according to an embodiment of the present invention; Figure 2 is a diagram of the segmentation result of the Cellpose model according to an embodiment of the present invention; Figure 3 2. YOLO model segmentation result diagram according to an embodiment of the present invention; Figure 4 3. This is a graph showing the result of YOLO model after overlap elimination and boundary optimization according to an embodiment of the present invention; Figure 5 4 is a segmentation result diagram of the Cellpose model combined with the YOLO model according to an embodiment of the present invention; Figure 6 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0026] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0027] Existing FISH image analysis relies on manual interpretation, which has problems such as heavy workload, strong subjectivity, and low efficiency. In addition, due to the difficulty of cell overlapping segmentation and interference from fluorescence signal attenuation, traditional automated methods lack accuracy.

[0028] Based on this, the present invention solves the problems existing in the prior art based on the Yolov12 model and the Cellpose model.

[0029] Example 1 The present invention aims to propose an automated cell detection method based on fluorescence in situ hybridization images. Figure 1 , the method comprises the following steps: S1. Data preparation: Acquire and annotate FISH images, and divide the cells in the images into the first category (cells requiring fluorescence point counting) and the second category (cells not yet analyzed). The first category of cells is complete, single cells in the field of view, with uniform size and consistent morphology. The second category of cells is cell clusters with incomplete boundaries or adhesions. In this example, LabelMe version 5.4.1 was used for labeling. The labeling process requires categorizing cells in the image into two categories: cell and other. The cell category refers to cells for which fluorescence point counting is required. These cells are complete and isolated within the image field of view, relatively large, uniform in size, and consistent in morphology. The other category refers to cells not currently being analyzed. These cells have incomplete boundaries within the image field of view, or contain adherent cell clusters, which are not suitable for subsequent fluorescence point counting.

[0030] The labeling process adopts a two-person labeling and verification method. For samples with ambiguous interpretation results between the two people, a third expert will be invited to interpret them, and the results of the third expert will be used as the final interpretation results.

[0031] For example, in the 500 annotated images, the ratio of cell to other categories is approximately 1:1. Samples with different imaging qualities and staining intensities are not included. The images all show cells with clear fields of view and uniform staining.

[0032] S2. Generate dataset: Convert the annotation results into the standard format required by the first model. The conversion includes coordinate normalization, category coding and quality control. In this embodiment, the json file annotated by the expert using Labelme is converted into the standard txt format required by the Yolo model. The specific steps are as follows: S2.1. Data preparation and input: Obtain the json annotation files generated by Labelme. Each file corresponds to a medical image. Read the polygon vertex coordinates and cell category labels from the json files, and read the original image size (width, height) for normalization.

[0033] S2.2 Coordinate Conversion and Normalization: Convert absolute pixel coordinates to normalized coordinates and encode the categories, where the "cell" category corresponds to the number 0 and the "other" category corresponds to the number 1. The specific vertex coordinate (x, y) conversion process is as follows:

[0034]

[0035] Finally, the result retains 6 decimal places of precision to reduce floating point errors.

[0036] S2.3. Quality control of converted datasets: During the conversion of JSON files into the standard txt format required by the Yolo model, strict quality control is required. Quality control includes the following three points: Filter invalid annotations: polygons with fewer than 3 vertices are automatically discarded.

[0037] Polygon closure check: When the number of first and last points is greater than 1 pixel, the polygon is automatically closed.

[0038] Vertex validation: Ensure that the coordinate values ​​of all points are within the range [0, width] and [0, height].

[0039] S3. First model training: Load the pre-trained parameters of the first model and fine-tune the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process uses a combination of multiple loss functions, combined with data augmentation and regularization strategies. In this example, the model parameters of the yolov12 model (first model) pre-trained on a large-scale general dataset are first loaded. Without modifying the specific architecture of the model, the parameters are fine-tuned using the annotation results of the FISH images, and the number of model categories is set to 2. During training, the relevant training parameters are set, with the training epochs set to 200, the batch size set to 96, and the image size set to 1024, so that the yolov12 model gradually adapts to the characteristics of the FISH images.

[0040] In addition, during the training process, the following training strategy adjustments are added: S3.1. Loss function optimization: To improve model performance, a combination of multiple loss functions is adopted, including Dice loss, Focal loss, Jaccard loss, mutual exclusion loss, and soft sampling weighted loss. They are combined into a custom combined loss, and the genetic algorithm is used to design the weights of multiple loss functions. The mean intersection-over-union (mIOU) result of the centralized segmentation result is verified as the fitness of the genetic algorithm to determine the weight of the combination of multiple loss functions. The final weights of Dice loss, Focal loss, Jaccard loss, mutual exclusion loss, and soft sampling weighted loss are 0.3:0.2:0.2:0.2:0.1.

[0041] Among them, the mean intersection-over-union (mIOU) is the average value of the intersection-over-union (IOU) of all categories, and its calculation formula is as follows:

[0042] The calculation formula for mIOU is as follows:

[0043] Where c is the number of categories; is the intersection-over-union ratio of the i-th category.

[0044] In the loss function, Dice loss is used to measure the degree of overlap between the predicted results and the true labels. Its expression is as follows:

[0045] In the above formula, The segmentation mask predicted by the model; is the true label; represents the intersection of the prediction and the true label; is a smoothing term, generally, the value is 1 , used to avoid the situation where the denominator is 0.

[0046] In addition, Focal loss is used to solve the problem of heavy category imbalance in classification tasks. Its standard formula is as follows:

[0047] In the above formula, It represents the model's predicted probability of the true category, which is used to measure the accuracy of sample classification. Represents the balance factor, which adjusts the weights of positive and negative samples. Represents the adjustment factor, which is used to reduce the weight of easily classified samples. Generally, the value is 2.

[0048] Jaccard loss is used to measure the degree of overlap between the predicted results and the true label. It is defined as the complement of the Jaccard coefficient. The calculation process of the Jaccard coefficient is as follows:

[0049] In the above formula, represents the segmentation mask predicted by the model, represents the true label, Indicates the size of the intersection between the prediction and the true label, represents the union size of the prediction and the true label, Represents a small constant used to ensure that the denominator is non-zero.

[0050] The calculation formula of Jaccard loss is:

[0051] Exclusive Loss refers to a loss function that forces the model to assign a unique category label to each pixel. Its calculation form is cross entropy loss:

[0052] in, Represents pixels i In category c The true label, Represents the predicted probability that the pixel i output by Softmax belongs to category c, represents the total number of categories, Indicates the total number of image pixels.

[0053] Soft sampling weighted loss dynamically adjusts the weights based on the difficulty of classifying samples, making the model pay more attention to samples that are difficult to classify correctly (hard samples), thereby improving the model's ability to identify boundary or minority class samples. Its basis is cross-entropy loss. After adding the weight modulation factor, the form is as follows:

[0054] In the above formula, represents the number of pixels in the image, represents the probability that the i-th sample predicted by the model belongs to its true category, Represents the weight of the i-th sample, which consists of two parts:

[0055] Represents the category weight, which is used to solve the category imbalance problem. Represents the difficulty weight, which is used to strengthen difficult samples. Represents the adjustment factor, which is used to reduce the weight of easily classified samples. Generally, the value is 2.

[0056] S3.2. Training strategy optimization: During the model training process, a series of optimization strategies are adopted.

[0057] In terms of data enhancement, data enhancement operations such as rotation, upside down flipping, left-right flipping, translation, and HSV space adjustment are used to amplify the input FISH images and improve the generalization performance of the model.

[0058] In terms of learning rate adjustment, the cosine annealing strategy is used to adjust the learning rate. The initial learning rate is adjusted to 0.001, the final learning rate is set to 0.1, the learning rate warm-up parameter is set to 5, and the momentum in the learning rate warm-up stage is set to 0.8.

[0059] At the same time, label smoothing, weight decay, and random dropout regularization methods are set to prevent model overfitting and improve model stability and accuracy. The AdamW optimizer is used, with the probability of label smoothing set to 0.1, the weight decay rate to 0.05, and the classifier dropout rate to 0.5.

[0060] S4. Second model training: Load the pre-trained parameters of the second model and adapt the input channels to suit the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data augmentation and dynamic training strategies. In this embodiment, the model parameters of Cellpose (the second model) pre-trained on the biomedical image dataset (cpsam) are loaded, and the original model architecture is kept unchanged. The input channel is adapted to 3 to adapt to the FISH image characteristics.

[0061] Set the training parameters: Epochs to 200, Batch Size to 32, Image size to 1024*1024 high-resolution images, and deprecate GPU acceleration and mixed precision training (AMP).

[0062] S4.1. Loss function used in Cellpose model training: Core loss function:

[0063] Among them, L mask represents the mask loss with a weight of 0.6. boundary L represents the boundary loss with a weight of 0.3. regularization represents the regularization loss with a weight of 0.1.

[0064] Mask loss ( ) The loss adopts the improved Dice-CE joint loss:

[0065] in, Represents Dice loss, which is more sensitive to segmentation boundaries and small objects by calculating the overlap ratio between the predicted area and the true area; The coefficient representing the Dice loss is usually set to 0.7; Represents cross entropy loss, which solves the column imbalance problem through the logarithmic penalty mechanism; The coefficient representing the cross entropy loss is usually set to 0.3.

[0066] The cross entropy loss here The standard calculation method is as follows: 1. Two-classification scenario (e.g., cells are divided into "first class" and "second class" in this invention): For a single sample, let the true label be , the model predicts that the probability that the sample belongs to category 1 is p, then the cross entropy loss is:

[0067] 2. For a batch of samples, take the average of all sample losses: The core logic is to penalize the "deviation between the predicted probability and the true label" through a logarithmic function (the less accurate the prediction, the greater the loss value). Especially for class imbalance scenarios (such as when the proportion of samples in a certain class is extremely low), further optimization can be achieved through weighting (such as introducing class weight α), namely:

[0068] In actual operation, you can choose a specific formula (such as the above-mentioned binary cross entropy or weighted cross entropy) according to your needs and adjust it according to the final effect.

[0069] Boundary optimization loss ( ) uses multi-scale matching loss:

[0070] in, represents the L1 norm, Indicates the calculation of the sum of the pixel-by-pixel absolute errors of the two gradient maps; Represents the downsampling scale, that is, the prediction mask ( ) and the true mask (M) are downsampled by 1 times (original scale), 2 times, and 4 times respectively to obtain the predicted masks at different sampling scales and the true mask ; It is the Sobel gradient operator, which calculates the gradient amplitude of the image at each pixel through convolution. The formula is:

[0071] is the input image, represents the gradient of the image in the x direction, Represents the gradient of the image in the y direction.

[0072] Regularization constraints ( ) using morphological consistency loss:

[0073] in, Cell masks predicted by the model; Prediction mask The area (unit: pixel²); is the preset mean cell area (unit: pixel²); To predict the circularity of cells (morphological parameter); is the preset mean value of cell circularity. The first item here is the area term, divided by the standard deviation of the area of ​​the model training set ( ), we get “(Area deviation) / "(dimensionless); the second term is the circularity term, divided by the standard deviation of the circularity of the model training set (σ cire ), we get "(circularity deviation) / σ cire ” (dimensionless). In this way, after normalization, both terms are converted into dimensionless standardized deviations and can be directly added.

[0074] S4.2, Data Enhancement Strategy: In terms of morphological enhancement, elastic deformation is used to simulate cell extrusion (α=120, σ=8, application probability 40%), and directional deformation is used to stretch along the long axis of the cell, with a deformation rate set to 15%. Among them, α here is the deformation strength parameter, which is used to control the amplitude (intensity) of the elastic deformation. The larger the α value, the more significant the degree of extrusion / distortion of the cell morphology; otherwise, the deformation is milder. In the present invention, α=120, indicating that the real state of cell extrusion is simulated by medium-intensity deformation. σ is the standard deviation of the Gaussian kernel, which is used to control the smoothness of the deformation field. The larger the σ value, the smoother the generated deformation field (the more natural the deformation transition); otherwise, local severe deformation may occur. In the present invention, σ=8 to ensure that the deformed cell morphology still conforms to the biological characteristics (avoiding morphological abnormalities caused by excessive distortion).

[0075] In terms of fluorescence enhancement, channel separation enhancement was used to independently process the HER2 / CEP17 channels. Signal attenuation was simulated to randomly reduce the fluorescence intensity in the edge areas.

[0076] In terms of adversarial sample generation, 15% of adversarial samples are added to the training set, and pre-trained GAN is used to generate samples such as adhesion cells and blurred boundaries.

[0077] S4.3. Dynamic training strategy: This step integrates learning rate scheduling, early stopping mechanism, and model preservation strategy. The specific process is as follows: S4.3.1. Dynamic adjustment of learning rate: Warm-up phase: The first five training rounds use a linear ramp-up strategy to increase the learning rate from the initial value to the target value, which is set to .

[0078] Cosine annealing decay, starting from the 6th round, according to the formula Adjust the learning rate to achieve smooth decay.

[0079] Among them, cosine annealing uses the periodic change of the cosine function to make the learning rate decay smoothly with the training rounds (epochs), avoid gradient mutations, and improve the convergence stability of the model. represents the learning rate; is the initial learning rate; epoch is the current iteration in the training process (an integer that increases gradually during training, such as epoch 1, epoch 2, etc.); π is the circumference ratio, which is used to construct the phase change of the cosine function; 100 represents the circular adjustment parameter, which is used to define the "half-cycle length" of the cosine decay.

[0080] Restart mechanism: If the validation loss decreases by less than 0.1% for 10 consecutive rounds, the learning rate is reset to , breaking the optimization stagnation.

[0081] S4.3.2 Early Stop and Checkpoint Control: Monitoring indicators: F1 score of the boundary ( ) as the core indicator, integrating the precision and recall rate of cell contour segmentation.

[0082] Early stopping mechanism: If 15 consecutive rounds If the improvement is less than 0.5%, the training is terminated to avoid overfitting.

[0083] Best model preservation: full recording The peak value of is saved, and the model and optimizer status of the corresponding round are saved to ensure that the final output is the optimal model.

[0084] S5. Dual-model segmentation prediction: Use the trained first model and second model to segment the FISH image, output the segmentation mask, category identifier and confidence of the first model, and the segmentation mask of the second model; In this example, the FISH image to be segmented is input, and the trained Cellpose model is called to disable segmentation prediction. The specific process is as follows: S5.1. Model call: Use the eval method of the trained Cellpose model to segment the input FISH image, such as Figure 2 shown.

[0085] S5.2. Segmentation parameter adjustment: Adjust the segmentation parameters according to the characteristics of the FISH image. The specific parameters are as follows: Cell diameter (diameter): This parameter specifies the approximate cell diameter. Properly setting the cell diameter helps improve segmentation accuracy. Setting this parameter to None allows the model to automatically estimate the cell diameter, adapting to variations in cell size across images and improving segmentation versatility and accuracy.

[0086] Flow field threshold (flow_threshold): This threshold controls the sensitivity of the flow field. In images with high cell density, lowering this threshold can detect more cells, but this can also introduce excessive noise. Here, a setting of 0.4 strikes a balance between detecting more nuclei and controlling noise, allowing for detection of most cells without introducing excessive noise interference.

[0087] Cell probability threshold (cellprob_threshold): The probability value used to filter cells. Setting the threshold to 0.0 means detecting as many cells as possible.

[0088] Minimum cell size (min_size): used to filter out cells that are too small. Here it is set to 15 to avoid mistaking noise or too small interference for cells.

[0089] Batch size (batch_size): The batch size determines the number of images processed at a time. Here it is set to 8 to balance between processing speed and memory resources.

[0090] Image normalization: Setting normalize to True normalizes the input image. Normalization adjusts the pixel values ​​of the image to a uniform range, which helps improve the stability of the model and the segmentation effect.

[0091] Data augmentation: Data augmentation operations (such as rotation, flipping, etc.) can increase the diversity of data, but here it is set to False and no data augmentation is required.

[0092] In this embodiment, the trained YOLO model is called and the predict method of the model is used to perform segmentation prediction on the input FISH image, such as Figure 3 As shown in the figure, the red border represents the "cell" category and the green border represents the "other" category. This method receives parameters such as image path, save option, device selection, whether to output bounding box, and confidence threshold as input.

[0093] Output results: After segmentation processing by the YOLO model, the segmentation results will be obtained, mainly including the following information: Prediction mask: It is a three-dimensional array, where each channel represents a predicted segmentation region. The mask can be used to determine the position and range of each predicted region in the image.

[0094] Category ID (cls_ids): indicates the category to which each predicted region belongs. Different category IDs correspond to different cell categories.

[0095] Confidence (confs): reflects the credibility of each prediction result. The higher the value, the more reliable the prediction result.

[0096] S5.3. Output results: After processing by the Cellpose model, three main output results are obtained: Cell mask: It is a two-dimensional array, each different integer value represents a segmented cell, and the value of the background area is 0. Through this mask, you can clearly see the position and range of each cell in the image.

[0097] Flow fields: contain the flow information of cells, which can be used to further analyze the morphology and movement characteristics of cells.

[0098] Style information (styles): reflects the overall characteristics of the image and can be used to classify or cluster the image.

[0099] S6. Optimization of the first model segmentation results: Post-process the segmentation results of the first model (YOLO), including confidence ranking, mask scaling, morphological operations and overlapping area removal, such as Figure 4 As shown; In this embodiment, the specific steps are as follows: S6.1. Confidence sorting to obtain a sorting index. Sort the segmentation masks from large to small based on their confidence, and rearrange the segmentation masks and category IDs based on the sorting index.

[0100] S6.2. Initialize the saved mask array: Initialize the saved mask data, set all values ​​to 0, and keep the size consistent with the actual height and width of the original image.

[0101] S6.3. Traverse the prediction results and process a single mask: After sorting by confidence, traverse each segmentation mask and its corresponding category ID, and perform the following processing on each mask in turn: S6.3.1. Mask image scaling: Scale the mask image to the source image size using quadratic spline interpolation.

[0102] S6.3.2 Mask dilation: Perform a morphological dilation operation on the binary mask. The structural element of the dilation process is a 3*3 circular element, which increases the area of ​​the binary mask.

[0103] S6.3.3. Single-cell image extraction: Convert the input FISH image from RGB to HSV color space, extract its luminance (V) component, and normalize it. Perform a logical AND operation on the luminance V component and the processed binary mask image to obtain a refined segmentation mask for the single-cell region.

[0104] S6.3.4. Threshold segmentation of cell regions: Use the Otsu threshold method to further segment the brightness (V) component of the true cell region to more accurately distinguish between cells and background regions, and obtain a refined segmentation mask for the cell region.

[0105] S6.3.5 Morphological Closing: Perform a morphological closing operation on the refined segmentation mask. The closing operation uses a 3*3 rectangular result element to eliminate holes in the binary area, making the cells more complete.

[0106] S6.3.6. Overlapping area removal: perform a negation operation on the saved mask array and perform a logical AND operation on the processed refined segmentation mask to obtain a non-overlapping part mask.

[0107] S6.3.7. Saved mask update: Perform a logical OR operation on the non-overlapping mask and the saved mask array, and update the saved mask array.

[0108] S7. Dual-model collaborative decision-making: Perform a logical AND operation on the segmentation mask of the second model (Cellpose) and the optimized segmentation mask of the first model, determine the cell category based on the number and distribution of connected domains in the operation result, and generate the final detection result.

[0109] In this embodiment, the FISH image is subjected to a logical operation on the segmentation result of the Cellpose model and the segmentation classification result of the Yolo model to obtain the final segmentation result, such as Figure 5 As shown, the red border represents the "cell" category and the green border represents the "other" category. The specific process is as follows: S7.1. Extract the segmentation results of the Cellpose model, perform a logical AND operation on the segmentation results of the Yolo model and the cells segmented by the Cellpose model, and count the results of the logical AND operation.

[0110] If there is only one connected domain after the AND operation, the classification of the cell is determined based on the classification results predicted by the YOLO model. Category ID = 0 represents "cell" and category ID = 1 represents "other".

[0111] If multiple connected domains exist after the AND operation, the cell classification is determined based on the number of pixels in the connected domain. The Yolo model prediction result with the largest number of pixels is used as the cell classification, with category ID = 0 representing "cell" and category ID = 1 representing "other."

[0112] If there is no connected domain after the AND operation, the cell category is determined using the following method: The Cellpose model's results are then dilated using a 1x1 square structuring element. The cells segmented by the Cellpose model are continuously dilated. After each dilation, the image is logically ANDed with the Yolo model results. If there is an overlap, dilation is stopped. Otherwise, dilation is continued until overlap with the Yolo model occurs. Finally, the Cellpose model's classification result is determined based on the YOLO model's classification of the overlapping area.

[0113] S7.2. Loop through all the cells segmented by the Cellpose model to confirm the classification categories and generate the classification results for the entire image.

[0114] Example 2 Based on the same concept, the present invention also proposes an automated cell detection device for fluorescence in situ hybridization images, comprising: The data preparation module acquires and annotates FISH images, dividing the cells in the images into the first category of cells for which fluorescence point counting is required, and the second category of cells that are not yet analyzed. The first category of cells are complete and single cells in the field of view with uniform size and consistent morphology, while the second category of cells are cell clusters with incomplete boundaries or adhesions. The dataset module converts the annotation results into the standard format required by the first model. The conversion includes coordinate normalization, category coding and quality control. The first model module loads the pre-trained parameters of the first model and fine-tunes the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process uses a combination of multiple loss functions and combines data enhancement and regularization strategies. The second model module loads the pre-trained parameters of the second model and adapts the input channels to suit the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data augmentation and dynamic training strategies; The dual-model segmentation prediction module segments the FISH image using the trained first model and the second model, and outputs the segmentation mask, category identifier, and confidence score of the first model, as well as the segmentation mask of the second model. The result optimization module performs post-processing on the segmentation results of the first model, including confidence-based sorting, mask scaling, morphological operations, and overlapping area removal; The collaborative decision-making module performs a logical AND operation on the segmentation mask of the second model and the optimized segmentation mask of the first model, determines the cell category based on the number and distribution of connected domains in the operation result, and generates the final detection result.

[0115] Example 3 This embodiment also provides an electronic device, referring to Figure 6 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0116] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0117] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0118] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0119] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the automated cell detection methods for fluorescence in situ hybridization images in the above embodiments.

[0120] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0121] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0122] The input / output device 408 is used to input or output information.

[0123] Example 4 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the automated cell detection method based on fluorescence in situ hybridization images according to the first embodiment.

[0124] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0125] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0126] Embodiments of the present invention can be implemented by computer software, which is executable by the data processor of the mobile device, such as in the processor entity, or is implemented by hardware, or is implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer executable components configured to perform the embodiment when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logical circuits, boxes and functions, or a combination of program steps and logical circuits, boxes and functions. The software can be stored in physical media such as memory chips or storage blocks implemented in the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. Physical media is non-transient media.

[0127] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. An automated cell detection method using fluorescence in situ hybridization images, characterized in that: The following steps are involved: S1. Data preparation: Acquire and annotate FISH images, and divide the cells in the images into the first category of cells for which fluorescence spot counting is required, and the second category of cells that are not yet analyzed. The first category of cells are complete and individual cells in the field of view with uniform size and consistent morphology. The second category of cells are cell clusters with incomplete boundaries or adhesions. S2. Generate a dataset: convert the annotation results into the standard format required by the first model, the conversion including coordinate normalization, category coding and quality control; S3. First model training: loading pre-trained parameters of the first model and fine-tuning the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process adopts a combination loss function including multiple loss functions, and combines data enhancement and regularization strategies. S4. Second model training: Load the pre-trained parameters of the second model and adapt the input channels to adapt to the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data enhancement and dynamic training strategies. S5. Dual-model segmentation prediction: Use the trained first model and second model to segment the FISH image, output the segmentation mask, category identifier and confidence of the first model, and the segmentation mask of the second model; S6. Optimizing the segmentation results of the first model: performing post-processing on the segmentation results of the first model, including confidence ranking, mask scaling, morphological operations, and overlapping area removal; S7. Dual-model collaborative decision-making: Perform a logical AND operation on the segmentation mask of the second model and the optimized segmentation mask of the first model, determine the cell category based on the number and distribution of connected domains in the operation result, and generate the final detection result.

2. The automated cell detection method of fluorescence in situ hybridization images according to claim 1, wherein: In step S1, the annotation is carried out by a double-person annotation verification method, and the ambiguous samples are finally interpreted by a third-party expert, and the annotation tool is an image annotation tool.

3. The automated cell detection method of fluorescence in situ hybridization images according to claim 1, wherein: In step S2, the quality control includes filtering invalid annotations, checking polygon closure, and verifying vertex coordinate ranges.

4. The automated cell detection method of fluorescence in situ hybridization images according to claim 1, wherein: In step S3, the first model is a Yolov12 model, the combined loss function includes a weighted combination of Dice loss, Focal loss, Jaccard loss, mutual exclusion loss and soft sampling weighted loss, the data enhancement includes rotation, flipping and color space adjustment, and the regularization strategy includes label smoothing, weight attenuation and random dropout.

5. The automated cell detection method of fluorescence in situ hybridization images according to claim 1, wherein: In step S4, the second model is a Cellpose model, the mask loss is a joint loss of Dice loss and cross entropy loss, the boundary loss is a multi-scale matching loss, the regularization loss is a morphological consistency loss, and the targeted data enhancement includes elastic deformation, fluorescence channel separation, and incorporation of adversarial samples.

6. The automated cell detection method of fluorescence in situ hybridization images according to claim 1, wherein: In step S4, the dynamic training strategy includes a learning rate warm-up, a cosine annealing decay and restart mechanism, and an early stopping mechanism based on a boundary evaluation index.

7. The automated cell detection method of fluorescence in situ hybridization images according to claim 6, characterized in that: In step S7, when the result of the logical AND operation contains a single connected domain, the category identifier predicted by the first model is directly used; when there are multiple connected domains, the category identifier of the first model corresponding to the connected domain with the largest number of pixels is used as the result; when there is no connected domain, the segmentation mask of the second model is expanded until it overlaps with the segmentation mask of the first model, and the category identifier of the first model corresponding to the overlapping area is used as the result.

8. An automated cell detection device for fluorescence in situ hybridization images, characterized in that: include: The data preparation module acquires and annotates FISH images, dividing the cells in the images into the first category of cells for which fluorescence point counting is required, and the second category of cells that are not yet analyzed. The first category of cells are complete and single cells in the field of view with uniform size and consistent morphology, while the second category of cells are cell clusters with incomplete boundaries or adhesions. The dataset module converts the annotation results into the standard format required by the first model. The conversion includes coordinate normalization, category coding and quality control. The first model module loads the pre-trained parameters of the first model and fine-tunes the parameters based on the dataset. The first model is a deep learning model with classification and detection capabilities. The training process uses a combination of multiple loss functions and combines data enhancement and regularization strategies. The second model module loads the pre-trained parameters of the second model and adapts the input channels to suit the FISH image characteristics. The second model is a deep learning model with fine boundary segmentation capabilities. The training process uses a combined loss function including mask loss, boundary loss, and regularization loss, combined with targeted data augmentation and dynamic training strategies; The dual-model segmentation prediction module segments the FISH image using the trained first model and the second model, and outputs the segmentation mask, category identifier, and confidence score of the first model, as well as the segmentation mask of the second model. The result optimization module performs post-processing on the segmentation results of the first model, including confidence-based sorting, mask scaling, morphological operations, and overlapping area removal; The collaborative decision-making module performs a logical AND operation on the segmentation mask of the second model and the optimized segmentation mask of the first model, determines the cell category based on the number and distribution of connected domains in the operation result, and generates the final detection result.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the automated cell detection method based on fluorescence in situ hybridization images according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the automated cell detection method based on fluorescence in situ hybridization images according to any one of claims 1 to 7.

Images