Intelligent counting method, system and device for fluorescence signals in fish images

Abstract

The application relates to the technical field of medical image processing, and discloses an intelligent counting method, system and device for fluorescent signals in a FISH image, wherein a full-automatic process is used to extract multi-dimensional characteristics of red and green fluorescent probes in a breast cancer FISH image, a differential Gaussian enhancement algorithm is combined with a neural network model of a multi-scale feature extraction and a self-attention mechanism to realize accurate counting of fluorescent signals. The differential Gaussian enhancement algorithm is used to suppress background noise and highlight fluorescent point regions; traditional characteristics are expanded to cover four dimensions of geometry, intensity, shape and quality, so that the fluorescent point characteristics are comprehensively described; a three-branch parallel feature extractor is used to capture feature information at different levels; important fluorescent point positions are automatically identified through a space-channel dual attention mechanism, and discriminative feature dimensions are intelligently selected; and a deep neural network is used to automatically learn optimal feature representation, automatically adjust weights during a training stage, and improve processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and in particular to an intelligent counting method, system and device for fluorescent signals in a FISH image. BACKGROUND

[0002] Fluorescence in situ hybridization (FISH) technology has become a core means for molecular pathological analysis of breast cancer due to its precise detection capability for specific gene amplification or deletion. In FISH detection, the accuracy of fluorescent signal counting directly affects the analysis and judgment of gene status.

[0003] At present, the fluorescence automatic method of FISH image is mainly based on traditional image processing technology, which relies on fixed threshold and manual setting rules, and has the following problems:

[0004] 1. The threshold parameter needs to be adjusted manually, which is difficult to adapt to the changes of imaging conditions of different samples;

[0005] 2. The recognition method based on simple geometric features is easily disturbed by background noise, signal diffusion, shape variation and other factors;

[0006] 3. Lack of self-adaptive learning ability, unable to learn and optimize feature expression from a large amount of data;

[0007] 4. The recognition accuracy of fluorescent points with fuzzy boundary and uneven intensity is not high.

[0008] The Chinese invention patent with publication number CN106296635A proposes a fluorescence in situ hybridization (FISH) image parallel processing and analysis method, which uses parallel processing to simultaneously detect chromosome fluorescent markers and detect and segment cell nuclei, and uses a watershed algorithm based on adaptive shape markers to improve the accuracy of relative position detection of cell nuclei and chromosome fluorescent markers in tumor cells. However, the patent uses traditional image processing methods such as adaptive threshold, morphological processing and watershed algorithm for FISH image analysis, and its performance depends on fixed parameters set manually, which is difficult to adapt to image quality differences caused by different staining batches and different imaging devices. It has poor robustness in the case of uneven brightness, high background noise or signal diffusion. Moreover, the fluorescent counting method is mainly based on simple geometric and gray level features such as area, shape and intensity threshold to detect fluorescent points, which has limited feature expression ability and cannot effectively deal with practical problems such as fluorescent signal overlap, breakage, weak signal or atypical morphology, which is easy to cause false detection or missed detection. SUMMARY

[0009] The application is to solve the technical problems of low recognition accuracy of fluorescent points and lack of self-adaptive learning ability, and proposes an intelligent counting method, system and device for fluorescent signals in a FISH image, a full-automatic process extracts multi-dimensional features of red and green fluorescent probes in a breast cancer FISH image, combines a differential Gaussian enhancement algorithm preprocessing, a multi-scale feature extraction and a neural network model of an attention mechanism, and realizes accurate counting of fluorescent signals.

[0010] To achieve the above-mentioned application purposes, the embodiments of the application provide the following technical solutions:

[0011] An intelligent counting method for fluorescent signals in a FISH image comprises the following sub-steps:

[0012] S1: inputting a breast cancer FISH image, enhancing fluorescent signals through a differential Gaussian enhancement algorithm, and extracting red and green fluorescent point candidate regions;

[0013] S2: performing connected domain analysis on the red and green fluorescent point candidate regions, and extracting a plurality of feature vectors;

[0014] S3: inputting the plurality of feature vectors into a multi-scale feature extractor, extracting feature vectors of different levels through a parallel branch network, and unifying the feature vectors of different levels into the same level;

[0015] S4: adopting a dual attention mechanism of channel attention and spatial attention for a feature matrix output by the multi-scale feature extractor, and automatically learning the relative importance between fluorescent points and the weight distribution of feature dimensions;

[0016] S5: using a self-attention mechanism for the feature vectors adopting the dual attention mechanism, weighting and aggregating all fluorescent point features in cells to form cell-level feature representations;

[0017] S6: predicting the cell-level feature representations through a double-head regression structure of a deep neural network, predicting the counting values of red and green fluorescent probes, adaptively optimizing a training process through a total loss function, and obtaining fluorescent probe counting results;

[0018] S7: repeating the dual attention mechanism fusion, cell-level feature aggregation and intelligent counting regression for all target cells in the breast cancer FISH image, and statistically obtaining fluorescent probe counting results of all cells.

[0019] Further, an intelligent counting method for fluorescent signals in a FISH image, S1 comprises the following sub-steps:

[0020] S11: performing RGB channel separation on the input breast cancer FISH image, mapping a gray scale to a red channel, a green channel and a blue channel, and extracting pixel values of each channel;

[0021] S12: Apply different scale Gaussian filter to the image of red channel and green channel respectively, calculate the difference enhancement result, and obtain the image highlighting the fluorescent point region;

[0022] S13: Perform binaryzation processing on the image highlighting the fluorescent point region by using adaptive threshold algorithm, and convert it into a binary image;

[0023] S14: Perform morphological optimization processing on the binary image, and obtain a plurality of red fluorescent point candidate regions and green fluorescent point candidate regions.

[0024] Specifically, S12 includes the following sub-steps:

[0025] S121: Apply small-scale Gaussian kernel and large-scale Gaussian kernel to the original image of red channel and green channel respectively, and obtain the filtering result:

[0026]

[0027] wherein, is the small-scale filtering result of red channel, which retains the fluorescent point details; is the large-scale filtering result of red channel, which smoothes the background of red channel; is the small-scale filtering result of green channel, which retains the fluorescent point details; is the large-scale filtering result of green channel, which smoothes the background of green channel;

[0028] wherein, is the original image of red channel; is the original image of green channel;

[0029] wherein, is performing Gaussian filtering process with as the scale, is performing Gaussian filtering process with as the scale; is performing Gaussian filtering process with as the scale, is performing Gaussian filtering process with as the scale;

[0030] wherein, is small-scale Gaussian kernel, which is used to capture the fluorescent point detail signal;

[0031] wherein, is large-scale Gaussian kernel, which is used to smooth the background noise;

[0032] S122: the filtering result is enhanced by using a differential Gaussian enhancement algorithm to obtain a red channel differential enhancement image and a green channel differential enhancement image highlighting the fluorescent point region:

[0033]

[0034] wherein, is the red channel differential enhancement image, is the green channel differential enhancement image, and 0.3 is an empirical optimal coefficient.

[0035] Further, an intelligent counting method of fluorescent signals in a FISH image, S2 includes the following sub-steps:

[0036] Calculate geometric features: the geometric features include area, average intensity, maximum intensity, eccentricity, and compactness;

[0037] Calculate shape features: the shape features include circularity, aspect ratio, and perimeter;

[0038] Extract intensity features: the intensity features include signal-to-noise ratio, intensity standard deviation, intensity range, and intensity coefficient of variation;

[0039] Extract quality features: the quality features include comprehensive quality score, tightness, and intensity-area product.

[0040] Further, an intelligent counting method of fluorescent signals in a FISH image, S3 includes the following sub-steps:

[0041] S31: input a plurality of feature vectors into a large-scale feature branch, a medium-scale feature branch, and a small-scale feature branch, respectively, the large-scale feature branch expands the feature dimension of the feature vector to 128 dimensions, the medium-scale feature branch expands the feature dimension of the feature vector to 96 dimensions, and the small-scale feature branch expands the feature dimension of the feature vector to 64 dimensions;

[0042] S32: concatenate the feature vectors of the three branches to obtain a 288-dimensional combined feature, and unify the 288-dimensional combined feature into a 128-dimensional feature vector through two-layer fusion layer operation.

[0043] Specifically, S31 includes the following sub-steps:

[0044] Large-scale feature branch: perform layer normalization processing on a plurality of feature vectors, a first fully connected layer operation processes the normalized feature vectors, maps the feature dimension to 64 dimensions, introduces a rectified linear unit activation function to express nonlinear features, cooperates with a random deactivation operation to suppress model overfitting, a second fully connected layer operation processes the fitted feature vectors, and maps the feature dimension to 128 dimensions,

[0045] The rectified linear unit activation function and the random deactivation operation are applied to suppress the model to complete extraction and optimization of large-scale features.

[0046] The mid-scale feature branch: a plurality of feature vectors are subjected to layer normalization processing, the normalized feature vectors are processed by a first full connection layer, the feature dimension is mapped to 48 dimensions, the rectified linear unit activation function is introduced to express nonlinear features, and the model is fitted in cooperation with the random deactivation operation to suppress overfitting, the fitted feature vectors are processed by a second full connection layer, and the feature dimension is mapped to 96 dimensions.

[0047] The rectified linear unit activation function and the random deactivation operation are applied to suppress the model to complete extraction and optimization of mid-scale features.

[0048] The small-scale feature branch: a plurality of feature vectors are subjected to layer normalization processing, the normalized feature vectors are processed by a first full connection layer, the feature dimension is mapped to 32 dimensions, the rectified linear unit activation function is introduced to express nonlinear features, and the model is fitted in cooperation with the random deactivation operation to suppress overfitting, the fitted feature vectors are processed by a second full connection layer, and the feature dimension is mapped to 64 dimensions.

[0049] The rectified linear unit activation function and the random deactivation operation are applied to suppress the model to complete extraction and optimization of small-scale features.

[0050] Specifically, S32 includes the following sub-steps:

[0051] S321: 128-dimensional large-scale features, 96-dimensional mid-scale features, and 64-dimensional small-scale features are dimensionally spliced to obtain 288-dimensional combined features, and multi-scale features are preliminarily integrated;

[0052] S322: The first fusion layer operation maps the 288-dimensional combined features to 256 dimensions, the rectified linear unit activation function is introduced to express nonlinear features, and the model is fitted in cooperation with the random deactivation operation to suppress overfitting, the second fusion layer operation processes the fitted feature vectors, and the 256-dimensional features are reduced to 128 dimensions to obtain 128-dimensional unified features.

[0053] Further, a method for intelligently counting fluorescent signals in a FISH image, S4 includes the following sub-steps:

[0054] Channel attention is adopted: the dimension of the feature vector is mapped and transformed by two convolution layers, the rectified linear unit is used to enhance the fitting ability of the fluorescent point features, the Sigmoid activation function is applied to generate a channel weight vector, and the feature dimensions with high discriminability are screened;

[0055] Adopting spatial attention: calculating the mean and maximum of the feature matrix, splicing the obtained mean map and maximum map, performing spatial feature fusion through a one-dimensional convolution layer, applying a Sigmoid activation function to generate a spatial weight vector, and focusing on screening the key fluorescent point positions in cells.

[0056] Specifically, adopting channel attention includes the following sub-steps:

[0057] Spatial average pooling: input The feature matrix is subjected to adaptive spatial average pooling along the spatial dimension, and multiple spatial features are compressed into a globally unified value;

[0058] First convolution layer mapping: obtaining The feature vector, The feature vector is mapped from 128 dimensions to 16 dimensions through the first convolution layer;

[0059] Applying a rectified linear unit: applying a rectified linear unit activation function to The feature vector introduces a nonlinear feature transformation to enhance the fitting ability of complex fluorescent point features;

[0060] Applying a Sigmoid activation function: restoring the enhanced feature vector to 128 dimensions through a second convolution layer, and applying a Sigmoid activation function to normalize the weight value to the [0, 1] interval to generate a 128-dimensional channel weight vector;

[0061] Wherein, z is the number of candidate fluorescent points, and the channel weight value is the contribution of the feature dimension to the identification of the true fluorescent point.

[0062] Specifically, adopting spatial attention includes the following sub-steps:

[0063] Calculating the mean and maximum: input The feature matrix is calculated along the channel dimension to obtain The mean map and the maximum map;

[0064] Splicing the mean map and the maximum map: the mean map and the maximum map are spliced through the channel dimension to complement the position sensitivity and obtain The feature map;

[0065] Convolution layer captures fluorescent points: The feature map is subjected to spatial feature fusion through a one-dimensional convolution layer to capture the local position correlation between fluorescent points;

[0066] Applying a Sigmoid activation function: applying a Sigmoid activation function to normalize the weight value to the [0, 1] interval to generate a spatial weight vector;

[0067] Wherein, the mean map reflects the global average information of the channel dimension, the maximum value map reflects the peak value information of the channel dimension, z is the number of candidate fluorescent points, and the spatial weight value is the probability that the candidate fluorescent point is a true red / green fluorescent signal.

[0068] Further, the S5 of the intelligent counting method of the fluorescent signal in the FISH image comprises the following sub-steps:

[0069] Multi-head self-attention calculation: linearly transform the feature vector, reshape the transformed feature into 8 attention heads, calculate the attention weight, and weight-sum the attention weight and the value to obtain the attention output;

[0070] Residual connection and layer normalization: add the attention output and the feature vector to form a residual connection, perform layer normalization processing to expand the feature dimension to twice, introduce a rectified linear unit activation function to express nonlinear features, cooperate with a random deactivation operation to suppress model overfitting, a second fully connected layer processes the fitted feature vector, restores the feature dimension to the original dimension, and performs residual connection and layer normalization again.

[0071] Weighted pooling aggregation: calculate the average value of all processed fluorescent point features to form a 128-dimensional feature representation at the cell level.

[0072] Specifically, the process of multi-head self-attention calculation: linearly transform the fluorescent point features through the query matrix, key matrix, and value matrix, reshape the transformed features into 8 attention heads, each head containing 16-dimensional features, calculate the dot product attention score of the query and the key, divide by the square root of the feature dimension for scaling, apply a soft-max function to obtain the attention weight, and weight-sum the weight and the value to obtain the attention output.

[0073] Specifically, the process of residual connection and layer normalization: add the attention output and the feature vector to form a residual connection, perform layer normalization processing, expand the feature dimension to twice through the first fully connected layer of the feedforward neural network, introduce a rectified linear unit activation function to express nonlinear features, cooperate with a random deactivation operation to suppress model overfitting, a second fully connected layer processes the fitted feature vector, restores the feature dimension to the original dimension, applies a rectified linear unit activation function and a random deactivation operation to suppress the model, and performs residual connection and layer normalization again.

[0074] Further, the S6 of the intelligent counting method of the fluorescent signal in the FISH image comprises the following sub-steps:

[0075] S61: map the 128-dimensional feature representation at the cell level to a 1-dimensional output through a fully connected network, and respectively predict the red fluorescent point count and the green fluorescent point count;

[0076] S62: Apply ReLU activation function to the prediction result to ensure that the count result is not less than zero, and use a dynamically weighted combination of mean square error loss function, smooth L1 loss function and Huber loss function to define an adaptive total loss function;

[0077] S63: Optimize the training process through the adaptive total loss function, dynamically adjust the weight according to the training progress, and obtain accurate count values of the red fluorescent probe and the green fluorescent probe.

[0078] Further, an intelligent counting method for fluorescent signals in a FISH image, the S62 includes the following sub-steps:

[0079] S621: Measure the squared difference between the predicted value and the true value to calculate the mean square error loss function:

[0080]

[0081] wherein, is the mean square error loss function, N is the number of fluorescent point count samples in the current batch, is the true value of the i-th sample, is the predicted value of the i-th sample;

[0082] S622: Combine the robustness of L1 loss and the smoothness of L2 loss to calculate the smooth L1 loss function:

[0083]

[0084] wherein, is the smooth L1 loss function;

[0085] S623: Combine the mean square error and the mean absolute error to calculate the Huber loss function:

[0086]

[0087] wherein, is the Huber loss function, is a hyperparameter;

[0088] S624: Dynamically weight the combination of mean square error loss function, smooth L1 loss function and Huber loss function to calculate the adaptive total loss function:

[0089]

[0090] wherein, is the adaptive total loss function, is the weight of is the weight of ​​is the weight.

[0091] Specifically, S63 includes the following sub-steps:

[0092] S631: each weight is dynamically adjusted according to the training progress:

[0093]

[0094] Wherein, progress represents the training progress, the value range is [0, 1].

[0095] An intelligent counting system for fluorescent signals in a FISH image, comprising an image preprocessing module, a feature extraction module, a deep learning coding module, an attention fusion module, a cell aggregation module, and an intelligent regression module.

[0096] The image preprocessing module is responsible for receiving breast cancer FISH images, enhancing fluorescent signals through a differential Gaussian enhancement algorithm, and extracting red and green fluorescent point candidate regions.

[0097] The feature extraction module is responsible for performing connected component analysis on the red and green fluorescent point candidate regions and extracting a plurality of feature vectors.

[0098] The deep learning coding module is responsible for receiving a plurality of feature vectors to a multi-scale feature extractor, extracting feature vectors at different levels through a parallel branch network, and unifying feature vectors at different levels to the same level.

[0099] The attention fusion module is responsible for using a dual attention mechanism of channel attention and spatial attention on the feature matrix output by the multi-scale feature extractor to automatically learn the relative importance between fluorescent points and the weight distribution of feature dimensions.

[0100] The cell aggregation module is responsible for using a self-attention mechanism on the feature vectors using the dual attention mechanism to weight and aggregate all fluorescent point features in the cell to form a cell-level feature representation.

[0101] The intelligent regression module is responsible for predicting the cell-level feature representation through a double-head regression structure of a deep neural network to predict the count values of red and green fluorescent probes, optimizing the training process with an adaptive total loss function, and obtaining fluorescent probe counting results. The double attention mechanism fusion, cell-level feature aggregation, and intelligent counting regression are repeated for all target cells in the breast cancer FISH image to obtain the fluorescent probe counting results of all cells.

[0102] An intelligent counting device for fluorescent signals in a FISH image, comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to execute the intelligent counting method for fluorescent signals in the FISH image.

[0103] The beneficial effects of the present application are: adopting a differential Gaussian enhancement algorithm to suppress background noise and highlight the fluorescent point area; expanding traditional features to cover four dimensions of geometry, intensity, shape and quality, and comprehensively describing the fluorescent point characteristics; adopting a three-branch parallel feature extractor to capture feature information at different levels; automatically identifying important fluorescent point positions through a space-channel dual attention mechanism and intelligently selecting discriminative feature dimensions; automatically learning the optimal feature representation through a deep neural network and automatically adjusting the weights according to the training phase; fully automated processing, eliminating subjective bias and improving processing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0104] Figure 1 is an example of a breast cancer FISH image.

[0105] Figure 2 is a flowchart of an intelligent counting method for fluorescent signals in a FISH image.

[0106] Figure 3 is a flowchart of multi-scale feature extraction.

[0107] Figure 4 is a flowchart of a self-attention mechanism.

[0108] Figure 5 is a structural diagram of an intelligent counting system for fluorescent signals in a FISH image. DETAILED DESCRIPTION

[0109] The present application will be further described, but the scope of protection of the present application is not limited to the following.

[0110] Specific embodiment one: an intelligent counting method for fluorescent signals in a FISH image.

[0111] In related technologies, existing breast cancer FISH image fluorescent counting relies on the defects of difficult fluorescent signal recognition, low counting efficiency, inability to fully automate counting, and incomplete fluorescent point characteristics.

[0112] In this embodiment, as shown in Figure 1 , a typical breast cancer fluorescence in situ hybridization image is input, with a size of 1024x768 pixels, and the image contains fluorescent cells-blue, human epidermal growth factor receptor 2 gene HER2 fluorescent probes-red, and 17 chromosome centromere CEP17 fluorescent probes-green.

[0113] As shown in Figure 2 , the intelligent counting method for fluorescent signals in a FISH image includes the following sub-steps:

[0114] S1: input a breast cancer FISH image, enhance the fluorescent signal through a differential Gaussian enhancement algorithm, and extract red and green fluorescent point candidate regions.

[0115] S1 specifically comprises the following sub-steps:

[0116] S11: RGB channel separation is performed on the input breast cancer FISH image, and gray scale is mapped to the red channel , green channel , and blue channel , and pixel values of each channel are extracted;

[0117] More specifically, the input red-green-blue image is separated into red, green, and blue channels, the gray scale value range of each channel is linearly mapped to the range of [0, 255], and the pixel values of the R channel, G channel, and B channel are extracted.

[0118] S12: Different scale Gaussian filters are applied to the images of the red channel and the green channel respectively, and the difference enhancement result is calculated to obtain an image highlighting the fluorescent point region;

[0119] S12 specifically comprises the following sub-steps:

[0120] S121: Small-scale and large-scale Gaussian kernels are respectively applied to the original images of the red channel and the green channel to obtain filter results:

[0121]

[0122] wherein, is the small-scale filter result of the red channel, which retains the fluorescent point details; is the large-scale filter result of the red channel, which smooths the red channel background; is the small-scale filter result of the green channel, which retains the fluorescent point details; is the large-scale filter result of the green channel, which smooths the green channel background;

[0123] wherein, is the original image of the red channel; is the original image of the green channel;

[0124] wherein, is a Gaussian filtering process with as the scale, is a Gaussian filtering process with as the scale; is a Gaussian filtering process with as the scale, is a Gaussian filtering process with as the scale;

[0125] wherein, , is a small-scale Gaussian kernel used to capture detailed signals of fluorescent dots;

[0126] in, , is a large-scale Gaussian kernel used to smooth background noise;

[0127] S122: The filtered results are enhanced using a differential Gaussian enhancement algorithm to obtain differentially enhanced images of the red channel and green channel highlighting the fluorescent point regions.

[0128]

[0129] in, It is a red channel differential enhancement image. This is a green channel differential enhancement image, and 0.3 is the empirically optimal coefficient.

[0130] S13: The image of the prominent fluorescent dot region is binarized using an adaptive thresholding algorithm and converted into a binary image;

[0131] S13 specifically includes the following sub-steps:

[0132] S131: Determine the background region. After sorting the pixel values ​​in the differentially enhanced image, identify the first 50% of the region using a background region mask.

[0133]

[0134] in It is the 50th percentile of pixel values ​​in the differentially enhanced image. It is a difference image in Gray values ​​in coordinates These are pixel coordinates. It is the background area mask.

[0135] S132: Calculate the mean and standard deviation of the background brightness based on the background region mask.

[0136]

[0137] in, yes The total number of pixels within, It is the average brightness of the background. It is the standard deviation of the background;

[0138] S133: Calculate the adaptive threshold and percentile threshold:

[0139]

[0140] in, It is an adaptive threshold. is the 85th percentile of the pixel values in the difference enhanced image; is the 85th percentile of the pixel values in the difference enhanced image;

[0141] S134: Based on the statistical characteristics that the fluorescent points in the breast cancer FISH image account for about 15% of the total pixels, the 85th percentile is selected, and the minimum value of the two threshold values is taken as the final threshold value for binarization:

[0142]

[0143] wherein, is the final threshold value for binarization;

[0144] S135: According to the final threshold value, the difference enhanced image is converted into a binary image:

[0145]

[0146] wherein, is the binary image.

[0147] S14: Morphological optimization processing is performed on the binary image to obtain a plurality of red fluorescent point candidate regions and green fluorescent point candidate regions.

[0148] S14 specifically includes the following steps:

[0149] S141: Open operation and closed operation are used to remove noise, a 3*3 circular structural element is used, open operation is used to remove small noise points, and closed operation is used to fill internal cavities to obtain red and green fluorescent point candidate regions.

[0150] In the embodiment, 124 red fluorescent point candidate regions and 89 green fluorescent point candidate regions are detected.

[0151] S2: Connected component analysis is performed on the red and green fluorescent point candidate regions to extract a plurality of feature vectors.

[0152] Specifically, the feature vectors in S2 include geometric features, shape features, intensity features, and quality features.

[0153] Geometric features: area, average intensity, maximum intensity, eccentricity, and compactness;

[0154] Shape features: circularity, aspect ratio, and perimeter;

[0155] Intensity features: signal-to-noise ratio, intensity standard deviation, intensity range, and intensity coefficient of variation;

[0156] Quality features: comprehensive quality score, tightness, and intensity-area product.

[0157] It should be noted that not all the above feature vectors are fixed, and it is not limited to the above feature vectors.

[0158] wherein the area is the number of pixels contained in the connected domain, the average intensity is the average intensity value of the pixels in the connected domain, the maximum intensity is the maximum intensity value of the pixels in the connected domain, the signal-to-noise ratio is the ratio of the average intensity to the background mean value, the intensity standard deviation is the standard deviation of the intensity of the pixels in the connected domain, the intensity range is the difference between the maximum and minimum values of the intensity of the pixels in the connected domain, the eccentricity is the eccentricity of the connected domain, the solidity is the ratio of the area of the connected domain to the area of the convex hull, the circularity is the ratio of 4π times the area to the square of the perimeter, the aspect ratio is the ratio of the length of the major axis to the length of the minor axis, and the comprehensive quality score is a comprehensive score based on the signal-to-noise ratio, the circularity, the solidity and the area. The intensity-area product is the product of the average intensity and the area, the intensity coefficient of variation is the ratio of the intensity standard deviation to the average intensity, and the compactness is the ratio of the area to the area of the convex hull.

[0159] More specifically, the geometric features are calculated: the connected domain area is calculated by contour analysis, the mask is created and the pixel values of the corresponding intensity region are extracted, and the average intensity and the maximum intensity are calculated; the shape features are calculated: the circularity is calculated by the contour perimeter, the formula is 4 times the circumference times the area divided by the square of the perimeter, when the number of contour points is not less than 5, the aspect ratio is calculated by ellipse fitting; the intensity features and quality features are extracted, including the signal-to-noise ratio, the intensity standard deviation, etc.

[0160] As shown in the embodiments, 15-dimensional feature vectors as shown in Table 1 are obtained.

[0161] Table 1: Example of 15-dimensional feature vector of single fluorescent spot

[0162]

[0163] S3: input a plurality of feature vectors to a multi-scale feature extractor, extract feature vectors of different levels through parallel branch networks, and unify feature vectors of different levels to the same level.

[0164] As shown in the embodiments, S3 specifically includes the following sub-steps: Figure 3

[0165] S31: input a plurality of feature vectors to a large-scale feature branch, a medium-scale feature branch and a small-scale feature branch respectively, the large-scale feature branch expands the feature dimension of the feature vector to 128 dimensions, the medium-scale feature branch expands the feature dimension of the feature vector to 96 dimensions, and the small-scale feature branch expands the feature dimension of the feature vector to 64 dimensions;

[0166] It should be noted that each branch adopts layer normalization + rectified linear unit activation function + random deactivation operation to inhibit the standardized structure of the model, thereby improving the training stability. ​

[0167] More specifically, S31 includes the following sub-steps:

[0168] Large-scale feature branch: perform layer normalization processing on a plurality of feature vectors, a first fully connected layer operation processes the layer normalized feature vectors, maps the feature dimension to 64 dimensions, introduces a rectified linear unit activation function to express non-linear features, and cooperates with a random deactivation operation to suppress model overfitting, a second fully connected layer operation processes the fitted feature vectors, maps the feature dimension to 128 dimensions, applies a rectified linear unit activation function and a random deactivation operation to suppress the model, and completes the extraction and optimization of large-scale features.

[0169] Medium-scale feature branch: perform layer normalization processing on a plurality of feature vectors, a first fully connected layer operation processes the layer normalized feature vectors, maps the feature dimension to 48 dimensions, introduces a rectified linear unit activation function to express non-linear features, and cooperates with a random deactivation operation to suppress model overfitting, a second fully connected layer operation processes the fitted feature vectors, maps the feature dimension to 96 dimensions, applies a rectified linear unit activation function and a random deactivation operation to suppress the model, and completes the extraction and optimization of medium-scale features.

[0170] Small-scale feature branch: perform layer normalization processing on a plurality of feature vectors, a first fully connected layer operation processes the layer normalized feature vectors, maps the feature dimension to 32 dimensions, introduces a rectified linear unit activation function to express non-linear features, and cooperates with a random deactivation operation to suppress model overfitting, a second fully connected layer operation processes the fitted feature vectors, maps the feature dimension to 64 dimensions, applies a rectified linear unit activation function and a random deactivation operation to suppress the model, and completes the extraction and optimization of small-scale features.

[0171] S32: Dimensionally splice the features of the three branches to obtain 288-dimensional combined features, and unify the 288-dimensional combined features into 128-dimensional feature vectors through two layers of fusion layer operations.

[0172] S32 includes the following sub-steps:

[0173] S321: Dimensionally splice the 128-dimensional large-scale features, 96-dimensional medium-scale features, and 64-dimensional small-scale features to obtain 288-dimensional combined features, and preliminarily integrate multi-scale features.

[0174] S322: A first fusion layer operation maps the 288-dimensional combined features to 256 dimensions, introduces a rectified linear unit activation function to express non-linear features, and cooperates with a random deactivation operation to suppress model overfitting, a second fusion layer operation processes the fitted feature vectors, reduces the 256-dimensional features to 128 dimensions, and obtains 128-dimensional unified features.

[0175] It should be noted that the random deactivation operation suppresses the model with a probability of 0.1.

[0176] In this embodiment, the input 15-dimensional fluorescent dot feature vector is specifically referenced in Table 1, covering geometric features (area 45, average intensity 187.3, maximum intensity 243, eccentricity 0.65, density 0.82), shape features (roundness 0.78, aspect ratio 1.45, perimeter 26.8), intensity features (signal-to-noise ratio 2.34, intensity standard deviation 23.7, intensity range 98, intensity coefficient of variation 0.127), and quality features (overall quality score 0.89, density 0.85, intensity-area product 8428.5). This ensures that the feature dimensions are fully matched with the input specifications of the multi-scale feature extractor, providing a standardized input basis for subsequent feature transformation.

[0177] S4: The feature matrix output by the multi-scale feature extractor adopts a dual attention mechanism of channel attention and spatial attention to automatically learn the relative importance between fluorescent points and the weight allocation of feature dimensions.

[0178] Specifically, channel attention involves the following sub-steps:

[0179] Spatial average pooling: input The feature matrix is ​​subjected to adaptive spatial average pooling along the spatial dimension to compress multiple spatial features into a globally uniform value;

[0180] First convolutional layer mapping: (Identifying) eigenvectors, The feature vectors are mapped from 128 dimensions to 16 dimensions through the first convolutional layer;

[0181] Applying corrected linear units: for The feature vectors are modified by applying the linear unit activation function and introducing nonlinear feature transformation to enhance the fitting ability to complex fluorescent point features.

[0182] Applying the Sigmoid activation function: The feature vector for enhanced fitting is restored to 128 dimensions through the second convolutional layer, and the weight values ​​are normalized to the [0,1] interval by applying the Sigmoid activation function to generate a 128-dimensional channel weight vector.

[0183] Where z is the number of candidate fluorescent spots, and the channel weight value is the contribution of the feature dimension to the identification of real fluorescent spots.

[0184] In this embodiment, the number of candidate fluorescent dots is 15; the spatial dimension is the same as the number of fluorescent dots; the kernel size of the first convolutional layer is 1×1, which compresses the dimension to 1 / 8 of the original size; the bottleneck structure enhances the nonlinear expression of key features and reduces computational complexity; the first convolutional layer has no bias and the kernel size is 1×1; the higher the channel weight value, the greater the contribution of the corresponding feature dimension to the recognition of real fluorescent dots, thus realizing the automatic selection of high-discriminative features.

[0185] Specifically, employing spatial attention involves the following sub-steps:

[0186] Calculate the mean and maximum: Input The mean and maximum values ​​of the feature matrix are calculated along the channel dimension to obtain... Mean and maximum value plots;

[0187] Stitching the mean and maximum plots: The mean and maximum plots are stitched together along the channel dimension, complementing each other to improve positional sensitivity and obtain... Feature map;

[0188] Convolutional layers capture fluorescent spots: The feature maps are fused spatially through a one-dimensional convolutional layer to capture the local positional relationships between fluorescent points.

[0189] Applying the Sigmoid activation function: Applying the Sigmoid activation function normalizes the weight values ​​to the [0,1] interval, generating a 15-dimensional weight vector.

[0190] Among them, the mean plot reflects the global average information of the channel dimension, the maximum plot reflects the peak information of the channel dimension, z is the number of candidate fluorescent spots, and the spatial weight value is the probability that the candidate fluorescent spot is a real red / green fluorescent signal.

[0191] In this embodiment, the number of candidate fluorescent spots is 15; the one-dimensional convolutional layer is unbiased, with a kernel size of 7 and a stride of 1; the higher the weight value of the 15-dimensional spatial weight vector, the higher the probability that the candidate fluorescent spot is a real HER2 / CEP17 signal, effectively suppressing the interference of background noise spots and pseudo fluorescent spots, and highlighting the contribution of key regions.

[0192] S5: Apply a self-attention mechanism to the feature vectors that employ a dual attention mechanism, and perform weighted aggregation of all fluorescent point features within the cell to form a cell-level feature representation.

[0193] Specifically, such as Figure 4 As shown, S5 includes the following sub-steps:

[0194] Multi-head self-attention calculation: The fluorescent point features are linearly transformed through the query matrix (Q), key matrix (K), and value matrix (V) respectively. The transformed features are reshaped into 8 attention heads (Head1, ..., Head8), each containing 16-dimensional features. The dot product attention score of query and key is calculated, and the result is scaled by the square root of the feature dimension. The soft maximization function is applied to obtain the attention weights. The weights and values ​​are weighted and summed to obtain the attention output.

[0195] Residual connection and layer normalization: the attention output is added to the feature vector to form a residual connection, and the layer normalization is processed. The feature dimension is expanded to twice through the first fully connected layer of the feedforward neural network. The rectified linear unit activation function is introduced to express the nonlinear characteristics. The random inactivation operation is used to suppress the model overfitting. The second fully connected layer operates the fitted feature vector to restore the feature dimension to the original dimension. The rectified linear unit activation function and the random inactivation operation are applied to the model to perform the residual connection and the layer normalization again.

[0196] Weighted pooling aggregation: the average value of all the processed fluorescent point features is calculated to form a 128-dimensional feature representation at the cell level.

[0197] In the embodiment, for an example cell containing 12 red fluorescent points and 8 green fluorescent points: the red fluorescent point feature is a 12*128 matrix, and the green fluorescent point feature is an 8*128 matrix. After the 8-head self-attention processing, the self-attention weight of the red fluorescent point is 8*12*12, and the mutual relationship between the red fluorescent points is learned. The self-attention weight of the green fluorescent point is 8*8*8, and the mutual relationship between the green fluorescent points is learned. Finally, the 128-dimensional cell-level features of red and green are aggregated respectively.

[0198] S6: The cell-level feature representation is predicted by the double-head regression structure of the deep neural network to predict the count values of the red fluorescent probe and the green fluorescent probe. The adaptive loss function is used to optimize the training process to obtain the fluorescent probe counting result.

[0199] Specifically, S6 includes the following sub-steps:

[0200] S61: The 128-dimensional cell-level feature representation is mapped to a 1-dimensional output through a fully connected network to predict the red fluorescent point count and the green fluorescent point count respectively.

[0201] More specifically, S611: the first layer of the fully connected network is mapped from 128 dimensions to 96 dimensions, and the rectified linear unit activation function is applied with a random inactivation probability of 0.2.

[0202] S612: the second layer is mapped to 64 dimensions, and the rectified linear unit activation function is applied with a random inactivation probability of 0.2.

[0203] S613: the third layer is mapped to 32 dimensions, and the rectified linear unit activation function is applied with a random inactivation probability of 0.1.

[0204] S614: the fourth layer is mapped to 16 dimensions, and the rectified linear unit activation function is applied. The fifth layer is mapped to a 1-dimensional output.

[0205] S62: Apply ReLU activation function to the prediction result to ensure that the counting result is not less than zero, and use a dynamic weighted combination of mean square error loss function, smooth L1 loss function and Huber loss function to define an adaptive total loss function;

[0206] S62 includes the following sub-steps:

[0207] S61: Map the 128-dimensional feature representation at the cell level to a 1-dimensional output through a fully connected network to predict the red fluorescent point count and the green fluorescent point count, respectively;

[0208] S62: Apply ReLU activation function to the prediction result to ensure that the counting result is not less than zero, and use a dynamic weighted combination of mean square error loss function, smooth L1 loss function and Huber loss function to define an adaptive total loss function;

[0209] S63: Optimize the training process through the adaptive total loss function, dynamically adjust the weight according to the training progress, and obtain accurate counting values of the red fluorescent probe and the green fluorescent probe.

[0210] Specifically, S62 includes the following sub-steps:

[0211] S621: Measure the squared difference between the predicted value and the true value to calculate the mean square error loss function:

[0212]

[0213] wherein, is the mean square error loss function, N is the number of fluorescent point counting samples in the current batch, is the true value of the i-th sample, is the predicted value of the i-th sample;

[0214] S622: Calculate the smooth L1 loss function by combining the robustness of L1 loss and the smoothness of L2 loss:

[0215]

[0216] wherein, is the smooth L1 loss function;

[0217] S623: Calculate the Huber loss function by combining the mean square error and the mean absolute error:

[0218]

[0219] wherein, is the Huber loss function, is a hyperparameter used to control the smoothness of the loss function and the sensitivity to outliers;

[0220] S624: dynamically combine the mean square error loss function, the smooth L1 loss function, and the Huber loss function to calculate an adaptive total loss function:

[0221]

[0222] wherein, is the adaptive total loss function, is a weight of is a weight of is a weight of is a weight of is a weight of is a weight of

[0223] Specifically, S63 includes the following sub-steps:

[0224] S631: each weight is dynamically adjusted according to the training progress:

[0225]

[0226] wherein, progress represents the training progress, and the value range is [0, 1].

[0227] It should be noted that the weight adjustment strategy ensures that the stability and robustness are focused on in the early stage of training (progress = 0), and each weight is (0.4, 0.4, 0.2); in the later stage of training (progress = 1), the counting accuracy is focused on, and each weight is (0.6, 0.3, 0.1).

[0228] In the embodiment, the single cell counting result is as shown in Table 2, and it can be seen that the prediction accuracy of the scheme has been greatly improved.

[0229] Table 2: Example of single cell counting result table

[0230]

[0231] S7: repeating the double attention mechanism fusion, cell-level feature aggregation, and intelligent counting regression for all target cells in the breast cancer FISH image, counting the fluorescence probe counting results of all cells, and generating an intelligent diagnosis report.

[0232] Specifically, S7: repeating S4 to S6 for all target cells in the fluorescence in situ hybridization image, counting the fluorescence probe counting results of all cells, and generating an intelligent diagnosis report.

[0233] In the embodiment, the complete multi-sample statistical analysis process is as shown in Table 3:

[0234] Table 3: Cell counting statistical result table of complete image

[0235]

[0236] In Table 3, image_id is the image number, cell_id is the cell number, R_Count is the counting result of the red fluorescent probe, G_Count is the counting result of the green fluorescent probe, and the mean values of R_Count and G_Count are given.

[0237] In the embodiment, all cells with an area > 500 pixels, a coefficient of variation of area < 0.3, a circularity > 0.6, and an eccentricity < 0.8 in the breast cancer FISH image are obtained, and the counting result (R_Count) of the red fluorescent probe and the counting result (G_Count) of the green fluorescent probe of all cells in the breast cancer FISH image are calculated.

[0238] Specific embodiment two, an intelligent counting system for fluorescent signals in a FISH image.

[0239] As shown in Figure 5 An intelligent counting system for fluorescent signals in a FISH image includes an image preprocessing module, a feature extraction module, a deep learning encoding module, an attention fusion module, a cell aggregation module, and an intelligent regression module.

[0240] The image preprocessing module is responsible for receiving a breast cancer FISH image, enhancing the fluorescent signal through a difference Gaussian enhancement algorithm, and extracting red and green fluorescent point candidate regions.

[0241] The feature extraction module is responsible for performing connected component analysis on the red and green fluorescent point candidate regions and extracting a plurality of feature vectors.

[0242] The deep learning encoding module is responsible for receiving a plurality of feature vectors to a multi-scale feature extractor, extracting feature vectors of different levels through a parallel branch network, and unifying feature vectors of different levels to the same level.

[0243] The attention fusion module is responsible for using a dual attention mechanism of channel attention and spatial attention on the feature matrix output by the multi-scale feature extractor, automatically learning the relative importance between fluorescent points and the weight distribution of feature dimensions.

[0244] The cell aggregation module is responsible for using a self-attention mechanism on the feature vectors using the dual attention mechanism, weighting and aggregating all fluorescent point features in the cell, and forming a cell-level feature representation.

[0245] The intelligent regression module is responsible for predicting the count values of the red fluorescent probe and the green fluorescent probe through the double-head regression structure of the deep neural network on the cell-level feature representation, optimizing the training process by the adaptive total loss function, and obtaining the fluorescent probe count result; and repeating the double attention mechanism fusion, cell-level feature aggregation, and intelligent counting regression on all target cells in the breast cancer FISH image to count the fluorescent probe count result of all cells.

[0246] Embodiments of the present application provide an intelligent counting device for fluorescent signals in a FISH image, comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to execute the intelligent counting method for fluorescent signals in a FISH image.

[0247] Embodiments of the present application provide a computer-readable storage medium storing instructions, when a computer executes the instructions, the computer executes the intelligent counting method for fluorescent signals in a FISH image.

[0248] The basic principles and main features of the present application and the advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent counting of fluorescent signals in a FISH image, characterized in that, The method comprises the following sub-steps: S1: inputting a breast cancer FISH image, enhancing fluorescent signals by a differential Gaussian enhancement algorithm, and extracting red and green fluorescent point candidate regions; S2: performing connected domain analysis on the red and green fluorescent point candidate regions to extract a plurality of feature vectors; S3: inputting the plurality of feature vectors into a multi-scale feature extractor, extracting feature vectors of different levels through a parallel branch network, and unifying the feature vectors of different levels into the same level; S4: adopting a dual attention mechanism of channel attention and spatial attention for a feature matrix output by the multi-scale feature extractor to automatically learn the relative importance between fluorescent points and the weight distribution of feature dimensions; S5: using a self-attention mechanism for the feature vectors adopting the dual attention mechanism to weight and aggregate all fluorescent point features in a cell to form a cell-level feature representation; S6: predicting the count values of red fluorescent probes and green fluorescent probes by a double-head regression structure of a deep neural network for the cell-level feature representation, adaptively optimizing a training process by using a total loss function, and obtaining fluorescent probe counting results; S7: repeating the dual attention mechanism fusion, cell-level feature aggregation, and intelligent counting regression for all target cells in the breast cancer FISH image to statistically obtain fluorescent probe counting results of all cells; The channel attention comprises the following sub-steps: Spatial average pooling: input The feature matrix is performed adaptive spatial average pooling along the spatial dimension, and multiple spatial features are compressed into a global unified value; The first convolutional layer maps: obtains a feature vector of dimension 128, The feature vector of dimension 128 is mapped by the first convolutional layer to a feature vector of dimension 16, The feature vector of dimension 128 is mapped by the first convolutional layer to a feature vector of dimension 16, Apply a rectified linear unit: to Apply a rectified linear unit activation function to the feature vector, introduce a nonlinear feature transformation, and enhance the fitting ability for complex fluorescent point features; applying a Sigmoid activation function: restoring the enhanced fitted feature vectors to 128 dimensions through a second convolutional layer, normalizing the weight values to the [0, 1] interval by applying the Sigmoid activation function, and generating a 128-dimensional channel weight vector; wherein z is the number of candidate fluorescent points, and the channel weight value is the contribution of the feature dimension to the identification of the true fluorescent point; The spatial attention comprises the following sub-steps: Compute mean and max: input The feature matrix is computed mean and max along the channel dimension, respectively, to obtain The mean map and max map of the feature matrix Concatenate mean and max maps: the mean and max maps are concatenated through the channel dimension, complementarily enhancing position sensitivity, obtaining the feature map of Convolutional layers capture fluorescent points: The feature map of the fluorescent point is fused by a one-dimensional convolutional layer to capture the local position correlation between the fluorescent points. applying a Sigmoid activation function: applying the Sigmoid activation function to normalize the weight values to the [0, 1] interval to generate a spatial weight vector; wherein the mean value map reflects the global average information of the channel dimension, the maximum value map reflects the peak value information of the channel dimension, z is the number of candidate fluorescent points, and the spatial weight value is the probability that the candidate fluorescent point is a true red / green fluorescent signal; The S6 comprises the following sub-steps: S61: mapping the 128-dimensional cell-level feature representation to a 1-dimensional output through a fully connected network to predict the red fluorescent point count and the green fluorescent point count, respectively; S62: applying a ReLU activation function to the prediction results to ensure that the counting results are not less than zero, and defining an adaptive total loss function by using a dynamic weighted combination of mean square error loss functions, smooth L1 loss functions, and Huber loss functions; S63: optimizing the training process by using the adaptive total loss function, dynamically adjusting the weight according to the training progress, and obtaining accurate count values of the red fluorescent probes and the green fluorescent probes.

2. The method of claim 1, wherein, The S1 comprises the following sub-steps: S11: performing RGB channel separation on the input breast cancer FISH image, mapping the gray scale to a red channel, a green channel, and a blue channel, and extracting pixel values of each channel; S12: Apply different scales of Gaussian filter to the images of red and green channels respectively, calculate the difference enhancement result, and obtain the image highlighting the fluorescent point region; S13: Perform binaryzation processing on the image highlighting the fluorescent point region using an adaptive threshold algorithm, and convert it into a binary image; S14: Perform morphological optimization processing on the binary image, and obtain a plurality of red fluorescent point candidate regions and green fluorescent point candidate regions.

3. The method of claim 1, wherein, The S2 includes the following sub-steps: Calculate geometric features: the geometric features include area, average intensity, maximum intensity, eccentricity, and compactness; Calculate shape features: the shape features include circularity and aspect ratio; Extract intensity features: the intensity features include signal-to-noise ratio, intensity standard deviation, intensity range, and intensity coefficient of variation; Extract quality features: the quality features include comprehensive quality score, tightness, and intensity-area product.

4. The method of claim 1, wherein, The S3 includes the following sub-steps: S31: input a plurality of feature vectors into a large-scale feature branch, a medium-scale feature branch, and a small-scale feature branch respectively, the large-scale feature branch expands the feature dimension of the feature vector to 128 dimensions, the medium-scale feature branch expands the feature dimension of the feature vector to 96 dimensions, and the small-scale feature branch expands the feature dimension of the feature vector to 64 dimensions; S32: perform dimension splicing on the feature vectors of the three branches to obtain a combined feature of 288 dimensions, and unify the combined feature of 288 dimensions into a feature vector of 128 dimensions through two layers of fusion layer operations.

5. The method of claim 1, wherein, The S4 includes the following sub-steps: Use channel attention: map and transform the dimension of the feature vector through two convolution layers, enhance the fitting ability of the feature vector to the fluorescent point feature by using a rectified linear unit, generate a channel weight vector by using a Sigmoid activation function, and screen feature dimensions with high discriminability; Use spatial attention: calculate the mean and maximum values of the feature matrix, splice the obtained mean image and maximum image, perform spatial feature fusion through a one-dimensional convolution layer, generate a spatial weight vector by using a Sigmoid activation function, and focus on screening the key fluorescent point positions in the cell.

6. The method of claim 1, wherein, The S5 includes the following sub-steps: Multi-head self-attention calculation: linearly transform the feature vector, reshape the transformed feature into 8 attention heads, calculate the attention weight, and obtain the attention output by weighting and summing the attention weight and the value; Residual connection and layer normalization: add the attention output and the feature vector to form a residual connection, perform layer normalization processing to expand the feature dimension to twice, introduce a rectified linear unit activation function to express nonlinear features, cooperate with a random inactivation operation to inhibit model overfitting, process the fitted feature vector through a second fully connected layer, restore the feature dimension to the original dimension, and perform residual connection and layer normalization again; Weighted pooling aggregation: calculate the average value of all the processed fluorescent point features to form a 128-dimensional feature representation at the cell level.

7. The method of claim 1, wherein, The S62 includes the following sub-steps: S621: measure the squared difference between the predicted value and the true value, and calculate the mean square error loss function: wherein, is the mean squared error loss function, N is the number of fluorescent dot count samples in the current batch, is the true value of the i-th sample, is the predicted value of the i-th sample; S622: combine the robustness of L1 loss and the smoothness of L2 loss, and calculate the smooth L1 loss function: wherein, is a smooth L1 loss function; S623: Calculate the Huber loss function by combining the mean square error and the mean absolute error: wherein, is the Huber loss function, is a hyperparameter; S624: Dynamically combine the mean square error loss function, the smooth L1 loss function, and the Huber loss function to calculate the adaptive total loss function: wherein, is an adaptive total loss function, is a weight of is a weight of is a weight of 8. An intelligent counting system of fluorescent signals in a FISH image, characterized in that, The image preprocessing module, the feature extraction module, the deep learning encoding module, the attention fusion module, the cell aggregation module, and the intelligent regression module are included. The image preprocessing module is responsible for receiving breast cancer FISH images, enhancing fluorescent signals through a differential Gaussian enhancement algorithm, and extracting red and green fluorescent point candidate regions. The feature extraction module is responsible for performing connected component analysis on the red and green fluorescent point candidate regions and extracting a number of feature vectors. The deep learning encoding module is responsible for receiving a number of feature vectors into a multi-scale feature extractor, extracting feature vectors at different levels through a parallel branch network, and unifying feature vectors at different levels to the same level. The attention fusion module is responsible for using a dual attention mechanism of channel attention and spatial attention on the feature matrix output by the multi-scale feature extractor to automatically learn the relative importance between fluorescent points and the weight distribution of feature dimensions. The cell aggregation module is responsible for using a self-attention mechanism on the feature vectors using a dual attention mechanism to weight and aggregate all fluorescent point features within a cell to form a cell-level feature representation. The intelligent regression module is responsible for predicting the count values of red and green fluorescent probes through a double-headed regression structure of a deep neural network, optimizing the training process with an adaptive total loss function, and obtaining fluorescent probe count results. The dual attention mechanism fusion, cell-level feature aggregation, and intelligent counting regression are repeated for all target cells in the breast cancer FISH image, and the fluorescent probe count results for all cells are calculated. The channel attention includes: Spatial average pooling: input The feature matrix is performed adaptive spatial average pooling along the spatial dimension, and multiple spatial features are compressed into a global unified value; The first convolutional layer maps: obtains a feature vector of dimension 128, The feature vector of dimension 128 is mapped by the first convolutional layer to a feature vector of dimension 16, The feature vector of dimension 128 is mapped by the first convolutional layer to a feature vector of dimension 16, Apply a rectified linear unit: to Apply a rectified linear unit activation function to the feature vector, introduce a nonlinear feature transformation, and enhance the fitting ability for complex fluorescent point features; Applying a Sigmoid activation function: The enhanced fitted feature vector is restored to 128 dimensions through the second convolutional layer, and the Sigmoid activation function is applied to normalize the weight values to the [0, 1] interval to generate a 128-dimensional channel weight vector. Where z is the number of candidate fluorescent points, and the channel weight value is the contribution of the feature dimension to the identification of the true fluorescent point. The spatial attention includes: Compute mean and max: input The feature matrix is computed mean and max along the channel dimension, respectively, to obtain The mean map and max map of the feature matrix Concatenate mean and max maps: the mean and max maps are concatenated through the channel dimension, complementarily enhancing position sensitivity, obtaining the feature map Convolutional layers capture fluorescent points: The feature map of the fluorescent point is fused by a one-dimensional convolutional layer to capture the local position correlation between the fluorescent points. Applying a Sigmoid activation function: The Sigmoid activation function is applied to normalize the weight values to the [0, 1] interval to generate a spatial weight vector. Where the mean value map reflects the global average information of the channel dimension, the maximum value map reflects the peak value information of the channel dimension, z is the number of candidate fluorescent points, and the spatial weight value is the probability that the candidate fluorescent point is a true red / green fluorescent signal. The double-headed regression structure for predicting the cell-level feature representation includes: Through a fully connected network, the 128-dimensional cell-level feature representation is mapped to a 1-dimensional output to predict the red and green fluorescent point counts, respectively. The predicted results are applied to the ReLU activation function to ensure that the count results are not less than zero, and the adaptive total loss function is defined by dynamically combining the mean square error loss function, the smooth L1 loss function, and the Huber loss function. The training process is optimized by an adaptive total loss function, and the weight is dynamically adjusted according to the training progress, so that the counting values of the red fluorescent probe and the green fluorescent probe are accurate.

9. An apparatus for intelligent counting of fluorescent signals in a FISH image, characterized in that The application relates to a computer program product, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the intelligent counting method of the fluorescent signals in the FISH image as described in any one of claims 1-7.

Images