Bacterial colony image time sequence classification tracking method based on dynamic time warping and label propagation

Abstract

The invention discloses a bacterial colony image time sequence classification tracking method based on dynamic time warping and label propagation, and solves the problems of inconsistent classification and accurate segmentation of bacterial colonies with different sizes caused by the change of bacterial colony morphology along with time. The method comprises the following steps: acquiring a bacterial colony image feature sequence and constructing a bacterial colony feature distance matrix; calculating an accumulated distance matrix based on a dynamic time warping algorithm, and calculating accumulated distance values point by point through a recursion formula; through a path backtracking algorithm, finding a time point matching path which minimizes an accumulated distance value between bacterial colonies, and obtaining an optimal time alignment relationship between feature sequences of different bacterial colonies; constructing a similarity matrix, and calculating a similarity value between bacterial colonies; and a classification corresponding relation between time points is established by adopting a cross-time-point target association algorithm, and the bacterial colony classification label at the first time point is propagated to the subsequent time point, so that the consistency of the time sequence classification labels is ensured.

Description

Technical Field

[0001] This invention relates to pathogenic microorganism colony image analysis and counting technology, and in particular to a colony image temporal classification and tracking method based on dynamic time warping and tag propagation. Background Technology

[0002] The technology for counting pathogenic microorganisms has evolved from traditional manual counting and basic instrument-assisted counting, to automatic counting dominated by traditional image processing techniques, and then to intelligent counting driven by deep learning and dynamic monitoring and multimodal data fusion counting. Currently, it mainly includes traditional image processing techniques based on CCD camera image acquisition-preprocessing-threshold segmentation-morphological operations-connected component counting processes, as well as end-to-end processing techniques based on deep learning models such as CNN and GAN.

[0003] In the prior art, US2018285624A1 discloses a method for colony culture detection and time measurement. This method receives a matrix image related to a first time, acquires the same matrix image related to a second time, and determines that a candidate colony and the first colony are the same colony by comparing a first region of the first image and a second region of the second image. However, this method only associates based on location information, lacks morphological feature analysis, does not consider complex situations such as colony division and fusion, and has limited ability to handle significant changes in colony morphology.

[0004] US2023060037A1 discloses a system for early detection and classification of live microorganisms based on time-series holographic imaging and deep learning. It uses a light source and incubator system to acquire time-series holographic images, moves an image sensor via a translation stage to capture multidimensional time-series data, and uses a trained deep neural network to process the time-series images for species identification. However, this method relies on holographic imaging equipment, is costly, has strict environmental requirements, and does not solve the problem of temporal consistency in colony morphological changes.

[0005] CN109948544B discloses an automatic target colony localization and identification method. This method involves acquiring color images of petri dishes and preprocessing them for morphological features to obtain binary images of colony outlines. Independent colonies are then selected based on morphological features, and target colonies are identified based on preset recognition conditions. However, this method only processes images at a single time point, lacks temporal analysis, has limited adaptability to changes in colony morphology, and does not consider the dynamic characteristics of the colony growth process.

[0006] Existing technologies suffer from the following technical defects: First, adhesion segmentation fails. Pathogenic microorganisms often adhere together due to their growth characteristics. For example, Klebsiella pneumoniae adheres in a mucus-like manner, and Streptococcus grows in chains. Traditional watershed algorithms are prone to over-segmentation or incomplete segmentation, resulting in a counting bias rate of 15%-30%. Second, time-series classification consistency is poor. The morphology of the same bacterium changes greatly at different time points, and there is no corresponding relationship between the classification numbers at different time points, making it impossible to effectively track the evolution of colonies. Third, traditional image processing techniques have weak anti-interference capabilities. When faced with clinical / environmental pathogenic microorganism samples containing a large number of impurities, the false positive rate is high. Deep learning methods are heavily dependent on rare pathogenic microorganism samples and have poor generalization ability. Summary of the Invention

[0007] The main technical problems with existing technologies are: the classification and tracking of colonies growing on the same petri dish at different time points, and the accurate segmentation of colonies of different sizes and densities in images. Specifically, this manifests as the contradiction between the significant changes in colony morphology over time and maintaining classification consistency, as well as the contradiction between improving detection efficiency and ensuring detection accuracy. It requires handling complex situations such as differences in colony growth rates, alignment of morphological changes, and unequal time sequences.

[0008] To address the aforementioned technical problems, this invention provides a colony image temporal classification and tracking method based on dynamic time warping and tag propagation, comprising the following steps: Step 1: Obtain the feature sequence of the colony image and construct the colony feature distance matrix. Specifically, this includes: first, obtaining the feature sequence of the colony from the colony image, where the feature sequence of the colony refers to the sequence formed by arranging the feature vectors of the same colony at different time points in chronological order; then, for the feature sequences of different colonies, using the Euclidean distance calculation formula, calculating the feature distance between any two colonies at different time points, and constructing the colony feature distance matrix. Step 2: Calculate the cumulative distance matrix based on the dynamic time warping algorithm, and calculate the cumulative distance value point by point using the recursive formula; Step 3: Find the time point matching path that minimizes the cumulative distance value between colonies in the cumulative distance matrix using the path backtracking algorithm, and obtain the optimal time alignment relationship between the feature sequences of different colonies; Step 4: Construct a similarity matrix based on the minimum cumulative distance between colonies and calculate the similarity values ​​between colonies; Step 5: Use a cross-time point target association algorithm to establish the classification correspondence between time points and generate the final colony classification and tracking results. Based on the established classification correspondence between time points, propagate the colony classification label of the first time point to subsequent time points to ensure the consistency of colony classification labels at different time points.

[0009] Preferably, in step one, the Euclidean distance calculation formula is D[i,j]=√(∑(k=1 to M)(A[i,k]-B[j,k]) 2 ), where A[i,k] represents the k-th feature vector of colony A at time i, B[j,k] represents the k-th feature vector of colony B at time j, k is the feature dimension index, M is the total dimension of the feature vector, and D[i,j] represents the feature distance between colony A at time i and colony B at time j. The feature vector includes colony area features, perimeter features, circularity features, RGB color features, and grayscale co-occurrence matrix texture features, wherein the circularity feature is calculated as 4π × area / perimeter. 2 The gray-level co-occurrence matrix texture features include four parameters: contrast, correlation, energy, and homogeneity. Contrast reflects the sharpness of the image, correlation reflects the similarity of gray-level co-occurrence matrix elements in the row or column direction, energy reflects the uniformity of gray-level distribution in the image, and homogeneity reflects the local changes in image texture. An 8×8 pixel window is used, and calculations are performed based on 256 gray levels in four directions: 0°, 45°, 90°, and 135°.

[0010] Preferably, in step two, the recursive formula is DTW[i,j]=D[i,j]+min(DTW[i-1,j],DTW[i,j-1],DTW[i-1,j-1]), where DTW[i,j] represents the cumulative distance between colony A from the first time point to the i-th time point and colony B from the first time point to the j-th time point, and the initialization condition is DTW[1,1]=D[1,1]; The constraint window size r of the dynamic time warping algorithm is adaptively adjusted according to the colony growth rate. The specific adjustment mechanism is as follows: first, the colony growth rate is measured, and when the growth rate does not exceed 2.5 mm... 2 When the growth rate is / hour, set the basic constraint window r = 0.3 × min(len(A), len(B)), when the growth rate exceeds 2.5 mm 2 When the time is / hour, expand the constraint window to r=0.5×min(len(A),len(B)), where len(A) and len(B) represent the time series lengths of colony A and colony B, respectively, i.e. the number of time points contained in each. The constraint window r of the dynamic time warping algorithm adopts Sakoe-Chiba band constraint, with the constraint condition max(|ij|)≤r. By limiting the search range of colony time series alignment, unreasonable time point matching is prevented. At the same time, the constraint window can be optimized by combining the colony growth kinetic model.

[0011] Preferably, in step three, the endpoint DTW[len(A),len(B)] is traced back to the starting point DTW[1,1], where len(A) and len(B) represent the time series lengths of colony A and colony B, respectively; combined with spatial position constraints, the offset of the colony center position at adjacent time points does not exceed 5mm.

[0012] Preferably, in step four, the similarity calculation formula S(A,B)=exp(-DTW_distance(A,B) / σ) is used to calculate the similarity value between colonies, where S(A,B) represents the similarity value between colony A and colony B, with a value range of 0 to 1, DTW_distance(A,B) represents the minimum cumulative distance value between colony A and colony B calculated by the dynamic time warping algorithm, and σ is a scale parameter used to adjust the sensitivity of similarity. A species-specific similarity threshold database is established for colony classification judgment, wherein the similarity judgment threshold for Escherichia coli is set to 0.65-0.75, the similarity judgment threshold for Staphylococcus aureus is set to 0.70-0.80, the similarity judgment threshold for Streptococcus is set to 0.75-0.85, and the similarity judgment threshold for Klebsiella pneumoniae is set to 0.68-0.78.

[0013] Preferably, in step five, the cross-time point target association algorithm is an improved algorithm based on the Hungarian algorithm, specifically including: using the Hungarian algorithm to find the optimal matching scheme based on the similarity matrix; when the similarity difference is detected to be greater than 1.5 times the similarity standard deviation, selecting the highest similarity match; when a one-to-many matching conflict is detected, calculating the similarity difference of all candidate matches, selecting the highest similarity match with a difference greater than 1.5 times the similarity standard deviation, and recalculating the secondary matching for the remaining clusters; The standard deviation of similarity is calculated using the formula σ_sim=√(∑(n=1 to N)(S_n-S_mean)). 2 The similarity is calculated as S_n / N, where S_n represents the nth similarity value, S_mean represents the average of all similarity values, and N represents the total number of similarity values.

[0014] Preferably, before obtaining the feature sequence of the colony in step one, a colony image grading processing step is included: setting three resolution levels for the colony image for multi-scale processing. The first resolution is set to twice the original resolution to improve the recognition accuracy of small colonies with a diameter of less than 2 mm. The second resolution is set to the original resolution to process medium-sized colonies with a diameter of 2 to 5 mm. The third resolution is set to 0.5 times the original resolution to efficiently process large colonies with a diameter of more than 5 mm. Finally, an adaptive weight allocation based on colony density and size distribution is used for weighted fusion to improve the overall recognition accuracy.

[0015] Preferably, in the colony image grading process, the weight allocation strategy is dynamically adjusted according to the image quality assessment, including three evaluation indicators: image sharpness, contrast, and noise level. When the image quality score is below 0.6, the weight of the high resolution level is increased to above 0.5. The image quality score is above 0.8 as excellent, 0.6 to 0.8 as good, 0.4 to 0.6 as medium, and below 0.4 as poor. Sharpness is calculated using the variance of the Laplacian operator, contrast is calculated using the standard deviation of gray levels, and noise level is calculated using the signal-to-noise ratio.

[0016] This invention is applicable to the temporal classification and tracking of colonies of at least one pathogenic microorganism, including *Escherichia coli*, *Klebsiella pneumoniae*, *Staphylococcus aureus*, and *Streptococcus*. The time interval is set to 8–12 hours, and the image sequence contains colony images from 3–8 time points. The culture conditions are 37℃±1℃ and relative humidity 60%–80%. The cross-time point target association algorithm combines morphological analysis and topological change detection mechanisms, adapting to biological processes such as colony division, fusion, and boundary blurring by setting a colony area change threshold of 25% and a shape change angle threshold of 20°. Using the species-specific threshold database established in step four, representative species are selected from the colony samples for subsequent DNA sequencing analysis, maintaining a species identification accuracy of over 95%. The matching accuracy of the time point classification correspondence established by the cross-time point target association algorithm reaches 83%–91%, validated under standard culture conditions using a test set containing more than 100 colony samples.

[0017] This invention also includes the following innovative technical features: Constraint Mechanism of Colony Growth Dynamics Model: Based on the biological laws of colony growth, a growth dynamics model is established to constrain and optimize the dynamic time warping algorithm. By analyzing the exponential characteristics of the colony area growth curve, a growth rate threshold of 2.5 mm is set. 2 / hour serves as the critical point for adjusting the constraint window. When the growth rate exceeds this threshold, the constraint window is automatically expanded to 0.5 times the sequence length, effectively adapting to the differences in growth characteristics of different bacterial species and solving the technical problem of not being able to handle differences in colony growth rate in existing technologies.

[0018] Multimodal feature fusion optimization strategy: Addressing the multi-scale characteristics of colony images, an innovative dynamic weight allocation mechanism based on image quality assessment is proposed. Through a comprehensive score across three dimensions—clarity, contrast, and noise level—the weight of high-resolution levels is automatically increased to above 0.5 when the image quality is below 0.6. This achieves optimal fusion of morphological, textural, and color features, significantly improving the recognition accuracy of both small and dense colonies and resolving the technical problem of a 15-30% counting bias rate inherent in traditional methods.

[0019] The present invention has the following advantages over the prior art: 1. It solves the problem of inconsistent classification caused by changes in colony morphology over time. It achieves flexible temporal alignment through dynamic time warping algorithm, handles differences in colony growth rate, and achieves a label matching accuracy of 83% to 91% across time points.

[0020] 2. It achieves accurate time-series tracking and can handle complex situations such as differences in growth rate, alignment of morphological changes, and unequal time sequences. It ensures that the optimal alignment path is found through the cumulative distance matrix and path backtracking mechanism, and effectively handles morphological changes such as colony area changes of up to 25% and shape changes of up to 20°.

[0021] 3. Improved the stability of identifying colonies of different sizes and densities. Through a colony grading process, different resolution levels were used to process small colonies with a diameter of less than 2 mm, medium colonies with a diameter of 2-5 mm, and large colonies with a diameter of more than 5 mm. This effectively solved the problem of a counting deviation rate of 15-30% in traditional methods, reducing the deviation rate to below 8%.

[0022] 4. Significantly improved detection efficiency: By setting a species-specific similarity threshold, representative species can be selected from colony samples for subsequent DNA sequencing analysis, while maintaining a species identification accuracy rate of over 95%.

[0023] 5. An innovative constraint mechanism for the colony growth kinetics model was proposed, by setting a growth rate threshold of 2.5 mm. 2 / hour serves as the critical point for adjusting the constraint window, effectively adapting to the differences in growth characteristics among different bacterial species and solving the technical problem of being unable to handle differences in colony growth rates in existing technologies.

[0024] 6. A multimodal feature fusion optimization strategy was established. Through a dynamic weight allocation mechanism based on image quality assessment, the optimal fusion of morphological features, texture features and color features was achieved, which significantly improved the recognition accuracy of small and dense colonies. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the colony image temporal classification and tracking method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the dynamic time warping algorithm processing flow according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the colony grading accuracy enhancement technology strategy of this invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0027] Example 1 This embodiment verifies the application effect of the dynamic time warping algorithm in colony time sequence alignment, how to handle the problem of colony time sequence alignment with different growth rates, and demonstrates the effect of the cross-time point target association algorithm in ensuring the consistency of time sequence classification, as well as how to establish the classification correspondence between different time points.

[0028] like Figure 1 and Figure 2 As shown, Escherichia coli colonies were selected as the test object, and time-series images were acquired under standard culture conditions (37℃±1℃, blood agar plate medium, relative humidity 65%).

[0029] Step 1: Obtain the feature sequence of the colony image and construct the colony feature distance matrix. Specifically, this includes: obtaining the feature sequence of the colony from the colony image, where the feature sequence of the colony refers to the sequence formed by arranging the feature vectors of the same colony at different time points in chronological order. Set up two colonies A and B. Colony A collects feature vectors [a1, a2, a3, a4, a5] at time points t=1, 2, 3, 4, and colony B collects feature vectors [b1, b2, b3, b4] at time points t=1, 2, 3, 4, with a time interval of 8 hours.

[0030] The colony feature vector includes colony area features, perimeter features, roundness features, RGB color features, and gray-level co-occurrence matrix texture features. The feature vector A[1] of colony A contains 12 dimensions, namely A[1,1]=3.500 (area, mm). 2 ), A[1,2]=6.800 (circumference, mm), A[1,3]=0.740 (circularity), A[1,4]=120, A[1,5]=85, A[1,6]=95 (RGB color features), A[1,7]=0.650 (contrast), A[1,8]=0.820 (correlation), A[1,9]=0.450 (energy), A[1,10]=0.780 (homogeneity), A[1,11]=0.230, A[1,12]=0.670 (additional texture parameters), fully represented as A[1,k] where k=1 to 12, total dimension M=12. Similarly, in the feature vector B[1,k] of colony B at the first time point, k=1 to 12 correspond to the same feature dimensions, B[1]=[3.3, 6.6, 0.76, 118, 87, 93, 0.63, 0.84, 0.43, 0.76, 0.21,0.65].

[0031] The formula for calculating circularity is 4π × area / perimeter. 2This method is applicable to colonies with a shape regularity evaluation range of 0.3–1.0. Outliers (<0.3 or >1.0) are recalculated using an ellipse fitting method. The gray-level co-occurrence matrix texture features include four parameters: contrast, correlation, energy, and homogeneity. Contrast reflects the sharpness of the image, correlation reflects the similarity of gray-level co-occurrence matrix elements in the row or column direction, energy reflects the uniformity of gray-level distribution in the image, and homogeneity reflects the local variations in image texture. An 8×8 pixel window is used, and calculations are performed based on 256 gray levels in four directions: 0°, 45°, 90°, and 135°.

[0032] Colony growth rate measurement method: Calculated by dividing the area change at two consecutive time points by the time interval. Area measurement uses a pixel calibration method with a calibration coefficient of 0.05 mm / pixel. The growth rate of colony A is (5.200 - 3.500) / 8 = 0.213 mm. 2 / hour, the growth rate of colony B is (4.800-3.300) / 8 = 0.188 mm 2 / hour, all less than 2.5mm 2 / hour threshold.

[0033] Construct a colony feature distance matrix and use the Euclidean distance calculation formula D[i,j]=√(∑(k=1 to M)(A[i,k]-B[j,k]) 2 ), calculate the characteristic distance between any two colonies at different time points, where D[i,j] represents the characteristic distance between colony A at time point i and colony B at time point j. Specifically, taking colony A at time point 1 and colony B at time point 1 as an example, D[1,1]=√[(3.5-3.3) 2 +(6.8-6.6) 2 +(0.74-0.76) 2 +(120-118) 2 +(85-87) 2 +(95-93) 2 +(0.65-0.63) 2 +(0.82-0.84) 2 +(0.45-0.43) 2 +(0.78-0.76) 2 +(0.23-0.21) 2 +(0.67-0.65) 2 =√[0.04+0.04+0.0004+4+4+4+0.0004+0.0004+0.0004+0.0004+0.0004]=√12.085=3.476.

[0034] To facilitate subsequent DTW algorithm processing, the distance values ​​are normalized using the maximum-min normalization method: D_norm[i,j]=(D[i,j]-D_min) / (D_max-D_min), where D_min and D_max are the minimum and maximum values ​​in the distance matrix, respectively. Calculations show D_min=2.845 and D_max=4.123, therefore, the normalized value of D[1,1] is (3.476-2.845) / (4.123-2.845)=0.494.

[0035] Step 2: Calculate the cumulative distance matrix based on the dynamic time warping algorithm. Initialize DTW[1,1]=D[1,1]=0.494, and then calculate the cumulative distance value point by point according to the recursive formula DTW[i,j]=D[i,j]+min(DTW[i-1,j],DTW[i,j-1],DTW[i-1,j-1]). DTW[i,j] represents the cumulative distance between colony A from time point 1 to time point i and colony B from time point 1 to time point j. Since the growth rates of both colonies are less than 2.5mm... 2 / hour, set the constraint window r=0.3×min(5,4)=1.2, the constraint window r adopts Sakoe-Chiba constraint, and the constraint condition is max(|ij|)≤1.2.

[0036] Step 3: Using a path backtracking algorithm, find the time-point matching path that minimizes the cumulative distance between colonies in the cumulative distance matrix, thus obtaining the optimal time alignment relationship between the feature sequences of different colonies. Backtracking from the endpoint DTW[5,4]=2.630 to the starting point DTW[1,1]=0.494, the optimal matching path is [(1,1), (2,2), (3,3), (4,4), (5,4)]. During the path backtracking process, spatial position constraints are verified for each matched time point pair. For example, the center coordinates of colony A at time points 1 and 2 are (25.2,18.6) and (25.8,19.1) respectively, with a position offset of √[(25.8-25.2)]. 2 +(19.1-18.6) 2 The value is 0.781mm < 5mm, which satisfies the constraint. This constraint mechanism eliminates unreasonable cross-distance matching and improves the accuracy of timing alignment.

[0037] Step 4: Construct a similarity matrix based on the minimum cumulative distance between colonies and calculate the similarity values ​​between colonies. The similarity calculation formula is S(A,B)=exp(-DTW_distance(A,B) / σ), where S(A,B) represents the similarity value between colony A and colony B, ranging from 0 to 1; DTW_distance(A,B) represents the minimum cumulative distance between colony A and colony B calculated using the dynamic time warping algorithm; and σ is a scale parameter used to adjust the sensitivity of the similarity. When σ is set to 0.5, and DTW_distance(A,B)=2.630, the calculated S(A,B)=exp(-2.630 / 0.5)=exp(-5.260)=0.005.

[0038] Step 5: Employ a cross-time point target association algorithm to establish classification correspondences between time points and generate the final colony classification and tracking results. Based on the established classification correspondences between time points, propagate the colony classification label from the first time point to subsequent time points to ensure consistency of colony classification labels across different time points.

[0039] The cross-time point target association algorithm is an improved version of the Hungarian algorithm. Specifically, it includes: using the Hungarian algorithm to find the optimal matching scheme based on the similarity matrix; when the similarity difference is detected to be greater than 1.5 times the similarity standard deviation, selecting the highest similarity match; when a one-to-many matching conflict is detected, calculating the similarity difference of all candidate matches, selecting the highest similarity match with a difference greater than 1.5 times the similarity standard deviation, and recalculating the secondary matching for the remaining clusters. To verify the algorithm's effectiveness on large-scale data, a culture dish containing 100 colonies was selected, and images were acquired and analyzed at three time points: t1, t2, and t3. At time point t1, the 100 colonies were divided into 3 categories {40 colonies in category A, 35 colonies in category B, and 25 colonies in category C}; at time point t2, they were divided into 4 categories {30 colonies in category X, 28 colonies in category Y, 25 colonies in category Z, and 17 colonies in category W}; and at time point t3, they were divided into 3 categories {42 colonies in category P, 33 colonies in category Q, and 25 colonies in category R}.

[0040] A similarity matrix is ​​constructed based on the minimum cumulative distance between colonies, and the similarity values ​​between colonies are calculated. The similarity calculation formula S(A,B)=exp(-DTW_distance(A,B) / σ) is used to calculate the similarity values ​​between colonies, where σ is set to 0.5. Taking class A in t1 and class X in t2 as examples, the DTW distances between the 40 colonies in class A and the 30 colonies in class X, calculated using the Dynamic Time Warping (DTW) algorithm, are as follows: 0.145, 0.167, 0.134, 0.189, 0.156, 0.178, 0.123, 0.198, 0.145, 0.167, 0.134, 0.189, 0.156, 0.178, 0.123, 0.198, 0.145, 0.167, 0.134, 0.189, 0.156, 0.178, 0.123, 0.198, 0.145, 0.167, 0.134, 0.189, 0.156. The average DTW distance was 0.156, and the similarity S(A,X) = exp(-0.156 / 0.5) = exp(-0.312) = 0.731. Since the test colonies were *E. coli*, and referring to the similarity threshold of 0.65–0.75 for *E. coli* in the species-specific threshold database, this similarity value of 0.731 falls within the threshold range, indicating that the two colony groups have high similarity and a corresponding relationship can be established. Using this threshold mechanism, the classification accuracy reached 84.2% in the test of 100 *E. coli* colonies.

[0041] A cross-time point target association algorithm is used to establish the classification correspondence between time points. The similarity standard deviation is calculated using the formula σ_sim=√(∑(n=1 to N)(S_n-S_mean)). 2 The similarity is calculated as S_n / N, where S_n represents the nth similarity value, S_mean represents the average of all similarity values, and N represents the total number of similarity values. The standard deviation of the similarity values ​​between all categories [0.731, 0.623, 0.445, 0.678, 0.589, 0.712, 0.534, 0.667, 0.598] is calculated as S_mean = (0.731 + 0.623 + 0.445 + 0.678 + 0.589 + 0.712 + 0.534 + 0.667 + 0.598) / 9 = 0.620, and σ_sim = √(∑(n=1 to 9)(S_n - 0.620)). 2 / 9)=0.089.

[0042] After constructing the cost matrix C[i,j]=1-S(i,j), the Hungarian algorithm is used to solve for the minimum cost matching. When the similarity difference between class B and classes Y and Z is detected as |0.678-0.623|=0.055, which is less than 1.5×0.089=0.134, according to the improved algorithm based on the Hungarian algorithm, class Y, which has higher similarity, is selected for matching. Unmatched class Z colonies re-enter the secondary matching calculation and are associated with the remaining t1 classes to ensure that all colonies can find a suitable temporal correspondence.

[0043] After processing with a cross-time point target association algorithm, the classification labels at time point t2 were reassigned to {30 in class A', 53 in class B', and 17 in class C'}, maintaining consistency with the classification at time point t1. The same method was used to process time point t3, ultimately achieving complete consistency of classification labels across the three time points.

[0044] This embodiment verifies that the dynamic time warping algorithm can effectively handle differences in colony growth rates and achieve accurate alignment of sequences with unequal time durations, providing a reliable similarity basis for subsequent tag propagation. In a test of 100 colony pairs, the temporal alignment accuracy reached 89.7% ± 3.2%.

[0045] This embodiment verifies that the cross-time point target association algorithm can effectively solve the problem of inconsistent classification labels across time points, establish a reliable temporal correspondence, and provide an accurate label basis for colony tracking. In the test of 100 colonies, the temporal label consistency reached 87.2% ± 2.8%.

[0046] Example 2 Based on Example 1, this example further verifies and demonstrates the comprehensive analytical effect of the colony grading process when processing colonies of different sizes and densities.

[0047] like Figure 3 As shown, the colony grading processing strategy includes an image quality assessment module, a resolution level selection module, and a weight allocation module. The image quality assessment module comprehensively scores the colonies based on three indicators: sharpness, contrast, and noise level. The resolution level selection module automatically selects the corresponding processing resolution based on the colony size. The weight allocation module dynamically adjusts the fusion weights of each level based on the image quality score to achieve the optimal comprehensive analysis effect.

[0048] A mixed colony culture dish containing significantly different colonies was selected, including 18 small colonies (0.8–1.5 mm in diameter), 15 medium colonies (2.5–4.2 mm in diameter), and 10 large colonies (5.5–8.0 mm in diameter), totaling 43 colonies. To verify the effectiveness of the dense distribution treatment, a dense distribution area containing 12 small colonies with a distance of less than 1 mm between colonies was set in the center of the culture dish.

[0049] Three resolution levels are set for processing: the first level is set to twice the original resolution (2048×2048 pixels) for processing small colonies, the second level is set to the original resolution (1024×1024 pixels) for processing medium colonies, and the third level is set to 0.5 times the original resolution (512×512 pixels) for processing large colonies.

[0050] Image quality assessment index calculation method: Sharpness is calculated using the variance of the Laplacian operator. , This is the Laplacian operator; contrast is calculated using the grayscale standard deviation, Contrast = √(∑(I_i - I_mean)). 2 / N); Noise level is calculated using signal-to-noise ratio (SNR) = 20 × logN; 10 (Signal_power / Noise_power). In this embodiment, the sharpness is 0.82, the contrast is 0.76, and the noise level is 0.15. The overall quality score is (0.82+0.76+(1-0.15)) / 3=0.81.

[0051] At the first resolution, the edge detection accuracy for small colonies reached 93.8%, successfully identifying 17 out of 18 small colonies, with one failing to separate due to severe overlap. In densely distributed areas, 11 out of 12 small colonies were successfully separated, resulting in an oversegmentation rate of 8.3%. At the second resolution, the segmentation accuracy for medium-sized colonies reached 95.2%, successfully identifying 14 out of 15 medium-sized colonies. At the third resolution, the processing efficiency for large colonies improved by 40%, with all 10 large colonies accurately identified, avoiding computational redundancy associated with high-resolution processing.

[0052] An adaptive weighting method based on colony density and size distribution is used for weighted fusion. The weighting formula is as follows: W_small = 0.4 × (1 + 0.25×(1-quality_score)); W_medium = 0.4 × (1 - 0.125×(1-quality_score)); W_large = 0.2 × (1 - 0.25×(1-quality_score)); Since the image quality score of 0.81 is higher than 0.6, standard weighting is used: small colony weight W_small = 0.4 × (1 + 0.25 × (1 - 0.81)) = 0.419, medium colony weight W_medium = 0.4 × (1 - 0.125 × (1 - 0.81)) = 0.391, and large colony weight W_large = 0.2 × (1 - 0.25 × (1 - 0.81)) = 0.190. The total weight is 1.000, which meets the normalization requirement. Figure 3 As shown, the weight allocation strategy is dynamically adjusted based on image quality assessment, including three evaluation indicators: image sharpness, contrast, and noise level. The weight allocation module dynamically adjusts the fusion weights of each level based on the image quality score to achieve the optimal comprehensive analysis effect. When the image quality score is below 0.6, the weight is adjusted to 0.5 for small colonies, 0.3 for medium colonies, and 0.2 for large colonies to enhance the ability to identify small colonies.

[0053] The final comprehensive analysis showed that 41 out of 43 colonies were successfully identified and accurately segmented, achieving an accuracy rate of 95.3%, which is 9.3 percentage points higher than the 86.0% accuracy rate of single-resolution processing methods. The processing time was approximately 68% of that of single-resolution methods, achieving a dual optimization of accuracy and efficiency.

[0054] This embodiment verifies that the colony grading process can effectively solve the problem of accurate segmentation of colonies of different sizes, and achieves optimal comprehensive analysis results through an image quality-driven resolution weight optimization mechanism. In eight test samples with different colony size distributions, the average identification accuracy was 94.1% ± 2.8%.

[0055] Example 3 Based on the aforementioned embodiments, this embodiment verifies the critical value of colony growth rate (2.5 mm). 2 The effect of the adaptive window adjustment mechanism for growth rate near ( / hour).

[0056] Select growth rates of 2.3 mm. 2 / hour, 2.5mm 2 / hour, 2.7mm 2 Three groups of E. coli colonies per hour were tested, each group containing 30 colonies, with a time series length of 6 time points.

[0057] For a growth rate of 2.3 mm 2For colony groups with a growth rate of 2.5 mm / hour, using a basic constraint window r = 0.3 × min(6,6) = 1.8, the DTW alignment accuracy was 89.7% ± 3.2%. 2 The colony size was at a critical value of 1.8 m / hour. Using a basic constraint window r = 0.3 × min(6,6) = 1.8, the DTW alignment accuracy was 90.1% ± 2.9%. For a growth rate of 2.7 mm... 2 With colony groups per hour, using an extended constraint window r=0.5×min(6,6)=3.0, the DTW alignment accuracy was 93.4%±2.5%.

[0058] Experimental results show that the growth rate adaptive window adjustment mechanism can dynamically adjust the size of the constraint window according to the colony growth characteristics, effectively improving the temporal alignment accuracy of fast-growing colonies.

[0059] Example 4 Based on the aforementioned embodiments, this embodiment verifies the rationality of setting the threshold boundary values ​​for similarity between different bacterial species.

[0060] Thirty colonies each of *Escherichia coli*, *Staphylococcus aureus*, and *Streptococcus* were selected and tested at similarity threshold boundaries. The classification accuracy rates for *E. coli* at thresholds of 0.64, 0.65, and 0.76 were 79.8%, 84.2%, and 83.1%, respectively; for *Staphylococcus aureus* at thresholds of 0.69, 0.70, and 0.81, the accuracy rates were 81.5%, 86.7%, and 85.9%; and for *Streptococcus* at thresholds of 0.74, 0.75, and 0.86, the accuracy rates were 83.2%, 88.4%, and 87.6%.

[0061] The experimental results verified the rationality of the species-specific similarity threshold setting, and each species could obtain the best classification effect within the recommended threshold range.

[0062] Example 5 Based on the aforementioned embodiments, this embodiment verifies the rationality of setting the colony area change threshold of 25% and the shape change angle threshold of 20°, as well as the effectiveness of the applicable bacterial species range and culture conditions.

[0063] Twenty-five colonies each of Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, and Streptococcus were selected and time-series images were acquired at 8-12 hour intervals under incubation conditions of 37℃±1℃ and relative humidity of 60%-80%. The image sequence contained 3-8 time points.

[0064] To investigate biological processes such as colony division, fusion, and boundary blurring, a colony area change threshold of 25% was set for testing. Among 100 tested colonies, 12 showed area changes exceeding 25%, of which 8 were due to normal growth and 4 were due to colony division. Morphological analysis, with a shape change angle threshold of 20°, successfully identified all division and fusion events with an accuracy rate of 95.8%.

[0065] Example 6 Based on the aforementioned embodiments, this embodiment verifies the technical effect of the cross-time point target association algorithm matching accuracy of 83%-91%.

[0066] Validation was performed using a test set containing more than 100 colony samples under standard culture conditions. A cross-timepoint target association algorithm was used to establish classification correspondences between time points. In 10 different test samples, the matching accuracies were 84.2%, 87.5%, 89.3%, 85.7%, 90.1%, 83.8%, 88.6%, 86.4%, 89.7%, and 87.2%, respectively, with an average matching accuracy of 87.25% and a standard deviation of 2.1%, validating the reliability of the 83%-91% accuracy range.

[0067] Comparative Example 1 The purpose of this comparative example is to verify the difference in performance between traditional location association methods when dynamic time warping is lacking, and to demonstrate the necessity and superiority of dynamic time warping.

[0068] The same test conditions and colony samples as in Example 1 were used, but a traditional simple association method based on location information was used instead of the dynamic time warping algorithm. This method assumes that the positional change of colonies at adjacent time points does not exceed a certain threshold (set to 5 mm), and association is performed by calculating the Euclidean distance between the colony centers.

[0069] For the temporal correlation of colony A and colony B, the traditional method calculates the center coordinates of the colonies at each time point. The center coordinates of colony A at the 5 time points are (25.2,18.6), (25.8,19.1), (26.5,19.8), (27.2,20.5), and (28.1,21.3), respectively. The center coordinates of colony B at the 4 time points are (45.1,32.4), (45.6,33.2), (46.8,34.5), and (47.9,35.8), respectively.

[0070] The association results obtained by following the principle of minimum distance are: A[1]→B[1], A[2]→B[2], A[3]→B[3], A[4]→B[4], and A[5] has no corresponding association. However, since the growth rate of colony B is relatively slow, A[2] should actually correspond to B[1] and A[3] should correspond to B[2]. The traditional method has resulted in a temporal misalignment.

[0071] Validated on a test set containing 100 colonies, the traditional location association method achieved a cross-time point matching accuracy of only 65.8% ± 6.1%, while the method using the dynamic time warping algorithm reached 89.7% ± 3.2%, a relative improvement of 36.3%. This is especially true for cases where the growth rate difference exceeds 1.5 mm. 2 For colony pairs per hour, the matching accuracy of traditional methods drops to 51.2%, while the dynamic time warping algorithm still maintains an accuracy of 78.9%.

[0072] The comparison shows that the lack of a dynamic time warping algorithm leads to a significant decrease in the accuracy of colony association, especially when dealing with colonies with large differences in growth rate. This demonstrates the necessity and superiority of the dynamic time warping algorithm in colony temporal alignment.

[0073] Comparative Example 2 The purpose of this comparative study is to verify the difference in performance between independent classification methods when cross-time point target association algorithms are lacking, and to demonstrate the important role of cross-time point target association algorithms in ensuring temporal consistency.

[0074] The same test conditions as in Example 1 were used, but colony classification was performed independently at different time points, and cross-time point target association algorithms were not used to establish the correspondence between time points. K-means clustering algorithm was used to classify independently at each time point.

[0075] At time point t1, the 100 colonies were divided into 3 categories: Category A (40 colonies, numbered 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, etc.), Category B (35 colonies, numbered 2, 4, 6, 8, 10, 12, 14, 16, etc.), and Category C (25 colonies, numbered 18, 20, 22, 24, 25, 26, 27, 28, etc.). At time point t2, the same 100 colonies were independently classified into 4 categories: Category X (30 colonies, numbered 1, 3, 5, 7, 9, 11, 13, 15, 17, etc.), Category Y (28 colonies, numbered 2, 4, 6, 8, 10, 12, 14, 16, etc.), Category Z (25 colonies, numbered 19, 21, 23, 18, 20, 22, 24, etc.), and Category W (17 colonies, numbered 25, 26, 27, 28, 29, 30, etc.).

[0076] By manually labeling to determine the true correspondence of colonies, a serious labeling inconsistency problem was found in the independent classification method. For example, colonies 19, 21, and 23 were assigned to class A at time t1, but to class Z at time t2; colonies 18, 20, 22, and 24 belonged to class C at time t1, but were assigned to class Z at time t2.

[0077] Statistical analysis shows that the temporal label consistency of the independent classification method is only 56.7% ± 8.3%, meaning that only 57 out of 100 colonies maintained the same classification label at different time points. In contrast, the method using a cross-time point target association algorithm achieved a temporal label consistency of 87.2% ± 2.8%, a relative improvement of 53.9%.

[0078] In a large-scale test involving 150 colonies, the average temporal consistency of the independent classification method was 59.3%, with a standard deviation of 9.2%, indicating unstable results. In contrast, the cross-timepoint target association algorithm showed an average temporal consistency of 85.8% and a standard deviation of 3.1%, demonstrating better stability and reliability.

[0079] The comparison shows that the lack of a cross-time point target association algorithm leads to a significant decrease in temporal classification consistency and makes it impossible to effectively track the evolution of colonies at different time points. This demonstrates the important role and technical advantages of the cross-time point target association algorithm in ensuring temporal consistency.

[0080] Comparative Example 3 The purpose of this comparative example is to verify the difference in effectiveness of a single-resolution processing method when a colony grading step is missing, and to demonstrate the superiority of the colony grading step in accurate colony segmentation.

[0081] The same mixed colony culture dishes as in Example 2 were used, but only a single resolution (1024×1024 pixels) was used for processing, without employing the comprehensive analysis strategy of colony grading.

[0082] Under single-resolution processing, the recognition performance of small colonies (0.8-1.5 mm in diameter) was poor, with only 13 out of 18 small colonies identified, resulting in a recognition rate of 72.2%. This was mainly due to insufficient resolution leading to the loss of edge details. Medium-sized colonies (2.5-4.2 mm in diameter) performed better, with 14 out of 15 colonies identified, resulting in a recognition rate of 93.3%. Processing large colonies (5.5-8.0 mm in diameter) involved computational redundancy; although all 10 colonies were identified, the processing time was relatively long.

[0083] Overall statistical results show that the single-resolution method achieved a comprehensive identification accuracy of 86.0% (37 / 43) for 43 colonies, a decrease of 9.3 percentage points compared to the 95.3% accuracy of the colony grading step. The processing time was 168 seconds, an efficiency improvement of 32.1% compared to the 114 seconds of the colony grading step.

[0084] Especially when dealing with densely distributed small colonies, the segmentation accuracy of single-resolution methods is significantly insufficient. In regions containing dense small colonies, the traditional watershed algorithm achieves an oversegmentation rate of 21.3%, while the colony grading step controls the oversegmentation rate to within 8.3% through high-resolution processing.

[0085] In eight test samples with different colony size distributions, the average recognition accuracy of the single-resolution method was 83.7% ± 7.2%, while the colony grading step reached 94.1% ± 2.8%, which not only had higher accuracy but also better stability.

[0086] The comparison shows that the lack of a colony grading process significantly reduces the accuracy of colony identification, especially in handling small and dense colonies. It also results in lower computational efficiency, demonstrating the necessity and superiority of the colony grading process in accurate colony segmentation.

[0087] Based on the above embodiments and comparative examples, this invention, through an innovative combination of dynamic time warping algorithm and cross-time point target association algorithm, systematically solves the problem of consistency in colony temporal classification for the first time, achieving flexible alignment of unequal time-length sequences and accurate propagation of labels across time points. The colony grading process further improves the identification accuracy of colonies of different sizes. Compared with existing technologies, it has significant improvements in temporal consistency, segmentation accuracy, and processing efficiency, providing a more accurate and efficient technical solution for pathogen detection.

Claims

1. A colony image temporal classification and tracking method based on dynamic time warping and tag propagation, characterized in that, Includes the following steps: Step 1: Obtain the feature sequence of the colony image and construct the colony feature distance matrix. Specifically, this includes: first, obtaining the feature sequence of the colony from the colony image, where the feature sequence of the colony refers to the sequence formed by arranging the feature vectors of the same colony at different time points in chronological order; then, for the feature sequences of different colonies, using the Euclidean distance calculation formula, calculating the feature distance between any two colonies at different time points, and constructing the colony feature distance matrix. Step 2: Calculate the cumulative distance matrix based on the dynamic time warping algorithm, and calculate the cumulative distance value point by point using the recursive formula; Step 3: Find the time point matching path that minimizes the cumulative distance value between colonies in the cumulative distance matrix using the path backtracking algorithm, and obtain the optimal time alignment relationship between the feature sequences of different colonies; Step 4: Construct a similarity matrix based on the minimum cumulative distance between colonies and calculate the similarity values ​​between colonies; Step 5: Use a cross-time point target association algorithm to establish the classification correspondence between time points and generate the final colony classification and tracking results. Based on the established classification correspondence between time points, propagate the colony classification label of the first time point to subsequent time points to ensure the consistency of colony classification labels at different time points.

2. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 1, characterized in that, In step one, the Euclidean distance calculation formula is D[i,j]=√(∑(k=1 to M)(A[i,k]-B[j,k]) 2 ), where A[i,k] represents the k-th feature vector of colony A at time i, B[j,k] represents the k-th feature vector of colony B at time j, k is the feature dimension index, M is the total dimension of the feature vector, and D[i,j] represents the feature distance between colony A at time i and colony B at time j. The feature vector includes colony area features, perimeter features, circularity features, RGB color features, and grayscale co-occurrence matrix texture features, wherein the circularity feature is calculated as 4π × area / perimeter. 2 The gray-level co-occurrence matrix texture features include four parameters: contrast, correlation, energy, and homogeneity. Contrast reflects the sharpness of the image, correlation reflects the similarity of gray-level co-occurrence matrix elements in the row or column direction, energy reflects the uniformity of gray-level distribution in the image, and homogeneity reflects the local changes in image texture. An 8×8 pixel window is used, and calculations are performed based on 256 gray levels in four directions: 0°, 45°, 90°, and 135°.

3. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 2, characterized in that, In step two, the recursive formula is DTW[i,j]=D[i,j]+min(DTW[i-1,j],DTW[i,j-1],DTW[i-1,j-1]), where DTW[i,j] represents the cumulative distance between colony A from the first time point to the i-th time point and colony B from the first time point to the j-th time point, and the initialization condition is DTW[1,1]=D[1,1]; The constraint window size r of the dynamic time warping algorithm is adaptively adjusted according to the colony growth rate. The specific adjustment mechanism is as follows: first, the colony growth rate is measured, and when the growth rate does not exceed 2.5 mm... 2 When the growth rate is / hour, set the basic constraint window r = 0.3 × min(len(A), len(B)), when the growth rate exceeds 2.5 mm 2 When the time is / hour, expand the constraint window to r=0.5×min(len(A),len(B)), where len(A) and len(B) represent the time series lengths of colony A and colony B, respectively, i.e. the number of time points contained in each. The constraint window r of the dynamic time warping algorithm adopts Sakoe-Chiba band constraint, and the constraint condition is max(|ij|)≤r.

4. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 3, characterized in that, In step three, backtrack from the endpoint DTW[len(A),len(B)] to the starting point DTW[1,1], where len(A) and len(B) represent the time series lengths of colony A and colony B, respectively; combined with spatial position constraints, the offset of the colony center position at adjacent time points does not exceed 5mm.

5. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 3, characterized in that, In step four, the similarity calculation formula S(A,B)=exp(-DTW_distance(A,B) / σ) is used to calculate the similarity value between colonies, where S(A,B) represents the similarity value between colony A and colony B, ranging from 0 to 1, DTW_distance(A,B) represents the minimum cumulative distance between colony A and colony B calculated by the dynamic time warping algorithm, and σ is a scale parameter used to adjust the sensitivity of similarity. A species-specific similarity threshold database is established for colony classification judgment, where the similarity judgment threshold for Escherichia coli is set to 0.65-0.75, the similarity judgment threshold for Staphylococcus aureus is set to 0.70-0.80, the similarity judgment threshold for Streptococcus is set to 0.75-0.85, and the similarity judgment threshold for Klebsiella pneumoniae is set to 0.68-0.

78.

6. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 1, characterized in that, In step five, the cross-time point target association algorithm is an improved algorithm based on the Hungarian algorithm. Specifically, it includes: using the Hungarian algorithm to find the optimal matching scheme based on the similarity matrix; when the similarity difference is detected to be greater than 1.5 times the similarity standard deviation, selecting the highest similarity match; when a one-to-many matching conflict is detected, calculating the similarity difference of all candidate matches, selecting the highest similarity match with a difference greater than 1.5 times the similarity standard deviation, and recalculating the secondary matching for the remaining clusters. The standard deviation of similarity is calculated using the formula σ_sim=√(∑(n=1 to N)(S_n-S_mean)). 2 The similarity is calculated as S_n / N, where S_n represents the nth similarity value, S_mean represents the average of all similarity values, and N represents the total number of similarity values.

7. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 1, characterized in that, Before obtaining the feature sequence of the colony in step one, a colony image hierarchical processing step is also included: three resolution levels are set for the colony image for multi-scale processing. The first resolution is set to twice the original resolution to improve the recognition accuracy of small colonies with a diameter of less than 2 mm. The second resolution is set to the original resolution to process medium-sized colonies with a diameter of 2 to 5 mm. The third resolution is set to 0.5 times the original resolution to efficiently process large colonies with a diameter of more than 5 mm. Finally, an adaptive weight allocation based on colony density and size distribution is used for weighted fusion to improve the overall recognition accuracy.

8. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 7, characterized in that, In the colony image grading process, the weight allocation strategy is dynamically adjusted based on the image quality assessment, including three evaluation indicators: image sharpness, contrast, and noise level. When the image quality score is below 0.6, the weight of the high resolution level is increased to above 0.

5. An image quality score of 0.8 or above is excellent, 0.6 to 0.8 is good, 0.4 to 0.6 is moderate, and below 0.4 is poor. Sharpness is calculated using the variance of the Laplacian operator, contrast is calculated using the standard deviation of gray levels, and noise level is calculated using the signal-to-noise ratio.

9. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 1, characterized in that, The cross-time point target association algorithm combines morphological analysis and topological change detection mechanism, and adapts to biological processes such as colony division, fusion and boundary blurring by setting a colony area change threshold of 25% and a shape change angle threshold of 20°. The colony image time-series tracking method is applicable to the time-series classification and tracking of colonies of at least one pathogenic microorganism among Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, and Streptococcus. The time interval is set to 8 to 12 hours, the image sequence contains colony images at 3 to 8 time points, and the culture temperature conditions are 37℃±1℃ and the relative humidity is 60% to 80%.

10. The colony image temporal classification and tracking method based on dynamic time warping and tag propagation according to claim 1, characterized in that, The cross-time point target association algorithm achieves a matching accuracy of 83% to 91% in establishing the classification correspondence between time points.

Images