High-throughput bacterial colony detection sensor based on multi-mode optics and spectrum fusion and detection method thereof

Abstract

The invention relates to the technical field of detection, in particular to a high-throughput bacterial colony detection sensor based on multi-mode optics and spectrum fusion and a detection method thereof.The method comprises the steps that data of a bright field image, a multi-channel fluorescence image and a near infrared spectrum of a bacterial colony to be detected are collected, and metadata in the collection process is recorded synchronously; preprocessing the data, and extracting multi-dimensional features of bacterial colonies from the preprocessed data; performing fusion processing on the multi-dimensional features in combination with space-time sequence information; identifying and classifying bacterial colonies according to the fused multi-dimensional features; and generating a detection report containing traceability information according to a result after identification and classification. Through multi-modal synchronous data acquisition, targeted data preprocessing, multi-dimensional feature fusion and standardized recognition and classification, high-throughput accurate detection of bacterial colonies is realized, manual intervention is reduced, and reliability and traceability are improved.

Description

Technical Field

[0001] This invention relates to the field of detection technology, and more specifically, to a high-throughput colony detection sensor and detection method based on multimodal optics and spectral fusion. Background Technology

[0002] As microbial testing is increasingly used in food and drug safety monitoring, environmental pollutant screening, and hospital infection control, the industry's demand for testing throughput, accuracy, and automation continues to rise. Multimodal information fusion technology has become a research hotspot because it can integrate multidimensional features to improve testing reliability. At the same time, advancements in embedded processing technology provide support for real-time analysis, driving the upgrade of colony detection from traditional offline analysis to online real-time monitoring.

[0003] Traditional colony detection methods often employ a single detection modality, such as relying solely on microscopic imaging to observe morphology or analyzing metabolic characteristics through a single spectral method, making it difficult to comprehensively capture colony information. Detection throughput is low, processing only a small number of samples per run, and image acquisition, data recording, and feature extraction require manual intervention, making the preprocessing cumbersome and prone to human error. Furthermore, identification and classification lack standardized matching mechanisms, confidence levels cannot be quantified, alarm thresholds are subjectively set, and traceability information in test reports is incomplete, making them unsuitable for large-scale, precise detection scenarios.

[0004] Traditional methods suffer from problems such as incomplete information on a single modality, low detection throughput, high susceptibility to errors due to excessive human intervention, lack of standardized identification and classification mechanisms, difficulty in quantifying confidence levels, subjective alarm responses, and incomplete traceability information.

[0005] Therefore, it is necessary to design a high-throughput colony detection sensor and its detection method based on multimodal optics and spectral fusion to solve the above problems. Summary of the Invention

[0006] In view of this, the present invention proposes a high-throughput colony detection sensor and its detection method based on multimodal optics and spectral fusion, aiming to solve the problems of incomplete information, low throughput, excessive manual intervention, lack of classification standards, difficulty in quantifying confidence and incomplete traceability information in traditional single-modal methods.

[0007] In one aspect, this invention proposes a high-throughput colony detection method based on multimodal optics and spectral fusion, comprising: Acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, and simultaneously record metadata of the acquisition process; The data is preprocessed, and multidimensional features of the colonies are extracted from the preprocessed data; Specifically, grayscale processing is performed on the preprocessed brightfield image data to obtain a grayscale image, and morphological features are extracted based on the grayscale image; The mean fluorescence intensity of each channel is calculated based on the preprocessed multi-channel fluorescence image; fluorescence features are extracted based on the mean fluorescence intensity. Peak identification is performed on the preprocessed near-infrared spectral data to mark the position of all peaks in the spectral curve; metabolic features are extracted based on the position of each marked peak. The multidimensional features are fused by combining spatiotemporal sequence information; Colonies are identified and classified based on the fused multidimensional features; The identification and classification includes: matching the fused multidimensional features with a preset standard multidimensional feature library and calculating the matching degree; and determining the output category, probability of harmful subspecies, and confidence interval of the colony to be detected based on the matching degree. Based on the confidence interval, the probability of harmful subspecies, and the preset confidence threshold and harmful subspecies probability threshold, an alarm flag is marked; A detection report containing traceability information is generated based on the identification and classification results.

[0008] Furthermore, the metadata includes the temperature and humidity at the time of acquisition, sample batch, culture medium type, image exposure parameters, and spectral integration time; The metadata of the synchronous recording and acquisition process includes: During the data acquisition process, temperature and humidity data are collected and recorded according to the preset recording intervals; Record the corresponding image exposure parameters when the bright-field image acquisition is completed; Record the corresponding image exposure parameters when the multi-channel fluorescence image acquisition is completed; Record the corresponding spectral integration time when the near-infrared spectral data acquisition is completed; After data collection is completed, the sample batch, culture medium type, and the collected data are associated and stored respectively.

[0009] Furthermore, when acquiring bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, a trigger signal is used to control the synchronous start of the three types of data acquisition actions, and the sending time of the trigger signal is determined according to the growth stage of the colony to be detected. Determining the transmission time of the trigger signal includes: The data collection cycle is preset according to the growth stage of the colony culture to be detected; The time interval between two consecutive data collections of the three types of data was compared with the preset data collection period. When the time interval between two consecutive data acquisitions is equal to the preset data acquisition period, the same trigger signal is sent to control the simultaneous acquisition of the three types of data. If the time interval between any two consecutive data acquisitions is not equal to the preset data acquisition period, a segmented trigger signal is sent. The preset data acquisition period is a variable of acquisition time interval corresponding to different growth stages determined by the growth rate test of the colonies to be detected.

[0010] Furthermore, the sending of the segmentation trigger signal includes: First, send the first control signal to control the simultaneous acquisition of bright-field images and multi-channel fluorescence images; After the bright-field image and multi-channel fluorescence image are acquired, a second control signal is sent to control the acquisition of near-infrared spectral data; The time interval between the first control signal and the second control signal does not exceed a preset time difference; The preset time difference is the maximum time interval variable determined through data correlation experiments to ensure the correlation of the three types of data.

[0011] Furthermore, the data is preprocessed, including: correcting and denoising the bright-field image and multi-channel fluorescence image data; First, bright-field images and multi-channel fluorescence images are distinguished, and then correction processing is performed on each type of image. After correction, denoising processing is performed on the two types of images. When performing the correction process, the correction parameters corresponding to the recorded culture medium type are retrieved. For bright field images, distortion correction is performed in particular, and for multi-channel fluorescence images, light intensity uniformity correction is performed in particular. When performing denoising, noise areas outside the bacterial colony areas in the image are first identified by grayscale comparison, and then denoising operations are performed on the noise areas.

[0012] Furthermore, the preprocessing of the data also includes: denoising and baseline subtraction of the spectral data; First, the denoising parameters corresponding to the recorded spectral integration time are retrieved, and denoising is performed using a filtering method. After denoising is completed, baseline subtraction is then performed.

[0013] Furthermore, the extraction of multidimensional features from the preprocessed data also includes: During the extraction process, the multidimensional features are associated with and stored with their corresponding metadata. The morphological features include colony boundaries, texture, and roughness; The fluorescence characteristics include multi-band intensity ratios; The metabolic characteristics include peak position, peak intensity, and half-maximum width.

[0014] Furthermore, the preset standard multidimensional feature library stores standard features and subspecies features of different types of bacterial colonies; Based on the matching degree, the output category, probability of harmful subspecies, and confidence interval of the colony are determined, including: Obtain the matching degree for different categories; When the matching degree exceeds the preset matching degree, the category with the highest matching degree is directly used as the output category, and the subspecies probability and confidence interval are calculated based on the deviation between the matching degree and the standard features. If the matching degree does not exceed the preset matching degree, then the candidate features in the preset standard multidimensional feature library that have a similarity to the multidimensional feature to be detected that exceeds the preset similarity are retrieved, the matching degree is recalculated, and then the output category, probability of harmful subspecies, and confidence interval are determined.

[0015] Furthermore, the alarm marking includes: When the lower limit of the confidence interval is lower than the preset confidence threshold, or the probability of a harmful subspecies exceeds the preset probability threshold of a harmful subspecies, an alarm flag is set. When the lower limit of the confidence interval is not lower than the preset confidence threshold and the probability of harmful subspecies does not exceed the preset probability threshold of harmful subspecies, no alarm flag is set.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention simultaneously acquires bright-field images, multi-channel fluorescence images, near-infrared spectral data, and metadata of the colonies to be detected. Compared with traditional single-modal detection, it can comprehensively capture colony morphology, fluorescence, and metabolic characteristics, avoiding information loss. After targeted data preprocessing, multi-dimensional features are extracted and fused with spatiotemporal sequences to improve the completeness and accuracy of feature representation. By matching the fused features with a standard feature library to calculate the matching degree, the output category, the probability of harmful subspecies, and the confidence interval are determined. With the help of preset threshold marking alarm signs, a standardized identification and classification mechanism is established to solve the problems of subjectivity and difficulty in quantifying confidence in traditional classification. Finally, a test report with traceability information is generated. The entire process is automated, reducing human intervention, reducing errors, and increasing detection throughput, adapting to the needs of accurate detection in multiple scenarios.

[0017] On the other hand, the present invention also proposes a high-throughput colony detection sensor based on multimodal optics and spectral fusion, which is applicable to a high-throughput colony detection method based on multimodal optics and spectral fusion, including: a data acquisition module, a data processing module, a feature processing module, and an identification control module.

[0018] Furthermore, the data acquisition module includes a microscopic imaging unit, a multi-channel fluorescence imaging unit, and a near-infrared spectroscopy unit; the microscopic imaging unit, the multi-channel fluorescence imaging unit, and the near-infrared spectroscopy unit are respectively used to acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, and simultaneously record metadata of the acquisition process; The data processing module is used to receive the data collected by the data acquisition module and to preprocess the data; The feature processing module is used to extract multidimensional features based on the preprocessed data, and to fuse the multidimensional features in combination with spatiotemporal sequence information to obtain fused multidimensional features. The identification and control module is used to identify and classify colonies based on the fused multidimensional features, and generate a test report containing traceability information based on the identification and classification results. The test report is also visualized in real time. The identification control module is electrically connected to the data acquisition module, data processing module, and feature processing module, and controls the operation of each module.

[0019] It is understood that the high-throughput colony detection sensor and detection method based on multimodal optics and spectral fusion in the above embodiments of the present invention have the same beneficial effects, and will not be described again. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a high-throughput colony detection method based on multimodal optics and spectral fusion provided in an embodiment of the present invention; Figure 2 This is a functional block diagram of a high-throughput colony detection sensor based on multimodal optics and spectral fusion, provided for an embodiment of the present invention. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] Traditional colony detection methods often employ a single detection modality, such as relying solely on microscopic imaging to observe morphology or analyzing metabolic characteristics through a single spectral method, making it difficult to comprehensively capture colony information. Detection throughput is low, processing only a small number of samples per run, and image acquisition, data recording, and feature extraction require manual intervention, making the preprocessing cumbersome and prone to human error. Furthermore, identification and classification lack standardized matching mechanisms, confidence levels cannot be quantified, alarm thresholds are subjectively set, and traceability information in test reports is incomplete, making them unsuitable for large-scale, precise detection scenarios.

[0023] Therefore, it is necessary to design a high-throughput colony detection sensor and its detection method based on multimodal optics and spectral fusion to solve the problems of traditional methods, such as incomplete information from a single modality, low detection throughput, high susceptibility to errors due to excessive human intervention, lack of standardized identification and classification mechanisms, difficulty in quantifying confidence levels, subjective alarms, and incomplete traceability information.

[0024] Reference Figure 1 In some embodiments of this application, a high-throughput colony detection method based on multimodal optics and spectral fusion includes: S1. Acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be tested, and simultaneously record metadata of the acquisition process.

[0025] S2. Preprocess the data and extract multidimensional features of colonies from the preprocessed data; Among them, grayscale processing is performed on the preprocessed brightfield image data to obtain a grayscale image, and morphological features are extracted based on the grayscale image; The mean fluorescence intensity of each channel is calculated based on the preprocessed multi-channel fluorescence image; fluorescence features are extracted based on the mean fluorescence intensity. Peak identification is performed on the preprocessed near-infrared spectral data to mark the position of all peaks in the spectral curve; metabolic features are extracted based on the position of each marked peak.

[0026] S3. Combine spatiotemporal sequence information to perform multidimensional feature fusion processing.

[0027] S4. Identify and classify colonies based on the fused multidimensional features.

[0028] The identification and classification process includes: matching the fused multidimensional features with a preset standard multidimensional feature library, calculating the matching degree, and determining the output category, probability of harmful subspecies, and confidence interval of the colony to be detected based on the matching degree; Alarm flags are set based on the confidence interval, the probability of harmful subspecies, and preset confidence thresholds and harmful subspecies probability thresholds; S5. Generate a test report containing traceability information based on the identification and classification results.

[0029] Understandably, by simultaneously acquiring bright-field images, multi-channel fluorescence images, and near-infrared spectral data and metadata of the colonies to be detected, compared to traditional single-modal detection, it can comprehensively capture colony morphology, fluorescence, and metabolic characteristics, avoiding information loss. After targeted data preprocessing, multi-dimensional features are extracted and combined with spatiotemporal sequence fusion to improve the completeness and accuracy of feature representation. By matching the fused features with a standard feature library to calculate the matching degree, the output category, the probability of harmful subspecies, and the confidence interval are determined. With the help of preset threshold marking alarm signs, a standardized identification and classification mechanism is established to solve the problems of subjectivity and difficulty in quantifying confidence in traditional classification. Finally, a test report with traceability information is generated. The entire process is automated, reducing human intervention, reducing errors, and increasing detection throughput, adapting to the needs of accurate detection in multiple scenarios.

[0030] Specifically, in S1, the bright-field image reflects the macroscopic morphology of the colony, such as its boundary, size, and texture, and is the core data for extracting colony morphological features; the multi-channel fluorescence image acquires the fluorescence signal image emitted by specific fluorescent substances (such as metabolites and fluorescent markers) within the colony, with different channels corresponding to the response characteristics of different fluorescent substances; and the near-infrared spectral data is the absorption / reflection spectrum generated by the molecular vibration of chemical substances (such as proteins, polysaccharides, and nucleic acids) within the colony, reflecting the chemical composition characteristics of the colony.

[0031] Specifically, metadata refers to key environmental and operational parameters during data acquisition, used for result traceability and repeatability verification to ensure the reproducibility and traceability of results. Metadata includes temperature and humidity during acquisition, sample batch, culture medium type, image exposure parameters, and spectral integration time.

[0032] Specifically, the metadata recorded synchronously during the data acquisition process includes: During the data acquisition process, temperature and humidity data are collected and recorded according to the preset recording intervals; Record the corresponding image exposure parameters when the bright-field image acquisition is completed; Record the corresponding image exposure parameters when the multi-channel fluorescence image acquisition is completed; Record the corresponding spectral integration time when the near-infrared spectral data acquisition is completed; After data collection is completed, the sample batch, culture medium type and collected data are associated and stored respectively.

[0033] Understandably, by recording operating parameters such as temperature, humidity, and exposure parameters in real time, when the test results fluctuate (such as large differences in colony characteristics between different wells in the same batch), it is possible to trace back whether the environmental temperature and humidity are stable and whether the exposure parameters are consistent, and locate the root cause of the error. By associating and storing sample batches, culture medium types, and collected data, the problem of inconsistent correspondence between samples and data in traditional methods can be solved, ensuring that the data of a single sample throughout the entire process can be accurately located.

[0034] Specifically, when collecting bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be tested, a trigger signal is used to control the synchronous initiation of the three types of data acquisition actions, and the sending time of the trigger signal is determined according to the growth stage of the colony culture.

[0035] Specifically, determining the timing of the trigger signal transmission includes: The data collection cycle is preset according to the growth stage of the colony culture to be tested; The time interval between two consecutive data collections of the three types of data was compared with the preset data collection period. When the time interval between two consecutive data acquisitions is equal to the preset data acquisition period, the same trigger signal is sent to control the simultaneous acquisition of the three types of data. If the time interval between any two consecutive data acquisitions is not equal to the preset data acquisition period, a segmented trigger signal is sent. Specifically, the preset data collection period is a variable of collection time intervals corresponding to different growth stages determined by the growth rate test of the colonies to be tested.

[0036] Specifically, the system records the time of the last complete data acquisition in real time and calculates the difference between this time and the current time (i.e., the time interval between two adjacent acquisitions); then it compares this difference with the preset data acquisition cycle. If the interval of all data is equal to the preset period, it means that the current time point is suitable for data collection, and the three types of data can be started synchronously; if the interval of any data is not equal to the preset period, segmented triggering is required to avoid invalid waiting, and controlling the time difference can ensure data correlation.

[0037] Specifically, sending segmentation trigger signals includes: First, send the first control signal to control the simultaneous acquisition of bright-field images and multi-channel fluorescence images; After the bright-field image and multi-channel fluorescence image acquisition are completed, a second control signal is sent to control the acquisition of near-infrared spectral data. The time interval between the first control signal and the second control signal shall not exceed the preset time difference; Specifically, the preset time difference is the maximum time interval variable that ensures the correlation of the three types of data, determined through data correlation experiments. If this difference is exceeded, the data will not be able to match the same colony state.

[0038] Understandably, given the characteristics of short imaging time and long spectral time, adopting a step-by-step start-up mode can improve the efficiency of data acquisition.

[0039] Understandably, bright-field images can directly reflect macroscopic morphological differences in colonies, forming the basis for initial colony identification; multi-channel fluorescence images can capture differences in specific metabolic activities within colonies, such as pathogenic colonies exhibiting unique fluorescence responses due to specific metabolites; near-infrared spectroscopy can reveal molecular-level differences in the chemical composition of colonies, enabling precise differentiation of colonies that are "morphologically similar but have different compositions." Single-modal data cannot fully cover these characteristics; fusing multi-modal data can achieve complementarity of "morphological-physiological-chemical" features. Synchronous acquisition ensures that data corresponds to the same colony state, avoiding feature misalignment caused by asynchronous acquisition; metadata association can establish a traceability chain of "sample-data-process," providing a basis for investigating the causes of abnormal results.

[0040] Specifically, in S2, data preprocessing includes: correcting and denoising the brightfield image and multi-channel fluorescence image data. First, distinguish between bright-field images and multi-channel fluorescence images, then perform correction processing on each type of image separately. After correction, perform denoising processing on both types of images. When performing the correction process, the correction parameters corresponding to the recorded culture medium type are retrieved. For bright field images, distortion correction is performed in particular, and for multi-channel fluorescence images, light intensity uniformity correction is performed in particular. When performing denoising, noise areas outside the bacterial colony areas in the image are first identified by grayscale comparison, and then denoising operations are performed on the noise areas.

[0041] Understandably, differentiating correction algorithms can be used to address the "insufficient targeting" of traditional uniform correction by targeting different sources of deviation in brightfield images (mainly distortion) and fluorescence images (mainly uneven light intensity). Different culture media have different background characteristics, and by using specific correction parameters, the impact of background interference on the correction effect can be reduced, thus improving the correction accuracy. Based on the significant difference in grayscale between colonies and the background, noise areas are located first, and then targeted denoising is performed to avoid the destruction of effective features by traditional global denoising and ensure the quality of subsequent feature extraction.

[0042] Specifically, the spectral data is denoised and baseline subtracted: First, retrieve the denoising parameters corresponding to the recorded spectral integration time, and perform denoising using a filtering method. After denoising is completed, perform baseline subtraction.

[0043] It is understandable that the integration time determines the noise intensity. By retrieving the matching denoising parameters, strong noise is filtered strongly and weak noise is filtered weakly to avoid insufficient denoising or excessive destruction of the peak shape. Denoising first and then baseline subtraction can prevent noise from being included in the baseline fitting (if the baseline is subtracted first, the noise will be misjudged as background, resulting in baseline fitting deviation), thus ensuring baseline accuracy.

[0044] It is understandable that the raw data collected contains noise, interference signals, and redundant information. For example, bright-field images may have deformed colony boundaries due to optical system distortion, fluorescence images may have intensity fluctuations due to uneven light source illumination, and spectral data may contain baseline drift due to ambient light interference. Preprocessing can eliminate these interferences and improve data quality.

[0045] Specifically, in S3, grayscale processing can eliminate color interference in image data and simplify the subsequent morphological feature extraction and calculation process; the mean fluorescence intensity is the average fluorescence intensity of all pixels in the colony area within a single channel in a multi-channel fluorescence image, reflecting the overall content of the fluorescent substance corresponding to that channel; peak identification is the analysis of the near-infrared spectral curve, marking the wavelength position corresponding to the local maximum value (peak) in the curve, and the peak is the characteristic absorption peak of a specific chemical substance.

[0046] Specifically, morphological characteristics are parameters reflecting the macroscopic morphology of colonies, including colony boundaries, texture, and roughness. Fluorescence characteristics are feature parameters calculated based on the average fluorescence intensity of different channels, including multi-band intensity ratios, which can eliminate interference from uneven illumination and reflect differences in the types of fluorescent substances. Metabolic characteristics are parameters reflecting the chemical composition of colony metabolites extracted from spectral peaks, including peak position, peak intensity, and full width at half maximum (FWHM).

[0047] Specifically, when extracting multidimensional features from preprocessed data, the multidimensional features are associated with and stored with their corresponding metadata.

[0048] As can be understood, feature extraction transforms preprocessed data into a "feature language" that can be used for identification. Different features correspond to different dimensions of the biological characteristics of the bacterial colony. Each extracted feature is bound and stored with its corresponding metadata to ensure feature traceability.

[0049] Specifically, in S4, the spatiotemporal sequence information is a sequence formed by collecting data of the same colony at different culture time points in chronological order, and then combined with the well location information of the colony in the culture plate to form multi-dimensional data association information.

[0050] Specifically, feature fusion combines morphological, fluorescence, and metabolic features, along with spatiotemporal sequence information, using a weighted fusion algorithm to form a numerical set containing all key information, resulting in a fused feature vector.

[0051] It is understandable that multidimensional features at a single point in time can only reflect the characteristics of a certain growth state of the colony, while the dynamic changes of features during colony growth (such as morphological expansion and accumulation of metabolites) can provide richer identification basis. Combining time series information can improve the stability of identification. Feature fusion can eliminate redundant information between different features and form a more distinctive comprehensive feature set.

[0052] Specifically, the preset standard multidimensional feature library stores standard features and subspecies features of different types of colonies, which serve as a reference for identification and classification. The standard feature set it contains covers common colony types, ensuring adaptability to most detection scenarios.

[0053] Specifically, the output category, probability of harmful subspecies, and confidence interval of the colony are determined based on the matching degree, including: Obtain the matching degree for different categories; When the matching degree exceeds the preset matching degree, the category with the highest matching degree is directly used as the output category, and the subspecies probability and confidence interval are calculated based on the deviation between the matching degree and the standard features. If the matching degree does not exceed the preset matching degree, then the candidate features in the preset standard multidimensional feature library that have a similarity to the multidimensional feature to be detected that exceeds the preset similarity are retrieved, the matching degree is recalculated, and then the output category, probability of harmful subspecies, and confidence interval are determined.

[0054] Specifically, the matching degree is the similarity between the fused multidimensional feature vector of the colony to be detected and the standard feature vector of a certain type of colony in the standard feature library, which can be calculated using a weighted cosine similarity algorithm. All matching degree values ​​and corresponding colony categories are recorded simultaneously during the matching degree calculation. The preset matching degree is the threshold for direct classification, determined experimentally. If the matching degree is greater than the preset matching degree, the species can be directly determined. The probability of a harmful subspecies is the likelihood that the colony to be detected belongs to a harmful subspecies, calculated based on the deviation between the detected feature and the subspecies feature. The confidence interval reflects the reliability of the classification result, determined by the matching degree and the stability of the feature deviation. The higher the matching degree and the more stable the deviation, the narrower the interval and the higher the lower limit of the interval.

[0055] Specifically, the preset similarity is a threshold for retrieving candidate features. When the matching degree is less than the preset matching degree, features in the library with a similarity to the feature to be detected that is greater than or equal to the preset similarity are selected as candidates to avoid missing variant colonies. Candidate features are features in the standard library that have a high similarity to the feature to be detected but do not reach the preset matching degree, and are used for secondary matching.

[0056] Understandably, matching degree calculation achieves objective classification by quantifying the similarity between the feature to be detected and the standard feature, avoiding the subjectivity of human judgment; the calculation of the probability of harmful subspecies and the confidence interval can quantify the risk level and reliability of the results; alarm flags are triggered by preset thresholds, which can ensure that high-risk results are not missed, while avoiding false alarms for low-risk results.

[0057] Specifically, the alarm flag is set when the lower limit of the confidence interval is lower than the preset confidence threshold, or when the probability of a harmful subspecies exceeds the preset harmful subspecies probability threshold. When the lower limit of the confidence interval is not lower than the preset confidence threshold and the probability of harmful subspecies does not exceed the preset probability threshold of harmful subspecies, no alarm flag is set.

[0058] Understandably, the preset confidence threshold is the minimum standard for judging the reliability of classification results. That is, the lower limit of the confidence interval must be greater than or equal to this value for the result to be considered reliable. If the lower limit is less than this value, it indicates that the result fluctuates greatly and an alarm should be triggered to indicate that the result is questionable.

[0059] Understandably, the preset threshold for the probability of harmful subspecies is a safe upper limit for determining the potential harm of a colony. That is, the probability of harmful subspecies must be less than or equal to this value for the risk to be considered controllable; if the probability is greater than this value, it means that the colony is likely to be a harmful subspecies, and an alarm should be triggered to indicate that the risk exceeds the standard.

[0060] Specifically, in S5, traceability information can be used to trace key information throughout the entire detection process, including the model version used for detection, all operating parameters, metadata, preprocessing and fusion algorithm parameters, etc., to ensure that the detection process is reproducible.

[0061] As can be seen, this embodiment achieves comprehensive capture of the "morphological-physiological-chemical" characteristics of colonies by fusing bright-field, fluorescence, and spectral multimodal data, which reduces the false positive rate compared with traditional single-modal methods; combined with spatiotemporal sequence information, it further enhances the ability to distinguish similar colonies and improves the accuracy of colony identification.

[0062] By standardizing feature libraries and matching processes, reliance on human experience is eliminated, reducing the detection error rate. Through the associated storage of metadata and full-process parameters, end-to-end traceability of "sample-data-process-result" is achieved. When results are questionable, the detection process can be quickly reproduced to investigate the cause of errors and improve the reliability of results. By triggering alarm flags with dual thresholds, the false negative rate of harmful subspecies is reduced, providing accurate evidence for risk management in scenarios such as food safety and clinical diagnosis, and lowering the incidence of safety incidents.

[0063] See Figure 2 As shown, in some embodiments of this application, a high-throughput colony detection sensor based on multimodal optics and spectral fusion is also provided, which is applicable to a high-throughput colony detection method based on multimodal optics and spectral fusion, including: a data acquisition module, a data processing module, a feature processing module, and an identification control module.

[0064] Specifically, the data acquisition module includes a microscopic imaging unit, a multi-channel fluorescence imaging unit, and a near-infrared spectroscopy unit; the microscopic imaging unit, the multi-channel fluorescence imaging unit, and the near-infrared spectroscopy unit are used to acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, respectively, and simultaneously record metadata of the acquisition process; The data processing module is used to receive data collected by the data acquisition module and to preprocess the data; The feature processing module is used to extract multidimensional features from the preprocessed data and fuse the multidimensional features with spatiotemporal sequence information to obtain fused multidimensional features. The identification and control module is used to identify and classify colonies based on the fused multidimensional features, and generate a test report with traceability information based on the identification and classification results. The test report is also visualized in real time. The identification control module is electrically connected to the data acquisition module, data processing module, and feature processing module, and controls the operation of each module.

[0065] It is understood that, in the above embodiments of the present invention, a high-throughput colony detection sensor and its detection method based on multimodal optics and spectral fusion have the same beneficial effects, and will not be described again.

[0066] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A high-throughput colony detection method based on multimodal optics and spectral fusion, characterized in that: Acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, and simultaneously record metadata of the acquisition process; The data is preprocessed, and multidimensional features of the colonies are extracted from the preprocessed data; Specifically, grayscale processing is performed on the preprocessed brightfield image data to obtain a grayscale image, and morphological features are extracted based on the grayscale image; The mean fluorescence intensity of each channel is calculated based on the preprocessed multi-channel fluorescence image; fluorescence features are extracted based on the mean fluorescence intensity. Peak identification is performed on the preprocessed near-infrared spectral data to mark the position of all peaks in the spectral curve; metabolic features are extracted based on the position of each marked peak. The multidimensional features are fused by combining spatiotemporal sequence information; Colonies are identified and classified based on the fused multidimensional features; The identification and classification includes: matching the fused multidimensional features with a preset standard multidimensional feature library, calculating the matching degree, and determining the output category, probability of harmful subspecies, and confidence interval of the colony to be detected based on the matching degree; Based on the confidence interval, the probability of harmful subspecies, and the preset confidence threshold and harmful subspecies probability threshold, an alarm flag is marked; A detection report containing traceability information is generated based on the identification and classification results.

2. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 1, characterized in that, The metadata includes the temperature and humidity at the time of acquisition, sample batch, culture medium type, image exposure parameters, and spectral integration time; The metadata of the synchronous recording and acquisition process includes: During the data acquisition process, temperature and humidity data are collected and recorded according to the preset recording intervals; Record the corresponding image exposure parameters when the bright-field image acquisition is completed; Record the corresponding image exposure parameters when the multi-channel fluorescence image acquisition is completed; Record the corresponding spectral integration time when the near-infrared spectral data acquisition is completed; After data collection is completed, the sample batch, culture medium type, and the collected data are associated and stored respectively.

3. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 2, characterized in that, When acquiring bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be tested, a trigger signal is used to control the synchronous start of the three types of data acquisition actions, and the sending time of the trigger signal is determined according to the growth stage of the colony to be tested. Determining the transmission time of the trigger signal includes: The data collection cycle is preset according to the growth stage of the colony culture to be detected; The time interval between two consecutive data collections of the three types of data was compared with the preset data collection period. When the time interval between two consecutive data acquisitions is equal to the preset data acquisition period, the same trigger signal is sent to control the simultaneous acquisition of the three types of data. If the time interval between any two consecutive data acquisitions is not equal to the preset data acquisition period, a segmented trigger signal is sent. The preset data acquisition period is a variable of acquisition time interval corresponding to different growth stages determined by the growth rate test of the colonies to be detected.

4. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 3, characterized in that, The segmentation trigger signal includes: First, send the first control signal to control the simultaneous acquisition of bright-field images and multi-channel fluorescence images; After the bright-field image and multi-channel fluorescence image are acquired, a second control signal is sent to control the acquisition of near-infrared spectral data; The time interval between the first control signal and the second control signal does not exceed a preset time difference; The preset time difference is the maximum time interval variable determined through data correlation experiments to ensure the correlation of the three types of data.

5. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 4, characterized in that, The data is preprocessed, including: correcting and denoising the bright-field image and multi-channel fluorescence image data; First, bright-field images and multi-channel fluorescence images are distinguished, and then correction processing is performed on each type of image. After correction, denoising processing is performed on the two types of images. When performing the correction process, the correction parameters corresponding to the recorded culture medium type are retrieved. For bright field images, distortion correction is performed in particular, and for multi-channel fluorescence images, light intensity uniformity correction is performed in particular. When performing denoising, noise areas outside the bacterial colony areas in the image are first identified by grayscale comparison, and then denoising operations are performed on the noise areas.

6. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 5, characterized in that, Preprocessing the data further includes: denoising and baseline subtraction of the spectral data; First, the denoising parameters corresponding to the recorded spectral integration time are retrieved, and denoising is performed using a filtering method. After denoising is completed, baseline subtraction is then performed.

7. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 6, characterized in that, The extraction of multidimensional features from the preprocessed data also includes: During the extraction process, the multidimensional features are associated with and stored with their corresponding metadata. The morphological features include colony boundaries, texture, and roughness; The fluorescence characteristics include multi-band intensity ratios; The metabolic characteristics include peak position, peak intensity, and half-maximum width.

8. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 7, characterized in that, The preset standard multidimensional feature library stores standard features and subspecies features of different types of bacterial colonies; Based on the matching degree, the output category, probability of harmful subspecies, and confidence interval of the colony are determined, including: Obtain the matching degree for different categories; When the matching degree exceeds the preset matching degree, the category with the highest matching degree is directly used as the output category, and the subspecies probability and confidence interval are calculated based on the deviation between the matching degree and the standard features. If the matching degree does not exceed the preset matching degree, then the candidate features in the preset standard multidimensional feature library that have a similarity to the multidimensional feature to be detected that exceeds the preset similarity are retrieved, the matching degree is recalculated, and then the output category, probability of harmful subspecies, and confidence interval are determined.

9. The high-throughput colony detection method based on multimodal optics and spectral fusion according to claim 8, characterized in that, The alarm marking includes: When the lower limit of the confidence interval is lower than the preset confidence threshold, or the probability of a harmful subspecies exceeds the preset probability threshold of a harmful subspecies, an alarm flag is set. When the lower limit of the confidence interval is not lower than the preset confidence threshold and the probability of harmful subspecies does not exceed the preset probability threshold of harmful subspecies, no alarm flag is marked.

10. A high-throughput colony detection sensor based on multimodal optics and spectral fusion, applicable to the high-throughput colony detection method based on multimodal optics and spectral fusion as described in any one of claims 1-9, characterized in that, include: Data acquisition module, data processing module, feature processing module, and recognition control module; Furthermore, the data acquisition module includes a microscopic imaging unit, a multi-channel fluorescence imaging unit, and a near-infrared spectroscopy unit; the microscopic imaging unit, the multi-channel fluorescence imaging unit, and the near-infrared spectroscopy unit are respectively used to acquire bright-field images, multi-channel fluorescence images, and near-infrared spectral data of the colonies to be detected, and simultaneously record metadata of the acquisition process; The data processing module is used to receive the data collected by the data acquisition module and to preprocess the data; The feature processing module is used to extract multidimensional features based on the preprocessed data, and to fuse the multidimensional features in combination with spatiotemporal sequence information to obtain fused multidimensional features. The identification and control module is used to identify and classify colonies based on the fused multidimensional features, and generate a test report containing traceability information based on the identification and classification results. The test report is also visualized in real time. The identification control module is electrically connected to the data acquisition module, data processing module, and feature processing module, and controls the operation of each module.

Images