DETR bacterial colony detection model training method, DETR bacterial colony detection model application method and related system

Abstract

The invention discloses a DETR bacterial colony detection model training method, an application method and a related system, and relates to the technical field of microorganism bacterial colony detection.The training method comprises the steps that a plurality of culture dish images are collected; each culture dish image is correspondingly marked with a real frame and a real category label; respectively inputting the plurality of culture dish images into a DETR bacterial colony detection model, and determining a loss value of a loss function according to a prediction frame and a prediction category label output by the DETR bacterial colony detection model and the corresponding real frame and the real category label; the DETR bacterial colony detection model is a model which is established based on a DETR network of a self-attention mechanism and is used for predicting the category and position of the microbial colony; parameters of the DETR bacterial colony detection model are optimized according to the loss value of the loss function, and a trained DETR bacterial colony detection model is obtained. According to the method, the precision and efficiency of microbial colony detection can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of microbial colony detection, and in particular to a training method, an application method and a related system of a DETR colony detection model. Background Art

[0002] The traditional clinical microbial colony detection process relies on temperature-controlled culture, which takes a long time and requires manual observation of changes in the culture dish to identify suspicious colonies in the culture dish. This process is not only time-consuming and inefficient, but also highly dependent on the operator's personal experience and subjective consciousness, prone to detection errors caused by human factors, and relatively low detection accuracy.

[0003] Based on this, how to provide a more accurate and efficient method for detecting and identifying microbial colonies has become a technical problem that needs to be urgently solved in this field. Summary of the invention

[0004] The purpose of this application is to provide a training method, application method and related system of a DETR colony detection model, which can effectively improve the accuracy and efficiency of detecting microbial colonies.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for training a DETR colony detection model, the training method for the DETR colony detection model comprising the following steps:

[0007] Collect a number of culture dish images; the culture dish images include the culture dish and the microbial colonies cultured in the culture dish, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony.

[0008] Several culture dish images are respectively input into the DETR colony detection model, and the loss value of the loss function is determined according to the prediction box and the prediction category label output by the DETR colony detection model and the corresponding true box and the true category label; the DETR colony detection model is a model established by the DETR network based on the self-attention mechanism for predicting the category and position of the microbial colony; the prediction box is used to circle the position of the microbial colony identified by the DETR colony detection model on the culture dish image, and the prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model.

[0009] The parameters of the DETR colony detection model are optimized according to the loss value of the loss function to obtain a trained DETR colony detection model.

[0010] Optionally, collecting a plurality of culture dish images specifically includes the following steps:

[0011] A plurality of culture dishes are used to culture different types of microbial colonies respectively, and a temperature-controlled culture device is used to control the temperature of each culture dish.

[0012] A timing camera is used to take timed shots of each of the culture dishes to obtain a number of original culture dish images.

[0013] The real categories and real positions of the microbial colonies in the original culture dish images are respectively marked to obtain a plurality of the culture dish images.

[0014] Optionally, the DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder and multiple prediction heads in sequence; wherein the backbone network is used to extract features from the culture dish image to obtain a feature map; the transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feedforward network, the transformer encoder is used to perform position encoding on the feature map, capture the global dependencies of each part of the feature map through the multi-head self-attention mechanism, and generate a final feature map including the semantic information and spatial position information of the culture dish image; the transformer decoder is used to decode the final feature map; the prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

[0015] Optionally, the loss function is a bipartite matching loss function.

[0016] In a second aspect, the present application provides an application method of a DETR colony detection model, the application method of the DETR colony detection model comprising the following steps:

[0017] A target culture dish image is collected; the target culture dish image is a culture dish image to be detected without a real frame and a real category label.

[0018] The target culture dish image is input into the DETR colony detection model to obtain the prediction box and prediction category label corresponding to the target culture dish image; the DETR colony detection model is a model trained according to the training method of the DETR colony detection model described in the first aspect.

[0019] In a third aspect, the present application provides a training system for a DETR colony detection model, the training system for the DETR colony detection model comprising:

[0020] A culture dish image acquisition module is used to acquire a number of culture dish images; the culture dish images include culture dishes and microbial colonies cultured in the culture dishes, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony.

[0021] The loss function loss value determination module is used to input several culture dish images into the DETR colony detection model respectively, and determine the loss value of the loss function according to the prediction box and prediction category label output by the DETR colony detection model and the corresponding real box and the real category label; the DETR colony detection model is a model established by the DETR network based on the self-attention mechanism for predicting the category and position of the microbial colony; the prediction box is used to circle the position of the microbial colony identified by the DETR colony detection model on the culture dish image, and the prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model.

[0022] The training module is used to optimize the parameters of the DETR colony detection model according to the loss value of the loss function to obtain a trained DETR colony detection model.

[0023] Optionally, the culture dish image acquisition module includes:

[0024] The culture unit is used to culture different types of microbial colonies using a plurality of the culture dishes, and to control the temperature of each of the culture dishes using a temperature control culture device.

[0025] The original culture dish image acquisition unit is used to use a timing camera to take timed photos of each of the culture dishes to obtain a plurality of original culture dish images.

[0026] The labeling unit is used to label the real category and real position of the microbial colony in the original culture dish image respectively to obtain a plurality of the culture dish images.

[0027] Optionally, the DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder and multiple prediction heads in sequence; wherein the backbone network is used to extract features from the culture dish image to obtain a feature map; the transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feedforward network, the transformer encoder is used to perform position encoding on the feature map, capture the global dependencies of each part of the feature map through the multi-head self-attention mechanism, and generate a final feature map including the semantic information and spatial position information of the culture dish image; the transformer decoder is used to decode the final feature map; the prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

[0028] Optionally, the loss function is a bipartite matching loss function.

[0029] In a fourth aspect, the present application provides an application system of a DETR colony detection model, the application system of the DETR colony detection model comprising:

[0030] The target culture dish image acquisition module is used to acquire the target culture dish image; the target culture dish image is the culture dish image to be detected without a real frame and a real category label.

[0031] The colony detection module is used to input the target culture dish image into the DETR colony detection model to obtain the prediction box and prediction category label corresponding to the target culture dish image; the DETR colony detection model is a model trained according to the training method of the DETR colony detection model described in the first aspect.

[0032] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0033] The present application provides a training method, application method and related system of a DETR colony detection model, a DETR colony detection model established by a DETR network based on a self-attention mechanism, and the DETR colony detection model is applied to the scene of microbial colony detection and identification, the DETR colony detection model takes a culture dish image as input, and takes a prediction box and a prediction category label corresponding to the culture dish image as output, to realize the detection and identification of the category and position of the microbial colony in the culture dish image. By training the DETR colony detection model, the trained DETR colony detection model has a strong performance in identifying microbial colonies, fully utilizing the characteristics of the DETR network and its self-attention mechanism, being able to integrate the global information of the culture dish image, and capturing long-distance dependencies in the entire culture dish image, and being able to accurately identify microbial colonies in complex backgrounds, without the need to manually observe the changes in the culture dish, and without relying on the personal experience and subjective consciousness of the operator for colony identification, taking less time and higher accuracy, and being able to effectively improve the accuracy and efficiency of detecting microbial colonies. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of a training method for a DETR colony detection model provided in Example 1 of the present application.

[0036] Figure 2 This is a schematic diagram of the structure of the DETR colony detection model provided in Example 1 of the present application.

[0037] Figure 3 A schematic flow chart of an application method of a DETR colony detection model provided in Example 2 of the present application.

[0038] Figure 4 This is a structural block diagram of a training system for a DETR colony detection model provided in Example 3 of the present application.

[0039] Figure 5 This is a structural block diagram of an application system of a DETR colony detection model provided in Example 4 of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] Example 1

[0043] like Figure 1 As shown, this embodiment proposes a training method for a DETR (Detection Transformer) colony detection model, which is mainly a method for training a DETR colony detection model, and the DETR colony detection model is mainly used in the scene of detecting and identifying the category and location of microbial colonies in a culture dish. The training method specifically includes the following steps:

[0044] Step S1, collecting several culture dish images.

[0045] In this embodiment, the culture dish image includes a culture dish and a microbial colony cultured in the culture dish, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony.

[0046] In this embodiment, step S1 collects a number of culture dish images, which specifically includes the following steps:

[0047] Step S11, using a plurality of the culture dishes to culture different types of microbial colonies respectively, and using a temperature control culture device to control the temperature of each of the culture dishes.

[0048] Step S12: using a timing camera to take timed shots of each of the culture dishes to obtain a number of original culture dish images.

[0049] Step S13: annotate the real categories and real positions of the microbial colonies in the original culture dish image respectively to obtain a plurality of the culture dish images.

[0050] Step S2: inputting the plurality of culture dish images into the DETR colony detection model respectively, and determining the loss value of the loss function according to the predicted box and predicted class label output by the DETR colony detection model and the corresponding real box and real class label, wherein the loss function is a Hungarian Loss function.

[0051] In this embodiment, the DETR colony detection model is a model established by a DETR network based on a self-attention mechanism for predicting the type and location of the microbial colony. Figure 2 As shown, the DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder, and multiple prediction heads in sequence. The backbone network is used to extract features from the culture dish image to obtain a feature map. The transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feed-forward network (FFN). The transformer encoder is used to perform positional encoding on the feature map, and the global dependency of each part in the feature map is captured through the multi-head self-attention mechanism to generate a final feature map including the semantic information and spatial position information of the culture dish image. The transformer decoder is used to decode the final feature map. The prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

[0052] In this embodiment, the backbone network of the DETR colony detection model includes a CNN (convolutional neural network) feature extractor. The culture dish image is first input into the CNN feature extractor, which is used to extract the basic features of the culture dish image and output a feature map with lower resolution. The commonly used CNN model is ResNet (Residual Network). After the feature map is extracted, position encoding needs to be added. This step ensures that the transformer encoder can know the spatial position information of the features in the feature map. Position encoding uses sine and cosine functions to encode the position information of each pixel in the feature map. The original image is converted into a set of features containing position information, which will serve as the input of the transformer encoder.

[0053] In this embodiment, the transformer encoder is used to process the image features extracted from the CNN feature extractor and capture the global dependencies of each part in the feature map through the self-attention mechanism. The task of the transformer encoder is to integrate the global and local features of the image so that the DETR colony detection model can better understand the relationship between complex backgrounds and objects. The transformer encoder consists of multiple layer structures, each of which includes a multi-head self-attention layer and a feedforward network. The multi-head self-attention layer uses the multi-head self-attention mechanism to focus on different areas in the feature map and capture long-distance dependencies. The feedforward network further processes the features that have passed the attention mechanism to increase the expressive power of the model. The task of the transformer decoder is to generate outputs for target detection. The query mechanism (object queries) is introduced in the DETR colony detection model. These query vectors interact with the features output by the transformer encoder to predict the location and category of objects in the image. The transformer decoder also consists of multiple layer structures, each of which also includes a multi-head self-attention layer and a feedforward network. The query vector interacts with the output of the transformer encoder, the number of query vectors is fixed, and each query vector predicts one target (i.e., colony). Each query vector predicts the location and category of the colony based on its similarity to the feature map, i.e., matching it with the information in the feature map through a self-attention mechanism.

[0054] In this embodiment, the role of the prediction head is to generate the final target detection results, including prediction boxes and prediction category labels, after the transformer decoder is processed. Specifically, each query vector passes through a feedforward network, which mainly outputs two results. The first result is the predicted category label, which is used to indicate the category of each predicted target, such as the category of a microbial colony; the second result is the prediction box, which is used to delineate the boundary position of each predicted target and mark the position of the target in the image. If some query vectors fail to match objects in the image, they will be marked as "no object", indicating that the query did not detect any valid targets. The final output is a set of predicted boxes and predicted category labels for the target generated by each query vector of the transformer decoder. These output results can be used for colony detection in culture dish images to mark the category and position of the colonies.

[0055] In this embodiment, the prediction box is used to delineate the position of the microbial colony identified by the DETR colony detection model on the culture dish image. The prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model.

[0056] In this embodiment, the feature extraction process is as follows:

[0057] (1) Input image processing: First, the DETR colony detection model receives the input culture dish image and performs preliminary processing on the image through a convolutional neural network (such as ResNet) to extract a set of feature maps. These feature maps represent the basic features in the input culture dish image, including edges, textures, and simple shapes.

[0058] (2) Feature map conversion to sequence: The feature map is flattened into a sequence, where each element corresponds to a pixel block of the feature map. These elements will be used as feature embeddings for the transformer encoder input sequence.

[0059] (3) Position encoding: In order to make the transformer encoder understand the information of each position in the feature map, the DETR colony detection model adds position encoding to each sequence element. Position encoding is a vector that encodes the spatial position information in the feature map, ensuring that the transformer encoder can recognize the features at a specific position in the feature map.

[0060] (4) Self-attention mechanism: The self-attention mechanism in the transformer encoder generates a weighted feature representation by calculating the similarity (attention weight) between each element in the sequence and other elements. Specifically, the self-attention mechanism considers the relationship between the features at each position in the feature map and all global features, thereby capturing important contextual information in the global scope of the feature map.

[0061] (5) Multi-head attention layer: In the multi-head self-attention layer of the transformer encoder, multiple attention heads operate in parallel, and each head learns different types of feature patterns in the feature map. Different types of feature patterns refer to various types of information learned by the DETR colony detection model in different regions and scales of the feature map. The outputs of these attention heads are then combined to generate a feature representation rich in global context information. The output of each attention head is a set of feature representations after self-attention calculation, which capture information at different levels and angles.

[0062] (6) Feature Mapping and Prediction: The sequence features processed by the multi-head self-attention layer are processed again through several layers of feed-forward networks to generate the final feature map. The final feature map includes high-level semantic information and spatial position information in the input culture dish image, representing the DETR colony detection model’s understanding of the culture dish image. The final feature map will be used to predict the object categories and their precise locations in the feature map.

[0063] (7) Output detection results: Finally, the output layer of the DETR colony detection model decodes the final feature map and generates a series of prediction boxes and corresponding prediction category labels to complete the microbial colony detection task. These detection results represent the location and category information of each microbial colony in the culture dish image.

[0064] In microbial experiments, different microbial colonies may have different morphologies and characteristics. Common microbial colony categories can be distinguished based on shape, color, and edge characteristics. For example, the colony shape of Escherichia coli is usually round or irregular, the color is generally off-white or milky white, smooth and moist, the edges are relatively smooth, the diameter is small, but it grows faster, and usually obvious colonies can be formed within 24 hours. The colony shape of Staphylococcus aureus is round, the surface is smooth, the color is usually yellow or golden yellow, the edges are neat, and the colonies of Staphylococcus aureus are usually thick and solid. The colony shape of Pseudomonas is irregular, the color is usually green or blue-green, and may be accompanied by fluorescence. The edges are usually irregular, and it is characterized by a special odor, and the surface of the colony is relatively moist. The colony shape of yeast is round, the color is milky white or light yellow, the edges are relatively neat, and the yeast colonies are usually thick, the surface is shiny, and the texture is like butter.

[0065] In this embodiment, the position encoding process is as follows:

[0066] (1) Generate position encoding vector: In the DETR colony detection model, the position encoding is a fixed set of vectors that matches the size of the feature map. The position encoding is generated by encoding the spatial dimensions of the feature map. Specifically, for each pixel position in the feature map, a corresponding vector is generated, and each dimension of the vector contains the spatial position information of the pixel position.

[0067] (2) Sine and cosine function encoding: The position encoding vector is usually generated using sine and cosine functions. For each position (x, y) in the feature map, its corresponding encoding vector is generated by the following formula:

[0068]

[0069] Among them, PE (x，y，2i) Indicates that for a two-dimensional coordinate (x, y), a sine function is used to generate the encoding value of the position on the 2ith dimension of the encoding vector; PE (x，y，2i+1) It means that for the two-dimensional coordinate (x, y), the cosine function is used to generate the encoding value of the position on the 2i+1th dimension of the encoding vector; x and y are the horizontal and vertical coordinates of a position in the feature map, respectively, i is the dimension index in the position encoding vector, and d is the total dimension of the position encoding vector.

[0070] (3) Two-dimensional position coding: The horizontal coordinate and the vertical coordinate are encoded by formula (1) and formula (2) respectively, thereby generating two vectors for each position, representing the position information in the horizontal direction and the position information in the vertical direction respectively. These two vectors are then merged to generate a complete two-dimensional position coding vector.

[0071] (4) Addition of position encoding: After the position encoding vector is generated, it will be added to the feature vector of each element (i.e., pixel block) in the feature map. This process ensures that the feature of each position contains not only visual information but also the spatial information of that position in the image.

[0072] (5) Application of position encoding: The position-encoded feature vector is input into the self-attention mechanism of the transformer encoder. This enables the transformer encoder to distinguish features at different positions in the image and understand the spatial relationship between features, thereby performing object detection more accurately.

[0073] Step S3: Optimize the parameters of the DETR colony detection model according to the loss value of the loss function to obtain a trained DETR colony detection model.

[0074] This embodiment uses the DETR colony detection model based on the self-attention mechanism to detect and identify microbial colonies, which can improve the DETR colony detection model's ability to identify microbial colony characteristics and achieve accurate and efficient microbial detection. By regularly collecting high-definition culture dish images and performing image analysis through the DETR colony detection model, different microbial colonies can be automatically identified and classified.

[0075] The DETR colony detection model in this embodiment uses the Transformer architecture and performs feature extraction and object detection through the self-attention mechanism. The self-attention mechanism can capture long-distance dependencies in the entire culture dish image, which is particularly important for the identification of microbial colonies in complex backgrounds. In addition, this embodiment can also achieve end-to-end detection. The DETR colony detection model simplifies the complex anchor settings and post-processing steps in the traditional detection framework, and directly realizes an end-to-end detection process from the culture dish image to the detection result. At the same time, in order to process the position information in the culture dish image, the DETR colony detection model in this embodiment introduces position encoding, so that Transf (position encoding is over described). The global information of the culture dish image can be integrated through the self-attention mechanism, which helps to improve the model's understanding of the overall shape of the colony and its tiny structural differences, thereby more accurately identifying microbial colonies.

[0076] In this embodiment, the microbial colony detection and identification process based on the DETR colony detection model is as follows:

[0077] (1) Feature extraction: First, the input culture dish image is passed through a pre-trained convolutional neural network (such as ResNet) to extract the feature map. Each pixel block in the feature map represents a part of the input culture dish image and contains rich local feature information.

[0078] (2) Position encoding: Position encoding is added to the feature map to preserve the spatial position information of each feature in the image. These position encodings are generated using sine and cosine functions to ensure that the transformer encoder can understand the spatial layout of each feature when processing it.

[0079] (3) Self-attention mechanism: The multi-head self-attention mechanism in the transformer encoder is used to integrate and process the global information of the feature map. The self-attention mechanism captures the long-distance dependencies and complex background information of the colonies in the feature map by calculating the correlation between each element in the feature map.

[0080] (4) Query and target matching: The DETR colony detection model introduces a fixed number of query vectors, which interact with the features in the feature map through the transformer encoder. The DETR colony detection model establishes a connection between these query vectors and the feature map, so that each query vector corresponds to an object detection result, thereby predicting the object detection result corresponding to each query vector. For the microbial colony detection scenario, the object refers to the microbial colony, and the object detection result refers to the location and category of the microbial colony. This design simplifies the traditional anchor mechanism (a traditional target detection mechanism), does not require pre-defined anchor boxes of different sizes and proportions, and directly performs target matching on a global scale.

[0081] (5) Loss calculation: The binary matching loss function is used. The goal of the binary matching loss function is to find the best match between the predicted box and the true box to minimize the loss between the prediction and the annotation, and to match and align the predicted result and the actual annotation. This method ensures the optimal matching relationship between the predicted box and the actual box by minimizing the matching loss, thereby improving the detection accuracy.

[0082] (6) Output results: The DETR colony detection model finally outputs a set of prediction boxes and their corresponding prediction category labels. The DETR colony detection model can accurately identify and locate microbial colonies in complex backgrounds, improving detection efficiency and accuracy.

[0083] This embodiment can learn from a large amount of sample data through the DETR colony detection model, effectively identify and classify microbial colonies of different forms, which not only improves the speed of detecting and identifying microbial colonies, but also improves the consistency and reliability of the detection results by reducing human intervention. In addition, the DETR colony detection model can establish global dependencies between different regions of the image through the self-attention mechanism, thereby effectively capturing microbial colonies of various sizes and shapes, thereby improving the detection accuracy of microbial colonies. Furthermore, the DETR colony detection model can significantly enhance the ability to capture detailed features in microbial identification, and can also effectively integrate local detail information and global context information, thereby further improving the recognition accuracy of microorganisms in complex backgrounds. In addition, this embodiment is based on the global feature calculation and end-to-end detection process of the DETR colony detection model, which has higher recognition accuracy and faster processing speed, improves the detection accuracy and efficiency of microbial colonies, reduces the need for manual operation and possible errors, and realizes the automation and optimization of the microbial detection process, which is particularly important for meeting the needs of efficient and accurate microbial colony detection in clinical and laboratory environments.

[0084] Example 2

[0085] like Figure 3 As shown, this embodiment provides an application method of a DETR colony detection model. The application method of the DETR colony detection model is actually a method for actually applying the trained DETR colony detection model after the DETR colony detection model is trained using the training method of Example 1, and specifically includes the following steps:

[0086] Step S101, collecting a target culture dish image, wherein the target culture dish image is a culture dish image to be detected without a real frame and a real category label.

[0087] Step S102: input the target culture dish image into the DETR colony detection model to obtain the prediction box and prediction category label corresponding to the target culture dish image; the DETR colony detection model is a model trained according to the training method of the DETR colony detection model described in Example 1.

[0088] The overall implementation process of microbial colony detection in this embodiment is as follows:

[0089] (1) Image data acquisition: Use a time-lapse camera to capture high-definition images of the culture dish during the temperature-controlled culture process.

[0090] (2) Image data annotation: Professional technicians annotate the microbial colony categories in the culture dish images. The annotation process is based on self-developed software. The rectangular coordinates of various positive sample colonies are annotated on the original culture dish images. Finally, the real box that encircles the location of the microbial colony and the real category label that represents the real category of the microbial colony are annotated on each culture dish image, thereby ensuring the accuracy and reliability of the training data.

[0091] (3) Build a deep learning model: Based on the DETR network architecture, build a DETR colony detection model and use the self-attention mechanism to optimize the detection and recognition of colonies in culture dish images.

[0092] (4) DETR colony detection model training and verification: The DETR colony detection model built in (3) is trained using the image data with the true box and true category label annotated in (2). The recognition rate and accuracy of the DETR colony detection model can then be tested on an independent verification dataset.

[0093] (5) System integration and implementation: Integrate temperature-controlled culture equipment and shooting equipment to form an integrated microbial colony detection and identification system, in which the shooting equipment can use a timer camera to collect images of the culture dish, and the temperature-controlled culture equipment is used to control the temperature of the culture dish. The trained inference model software can be installed in the industrial computer of the temperature-controlled culture equipment, and an API interface is provided. After the control software of the temperature-controlled culture equipment controls the timer camera to take pictures of the culture dish, the API interface is called to return the identification structure of the DETR colony detection model, and the relevant data is provided to downstream applications to ensure the stability of the culture process and the image quality, and realize the full automation of the system.

[0094] (6) Identification result determination: Generally, all artificial intelligence systems set a threshold for the result probability during reasoning. In this embodiment, when the probability of the DETR colony detection model identifying a microbial colony exceeds a preset threshold, the system automatically outputs the identification result, including a prediction box that delineates the location of the microbial colony and a prediction category label that characterizes the category of the microbial colony.

[0095] Example 3

[0096] like Figure 4 As shown, this embodiment provides a training system for a DETR colony detection model. The training system for the DETR colony detection model is actually a system corresponding to the training method of Example 1. The training system specifically includes the following functional modules:

[0097] The culture dish image acquisition module M1 is used to acquire a number of culture dish images; the culture dish image includes a culture dish and a microbial colony cultured in the culture dish, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony.

[0098] The loss function loss value determination module M2 is used to input several of the culture dish images into the DETR colony detection model respectively, and determine the loss value of the loss function according to the prediction box and prediction category label output by the DETR colony detection model and the corresponding real box and the real category label; the DETR colony detection model is a model established by the DETR network based on the self-attention mechanism for predicting the category and position of the microbial colony; the prediction box is used to circle the position of the microbial colony identified by the DETR colony detection model on the culture dish image, and the prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model.

[0099] In this embodiment, the loss function is a bipartite matching loss function.

[0100] The training module M3 is used to optimize the parameters of the DETR colony detection model according to the loss value of the loss function to obtain a trained DETR colony detection model.

[0101] In this embodiment, the culture dish image acquisition module M1 specifically includes:

[0102] The culture unit M11 is used to culture different types of microbial colonies using a plurality of culture dishes, and to control the temperature of each culture dish using a temperature control culture device.

[0103] The original culture dish image acquisition unit M12 is used to use a timing camera to take timed photos of each of the culture dishes to obtain a plurality of original culture dish images.

[0104] The labeling unit M13 is used to label the real category and real position of the microbial colony in the original culture dish image respectively to obtain a plurality of the culture dish images.

[0105] In this embodiment, the DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder and multiple prediction heads in sequence; wherein the backbone network is used to extract features from the culture dish image to obtain a feature map; the transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feedforward network, the transformer encoder is used to perform position encoding on the feature map, and capture the global dependencies of each part of the feature map through the multi-head self-attention mechanism to generate a final feature map including the semantic information and spatial position information of the culture dish image; the transformer decoder is used to decode the final feature map; the prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

[0106] Example 4

[0107] like Figure 5 As shown, this embodiment provides an application system of a DETR colony detection model. The application system of the DETR colony detection model is actually a system corresponding to the application method of Example 2. The application system specifically includes the following functional modules:

[0108] The target culture dish image acquisition module N1 is used to acquire the target culture dish image, wherein the target culture dish image is the culture dish image to be detected without a real frame and a real category label.

[0109] The colony detection module N2 is used to input the target culture dish image into the DETR colony detection model to obtain the prediction box and prediction category label corresponding to the target culture dish image. The DETR colony detection model is a model trained according to the training method of the DETR colony detection model described in Example 1.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A training method for a DETR colony detection model, characterized in that: The training method of the DETR colony detection model includes: Collecting a number of culture dish images; the culture dish images include culture dishes and microbial colonies cultured in the culture dishes, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony; Input several culture dish images into the DETR colony detection model respectively, and determine the loss value of the loss function according to the prediction box and the prediction category label output by the DETR colony detection model and the corresponding true box and the true category label; the DETR colony detection model is a model established by the DETR network based on the self-attention mechanism for predicting the category and position of the microbial colony; the prediction box is used to circle the position of the microbial colony identified by the DETR colony detection model on the culture dish image, and the prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model; The parameters of the DETR colony detection model are optimized according to the loss value of the loss function to obtain a trained DETR colony detection model.

2. The training method of the DETR colony detection model according to claim 1, characterized in that: Collect several images of the culture dishes, including: Using a plurality of the culture dishes to culture different types of microbial colonies respectively, and using a temperature control culture device to control the temperature of each of the culture dishes; Using a timing camera to take timed photos of each of the culture dishes to obtain a number of original culture dish images; The real categories and real positions of the microbial colonies in the original culture dish images are respectively marked to obtain a plurality of the culture dish images.

3. The training method of the DETR colony detection model according to claim 1, characterized in that: The DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder and multiple prediction heads in sequence; wherein the backbone network is used to extract features from the culture dish image to obtain a feature map; the transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feedforward network, the transformer encoder is used to perform position encoding on the feature map, capture the global dependencies of each part of the feature map through the multi-head self-attention mechanism, and generate a final feature map including the semantic information and spatial position information of the culture dish image; the transformer decoder is used to decode the final feature map; the prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

4. The training method of the DETR colony detection model according to claim 1, characterized in that: The loss function is a bipartite matching loss function.

5. An application method of a DETR colony detection model, characterized in that: The application method of the DETR colony detection model includes: Collecting a target culture dish image; the target culture dish image is a culture dish image to be detected without a real frame and a real category label; The target culture dish image is input into the DETR colony detection model to obtain the prediction box and prediction category label corresponding to the target culture dish image; the DETR colony detection model is a model trained according to the training method of the DETR colony detection model according to claim 1.

6. A training system for a DETR colony detection model, characterized in that: The training system of the DETR colony detection model includes: A culture dish image acquisition module, used to acquire a number of culture dish images; the culture dish images include culture dishes and microbial colonies cultured in the culture dishes, and each of the culture dish images is correspondingly annotated with a real frame and a real category label, the real frame is used to circle the real position of the microbial colony on the culture dish image, and the real category label is used to indicate the real category of the microbial colony; A loss function loss value determination module, used to input several culture dish images into the DETR colony detection model respectively, and determine the loss value of the loss function according to the prediction box and prediction category label output by the DETR colony detection model and the corresponding true box and the true category label; the DETR colony detection model is a model established by the DETR network based on the self-attention mechanism for predicting the category and position of the microbial colony; the prediction box is used to circle the position of the microbial colony identified by the DETR colony detection model on the culture dish image, and the prediction category label is used to indicate the category of the microbial colony identified by the DETR colony detection model; The training module is used to optimize the parameters of the DETR colony detection model according to the loss value of the loss function to obtain a trained DETR colony detection model.

7. The training system of the DETR colony detection model according to claim 6, characterized in that: The culture dish image acquisition module comprises: A culture unit, used to culture different types of microbial colonies using a plurality of culture dishes, and to control the temperature of each culture dish using a temperature control culture device; An original culture dish image acquisition unit is used to use a timing camera to take timed photos of each of the culture dishes to obtain a plurality of original culture dish images; The labeling unit is used to label the real category and real position of the microbial colony in the original culture dish image respectively to obtain a plurality of the culture dish images.

8. The training system of the DETR colony detection model according to claim 6, characterized in that: The DETR colony detection model includes a backbone network, a transformer encoder, a transformer decoder and multiple prediction heads in sequence; wherein the backbone network is used to extract features from the culture dish image to obtain a feature map; the transformer encoder and the transformer decoder both include multiple layer structures, each of which includes a multi-head self-attention mechanism and a feedforward network, the transformer encoder is used to perform position encoding on the feature map, capture the global dependencies of each part of the feature map through the multi-head self-attention mechanism, and generate a final feature map including the semantic information and spatial position information of the culture dish image; the transformer decoder is used to decode the final feature map; the prediction head is used to generate and output a prediction box and a predicted category label based on the decoded final feature map.

9. The training system of the DETR colony detection model according to claim 6, characterized in that: The loss function is a bipartite matching loss function.

10. An application system of a DETR colony detection model, characterized in that: The application system of the DETR colony detection model includes: A target culture dish image acquisition module is used to acquire a target culture dish image; the target culture dish image is an image of a culture dish to be detected without a real frame and a real category label; A colony detection module is used to input the target culture dish image into a DETR colony detection model to obtain a prediction box and a prediction category label corresponding to the target culture dish image; the DETR colony detection model is a model trained according to the training method of the DETR colony detection model according to claim 1.

Images