A general method and device for constructing a bacterial detection network model, and a storage medium

Abstract

The present invention provides a general method and device for constructing a bacterial detection network model, and a storage medium. In the STE pre-training stage, by fusing explicit temporal encoding with image information of corresponding growth stages, the spatio-temporal context across growth stages is encoded, reducing false negatives caused by adhesion clusters. In the DG-MoE migration stage, dual-grained experts are used to extract global features and local features respectively, so as to take into account the detection of tiny bacteria and normal bacteria. Among them, the tiny object expert focuses on extracting local texture features to enhance position distribution information; the normal object expert focuses on extracting global morphological features to enhance boundary information. The dual-grained experts are used to solve the problems of morphological differences between tiny bacteria and normal bacteria and misdetection of impurities with similar morphology. The present invention is model-independent and can be compatible with existing object detection algorithms, thereby achieving a consistent performance improvement.

Description

Technical Field

[0001] The present invention relates to the technical field of biotechnology, and particularly relates to a method and device for constructing a general bacterial detection network model, and a storage medium. Background Art

[0002] Microscopic time-lapse imaging technology has been widely used in the field of microbial analysis such as bacteria because it can provide dynamic information about the behavior of microorganisms such as bacteria, such as revealing the regulatory mechanisms of each stage of the cell cycle, studying the formation process of biofilms, and revealing the interaction between microorganisms and hosts. The analysis of time-lapse image sequences involves other processing processes such as the tracking and counting of microorganisms such as bacteria. These processes usually require a large amount of manual operations. Especially when the bacterial morphology is complex or the image quality is poor, traditional algorithms may not be able to accurately identify the bacterial boundaries. Manual operations are not only time-consuming and laborious, but also have poor consistency and cannot meet the requirements of high throughput in clinical practice. Therefore, researchers have proposed an automatic bacterial detection method based on a single growth stage, which has significantly improved the accuracy and efficiency of clinical microbiological analysis.

[0003] The purpose of bacterial detection is to detect the bacterial information in a single image, including position and size information. In recent years, there have also been many related works dedicated to bacterial detection, which can be mainly divided into two categories: based on artificial features and based on deep learning. The methods based on artificial features usually manually design and extract relevant features according to expert knowledge, and then use machine learning methods to detect bacteria based on the extracted artificial features. For example: in existing research, extracting basic information of patients, types of specimen carriers, etc. are incorporated into the artificial feature set to achieve rapid detection of ciprofloxacin resistance. In existing research, edge detection technology is also used to achieve automatic colony counting. In existing research, there is also the use of a feedback-based watershed algorithm to assist multi-threshold segmentation for robust automatic colony-forming unit (CFU) / cell analysis. These methods are limited by the representation ability of artificial features, and their robustness and generalization are still restricted. Therefore, some researchers are dedicated to the research of bacterial detection methods based on deep learning, and use deep learning methods to automatically extract features to achieve bacterial detection. For example: in existing research, the YOLOv4 deep learning framework for object detection is used for cell detection in bright-field images. In existing research, EfficientNet-Transformer is also used to classify and quantify bacteria. The above methods based on deep learning have greatly improved the accuracy of bacterial detection, but almost all of these methods are designed for natural images, and bacterial images introduce some unique challenges, such as Figure 1 as shown Figure 1The 0-H to 3-H in it represent different growth stages. The rectangular boxes represent bacteria, the circles represent impurities, different colors represent different instances, and the arrows represent the corresponding relationships of the same instance): (1) There are significant morphological differences between tiny bacteria and normal bacteria, and most detectors cannot handle both; (2) The morphology of some impurities is highly similar to that of bacteria; (3) Bacteria will form indistinguishable adhesion clusters as they grow.

[0004] Therefore, the existing bacteria detection methods have the following disadvantages:

[0005] Most of the existing bacteria detection methods are designed for natural images, do not consider the unique challenges of bacteria images, and lack the alignment of spatio-temporal information across growth stages.

[0006] The existing bacteria detection methods use the same detector for bacteria of different scales, but the morphological differences of bacteria of different scales are large, and it is impossible to handle the detection of bacteria of different scales.

[0007] The MoE method is widely used in natural images, but it is still blank in the field of bacteria detection. And the expert structure of the current MoE method is not specifically designed for downstream tasks, which will lead to differences in migration. Summary of the Invention

[0008] To achieve the above objects and other advantages of the present invention, the first object of the present invention is to provide a general method for constructing a bacteria detection network model, including the following steps:

[0009] Obtain bacteria images with annotations and verifications;

[0010] Train a bacteria detection network model based on cross-growth stage spatio-temporal context and dual-grained mixture of experts through the bacteria images:

[0011] In the STE pre-training stage, encode the spatio-temporal context across growth stages by fusing explicit temporal encoding with image information of the corresponding growth stage;

[0012] In the DG-MoE migration stage, extract global features and local features respectively through dual-grained experts to handle the detection of tiny bacteria and normal bacteria.

[0013] Further, before the step of obtaining bacteria images with annotations and verifications, it further includes:

[0014] Screen the prepared bacteria images through label checking to remove images without bacteria, and use the remaining images for training and testing;

[0015] Identify the target size of the bacteria images through the standard definition of the MS COCO dataset;

[0016] Collect auxiliary datasets For pre-training auxiliary samples across growth stages, each group of samples contains multiple registered time-lapse images, each representing a growth stage, and the multiple registered time-lapse images represent a complete bacterial growth cycle;

[0017] All images are labeled and verified.

[0018] Furthermore, in the STE pre-training stage, the steps of encoding spatio-temporal context across growth stages by fusing explicit temporal encoding with image information of corresponding growth stages include:

[0019] Using multiple images of consecutive growth stages as joint input to detect bacteria in the target growth stage, and the specific process is shown as follows:

[0020] ;

[0021] where STE represents the spatio-temporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacterial detection;

[0022] ;

[0023] where represents the image of the i-th growth stage as input, represents concatenation along the channel dimension, represents 2D convolution, represents depthwise separable convolution with a kernel size of 31x31, represents the temporal encoding corresponding to the i-th growth stage, as shown in the following formula:

[0024] ;

[0025] where t and c represent the growth stage and the channel respectively.

[0026] Furthermore, in the DG-MoE migration stage, the steps of extracting global features and local features respectively by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria include:

[0027] Extracting local texture features through the tiny object expert to enhance the position distribution information;

[0028] Extracting global morphological features through the normal object expert to enhance the boundary information;

[0029] Processing the detection results obtained by the two experts through a soft router.

[0030] Furthermore, the overall process of the tiny object expert is shown as follows:

[0031] ;

[0032] Among them, is a detector for bacteria detection, and NWD represents the NWD metric for measuring bacteria at the micro level, as shown in formula (5), as shown in formula (6):

[0033] ;

[0034] Among them, respectively represent the detection box a and the detection box b, represents the L2 norm operation, C represents the modulation coefficient, respectively represent the center point x coordinates of the detection box a, the center point y coordinates, width, and height; respectively represent the center point x coordinates of the detection box b, the center point y coordinates, width, and height;

[0035] ;

[0036] Among them, after downsampling, 4 different-scale features are obtained, namely , represents the feature of the i-th scale, is the corresponding fused feature, is the feature fused with the feature, represents the upsampling operation with a factor of 2, represents the 2D convolution operation with a kernel size of 3x3.

[0037] Furthermore, the overall process of the normal target expert is as shown in the following formula:

[0038] ,

[0039] Among them, is a detector for bacteria detection, and IoU represents the IoU metric for measuring bacteria at the normal level, as shown in formula (8), as shown in formula (9);

[0040] .

[0041] Furthermore, when the input image is and respectively obtain the corresponding detection results and The detection results obtained by processing the two experts through the soft router are specifically as follows:

[0042] ;

[0043] where is the union of the detection results of the two experts, is one of denotes the router, which is used to score the prediction boxes, S is the detection result after selection, is S one of , and k candidate bounding boxes are routed and selected based on the score given by the router, that is .

[0044] Furthermore, the loss function adopted by the bacterial detection network model based on cross-growth-stage spatio-temporal context and dual-granularity mixture experts is as follows:

[0045] ;

[0046] where represents the L1 loss, represents the binary cross-entropy loss, P and respectively represent the categories of the targets of the true box and the prediction box, and respectively represent the coordinate information of the true box and the prediction box, respectively represent the coordinates of the center point x coordinates, the center point y coordinates, width, height.

[0047] The second object of the present invention is to provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0048] The third object of the present invention is to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] The present invention provides a general bacterial detection framework, which is specifically designed for the unique challenges of bacterial drug sensitivity images and is better applicable to bacterial detection based on a single image.

[0051] In the STE pre-training stage of the present invention, by fusing explicit temporal encoding with image information of corresponding growth stages, the spatio-temporal context across growth stages is encoded, reducing false negatives caused by adhesion clusters; in the DG-MoE migration stage of the present invention, global features and local features are respectively extracted by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria. Among them, the tiny-object expert focuses on extracting local texture features to enhance position distribution information; the normal-object expert focuses on extracting global morphological features to enhance boundary information. The dual-granularity experts are used to solve the morphological difference problem between tiny bacteria and normal bacteria and the problem of misdetection of impurities with similar morphology.

[0052] The present invention is model-independent and can be compatible with existing object detection algorithms, thereby achieving a consistent performance improvement.

[0053] The present invention has been actually tested on a clinical dataset, and the results have reached the state-of-the-art performance. It provides an accurate, fast, and simple method for bacterial detection for grass-roots experiments and the clinical front line.

[0054] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and implement it in accordance with the content of the specification, the following will detail the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation manners of the present invention are given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0056] Figure 1 It is a schematic diagram of a sequence of bacterial images for a complete growth cycle;

[0057] Figure 2 It is a general method flow for constructing a bacterial detection network model based on spatio-temporal context across growth stages and dual-granularity hybrid experts Figure 1 ;

[0058] Figure 3 It is a flowchart for training a bacterial detection network model based on spatio-temporal context across growth stages and dual-granularity hybrid experts using the bacterial images;

[0059] Figure 4 It is a general method flow for constructing a bacterial detection network model based on spatio-temporal context across growth stages and dual-granularity hybrid experts Figure 2 ;

[0060] Figure 5 It is a flowchart of the STE pre-training;

[0061] Figure 6 For the DG-MoE migration process Figure 1 ;

[0062] Figure 7 For the flowchart of bacterial image processing;

[0063] Figure 8 For the DG-MoE migration process Figure 2 ;

[0064] Figure 9 For the schematic diagram of a computer device;

[0065] Figure 10 For the schematic diagram of a computer-readable storage medium. Detailed implementation manners

[0066] Next, in combination with the accompanying drawings and specific implementation manners, the present invention will be further described. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be noted that, on the premise of no conflict, the following described embodiments or various technical features can be combined arbitrarily to form new embodiments.

[0067] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0068] In this application, the accompanying drawing numbers are only used to distinguish each step in the solution and are not used to limit the execution order of each step. The specific execution order shall be subject to the description in the specification.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0070] The present invention proposes a general bacteria detection framework based on contrast learning with cross-growth stage information, which performs excellently in dealing with bacteria detection based on AST images. Its complete technical solution is as follows. Embodiment 1

[0071] A method for constructing a general bacteria detection network model, as Figures 2 - 4 shown, includes the following steps:

[0072] S100. Obtain bacteria images with annotations and validations;

[0073] S200. Train the bacterial detection network model based on spatio-temporal context across growth stages and dual-grained mixture of experts using the bacterial images:

[0074] S210. In the STE pre-training stage, encode the spatio-temporal context across growth stages by fusing explicit temporal encoding with image information corresponding to the growth stages;

[0075] S220. In the DG-MoE transfer stage, extract global features and local features respectively by dual-grained experts to take into account the detection of tiny bacteria and normal bacteria.

[0076] In some embodiments, as Figure 7 shown, before the step of obtaining the labeled and verified bacterial images, it further includes:

[0077] S110. Screen the prepared bacterial images through label checking to remove images without bacteria, and use the remaining images for training and testing;

[0078] The resolution of the bacterial images prepared in this embodiment is between 3842x2198 and 5453x3599, approximately equal to 3000x4000 pixels. A total of 1220 images were collected, and after screening (label checking, removal of images without bacteria), 1049 images remained for training and testing the method of this embodiment.

[0079] S120. Identify the target sizes of the bacterial images through the standard definition of the MS COCO dataset;

[0080] The target size distribution is shown in Table 1, where the definitions of tiny, medium, and large come from the standard definition of the MS COCO dataset.

[0081] Table 1 Target size distribution

[0082]

[0083] S130. Collect an auxiliary dataset for pre-training 258 groups of auxiliary samples across growth stages. Each group of samples contains multiple registered time-lapse images, each representing a growth stage, and multiple registered time-lapse images represent a complete bacterial growth cycle;

[0084] In this embodiment, each group of samples contains 4 registered time-lapse images, each representing a growth stage (0-H to 3-H), and 4 images represent a complete bacterial growth cycle.

[0085] S140. Label and verify all images. For example, all the above images are manually labeled and verified by relevant experts.

[0086] The architecture of the bacterial detection framework proposed in this embodiment is as follows Figures 4 - 6 and includes two training stages, namely the Spatial-temporal Encode (STE) pre-training stage and the Dual-granularity Mixture-of-experts (DG-MoE) transfer stage.

[0087] Furthermore, as shown in Figure 5 , in the STE pre-training stage, the steps of encoding the spatio-temporal context across growth stages by fusing explicit temporal encoding and image information of the corresponding growth stages include:

[0088] Using multiple consecutive growth stage images as the joint input to detect bacteria in the target growth stage. For example, using four consecutive growth stage images as the joint input to detect bacteria in the target growth stage, where the target growth stage is selected from the joint input. The specific process is as shown in the following formula:

[0089] ;

[0090] where STE represents the spatio-temporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacterial detection;

[0091] ;

[0092] where represents the image of the i-th growth stage of the input, represents concatenation along the channel dimension, represents 2D convolution, represents depthwise separable convolution with a kernel size of 31x31, represents the temporal encoding corresponding to the i-th growth stage, as shown in the following formula:

[0093] ;

[0094] where t and c represent the growth stage and the channel respectively.

[0095] Furthermore, as shown in Figure 6 and Figure 8 , in the DG-MoE transfer stage, the steps of extracting global features and local features by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria include:

[0096] S221. Extract local texture features through the tiny object expert to enhance the position distribution information;

[0097] S222. Through the normal object expert Extract global morphological features and enhance boundary information;

[0098] S223. Process the detection results obtained by the two experts through a soft router.

[0099] That is, the DG-MoE fine-tuning stage: DG-MoE mainly consists of two experts, namely the small target expert and the normal target expert and a soft router, which will be introduced separately below.

[0100] Furthermore, the overall process of the small target expert is shown as follows:

[0101] ;

[0102] Among them, is a detector for bacteria detection (FasterRCNN is used in this embodiment), and NWD represents the NWD metric for measuring bacteria at the small level, as shown in formula (5), as shown in formula (6):

[0103] ;

[0104] Among them, respectively represent the detection box a and the detection box b, represents the two-norm operation, C represents the modulation coefficient, respectively represent the center point x coordinates of the detection box a, the center point y coordinates, width, and height; respectively represent the center point x coordinates of the detection box b, the center point y coordinates, width, and height; ;

[0105] Among them, after downsampling, 4 different-scale features are obtained, namely , represents the feature of the i-th scale, is the corresponding fused feature, is the feature fused with the feature, represents the upsampling operation with a factor of 2, represents the 2D convolution operation with a kernel size of 3x3.

[0106] Furthermore, the overall process of the normal target expert is shown as follows:

[0107] ;

[0108] Among them, is a detector for bacterial detection (Faster RCNN is used in this embodiment), and IoU represents the IoU metric for measuring bacteria at the normal level, as shown in formula (8), as shown in formula (9).

[0109] ;

[0110] .

[0111] Furthermore, when the input image is input, through and , the corresponding detection results and are obtained respectively. The detection results obtained by processing the two experts through the soft router are specifically as follows:

[0112] ;

[0113] Among them, is the union of the detection results of the two experts, is one of , represents the router, which is used to score the prediction boxes, S is the detection result after selection, is one of S . In this embodiment, the confidence score is used to score it. Based on the score given by the router, k candidate bounding boxes are routed and selected, that is, , and then the final detection result is obtained.

[0114] Furthermore, the loss function adopted by the bacterial detection network model based on cross-growth-stage spatio-temporal context and dual-granularity hybrid experts is as follows:

[0115] ;

[0116] Among them, represents the L1 loss, represents the binary cross-entropy loss, P and represent the categories of the targets of the true box and the predicted box respectively, and represent the coordinate information of the true box and the predicted box respectively, represent the coordinates of the center point x , the coordinates of the center point y , width, and height.

[0117] In some embodiments, data preparation: Both the training data and the test data are from actual clinical collections. A total of four CLSI standard strains are used in the experiment, including 2 Gram-positive bacteria (Staphylococcus aureus ATCC29213, Enterococcus faecalis ATCC 29212) and 2 Gram-negative bacteria (Pseudomonas aeruginosa ATCC 27853, Escherichia coli ATCC 25922). All strains are purchased from Beijing Baocang Biotechnology Co., Ltd. (Beijing, China). The bacteria are cultured overnight in a shaker containing Mueller Hinton medium (MHB) under the conditions of 200 rpm and 37°C. Subsequently, their concentration is diluted to 10^6 CFU / ml and added to the flat agarose wells of a 96-well plate. A self-made up-view microscope is used in the experiment, and combined with a heating system for bacterial imaging. The microscope is equipped with an MPlanFLN 20 × / 0.30 lens (Olympus, Japan). The size of the imaging area is 800µm × 800µm. Through a self-made autofocus and positioning system, time-lapse images in the same area are captured at an interval of 1 hour, with one photo taken each time, and the shooting time is within 1 second.

[0118] Experimental setup: The optimizer used for training is the SGD optimizer, and the parameter settings are as follows: learning rate lr = 0.02, momentum = 0.9, weight decay = 0.0001. After training is completed, the model with the best mAP.50 is saved.

[0119] Testing: The detector trained successively through two stages of STE and DG-MoE is directly used for inference, and the image to be tested is directly input into the detector to obtain the prediction result.

[0120] The comparison results of this method with other state-of-the-art detectors are shown in Table 2.

[0121] Table 2 Comparison results of different methods

[0122]

[0123] The results show that the method of the present invention can outperform most SOTA detectors, including single-stage, two-stage, and end-to-end ones, when combined with Faster RCNN only, and can match the performance of the state-of-the-art detector, namely RT-DETR, indicating the outstanding advantages of this method.

[0124] The present invention provides a general three-stage contrastive learning bacterial detection framework with cross-growth stage information, including a Spatio-Temporal Encoding (STE) pre-training stage and a Dual-Granularity Mixture of Experts (DG-MoE) transfer stage. Among them, in the STE pre-training stage, by fusing explicit temporal encoding with the image information of the corresponding growth stage, the spatio-temporal context across growth stages is encoded to reduce false negatives caused by adhesion clusters; in the DG-MoE transfer stage, global features and local features are respectively extracted by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria. Among them, the tiny object expert focuses on extracting local texture features to enhance the position distribution information; the normal object expert focuses on extracting global morphological features to enhance the boundary information. The morphological differences between tiny bacteria and normal bacteria and the misdetection problem of impurities with similar morphology are solved by dual-granularity experts. Example 2

[0125] A computer device 300, as Figure 9 shown, includes a memory 310, a processor 320, and a computer program 330 stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a general method for constructing a bacterial detection network model. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here. Example 3

[0126] A computer-readable storage medium, as Figure 10 shown, stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of a general method for constructing a bacterial detection network model. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here.

[0127] The number of devices and the processing scale described here are used to simplify the description of the present invention. Applications, modifications, and variations of the present invention will be obvious to those skilled in the art.

[0128] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the field, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to specific details and the examples shown and described here.

[0129] The device, computer equipment, non-volatile computer storage medium, and method provided by the embodiments of this specification are corresponding. Therefore, the device, computer equipment, and non-volatile computer storage medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding device, computer equipment, and non-volatile computer storage medium will not be elaborated here.

[0130] Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the method steps can be logically programmed to enable the controller to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same function. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software units for implementing the method and the structures within the hardware component.

[0131] The systems, devices, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0132] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, system, or computer program product. Therefore, the embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0133] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0137] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program units may be located in local and remote computer storage media, including storage devices.

[0138] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0139] The above is only described for the embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A general method for constructing a bacterial detection network model, characterized in that, It includes the following steps: Obtain the bacteria images with annotations and verifications; Train the bacteria detection network model based on cross-growth-stage spatio-temporal context and dual-granularity mixture of experts through the said bacteria images: In the STE pre-training stage, encode the spatio-temporal context across growth stages by fusing explicit temporal encoding and image information of corresponding growth stages; In the DG-MoE transfer stage, extract global features and local features respectively by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria; The step of encoding the spatio-temporal context across growth stages by fusing explicit temporal encoding and image information of corresponding growth stages in the STE pre-training stage includes: Use multiple consecutive growth-stage images as joint inputs to detect bacteria in the target growth stage, and the specific process is shown as follows: ； Among them, STE represents the spatio-temporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacteria detection; ； Among them, represents the image of the i-th growth stage of the input, represents concatenation along the channel dimension, represents 2D convolution, represents depthwise separable convolution with a kernel size of 31x31, represents the temporal encoding corresponding to the i-th growth stage, as shown in the following formula: ； Among them, t and c represent the growth stage and the channel respectively; The step of extracting global features and local features respectively by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria in the DG-MoE transfer stage includes: By micro-target experts Extract local texture features and enhance position distribution information; Through normal target experts Extract global morphological features and enhance boundary information; Process the detection results obtained by the two experts through a soft router.

2. The general method for constructing a bacterial detection network model according to claim 1, wherein Before the step of obtaining the bacteria images with annotations and verifications, it also includes: Screen the prepared bacteria images through label checking to remove the images without bacteria, and use the remaining images for training and testing; Identify the target size of the said bacteria images according to the standard definition of the MS COCO dataset; Collect auxiliary data sets For pre-training auxiliary samples across growth stages, each group of samples contains multiple registered time-lapse images, each representing a growth stage, and the multiple registered time-lapse images represent a complete bacterial growth cycle; Perform annotations and verifications on all images.

3. A general method for constructing a bacterial detection network model according to claim 1, characterized in that The micro-target expert The overall process is shown as follows: ； Among them, is a detector for bacteria detection. NWD represents the NWD metric for measuring bacteria at a microscopic level, as shown in formula (5), as shown in formula (6): ； Among them, respectively represent the detection box a and the detection box b, represents the two-norm operation, C represents the modulation coefficient, respectively represent the center point x coordinates of the detection box a, the center point y coordinates, width, and height; respectively represent the center point x coordinates of the detection box b, the center point y coordinates, width, and height; ； Among them, 4 features of different scales are obtained after downsampling, namely , represents the feature of the i-th scale, is the corresponding fused feature, is the feature fused with features, represents the upsampling operation with a factor of 2, represents a 2D convolution operation with a kernel size of 3x3.

4. The general bacterial detection network model construction method according to claim 3, characterized in that The normal target expert The overall process is as shown in the following formula: ； Among them, is a detector for bacteria detection. IoU represents the IoU metric used to measure bacteria at the normal level, as shown in formula (8), as shown in formula (9); ； 。 5. A general method for constructing a bacterial detection network model according to claim 4, characterized in that When the input image is processed, corresponding detection results and are obtained respectively and . The detection results obtained by processing the two experts through the soft router are specifically as follows: ； Among them, is the union of the detection results of two experts, is one of them, representing a router, used to score the prediction boxes, S is the detection result after selection, is S one of them, scored using a confidence score; based on the score given by the router, k candidate bounding boxes are routed and selected, that is , and then the final detection result is obtained.

6. The general method for constructing a bacterial detection network model according to claim 5, wherein: The loss function adopted by the bacteria detection network model based on cross-growth-stage spatio-temporal context and dual-granularity mixture of experts is as follows: ； Among them, represents the L1 loss, represents the binary cross-entropy loss, P and represent the categories of the targets of the ground truth box and the predicted box respectively, and represent the coordinate information of the ground truth box and the predicted box respectively, represent the x coordinate of the center point, the y coordinate of the center point, width, and height.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the said computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the said computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images