Universal bacterium detection network model construction method and device, and storage medium

Abstract

The invention provides a universal bacterium detection network model construction method and device, and a storage medium, in an STE pre-training stage, through fusion of explicit time sequence coding and image information of a corresponding growth stage, a spatio-temporal context across the growth stage is coded, and false negative caused by adhesion clusters is reduced; in the DG-MoE migration stage, global features and local features are respectively extracted through dual-granularity experts, so that detection of both tiny bacteria and normal bacteria is considered. Wherein a tiny target expert focuses on extracting local texture features and enhancing position distribution information; normal target experts focus on extracting global morphological features and enhancing boundary information. The problem of morphological difference between tiny bacteria and normal bacteria and the problem of false detection of impurities with similar morphologies are solved through double-granularity experts. The method is irrelevant to a model and can be compatible with an existing target detection algorithm, so that the performance improvement of consistency is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of biotechnology, and particularly relates to a method and device for constructing a general bacterial detection network model, and a storage medium. Background Art

[0002] Due to the ability to provide dynamic information about the behavior of bacteria and other microorganisms, time-lapse microscopy has been widely used in the field of microbial analysis, such as revealing the regulatory mechanisms of each stage of the cell cycle, studying the formation process of biofilms, and revealing the interaction between microorganisms and hosts. The analysis of time-lapse image sequences involves other processing processes such as the tracking and counting of bacteria and other microorganisms. These processes usually require a large amount of manual operations. Especially when the bacterial morphology is complex or the image quality is poor, traditional algorithms may not be able to accurately identify the bacterial boundary. Manual operations are not only time-consuming and laborious, but also have poor consistency and cannot meet the requirements of high throughput in clinical practice. Therefore, researchers have proposed an automatic bacterial detection method based on a single growth stage, which has significantly improved the accuracy and efficiency of clinical microbial analysis.

[0003] The purpose of bacterial detection is to detect the bacterial information in a single image, including position and size information. In recent years, there have also been many related works dedicated to bacterial detection, which can be mainly divided into two categories: based on artificial features and based on deep learning. The methods based on artificial features usually manually design and extract relevant features according to expert knowledge, and then use machine learning methods to detect bacteria based on the extracted artificial features. For example, in existing research, basic information of patients, types of specimen carriers, etc. are included in the artificial feature set to achieve rapid detection of ciprofloxacin resistance. In existing research, edge detection technology is also used to achieve automatic colony counting. In existing research, a feedback-based watershed algorithm is also used to assist multi-threshold segmentation for robust automatic colony-forming unit (CFU) / cell analysis. These methods are limited by the representation ability of artificial features, and their robustness and generalization are still restricted. Therefore, some researchers are dedicated to the research of bacterial detection methods based on deep learning, and use deep learning methods to automatically extract features to achieve bacterial detection. For example, in existing research, the YOLOv4 deep learning framework for object detection is used for cell detection in bright-field images. In existing research, EfficientNet-Transformer is also used to classify and quantify bacteria. The above methods based on deep learning have greatly improved the accuracy of bacterial detection, but almost all of these methods are designed for natural images, and bacterial images introduce some unique challenges, such as Figure 1 as shown Figure 1The 0-H to 3-H in it represent different growth stages. The rectangular boxes represent bacteria, the circles represent impurities, different colors represent different instances, and the arrows represent the corresponding relationships of the same instance): (1) There are significant morphological differences between tiny bacteria and normal bacteria, and most detectors cannot handle both; (2) The morphology of some impurities is highly similar to that of bacteria; (3) Bacteria will form indistinguishable adhesion clusters as they grow.

[0004] Therefore, the existing bacteria detection methods have the following disadvantages: Most of the existing bacteria detection methods are designed for natural images, without considering the unique challenges of bacteria images and lacking the alignment of spatio-temporal information across growth stages.

[0005] The existing bacteria detection methods use the same detector for bacteria of different scales, but the morphological differences between bacteria of different scales are significant, and it is impossible to handle the detection of bacteria of different scales.

[0006] The MoE method is widely used in natural images, but it is still blank in the field of bacteria detection. Moreover, the expert structure of the current MoE method is not specifically designed for downstream tasks, which will lead to migration differences. Summary of the Invention

[0007] To achieve the above objects and other advantages of the present invention, the first object of the present invention is to provide a method for constructing a general bacteria detection network model, including the following steps: Obtain bacteria images with annotations and validations; Train a bacteria detection network model based on cross-growth stage spatio-temporal context and dual-granularity mixture of experts through the bacteria images: In the STE pre-training stage, encode the spatio-temporal context across growth stages by fusing explicit temporal encoding and image information of the corresponding growth stage; In the DG-MoE migration stage, extract global features and local features respectively through dual-granularity experts to handle the detection of tiny bacteria and normal bacteria.

[0008] Furthermore, before the step of obtaining bacteria images with annotations and validations, it also includes: Screen the prepared bacteria images through label checking to remove images without bacteria, and use the remaining images for training and testing; Identify the target size of the bacteria images through the standard definition of the MS COCO dataset; Collect auxiliary datasets For cross-growth stage pre-training auxiliary samples, each group of samples contains multiple registered time-lapse images, each representing a growth stage, and multiple registered time-lapse images represent a complete bacteria growth cycle; Annotate and validate all images.

[0009] Further, in the STE pre-training stage, the step of encoding spatio-temporal context across growth stages by fusing explicit temporal encoding with image information corresponding to the growth stages includes: Using multiple consecutive growth-stage images as joint inputs to detect bacteria in the target growth stage, and the specific process is shown as follows: ; where STE represents the spatio-temporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacteria detection; ; where, represents the image of the i-th growth stage as input, represents concatenation along the channel dimension, represents 2D convolution, represents depthwise separable convolution with a kernel size of 31x31, represents the temporal encoding corresponding to the i-th growth stage, as shown in the following formula: ; where t and c represent the growth stage and the channel respectively.

[0010] Further, in the DG-MoE migration stage, the step of respectively extracting global features and local features by dual-grained experts to take into account the detection of tiny bacteria and normal bacteria includes: Extracting local texture features through the tiny object expert to enhance the position distribution information; Extracting global morphological features through the normal object expert to enhance the boundary information; Processing the detection results obtained by the two experts through a soft router.

[0011] Further, the overall process of the tiny object expert is shown as follows: ; where, is the detector for bacteria detection, NWD represents the NWD metric for measuring tiny-level bacteria, as shown in formula (5), as shown in formula (6): ; where, respectively represent detection box a and detection box b, represents the L2 norm operation, C represents the modulation coefficient, respectively represent the center point of the detection box a x coordinates, center point y coordinates, width, height; respectively represent the center point of the detection box b x coordinates, center point y coordinates, width, height; ; Among them, after downsampling, 4 different-scale features are obtained, namely , represents the feature of the i-th scale, is the corresponding fused feature, is the feature fused with ; represents the upsampling operation with a factor of 2, represents the 2D convolution operation with a kernel size of 3x3.

[0012] Furthermore, the normal target expert The overall process is shown as follows: , wherein, is the detector for bacteria detection, IoU represents the IoU metric for measuring bacteria at the normal level, as shown in formula (8), as shown in formula (9); .

[0013] Furthermore, when the input image is input, through and the corresponding detection results and are obtained respectively. The detection results obtained by processing the two experts through the soft router are specifically as follows: ; wherein, is the union of the detection results of the two experts, is one of , represents the router, which is used to score the prediction boxes, S is the selected detection result, is one of S , and it is scored using the confidence score; based on the score given by the router, k candidate bounding boxes are routed and selected, that is , and then the final detection result is obtained.

[0014] Furthermore, the loss function adopted by the bacterial detection network model based on cross-growth-stage spatio-temporal context and dual-granularity mixture of experts is as follows: ; where represents the L1 loss, represents the binary cross-entropy loss, P and respectively represent the categories of the targets of the ground truth box and the predicted box, and respectively represent the coordinate information of the ground truth box and the predicted box, respectively represent the center point x coordinate, center point y coordinate, width, and height.

[0015] The second objective of the present invention is to provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0016] The third objective of the present invention is to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a general bacterial detection framework, which is specifically designed for the unique challenges of bacterial drug sensitivity images and is better applicable to bacterial detection based on a single image.

[0018] In the STE pre-training stage of the present invention, by fusing explicit temporal encoding with image information of corresponding growth stages, the spatio-temporal context across growth stages is encoded, reducing false negatives caused by adhesion clusters; in the DG-MoE migration stage of the present invention, global features and local features are respectively extracted by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria. Among them, the tiny target expert focuses on extracting local texture features to enhance position distribution information; the normal target expert focuses on extracting global morphological features to enhance boundary information. The morphological differences between tiny bacteria and normal bacteria and the problem of misdetection of impurities with similar morphology are solved by dual-granularity experts.

[0019] The present invention is model-independent and can be compatible with existing object detection algorithms, thereby achieving a consistent performance improvement.

[0020] The present invention has been actually tested on a clinical dataset, and the results have achieved state-of-the-art performance. It provides an accurate, fast, and simple bacterial detection method for grass-roots experiments and clinical frontlines.

[0021] The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly and be able to implement it according to the content of the description, the following takes the preferred embodiments of the present invention and combines with the accompanying drawings to describe in detail as follows. The specific implementation manners of the present invention are given in detail by the following embodiments and their accompanying drawings. Description of the Drawings

[0022] The drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 It is a schematic diagram of a sequence of images of bacteria for a complete growth cycle; Figure 2 It is a general method flow for constructing a bacteria detection network model based on spatio-temporal context across growth stages and dual-granularity mixture of experts Figure 1 ; Figure 3 It is a training flow chart of the bacteria detection network model based on spatio-temporal context across growth stages and dual-granularity mixture of experts through the said bacteria images; Figure 4 It is a general method flow for constructing a bacteria detection network model based on spatio-temporal context across growth stages and dual-granularity mixture of experts Figure 2 ; Figure 5 It is a pre-training flow chart of STE; Figure 6 It is a DG-MoE migration process Figure 1 ; Figure 7 It is a flow chart of bacteria image processing; Figure 8 It is a DG-MoE migration process Figure 2 ; Figure 9 It is a schematic diagram of a computer device; Figure 10 It is a schematic diagram of a computer-readable storage medium. Specific Embodiments

[0023] Next, in combination with the accompanying drawings and specific implementation manners, the present invention will be further described. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. It should be noted that, on the premise of no conflict, any combination of the following described embodiments or technical features can form a new embodiment.

[0024] All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0025] In this application, the attached drawing numbers are only used to distinguish each step in the solution and are not used to limit the execution order of each step. The specific execution order shall be subject to the description in the specification.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs. The terms used in the specification of this invention are only for the purpose of describing specific embodiments and are not intended to limit this invention.

[0027] The present invention proposes a general bacteria detection framework based on contrastive learning with cross-growth-stage information, which performs excellently in dealing with bacteria detection based on AST images. The complete technical solution is as follows. Example 1

[0028] A method for constructing a general bacteria detection network model, as Figures 2 - 4 shown, includes the following steps: S100. Obtain bacteria images with annotations and verifications; S200. Train a bacteria detection network model based on cross-growth-stage spatio-temporal context and dual-granularity mixture of experts through the bacteria images: S210. In the STE pre-training stage, encode the spatio-temporal context of cross-growth stages by fusing explicit temporal encoding and corresponding growth-stage image information; S220. In the DG-MoE migration stage, extract global features and local features respectively through dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria.

[0029] In some embodiments, as Figure 7 shown, before the step of obtaining bacteria images with annotations and verifications, it further includes: S110. Screen the prepared bacteria images through label checking to remove images without bacteria, and use the remaining images for training and testing; The resolution of the bacteria images prepared in this embodiment is between 3842x2198 and 5453x3599, approximately equal to 3000x4000 pixels. A total of 1220 images were collected, and after screening (label checking, removal of images without bacteria), a total of 1049 images remained for training and testing the method of this embodiment.

[0030] S120. Identify the target size of the bacteria images through the standard definition of the MS COCO dataset; The target size distribution is shown in Table 1, where the definitions of tiny, medium, and large come from the standard definition of the MS COCO dataset.

[0031] Table 1 Target Size Distribution

[0032] S130, Collect auxiliary data sets For pre-training 258 groups of auxiliary samples across growth stages, each group of samples contains multiple registered time-lapse images, each representing a growth stage, and multiple registered time-lapse images represent a complete bacterial growth cycle; In this embodiment, each group of samples contains 4 registered time-lapse images, each representing a growth stage (0-H to 3-H), and 4 images represent a complete bacterial growth cycle.

[0033] S140, Annotate and verify all images. For example, all the above images are manually annotated and verified by relevant experts.

[0034] The architecture of the bacterial detection framework proposed in this embodiment is as Figures 4 - 6 shown, and it includes two training stages, namely the Spatial-temporal Encode (STE) pre-training stage and the Dual-granularity Mixture-of-experts (DG-MoE) transfer stage.

[0035] Furthermore, as Figure 5 shown, in the STE pre-training stage, the steps of encoding the spatio-temporal context across growth stages by fusing explicit temporal encoding with image information of the corresponding growth stage include: Using multiple consecutive growth stage images as joint inputs to detect bacteria in the target growth stage. For example, using four consecutive growth stage images as joint inputs to detect bacteria in the target growth stage, where the target growth stage is selected from the joint inputs. The specific process is shown as follows: ; where STE represents the spatio-temporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacterial detection; ; where represents the image of the i-th growth stage of the input, represents concatenation along the channel dimension, represents 2D convolution, represents depthwise separable convolution with a kernel size of 31x31, represents the temporal encoding corresponding to the i-th growth stage, as shown in the following formula: ; where t and c represent the growth stage and the channel respectively.

[0036] Furthermore, as shown in Figure 6 and Figure 8 , in the DG-MoE migration stage, the steps of respectively extracting global features and local features through dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria include: S221. Extract local texture features through the tiny-object expert to enhance the position distribution information; S222. Extract global morphological features through the normal-object expert to enhance the boundary information; S223. Process the detection results obtained by the two experts through the soft router.

[0037] That is, the DG-MoE fine-tuning stage: DG-MoE mainly includes two experts, namely the tiny-object expert and the normal-object expert and a soft router, which will be introduced separately below.

[0038] Furthermore, the overall process of the tiny-object expert is shown as follows: ; where is a detector for bacteria detection (FasterRCNN is used in this embodiment), NWD represents the NWD metric for measuring tiny-level bacteria, as shown in formula (5) as shown in formula (6): ; where respectively represent detection box a and detection box b, represents the two-norm operation, C represents the modulation coefficient, respectively represent the center point x coordinates, center point y coordinates, width, and height of detection box a; respectively represent the center point x coordinates, center point y coordinates, width, and height of detection box b; ; where, after downsampling, 4 different-scale features are obtained, namely , represents the feature of the i-th scale, is the corresponding fused feature, is the feature fused with , represents the upsampling operation with a factor of 2 Represents a 2D convolution operation with a kernel size of 3x3.

[0039] Furthermore, the normal target expert The overall process is shown as follows: ; Among them, is a detector for bacteria detection (Faster RCNN is used in this embodiment), IoU represents the IoU metric for measuring bacteria at the normal level, as shown in formula (8), as shown in formula (9).

[0040] ; .

[0041] Furthermore, when the input image is, through and the corresponding detection results and are obtained respectively. The detection results obtained by processing the two experts through the soft router are specifically as follows: ; Among them, is the union of the detection results of the two experts, is one of the items in, represents the router, which is used to score the prediction boxes, S is the detection result after selection, is S one of the items in. In this embodiment, the confidence score is used to score it. Based on the score given by the router, k candidate bounding boxes are routed and selected, that is, , and then the final detection result is obtained.

[0042] Furthermore, the loss function adopted by the bacteria detection network model based on cross-growth-stage spatio-temporal context and dual-granularity hybrid experts is as follows: ; Among them, represents the L1 loss, represents the binary cross-entropy loss, P and respectively represent the categories of the targets of the true box and the predicted box, and respectively represent the coordinate information of the true box and the predicted box, respectively represent the coordinates of the center point x coordinates, center pointy Coordinates, width, and height.

[0043] In some embodiments, data preparation: Both the training data and the test data are from actual clinical collections. A total of four CLSI standard strains were used in the experiment, including 2 Gram-positive bacteria (Staphylococcus aureus ATCC29213, Enterococcus faecalis ATCC 29212) and 2 Gram-negative bacteria (Pseudomonas aeruginosa ATCC 27853, Escherichia coli ATCC 25922). All strains were purchased from Beijing Baocang Biotechnology Co., Ltd. (Beijing, China). The bacteria were cultured overnight in a shaker containing Mueller Hinton medium (MHB) under the conditions of 200 rpm and 37°C. Subsequently, their concentration was diluted to 10^6 CFU / ml and added to the flat agarose wells of a 96-well plate. A self-made up-view microscope was used in the experiment, and combined with a heating system for bacterial imaging. The microscope was equipped with an MPlanFLN 20 × / 0.30 lens (Olympus, Japan). The size of the imaging area was 800µm × 800µm. Through a self-made autofocus and positioning system, time-lapse images in the same area were captured at an interval of 1 hour, with one photo taken each time, and the shooting time was within 1 second.

[0044] Experimental setup: The optimizer used for training was the SGD optimizer, and the parameter settings were as follows: learning rate lr = 0.02, momentum = 0.9, weight decay = 0.0001. After training was completed, the model with the best mAP.50 was saved.

[0045] Testing: The detector trained successively through the two stages of STE and DG-MoE was directly used for inference, and the image to be tested was directly input into the detector to obtain the prediction result.

[0046] The comparison results of this method with other state-of-the-art detectors are shown in Table 2.

[0047] Table 2 Comparison results of different methods

[0048] The results show that the method of the present invention can outperform most SOTA detectors, including single-stage, two-stage, and end-to-end ones, only when combined with Faster R-CNN, and can match the performance of the state-of-the-art detector, namely RT-DETR, indicating the outstanding advantages of this method.

[0049] The present invention provides a general three-stage contrastive learning bacterial detection framework with cross-growth-stage information, including a spatio-temporal encoding (STE) pre-training stage and a dual-granularity mixture-of-experts (DG-MoE) transfer stage. Among them, in the STE pre-training stage, by fusing explicit temporal encoding with the image information of the corresponding growth stage, the spatio-temporal context across growth stages is encoded to reduce false negatives caused by adhesion clusters; in the DG-MoE transfer stage, global features and local features are respectively extracted by dual-granularity experts to take into account the detection of tiny bacteria and normal bacteria. Among them, the tiny-object expert focuses on extracting local texture features to enhance the position distribution information; the normal-object expert focuses on extracting global morphological features to enhance the boundary information. The morphological differences between tiny bacteria and normal bacteria and the problem of misdetection of impurities with similar morphology are solved by the dual-granularity experts. Embodiment 2

[0050] A computer device 300, as Figure 9 shown, includes a memory 310, a processor 320, and a computer program 330 stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a general method for constructing a bacterial detection network model. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here. Embodiment 3

[0051] A computer-readable storage medium, as Figure 10 shown, stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of a general method for constructing a bacterial detection network model. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here.

[0052] The number of devices and the processing scale described here are used to simplify the description of the present invention. Applications, modifications, and variations of the present invention will be obvious to those skilled in the art.

[0053] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the field, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to specific details and the examples shown and described here.

[0054] The device, computer device, non-volatile computer storage medium, and method provided by the embodiments of this specification are corresponding. Therefore, the device, computer device, and non-volatile computer storage medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding device, computer device, and non-volatile computer storage medium will not be elaborated here.

[0055] Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the method steps can be logically programmed to enable the controller to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same function. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software units for implementing the method and structures within the hardware component.

[0056] The systems, devices, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0057] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, system, or computer program product. Therefore, the embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified function in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0061] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0062] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program units may be located in local and remote computer storage media, including storage devices.

[0063] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0064] The above is only described for the embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A general method for constructing a bacterial detection network model, characterized in that: The following steps are involved: Obtain annotated and verified bacterial images; The bacterial images are used to train a bacterial detection network model based on spatiotemporal context across growth stages and dual-granularity hybrid experts: In the STE pre-training stage, the spatiotemporal context across growth stages is encoded by fusing the explicit temporal coding with the image information of the corresponding growth stage; In the DG-MoE migration stage, global features and local features are extracted respectively by dual-granularity experts to take into account the detection of both microscopic bacteria and normal bacteria.

2. A general bacteria detection network model construction method as claimed in claim 1, characterized in that: Before the step of obtaining the labeled and verified bacterial image, the method further includes: The prepared bacterial images are screened by label checking to remove bacteria-free images, and the remaining images are used for training and testing; Identify the target size of the bacteria image using the standard definition of the MS COCO dataset; Collect auxiliary datasets Used for pre-training auxiliary samples across growth stages. Each set of samples contains multiple registered time-lapse images, each representing a growth stage. Multiple registered time-lapse images represent a complete bacterial growth cycle. All images are annotated and verified.

3. A general bacteria detection network model construction method as claimed in claim 1, characterized in that: In the STE pre-training stage, the step of encoding the spatiotemporal context across growth stages by fusing the explicit temporal coding with the image information of the corresponding growth stage includes: Multiple images of consecutive growth stages are used as joint input to detect bacteria in the target growth stage. The specific process is shown in the following formula: ； Among them, STE represents the spatiotemporal encoder, as shown in formula (2), and DG-MoE represents the detector used for subsequent bacteria detection; ； in, represents the input image at the i-th growth stage, represents splicing along the channel dimension, represents 2D convolution, represents a depthwise separable convolution with a kernel size of 31x31, It represents the time series encoding corresponding to the i-th growth stage, as shown in the following formula: ； Here, t and c represent the growth stage and channel, respectively.

4. A general bacteria detection network model construction method as claimed in claim 3, characterized in that: In the DG-MoE migration stage, the steps of extracting global features and local features respectively by dual-granularity experts to take into account the detection of micro bacteria and normal bacteria include: By Micro Target Expert Extract local texture features and enhance location distribution information; By Normal Target Expert Extract global morphological features and enhance boundary information; The detection results obtained by the two experts are processed by the soft router.

5. A general bacteria detection network model construction method as claimed in claim 4, characterized in that: The Micro Target Expert The overall process is shown as follows: ； in, is a detector for bacteria detection, and NWD represents the NWD measurement used to measure microscopic bacteria, as shown in formula (5). As shown in formula (6): ； in, , Respectively represent detection box a and detection box b, represents the two-norm operation, C represents the modulation coefficient, Respectively represent the center point of the detection box a x Coordinates, center point y Coordinates, width, height; Represents the center point of the detection box b respectively x Coordinates, center point y Coordinates, width, height; ； Among them, after downsampling, four features of different scales are obtained, namely , represents the feature of the i-th scale, is the corresponding fused feature, For the fusion Features, represents an upsampling operation with a factor of 2, Represents a 2D convolution operation with a kernel size of 3x3.

6. A general bacteria detection network model construction method as claimed in claim 5, characterized in that: The normal target expert The overall process is as follows: ； in, is a detector for bacteria detection, and IoU represents the IoU metric used to measure normal-level bacteria, as shown in formula (8). As shown in formula (9); 。 7. A general bacteria detection network model construction method as claimed in claim 6, characterized in that: When the input image When, through and Get the corresponding test results respectively and , the detection result obtained by processing the two experts through the soft router is as follows: ； in, is the union of the detection results of the two experts, for One of them represents the router, which is used to score the prediction box. S is the test result after selection. for S One of them is scored with a confidence score; based on the score given by the router, k candidate bounding boxes are selected by the router, that is, , and then get the final test results .

8. A general bacteria detection network model construction method as claimed in claim 7, characterized in that: The loss function used by the bacterial detection network model based on spatiotemporal context across growth stages and dual-granularity mixed experts is as follows: ； in, represents L1 loss, represents the binary cross entropy loss, P and Represent the categories of the targets of the real box and the predicted box respectively, and Respectively represent the coordinate information of the real box and the predicted box, Respectively represent the center point x Coordinates, center point y Coordinates, width, height.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images