Multi-category cell recognition and counting method and system based on convolutional neural network

Abstract

The present application provides a multi-category cell recognition and counting method and system based on convolutional neural network, including: obtaining an initial model for multi-cell type target detection; optimizing the initial model for multi-cell type target detection to obtain an optimized model for multi-cell type target detection; the convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight; training the optimized model for multi-cell type target detection using a training set to obtain an optimal multi-cell type target detection model; using the optimal multi-cell type target detection model, performing detection operations on different types of cell images of the image to be detected, and outputting the target detection results. The present invention realizes high-precision and real-time multi-cell type target detection, providing a powerful tool for biomedical research and clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a multi-category cell recognition and counting method and system based on convolutional neural network. Background Art

[0002] In biomedical research and clinical diagnosis, real-time and accurate identification and counting of multiple cells under a microscope are crucial. However, traditional methods face huge challenges in processing images with complex backgrounds and coexistence of multiple cell types. First, the morphological characteristics of different types of cells are similar, and the boundaries are blurred, making it difficult to accurately distinguish them. Secondly, the large number of cells and their dense distribution can easily lead to missed detections and false detections. Furthermore, the unstable imaging quality of the microscope, and factors such as uneven lighting and low contrast affect the detection effect.

[0003] In addition, there is a contradiction between real-time requirements and detection accuracy. Improving accuracy often reduces processing speed. At the same time, due to the scarcity of samples of certain cell types, unbalanced training data also restricts model performance. The combination of these problems makes it difficult for existing technologies to simultaneously achieve high-precision identification, accurate counting, and real-time detection of multiple cell types.

[0004] Cell identification and counting are key tasks in medical diagnosis, drug development, and basic biological research. Traditional methods are time-consuming and susceptible to human factors. How to overcome the interference of complex image environments, balance detection speed and accuracy, and achieve rapid and accurate identification and counting of multiple cells under limited computing resources has become a key technical problem that needs to be solved urgently. This is not only related to the efficiency and quality of biomedical research, but also directly affects the accuracy and timeliness of clinical diagnosis. Summary of the invention

[0005] Based on the fact that the above-mentioned existing cell detection cannot achieve high-precision identification, accurate counting and real-time detection of multiple cell types, the present invention provides a multi-category cell recognition and counting method and system based on convolutional neural network, aiming to solve the problems that the existing cell detection cannot achieve high-precision identification, accurate counting and real-time detection of multiple cell types.

[0006] The present invention provides a multi-category cell recognition and counting method based on convolutional neural network, which mainly includes:

[0007] Obtaining an initial model for multi-cell type target detection; the initial model for multi-cell type target detection includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through a convolution operation, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling on a spatial dimension, a channel dimension, and a feedback connection to enhance feature extraction capability;

[0008] Optimizing the multi-cell type target detection initial model to obtain a multi-cell type target detection optimization model; the multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module; the convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight;

[0009] Using the training set to train the multi-cell type target detection optimization model to obtain an optimal multi-cell type target detection model;

[0010] The optimal multi-cell type target detection model is used to perform detection operations on different types of cell images of the image to be detected, and the target detection result is output.

[0011] Furthermore, before the multi-cell type target detection optimization model is trained using the training set to obtain the optimal multi-cell type target detection model, the method further includes:

[0012] Acquire a public sample data set, preprocess the public sample data set, and obtain an updated sample data set;

[0013] Add custom datasets; and,

[0014] Merging and preprocessing the self-made data set with the updated sample data set to obtain a merged data set;

[0015] The merged data set is divided to obtain divided data sets; the divided data sets include the training set.

[0016] Furthermore, the divided data set further includes a test set and a validation set, and the dividing the merged data set to obtain the divided data set includes:

[0017] Using a random number generator to randomly shuffle all the data of the merged data set;

[0018] The data in the merged data set is divided into different data sets according to a preset allocation ratio to obtain the training set, the test set and the validation set.

[0019] Furthermore, the obtaining of a public sample data set, preprocessing the public sample data set, and obtaining an updated sample data set includes:

[0020] Get public sample datasets;

[0021] Extract image features and cell bounding box information from the public sample dataset, perform feature extraction, data enhancement, data cleaning, and label processing operations on the public sample dataset, and remove images with incorrect annotations or low quality and clarity to obtain an updated sample dataset.

[0022] Furthermore, before adding the self-made data set, the method further includes:

[0023] The homemade dataset was constructed by acquiring a T lymphocyte image dataset in a standard biological laboratory using an inverted biological microscope.

[0024] Furthermore, dividing the data in the merged data set into different data sets according to a preset distribution ratio to obtain the training set, the test set and the validation set includes:

[0025] According to a preset distribution ratio, a stratified sampling method is used to divide the data in the merged data set into different data sets to obtain the training set, the test set and the validation set; the deviation between the distribution ratio of each category of cells in the validation set and the test set and the distribution ratio of each category of cells in the training set is within a preset deviation range.

[0026] Furthermore, the method of using the training set to train the multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model includes:

[0027] Setting training parameters; the training parameters include input image size, batch size, number of training iterations, learning rate, and optimizer;

[0028] Using the training parameters and the training set to train the multi-cell type target detection optimization model to obtain a trained model weight file;

[0029] The multi-cell type target detection optimization model is updated according to the model weight file to obtain an optimal multi-cell type target detection model.

[0030] Furthermore, the adopting of the training parameters and the use of the training set to train the multi-cell type target detection optimization model to obtain a trained model weight file includes:

[0031] In the process of training the multi-cell type target detection optimization model using the training set by adopting the training parameters, regularly evaluating the performance of the multi-cell type target detection optimization model by using the validation set to obtain loss function changes and evaluation indicators;

[0032] Monitor the loss function change and the evaluation index exceeds the preset monitoring index, and adjust the training parameters;

[0033] According to the evaluation results of the validation set in each period, the weight file with the highest accuracy is selected as the trained model weight file.

[0034] Further, updating the multi-cell type target detection optimization model according to the model weight file to obtain an optimal multi-cell type target detection model includes:

[0035] Update the multi-cell type target detection optimization model according to the model weight file to obtain a trained and updated multi-cell type target detection optimization model;

[0036] The images in the test set are predicted respectively using the multi-cell type target detection optimization model and other cell detection models after training and updating, so as to obtain the detected cell objects in the test set;

[0037] By traversing the cell objects and counting them according to categories, a first test result of the multi-cell type target detection optimization model after training and updating to detect each category of cells on the test set is obtained; the first test result includes recognition performance parameters and counting performance parameters of the multi-cell type target detection optimization model after training and updating to detect each category of cells on the test set; and

[0038] Using other cell detection models to predict the images in the test set respectively, obtaining the detected cell objects in the test set, and obtaining the second test results of the other cell detection model detecting each category of cells in the test set by traversing the cell objects and counting them according to categories; the second test results include recognition performance parameters and counting performance parameters of the other cell detection model detecting each category of cells in the test set;

[0039] If the first test result is higher than the second test result, the trained and updated multi-cell type target detection optimization model is used as the optimal multi-cell type target detection model.

[0040] Furthermore, it also includes:

[0041] Converting the target detection result into a visualization result; and

[0042] The visualization result is processed by using a counting function, cell counting and recognition are performed on multiple types of cell images, and the identification of each type of cell and the counting result identification are visually output.

[0043] Furthermore, the optimal multi-cell type target detection model is used to perform detection operations on different types of cell images of the image to be detected, and before outputting the target detection result, it also includes:

[0044] Obtaining the image to be detected uploaded by the user terminal; and / or,

[0045] Obtain the image to be detected copied and pasted by the user; and / or,

[0046] Get the image to be detected that the user takes and sends through the camera.

[0047] Furthermore, the present invention also provides a multi-category cell recognition and counting system based on a convolutional neural network, which is used to implement any of the multi-category cell recognition and counting methods based on a convolutional neural network described above, and the multi-category cell recognition and counting system based on a convolutional neural network comprises:

[0048] An acquisition module is used to obtain an initial model for multi-cell type target detection; the initial model for multi-cell type target detection includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through a convolution operation, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling in spatial dimensions, channel dimensions and feedback connections to enhance feature extraction capabilities;

[0049] An optimization processing module, used for optimizing the initial model of multi-cell type target detection to obtain a multi-cell type target detection optimization model; the multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module; the convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight;

[0050] A training module, used to train the multi-cell type target detection optimization model using a training set to obtain an optimal multi-cell type target detection model;

[0051] The detection module is used to use the optimal multi-cell type target detection model to perform detection operations on different types of cell images of the image to be detected, and output the target detection result.

[0052] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0053] The present invention discloses a multi-category cell recognition and counting method and system based on convolutional neural network, which specifically solves the problem of recognition and counting of different types of cells under biological microscope. The method first obtains and optimizes the real-time target detection algorithm structure, introduces a convolution operation enhancement module, a feature interaction capability improvement module and a feature extraction enhancement module to improve the model performance. Then, by integrating the public sample data set and the self-made T-lymphocyte category data set, data preprocessing and division are performed to form a training set, a test set and a verification set. The optimization model is trained using the training set, and the optimal model is obtained through verification set evaluation and parameter adjustment. Finally, the method can detect, identify and count multiple types of cells for the pictures to be detected uploaded or taken by the user, and output visualization results. The present invention realizes high-precision, real-time multi-cell type target detection, providing a powerful tool for biomedical research and clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a multi-category cell recognition and counting method based on a convolutional neural network is provided in an embodiment of the present invention.

[0055] Figure 2 An example diagram of the YOLOv8 model optimization and improvement structure for a multi-cell type target detection optimization model provided in an embodiment of the present invention.

[0056] Figure 3 This is a diagram showing an example structure of a C2f dynamic deformable convolution module provided in an embodiment of the present invention.

[0057] Figure 4 A flowchart of another multi-category cell recognition and counting method based on a convolutional neural network provided in an embodiment of the present invention.

[0058] Figure 5 A flowchart of another multi-category cell recognition and counting method based on a convolutional neural network provided in an embodiment of the present invention.

[0059] Figure 6 A structural diagram of a multi-category cell recognition and counting system based on a convolutional neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0061] See also Figure 1 , Figure 1 The present invention provides a flowchart of a multi-category cell recognition and counting method based on a convolutional neural network. Figure 1As shown, the multi-category cell recognition and counting method based on convolutional neural network provided in this embodiment specifically includes the following steps:

[0062] Step S110, obtaining an initial model for detecting multiple cell types;

[0063] Among them, the initial model for multi-cell type target detection includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through convolution operations, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling in spatial dimensions, channel dimensions and feedback connections to enhance feature extraction capabilities.

[0064] In the process of obtaining the initial model of multi-cell type target detection, it is necessary to select a suitable real-time target detection algorithm structure as the basis. Common real-time target detection algorithm structures include the single-stage target detection algorithm (You Only LookOnce, YOLO) series, the single shot multi-box detector (Single Shot MultiBox Detector, SSD) series, and the single-stage target detection algorithm based on deep convolutional neural network (RetinaNet). The YOLO series algorithm is a single-stage target detection algorithm that mainly solves the target detection problem as a regression problem. The entire process of target recognition is completed through one forward propagation, including the category and location of the detected target. Due to only one forward propagation, YOLO has a very fast detection speed and is suitable for real-time applications. SSD is also a single-stage target detection algorithm that uses feature maps of multiple scales for detection and predicts the location and category of the target through multiple predefined anchor boxes. SSD is more accurate than YOLO, but slightly slower. RetinaNet is a single-stage target detection algorithm based on deep convolutional neural network to solve the problem of imbalance between positive and negative samples and improve the accuracy of the algorithm. RetinaNet outperforms YOLO and SSD in both speed and accuracy.

[0065] The above algorithm structures all have convolution blocks, which are used to perform feature extraction on the input image through convolution operations. Convolution blocks are usually composed of multiple convolution layers, pooling layers, activation functions, etc., which extract low-level to high-level features of the image layer by layer to form a semantic representation of the target object. In addition to convolution blocks, the real-time target detection algorithm structure also requires a feature fusion module, which is used to perform pooling in spatial dimensions, channel dimensions, and feedback connections to enhance feature extraction capabilities. Pooling in the spatial dimension can expand the receptive field and capture the global information of the target object; pooling in the channel dimension can fuse feature maps at different scales to improve the representation ability of the feature; feedback connections can feed back high-level semantic features to the lower layers, guiding the lower-level feature learning to be more accurate.

[0066] Common feature fusion modules include Feature Pyramid Network (FPN), Path Aggregation Network (PAN), Bidirectional Feature Pyramid Networks (BiFPN), etc. They build multi-scale feature pyramids through top-down and bottom-up feature fusion to improve detection accuracy. When obtaining the initial model for multi-cell type target detection, it is necessary to select the appropriate real-time target detection algorithm structure and feature fusion module according to the characteristics of the cell image dataset. For example, if the target scale in the cell image varies greatly, you can choose an algorithm structure with powerful multi-scale feature extraction capabilities, such as YOLOv3, YOLOv4, etc.; if there are many types of cells, you can choose a module with deeper feature fusion capabilities, such as PAN, BiFPN, etc. At the same time, it is also necessary to weigh the number of model parameters and calculations according to hardware conditions and real-time requirements, select a suitable backbone network and feature fusion method, and strike a balance between accuracy and speed. In short, obtaining the initial model for multi-cell type target detection is a comprehensive process, which requires selecting the appropriate real-time target detection algorithm structure and feature fusion module according to task requirements, data characteristics, hardware conditions and other factors, and making appropriate modifications and optimizations to build an efficient and accurate cell detection model. This lays a good foundation for subsequent model training and application.

[0067] In this embodiment, YOLOv8 is selected as the initial model for multi-cell type target detection.

[0068] YOLOv8 is a powerful target detection model developed by Ultralytics and is the latest iteration of the YOLO (YouOnly Look Once) series. Like previous generations of public YOLO models, YOLOv8 uses an end-to-end learning approach to directly predict the location and category of the target object from the input image. The main features of YOLOv8 are high speed and high precision, that is, YOLOv8 has significant improvements in speed and precision, fast inference speed, and can perform real-time target detection.

[0069] In addition, YOLOv8 implements a modular design, that is, the architecture of YOLOv8 is more modular, easy to customize and modify, and developers can adjust the structure and parameters of the model according to their needs.

[0070] In addition, compared with other YOLO models, YOLOv8 achieves efficient data enhancement, that is, YOLOv8 uses a variety of efficient data enhancement strategies, such as mixed enhancement and clipping enhancement, to improve the generalization ability of the model.

[0071] Step S120, optimizing the initial model for detecting multiple cell types to obtain an optimized model for detecting multiple cell types;

[0072] Among them, the multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module.

[0073] Optimize the initial model of multi-cell type target detection and obtain the optimized model of multi-cell type target detection, which mainly includes the following aspects:

[0074] First, in this embodiment, in the optimized real-time target detection algorithm structure, the convolution operation enhancement module of the convolution block is replaced to enhance the convolution operation function by dynamically adjusting the convolution kernel weight.

[0075] The convolution operation is the core of the convolutional neural network, which extracts local features by sliding the convolution kernel on the input feature map. Traditional convolution operations use convolution kernels of fixed size and weight, but this may not adapt well to targets of different scales and shapes. Therefore, deformable convolution can be used to replace ordinary convolution, so that the position and shape of the convolution kernel can be adaptively adjusted. Specifically, an offset is added to each convolution position so that the convolution kernel can deform freely within a certain range, thereby better aligning the target contour. In addition, an attention mechanism can be introduced to adaptively adjust the convolution kernel weights according to the content of the feature map to highlight the significant areas. These improvements can enhance the ability of convolution operations to extract cell features.

[0076] The convolution operation enhancement module realizes the enhancement of the convolution operation function by dynamically adjusting the convolution kernel weights. Specifically, the module adopts multiple groups of convolution kernels, and the weights of each group of convolution kernels are dynamically generated by the input data. This way of dynamically generating weights can make the convolution operation more flexible and adaptable. For example, for different input images, the module can automatically adjust the weights of the convolution kernels according to the characteristics of the image, so as to extract more effective and discriminative features. For another example, by introducing the attention mechanism, the module can weight the convolution kernel weights according to the importance of the input data, so that the convolution operation pays more attention to important feature areas. In addition, dynamic convolution can also increase the expressive power of the model by generating diversified convolution kernels, so that the model can handle more complex and changeable data. In short, by dynamically adjusting the convolution kernel weights, the convolution operation enhancement module can significantly improve the performance and generalization ability of the convolutional neural network. The multi-cell type target detection optimization model is trained using the training set to obtain the optimal multi-cell type target detection model.

[0077] For example, by replacing the C2f layer convolution block in YOLOv8 with the C2f Dynamic Convolution (DynamicConv) architecture, C2f refers to convolution to feature, which means that the convolution operation is used to extract image features. DynamicConv refers to dynamic convolution, which means that the parameters of this convolution operation can be dynamically adjusted according to the input image.

[0078] Secondly, in this embodiment, the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module.

[0079] For example, the AIFI module (Attention-based Intrascale Feature Interaction I) and the convolution (Convolution, Conv) module are combined to replace the spatial pyramid pooling structure in YOLOv8 to optimize the YOLOv8 target detection algorithm and build a multi-category cell detection optimization model. The optimized multi-category cell detection optimization model is trained using the training set data to obtain a trained target detection model as the optimal multi-category cell detection model in this embodiment.

[0080] The AIFI module (Attention-based Intrascale Feature Interaction I) is a technical module designed to improve the ability of neural networks to handle complex tasks. Its main purpose is to optimize information processing and task performance through flexible feature fusion methods. The core feature of the AIFI module is adaptive information fusion. The AIFI module can dynamically adjust the feature fusion strategy based on input data and task requirements. This adaptive mechanism enables the network to more effectively combine information from different sources and improve the understanding and processing capabilities of complex data.

[0081] In this implementation, a convolution (Convolution, Conv) module can be used as a feature extraction enhancement module.

[0082] In this embodiment, the AIF module and the Conv module are combined to replace the Spatial Pyramid Pooling Feature (SPPF) in YOLOv8, and the AIFI module further reduces computational redundancy. By applying self-attention operations to high-level features with rich semantic concepts, the connection between conceptual entities in the image is captured. This processing helps subsequent modules to detect and recognize objects in the image more effectively. Due to the lack of semantic concepts in low-level features and the risk of duplication and confusion when interacting with high-level features, the AIFI module does not perform internal scale interactions on low-level features. Therefore, the main idea of ​​the AIFI module is to focus on processing high-level image features through a self-attention mechanism, thereby improving the performance of the model in object detection and recognition while reducing unnecessary computational consumption.

[0083] See also Figure 2 , Figure 2 This is an example diagram of the YOLOv8 model optimization and improvement structure for the multi-cell type target detection optimization model provided by the embodiment of the present invention. Figure 2 As shown, the embodiment of the present invention replaces the C2f layer in YOLOv8 with C2f_DynamicConv, combines the AIFI (Attention-based Intrascale Feature InteractionI) module and the Conv module to replace the SPPF (Spatial Pyramid Pooling Feature) spatial pyramid pooling structure in YOLOv8, and then performs training to output the detection result Detect. Figure 2As shown, the optimization model includes a backbone network, a neck network and a head network. The head network outputs a detection result Detect. The backbone network includes a first convolution module p0, a second convolution module p1, a first dynamic deformable convolution module p2, a third convolution module p3, a second dynamic deformable convolution module p4, a fourth convolution module p5, a third dynamic deformable convolution module p6, a fifth convolution module p7, a fourth dynamic deformable convolution module p8, a sixth convolution module p9, and an attention feature interaction module p10, which are connected in sequence. The attention feature interaction module p10 can be implemented by the above-mentioned AIFI (Attention-based Intrascale Feature Interaction I) module. The neck network includes a first upsampling layer p11, a first fully connected layer p12, a fifth dynamic deformable convolution module p13, a second upsampling layer p14, a second fully connected layer p15, a sixth dynamic deformable convolution module p16, a seventh convolution module p17, a third fully connected layer p18, a seventh dynamic deformable convolution module p19, an eighth convolution module p20, a fourth fully connected layer p21, and an eighth dynamic deformable convolution module p22.

[0084] In this embodiment, all C2f layers in the original YOLOv8 are replaced with C2f_DynamicConv modules (dynamic deformable convolution modules). The structure diagram of the DynamicConv module is as follows: Figure 3 As shown. The data processing process of the DynamicConv module is that the input data is processed by the linear connection module Linear, the gated linear unit GLU, the convolution layer Lconv, and another linear connection module Lineard. In the data processing process from the gated linear unit GLU to the convolution layer Lconv, the dynamic weight Dynamic weight is realized through another linear connection module Linear. Dynamic weight refers to the mechanism in which the weight value will continue to change as the training progresses during the machine learning model training process. This is different from traditional machine learning algorithms, which usually use fixed values ​​as weights. The advantage of dynamic weights is that a more accurate model can be achieved. Since the weight value will be continuously adjusted according to the training data, the model can better fit the data, thereby improving the prediction accuracy. And the model can achieve more flexible learning, that is, the model can adapt to different types of input data and dynamically adjust the weight according to data changes, making it more flexible.

[0085] The present invention replaces the ordinary convolution in the C2f layer with a dynamic convolution, and the replaced module is the C2f_DynamicConv module. Dynamic Convolution increases the number of network parameters without adding any additional floating point operations (FLOPs). Dynamic Convolution processes input data by dynamically selecting or combining different convolution kernels (called "experts") for each input sample. This method can be regarded as an extension of the traditional convolution operation, which allows the network to adaptively adjust its parameters according to the input. In Dynamic Convolution, instead of using a fixed convolution kernel for all inputs, there are multiple convolution kernels (or parameter sets), and which kernel to use is dynamically selected based on the characteristics of the input. This selection is dynamically generated through a learned function (for example, using a multi-layer perceptron (MLP) and a softmax function) to control the weights of the contributions of each convolution kernel.

[0086] Consider efficient dynamic convolution, which increases the number of parameters by several times with almost no additional FLOPs.

[0087] The dynamic convolution with M dynamic experts can be written as:

[0088]

[0089] Among them, W i is the i-th convolution weight tensor; α i is the corresponding dynamic coefficient.

[0090] The coefficients αi are dynamically generated based on different input samples. A typical way is to generate them based on the input using an MLP module:

[0091]

[0092] Where α∈RM. The coefficient generation in the equation only brings negligible FLOPs compared to the original convolution layer. In this way, ParameterNet implemented with dynamic convolution can significantly increase more parameters while minimizing the increase in FLOPs.

[0093] For a standard convolutional layer, the number of parameters is C out × C i n ×K×K. The number of FLOPs is H ′×W ′× C out × C i n ×K×K. Dynamic convolution includes coefficient generation module, dynamic weight fusion and convolution process.C i n The coefficient generation module for hidden dimensions requires C in ×2+ C i n ×M parameters and C i n ×2+ C i n ×M FLOPs. Dynamic weight fusion is parameter-free and has M × C out × C i n × K × K FLOPs. Therefore, the total parameters and FLOPs of dynamic convolution are:

[0094]

[0095] The parameter ratio of dynamic convolution to standard convolution is:

[0096]

[0097] The FLOPs ratio is:

[0098]

[0099] Therefore, compared to standard convolution, dynamic convolution has approximately M times fewer parameters while the additional FLOPs are negligible.

[0100] By sharing and dynamically combining convolution kernels, Dynamic Convolution can significantly increase the number of model parameters with minimal additional computational cost. Since the convolution kernel is dynamically selected for each input, this method can better adapt to different input features and theoretically improve the generalization ability of the model. Dynamic Convolution allows the model to deploy more complex network structures in resource-limited environments (such as mobile devices) without significantly increasing the computational burden. The design idea of ​​dynamic convolution breaks through the limitations of traditional convolutional network structures and achieves the goal of improving network performance under low FLOPs conditions by dynamically adjusting and optimizing the use of computing resources.

[0101] In this embodiment, the AIF module and the Conv module are combined to replace the SPPF spatial pyramid pooling structure in YOLOv8. AIFI refers to attention-based internal scale feature interaction. The AIF module is a key component, which together with the cross-scale feature fusion module (CCFM) constitutes the encoder part of the model. AIFI is an attention-based feature processing. The AIFI module uses the self-attention mechanism to process high-level features in the image. Self-attention is a mechanism that allows the model to take into account other relevant parts of the data while processing a specific part of the data. This method is particularly suitable for processing high-level image features with rich semantic information. In addition, the AIF module has selective feature interaction, and the AIFI module focuses on internal scale interactions at the high-level feature layer level (i.e., S5). This is based on the recognition that the high-level feature layer contains richer semantic concepts and can more effectively capture the connection between conceptual entities in the image. At the same time, avoid the same interaction at the low-level feature layer because the low-level features lack the necessary semantic depth and may cause duplication and confusion in data processing. The main idea of ​​the AIFI module is to focus on processing high-level image features through the self-attention mechanism, thereby improving the performance of the model in object detection and recognition while reducing unnecessary computational consumption.

[0102] The multi-category cell recognition and counting method based on convolutional neural network provided by the embodiment of the present invention replaces all C2f layers in YOLOv8 with C2f_DynamicConv modules, and combines the AIFI module and the Conv module to replace the SPPF spatial pyramid pooling structure in YOLOv8, so that the model can have more superior performance in target detection, and finally enhances the detection accuracy, computing power and generalization computing ability of the model.

[0103] For example, in other implementations, in the target detection network, it is necessary to fuse shallow high-resolution features and deep high-semantic features to take into account both precise positioning and semantic discrimination. Traditional feature fusion methods such as FPN (Feature Pyramid Network) use top-down feature transfer and layer-by-layer upsampling, but this unidirectional information flow may cause a semantic gap. In order to enhance the feature interaction capability, bidirectional feature fusion can be introduced, that is, while adding a bottom-up feature transfer path from top to bottom, so that features at different levels can fully exchange information. For example, PANet (Path Aggregation Network) uses a bidirectional feature pyramid structure. In addition, in order to further improve the expressiveness of features, an attention module can be applied to the fused feature map to adaptively calibrate the feature response. Common attention modules include SE (Squeeze-and-Excitation), CBAM (Convolutional Block Attention Module), etc., which can model the dependencies between feature channels and spatial dimensions and enhance the features of salient targets.

[0104] In addition, in the optimized real-time target detection algorithm structure, the feature extraction enhancement module can improve the backbone network's ability to extract cell features. Traditional backbone networks such as Residual Network (ResNet) and Mobile Neural Network (MobileNet) use layer-by-layer stacked convolution and downsampling designs, but this may lead to a mismatch between feature resolution and receptive field. In order to better balance the resolution and receptive field of features, multi-scale feature aggregation modules such as SPP (Spatial Pyramid Pooling) and ASPP (Atrous Spatial Pyramid Pooling) can be introduced into the backbone network. These modules aggregate features at multiple scales through dilated convolution or pooling operations with different dilation rates, expanding the receptive field while maintaining resolution. In addition, high-resolution feature connections such as the RFB (Receptive Field Block) module can be introduced in the shallow layer of the backbone network to fuse high- and low-level features to supplement spatial detail information. These improvements can enhance the backbone network's ability to extract multi-scale and multi-level cell features. Through the above optimizations, the multi-cell type target detection model can better adapt to the characteristics of cell images and achieve a balance in positioning accuracy, classification accuracy, and inference speed. The convolution operation enhancement module enables the feature extraction process to adapt to the cell shape, while the feature interaction capability improvement module and the feature extraction enhancement module enhance the feature expression capabilities from the two levels of feature fusion and backbone network respectively. These improvements ultimately lead to improved detection performance and lay a high-quality detection foundation for subsequent cell counting and analysis.

[0105] Step S130, using the training set to train the multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model.

[0106] During the training process, we first need to prepare a well-labeled cell image dataset, which should contain multiple types of cells, and each cell should have accurate location and category annotations. Then, the dataset is divided into a training set and a validation set according to a certain ratio, where the training set is used for model training and the validation set is used to evaluate the model performance. During the training phase, the optimized target detection model will forward propagate the images in the training set, calculate the predicted cell positions and categories, compare them with the true annotations, and calculate the loss function value. Then, the parameters of the model are updated through the back-propagation algorithm so that the model's prediction results are closer to the true annotations. This process will be iterated many times until the model's performance on the validation set is optimal. In the end, we can get a multi-cell type target detection model with excellent performance, which can accurately detect different types of cells in the image and give their location and category information. This training method can effectively improve the generalization ability and robustness of the model, so that it can adapt to cell detection tasks in different scenarios.

[0107] In step S130 provided in this embodiment, that is, using the training set to train the multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model, specifically includes:

[0108] Setting training parameters; the training parameters include input image size, batch size, number of training iterations, learning rate, and optimizer;

[0109] Using the training parameters and the training set to train the multi-cell type target detection optimization model to obtain a trained model weight file;

[0110] The multi-cell type target detection optimization model is updated according to the model weight file to obtain an optimal multi-cell type target detection model.

[0111] Specifically, in the training phase, the updated YOLOv8 optimization model can be trained using a dataset containing 5,000 images of different types of cells, with 100 training rounds, a learning rate of 0.01, and a batch size of 8. During the training process, the model is optimized using the cross entropy loss function and Adaptive Moment Estimation (Adam), and finally an optimal multi-cell type target detection model with an average precision mAP (Mean Average Precision) of 95% is obtained.

[0112] In other embodiments, according to the preprocessed training set, the training parameters are set, including the input image size of 640*640, the batch size of 16, the number of training iterations of 300, the initial learning rate of 0.01, the learning rate dynamic of 0.937, the weight decay coefficient of 0.0005, and the optimizer using the stochastic gradient descent optimizer (SGD). The set training parameters are used to train the optimized multi-cell type target detection model using the preprocessed training set, and the model parameters are continuously updated through the forward propagation and back propagation processes to obtain the trained model weight file. According to the model weight file obtained by training, the optimized multi-cell type target detection model is updated to obtain the optimal multi-cell type target detection model.

[0113] Specifically, when training the multi-cell type target detection model, the input image size is first set to 640*640 pixels, the batch size is 16, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, and SGD is selected as the optimizer. Then, the pre-processed training set data is used to calculate the loss function through forward propagation, and then the model parameters are updated through back propagation. The optimization is continuously iterated to finally obtain the trained model weight file. The optimized detection model is updated according to the weight file to obtain the cell detection model with the best performance. When using the trained model for cell detection, the image to be detected is input into the model, and the category, position coordinates and confidence scores of each cell target are obtained through operations such as feature extraction and target box regression. Then, the non-maximum suppression algorithm is used to filter redundant and overlapping detection frames with a threshold of 0.5 to obtain the final cell detection result. On this basis, by adding a counting module, various types of cells are counted and counted according to the category label and position information of the cell target to obtain the number of different cell types. Finally, the graphical model demonstration framework (Gradio) is used for visualization, and the result image with detection box and quantity annotation is presented to the user. If it is a real-time detection application, an interactive interface is built through Gradio to receive the images to be detected uploaded by the user in real time and dynamically return the detection count results; if it is an offline batch detection, the result image and statistical data are saved in a specified folder for further analysis and application.

[0114] In this embodiment, the training parameters are used to train the multi-cell type target detection optimization model using the training set to obtain a trained model weight file, including:

[0115] In the process of training the multi-cell type target detection optimization model using the training set by adopting the training parameters, regularly evaluating the performance of the multi-cell type target detection optimization model by using the validation set to obtain loss function changes and evaluation indicators;

[0116] Monitor the loss function change and the evaluation index exceeds the preset monitoring index, and adjust the training parameters;

[0117] According to the evaluation results of the validation set in each period, the weight file with the highest accuracy is selected as the trained model weight file.

[0118] In this embodiment, according to the training parameters, the training set is used to train the multi-cell type target detection optimization model to obtain the loss function change and evaluation index during the training process. It is determined whether the loss function change and the evaluation index exceed the preset monitoring index. If exceeded, the training parameters are adjusted and the training continues; otherwise, the next step is entered. The performance of the multi-cell type target detection optimization model is regularly evaluated through the validation set to obtain the evaluation results of each validation set. Compare the evaluation results of each validation set, and select the weight file with the highest accuracy as the model weight file after training. Save the trained model weight file as the best.pt file as the weight file for subsequent detection of cell images.

[0119] Specifically, according to the training parameters, such as the input image size is set to 640*640, the batch size batch-size is 16, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer uses SGD, the random seed is fixed during the experiment, and the pre-trained weights are not used in the whole process. The training set is used to train the multi-cell type target detection optimization model, and the loss function change and mAP and other evaluation indicators during the training process are obtained. It is judged whether the loss function change and the evaluation index exceed the preset monitoring index (such as the loss function change is less than 0.01, and the mAP is greater than 0.9). If it exceeds, the training parameters are adjusted (such as increasing the learning rate or batch-size) and the training is continued; otherwise, the model performance is evaluated every 50 iterations through the validation set to obtain the mAP of each validation set. Compare the mAP of each validation set, and select the weight file best.pt with the highest mAP (such as 0.95) as the weight file of the trained model. Read best.pt as the model weight, take the image of the cell to be detected as input, use the trained model to predict it, and obtain the coordinates, category, and confidence of the detected cell object. Traverse the detection results, count according to the category labels (such as lymphocytes, red blood cells, etc.), and get the number of cells in each category. Finally, superimpose the category, coordinate box, and quantity information on the detected image, and visualize it through the Gradio framework to complete multi-category cell recognition and counting.

[0120] Preferably, for example, during the training process, the superiority of the model can be determined by various model evaluation indicators. The indicators for evaluating superiority specifically include: Precision (precision), Recall (recall), GFLOPS (number of floating point operations), AP (Average Precision), a curve drawn with recall as the horizontal axis and precision as the vertical axis is the PR curve, the area under the PR curve is defined as AP, and mAP is the average value of AP of each category).

[0121] The calculation formulas for each evaluation index are as follows:

[0122]

[0123] Among them, P is the precision, R is the recall, TP means the number of positive classes predicted as positive classes, FP means the number of negative classes predicted as positive classes, and FN means the number of negative classes predicted as negative classes.

[0124] The ablation experiment results of the embodiment of the present invention are shown in Table 1:

[0125] Table 1 Ablation experiment

[0126] AIFI C2f_DynamicConv mAP50 mAP50-95 Precision Recall GFLOP × × 0.601 0.341 0.704 0.578 78.7 √ × 0.624 0.348 0.725 0.595 77.8 × √ 0.604 0.341 0.712 0.588 59.4 √ √ 0.625 0.353 0.739 0.602 58.5

[0127] The comparison of the detection results of the embodiment of the present invention and the YOLOv8m model is shown in Table 2:

[0128] Table 2 Comparative experiment

[0129] Experimental content mAP50 mAP50-95 Precision Recall GFLOP YOLOv8m 0.601 0.341 0.704 0.578 78.7 Ours 0.625 0.353 0.739 0.602 58.5

[0130] It can be seen from Table 2 that the embodiment of the present invention has significantly improved performance after making improvements on YOLOv8m. It is 2.4% higher than YOLOv8m in mAP50, 1.2% higher than YOLOv8m in mAP50-95, 3.5% higher than YOLOv8m in Precision, 2.4% higher than YOLOv8s in Recall, and 25.7% lower in GFLOP.

[0131] The comparison of AP detection results of 7 cell types before and after the model structure improvement is shown in Table 3:

[0132] Table 3 Comparison of categories

[0133] Category YOLOv8m Ours All 0.601 0.625 Epithelial 0.465 0.474 Lymphocyte 0.727 0.732 Neutrophil 0.573 0.583 Macrophage 0.552 0.527 Epithelial Nuclei 0.558 0.628 IEL 0.456 0.516 T-lymphocyte 0.880 0.918

[0134] It can be seen from Table 3 above that among the seven types of cells detected, the AP of six types has been improved, indicating that the embodiment of the present invention has effectively improved the performance of multiple types of cell detection algorithms.

[0135] In order to verify the effectiveness of the improved modules in the embodiment of the present invention on the model improvement, under the same parameter conditions, the embodiment of the present invention conducted the following four groups of ablation experiments on two innovative points. Both the introduction of the C2f_DynamicConv module and the introduction of the AIFI module achieved good experimental results in the experiments, but the combination of the two achieved the best effect.

[0136] The effective improvement of the algorithm provided by the embodiment of the present invention can be clearly felt through the data in the three tables in the above comparative experiment and the ablation experiment.

[0137] Step S140 , using the optimal multi-cell type target detection model, performing detection operations on different types of cell images of the image to be detected, and outputting the target detection result.

[0138] In this embodiment, based on the above step S130, that is, based on the preprocessed data set, the optimized multi-cell type target detection model is used for training. After the optimal multi-cell type target detection model is obtained through iterative optimization, the cell image to be detected is read and used as a data source, and input into the trained optimal multi-cell type target detection model. The input cell image is predicted using the trained optimal multi-cell type target detection model to obtain the detected cell objects and their categories. The detected cell objects are traversed and counted according to the categories to obtain the recognition and counting results of each category of cells. The detected images and the recognition and counting results of each category of cells are visualized and output to complete the multi-category cell recognition and counting tasks.

[0139] In this embodiment, after obtaining the cell image to be detected, the image preprocessing technology is used to perform denoising, enhancement and other operations on the image to improve the image quality and prepare for subsequent detection. The trained optimal multi-cell type target detection model is loaded, the preprocessed cell image to be detected is read, and the image is input into the model for inference prediction. The model extracts and analyzes the input cell image, and captures the key feature information of the cell image through multi-level feature learning of the convolutional neural network. Using the features learned by the model, a sliding window detection is performed in the cell image to identify the different types of cell regions in the image. The detected cell region is subjected to bounding box regression and category judgment to determine the specific location coordinates and cell type of each cell region. According to the detection results, the bounding boxes and category labels of the different types of cells identified are drawn on the original image to intuitively display the detection effect. The detection results are post-processed, including removing overlapping detection boxes, merging similar cell regions, etc., to improve the accuracy and reliability of the detection. The final cell detection results are output, including the location information, type information, etc. of various types of cells, to provide data support for subsequent cell counting and analysis.

[0140] In the detection operation stage, when performing detection operations on different types of cell images of the detected image, the input 1024x1024 pixel cell image can be preprocessed first, including normalization and pixel value scaling. Then the preprocessed image is input into the optimal multi-cell type target detection model for detection reasoning. The model obtains the location and category information of the cell area through feature extraction. The non-maximum suppression (NMS) algorithm is used to filter the detection results to remove redundant detection boxes with an overlap of more than 0.5. Finally, the detected cell bounding boxes and category labels are drawn on the original image. Different types of cells are represented by bounding boxes of different colors, and the detection results are output in the JavaScript Object Notation (JSON) format, which contains the location coordinates, confidence and category information of each cell for subsequent cell counting and analysis tasks. The entire detection process runs on a high-performance graphics processing unit (GPU). The detection time for a single image is less than 50 milliseconds, meeting the real-time requirements. The high-performance GPU can be an NVIDIA Tesla V100 GPU, which is a high-performance GPU suitable for data centers and is designed for high-performance computing tasks such as deep learning and artificial intelligence. The main feature is that it can significantly accelerate deep learning training and reasoning. It has up to 5120 parallel computing cores, providing powerful parallel processing capabilities, and is equipped with up to 32GB of high-speed hybrid memory interface (HBM2) wide area memory, which can support large models and data sets, as well as high-performance GPU communication through high-bandwidth interconnect technology and virtual interconnect switch connections. In addition, it also has a broad software ecosystem that supports a variety of deep learning frameworks and software tools.

[0141] In other embodiments, the trained optimal multi-cell type target detection model and the image to be detected are used to perform forward reasoning on different types of cell images, and the detection results such as the category, position and confidence of the cell target are obtained through feature extraction and target frame regression of the model. The detection results output by the model are post-processed, and redundant and overlapping detection frames are removed by a non-maximum suppression algorithm to obtain the final cell target detection results. A cell counting module is added to count various types of cells according to the category and position information of the detected cell targets, and obtain the quantitative statistical results of each cell category. The Gradio framework is used to visualize the cell detection and counting results, and generate a result image with detection frames and quantity annotations.

[0142] If real-time detection and counting are required, the image to be detected is input into the user interface built by Gradio to trigger the detection and counting process and dynamically display the results; if it is offline batch detection, the result image and statistical data are saved to the specified path. According to the application requirements, it is determined whether the detection and counting results need to be further analyzed and used. If so, the result data is passed to the downstream analysis module; if not, the entire cell detection and counting process is completed.

[0143] See also Figure 4 , Figure 4 Flow chart of another multi-category cell recognition and counting method based on convolutional neural network provided by an embodiment of the present invention. Figure 4 As shown, Figure 1 The difference between the multi-category cell recognition and counting method based on convolutional neural network shown in Figure 1 Before step S130, that is, before the multi-cell type target detection optimization model is trained using the training set to obtain the optimal multi-cell type target detection model, the following steps S121 to S124 are also included:

[0144] Step S121, obtaining a public sample data set, preprocessing the public sample data set, and obtaining an updated sample data set;

[0145] The obtaining of the public sample data set, preprocessing the public sample data set, and obtaining the updated sample data set specifically includes the following steps:

[0146] Get public sample datasets;

[0147] Extract image features and cell bounding box information from the public sample dataset, perform feature extraction, data enhancement, data cleaning, and label processing operations on the public sample dataset, and remove images with incorrect annotations or low quality and clarity to obtain an updated sample dataset.

[0148] In other embodiments, a public sample data set is obtained, and image data in the data set is preprocessed, including operations such as image size normalization and image enhancement, to obtain a preprocessed updated sample data set.

[0149] In this embodiment, the public sample datasets include Multicellular Organism Nuclei Segmentation And Classification (MoNuSAC) and Multiple Cell Types Dataset (MuCeD). Among them, MoNuSAC is a large dataset containing nucleus boundary annotations and category labels. The dataset contains more than 46,000 nuclei from 37 hospitals, 71 patients, four organs and four nucleus types. Since MoNuSAC was originally a segmentation dataset, in order to use it for the Multimodal Complex Conversational Chinese Dataset (MC2DC), the embodiment of the present invention pre-processes it by converting each segmentation mask into a bounding box. In addition, the embodiment of the present invention uses an inverted biological microscope MI52-N to obtain images of the T-lymphocyte category in a standard biological laboratory, and cuts, divides and screens the acquired images. Finally, the images and labels of the three datasets are merged, and the processed datasets are divided into training sets, validation sets and test sets. The H&E stained histopathology images of human duodenum in the MuCeD dataset were captured by an Olympus BX50 microscope using a DP26 camera at 20x zoom, and the size of each image is 1920×2148. The dataset has 55 images, and the bounding boxes were annotated using the open source image annotation tool (LabelMe) software and further verified by multiple pathologists. The cells were selected from the epithelial region. The epithelial region represents the area of ​​continuous villi and is used for cell detection, while the rest of the area is masked. In addition, each image was cut into 9 sub-images, and each sub-image was rescaled to 640x640.

[0150] Specifically, after obtaining the MoNuSAC and MuCeD public sample datasets, the image data is resized and uniformly adjusted to 512*512 pixels, and image enhancement techniques such as random horizontal flipping and random cropping are used to expand the dataset.

[0151] Step S122, adding a self-made data set; and,

[0152] Step S123, merging and preprocessing the self-made data set and the updated sample data set to obtain a merged data set;

[0153] The preprocessed updated sample dataset is merged with the self-made dataset, and the merged dataset is preprocessed by label unification, data cleaning and other preprocessing operations to obtain the merged dataset.

[0154] Specifically, the preprocessed public dataset can be merged with the self-made dataset, the cell category labels can be encoded using One-Hot encoding, and missing or abnormal data can be removed to obtain a merged dataset.

[0155] In other embodiments, specifically, after acquiring MoNuSAC, MuCeD, and homemade T lymphocyte image datasets, the bounding box of the MoNuSAC dataset can be fine-tuned at the pixel level, and the labels of the three datasets can be mapped to a unified category space to obtain a merged dataset containing 7 cell types.

[0156] Step S124, dividing the merged data set to obtain a divided data set; the divided data set includes the training set.

[0157] In step S124, the divided data set further includes a test set and a validation set, and the dividing the merged data set to obtain the divided data set includes the following steps:

[0158] Using a random number generator to randomly shuffle all the data of the merged data set;

[0159] The data in the merged data set is divided into different data sets according to a preset allocation ratio to obtain the training set, the test set and the validation set.

[0160] Partitioning the data in the merged data set into different data sets according to a preset allocation ratio to obtain the training set, the test set, and the validation set specifically includes:

[0161] According to a preset distribution ratio, a stratified sampling method is used to divide the data in the merged data set into different data sets to obtain the training set, the test set and the validation set; wherein the deviation between the distribution ratio of each category of cells in the validation set and the test set and the distribution ratio of each category of cells in the training set is within a preset deviation range.

[0162] In this embodiment, a stratified sampling method can be used to divide the merged data set, and samples are randomly selected according to a certain ratio to form a training set, a validation set, and a test set to obtain a divided data set. Specifically, a stratified sampling method can be used to divide the merged data set into a training set, a validation set, and a test set according to a ratio of 8:1:1.

[0163] And, determine whether the deviation between the distribution ratio of each category of cells in the validation set and the test set and the distribution ratio in the training set is within a preset deviation range. If the deviation exceeds the preset range, return to the preset distribution ratio and re-adopt the stratified sampling method to divide the data in the merged data set into different data sets until the deviation between the distribution ratio of each category of cells in the validation set and the test set and the distribution ratio of each category of cells in the training set is within the preset deviation range.

[0164] For example, a stratified sampling method was used to divide the training set, validation set, and test set in a ratio of 8:1:1 to ensure that the distribution ratio deviation of each cell category in the three data sets did not exceed 5%.

[0165] In this embodiment, during the training phase, a data set containing 5,000 images of different types of cells can be used to train the updated YOLOv8 model. The YOLOv8 model serves as the initial model for multi-cell type target detection in this embodiment, and the updated YOLOv8 model serves as the optimized model for multi-cell type target detection in this embodiment.

[0166] Specifically, by replacing the C2f layer in YOLOv8 with C2f_DynamicConv, the AIFI module and the Conv module are combined to replace the SPPF spatial pyramid pooling structure in YOLOv8, the YOLOv8 target detection algorithm is optimized, and a multi-category cell detection optimization model is constructed. The optimized multi-category cell detection optimization model is trained using the training set data to obtain a trained target detection model as the optimal multi-category cell detection model in this embodiment.

[0167] In this embodiment, a cell image to be detected is obtained, and the trained optimal multi-category cell detection model is used to identify different types of cells in the image. According to the recognition results, the cells of each category are counted to obtain the number of cells in each category. The trained optimal multi-category cell detection model is tested using the test set data to evaluate the detection accuracy and computing power of the model.

[0168] After obtaining the training set according to step S124, execute Figure 1 Step S130 shown, i.e., using the training set to train the multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model, specifically includes:

[0169] According to the training set in the divided data set and the initialization of the training model by setting the hyperparameters, a deep learning framework is used to build a training multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model.

[0170] In other embodiments, the training set can be input into an initialized multi-cell type target detection optimization model, and the model parameters can be optimized using a back propagation algorithm. The training is continuously iterated until the model converges or reaches a preset number of iterations to obtain a trained multi-cell type target detection model.

[0171] The performance of the trained multi-cell type target detection optimization model is evaluated using the validation set in the partitioned data set. By calculating indicators such as the precision and recall rate of the multi-cell type target detection optimization model on the validation set, it is determined whether the trained multi-cell type target detection optimization model has achieved the expected performance.

[0172] If the performance of the multi-cell type target detection optimization model after training does not meet expectations, return to adjust the hyperparameter initialization training model structure or hyperparameters, continue to use the back propagation algorithm to optimize the parameters of the hyperparameter initialization training model, continuously iterate the training, and retrain; if the performance of the multi-cell type target detection optimization model has met expectations, it will be determined as the optimal multi-cell type target detection model for subsequent cell detection tasks.

[0173] Specifically, a multi-cell type target detection optimization model with a feature pyramid network (FPN) training model structure can be constructed based on a deep learning framework (PyTorch), with an initial learning rate of 0.01, a batch size of 8, and an optimizer of Adam. The training set data is input into the model, and the model parameters are optimized by the gradient descent algorithm, with 50 iterations of training. The model performance is evaluated on the validation set, and when the mAP (mean Average Precision) index reaches above 0.85, it is determined to be the optimal multi-cell type target detection model for subsequent cell detection tasks.

[0174] See also Figure 5 , Figure 5 Flow chart of another multi-category cell recognition and counting method based on convolutional neural network provided by an embodiment of the present invention. Figure 5 As shown, Figure 4 The difference of the multi-category cell recognition and counting method based on convolutional neural network shown is that before step S122, that is, before adding the self-made data set, the following steps are also included:

[0175] Step S120, using an inverted biological microscope to obtain a T lymphocyte image dataset in a standard biological laboratory to construct the self-made dataset.

[0176] Among them, the T-Lymphocyte Image Dataset (T-lymphocyte) is a database containing T lymphocyte images. T lymphocytes (Thymus lymphocyte, T-lymphocyte) are lymphocytes that mature in the thymus.

[0177] It is closely related to many diseases, such as cancer and AIDS. Analyzing T lymphocyte images can help early diagnosis and monitor disease progression, and even help develop new treatments. The main functions of the T lymphocyte image dataset include T lymphocyte identification and classification, which is used to train machine learning models to identify and classify different types of human T lymphocytes. For the study of T lymphocyte function, the T lymphocyte dataset can be used to study the morphological, structural characteristics and functions of T lymphocytes to help understand the working mechanism of the immune system.

[0178] In this embodiment, an inverted biological microscope (MI52-N, MI52-N) is used to collect cell images of the T-lymphocyte category in a standard biological laboratory environment to obtain a T-lymphocyte image dataset. The acquired T-lymphocyte image dataset is cut and divided to obtain a cut T-lymphocyte image dataset, which is used as a self-made dataset.

[0179] Specifically, we can use the MI52-N inverted biological microscope to collect 100 T-lymphocyte cell images in a biological laboratory environment that meets the GB / T 35746-2017 standard, cut the images into 256*256 pixel image blocks using the sliding window method, and construct a self-made dataset.

[0180] In this embodiment, the updating of the multi-cell type target detection optimization model according to the model weight file to obtain the optimal multi-cell type target detection model includes:

[0181] Update the multi-cell type target detection optimization model according to the model weight file to obtain a trained and updated multi-cell type target detection optimization model;

[0182] The images in the test set are predicted respectively using the multi-cell type target detection optimization model and other cell detection models after training and updating, so as to obtain the detected cell objects in the test set;

[0183] By traversing the cell objects and counting them according to categories, a first test result of the multi-cell type target detection optimization model after training and updating to detect each category of cells on the test set is obtained; the first test result includes recognition performance parameters and counting performance parameters of the multi-cell type target detection optimization model after training and updating to detect each category of cells on the test set; and

[0184] Using other cell detection models to predict the images in the test set respectively, obtaining the detected cell objects in the test set, and obtaining the second test results of the other cell detection model detecting each category of cells in the test set by traversing the cell objects and counting them according to categories; the second test results include recognition performance parameters and counting performance parameters of the other cell detection model detecting each category of cells in the test set;

[0185] If the first test result is higher than the second test result, the trained and updated multi-cell type target detection optimization model is used as the optimal multi-cell type target detection model.

[0186] Update the multi-cell type target detection optimization model according to the model weight file to obtain the multi-cell type target detection optimization model after training and updating. Use the multi-cell type target detection optimization model after training and updating and other cell detection models to predict the images in the test set respectively, and obtain the cell objects detected in the test set. By traversing the cell objects and counting them according to the categories, obtain the first test results of the multi-cell type target detection optimization model after training and updating on the test set, including recognition performance parameters and counting performance parameters. Use other cell detection models to predict the images in the test set respectively, obtain the cell objects detected in the test set, and by traversing the cell objects and counting them according to the categories, obtain the second test results of the other cell detection models on the test set for detecting each category of cells, including recognition performance parameters and counting performance parameters. Determine whether the first test result is higher than the second test result. If so, use the multi-cell type target detection optimization model after training and updating as the optimal multi-cell type target detection model. Use the optimal multi-cell type target detection model and the image to be detected to detect different types of cell images and output the target detection results. Add a counting module to count cells in the detected image and obtain the counting results of each category of cells. The output results are visualized using the Gradio framework to generate intuitive recognition and counting results. The final cell recognition and counting results of multiple types of cell images are output to complete multi-category cell recognition and counting based on convolutional neural networks.

[0187] Specifically, the multi-cell type target detection optimization model is updated according to the model weight file to obtain the multi-cell type target detection optimization model after training and updating, for example, using the Adam optimizer, with the learning rate set to 0.001 and the number of iterations set to 100. The multi-cell type target detection optimization model after training and updating and other cell detection models such as the Faster Regions With Convolutional Neural Networks (Faster R-CNN) and the Single Shot MultiBox Detector (SSD) model are used to predict 1,000 images in the test set and obtain the detected cell objects in the test set.

[0188] In this implementation, it is determined whether the first test result is higher than the second test result. If so, the updated multi-cell type target detection optimization model is used as the optimal multi-cell type target detection model. The optimal multi-cell type target detection model and the image to be detected are used to detect different types of cell images, output the target detection results, and add a counting module to count the cells in the detected image to obtain the counting results of each category of cells, such as 100 red blood cells, 50 white blood cells, 200 platelets, etc. Finally, the Gradio framework is used to visualize the output results, generate an intuitive recognition and counting result display, output the final cell recognition and counting results of multiple types of cell images, and complete the multi-category cell recognition and counting based on convolutional neural networks.

[0189] Combined with the above Figure 1 , Figure 4 as well as Figure 5 The flowchart of the multi-category cell recognition and counting method based on convolutional neural network is shown. The embodiment of the present invention discloses a multi-category cell recognition and counting method based on convolutional neural network, which specifically solves the problem of recognition and counting of different types of cells under biological microscope.

[0190] The method first obtains and optimizes the real-time target detection algorithm structure, introduces a convolution operation enhancement module, a feature interaction capability improvement module and a feature extraction enhancement module to improve model performance. Then, by integrating public sample data sets and self-made T-lymphocyte category data sets, data preprocessing and division are performed to form training sets, test sets and verification sets. The optimization model is trained using the training set, and the optimal model is obtained through verification set evaluation and parameter adjustment. Finally, the method can detect, identify and count multiple types of cells for images to be detected uploaded or taken by users, and output visualization results. The present invention realizes high-precision, real-time detection of multiple cell types, providing a powerful tool for biomedical research and clinical diagnosis.

[0191] Specifically, after obtaining the MoNuSAC, MuCeD and self-made datasets, the frames of the MoNuSAC dataset were annotated at the pixel level. 500 images of the T-lymphocyte category were obtained using an inverted biological microscope MI52-N with a 40x lens. The image resolution was 1024*1024, and the acquired images were cut and divided into 256*256 using a sliding window algorithm. A total of 8,000 sub-images were obtained. The labels of the three datasets were mapped to a unified category label space to obtain a total of 50,000 merged datasets. The Mersenne rotation algorithm was used to randomly shuffle the merged dataset and divide it into a training set, a test set, and a validation set at a ratio of 8:1:1, containing 40,000, 5,000, and 5,000 images, respectively. The balance of the dataset division was verified by calculating the mean, variance and other statistics of each dataset, and the random algorithm was used to visualize the divided data to intuitively evaluate the category distribution. Build a data loader based on the divided data set, set the batch size to 32, use 4 processes to load data, and test the data loader through the iterator method to ensure that the data is loaded normally. Finally, save the divided data set in the preset format to the specified path, and save the parameter configuration of the data holiday as a JSON object format file to facilitate the subsequent model training and evaluation process calls.

[0192] In other embodiments, MoNuSAC, MuCeD and self-made data sets can be obtained, the frame of the MoNuSAC data set is processed, the image of the T-lymphocyte category is obtained using an inverted biological microscope MI52-N, and the acquired image is cut and divided, and the labels of the three data sets are merged to obtain a merged data set. In addition, a random number generator is used to randomly shuffle all the data of the merged data set to ensure the randomness and uniformity of the data distribution. According to the preset allocation ratio, the data in the merged data set is divided into different data sets to obtain a training set, a test set and a validation set respectively.

[0193] In other embodiments, it is determined whether the data volumes of the divided training set, test set, and validation set meet preset requirements. If not, the preset allocation ratio is adjusted and the data set is re-divided until the requirements are met.

[0194] And, in other embodiments, by performing statistical analysis on the divided training set, test set and validation set, information such as the number of samples and category distribution of each data set is obtained to determine the rationality of the data set division.

[0195] Furthermore, in other embodiments, data visualization technology may be used to visualize the sample distribution of the divided training set, test set, and validation set, so as to intuitively evaluate the effect of data set division.

[0196] And, in other embodiments, according to the divided training set, test set and verification set, corresponding data loaders are respectively constructed for subsequent model training, testing and verification. By testing the data loader, it is determined whether the data can be loaded and transmitted normally. If there is a problem, the data set division and data loader are debugged and optimized. The divided training set, test set and verification set and the corresponding data loader are saved to a specified path so that the subsequent model training and evaluation process can be called to ensure the persistence and reusability of the data set division results. The public sample data sets MoNuSAC and MuCeD are obtained, and the public sample data sets are preprocessed to obtain the preprocessed public sample data sets. The image features and cell bounding box information in the preprocessed public sample data sets are extracted, and the image features are extracted using a feature extraction algorithm to obtain the extracted image feature data. According to the extracted image feature data, the image data is enhanced by using a data enhancement technology, and the enhanced image data is obtained by rotation, translation, scaling and other operations. The enhanced image data is cleaned, and the cleaned image data is obtained by removing images with incorrect annotations or low quality clarity. According to the cleaned image data, the image data is labeled, and the processed label data is obtained by marking the cell category and location information. The processed label data is fused with the cleaned image data to obtain an updated sample data set. Some self-made data sets are added to the updated sample data set, and the self-made data sets are fused with the updated sample data set through data fusion technology to obtain a fused data set. The fused data set is divided into a training set, a validation set, and a test set through a random sampling method to obtain a divided data set. The divided training set, validation set, and test set are stored separately, and different data sets are stored in designated locations through data storage technology to prepare data for subsequent model training.

[0197] Specifically, after obtaining the public sample datasets MoNuSAC and MuCeD, they were preprocessed, including resizing the image to 512x512 pixels and normalizing the image to the range of 0-1. Then, the image features were extracted to obtain a 4096-dimensional feature vector. Then, the dataset was expanded to 5 times the original data volume through data enhancement operations such as random horizontal flipping, vertical flipping, and rotation of ±30 degrees. In the data cleaning stage, the image clarity was calculated, low-quality images were removed, and incorrectly labeled data were removed through manual inspection. For the remaining data, label processing was performed according to the cell category and location information to generate a label file in YOLO format. The processed image data was fused with the label data to obtain an updated sample dataset. Then, 1000 self-made cell image data were added, and data fusion techniques such as histogram matching were used to make them consistent with the distribution characteristics of the updated sample dataset. Finally, a stratified sampling method was used to divide the dataset into training, validation, and test sets in a ratio of 8:1:1, and they were stored separately in a preset format for subsequent model training and evaluation.

[0198] Optionally, the above method further includes:

[0199] Converting the target detection result into a visualization result; and

[0200] The visualization result is processed by using a counting function, cell counting and recognition are performed on multiple types of cell images, and the identification and counting results of each type of cells are visually output.

[0201] The Gradio framework is used to visualize the detected image and the identification and counting results of each category of cells, so as to realize the visualization transformation of the target detection results. Obtain the visual output detection result image, and count each category of cells in the image through the counting function. If the counting function is successfully processed, the counting result is combined with the cell category identification to form a complete cell identification and counting visualization result. According to the visualization result display function of the Gradio framework, the cell identification and counting visualization results are displayed on the interface, and the identification of each category of cells and the corresponding count values ​​are intuitively presented. If the visualization results need to be saved, the cell identification and counting visualization results are saved in the form of an image file through the result saving interface provided by the Gradio framework. According to actual needs, determine whether the visualization results need to be further analyzed or applied. If necessary, the visualization results are passed to subsequent analysis modules or applications for processing.

[0202] Specifically, when the output results are visualized using the Gradio framework, the detected image and the identification and counting results of each category of cells are passed into the Gradio interface component to achieve the visualization transformation of the target detection results. The image processing function provided by the Gradio framework is used to count the visual output detection result image. If the counting function is successfully processed, the counting result is combined with the cell category identification to form a complete cell identification and counting visualization result. Finally, according to the visualization result display function of the Gradio framework, the cell identification and counting visualization results are intuitively presented in the form of an interface, showing the identification of each category of cells and the corresponding count values, and judging whether the visualization results need to be saved as an image file or passed to the subsequent analysis module for further processing according to actual needs.

[0203] Optionally, the method of using the optimal multi-cell type target detection model to perform detection operations on different types of cell images of the image to be detected, and before outputting the target detection result, further includes:

[0204] Obtaining the image to be detected uploaded by the user terminal; and / or,

[0205] Obtain the image to be detected copied and pasted by the user; and / or,

[0206] Get the image to be detected that the user takes and sends through the camera.

[0207] Specifically, obtain the picture to be detected uploaded by the user terminal. If the user has not uploaded the picture, determine whether the user has copied and pasted the picture to be detected. If the user has not copied and pasted the picture, the picture to be detected is obtained by calling the camera to shoot. The obtained picture to be detected is preprocessed, including adjusting the image size to 640*640, and converting the picture to the input format required by the YOLOv8 model to obtain the preprocessed picture data to be detected. Load the weight file best.pt of the optimal multi-class cell detection and counting model trained in the above steps, and use the model to perform inference detection on the preprocessed picture data to be detected to obtain the cell detection result. Parse the cell detection results, extract the position coordinates, confidence and other information of each category of cells detected, and according to the preset confidence threshold, filter out the cell detection box with a confidence higher than the threshold, and determine the final cell detection result. According to the cell detection results, count the cells of each category, count the number of cells of each category, and obtain the cell counting result. The cell detection results and cell counting results are visualized, and the detected cell frame is drawn on the original image to be detected, and the cell category and confidence are marked next to the frame. At the same time, the number statistics of cells of each category are marked on the image. The Gradio framework is used to compare and display the visualized cell detection and counting result images with the original images, and a user-friendly interactive interface is generated to facilitate users to view and analyze cell detection and counting results. If the user is not satisfied with the detection and counting results, the functions provided by the interface can be used to manually adjust the cell detection frame or delete the wrong detection frame, correct the cell category or number, and obtain the corrected cell detection and counting results. The final cell detection and counting results are exported in the format required by the user, such as exporting to image files, table files, or report files, so that users can conduct subsequent analysis and application.

[0208] Specifically, users can upload local images through the mobile app, copy and paste the image address through the web page of the personal computer (PC), or take pictures through the device camera and upload them. The system will automatically determine the source of the image and obtain the image data. The obtained image will first be adjusted to a uniform size of 640*640 and converted to an RGB three-channel array format. Then, the trained YOLOv8 cell detection model is loaded, and the image array is used as input for inference prediction to obtain the detection results containing cell position coordinates, categories and confidence. According to the preset confidence threshold of 0.8, the detection box with higher confidence is selected. Then, the category labels of the detection box are traversed, and the number of cells in each category is accumulated and counted. The detection box and category labels are drawn on the original image, and the statistical results of the number of cells in each category are added to the image. Gradio is used to build an interactive interface to display the result image side by side with the original image. Users can modify the detection box or cell category through interface operations, such as merging overlapping detection boxes, splitting adherent cells, etc., to obtain the corrected detection results. Finally, the detection results are exported as Excel tables, including the location, category, and confidence data of each cell, as well as the number of cells in each category. Users can use these data for subsequent analysis.

[0209] The embodiment of the present invention also provides a multi-category cell recognition and counting system based on a convolutional neural network, which is used to implement the multi-category cell recognition and counting method based on a convolutional neural network described above, such as Figure 6 As shown, the multi-category cell recognition and counting system based on convolutional neural network provided in this embodiment includes:

[0210] An acquisition module 401 is used to obtain an initial model for detecting multiple cell types;

[0211] The multi-cell type target detection initial model includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through a convolution operation, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling on a spatial dimension, a channel dimension, and a feedback connection to enhance feature extraction capability;

[0212] An optimization processing module 402 is used to optimize the initial model for multi-cell type target detection to obtain an optimized model for multi-cell type target detection;

[0213] The multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module; the convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight;

[0214] A training module 403 is used to train the multi-cell type target detection optimization model using a training set to obtain an optimal multi-cell type target detection model;

[0215] The detection module 404 is used to use the optimal multi-cell type target detection model to perform detection operations on different types of cell images in the image to be detected, and output the target detection result.

[0216] In summary, a multi-category cell recognition and counting method and system based on a convolutional neural network disclosed in an embodiment of the present invention specifically solves the problem of recognition and counting of different types of cells under a biological microscope. The method first obtains and optimizes the real-time target detection algorithm structure, introduces a convolution operation enhancement module, a feature interaction capability improvement module, and a feature extraction enhancement module to improve model performance. Then, by integrating public sample data sets and self-made T-lymphocyte category data sets, data preprocessing and division are performed to form a training set, a test set, and a verification set. The optimization model is trained using the training set, and the optimal model is obtained through verification set evaluation and parameter adjustment. Finally, the method can detect, identify, and count multiple types of cells for images to be detected uploaded or taken by users, and output visualization results. The present invention realizes high-precision, real-time multi-cell type target detection, providing a powerful tool for biomedical research and clinical diagnosis.

[0217] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-category cell recognition and counting method based on convolutional neural network, characterized in that: include: Obtain an initial model for multi-cell type target detection; The multi-cell type target detection initial model includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through a convolution operation, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling on a spatial dimension, a channel dimension, and a feedback connection to enhance feature extraction capability; The multi-cell type target detection initial model includes a modularly designed YOLOv8 target detection model; Optimizing the initial model for multi-cell type target detection to obtain an optimized model for multi-cell type target detection; The multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module; The convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight; The feature interaction capability improvement module is used to dynamically adjust the feature fusion strategy according to input data and task requirements; The feature extraction enhancement module is used to improve the cell feature extraction capability of the backbone network; Using the training set to train the multi-cell type target detection optimization model to obtain an optimal multi-cell type target detection model; The optimal multi-cell type target detection model is used to perform different types of cell image detection operations on the image to be detected, and the target detection result is output.

2. The multi-category cell recognition and counting method based on convolutional neural network according to claim 1, characterized in that: Before the multi-cell type target detection optimization model is trained using the training set to obtain the optimal multi-cell type target detection model, the method further includes: Acquire a public sample data set, preprocess the public sample data set, and obtain an updated sample data set; Added self-made dataset; And, merging and preprocessing the self-made data set and the updated sample data set to obtain a merged data set; dividing the merged data set to obtain a divided data set; The partitioned data set includes the training set.

3. The multi-category cell recognition and counting method based on convolutional neural network according to claim 2, characterized in that: The divided data set also includes a test set and a validation set, and the dividing the merged data set to obtain the divided data set includes: Using a random number generator to randomly shuffle all the data of the merged data set; The data in the merged data set is divided into different data sets according to a preset allocation ratio to obtain the training set, the test set and the validation set.

4. The multi-category cell recognition and counting method based on convolutional neural network according to claim 2, characterized in that: The step of obtaining a public sample data set and preprocessing the public sample data set to obtain an updated sample data set includes: Get public sample datasets; Extract image features and cell bounding box information from the public sample dataset, perform feature extraction, data enhancement, data cleaning, and label processing operations on the public sample dataset, and remove images with incorrect annotations or low quality and clarity to obtain an updated sample dataset.

5. The multi-category cell recognition and counting method based on convolutional neural network according to claim 2, characterized in that: Before adding the self-made dataset, it also includes: The homemade dataset was constructed by acquiring a T lymphocyte image dataset in a standard biological laboratory using an inverted biological microscope.

6. The multi-category cell recognition and counting method based on convolutional neural network according to claim 3, characterized in that: The dividing the data in the merged data set into different data sets according to a preset distribution ratio to obtain the training set, the test set and the validation set includes: According to a preset allocation ratio, the data in the merged data set is divided into different data sets by using a stratified sampling method to obtain the training set, the test set and the validation set; The deviation between the distribution ratio of each category of cells in the validation set and the test set and the distribution ratio of each category of cells in the training set is within a preset deviation range.

7. The multi-category cell recognition and counting method based on convolutional neural network according to claim 6, characterized in that: The method of using the training set to train the multi-cell type target detection optimization model to obtain the optimal multi-cell type target detection model includes: Set training parameters; The training parameters include input image size, batch size, number of training iterations, learning rate and optimizer; Using the training parameters and the training set to train the multi-cell type target detection optimization model to obtain a trained model weight file; Update the multi-cell type target detection optimization model according to the model weight file to obtain an optimal multi-cell type target detection model; The adopting the training parameters and using the training set to train the multi-cell type target detection optimization model to obtain a trained model weight file includes: In the process of training the multi-cell type target detection optimization model using the training set by adopting the training parameters, regularly evaluating the performance of the multi-cell type target detection optimization model by using the validation set to obtain loss function changes and evaluation indicators; Monitor the loss function change and the evaluation index exceeds the preset monitoring index, and adjust the training parameters; According to the evaluation results of the validation set in each period, the weight file with the highest accuracy is selected as the trained model weight file.

8. The multi-category cell recognition and counting method based on convolutional neural network according to claim 7, characterized in that: The step of updating the multi-cell type target detection optimization model according to the model weight file to obtain an optimal multi-cell type target detection model includes: Update the multi-cell type target detection optimization model according to the model weight file to obtain a trained and updated multi-cell type target detection optimization model; Using the trained and updated multi-cell type target detection optimization model and other cell detection models to respectively predict the images in the test set, and obtain the detected cell objects in the test set; By traversing the cell objects and counting them according to categories, a first test result of the multi-cell type target detection optimization model after training and updating to detect cells of each category on the test set is obtained; The first test result includes recognition performance parameters and counting performance parameters of each type of cells detected by the training and updated multi-cell type target detection optimization model on the test set; And, using other cell detection models to predict the images in the test set respectively, obtaining the detected cell objects in the test set, and obtaining the second test results of the other cell detection models detecting each category of cells in the test set by traversing the cell objects and counting them according to categories; The second test result includes recognition performance parameters and counting performance parameters of each type of cells detected by the other cell detection model on the test set; If the first test result is higher than the second test result, the updated multi-cell type target detection optimization model is trained as the optimal multi-cell type target detection model.

9. The multi-category cell recognition and counting method based on convolutional neural network according to claim 1, characterized in that: Also includes: Converting and outputting the target detection result into a visualization result; And, using a counting function to perform cell counting and recognition on multiple types of cell images, processing the visualization results, and visually outputting the identification of each type of cell and the identification of the counting results; The method further comprises: using the optimal multi-cell type target detection model to perform different types of cell image detection operations on the image to be detected, and outputting the target detection result before the target detection result is output; Obtain the image to be detected uploaded by the user terminal; And / or, obtaining the image to be detected copied and pasted by the user; And / or, obtain the picture to be detected that is taken and transmitted by the user through the camera.

10. A multi-category cell recognition and counting system based on convolutional neural network, characterized in that: include: An acquisition module is used to obtain an initial model for detecting targets of multiple cell types; The multi-cell type target detection initial model includes a real-time target detection algorithm structure, the real-time target detection algorithm structure includes a convolution block for performing feature extraction processing through a convolution operation, and the real-time target detection algorithm structure also includes a feature fusion module for performing pooling on a spatial dimension, a channel dimension, and a feedback connection to enhance feature extraction capabilities; the multi-cell type target detection initial model includes a modularly designed YOLOv8 target detection model; An optimization processing module, used for optimizing the initial model for multi-cell type target detection to obtain an optimized model for multi-cell type target detection; The multi-cell type target detection optimization model includes an optimized real-time target detection algorithm structure, the optimized real-time target detection algorithm structure includes a convolution operation enhancement module that replaces the convolution block, and the optimized real-time target detection algorithm structure also includes a feature interaction capability improvement module and a feature extraction enhancement module that replace the feature fusion module; The convolution operation enhancement module is used to enhance the convolution operation function by dynamically adjusting the convolution kernel weight; The feature interaction capability improvement module is used to dynamically adjust the feature fusion strategy according to input data and task requirements; The feature extraction enhancement module is used to improve the cell feature extraction capability of the backbone network; A training module, used to train the multi-cell type target detection optimization model using a training set to obtain an optimal multi-cell type target detection model; The detection module is used to use the optimal multi-cell type target detection model to perform different types of cell image detection operations on the image to be detected and output the target detection result.

Images