A recognition model training method for gynecological microorganism fluorescence images
By using residual network and bidirectional feature pyramid network in gynecological microbial fluorescence image recognition, combining knowledge distillation and data preprocessing, the problems of insufficient recognition accuracy and large model scale are solved, and efficient and accurate microbial identification and classification are achieved. for medical equipment with limited resources.
Patent Information
- Application Number
- CN202410796305.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-06-20
AI Technical Summary
The prior art has insufficient recognition accuracy, limited classification range, and large scale of the recognition model in the recognition of gynecological microbial fluorescent images, making it difficult to design an intelligent recognition system that can fully cover all types of microbial cells and maintain high accuracy.
Using residual network as the backbone network and bidirectional feature pyramid network as the neck network, combined with the head network design of classifiers and regressors, a compact student network is refined from a large-scale teacher network through knowledge distillation, combining data preprocessing and Image enhancement strategy, optimized comprehensive loss function to improve recognition accuracy and positioning accuracy.
Significantly enhance the model's ability to extract complex image features and integrate multi-level information, maintain recognition accuracy and greatly reduce the model size, improve the robustness and generalization ability of image changes, and is suitable for resource-constrained environments. , such as portable medical devices and remote diagnostic systems.
Smart Images

Figure CN118629038B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a recognition model training method and system for gynecological microorganism fluorescence images, and a gynecological microorganism fluorescence image recognition and classification method. Background Art
[0002] With the rapid development of medical technology, precision medicine has become one of the core pursuits in today's medical field. In the diagnosis and treatment of gynecological diseases, accurate identification and classification of vaginal microbial communities is an important part of assessing the health status of patients. Traditional microbial detection methods, such as smear microscopy, are highly dependent on the experience and skills of pathologists. They are not only time-consuming and labor-intensive, but also have problems with diagnostic inconsistency and high error rates due to human factors. Especially when faced with a large number of samples, this manual-dependent detection method is particularly inefficient and unreliable, which can easily lead to a waste of medical resources and may affect the timely diagnosis and treatment of patients.
[0003] In recent years, with the rapid progress of computer vision, machine learning, especially deep learning technology, its application in the field of medical image analysis has become increasingly widespread, providing a new way to solve the above problems. By automatically learning features from massive data, deep learning models can achieve efficient recognition and classification of complex images, significantly improving the accuracy and efficiency of diagnosis. In the field of gynecological microbiology, fluorescence imaging technology has become an ideal tool for capturing microbial morphological characteristics due to its high sensitivity and specificity. Combined with deep learning algorithms, it makes it possible to realize automated and intelligent recognition and classification of gynecological microbial cells.
[0004] However, although some studies have explored the possibility of applying computer technology to gynecological microbial image analysis, most of these technologies are in the preliminary stage and have problems such as limited recognition accuracy, narrow classification range, and large recognition model size. In particular, given the complexity of gynecological microbial diversity, how to design an intelligent recognition system that can comprehensively cover various types of microbial cells (such as epithelial cells, lactobacilli, different types of white blood cells and bacteria, etc.) while maintaining a high degree of accuracy, while minimizing the size of the recognition model, has become the key to current technological breakthroughs.
[0005] Therefore, there is an urgent need to develop a technology that can intelligently identify and accurately classify multiple microorganisms in gynecological microbial fluorescence images. Summary of the invention
[0006] 1. Technical issues to be resolved
[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a recognition model training method and system for gynecological microorganism fluorescence images, and a gynecological microorganism fluorescence image recognition and classification method, which solve the technical problems of insufficient accuracy, limited classification range, and large recognition model scale in gynecological microorganism fluorescence image recognition.
[0008] (II) Technical solution
[0009] In order to achieve the above object, the main technical solutions adopted by the present invention include:
[0010] In a first aspect, an embodiment of the present invention provides a method for training a recognition model for gynecological microorganism fluorescence images, comprising:
[0011] S10, obtaining an initial image set of gynecological microorganism fluorescence, screening and cleaning the initial image set, and annotating the screened and cleaned image set to obtain a training image set with annotated information;
[0012] S20, training the pre-constructed model architecture to be trained according to the training image set, and using a preset loss function to calculate the loss value between the output result of the model and the corresponding true label, until the loss value is less than a preset threshold, to obtain a trained first gynecological microorganism fluorescence image recognition model;
[0013] The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism;
[0014] S30, constructing a student network model to be trained, taking the first gynecological microorganism fluorescence image recognition model as the teacher network model, performing knowledge distillation operation using the teacher network model, training the student network model to be trained, and obtaining a second gynecological microorganism fluorescence image recognition model;
[0015] The scale of the second gynecological microorganism fluorescence image recognition model is smaller than the first gynecological microorganism fluorescence image recognition model.
[0016] Optionally, the S30 includes:
[0017] S31, constructing a student network model to be trained;
[0018] S32, using the trained first gynecological microorganism fluorescence image recognition model as a teacher network model, and selecting the middle layer used for distillation in the teacher network model as a distillation layer;
[0019] S33. For each selected distillation layer, use the same input x to pass through the teacher network and the student network respectively to obtain the feature Ft =f t (x) and F s =f s (x), and according to the preset conversion function Φ t and Φ s , the feature F t =f t (x) and F s =f s (x) are transformed respectively to obtain Φ t (F t ) and Φ s (F s ), and the knowledge transfer formula is used to calculate the feature distillation loss L between the teacher network model and the student network model FeaD (F t ,F s );
[0020] The knowledge transfer formula is:
[0021] L FeaD (F t ,F s )=L F (Φ t (F t ),Φ s (F s ));
[0022] Among them, L FeaD (F t ,F s ) is the feature distillation loss between the teacher network model and the student network model, F t and F s are the features output by the teacher network model and the student network model, L F is a similarity calculation function for matching the teacher network and the student network;
[0023] S34. Use the total loss function to train the student network model until the total loss L total The value of approaches the preset value, and a trained student network model is obtained, namely, the second gynecological microorganism fluorescence image recognition model;
[0024] The expression of the total loss function is:
[0025] ;
[0026] Among them, L total is the total loss, α and β are the preset weight factors, L CE is the cross entropy loss between the teacher network and the student network, L FeaDIt is the feature distillation loss between the teacher network model and the student network model.
[0027] Optionally, after S10, the method further includes:
[0028] S10a, performing spatial transformation on each image in the image set with the annotation information to obtain a set of images after spatial transformation;
[0029] The set of images after spatial transformation includes all images before and after spatial transformation;
[0030] S10b, scaling or cropping the images in the image set after the spatial transformation, so that the sizes of all images are unified to a preset size;
[0031] S10c, scaling the pixel values in the image after the size is unified to between 0 and 1 or between -1 and 1, and converting the pixel values in the image into a distribution with a mean of 0 and a variance of 1.
[0032] Optionally, the stage of training the backbone network in S20 includes:
[0033] S21, input the image in the training image set with labeled information into the residual network, the image is first convolved with a convolution kernel of size 7x7 and stride 2 to preliminarily extract the low-level features of the image, the image size is halved and the number of channels is increased to 64 to obtain a feature map;
[0034] S22, normalize the input image data through the batch normalization layer, and increase the nonlinearity of the model through the ReLU activation function;
[0035] S23, the size of the feature map is further halved through the maximum pooling layer, and higher-level features are extracted;
[0036] S24, the feature map enters the residual block sequence, passes through multiple convolutional layers with a step size of 2, the feature map size is reduced, the number of channels is increased, and higher-level features are continuously extracted until the pre-set network depth of the backbone network is reached.
[0037] Optionally, the stage of training the neck network in S20 includes:
[0038] S25, the neck network receives feature maps of different scales output from the backbone network, starting from the high-level feature map, increases its size through upsampling, uses 1x1 convolution to adjust the number of channels to match the number of channels in the adjacent layer, and fuses the upsampled feature map with the feature map below. The low-level feature map is reduced in size through downsampling, and the number of channels is adjusted using 1x1 convolution, and then fused with the upper-level feature map;
[0039] S26. At each fusion point, feature maps from different scales are merged and integrated through lateral connections.
[0040] Optionally, the stage of training the head network in S30 includes:
[0041] S27, sending the merged feature map to the classifier, using the cross entropy loss function to measure the gap between the predicted probability distribution and the true category label, and estimating the probability of each predefined category appearing in a given feature map area;
[0042] S28, the merged feature map is fed into the regressor, and the regressor predicts the bounding box parameters of each potential target through the fully connected layer;
[0043] The bounding box parameters include the starting coordinates, width, and height of the bounding box;
[0044] S29. Calculate the classification loss function of the classifier and the regression loss function of the regressor respectively, and calculate the weighted sum of the classification loss function and the regression loss function according to the preset weights, that is, the total loss function. When the total loss function converges, the trained first gynecological microorganism fluorescence image recognition model is obtained.
[0045] Optionally, the annotation information includes location information and category information of the target microorganism in the image;
[0046] The categories of target microorganisms include one or more of epithelial cells, lactobacilli, leukocytes, microbacillum, hyphae, blastospores, spores, clue cells, cocci, trichomonas, naked nuclei, streptococci, diplococci, and bacilli aggregates.
[0047] In a second aspect, an embodiment of the present invention provides a method for identifying and classifying gynecological microorganism fluorescence images, comprising:
[0048] Z10, obtaining a fluorescent image of gynecological microorganisms to be identified and classified;
[0049] Z20. Input the gynecological microorganism fluorescence image to be identified and classified into the second gynecological microorganism fluorescence image recognition model. The residual network extracts multi-scale features from the gynecological microorganism fluorescence image and passes them to the bidirectional feature pyramid network. The bidirectional feature pyramid network fuses features of different levels through top-down and bottom-up paths, and transmits the fused features to the head network. Based on the fused feature information, the classifier in the head network predicts the category of the target microorganism in the image, and outputs the probability distribution of each category by comparing the features with the categories learned during training. The regressor in the head estimates the specific location of the target microorganism based on the feature information.
[0050] The second gynecological microorganism fluorescence image recognition model is obtained based on the recognition model training method for gynecological microorganism fluorescence images as described in the first aspect above;
[0051] Z30, output recognition and classification results according to the category probability distribution and the specific location of the target microorganism;
[0052] The identification and classification result includes category information of the microorganism, location information on the image, and a corresponding confidence score.
[0053] Optionally, after the Z30, the method further includes:
[0054] Z41, obtain all candidate boxes and corresponding confidence scores output by the second gynecological microorganism fluorescence image recognition model, for each candidate box b i , calculate the intersection-and-merge area ratio between it and the current highest scoring box M;
[0055] Z42, for each candidate box bi, use the Gaussian decay function to smoothly adjust the corresponding confidence score;
[0056] The expression of the Gaussian attenuation function is:
[0057]
[0058] Among them, S j is the candidate box b i After adjusting the confidence score; S i is the candidate box b i Confidence score before adjustment; iou(M,b i ) is the candidate box b i The intersection-and-combination area ratio with the current highest scoring box M; e is the base of the natural logarithm; σ is the preset hyperparameter for controlling the width of the Gaussian curve;
[0059] Z43, reorder the candidate boxes according to the adjusted scores, and delete the candidate boxes whose confidence scores are lower than the specified threshold;
[0060] Z44, select the candidate box with the highest current score as one of the final detection results, remove other candidate boxes whose intersection area ratio with the candidate box with the highest current score exceeds a preset threshold from the candidate box set, repeat steps Z42-Z43 until the number of retained candidate boxes is lower than the predetermined maximum detection number, and output the final retained candidate box;
[0061] Each output box carries the corresponding class label and an adjusted confidence score.
[0062] In a third aspect, an embodiment of the present invention provides a gynecological microorganism fluorescence image recognition and classification system, comprising:
[0063] A labeling module is used to obtain an initial image set of gynecological microorganism fluorescence, screen and clean the initial image set, and label the screened and cleaned image set to obtain a training image set with labeling information;
[0064] A first gynecological microorganism fluorescence image recognition model training module is used to train the pre-constructed model architecture to be trained according to the training image set, and use a preset loss function to calculate the loss value between the output result of the model and the corresponding true label until the loss value is less than a preset threshold, thereby obtaining a trained first gynecological microorganism fluorescence image recognition model;
[0065] The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism;
[0066] The second gynecological microorganism fluorescence image recognition model training module is used to construct a student network model to be trained, take the first gynecological microorganism fluorescence image recognition model as the teacher network model, use the teacher network model to perform knowledge distillation operation, train the student network model to be trained, and obtain the second gynecological microorganism fluorescence image recognition model;
[0067] The scale of the second gynecological microorganism fluorescence image recognition model is smaller than the first gynecological microorganism fluorescence image recognition model.
[0068] (III) Beneficial effects
[0069] The present invention provides a recognition model training method and system for gynecological microorganism fluorescence images, and a gynecological microorganism fluorescence image recognition and classification method. Due to the use of a residual network as a backbone network, a bidirectional feature pyramid network (BiFPN) as a neck network, and a head network design combining a classifier and a regressor, the model's capabilities in complex image feature extraction and multi-level information integration are significantly enhanced. In particular, by means of knowledge distillation, a more compact student network is refined from a large-scale teacher network, which not only maintains recognition accuracy, but also greatly reduces the model size, which is conducive to deployment in resource-constrained environments, such as portable medical devices and remote diagnosis systems.
[0070] In addition, through meticulous data preprocessing steps and image enhancement strategies, the recognition model training method for gynecological microorganism fluorescence images ensures robustness to image changes and improves the generalization ability of the model. In the classification and positioning tasks, the optimization strategy of the comprehensive loss function is used to achieve a balanced improvement in classification accuracy and positioning accuracy, especially in the recognition of images of various types of gynecological microorganisms, which can effectively distinguish and accurately determine the location of microorganisms, which is of great significance for clinical diagnosis.
[0071] In summary, the present invention not only overcomes the problems of insufficient recognition accuracy, limited classification range, and excessive model scale in the prior art, but also provides an efficient and feasible solution for the automated, high-precision recognition and classification of gynecological microorganisms through the deep integration of deep learning and medical image analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 A schematic flow chart of a method for training a recognition model for gynecological microorganism fluorescence images provided by an embodiment of the present invention;
[0073] Figure 2 A schematic diagram of a flow chart of a method for identifying and classifying gynecological microorganism fluorescence images provided by an embodiment of the present invention;
[0074] Figure 3 A schematic diagram of labeling a gynecological microorganism fluorescence image provided by an embodiment of the present invention;
[0075] Figure 4 A fluorescence image of gynecological microorganisms with bubbles is provided in one embodiment of the present invention;
[0076] Figure 5a An unrotated epithelial cell fluorescence image provided by an embodiment of the present invention;
[0077] Figure 5b A method provided by an embodiment of the present invention Figure 5a Fluorescence image of epithelial cells after rotation;
[0078] Figure 5c A method provided by an embodiment of the present invention Figure 5b Fluorescence image of epithelial cells after spatial distortion;
[0079] Figure 6 An embodiment of the present invention provides a method for Figure 5a Cell fluorescence images under different color illumination;
[0080] Figure 7 A fluorescence image of gynecological microorganisms produced by overexposure provided by an embodiment of the present invention;
[0081] Figure 8 A fluorescence image of hyphae provided by an embodiment of the present invention;
[0082] Fig. 9 A schematic diagram of top-down and bottom-up feature fusion in a neck network provided by an embodiment of the present invention;
[0083] Fig.10A schematic diagram of a recognition and classification result output by a second gynecological microorganism fluorescence image recognition model provided in one embodiment of the present invention;
[0084] Fig.11 A schematic diagram of selecting incomplete cells at the edge of an image in a post-processing stage according to an embodiment of the present invention. DETAILED DESCRIPTION
[0085] In order to better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein.
[0086] Embodiment 1
[0087] like Figure 1 As shown, this embodiment provides a recognition model training method for gynecological microorganism fluorescence images, which may include:
[0088] S10, obtaining an initial image set of gynecological microorganism fluorescence, screening and cleaning the initial image set, and annotating the screened and cleaned image set to obtain a training image set with annotated information.
[0089] The annotation information includes the location information and category information of the target microorganisms in the image;
[0090] The categories of target microorganisms include epithelial cells, lactobacilli, leukocytes, microbacillum, hyphae, blastospores, spores, clue cells, cocci, trichomonas, naked nuclei, streptococci, diplococci and bacilli aggregates.
[0091] Before labeling, the data is cleaned and screened: images that are not relevant to the detection task are removed, including background images, non-target category images, etc.; duplicate images and low-quality images are deleted, such as blurred, severely exposed, or noisy images; for example, when abnormal bubbles appear in the image (such as Figure 4 If the image is not displayed, delete it;
[0092] In this embodiment, a threshold value can be set according to image quality indicators (clarity, contrast, brightness, etc.) for automatic screening to ensure image quality.
[0093] In addition, after labeling, the labeling information can be used to filter images, retaining images containing target categories and excluding images with incorrect or incomplete labeling.
[0094] Missing value processing: For missing values in the data, appropriate strategies are adopted to fill them or delete them directly, which can be decided based on the specific situation.
[0095] The labeling process (e.g. Figure 3As shown in the figure): In this embodiment, in the initial stage, tens of thousands of images are manually annotated to accumulate a large amount of annotated data containing location and category information; in the mid-term, a semi-automatic annotation strategy is adopted, and the annotation is predicted by the initially trained model, and manually reviewed, verified and corrected to improve efficiency; in the later stage, after the model matures, only special cases or incorrect annotations that are difficult for the model to recognize are corrected by manual intervention. In the entire annotation process, the annotation consistency should be ensured to ensure that the same target is annotated in the same way in each image, reduce annotation differences, and facilitate effective model learning.
[0096] In order to enhance the generalization and robustness of the model, while reducing the reliance on a large amount of annotated data and preventing overfitting, a series of carefully designed data preprocessing and enhancement operations were performed on the original microbial cell image dataset:
[0097] Spatial transformation: By applying random scaling, shearing, rotation, and spatial distortion to the image, the actual observation conditions are simulated, so that the model learns the invariance to translation, scale change, rotation, and complex distortion. For example, the rotation and spatial distortion transformation of cell images (such as Figure 5a , Figure 5b , Figure 5c As shown in Figure 3), it not only increases data diversity, but also improves the distortion invariance of the model.
[0098] Resizing: To adapt to the fixed input size requirements of the model, intelligent scaling (such as bilinear interpolation, nearest neighbor interpolation) and cropping (random cropping, center cropping) are performed on the images to unify the size of all images to the preset size. This not only ensures feature integrity, but also enriches the diversity of the data set, ensuring that the model can effectively handle targets of different sizes and positions.
[0099] Lighting and color correction: Through image normalization and standardization, normalization is to scale the pixel values to a certain range (such as 0-1 or -1-1) so that the model can learn the features of the image more effectively. Standardization is to convert the pixel values into a distribution with a mean of 0 and a variance of 1 to reduce the dimensional differences between different features. Normalization and standardization can improve the convergence speed and stability of the model, so that the model can better adapt to different data sets and task requirements. It eliminates the effects caused by differences in light intensity and color (such as Figure 6 and Figure 7 As shown in Figure 2, the model is trained to focus on essential features rather than changing background conditions. This includes adjustments for overexposed images and images with inconsistent colors, ensuring that the model works stably under a variety of visual conditions.
[0100] Advanced preprocessing and enhancement: For specific microbial features, color space conversion (such as RGB to HSV / Lab), histogram equalization, filtering technology, etc. are used to further enhance image quality and improve image contrast and clarity, especially for categories with significant color differences (such as Figure 5a , Figure 5b , Figure 5c and Figure 8 These operations help the model to more accurately identify and distinguish different cell structures.
[0101] S20. According to the training image set, the pre-constructed model architecture to be trained is trained, and a preset loss function is used to calculate the loss value between the output result of the model and the corresponding true label until the loss value is less than a preset threshold, thereby obtaining a trained first gynecological microorganism fluorescence image recognition model.
[0102] The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism.
[0103] The stage of training the backbone network: the images in the training image set with labeled information are input into the residual network. The images are first convolved through a 7x7 convolution kernel with a stride of 2 to preliminarily extract the low-level features of the image, halve the image size and increase the number of channels to 64 to obtain the feature map; the input image data is normalized through the batch normalization layer, and the nonlinearity of the model is increased through the ReLU activation function; the size of the feature map is further halved through the maximum pooling layer, and higher-level features are extracted; the feature map enters the residual block sequence, and through multiple convolution layers with a stride of 2, the feature map size is reduced, the number of channels is increased, and higher-level features are continuously extracted until the pre-set network depth of the backbone network is reached.
[0104] Training phase of the neck network: The neck network receives feature maps of different scales output from the backbone network. Starting from the high-level feature map, its size is increased by upsampling, and the number of channels is adjusted using 1x1 convolution to match the number of channels in the adjacent layer. The upsampled feature map is fused with the feature map below. The low-level feature map is reduced in size by downsampling, and the number of channels is adjusted using 1x1 convolution, and then fused with the upper-level feature map. At each fusion point, the feature maps from different scales are merged and integrated through lateral connections (such as Fig. 9 as shown).
[0105] The stage of training the head network: the merged feature map is sent to the regressor, and the regressor predicts the bounding box parameters (starting coordinates, width and height) of each potential target through the fully connected layer; the classification loss function of the classifier and the regression loss function of the regressor are calculated respectively, and according to the preset weights, the weighted sum of the classification loss function and the regression loss function, that is, the total loss function, is calculated. When the total loss function converges, the trained first gynecological microorganism fluorescence image recognition model is obtained.
[0106] S30. Construct a student network model to be trained, use the first gynecological microorganism fluorescence image recognition model as the teacher network model, perform knowledge distillation operation using the teacher network model, train the student network model to be trained, and obtain a second gynecological microorganism fluorescence image recognition model.
[0107] The scale of the second gynecological microorganism fluorescence image recognition model is smaller than the first gynecological microorganism fluorescence image recognition model.
[0108] Specifically, a student network model to be trained is constructed;
[0109] The trained first gynecological microorganism fluorescence image recognition model is used as a teacher network model, and the middle layer used for distillation in the teacher network model is selected as a distillation layer;
[0110] For each selected distillation layer, the same input x is passed through the teacher network and the student network to obtain the feature F t =f t (x) and F s =f s (x), and according to the preset conversion function Φ t and Φ s , the feature F t =f t (x) and F s =f s (x) are transformed respectively to obtain Φ t (F t ) and Φ s (F s ), and the knowledge transfer formula is used to calculate the feature distillation loss L between the teacher network model and the student network model FeaD (F t ,F s );
[0111] The knowledge transfer formula is:
[0112] L FeaD (F t ,F s )=L F (Φ t (F t),Φ s (F s ));
[0113] Among them, L FeaD (F t ,F s ) is the feature distillation loss between the teacher network model and the student network model, F t and F s are the features output by the teacher network model and the student network model, L F is a similarity calculation function for matching the teacher network and the student network;
[0114] Use the total loss function to train the student network model until the total loss L total The value of approaches the preset value, and a trained student network model is obtained, namely, the second gynecological microorganism fluorescence image recognition model;
[0115] The expression of the total loss function is:
[0116] ;
[0117] Among them, L total is the total loss, α and β are the preset weight factors, L CE is the cross entropy loss between the teacher network and the student network, L FeaD It is the feature distillation loss between the teacher network model and the student network model.
[0118] Through knowledge distillation, a more compact student network is refined from a large-scale teacher network, which not only maintains the recognition accuracy but also greatly reduces the model size, which is conducive to deployment in resource-constrained environments such as portable medical devices and remote diagnosis systems.
[0119] In addition, two key technical strategies can be adopted to optimize the inference speed of the model: pruning technology and CUDA inference acceleration to ensure that the execution efficiency and deployment flexibility of the model are improved without sacrificing detection accuracy.
[0120] Embodiment 2
[0121] like Figure 2 As shown, this embodiment provides a gynecological microorganism fluorescence image recognition and classification method, which may include:
[0122] Z10. Obtain a fluorescent image of gynecological microorganisms to be identified and classified.
[0123] Z20. Input the gynecological microorganism fluorescence image to be identified and classified into the second gynecological microorganism fluorescence image recognition model. The residual network extracts multi-scale features from the gynecological microorganism fluorescence image and passes them to the bidirectional feature pyramid network. The bidirectional feature pyramid network fuses features of different levels through top-down and bottom-up paths, and transmits the fused features to the head network. Based on the fused feature information, the classifier in the head network predicts the category of the target microorganism in the image, and outputs the probability distribution of each category by comparing the features with the categories learned during training. The regressor in the head estimates the specific location of the target microorganism based on the feature information.
[0124] The second gynecological microorganism fluorescence image recognition model is obtained based on the recognition model training method for gynecological microorganism fluorescence images as described in the first embodiment.
[0125] Z30, outputs the recognition and classification results (such as Fig.10 as shown).
[0126] The identification and classification result includes category information of the microorganism, location information on the image, and a corresponding confidence score.
[0127] In this embodiment, the following method is used to filter the multiple rectangular candidate boxes output by the model:
[0128] Get all candidate boxes and corresponding confidence scores output by the second gynecological microorganism fluorescence image recognition model. For each candidate box b i , calculate the intersection-and-merge area ratio between it and the current highest scoring box M;
[0129] For each candidate box bi, a Gaussian decay function is used to smoothly adjust the corresponding confidence score;
[0130] The expression of the Gaussian attenuation function is:
[0131]
[0132] Among them, S j is the candidate box b i After adjusting the confidence score; S i is the candidate box b i Confidence score before adjustment; iou(M,b i ) is the candidate box b i The intersection-and-combination area ratio with the current highest scoring box M; e is the base of the natural logarithm; σ is the preset hyperparameter for controlling the width of the Gaussian curve;
[0133] Re-rank candidate boxes according to the adjusted scores and remove candidate boxes with confidence scores below the specified threshold;
[0134] Select the candidate box with the highest current score as one of the final detection results, remove other candidate boxes whose intersection area ratio with the candidate box with the highest current score exceeds a preset threshold from the candidate box set, repeat steps Z42-Z43 until the number of retained candidate boxes is lower than the predetermined maximum detection number, and output the final retained candidate box;
[0135] Each output box carries the corresponding class label and an adjusted confidence score.
[0136] Finally, the method also includes a post-processing stage, which is mainly for fine screening and optimization of specific application scenarios to ensure the accuracy and practicality of data analysis. The key operations in this stage include:
[0137] Edge cell screening: By reviewing the model prediction results, identify and remove those incomplete cells located at the edge of the image. Fig.11 As shown in Figure 2, even if the model successfully identifies the nucleus and some structures of epithelial cells, these predictions will be eliminated in the statistical analysis due to the incompleteness of the cell as a whole. This is because the edge cells may not provide a complete picture of the cell, affecting the correct assessment of the cell state or characteristics.
[0138] Associated deletion: When processing edge cells, not only incomplete cells should be removed, but also the cell components closely associated with them, such as the nucleus, should be deleted. This is because the identification and counting of nuclei should be based on the complete cell background to avoid erroneous statistics caused by cell fragments. For example, if the nucleus is identified in an incomplete epithelial cell, the nucleus marker will also be removed from the final analysis results to maintain the accuracy and consistency of the data.
[0139] Embodiment 3
[0140] This embodiment provides a recognition model training system for gynecological microorganism fluorescence images, including:
[0141] A labeling module is used to obtain an initial image set of gynecological microorganism fluorescence, screen and clean the initial image set, and label the screened and cleaned image set to obtain a training image set with labeling information;
[0142] A first gynecological microorganism fluorescence image recognition model training module is used to train the pre-constructed model architecture to be trained according to the training image set, and use a preset loss function to calculate the loss value between the output result of the model and the corresponding true label until the loss value is less than a preset threshold, thereby obtaining a trained first gynecological microorganism fluorescence image recognition model;
[0143] The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism;
[0144] The second gynecological microorganism fluorescence image recognition model training module is used to construct a student network model to be trained, take the first gynecological microorganism fluorescence image recognition model as the teacher network model, use the teacher network model to perform knowledge distillation operation, train the student network model to be trained, and obtain the second gynecological microorganism fluorescence image recognition model;
[0145] The scale of the second gynecological microorganism fluorescence image recognition model is smaller than the first gynecological microorganism fluorescence image recognition model.
[0146] Embodiment 4
[0147] This embodiment provides a computing device, including:
[0148] at least one processor; and a memory communicatively coupled to the at least one processor;
[0149] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the recognition model training method for gynecological microorganism fluorescence images described in the first embodiment of the claims.
[0150] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0151] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0152] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.
[0153] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0154] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A recognition model training method for gynecological microorganism fluorescence images, It is characterized in that include: S10, obtaining an initial image set of gynecological microorganism fluorescence, screening and cleaning the initial image set, and annotating the screened and cleaned image set to obtain a training image set with annotated information; S20, training the pre-constructed model architecture to be trained according to the training image set, and using a preset loss function to calculate the loss value between the output result of the model and the corresponding true label, until the loss value is less than a preset threshold, to obtain a trained first gynecological microorganism fluorescence image recognition model; The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism; S30, constructing a student network model to be trained, taking the first gynecological microorganism fluorescence image recognition model as the teacher network model, performing knowledge distillation operation using the teacher network model, training the student network model to be trained, and obtaining a second gynecological microorganism fluorescence image recognition model; The scale of the second gynecological microorganism fluorescence image recognition model is smaller than that of the first gynecological microorganism fluorescence image recognition model.
2. The recognition model training method for gynecological microorganism fluorescence images according to claim 1, It is characterized in that The S30 includes: S31, constructing a student network model to be trained; S32, using the trained first gynecological microorganism fluorescence image recognition model as a teacher network model, and selecting the middle layer used for distillation in the teacher network model as a distillation layer; S33. For each selected distillation layer, use the same input x to pass through the teacher network and the student network respectively to obtain the feature F t =f t (x) and F s =f s (x), and according to the preset conversion function Φ t and Φ s , the feature F t =f t (x) and F s =f s (x) are transformed respectively to obtain Φ t (F t ) and Φ s (F s ), and the knowledge transfer formula is used to calculate the feature distillation loss L between the teacher network model and the student network model FeaD (F t ,F s ); The knowledge transfer formula is: L FeaD (F t ,F s )=L F (F t (F t ),F s (F s )); Among them, L FeaD (F t ,F s ) is the feature distillation loss between the teacher network model and the student network model, F t and F s are the features output by the teacher network model and the student network model, L F is a similarity calculation function for matching the teacher network and the student network; S34. Use the total loss function to train the student network model until the total loss L total The value of approaches the preset value, and a trained student network model is obtained, namely, the second gynecological microorganism fluorescence image recognition model; The functional expression of the total loss is: ; Among them, L total is the total loss, α and β are the preset weight factors, L CE is the cross entropy loss between the teacher network and the student network, L FeaD It is the feature distillation loss between the teacher network model and the student network model.
3. The recognition model training method for gynecological microorganism fluorescence images according to claim 1, It is characterized in that After S10, the method further includes: S10a, performing spatial transformation on each image in the image set with the annotation information to obtain a set of images after spatial transformation; The set of images after spatial transformation includes all images before and after spatial transformation; S10b, scaling or cropping the images in the image set after the spatial transformation, so that the sizes of all images are unified to a preset size; S10c, scaling the pixel values in the image after the uniform size to between 0 and 1 or between -1 and 1, and converting the pixel values in the image into a distribution with a mean of 0 and a variance of 1.
4. The recognition model training method for gynecological microorganism fluorescence images according to claim 3, It is characterized in that The stage of training the backbone network in S20 includes: S21, input the image in the training image set with labeled information into the residual network, the image is first convolved with a convolution kernel of size 7x7 and stride 2 to preliminarily extract the low-level features of the image, the image size is halved and the number of channels is increased to 64 to obtain a feature map; S22, normalize the input image data through the batch normalization layer, and increase the nonlinearity of the model through the ReLU activation function; S23, the size of the feature map is further halved through the maximum pooling layer, and higher-level features are extracted; S24, the feature map enters the residual block sequence, passes through multiple convolutional layers with a step size of 2, the feature map size is reduced, the number of channels is increased, and higher-level features are continuously extracted until the pre-set network depth of the backbone network is reached.
5. The recognition model training method for gynecological microorganism fluorescence images according to claim 4, It is characterized in that The stage of training the neck network in S20 includes: S25, the neck network receives feature maps of different scales output from the backbone network, starting from the high-level feature map, increases its size through upsampling, uses 1x1 convolution to adjust the number of channels to match the number of channels in the adjacent layer, and fuses the upsampled feature map with the feature map below. The low-level feature map is reduced in size through downsampling, and the number of channels is adjusted using 1x1 convolution, and then fused with the upper-level feature map; S26. At each fusion point, feature maps from different scales are merged and integrated through lateral connections.
6. The recognition model training method for gynecological microorganism fluorescence images according to claim 5, It is characterized in that The stage of training the head network in S30 includes: S27, sending the merged feature map to the classifier, using the cross entropy loss function to measure the gap between the predicted probability distribution and the true category label, and estimating the probability of each predefined category appearing in a given feature map area; S28, the merged feature map is fed into the regressor, and the regressor predicts the bounding box parameters of each potential target through the fully connected layer; The bounding box parameters include the starting coordinates, width, and height of the bounding box; S29. Calculate the classification loss function of the classifier and the regression loss function of the regressor respectively, and calculate the weighted sum of the classification loss function and the regression loss function according to the preset weights, that is, the total loss function. When the total loss function converges, the trained first gynecological microorganism fluorescence image recognition model is obtained.
7. The recognition model training method for gynecological microorganism fluorescence images according to claim 1, It is characterized in that The annotation information includes the location information and category information of the target microorganisms in the image; The categories of target microorganisms include one or more of epithelial cells, lactobacilli, leukocytes, microbacillum, hyphae, spores, clue cells, cocci, trichomonas, naked nuclei, streptococci, diplococci, and bacilli aggregates.
8. A method for identifying and classifying gynecological microorganism fluorescence images. It is characterized in that include: Z10, obtaining a fluorescent image of gynecological microorganisms to be identified and classified; Z20. Input the gynecological microorganism fluorescence image to be identified and classified into the second gynecological microorganism fluorescence image recognition model. The residual network extracts multi-scale features from the gynecological microorganism fluorescence image and passes them to the bidirectional feature pyramid network. The bidirectional feature pyramid network fuses features of different levels through top-down and bottom-up paths, and transmits the fused features to the head network. Based on the fused feature information, the classifier in the head network predicts the category of the target microorganism in the image, and outputs the probability distribution of each category by comparing the fused features with the categories learned during training. The regressor in the head estimates the specific location of the target microorganism based on the feature information. Wherein, the second gynecological microorganism fluorescence image recognition model is obtained based on the recognition model training method for gynecological microorganism fluorescence images according to any one of claims 1 to 7; Z30, output recognition and classification results according to the category probability distribution and the specific location of the target microorganism; The identification and classification result includes category information of the microorganism, location information on the image, and a corresponding confidence score.
9. The gynecological microorganism fluorescence image recognition and classification method according to claim 8, It is characterized in that The Z30 also includes: Z41, obtain all candidate boxes and corresponding confidence scores output by the second gynecological microorganism fluorescence image recognition model, for each candidate box b i , calculate the intersection-and-merge area ratio between it and the current highest scoring box M; Z42, for each candidate box bi, use the Gaussian decay function to smoothly adjust the corresponding confidence score; The expression of the Gaussian attenuation function is: ; Among them, S j is the candidate box b i After adjusting the confidence score; S i is the candidate box b i Confidence score before adjustment; iou(M,b i ) is the candidate box b i The intersection-and-combination area ratio with the current highest scoring box M; e is the base of the natural logarithm; σ is the preset hyperparameter for controlling the width of the Gaussian curve; Z43, reorder the candidate boxes according to the adjusted scores, and delete the candidate boxes whose confidence scores are lower than the specified threshold; Z44, select the candidate box with the highest current score as one of the final detection results, remove other candidate boxes whose intersection area ratio with the candidate box with the highest current score exceeds a preset threshold from the candidate box set, repeat steps Z42-Z43 until the number of retained candidate boxes is lower than the predetermined maximum detection number, and output the final retained candidate box; Each output box carries the corresponding class label and an adjusted confidence score.
10. A recognition model training system for gynecological microbial fluorescence images, It is characterized in that include: A labeling module is used to obtain an initial image set of gynecological microorganism fluorescence, screen and clean the initial image set, and label the screened and cleaned image set to obtain a training image set with labeling information; A first gynecological microorganism fluorescence image recognition model training module is used to train the pre-constructed model architecture to be trained according to the training image set, and use a preset loss function to calculate the loss value between the output result of the model and the corresponding true label until the loss value is less than a preset threshold, thereby obtaining a trained first gynecological microorganism fluorescence image recognition model; The backbone network of the model architecture to be trained is a residual network, the neck network of the model architecture to be trained is a bidirectional feature pyramid network, and the head network of the model architecture to be trained includes a classifier for determining the category of the target microorganism and a regressor for predicting the position of the target microorganism; The second gynecological microorganism fluorescence image recognition model training module is used to construct a student network model to be trained, take the first gynecological microorganism fluorescence image recognition model as the teacher network model, use the teacher network model to perform knowledge distillation operation, train the student network model to be trained, and obtain the second gynecological microorganism fluorescence image recognition model; The scale of the second gynecological microorganism fluorescence image recognition model is smaller than that of the first gynecological microorganism fluorescence image recognition model.
Citation Information
Patent Citations
Training method of image recognition model and image recognition method
CN112016591A
Fluorescent staining solution and application thereof
CN112461630A
Oil tank image recognition model training method, oil tank image recognition method and device
CN113592033A
Three-stage pest image recognition method based on adaptive feature fusion pyramid network
CN114565048A
System and method for classifying and segmenting microscopy images with deep multiple instance learning
US20180137338A1
