A method and system for identifying mold based on liquid-based cytology of cervical cells

Abstract

The present invention discloses a method and system for mold recognition based on liquid-based cytological preparation of cervical cells. Embodiments of the present invention are trained based on a Transformer neural network, and after passing the test, a mold recognition model is obtained. The image of the liquid-based cytological preparation of cervical cells after image processing is input into this model, and the mold recognition result of this image is output. Since the mold recognition model set in the embodiments of the present invention does not adopt the commonly used convolutional neural network structure, but is constructed by an improved Transformer neural network, it can accurately recognize the microorganisms mainly composed of molds with the morphology of long hyphae in the image, so as to accurately recognize the molds in the image based on the liquid-based cytological preparation of cervical cells.

Description

Technical Field

[0001] The present invention relates to the technical field of medical cell image processing, and particularly to a method and system for identifying mold based on liquid-based cervical cytology preparations. Background Art

[0002] Cervical cancer is the malignant tumor with the highest incidence and mortality rate in the global female reproductive health. Cervical cytology examination is the most common, economical and effective cervical cancer screening method at present, which can effectively reduce the incidence and mortality rate of cervical cancer. In cervical cytology screening, in addition to paying attention to squamous cell and glandular cell lesions related to cervical cancer, the situation of cervical microbial infection also needs to be noted. The current epidemiological characteristics of cervical microbial infection are that the infection rates of microorganisms related to sexual life such as trichomonas, mold and virus remain high, especially in young and middle-aged women aged 20-40. According to the literature report, the infection rate of cervical specific microorganisms is 12.6%-15.1%, mold 5.0%-6.1%, bacteria 3.6%-6.0%, trichomonas 2.5%-3.4%, HPV 2.6%-3.0%, and herpes 0.09%-0.90%. Cervical cytology examination can generally clearly detect the infected pathogen or the characteristic cell changes caused by the pathogen, and the method is simple and reliable, which has important guiding significance for clinical treatment.

[0003] Among them, as a microorganism species with a relatively high infection rate of cervical microorganisms, mold can cause vulvovaginal inflammatory diseases. The vast majority of cases of mold infection are caused by Candida albicans infection. Finding Candida hyphae or spores is an important indication for diagnosis. The pseudohyphae of Candida albicans are eosinophilic, gray-brown when degenerated, in a bamboo joint shape, with acute-angle branches; the spores are round or oval with a thin envelope, and budding phenomenon can be seen. The pseudohyphae and long-shaped spores are arranged along the longitudinal axis. Under the microscope, leukocyte nuclear fragments and coin-shaped squamous cells "strung up" by hyphae can be seen. When identifying mold, it is directly identified by observing liquid-based cervical cytology preparations. However, in the process of mold identification, affected by the staining environment, mold is often not easily observed on liquid-based cervical cytology preparations, and due to its extremely subtle characteristics, it often requires the observer to carefully distinguish it from thin strip impurities, and the number of molds on a preparation may be very small, which greatly increases the difficulty of locating and identifying mold. Therefore, there is an urgent need for a method that can efficiently identify mold to reduce the burden of observation.

[0004] With the development of medical imaging technology, the application of artificial intelligence technology in medical imaging technology has gradually transformed medical imaging from an auxiliary examination method into the most important clinical diagnosis and differential diagnosis method in modern medicine. The use of artificial intelligence technology to set up an automatic film reading system has gradually matured in the field of cervical cancer screening and begun to be applied. Currently, the automatic film reading system is generally trained based on neural networks and can identify various squamous cell and adenocarcinoma cell lesions related to cervical cancer. However, the recognition accuracy for microorganisms mainly composed of molds still needs to be improved. This is because molds have many long and disordered hyphae, and the pixel ratio they occupy in the entire image of cervical cell liquid-based preparation is very small, which is significantly different from the common cell morphology. The commonly used convolutional neural network adopted by the automatic film reading system performs convolution on the image in a sliding window manner, which is more suitable for identifying images with continuous target pixels concentrated in a certain area. Therefore, it has an inherent disadvantage in recognizing the long and thin hyphae of molds.

[0005] In summary, how to accurately identify molds in images based on cervical cell liquid-based preparation has become an urgent problem to be solved. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a method for identifying molds based on cervical cell liquid-based preparation, which can accurately identify molds in images based on cervical cell liquid-based preparation.

[0007] An embodiment of the present invention also provides a system for identifying molds based on cervical cell liquid-based preparation, which can accurately identify molds in images based on cervical cell liquid-based preparation.

[0008] The embodiment of the present invention is implemented as follows:

[0009] A method for identifying molds based on cervical cell liquid-based preparation, the method includes:

[0010] Set up a mold recognition model, which is trained based on the Transformer neural network and obtained after passing the test;

[0011] Perform image processing on the cervical cell liquid-based preparation to obtain an image of the cervical cell liquid-based stained slide;

[0012] Input the obtained image of the cervical cell liquid-based stained slide into the mold recognition model, and output the mold recognition result of the image.

[0013] Preferably, the training based on the Transformer neural network includes:

[0014] Provide a positive sample set of images based on cervical cell liquid-based stained slides, and a negative sample set of images based on cervical cell liquid-based stained slides;

[0015] Perform data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set using a preset data augmentation method;

[0016] Use the processed positive sample set and negative sample set to train the structure of the mold recognition model.

[0017] Preferably, the process of passing the test based on the Transformer neural network includes:

[0018] Provide a test sample set based on the images of cervical cytology liquid-based stained slides;

[0019] Perform data augmentation processing on the test sample set using a preset data augmentation method;

[0020] Use the processed test sample set to test the trained mold recognition model. After the test meets the preset mold recognition probability value, the test passes.

[0021] Preferably, the image processing of the cervical cytology liquid-based preparation includes:

[0022] Obtain the image of the cervical cytology liquid-based preparation;

[0023] Perform processing using a preset data augmentation method on the obtained image of the cervical cytology liquid-based preparation.

[0024] Preferably, the data augmentation processing using a preset data augmentation method includes:

[0025] Data augmentation methods such as rotating the image in a set manner, changing the color of the image, or / and performing blurring processing on the image;

[0026] Or / and, select one of a batch of images as a data augmentation sub-strategy, and determine the best strategy based on a search algorithm to obtain the data augmentation method of the image;

[0027] Or / and, a data augmentation method of cropping the image to a set size.

[0028] Preferably, the structure of the mold recognition model is: a convolutional layer, a Transformer layer with multiple Transformer blocks, and a fully connected layer, where,

[0029] The convolutional layer performs convolutional processing;

[0030] The Transformer model is connected in series among multiple Transformer blocks. Among them, the Transformer block connected to the convolutional layer processes the convolved image and then outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer;

[0031] During the process of each Transformer block processing the image: the image is sequentially subjected to linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing;

[0032] The fully connected layer classifies the image processed by the Transformer layer.

[0033] Preferably, the linear regularization processing includes: performing linear regularization processing on the image twice;

[0034] The self-attention mechanism processing includes: calculating the horizontal self-attention and the vertical self-attention respectively. After obtaining the horizontal self-attention mechanism feature and the vertical self-attention base station feature, adding these two attention features to obtain the attention feature of the image;

[0035] The residual processing is performed twice;

[0036] The fully connected layer includes a classifier for the microorganism recognition task and a classifier for the mold recognition task. After classifying the image processed by the Transformer layer respectively, the classification result for the microorganism recognition task and the classification result for the mold recognition task obtained are used to calculate the loss using the set binary cross-entropy loss function to obtain the mold recognition result.

[0037] A mold recognition system based on cervical cytology liquid-based preparation, the system includes: a training model unit, an image processing unit, and a model processing unit, where,

[0038] The training model unit is used to set the mold recognition model, and the model is trained based on the Transformer network and obtained after passing the test;

[0039] The image processing unit is used to perform image processing on the cervical cytology liquid-based preparation to obtain a digital image of the cervical cytology liquid-based stained slide;

[0040] The model processing unit is used to input the image of the cervical cytology liquid-based stained slide obtained into the mold recognition model and output the mold recognition result of the image.

[0041] Preferably, the training model unit is further configured to provide a positive sample set based on the images of cervical cytology liquid-based stained slides, and a negative sample set based on the images of cervical cytology liquid-based stained slides; perform data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set by using a preset data augmentation method; use the processed positive sample set and negative sample set to train the structure of the mold recognition model.

[0042] The training model unit is further configured to provide a test sample set based on the images of cervical cytology liquid-based stained slides; perform data augmentation processing on the test sample set by using a preset data augmentation method; use the processed test sample set to test the trained mold recognition model, and the test passes after the test meets a preset mold recognition probability value.

[0043] Preferably, the structure of the mold recognition model is: a convolutional layer, a Transformer layer with multiple Transformer blocks, and a fully connected layer, wherein the convolutional layer performs convolutional processing.

[0044] The Transformer model is connected in series among multiple Transformer blocks. After the Transformer block connected to the convolutional layer processes the convolved image, it outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer.

[0045] In the process of each Transformer block processing the image: perform linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing on the image in sequence.

[0046] The fully connected layer performs classification processing on the image processed by the Transformer layer.

[0047] As can be seen above, the embodiments of the present invention are trained based on a Transformer neural network, and after passing the test, a mold recognition model is obtained. The image of the cervical cytology liquid-based preparation after image processing is input into this model, and the mold recognition result of this image is output. Since the mold recognition model set in the embodiments of the present invention does not adopt the commonly used convolutional neural network structure, but is constructed by an improved Transformer neural network, it can accurately identify the microorganisms mainly composed of molds with the morphology of long hyphae in the image, so as to accurately identify the molds in the image based on the cervical cytology liquid-based preparation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of the mold recognition method based on cervical cytology liquid-based preparation provided by the embodiments of the present invention;

[0049] Figure 2 It is a schematic diagram of the training samples and test samples adopted by the mold recognition model provided by the embodiments of the present invention;

[0050] Figure 3 It is a schematic diagram of the process of performing self-attention mechanism processing provided by the embodiments of the present invention

[0051] Figure 4 It is a schematic diagram of the structure of the mold recognition model provided by the embodiments of the present invention;

[0052] Figure 5 It is a flowchart of the construction of the mold recognition model provided by the embodiments of the present invention;

[0053] Figure 6 It is a schematic diagram of the structure of the mold recognition system based on cervical cytology liquid-based preparation provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0055] In the description and claims of the present invention and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0056] The technical solution of the present invention will be described in detail below with specific embodiments. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0057] As can be seen from the background art, the main reason why the automatic film reading system cannot accurately identify molds in images based on cervical cytology liquid-based preparations is that: the automatic film reading system uses the common convolutional neural network to perform convolution on the image in a sliding window manner and does not recognize molds with long and slender hyphal forms in the image. Therefore, in order to overcome the above problems, in the embodiment of the present invention, when constructing the mold recognition model, it is trained based on the Transformer neural network and obtained after passing the test. The image of the cervical cytology liquid-based preparation after image processing is input into the model, and the mold recognition result of the image is output. Since the mold recognition model set in the embodiment of the present invention is not constructed by using the common convolutional neural network, but is constructed by using an improved Transformer neural network, the improved Transformer neural network can accurately identify microorganisms mainly composed of molds with long hyphal forms in the image, so that the molds in the image based on cervical cytology liquid-based preparations can be accurately identified.

[0058] It can be seen that the embodiment of the present invention improves the structure of the Transformer model by integrating artificial intelligence technology and pathological knowledge, constructs a mold recognition model in cervical liquid-based cytology, and further promotes the implementation and application of the automatic film reading system for cervical liquid-based cytology.

[0059] Figure 1 The following is a flowchart of the mold recognition method based on cervical cytology liquid-based preparations provided by the embodiment of the present invention, and the specific steps include:

[0060] Step 101, set a mold recognition model, and the model is obtained by training based on the Transformer neural network and passing the test;

[0061] Step 102: Perform image processing on the liquid-based preparation of cervical cells to obtain an image of the liquid-based stained cervical cell slide.

[0062] Step 103: Input the obtained image of the liquid-based stained cervical cell slide into the mold identification model, and output the mold identification result of this image.

[0063] In this method, the training based on the Transformer neural network includes:

[0064] Provide a positive sample set based on the images of liquid-based stained cervical cell slides, and a negative sample set based on the images of liquid-based stained cervical cell slides;

[0065] Perform data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set by using a preset data augmentation method;

[0066] Use the processed positive sample set and negative sample set to train the structure of the mold identification model.

[0067] In this method, the process of passing the test based on the Transformer neural network includes:

[0068] Provide a test sample set based on the images of liquid-based stained cervical cell slides;

[0069] Perform data augmentation processing on the test sample set by using a preset data augmentation method;

[0070] Use the processed test sample set to test the trained mold identification model. After the test meets the preset mold identification probability value, the test passes.

[0071] Here, the structure of the mold identification model is: a convolutional layer, a Transformer layer with multiple Transformer blocks, and a fully connected layer, where

[0072] The convolutional layer performs convolutional processing;

[0073] The Transformer model is composed of multiple Transformer blocks connected in series. Among them, the Transformer block connected to the convolutional layer processes the convolved image and then outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer;

[0074] In the process of each Transformer block processing the image: the image is successively subjected to linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing;

[0075] The fully connected layer classifies the image processed by the Transformer layer.

[0076] Here, the cascading between multiple Transformer blocks in the Transformer layer is based on 3*1 and 1*3 convolutional connections. In this way, the Transformer blocks constructed in the embodiments of the present invention perform relationship modeling and global feature synthesis on the image, improving the recognition accuracy of molds in cervical cell smears.

[0077] Among them, the linear regularization processing includes: performing two linear regularization processes on the image; the self-attention mechanism processing includes: calculating the horizontal self-attention and the vertical self-attention respectively, and after obtaining the horizontal self-attention mechanism feature and the vertical self-attention base station feature, adding these two attention features to obtain the attention feature of the image. The residual processing is performed twice.

[0078] Among them, the fully connected layer includes a classifier for the microorganism recognition task and a classifier for the mold recognition task. After classifying the image processed by the Transformer layer respectively, the classification result for the microorganism recognition task and the classification result for the mold recognition task obtained are used to calculate the loss using the set binary cross-entropy loss function to obtain the mold recognition result, that is, the positive sample result and the negative sample result.

[0079] It can be seen that the fully connected layer better integrates the judgment criteria of pathologists into the classification task training through the multi-task learning method, and integrates the rootless structure in the image as a kind of supervision information into the model training process, further improving the accuracy and robustness of the model.

[0080] In this method, the image processing of the cervical cell liquid-based smear includes:

[0081] Obtaining an image of the cervical cell liquid-based smear;

[0082] Processing the obtained image of the cervical cell liquid-based smear by a preset data augmentation method.

[0083] Here, the preset data augmentation method includes:

[0084] Data augmentation methods of rotating the image in a set manner, changing the color of the image, and performing blurring processing on the image;

[0085] Or / and, select one from a batch of images as a data augmentation sub-strategy, and determine the best strategy based on a search algorithm to obtain the data augmentation method of the images;

[0086] Or / and, a data augmentation method of cropping the image to a set size.

[0087] As can be seen from the above method, the mold recognition model based on the improved Transformer neural network proposed in the embodiment of the present invention can not only extract the local features of the image, but also more conveniently perform relationship modeling and global feature synthesis on the features at different spatial positions, thereby further improving the accuracy of mold recognition.

[0088] The following gives a specific example to illustrate the process of training based on the Transformer neural network and obtaining the mold recognition model after passing the test, which specifically includes the following steps:

[0089] The first step: Preparation of image samples of cervical cytology liquid-based preparations.

[0090] In this step, in order to accurately identify the molds in the images and exclude the interference of impurities such as mucus filaments similar to the morphology of molds, professional pathologists annotate the mold image set and the impurity image set similar to the morphology of molds at the same time. The above image sets are divided into a training set and a test set according to a ratio of 8:2, as the training sample set and the test sample set of the mold recognition model.

[0091] Since the mycelium of the mold will differentiate into root-like structures when it contacts the nutrient matrix of the cells, in the training sample set and the test sample set, they are respectively divided into three parts according to whether the sample has root-like structures. One part is the positive sample with root-like structures, one part is the negative sample with root-like structures, and one part is the negative sample without root-like structures. The specific example samples are as Figure 2 shown, Figure 2 which is a schematic diagram of the training samples and test samples adopted by the mold recognition model provided by the embodiment of the present invention. The left image in the figure represents the positive sample with root-like structures, that is, the sample of the mold; the middle image in the figure represents the negative sample with root-like structures, that is, the sample of one of the impurities; the right image in the figure represents the negative sample without root-like structures, that is, the sample of another impurity. In this example, there are 6479 images in the positive sample set with root-like structure features in the training set, 3977 images in the negative sample set with root-like structure features, and 17490 images in the negative sample set without root-like structure features; in the test sample set, there are 1618 images in the positive sample set with root-like structure features, 990 images in the negative sample set with root-like structure features, and 4372 images in the negative sample set without root-like structure features.

[0092] Second step: Preprocess the prepared image samples.

[0093] In this step, data augmentation is very important for improving the recognition accuracy of the mold recognition model. Three data augmentation methods are used for the images:

[0094] a. Conventional data augmentation methods such as randomly rotating from -180 degrees to 180 degrees, randomly flipping vertically, randomly flipping horizontally, randomly changing colors, and randomly applying Gaussian blur.

[0095] b. Automatic data augmentation method. Automatic data augmentation means randomly selecting a data augmentation sub-strategy for each image in each batch and using a search algorithm to find the best strategy to extract the image.

[0096] c. According to the characteristic that the scales of the sampled images vary greatly, a data increase and decrease method of randomly cropping large images is designed. The process is as follows: First, judge the aspect ratio of the training image. If the aspect ratio is too large or too small (less than the preset ratio), then discard this image. If both the length and width of the image are greater than a set preset length and width value, randomly crop an area larger than one-fourth of the length and width as the image.

[0097] Third step: Training of the mold recognition model

[0098] In this step, considering the characteristic that microorganisms have root-like structures under stained slides, a mold recognition model is set up for training through an improved Transformer neural network. The Transformer neural network directly divides the input image into N S*S regions, where N is a natural number and S is the number of pixel points, and then processes it using the self-attention mechanism to obtain the output result. And the embodiment of the present invention improves the Transformer neural network with multiple Transformer blocks. Specifically, the structure in each Transformer Block is set. After two linear regularizations, one depthwise separable convolution process, and one self-attention mechanism process, it is output through two residual connection processes. Among them, in the self-attention mechanism process, the horizontal self-attention and the vertical self-attention are calculated respectively. After obtaining the horizontal self-attention mechanism feature and the vertical self-attention mechanism feature, these two attention features are added to obtain the attention feature of the image. The above process is as Figure 3 shown, Figure 3 which is a schematic diagram of the process of the self-attention mechanism provided by the embodiment of the present invention.

[0099] Specifically, combined with Figure 4 , Figure 4 which is a schematic diagram of the structure of the mold recognition model provided by the embodiment of the present invention, the construction process of the mold recognition model provided by the embodiment of the present invention is described. As Figure 5 shown,Figure 5 This is the flowchart for constructing a mold recognition model provided by an embodiment of the present invention, and its specific process includes:

[0100] In the first step, an image with a size of H*W*3 passes through N convolutional kernels with a size of 3*3 to obtain a feature map with a size of (H / 2)*(W / 2)*N; where H is the length of the image, in pixel values, W is the width of the image, in pixel values, and N is a natural number, generally set to 96.

[0101] In the second step, the obtained feature map passes through n Transformer Blocks, and each Block is connected by 3*1 and 1*3 convolutions. Where n is a natural number and can be 3.

[0102] In this step, using 3*1 and 1*3 convolutions to connect between Transformer Blocks in the mold recognition model can enable the model to better adapt to the unique root-like structure characteristics of molds. At the same time, Transformer Blocks can not only extract image features through horizontal and vertical self-attention mechanisms, but also more conveniently perform relationship modeling and global feature synthesis on the features, ensuring further improvement in the accuracy of mold recognition.

[0103] In the third step, classify and perform multi-task training on the mold recognition model.

[0104] In this step, according to the characteristics of molds, multiple tasks are established, including: 1) First, fuse the set diagnostic criteria, take the presence or absence of a root-like structure as an important feature, and set a classifier for the presence or absence of a root-like structure and an auxiliary loss module; 2) Set a sample classifier to perform positive and negative sample classification on the entire image.

[0105] Through multi-task training, the set diagnostic criteria can be better fused, and the classification of the presence or absence of a root-like structure is added as a constraint condition to the training of the mold recognition model to assist in obtaining better recognition accuracy for molds. The specific implementation method is: the classification vectors output by the mold recognition model are respectively input into different fully connected networks, and then two classification scores are obtained through a logistic regression layer (softmax), one representing the probability of a positive sample and the other representing the probability of having a root-like structure. In the training stage of the mold recognition model, for an image, it is simultaneously judged whether it is a microorganism and whether it has a root-like structure, and the binary cross-entropy loss function is used to supervise the training. In the testing stage of the mold recognition model, it is directly judged according to the probability of the positive sample. If the probability of the positive sample is greater than the preset probability threshold p, it is considered correct, otherwise it is judged that there is no microorganism.

[0106] In this step, the formula of the binary cross-entropy loss function adopted is as follows:

[0107]

[0108] Here, x is the prediction vector, and gt is the index corresponding to the true label.

[0109] Figure 6 The figure is a schematic structural diagram of a mold recognition system based on liquid-based preparation of cervical cells provided by an embodiment of the present invention. The system includes: a training model unit, an image processing unit, and a model processing unit. Among them,

[0110] The training model unit is used to set a mold recognition model, and the model is trained based on a Transformer neural network and obtained after passing the test;

[0111] The image processing unit is used to perform image processing on the liquid-based preparation of cervical cells to obtain an image of the liquid-based stained slide of cervical cells;

[0112] The model processing unit is used to input the obtained image of the liquid-based stained slide of cervical cells into the mold recognition model and output the mold recognition result of the image.

[0113] In this system, the training model unit is further used to provide a positive sample set based on the image of the liquid-based stained slide of cervical cells and a negative sample set based on the image of the liquid-based stained slide of cervical cells; perform data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set by using a preset data augmentation method; use the processed positive sample set and negative sample set to train the structure of the mold recognition model.

[0114] In this system, the training model unit is further used to provide a test sample set based on the image of the liquid-based stained slide of cervical cells; perform data augmentation processing on the test sample set by using a preset data augmentation method; use the processed test sample set to test the trained mold recognition model, and if the test meets a preset mold recognition probability value, the test passes.

[0115] In this system, the structure of the mold recognition model is: a convolutional layer, a Transformer layer with multiple Transformer blocks, and a fully connected layer. Among them, the convolutional layer performs convolutional processing;

[0116] Multiple Transformer blocks in the Transformer layer are connected in series. Among them, after the Transformer block connected to the convolutional layer processes the convolved image, it outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer;

[0117] In the process of each Transformer block processing the image: the image is sequentially subjected to linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing;

[0118] The fully connected layer classifies the image processed by the Transformer layer.

[0119] As can be seen from the above methods and systems provided by the embodiments of the present invention, the embodiments of the present invention construct a mold recognition model integrating professional knowledge in the medical field, which can efficiently identify mold cells and reduce the burden on doctors; the embodiments of the present invention improve the recognition accuracy of molds in cervical cell preparations, and further promote the implementation and practical application of the automatic image reading system based on cervical cell liquid-based preparations.

[0120] The flowcharts and block diagrams in the accompanying drawings of the present application illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments disclosed in the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in the order marked in different drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0121] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope disclosed in the present application.

[0122] In this article, specific embodiments are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention, and is not used to limit the present application. For those skilled in the art, based on the ideas, spirit and principles of the present invention, changes can be made in the specific implementation manners and application scopes, and any modifications, equivalent replacements, improvements, etc. made by them shall be included in the scope protected by the present application.

Claims

1. A method for identifying mold based on liquid-based cytology of cervical cells, characterized in that, The method includes: Setting up a mold recognition model, which is trained based on the Transformer neural network and obtained after passing the test; the structure of the mold recognition model includes a convolutional layer, a Transformer layer, and a fully connected layer; the steps of each Transformer block in the Transformer layer for processing an image include: sequentially performing linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing on the image; the linear regularization processing includes: performing two linear regularization processes on the image; the self-attention mechanism processing includes: calculating the horizontal self-attention and the vertical self-attention respectively, obtaining the horizontal self-attention mechanism feature and the vertical self-attention base station feature, and then adding these two attention features to obtain the attention feature of the image; the residual processing is performed twice; the fully connected layer includes a classifier for the microorganism recognition task and a classifier for the mold recognition task. After classifying the image processed by the Transformer layer respectively, the classification result for the microorganism recognition task and the classification result for the mold recognition task obtained are used to calculate the loss using the set binary cross-entropy loss function to obtain the mold recognition result; Performing image processing on the cervical cytology liquid-based preparation to obtain an image of the cervical cytology liquid-based stained slide; Inputting the obtained image of the cervical cytology liquid-based stained slide into the mold recognition model, and outputting the mold recognition result of the image.

2. The recognition method according to claim 1, wherein Training based on the Transformer neural network includes: Providing a positive sample set based on the images of cervical cytology liquid-based stained slides and a negative sample set based on the images of cervical cytology liquid-based stained slides; Performing data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set using a preset data augmentation method; Using the processed positive sample set and negative sample set to train the structure of the mold recognition model.

3. The recognition method according to claim 1, wherein The process of passing the test based on the Transformer neural network includes: Providing a test sample set based on the images of cervical cytology liquid-based stained slides; Performing data augmentation processing on the test sample set using a preset data augmentation method; Using the processed test sample set to test the trained mold recognition model. After the test meets the preset mold recognition probability value, the test passes.

4. The recognition method according to claim 1, wherein, The performing image processing on the cervical cytology liquid-based preparation includes: Obtaining an image of the cervical cytology liquid-based preparation; Performing processing using a preset data augmentation method on the obtained image of the cervical cytology liquid-based preparation.

5. The recognition method according to claim 2, 3 or 4, characterized in that, The performing data augmentation processing using a preset data augmentation method includes: A data augmentation method of rotating the image in a set manner, changing the color of the image, or / and performing blurring processing on the image; Or / and, selecting one of a batch of images as a data augmentation sub-strategy, and determining the best strategy based on a search algorithm to obtain the data augmentation method of the image; Or / and, a data augmentation method of cropping the image to a set size.

6. The recognition method according to claim 1, wherein the convolutional layer performs convolutional processing; the Transformer layer is connected in series among multiple Transformer blocks. After the Transformer block connected to the convolutional layer processes the convolved image, it outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer.

7. A mold recognition system based on liquid-based preparation of cervical cells, characterized in that, The system includes: a training model unit, an image processing unit, and a model processing unit, wherein The training model unit is used to set up a mold recognition model, which is trained based on the Transformer network and obtained after passing the test; the structure of the mold recognition model includes a convolutional layer, a Transformer layer, and a fully connected layer; the steps for each Transformer block in the Transformer layer to process the image include: sequentially performing linear regularization processing, convolutional processing, self-attention mechanism processing, and residual processing on the image; the linear regularization processing includes: performing two linear regularization processes on the image; the self-attention mechanism processing includes: respectively calculating the horizontal self-attention and the vertical self-attention, obtaining the horizontal self-attention mechanism feature and the vertical self-attention base station feature, and then adding these two attention features to obtain the attention feature of the image; the residual processing is performed twice; the fully connected layer includes a classifier for the microorganism recognition task and a classifier for the mold recognition task. After classifying the image processed by the Transformer layer respectively, the classification result for the microorganism recognition task and the classification result for the mold recognition task obtained are used to calculate the loss using the set binary cross-entropy loss function to obtain the mold recognition result The image processing unit is used to perform image processing on the liquid-based smear of cervical cells to obtain a digital image of the liquid-based stained smear of cervical cells; The model processing unit is used to input the obtained image of the liquid-based stained smear of cervical cells into the mold recognition model and output the mold recognition result of the image.

8. The system according to claim 7, wherein, The training model unit is further used to provide a positive sample set based on the image of the liquid-based stained smear of cervical cells and a negative sample set based on the image of the liquid-based stained smear of cervical cells; perform data augmentation processing on the positive samples in the positive sample set and the negative samples in the negative sample set using a preset data augmentation method; Use the processed positive sample set and negative sample set to train the structure of the mold recognition model; The training model unit is further used to provide a test sample set based on the image of the liquid-based stained smear of cervical cells; perform data augmentation processing on the test sample set using a preset data augmentation method; Using the processed test sample set, the trained mold recognition model is tested. After the test meets the preset mold recognition probability value, the test passes.

9. The system according to claim 7, wherein In the mold recognition model for setting the mold recognition model, the convolutional layer performs convolutional processing; the Transformer layer is connected in series between multiple Transformer blocks. Among them, the Transformer block connected to the convolutional layer processes the convolved image and then outputs the image processed by the Transformer block. Other Transformer blocks process the image processed by the previous connected Transformer block until the Transformer block connected to the fully connected layer finishes processing and outputs the image processed by the Transformer layer.

Images