A cell recognition method and system in a cell image

Abstract

The present invention discloses a method and system for identifying cells in a cell image. The embodiment of the present invention trains a classification neural network, which is composed of a sub-backbone network, a sub-feature pyramid network (FPN, feature pyramid networks), a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit. When the classification neural network receives a cell image, after extracting the features of the cell image by the sub-backbone, a multi-scale feature is generated through FPN, and the multi-scale features are respectively input to the cell classification branch unit for cell classification to obtain cell classification branch features, and input to the cell nucleus detection branch unit for cell nucleus region identification and obtain cell nucleus detection branch features; the collaborative analysis unit is used to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification result of the cell image. In this way, the embodiment of the present invention can accurately identify cells in cell images based on cell smears.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular to a method and system for identifying cells in a cell image. Background Art

[0002] Cervical cancer is a disease with a clear cause that can be detected and prevented early. Since the advent of cervical cytology Pap pictures, cytology examinations have played an indelible role in the early detection of cervical cancer. However, due to the shortage of cervical cytology personnel, limited experience in reading cervical cell smears, and the high pressure of reading a large number of films, the readers are tired and subjective, resulting in low cell-specific recognition of cervical cell smears, and a certain amount of missed diagnoses and misdiagnoses. With the rapid development of computer vision technology and the increasing maturity of digital image processing technology, the use of computer-assisted screening systems to perform quantitative and qualitative analysis of cell images based on cell images, especially cell images based on cervical cell images, to identify cell types, and to assist doctors in screening for cervical cancer lesions based on cell images has gradually become a research focus.

[0003] At present, computer-aided screening systems mainly conduct pathological quantitative analysis on cell smears or cell images of tissue sections of specific parts to obtain characteristic parameters of the degree of cell pathology, thereby judging whether the cells are pathological and obtaining classification results for pathological cells. The key technology of computer-aided screening systems is to accurately identify cells in cell images and confirm whether pathology has occurred.

[0004] Computer-aided systems usually classify and identify cells in cell images based on convolutional neural networks. However, due to the differences in the preparation and staining methods of cervical cell smears, the complexity of the background, the diversity and irregularity of cell morphology, and the overlap between cells, the superposition of multiple factors makes it difficult to identify cells in cell images. On the other hand, when the computer-aided system extracts cell features in cell images to achieve cell classification, the feature description of cervical cells is not comprehensive, and it is impossible to accurately classify cervical cells in cell images based on cervical cell smears.

[0005] Therefore, how to accurately identify cells in cell images based on cell smears, especially how to accurately identify cervical cells in cell images based on cervical cell smears, has become a problem that needs to be solved urgently. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method for identifying cells in a cell image, which can accurately identify cells in a cell image based on a cell smear.

[0007] The embodiment of the present invention further provides a cell classification system in a cell image, which can accurately identify cells in a cell image based on a cell smear.

[0008] The embodiment of the present invention is implemented as follows:

[0009] A method for identifying cells in a cell image, comprising:

[0010] A classification neural network is obtained by training, wherein the classification neural network is composed of a sub-backbone network, a sub-feature pyramid FPN, a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit;

[0011] When the classification neural network receives a cell image, the sub-backbone extracts the features of the cell image, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region recognition to obtain cell nucleus detection branch features;

[0012] The classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, so as to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; and uses a collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, so as to obtain the cell classification result of the cell image.

[0013] Preferably, the cell classification branch unit is implemented by using a first sub-convolutional neural network with a spatial attention mechanism to focus on the cells in the cell image and identify and obtain cell classification branch features;

[0014] The cell nucleus detection branch unit is implemented by using a second sub-convolutional neural network with an attention mechanism. By detecting the cell nucleus area, the cell nucleus area in the cell image is explicitly focused on to identify the cell nucleus detection branch features.

[0015] Preferably, the collaborative analysis unit is implemented using a third sub-convolutional neural network with an attention mechanism, which guides and fuses the cell nucleus detection branch features according to the cell classification branch features, and guides and fuses the cell classification branch features according to the cell nucleus detection branch features.

[0016] Preferably, the training of a classification neural network includes:

[0017] Acquire cell image samples, and label the cell category in each cell image sample;

[0018] In the cell image samples with labeled cell categories, part of the cell nucleus area is labeled;

[0019] Input the cell image samples with the cell categories and partial cell nucleus regions marked into the set classification neural network;

[0020] The sub-backbone in the classification neural network extracts the features of the cell image based on the cell category annotations and partial cell nucleus region annotations of the cell image sample, and then generates multi-scale features through FPN. The multi-scale features are respectively input into the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and are input into the cell nucleus detection branch unit for cell nucleus region recognition and training to obtain cell nucleus detection branch features;

[0021] The classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification results; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification results of the cell image.

[0022] Preferably, the cell image is a cell image based on a cervical cell smear.

[0023] A cell recognition system in a cell image, the system comprising: a training unit, a storage unit and a processing unit, wherein:

[0024] A training unit, used for training to obtain a classification neural network, wherein the classification neural network is composed of a sub-backbone, a sub-FPN, a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit;

[0025] A storage unit, used for storing the trained classification neural network;

[0026] The processing unit is used to obtain the classification neural network from the storage unit when receiving the cell image, extract the features of the cell image by the sub-backbone, generate multi-scale features through FPN, and input the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and input them to the cell nucleus detection branch unit for cell nucleus area recognition to obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus area in the cell and the cell nucleus classification result; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification result of the cell image.

[0027] Preferably, the processing unit is also used for the cell classification branch unit to be implemented by a first sub-convolutional neural network with a spatial attention mechanism, which pays attention to the cells in the cell image and identifies the cell classification branch features; the cell nucleus detection branch unit is implemented by a second sub-convolutional neural network with an attention mechanism, which detects the cell nucleus area, explicitly pays attention to the cell nucleus area in the cell image, and identifies the cell nucleus detection branch features.

[0028] Preferably, the processing unit is also used for the collaborative analysis unit to be implemented using a third sub-convolutional neural network with an attention mechanism, to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and to guide and fuse the cell classification branch features according to the cell nucleus detection branch features.

[0029] Preferably, the training unit is also used for the training to obtain the classification neural network, including: obtaining cell image samples, and marking the cell category in each cell image sample; marking part of the cell nucleus region in the cell image samples with the cell category marked; inputting the cell image samples with the cell category marked and the part of the cell nucleus region marked into the set classification neural network; the sub-backbone in the classification neural network extracts the features of the cell image according to the cell category marking and the part of the cell nucleus region marking of the cell image sample, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region identification and training to obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; uses the collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification result of the cell image.

[0030] Preferably, the cell image is a cell image based on a cervical cell smear.

[0031] As shown above, the classification neural network is trained by the embodiment of the present invention, and the classification neural network is composed of a sub-backbone network (backbone), a sub-feature pyramid network (FPN, feature pyramid networks), a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit. When the classification neural network receives a cell image, after the sub-backbone extracts the features of the cell image, the multi-scale features are generated through FPN, and the multi-scale features are respectively input to the cell classification branch unit for cell classification to obtain cell classification branch features, and input to the cell nucleus detection branch unit for cell nucleus region identification and obtain cell nucleus detection branch features; the collaborative analysis unit is used to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification results; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification results of the cell image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flow chart of a method for identifying cells in a cell image provided by an embodiment of the present invention;

[0033] Figure 2 A flow chart of a training method for a convolutional neural network with an attention mechanism unit provided in an embodiment of the present invention;

[0034] Figure 3 A flowchart of a specific example process of training a convolutional neural network provided in an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the structure of a cell recognition system in a cell image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0038] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0039] In the background art, when a computer-aided system based on a convolutional neural network is used to identify cells in a cell image, it is not possible to accurately identify the cells in the cell image, especially it is not possible to accurately identify cervical cells in a cell image based on a cervical cell smear. In fact, when doctors identify cells in a cell smear, the morphology and color of the cell nucleus are important reference indicators, such as the size of the cell nucleus, the depth of the nucleus, and the nuclear-cytoplasmic ratio. The essence of using a convolutional neural network to identify cell images is to obtain the cell nucleus focus area of ​​each cell image through a large amount of data training, and classify it, so as to train a convolutional neural network for subsequent cell image cell recognition. However, the focus area in the cell image is actually predicted, and there is instability and unexplainability, which leads to the inability of the trained convolutional neural network to accurately obtain the position of the cell nucleus area in the cell image when it is actually applied.

[0040] In order to solve the above problems, the classification neural network is trained in the embodiment of the present invention, and the classification neural network is composed of a sub-backbone network (backbone), a sub-feature pyramid network (FPN, feature pyramid networks), a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit. When the classification neural network receives a cell image, after the sub-backbone extracts the features of the cell image, the multi-scale features are generated through FPN, and the multi-scale features are respectively input to the cell classification branch unit for cell classification to obtain cell classification branch features, and input to the cell nucleus detection branch unit for cell nucleus region identification and obtain cell nucleus detection branch features; the collaborative analysis unit is used to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification result of the cell image.

[0041] In this way, the classification neural network trained by the embodiment of the present invention classifies the cell nucleus region in the cell image, thereby introducing an explicit cell nucleus attention mechanism to improve the accuracy of cell classification in the cell image, and subsequently adding a collaborative analysis unit to guide and fuse the cell classification branch features and the cell nucleus detection branch features, thereby obtaining accurate cell classification results of the cell image and obtaining the location information of the cell nucleus region in the cell and the cell nucleus classification results. Especially for cell images based on cervical cell smears, cervical cells in cell images based on cervical cell smears can be accurately classified and identified.

[0042] Therefore, the embodiment of the present invention can accurately identify cells in a cell image based on a cell smear.

[0043] In addition, in practical applications, the difficulty of object detection data annotation far exceeds that of image classification data set annotation, especially in the field of cell recognition, this problem is more obvious. Therefore, the classification neural network provided by the embodiment of the present invention performs collaborative training on cell classification and cell nucleus region recognition in cell images, and only requires annotation information of cell nucleus regions in some cell images during training, which greatly reduces the labor cost of annotation.

[0044] Figure 1 A flow chart of a method for identifying cells in a cell image provided by an embodiment of the present invention, wherein the specific steps include:

[0045] Step 101, training to obtain a classification neural network, wherein the classification neural network is composed of a sub-backbone, a sub-FPN, a cell classification branch unit, a cell nucleus detection branch unit, and a collaborative analysis unit;

[0046] Step 102, when the classification neural network receives a cell image, the sub-backbone extracts the features of the cell image, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region recognition to obtain cell nucleus detection branch features;

[0047] Step 103, the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, so as to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; and uses a collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, so as to obtain the cell classification result of the cell image.

[0048] In the method, the cell image is a cell image based on a cervical smear.

[0049] In this method, the cell classification branch unit is implemented using a first sub-convolutional neural network with a spatial attention mechanism, which focuses on the cells in the cell image and identifies the cell classification branch features.

[0050] The cell nucleus detection branch unit is implemented by using a second sub-convolutional neural network with an attention mechanism. By detecting the cell nucleus area, the cell nucleus area in the cell image is explicitly focused on to identify the cell nucleus detection branch features.

[0051] In this method, the collaborative analysis unit is implemented using a third sub-convolutional neural network with an attention mechanism, which guides and fuses the cell nucleus detection branch features according to the cell classification branch features, and guides and fuses the cell classification branch features according to the cell nucleus detection branch features.

[0052] In this method, the training of a classification neural network includes:

[0053] Acquire cell image samples, and label the cell category in each cell image sample;

[0054] In the cell image samples with labeled cell categories, part of the cell nucleus area is labeled;

[0055] Input the cell image samples with the cell categories and partial cell nucleus regions marked into the set classification neural network;

[0056] The sub-backbone in the classification neural network extracts the features of the cell image based on the cell category annotations and partial cell nucleus region annotations of the cell image sample, and then generates multi-scale features through FPN. The multi-scale features are respectively input into the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and are input into the cell nucleus detection branch unit for cell nucleus region recognition and training to obtain cell nucleus detection branch features;

[0057] The classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification results; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification results of the cell image.

[0058] When identifying cells in a cell image, the cell nucleus in the cell image is an important reference indicator for judging the degree of cell pathology. The embodiment of the present invention is to set a cell nucleus detection branch function in the classification neural network, introduce the cell nucleus region as an important feature to be processed by the classification neural network, thereby improving the recognition accuracy. The classification neural network provided by the embodiment of the present invention also has a collaborative analysis function, which can well integrate the cell classification branch features and cell nucleus detection branch features in the cell image, that is, integrate the cell nucleus region features and the cell global features, improve the generalization and accuracy of the branch neural network, and the classification neural network supports end-to-end training and semi-supervised training during training, reduces the cost of manual annotation, and supports the simultaneous output of the cell classification results of the cell image and the position information of the cell nucleus region in the cell and the cell nucleus classification results.

[0059] The embodiments of the present invention can be applied to most practical scenarios, not just medical images, and have high use value. Based on a large number of reliable cell image samples and the support of deep learning technology, the trained classification neural network has excellent generalization ability, which can provide a more accurate reference for the final reading of cell images and improve the accuracy of the interpretation of cell smears collected by cell images.

[0060] Figure 2 A flow chart of a training method for a convolutional neural network with an attention mechanism unit provided in an embodiment of the present invention, wherein the specific steps include:

[0061] Step 201: Obtain cell image samples and label the cell category in each cell image sample;

[0062] Step 202: in the cell image sample with the cell category labeled, labeling a part of the cell nucleus region;

[0063] Step 203, inputting the cell image samples with the cell category and the partial cell nucleus region annotated into the set classification neural network; the sub-backbone in the classification neural network extracts the features of the cell image according to the cell category annotations and the partial cell nucleus region annotations of the cell image samples, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region recognition and training to obtain cell nucleus detection branch features;

[0064] In this step, the classification neural network is pre-built based on the expert’s foreknowledge of film reading experience information;

[0065] Step 204, the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; uses a collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification result of the cell image.

[0066] In the above method, the implementation of step 203 and step 204 is as follows Figure 3 As shown, Figure 3 A flowchart of a specific example process of training a convolutional neural network provided in an embodiment of the present invention.

[0067] In the above method, the classification neural network set up includes a cell classification branch unit and a cell nucleus detection branch unit. The cell classification branch unit is implemented by a convolutional neural network with a first subspace attention mechanism, and the cell nucleus detection branch unit is implemented by a convolutional neural network with a second subchannel attention mechanism. By detecting the cell nucleus area, the cell nucleus area in the cell image is explicitly paid attention to. Therefore, the classification neural network takes into account the global features in the cell image when setting it up, captures the long-distance dependencies of the cell image, and can effectively improve the accuracy and robustness of cell classification and recognition in the cell image when introducing fewer cell image samples.

[0068] The constructed and trained classification neural network is described in detail below.

[0069] Step 1: Construct a classification neural network, the structure of which includes backbone and FPN.

[0070] In this step, the classification neural network includes a backbone for extracting features of cell images. In order to combine multi-scale features, an FPN is also set in the classification neural network in the embodiment of the present invention. In this way, the influence of factors such as differences in scanner magnifications and large differences in resolution of different types of cells on the final classification accuracy of cell images when acquiring cell images can be greatly reduced.

[0071] Step 2: The constructed classification neural network also includes two attention mechanism units, which are used to detect the cell classification branches and the cell nucleus area in the cell image respectively.

[0072] In this step, when classifying cells in a cell image, the set classification neural network is required not only to learn the local features of the cells in the cell image, but also to learn the global features of the cells in the cell image. The convolutional neural network provided by the background technology extracts cell features in a cell image in a stacked manner, and can only learn the local features of the cells in the cell image, which affects the lack of diversity in the extraction of cell features in the cell image. In this step, the classification neural network introduces two attention mechanism units, which are implemented using a spatial attention mechanism and a channel attention mechanism respectively, so as to model the global context information of the cell image, enhance the representation ability of the classification neural network in extracting cell features in the cell image, and improve the diversity of extraction when extracting cell features in the cell image, which can effectively improve the accuracy and robustness of cell classification and recognition.

[0073] Step 3: The cell nucleus detection branch unit in the constructed classification neural network is implemented using a convolutional neural network with a spatial attention mechanism.

[0074] In this step, the nucleus morphology and color in the cell image are an important reference quality assurance for doctors to accurately classify cells. Although the classification neural network includes two attention mechanism units, which are used to classify the cell branches in the cell image and detect the nucleus region, the important regions predicted by the network in the cell image still have some instability and unexplainability, which leads to the lack of stable focus on learning the nucleus region in the cell image in the constructed classification neural network, resulting in the lack of some important prior knowledge. Therefore, in order to better utilize the nucleus features in the cell image to assist the classification neural network in accurately classifying cells, the convolutional neural network with a spatial attention mechanism is used to detect the nucleus region in the classification neural network, that is, to introduce supervision information in the classification neural network, so as to prompt the classification neural network to focus on the nucleus region in the cell image.

[0075] During implementation, a convolutional neural network with a spatial attention mechanism is set in the classification neural network. The cell features extracted by FPN are used as input and processed in the convolutional neural network with a spatial attention mechanism to obtain the cell nucleus features in the cell image, including the location information of the cell nucleus area, the cell nucleus classification results, the cell category information and the confidence score.

[0076] In this step, the convolutional neural network with spatial attention mechanism also includes a loss module for predicting the cell nucleus region, thereby accurately predicting the cell nucleus region.

[0077] Step 4: The constructed classification neural network includes a collaborative analysis unit for collaboratively analyzing the cell classification branch characteristics and the cell nucleus detection branch characteristics.

[0078] In this step, before outputting the final result based on the cell classification branch features and the cell nucleus detection branch features, the cell classification branch features and the cell nucleus detection branch features are guided and fused through the set collaborative analysis unit, and the cell nucleus detection branch features containing cell nucleus supervision information are used to guide the cell classification branch features to focus on the cell nucleus area, so that the cell classification branch can learn more local information of the cell nucleus area; at the same time, the cell classification branch features are used to guide and fuse the cell nucleus detection branch features, so that the cell nucleus detection branch features can learn more global features.

[0079] Step 5: The constructed classification neural network simultaneously obtains the cell classification result of the cell image and the position information of the cell nucleus region in the cell and the cell nucleus classification result.

[0080] In this step, the constructed classification neural network is a neural network that supports end-to-end training. Different from the convolutional neural network in the background technology that first captures the cell nucleus features in the cell image and then performs multi-step training on the captured cell nucleus features, the constructed classification neural network can simultaneously obtain the cell classification results of the cell image and the position information of the cell nucleus region in the cell and the cell nucleus classification results.

[0081] Figure 4 A schematic diagram of a cell recognition system in a cell image provided by an embodiment of the present invention includes: a training unit, a storage unit and a processing unit, wherein:

[0082] A training unit, used for training to obtain a classification neural network, wherein the classification neural network is composed of a sub-backbone, a sub-FPN, a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit;

[0083] A storage unit, used for storing the trained classification neural network;

[0084] The processing unit is used to obtain the classification neural network from the storage unit when receiving the cell image, extract the features of the cell image by the sub-backbone, generate multi-scale features through FPN, and input the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and input them to the cell nucleus detection branch unit for cell nucleus area recognition to obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus area in the cell and the cell nucleus classification result; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification result of the cell image.

[0085] In the system, the cell image is a cell image based on a cervical cell smear.

[0086] In the system, the processing unit is also used for the cell classification branch unit to be implemented by a first sub-convolutional neural network with a spatial attention mechanism, to pay attention to the cells in the cell image, and to identify the cell classification branch features; the cell nucleus detection branch unit is implemented by a second sub-convolutional neural network with an attention mechanism, to detect the cell nucleus area, to explicitly pay attention to the cell nucleus area in the cell image, and to identify the cell nucleus detection branch features; the collaborative analysis unit is implemented by a third sub-convolutional neural network with an attention mechanism, to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and to guide and fuse the cell classification branch features according to the cell nucleus detection branch features.

[0087] In the system, the training unit is also used for the training to obtain the classification neural network, including: obtaining cell image samples, marking the cell category in each cell image sample; marking part of the cell nucleus region in the cell image samples with the cell category marked; inputting the cell image samples with the cell category marked and part of the cell nucleus region marked into the set classification neural network; the sub-backbone in the classification neural network extracts the features of the cell image according to the cell category marking and part of the cell nucleus region marking of the cell image sample, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region recognition and training to obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; uses the collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification result of the cell image.

[0088] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0089] An embodiment of the present invention further provides an electronic device, in which a device for implementing the method according to an embodiment of the present application can be integrated.

[0090] Specifically:

[0091] The electronic device may include a processor with one or more processing cores, a memory with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When executing the program in the memory, the cell recognition method in the cell image described above may be implemented.

[0092] Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope of the present disclosure.

[0093] In this article, specific embodiments are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention, and is not used to limit the present application. For those skilled in the art, according to the ideas, spirit and principles of the present invention, changes can be made in the specific implementation manners and application scopes, and any modifications, equivalent replacements, improvements, etc. made by them shall be included within the scope of protection of the present application.

Claims

1. A method for identifying cells in a cell image, It is characterized in that include: A classification neural network is obtained by training, wherein the classification neural network is composed of a sub-backbone network, a sub-feature pyramid FPN, a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit; When the classification neural network receives a cell image, the sub-backbone extracts the features of the cell image, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus region recognition to obtain cell nucleus detection branch features; the cell nucleus detection branch unit is implemented by a second sub-convolutional neural network with an attention mechanism, and by detecting the cell nucleus region, explicitly pays attention to the cell nucleus region in the cell image, and recognizes and obtains cell nucleus detection branch features; The classification neural network uses a collaborative analysis unit to guide and fuse the characteristics of the cell nucleus detection branch according to the characteristics of the cell classification branch, and obtains the location information of the cell nucleus region in the cell and the cell nucleus classification result; A collaborative analysis unit is used to guide and fuse cell classification branch features according to cell nucleus detection branch features to obtain a cell classification result of the cell image; the collaborative analysis unit is implemented using a third sub-convolutional neural network with an attention mechanism.

2. The method according to claim 1, It is characterized in that The cell classification branch unit is implemented by using a first sub-convolutional neural network with a spatial attention mechanism to focus on the cells in the cell image and identify the cell classification branch features.

3. The method according to claim 1, It is characterized in that The training to obtain a classification neural network includes: Acquire cell image samples, and label the cell category in each cell image sample; In the cell image samples with labeled cell categories, part of the cell nucleus area is labeled; Input the cell image samples with the cell categories and partial cell nucleus regions marked into the set classification neural network; The sub-backbone in the classification neural network extracts the features of the cell image based on the cell category annotations and partial cell nucleus region annotations of the cell image sample, and then generates multi-scale features through FPN. The multi-scale features are respectively input into the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and are input into the cell nucleus detection branch unit for cell nucleus region recognition and training to obtain cell nucleus detection branch features; The classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification results; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification results of the cell image.

4. The method according to claim 1, It is characterized in that The cell image is a cell image based on a cervical cell smear.

5. A cell recognition system in cell images, It is characterized in that The system comprises: a training unit, a storage unit and a processing unit, wherein: A training unit, used for training to obtain a classification neural network, wherein the classification neural network is composed of a sub-backbone, a sub-FPN, a cell classification branch unit, a cell nucleus detection branch unit and a collaborative analysis unit; A storage unit, used for storing the trained classification neural network; The processing unit is used to obtain the classification neural network from the storage unit when receiving the cell image, extract the features of the cell image by the sub-backbone, generate multi-scale features through FPN, and input the multi-scale features to the cell classification branch unit for cell classification to obtain cell classification branch features, and input them to the cell nucleus detection branch unit for cell nucleus region identification and obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features to obtain the position information of the cell nucleus region in the cell and the cell nucleus classification result; the collaborative analysis unit is used to guide and fuse the cell classification branch features according to the cell nucleus detection branch features to obtain the cell classification result of the cell image; the collaborative analysis unit is implemented by a third sub-convolutional neural network with an attention mechanism; The processing unit is also used for the cell nucleus detection branch unit to be implemented by using a second sub-convolutional neural network with an attention mechanism. By detecting the cell nucleus area, the cell nucleus area in the cell image is explicitly paid attention to, and the cell nucleus detection branch features are identified.

6. The system according to claim 5, It is characterized in that The processing unit is also used for the cell classification branch unit to adopt a first sub-convolutional neural network with a spatial attention mechanism to focus on the cells in the cell image and identify and obtain cell classification branch features.

7. The system according to claim 5, It is characterized in that The training unit is also used for the training to obtain a classification neural network, including: obtaining cell image samples, marking the cell category in each cell image sample; marking part of the cell nucleus area in the cell image samples with the cell category marked; inputting the cell image samples with the cell category marked and the part of the cell nucleus area marked into the set classification neural network; the sub-backbone in the classification neural network extracts the features of the cell image according to the cell category marking and the part of the cell nucleus area marking of the cell image sample, generates multi-scale features through FPN, and inputs the multi-scale features to the cell classification branch unit for cell classification branch training to obtain cell classification branch features, and inputs the multi-scale features to the cell nucleus detection branch unit for cell nucleus area recognition and training to obtain cell nucleus detection branch features; the classification neural network uses a collaborative analysis unit to guide and fuse the cell nucleus detection branch features according to the cell classification branch features, and trains to obtain the position information of the cell nucleus area in the cell and the cell nucleus classification result; uses the collaborative analysis unit to guide and fuse the cell classification branch features according to the cell nucleus detection branch features, and trains to obtain the cell classification result of the cell image.

8. The system according to claim 5, It is characterized in that The cell image is a cell image based on a cervical cell smear.

Images