A method and system for identifying glandular cells

Abstract

The embodiment of the present invention discloses a method and system for identifying glandular cells. The embodiment of the present invention uses a trained target detection model to perform target detection on the obtained glandular cell image, and then identifies the information features of a single glandular cell and the cell nucleus in the single glandular cell; inputs the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extracts the arrangement information features of the glandular cells; uses the trained convolutional neural network to process the obtained glandular cell image, and outputs the information features such as the texture of the glandular cells; splices the obtained arrangement information features of the glandular cells and the information features such as the texture of the glandular cells, and then classifies them to obtain the identification results of the glandular cells. In this way, the embodiment of the present invention improves the recognition accuracy of identifying glandular cells.

Description

Technical Field

[0001] The present invention relates to image recognition technology, and in particular to a method and system for recognizing glandular cells. Background Art

[0002] Glandular cells are cells in the epidermis of body tissues that can produce and secrete certain liquid substances, such as cells that constitute the prostate, lacrimal glands, pancreatic islets, thyroid glands, etc. Glandular cells can be affected by various internal and external stimuli and develop lesions, and even develop into adenocarcinoma. Adenocarcinoma accounts for more than 95% of the incidence of rectal cancer and gastric cancer, and is also a type of cancer in cervical cancer that seriously affects women's reproductive health. Cytology is a common, economical and effective screening method, which is widely used in the diagnosis of various cancers including adenocarcinoma. Timely detection of atypical glandular cells is an important means of preventing and treating adenocarcinoma. However, there is a shortage of cytology readers, which makes it difficult to meet the current needs of cytology diagnosis. In addition, due to the high difficulty in distinguishing atypical glandular cells, higher requirements are placed on the experience and diagnostic level of cytology readers. Under the influence of insufficient experience or subjective factors of cytology readers, there is a certain risk of missed diagnosis and misdiagnosis.

[0003] In recent years, artificial intelligence technology has been widely used in the field of medical image analysis. Artificial intelligence technology can reconstruct and analyze medical image data, extract key features of lesions and establish recognition models, and then produce disease analysis reports and auxiliary diagnosis plans, which reduces the heavy workload of cytology film readers and helps identify rare diseases and standardize analysis reports. At present, artificial intelligence technology can achieve accurate classification of some cancer cells, but the recognition accuracy of glandular cells is still difficult to meet the needs of actual scenarios. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method for identifying glandular cells, which can improve the accuracy of identifying glandular cells.

[0005] The embodiment of the present invention further provides a gland cell recognition system, which can improve the recognition accuracy of gland cells.

[0006] The embodiment of the present invention is implemented as follows:

[0007] A method for identifying glandular cells, comprising:

[0008] After the obtained glandular cell images are subjected to target detection using the trained target detection model, information features of individual glandular cells and cell nuclei in the individual glandular cells are identified;

[0009] Inputting the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extracting the arrangement information features of the glandular cells;

[0010] The obtained glandular cell images are processed by the trained convolutional neural network to output the texture information features of the glandular cells;

[0011] The obtained arrangement information features of the glandular cells and the texture information features of the glandular cells are spliced ​​and then classified to obtain the identification results of the glandular cells.

[0012] Preferably, when the gland cell image includes an atypical gland cell image, the arrangement information features of the gland cells obtained by extraction are the arrangement information features of the atypical gland cells, the texture information features of the output gland cells are the texture information of the atypical gland cells, and the obtained gland cell recognition result is the recognition result of the atypical gland cells.

[0013] Preferably, the target detection model obtained through training adopts a target detection model improved based on CenterNet network architecture, and the training includes tilt positioning of a single gland cell in the gland cell image and segmentation of the cell nucleus in a single gland cell.

[0014] Preferably, training the tilt positioning of the glandular cells in the glandular cell image comprises:

[0015] Based on the set tilt frame angle, the tilt positioning training of individual gland cells was performed.

[0016] Preferably, the target detection model improved based on the CenterNet network architecture is also established based on an attention mechanism and a multi-scale image preprocessing mechanism.

[0017] Preferably, the step of inputting the information features of the single gland cell and the cell nucleus in the single gland cell into the trained graph convolutional neural network for processing comprises:

[0018] The graph convolutional neural network layer selects the input nucleus center position of each gland cell and the center position of each gland cell as the network structure data node;

[0019] The graph structure of the graph convolutional network is obtained by fully connecting the network structure data nodes, and the arrangement information characteristics of the glandular cells are obtained after processing by setting five layers of graph convolutional layers.

[0020] Preferably, the central position of each glandular cell includes:

[0021] The center position of the tilted frame of each gland cell is taken as the center position of the gland cell.

[0022] Preferably, the convolutional neural network adopts the SE-ResNet50 network architecture, and the SE-ResNet50 network architecture is composed of basic elements based on the attention mechanism and the residual network.

[0023] Preferably, the reclassification is performed using a perceptron with a set number of layers.

[0024] A gland cell recognition system comprises: an acquisition unit, a target detection unit, a gland cell arrangement information feature extraction unit, a gland cell texture information feature output unit and a classification unit, wherein:

[0025] an acquisition unit, used for acquiring gland cell images;

[0026] A target detection unit is used to perform target detection on the obtained glandular cell image using the trained target detection model to identify and obtain information features of a single glandular cell and a cell nucleus in the single glandular cell;

[0027] A unit for extracting arrangement information features of glandular cells is used to input information features of the single glandular cells and the cell nuclei in the single glandular cells into the trained graph convolutional neural network for processing, so as to extract arrangement information features of the glandular cells;

[0028] The unit for outputting the texture information feature of the glandular cells is used to process the obtained glandular cell image using the trained convolutional neural network to output the texture information feature of the glandular cells;

[0029] The classification unit is used to combine the obtained arrangement information features of the glandular cells and the texture information features of the glandular cells, and then classify them to obtain the identification results of the glandular cells.

[0030] As can be seen above, the embodiment of the present invention uses the trained target detection model to perform target detection on the obtained glandular cell image, and then identifies the information features of the single glandular cell and the cell nucleus in the single glandular cell; the information features of the single glandular cell and the cell nucleus in the single glandular cell are input into the trained graph convolutional neural network for processing, and the arrangement information features of the glandular cells are extracted; the obtained glandular cell image is processed by the trained convolutional neural network, and the information features such as the texture of the glandular cells are output; the obtained arrangement information features of the glandular cells and the information features such as the texture of the glandular cells are spliced, and then classified to obtain the identification results of the glandular cells. In this way, since the embodiment of the present invention introduces the target detection model to segment the glandular cells in the glandular cell image and identify the cell nuclei in the glandular cells, and introduces the graph convolutional neural network to extract the arrangement information features of the glandular cells, the recognition accuracy is higher and the recognition is more accurate in the subsequent glandular cell recognition based on the arrangement information of the glandular cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flow chart of a method for identifying glandular cells provided in an embodiment of the present invention;

[0032] Figure 2 A flow chart showing an example of a method for identifying glandular cells provided in an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a process for performing target detection using an improved CenterNet network provided in an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of information feature detection results of a single gland cell provided by an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of an example of information feature expression of glandular cells provided in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the structure of a gland cell recognition system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0038] By introducing artificial intelligence technology into the identification process of atypical glandular cells, a convolutional neural network can be trained. After the acquired glandular cell images are classified using the trained convolutional neural network, the identification results of atypical glandular cells can be obtained. However, the classification results of atypical glandular cells obtained in this way are not very accurate and the recognition accuracy is low.

[0039] In order to overcome this problem, the embodiment of the present invention uses the trained target detection model to perform target detection on the obtained glandular cell image, and then identifies the information features of the single glandular cell and the cell nucleus in the single glandular cell; inputs the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extracts the arrangement information features of the glandular cells; processes the obtained glandular cell image using the trained convolutional neural network, and outputs the texture information features of the glandular cells; splices the obtained arrangement information features of the glandular cells and the texture information features of the glandular cells, and then classifies them to obtain the recognition result of the glandular cells.

[0040] In this way, since the embodiment of the present invention introduces a target detection model to segment the glandular cells in the glandular cell image and identify the cell nuclei in the glandular cells, and introduces a graph convolutional neural network to extract the arrangement information features of the glandular cells, the recognition accuracy is higher and the recognition is more accurate in the subsequent glandular cell recognition based on the arrangement information of the glandular cells.

[0041] Figure 1 The flowchart of the method for identifying glandular cells provided by the embodiment of the present invention includes the following specific steps:

[0042] Step 101: After performing target detection on the obtained glandular cell image using the trained target detection model, information features of a single glandular cell and a cell nucleus in the single glandular cell are identified;

[0043] Step 102, inputting the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extracting the arrangement information features of the glandular cells;

[0044] Step 103, processing the obtained glandular cell image using the trained convolutional neural network to output the texture information features of the glandular cells;

[0045] Here, the output includes not only the texture information features of the glandular cells, but also other information features of the glandular cells, which are not limited here;

[0046] Step 104: After the obtained arrangement information features of the glandular cells and the texture information features of the glandular cells are spliced, they are classified to obtain the identification results of the glandular cells.

[0047] In this method, the execution order of step 103 may not be after step 102, and may be executed in parallel with step 101 and step 102, which is not limited here.

[0048] In this method, when the glandular cell image includes an atypical glandular cell image, the arrangement information features of the glandular cells extracted are the arrangement information features of the atypical glandular cells, the texture information features of the output glandular cells are the texture information of the atypical glandular cells, and the identification results of the glandular cells obtained are the identification results of the atypical glandular cells. Here, the atypical glandular cell image includes a large number of atypical glandular cell information. In this way, since the glandular cell image contains both typical glandular cells and atypical glandular cells, these atypical glandular cells should be particularly identified during identification to ensure the accuracy of identifying the glandular cells.

[0049] In the method, the target detection model obtained by the training adopts a target detection model improved based on the CenterNet network architecture, and the training includes the tilt positioning of a single gland cell in the gland cell image and the segmentation of the nucleus in the single gland cell. Here, in particular, the training of the tilt positioning of the gland cell in the gland cell image includes: based on the set tilt frame angle, the tilt positioning training of the single gland cell is performed.

[0050] In this method, the target detection model improved based on the CenterNet network architecture is also established based on the attention mechanism and the multi-scale image preprocessing mechanism.

[0051] In this method, the step of inputting the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing comprises:

[0052] The graph convolutional neural network layer selects the center position of the nucleus of each gland cell and the center position of each gland cell as the network structure data nodes, and obtains the graph structure of the graph convolutional network after fully connecting the network structure data nodes. After processing by setting five layers of graph convolutional layers, the arrangement information characteristics of the gland cells are obtained. Here, when selecting the center position of each gland cell, the center position of the tilted frame positioning of each gland cell can be used as the center position of the gland cell.

[0053] In the method, the convolutional neural network adopts the SE-ResNet50 network architecture, and the SE-ResNet50 network architecture is based on the basic elements of the attention mechanism and the residual network.

[0054] In this method, the reclassification is performed using a perceptron with a set number of layers.

[0055] A specific example is given to explain the embodiment of the present invention in detail.

[0056] Figure 2 The flowchart of the method for identifying glandular cells provided in the embodiment of the present invention is given as an example, and the specific steps thereof include:

[0057] Step 201: Prepare a liquid-based smear of glandular cells.

[0058] Step 202: After scanning the liquid-based smear of glandular cells, an image of the glandular cells is obtained.

[0059] Step 203: After performing target detection on the glandular cell image using the trained CenterNet network, information features of a single glandular cell and a cell nucleus in the single glandular cell are identified.

[0060] In this step, the segmentation process of single glandular cells and the nuclei in single glandular cells, especially for atypical glandular cells, is mainly difficult because: the unclear cytoplasmic boundaries and large scale variations in glandular cell images, as well as the diversity and compactness of glandular cell arrangement, make it difficult to locate the target; and glandular cells with different degrees of lesions have certain similarities in morphology, which causes confusion in model training. Therefore, in order to overcome these difficulties, such as Figure 3 As shown, Figure 3The present invention provides a process diagram of target detection based on an improved CenterNet network. Improvements are made in the CenterNet network, including: introducing training parameters to control the rotation angle of the detection frame, which can be set to an oblique frame to detect and locate glandular cells; b. introducing a cell nucleus segmentation branch to segment the cell nuclei while locating the glandular cells. The network outputs the location information features of the glandular cells and the information features of the cell nucleus segmentation, which reduces the impact of the huge glandular cell graphic information on cell detection and improves recall; at the same time, a multi-scale image preprocessing strategy can be added to locate the glandular cell area at different resolutions, better train the size and contour information of the cells, and enhance the model's ability to detect glandular cells when the boundaries are unclear; in the network, an attention mechanism can also be used to focus on the most important areas of the glandular cells to reduce category confusion of similar cells. Finally, the following is obtained: Figure 4 The test results are shown. Figure 4 A schematic diagram of information feature detection results of a single gland cell provided in an embodiment of the present invention.

[0061] Step 204: Input the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extract the arrangement information features of the glandular cells.

[0062] In this step, the graph convolutional neural network is introduced to improve the recognition accuracy by using the arrangement information characteristics of glandular cells. Specifically, the graph structure of glandular cells needs to be constructed first. Figure 4 The detection results shown in the figure directly locate the position of a single cell and segment the cell nucleus region therein, directly select the center position of the cell nucleus of each cell and the center position of the inclined box positioning of each cell as the graph structure data node, and fully connect each cell nucleus center point and each inclined box center point to construct a graph structure; secondly, a five-layer graph convolutional neural network is set to mine the structural relationship between data nodes. Here, a matrix containing only 0 or 1 is constructed based on the data nodes, and the data of the data nodes is the point coordinate vector of the corresponding node of the graph structure. By performing multi-layer graph convolution operations on the graph structure data nodes, a higher-level feature map is generated on the graph structure, that is, the arrangement information characteristics of glandular cells are obtained through training. Figure 5 As shown in the first line of Figure 5 This is a schematic diagram of an example of information feature expression of glandular cells provided in an embodiment of the present invention.

[0063] Steps 205-206: Process the glandular cell image using the trained convolutional neural network, output information features such as the texture of the glandular cells, and splice the obtained information features such as the arrangement of the glandular cells and the information features such as the texture of the glandular cells.

[0064] In this step, in addition to the arrangement information features of glandular cells, it is also necessary to extract information features such as the texture of glandular cells in the cell image. The process is as follows: the widely used convolutional neural network architecture SE-ResNet50 network is used, which is reconstructed by the attention mechanism (Attention) and the basic elements of ResNet to extract the global high-level features of glandular cells, and finally obtain 512-D embedded features; secondly, the arrangement information features extracted by the graph convolutional neural network are fused to obtain the arrangement information features and texture information features of the spliced ​​glandular cells, and obtain a more expressive glandular cell feature vector, such as Figure 5 The second row is shown. The fused feature is the multi-dimensional information representation of glandular cells. The characteristics of the glandular cells of interest are integrated into the overall interpretation, which can ultimately further improve the accurate identification of glandular cells.

[0065] Step 207: classify the obtained global information features of the glandular cells to obtain the identification results of the glandular cells.

[0066] In this step, a multilayer perceptron, i.e. a feedforward artificial neural network model, is set for classification, and the number of layers can be set to 2.

[0067] In this way, the embodiment of the present invention repeatedly considers the possible glandular cell abnormalities in the glandular cell image, improves the recognition accuracy, and allows artificial intelligence to play a more important role in cytological examination. In the medical field, it will reduce the pressure on pathologists to read films and reduce the misdiagnosis rate of adenocarcinoma, which is of great significance.

[0068] Figure 6 The schematic diagram of the structure of the gland cell recognition system provided by the embodiment of the present invention includes: an acquisition unit, a target detection unit, a gland cell arrangement information feature extraction unit, a gland cell texture information feature output unit and a classification unit, wherein:

[0069] an acquisition unit, used for acquiring gland cell images;

[0070] A target detection unit is used to perform target detection on the obtained glandular cell image using the trained target detection model to identify and obtain information features of a single glandular cell and a cell nucleus in the single glandular cell;

[0071] A unit for extracting arrangement information features of glandular cells is used to input information features of the single glandular cells and the cell nuclei in the single glandular cells into the trained graph convolutional neural network for processing, so as to extract arrangement information features of the glandular cells;

[0072] The unit for outputting the texture information feature of the glandular cells is used to process the obtained glandular cell image using the trained convolutional neural network to output the texture and other information features of the glandular cells;

[0073] The classification unit is used to combine the obtained arrangement information features of the glandular cells and the information features such as the texture of the glandular cells, and then classify them to obtain the identification results of the glandular cells.

[0074] In this system, when the gland cell image is an atypical gland cell image, the arrangement information features of the gland cells obtained by extraction are the arrangement information features of the atypical gland cells, the texture information features of the output gland cells are the texture information of the atypical gland cells, and the identification result of the gland cells obtained is the identification result of the atypical gland cells.

[0075] In the system, the target detection unit is also used for the target detection model obtained by the training to adopt a target detection model improved based on the CenterNet network architecture, and the training includes the tilt positioning of a single gland cell in the gland cell image and the segmentation of the cell nucleus in a single gland cell.

[0076] In the system, the arrangement information feature unit of the glandular cells is extracted, and is also used to input the information features of the single glandular cells and the cell nuclei in the single glandular cells into the trained graph convolutional neural network for processing, including:

[0077] The graph convolutional neural network layer selects the center position of the nucleus of each gland cell and the center position of each gland cell as the network structure data nodes, and obtains the graph structure of the graph convolutional network after fully connecting the network structure data nodes. After processing by setting five layers of graph convolutional layers, the arrangement information characteristics of the gland cells are obtained.

[0078] In this system, the texture information feature unit of the output gland cells is also used in the convolutional neural network using the SE-ResNet50 network architecture, which is based on the basic elements of the attention mechanism and the residual network.

[0079] In the system, the classification unit is also used for performing the reclassification using a perceptron with a set number of layers.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying glandular cells, characterized in that: include: After the obtained glandular cell image is subjected to target detection using the trained target detection model, information features of a single glandular cell and a cell nucleus in the single glandular cell are identified, wherein the trained target detection model uses a target detection model improved based on the CenterNet network architecture, and the training includes tilt positioning of a single glandular cell in the glandular cell image and segmentation of a cell nucleus in the single glandular cell, and training parameters are introduced into the CenterNet network to control the rotation angle of the detection frame to perform tilt positioning of the single glandular cell and the cell nucleus in the single glandular cell, and a cell nucleus segmentation branch is introduced to segment the cell nucleus in the middle of the single glandular cell while performing tilt positioning, and the CenterNet network outputs the position information features of the glandular cell and the segmentation information features of the cell nucleus; Inputting the information features of the single glandular cell and the cell nucleus in the single glandular cell into the trained graph convolutional neural network for processing, and extracting the arrangement information features of the glandular cells; The obtained glandular cell images are processed by the trained convolutional neural network to output the texture information features of the glandular cells; After the obtained arrangement information features of the glandular cells and the texture information features of the glandular cells are spliced, they are classified to obtain the identification results of the glandular cells; When the gland cell image includes an atypical gland cell image, the arrangement information feature of the gland cells obtained by extraction is the arrangement information feature of the atypical gland cells, the texture information feature of the output gland cells is the texture information of the atypical gland cells, and the identification result of the gland cells obtained is the identification result of the atypical gland cells; in, The step of inputting the information features of the single gland cell and the cell nucleus in the single gland cell into the trained graph convolutional neural network for processing comprises: The center position of the nucleus of each glandular cell and the center position of each glandular cell are selected to construct a data node, wherein the data node only contains a matrix of 0 or 1, and the data in the data node is the point coordinate vector of the node corresponding to the graph structure, and the constructed data nodes are fully connected to construct a graph structure. The graph convolutional neural network layer processes the graph structure by setting five layers of graph convolutional layers to obtain the arrangement information characteristics of glandular cells.

2. The method according to claim 1, characterized in that Training the tilt positioning of the glandular cells in the glandular cell image includes: Based on the set tilt frame angle, the tilt positioning training of individual gland cells was performed.

3. The method according to claim 1, characterized in that The target detection model improved based on the CenterNet network architecture is also established based on the attention mechanism and the multi-scale image preprocessing mechanism.

4. The method according to claim 3, characterized in that The central location of each glandular cell includes: The center position of the tilted frame of each gland cell is taken as the center position of the gland cell.

5. The method according to claim 1, characterized in that The convolutional neural network adopts the SE-ResNet50 network architecture, which is based on the basic elements of the attention mechanism and the residual network.

6. The method according to claim 1, characterized in that The reclassification is performed using a perceptron with a set number of layers.

7. A gland cell recognition system, characterized in that: include: An acquisition unit, a target detection unit, a unit for extracting the arrangement information feature of glandular cells, a unit for outputting the texture information feature of glandular cells, and a classification unit, wherein: an acquisition unit, used for acquiring gland cell images; A target detection unit is used to perform target detection on the obtained glandular cell image using a trained target detection model, and identify and obtain information features of single glandular cells and cell nuclei in the single glandular cells, wherein the trained target detection model uses a target detection model improved based on the CenterNet network architecture, and the training includes tilt positioning of single glandular cells in the glandular cell image and segmentation of cell nuclei in the single glandular cells, and training parameters are introduced into the CenterNet network to control the rotation angle of the detection frame to perform tilt positioning on single glandular cells and cell nuclei in the single glandular cells, and a cell nucleus segmentation branch is introduced to segment the cell nucleus in the middle of the single glandular cell while performing tilt positioning, and the CenterNet network outputs the position information features of the glandular cells and the segmentation information features of the cell nuclei; A unit for extracting arrangement information features of glandular cells is used to input information features of the single glandular cells and the cell nuclei in the single glandular cells into the trained graph convolutional neural network for processing, so as to extract arrangement information features of the glandular cells; The unit for outputting the texture information feature of the glandular cells is used to process the obtained glandular cell image using the trained convolutional neural network to output the texture information feature of the glandular cells; A classification unit is used to combine the obtained arrangement information features of the glandular cells and the texture information features of the glandular cells, and then classify them to obtain the identification results of the glandular cells; When the gland cell image includes an atypical gland cell image, the arrangement information feature of the gland cells obtained by extraction is the arrangement information feature of the atypical gland cells, the texture information feature of the output gland cells is the texture information of the atypical gland cells, and the identification result of the gland cells obtained is the identification result of the atypical gland cells; The step of inputting the information features of the single gland cell and the cell nucleus in the single gland cell into the trained graph convolutional neural network for processing comprises: The center position of the nucleus of each glandular cell and the center position of each glandular cell are selected to construct a data node, wherein the data node only contains a matrix of 0 or 1, and the data in the data node is the point coordinate vector of the node corresponding to the graph structure, and the constructed data nodes are fully connected to construct a graph structure. The graph convolutional neural network layer processes the graph structure by setting five layers of graph convolutional layers to obtain the arrangement information characteristics of glandular cells.

Images