P2Pnet-based intraoperative rapid pathological auxiliary diagnosis method and equipment for ovarian metastatic signet-ring cell carcinoma, and medium

Abstract

The invention belongs to the technical field of medical treatment, and particularly discloses an intraoperative rapid pathological auxiliary diagnosis method and device for ovarian metastatic signet-ring cell carcinoma based on P2Pnet and a medium. Taking the intercepted image block as the input of a deep convolutional neural network, training in a backbone network VGG-16BN to obtain a deep feature, sampling the feature, fusing the feature with an original feature to obtain a new feature, obtaining coordinate regression and confidence classification through convolution, and then matching a prediction point with a marking point by using a one-to-one matching algorithm to obtain a prediction result; and discarding the prediction cells with low confidence coefficient, and finally obtaining the predicted signet ring cell carcinoma, thereby realizing auxiliary rapid pathological diagnosis in the ovarian metastatic signet ring cell carcinoma operation.

Description

Technical Field

[0001] The present invention belongs to the field of medical technology, and specifically relates to a method, device and medium for rapid intraoperative pathological auxiliary diagnosis of ovarian metastatic signet ring cell carcinoma based on P2Pnet. Background Art

[0002] Ovarian metastatic signet ring cell carcinoma is a highly malignant adenocarcinoma. It is named because the cytoplasm of the cancer cells is filled with mucus, which squeezes the cell nucleus to one side, resembling a "ring". 90% of ovarian metastatic signet ring cell carcinomas originate from the stomach and are easily implanted and metastasized to the ovaries. Most patients with ovarian metastatic signet ring cell carcinoma come to the doctor with ovarian tumors as the first symptom.

[0003] The ovaries are located deep and difficult to puncture and sample, and there are large differences in the sensitivity and specificity of imaging and serum markers. Therefore, it is rare to obtain a clear qualitative diagnosis of ovarian metastatic signet ring cell carcinoma before surgery. Therefore, the first clear diagnosis is a rapid intraoperative pathological diagnosis. The key to the rapid intraoperative pathological diagnosis of ovarian metastatic signet ring cell carcinoma is to identify signet ring-like cancer cells. Once diagnosed during surgery, the surgeon should be immediately advised to explore the gastrointestinal tract.

[0004] There are many challenges in the process of rapid intraoperative pathological diagnosis, which makes the work of pathologists difficult. The specific challenges are detailed as follows: Rapid intraoperative pathological diagnosis requires pathologists to perform pathological examinations with frozen tissue sections within 30 minutes to help surgeons determine the nature of the lesion and the scope of the operation, adjust the surgical plan in a timely manner, and improve the effect of surgical treatment. It requires extremely high diagnostic speed and accuracy. However, due to the limitations of frozen sections used for rapid intraoperative pathological diagnosis, the temperature of the tissue drops sharply during freezing, especially in the water-rich ovarian tissue, which is more likely to form ice crystals, resulting in poor quality of tissue sections. In addition, frozen sections are prone to cell deformation such as large nuclei and atypicality due to the lack of dehydration and fixation of the tissue, which increases the risk and difficulty of rapid intraoperative pathological diagnosis. In addition, in rare cases, when ovarian metastatic signet ring cell carcinoma is combined with ovarian cysts, the presence of ovarian cysts masks the true lesions and easily leads to missed diagnosis. The hidden pathological characteristics of signet ring cancer cells and the subjectivity of pathologists in reading films lead to underdiagnosis or even misdiagnosis during the rapid pathological diagnosis of ovarian metastatic signet ring cell carcinoma during surgery, which affects the choice of surgical plan and thus the patient's surgical treatment effect, fertility and quality of life.

[0005] In the past, tumor cell identification and pathological diagnosis mainly relied on traditional microscopic observation and manual analysis, which is time-consuming, labor-intensive and error-prone. With the rise of deep learning, especially the development of convolutional neural networks, deep learning methods have begun to be applied in the field of pathological images. Therefore, using deep learning methods to study a new type of pathological auxiliary diagnosis method for tumor cell identification in tissues is still one of the main topics that researchers are currently working on. Summary of the invention

[0006] The purpose of the present invention is to provide a method, device and medium for rapid intraoperative pathological auxiliary diagnosis of ovarian metastatic signet ring cell carcinoma based on P2Pnet, which solves the problem of difficulty in rapid pathological diagnosis of ovarian metastatic signet ring cell carcinoma in the prior art.

[0007] The present invention also aims to provide an intraoperative pathology auxiliary diagnosis device.

[0008] Another object of the present invention is to provide a storage medium.

[0009] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a method for rapid intraoperative pathological auxiliary diagnosis of ovarian metastatic signet ring cell carcinoma based on P2Pnet, specifically comprising the following steps:

[0010] S1, the image blocks are obtained by intercepting patches at 10x field of view magnification of the intraoperative frozen section digital pathology full-field image of ovarian metastatic signet ring cell carcinoma, and distributed to different nodes in P2PNet;

[0011] S2, the model of each node receives image data and performs data preprocessing;

[0012] S3. Extracting deep features from the data preprocessing results using the VGG-16_BN network model;

[0013] S4, using two parallel branches to classify the point positions and confidence levels of the depth features respectively, to obtain predicted points and marked points;

[0014] S5. Use the Hungarian matching algorithm to perform one-to-one matching between the predicted points and the marked points. After the matching is successful, use Euclidean loss supervised point regression and cross entropy loss training classification respectively, and then train the model to obtain the prediction result.

[0015] Preferably, in S1, the size of the image block is consistent with the size of an input image of an artificial neural network model for region recognition.

[0016] Preferably, in S2, the model of each node receives a color image as input, and the color image is an image with three channels: red, green and blue.

[0017] Preferably, in S3, the first thirteen convolutional layers of the VGG-16_BN network model are used to extract deep features.

[0018] Preferably, in S4, the architecture of the two branches remains the same, consisting of three stacked convolutions associated with a ReLU activation function.

[0019] Preferably, in S5, the function of the loss is as follows:

[0020] Loss = L cls +λL loc

[0021] Where L cls is the cross entropy loss, which is used to calculate the gap between the predicted category and the labeled category, λ is the weight term that balances the impact of regression loss, L loc is the Euclidean loss, which is used to calculate the distance between the predicted point and the labeled point.

[0022] The second technical solution of the present invention is implemented as follows: an intraoperative pathology-assisted diagnosis device, the device comprising: a memory, a processor, and a pathology-assisted diagnosis program stored in the memory and executable on the processor, the pathology-assisted diagnosis program being configured to implement the steps of the pathology-assisted diagnosis method as described in any one of claims 1-6.

[0023] The third technical solution of the present invention is implemented as follows: a storage medium, on which a pathology-assisted diagnosis program is stored, and when the pathology-assisted diagnosis program is executed by a processor, the steps of the pathology-assisted diagnosis method as described in any one of claims 1-6 are implemented.

[0024] Compared with the prior art, a P2Pnet-based intraoperative rapid pathological auxiliary diagnosis method for ovarian metastatic signet ring cell carcinoma according to an embodiment of the present application is used. The image blocks of the digital pathological images of ovarian metastatic signet ring cell carcinoma are obtained by intercepting patches. On this basis, features are extracted through a deep convolutional neural network and fused with the original features to obtain new features. Softmax is used to normalize the output confidence score and the inherent translation invariance of the convolutional layer is used to predict the offset of the point coordinates. The Hungarian matching algorithm and contrastive learning are used to perform matching constraints between the predicted points and the marked points, and the consistency of the encoding of the same type of cells is learned, and the difference in the characteristics of different types of cells is widened, so as to obtain a more accurate recognition and prediction result of ovarian metastatic signet ring cell carcinoma, thereby realizing the judgment of ovarian cancer metastatic signet ring cell carcinoma in frozen sections during surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 This is a schematic diagram of the overall architecture of the P2PNet model;

[0027] Figure 2 Visualization of the predictions identified for ovarian metastatic signet ring cell carcinoma. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] In the past, cell recognition mainly relied on traditional microscopic observation and manual analysis, which is time-consuming, labor-intensive and error-prone. With the rise of deep learning, especially the development of convolutional neural networks, deep learning methods have begun to be applied in the field of pathological images. In the development of deep learning cell recognition, researchers have proposed many neural network-based models to solve the problem of cell recognition, but the recognition of signet ring-like cancer cells unique to ovarian metastatic signet ring cell carcinoma has been less involved.

[0030] At the same time, clinical diagnosis urgently needs a convenient and effective technology for judging ovarian metastatic signet ring cell carcinoma, which can quickly and accurately give the situation of the entire pathological section. This application is proposed to solve the clinical-level judgment needs of ovarian metastatic signet ring cell carcinoma. Secondly, in the process of cell recognition, how to guide the convolutional neural network to explicitly encode the consistent characteristics of cells, and how to fully decode the consistent cell characteristics to obtain accurate signet ring-like cancer cells in ovarian metastatic signet ring cell carcinoma are also urgent problems to be solved.

[0031] In response to the above technical problems, the concept of the embodiments of the present invention is: unlike natural images, the number of cells in pathological diagnostic images is very large, and a large number of cells may overlap, making it difficult to accurately distinguish the boundaries and morphological characteristics of each cell. There are great differences in cell morphology, including changes in size, shape, color, etc. Due to the complexity of cell characteristics, cell identification becomes difficult.

[0032] In view of this, the present application proposes a P2Pnet-based ovarian metastatic signet ring cell carcinoma recognition and detection method to judge ovarian metastatic signet ring cell carcinoma in intraoperative frozen pathological sections. The P2Pnet-based ovarian metastatic signet ring cell carcinoma recognition and detection method can use the patch image block of the digital pathology full-field image as the input of the deep convolutional neural network, and obtain deep features after training on the backbone network VGG-16_BN. The features are upsampled and fused with the original features to obtain new features, and coordinate regression and confidence classification are obtained after convolution. After that, a one-to-one matching algorithm is used to match the predicted points with the marked points, and the predicted cells with low confidence are discarded. Finally, the predicted ovarian metastatic signet ring cell carcinoma is obtained, thereby realizing the judgment of ovarian metastatic signet ring cell carcinoma in intraoperative frozen pathological sections.

[0033] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0034] Example 1

[0035] This embodiment provides a P2Pnet-based rapid intraoperative pathological auxiliary diagnosis method for ovarian metastatic signet ring cell carcinoma, including:

[0036] The P2PNet model is used to detect ovarian metastatic signet ring cell carcinoma. The backbone network is VGG-16_BN. The P2PNet model is a distributed neural network model, which is usually used for image detection tasks. The overall architecture of the model is as follows: Figure 1 As shown in the figure. It trains the neural network by collaborating multiple nodes, where the nodes are often neural network models, which improves performance, robustness and security. Multiple nodes communicate directly and work together to solve image detection problems. In addition, contrastive learning of cell recognition is used to provide strong supervision guidance for the mapping relationship of the model to better mine the fine-grained features of the model. Contrastive learning of cell detection aims to encourage convolution kernels to learn important features and increase the distinguishability of features. This enables the network to make better predictions. Based on the contrastive learning method, the features of non-high-confidence cell predictions are taken out and sent to the fully connected neural network for category re-prediction and correction.

[0037] S1. Image blocks are obtained by intercepting patches at 10x magnification of the full-field image of frozen digital pathology sections of ovarian metastatic signet ring cell carcinoma, and then distributed to different nodes in P2PNet. Each node will be responsible for processing part of the data.

[0038] The experimental data set uses 42 1920*1080 jpg data and 42 json annotations, with a total of 11194 cells. Since only targets such as ovarian metastatic signet ring cell carcinoma need to be detected, it is only necessary to find the target location. The pathological slice has huge pixels. If it is directly input into the processor, it will cause memory overflow. Therefore, it is necessary to obtain image blocks by intercepting patches at a 10x field of view magnification of the full-field image of the digital slice. The image block size is 512*512, and only patches with a cell number greater than or equal to 10 are retained, totaling 728, and the corresponding annotation txt is generated at the same time. Then the patch data is divided into a test set (146 images), a validation set (146 images), and a training set (436 images) in a ratio of 1:1:3. The size of the image block is consistent with the size of the input image of the artificial neural network model used for region recognition.

[0039] S2 inputs the image data.

[0040] Each node model receives a color image as input, usually a three-channel image (red, green, and blue channels). The mean and variance of the input image are normalized.

[0041] S3. Use VGG convolutional neural network.

[0042] The input image first passes through a backbone network VGG-16_BN, and the first thirteen convolutional layers of VGG-16_BN are used to extract deep features. For the output feature map, the nearest neighbor interpolation algorithm is used to upsample it to twice. The upsampled image and the feature map are horizontally connected by element-by-element addition to generate a merged feature map. The merged feature map is convolved by 3*3 to obtain the deep feature map F s , where s is the downsampling step size.

[0043] S4. Use P2Pnet model.

[0044] According to the obtained F s Two parallel branches are used for point coordinate regression and confidence classification, where the architecture of the two branches remains the same, consisting of three stacked convolutions associated with ReLU activation functions. For the classification branch, the confidence score is output by Softmax normalization; for the regression branch, due to the inherent translation invariance of the convolutional layer, it relies on the offset of the predicted point coordinates. Specifically, each pixel on Fs should correspond to a block of size s*s in the input image. The confidence score is obtained based on the density normalized average precision, which is calculated based on the area under the precision-recall curve.

[0045] S5. Use a one-to-one matching strategy.

[0046] The Hungarian matching algorithm is used to match the predicted points and the labeled points. After the matching is successful, the Euclidean loss supervised point regression and the cross entropy loss training classification are used to train the model and obtain the prediction results.

[0047] The loss function is as follows:

[0048] Loss = L cls +λL loc

[0049] Where L cls is the cross entropy loss, which is used to calculate the gap between the predicted category and the labeled category, λ is the weight term that balances the impact of regression loss, L loc is the Euclidean loss, which is used to calculate the distance between the predicted point and the labeled point.

[0050] Experimental results:

[0051] Table 1 shows the model results trained for ovarian metastatic signet ring cell carcinoma recognition. After analysis, it was found that the indicators of the training set were nearly 10 points higher than those of the validation set and the test set. The reason was that the training was overfitted due to the small amount of data. Therefore, the performance on the training set was good, but the validation and test indicators were slightly worse. This phenomenon can be attributed to the complexity of the cell background and the similarity of the features, which led to the poor validation and test results compared with the training set results.

[0052] Table 1 shows the model results trained for ovarian metastatic signet ring cell carcinoma

[0053] index Training set Validation set Test Set Precision 0.8183 0.7132 0.7176 Recall 0.8334 0.7094 0.7205 F1 0.8258 0.7113 0.7191

[0054] Figure 2 The visualization of the prediction results of ovarian metastatic signet ring cell carcinoma recognition is shown, where the red squares are signet ring-like cancer cells annotated by Ground Truth, and the green circles are the areas predicted by the model as ovarian metastatic signet ring cell carcinoma. By observing the images, it can be found that the model can accurately detect ovarian metastatic signet ring cell carcinoma in the first three pictures, while the effect of the fourth picture is slightly worse. This may be because the complex characteristics of the cell background are similar, resulting in poor results. In this case, the model may mistakenly predict some background cells as ovarian metastatic signet ring cell carcinoma, thereby reducing the accuracy of the detection.

[0055] Nevertheless, the ovarian metastatic signet ring cell carcinoma recognition and detection method of the present invention can still effectively detect ovarian metastatic signet ring cell carcinoma. In short, the training method of the present invention provides an effective solution for the detection of ovarian metastatic signet ring cell carcinoma, and provides strong support for medical research and clinical diagnosis.

[0056] Example 2

[0057] Embodiment 2 of the present invention provides an electronic device, comprising a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the ovarian metastatic signet ring cell carcinoma detection algorithm.

[0058] Example 3

[0059] Embodiment 2 of the present invention provides a computer-readable medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the method for automatic scoring of intraoperative rapid pathological diagnosis sections of ovarian metastatic signet ring cell carcinoma.

[0060] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for automatic scoring of intraoperative rapid pathological diagnosis sections for identifying ovarian metastatic signet ring cell carcinoma according to various embodiments of the present application described in the above-mentioned "Exemplary Methods" section of this specification.

[0061] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0062] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the ovarian metastatic signet ring cell carcinoma detection algorithm according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0063] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0064] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method, device and medium for rapid intraoperative pathological auxiliary diagnosis of ovarian metastatic signet ring cell carcinoma based on P2Pnet, characterized in that: The specific steps include: S1, the image blocks are obtained by intercepting patches at 10x field of view magnification of the intraoperative frozen section digital pathology full-field image of ovarian metastatic signet ring cell carcinoma, and distributed to different nodes in P2PNet; S2, the model of each node receives image data and performs data preprocessing; S3. Extracting deep features from the data preprocessing results using the VGG-16_BN network model; S4, using two parallel branches to classify the point positions and confidence levels of the depth features respectively, to obtain predicted points and marked points; S5. Use the Hungarian matching algorithm to perform one-to-one matching between the predicted points and the marked points. After the matching is successful, use Euclidean loss supervised point regression and cross entropy loss training classification respectively, and then train the model to obtain the prediction result.

2. The pathology-assisted diagnosis method according to claim 1, characterized in that: In S1, the size of the image block is consistent with the size of the input image of the artificial neural network model used for region recognition.

3. The pathology-assisted diagnosis method according to claim 1, characterized in that: In S2, the model of each node receives a color image as input, and the color image is an image with three channels: red, green, and blue.

4. The pathology-aided diagnosis method according to claim 1, characterized in that: In S3, the first thirteen convolutional layers of the VGG-16_BN network model are used to extract deep features.

5. The pathology-assisted diagnosis method according to claim 1, characterized in that: In S4, the architecture of the two branches remains the same, consisting of three stacked convolutions associated with ReLU activation functions.

6. The pathology-assisted diagnosis method according to claim 1, characterized in that: In S5, the loss function is as follows: Loss=L cls +λL loc Where L cls is the cross entropy loss, which is used to calculate the gap between the predicted category and the labeled category, λ is the weight term that balances the impact of regression loss, L loc is the Euclidean loss, which is used to calculate the distance between the predicted point and the labeled point.

7. An intraoperative pathology auxiliary diagnosis device, characterized in that: The device comprises: a memory, a processor, and a pathology-assisted diagnosis program stored in the memory and executable on the processor, wherein the pathology-assisted diagnosis program is configured to implement the steps of the pathology-assisted diagnosis method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium stores a pathology-assisted diagnosis program, and when the pathology-assisted diagnosis program is executed by the processor, the steps of the pathology-assisted diagnosis method according to any one of claims 1 to 6 are implemented.

Images