A method for automatically detecting lesions in real-time pathological microscopic images
By combining CNN and deep learning networks, a real-time automatic detection method for pathological microscopic images was developed, enabling real-time screening and lesion localization of pathological microscopic images. This solves the problems of poor real-time performance and high error risk in existing technologies, and improves diagnostic efficiency and accuracy.
Patent Information
- Application Number
- CN202410662578.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-27
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2044-05-27
AI Technical Summary
Existing pathological image detection methods have poor real-time performance, cumbersome operation procedures, require a large amount of human intervention, and are prone to error, making it difficult to achieve rapid and accurate lesion localization.
A real-time automatic detection method for pathological microscopic images is adopted. Through a real-time image display module, a screening module, a magnification recognition module, a preprocessing module, a classification prediction module, and a report generation module, combined with CNN convolutional neural network and deep learning neural network, the real-time screening, cutting, and lesion localization of pathological microscopic images can be achieved.
It improves the efficiency and accuracy of pathological microscopic image diagnosis, reduces the manpower burden, provides clear diagnostic heat maps for doctors to accurately locate lesions, and reduces the risk of errors.
Smart Images
Figure CN118537314B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of microscopic pathological image processing, and particularly relates to a method for automatically detecting lesions in real-time pathological microscopic images. BACKGROUND
[0002] In the existing cytological examination or histological examination, tumor tissue samples or cell samples are obtained for microscopic observation and cytological analysis to further confirm the presence or absence of tumors, the type and nature of the tumors. The entire diagnostic process involves the following procedures: the doctor observes the structure and morphology of the sample under the microscope after obtaining the stained sample; then consults relevant pathological diagnosis auxiliary materials according to the microscopic image to make comprehensive judgment and differential diagnosis, determine the disease type and severity; finally, the diagnostic results are recorded in the pathological report, and the diagnostic report is written. Usually, more than one patient's tissue sample will be detected by the doctor, and the doctor will continue to repeat the above steps. As can be seen, more human participation is needed in the above process to complete the entire process, which not only wastes time but also requires a large amount of manpower and may have the risk of human error.
[0003] Nowadays, existing hardware devices and image processing technology have supported real-time capture of video frames under the microscope and screening and image processing, and the development of artificial intelligence technology has made it possible to automatically identify the magnification of the microscope objective and diagnose the pathological type of the captured image. Artificial intelligence technology represented by deep learning (such as CNN) has begun to involve lesion classification and other directions, which will further assist doctors in quickly identifying diseases and also help young doctors as a reference for judgment.
[0004] Currently, most of the detection methods for pathological images are based on single picture shooting and saving or multiple high magnification microscopic picture splicing methods, which have poor real-time performance and are cumbersome to operate. Therefore, a method is needed that can select effective frames directly from real-time video streams and input them into a model for disease diagnosis, accurately locate the position of cancer-related lesions, and achieve what you see is what you get. SUMMARY
[0005] In order to solve the problems in the background art, the purpose of the present application is to provide a method for automatically detecting lesions in real-time pathological microscopic images, which can assist doctors in quickly and accurately locating the position of related lesions during clinical diagnosis.
[0006] The technical solution adopted by the present application is as follows, comprising the following steps:
[0007] Step S1, acquiring the pathological microscopic image collected and magnified by the microscope and inputting it into an image real-time display module;
[0008] Step S2, the image screening module screens each frame of pathological microscopic image input by the image real-time display module to obtain a standardized pathological microscopic image, and inputs the screened standardized pathological microscopic image to the image magnification recognition module, the image preprocessing module and the report generation and review module;
[0009] Step S3, the image magnification recognition module obtains the magnification of the pathological microscopic image input by the image screening module, and inputs the magnification of the pathological microscopic image to the image preprocessing module;
[0010] Step S4, the image preprocessing module cuts the standardized pathological microscopic image according to the magnification to obtain an image block with consistent field of view after cutting, and inputs the pathological microscopic image block to the image classification prediction module;
[0011] Step S5, the image classification prediction module inputs the cut image block into the trained neural network model for pathological analysis and prediction, and inputs the pathological analysis and prediction result to the report generation and review module;
[0012] Step S6, the report generation and review module reads and stores the pathological analysis and prediction result, and generates a pathological report according to the pathological analysis and prediction result combined with the standardized pathological microscopic image.
[0013] In the step S3, the image magnification recognition module mainly consists of a CNN convolutional neural network, which is used to realize four classifications of the magnification of the pathological microscopic image, including 2 times, 4 times, 10 times and 20 times magnification.
[0014] First, the standardized pathological microscopic image output by the image screening module is down-sampled to obtain a 300*300 pixel size pathological microscopic image, and then the 300*300 pixel size pathological microscopic image is input into the trained CNN convolutional neural network for magnification classification. The CNN convolutional neural network outputs four classification results: 2 times, 4 times, 10 times and 20 times magnification.
[0015] In the step S4, the pathological microscopic image is cut according to the magnification, and the specific method is as follows:
[0016] Step S4.1, first obtain the magnification of the pathological microscopic image and make the following judgments:
[0017] If the magnification of the pathological microscopic image is 20 times, the pathological microscopic image is not cut, and the pathological microscopic image is taken as an image block, which directly enters step S4.2;
[0018] Otherwise, a fixed-size sliding window is set according to the magnification of the pathological microscopic image, the pathological microscopic image is traversed in the sliding window mode, the image in the sliding window is cut to obtain a plurality of cut image blocks, and then step S4.2 is entered.
[0019] The size of the sliding window corresponding to each frame of the pathological microscopic image is determined according to the magnification:
[0020] When the magnification of the pathological microscopic image is 2 times, the size of the sliding window is h*h, wherein h represents a preset side length; when the magnification of the pathological microscopic image is 4 times, the size of the sliding window is 2h*2h; and when the magnification of the pathological microscopic image is 10 times, the size of the sliding window is 5h*5h.
[0021] In step S4.2, all the image blocks obtained in step S4.1 are down-sampled to obtain pathological microscopic image blocks with a size of 100*100 pixels.
[0022] The image classification prediction module adopts a trained deep learning neural network model, the image block cut in step S5 with a size of 100*100 pixels is input into the trained deep learning neural network model, and the deep learning neural network model outputs two classification results of each image block: negative and positive.
[0023] In step S6, the report generation and review module draws a heat map according to the standardized pathological microscopic image output by the image screening module and the classification result of the image block output by the image classification prediction module, and obtains the detection result of the tissue according to the heat map:
[0024] If the classification result of all the image blocks in the heat map is negative, it indicates that the detection result of the tissue in the heat map is negative.
[0025] Otherwise, it indicates that the detection result of the tissue in the heat map is positive.
[0026] Finally, a pathological report of the tissue part is generated according to the detection results of all the heat maps.
[0027] The method of the present application adopts a real-time pathological microscopic image automatic analysis system, which comprises an image real-time display module, an image screening module, an image magnification recognition module, an image preprocessing module, an image classification prediction module and a report generation and review module.
[0028] The image real-time display module is used for displaying the input pathological microscopic image of the tissue and inputting the pathological microscopic image into the image screening module.
[0029] An image screening module is configured to screen the pathological microscopic images input by the image real-time display module to obtain standardized pathological microscopic images, and input the screened pathological microscopic images to the image magnification recognition module, the image preprocessing module and the report generation and review module.
[0030] An image magnification recognition module is configured to obtain the magnification of the pathological microscopic images input by the image screening module, and input the magnification corresponding to the pathological microscopic images to the image preprocessing module.
[0031] An image preprocessing module is configured to cut the standardized pathological microscopic images according to the magnification to obtain image blocks with consistent fields of view after cutting, and input the pathological image blocks to the image classification prediction module.
[0032] An image classification prediction module is configured to input the cut image blocks into a neural network model for pathological analysis and prediction, and input the pathological analysis and prediction results to the report generation and review module.
[0033] A report generation and review module is configured to store the pathological analysis and prediction results, and generate a pathological report according to the pathological analysis and prediction results in combination with the standardized pathological microscopic images.
[0034] The pathological microscopic images in the image real-time display module are obtained in the following manner: reading the pre-stored pathological videos in a local storage, or inputting the pathological images collected by an external microscope.
[0035] The image screening module screens the pathological microscopic images in the following manner: detecting and removing the black transition frames, overexposed frames, blurred frames and similar frames in all the pathological microscopic images to standardize the images.
[0036] The image automatic analysis method of the present application can display the video frames under the external lens of the microscope in real time, and input the neural network after image validity discrimination, blur detection and similarity detection of each frame to judge the magnification, so as to complete the image cutting and input the neural network model to obtain the pathological detection results. The system can improve the work efficiency and accuracy of the microscope in diagnosing pathological sections, reduce the work burden of doctors and operators, and at the same time, provide clear diagnostic heat maps for doctors to accurately locate the positions of related lesions, further improving the accuracy of diagnosis.
[0037] The beneficial effects of the present application are as follows:
[0038] 1. The method of the present application can not only detect pathology through local video stream, but also receive video stream transmitted from external image acquisition device of microscope in real time, filter out some invalid, blurred and similar frames based on video stream frame by frame, perform reasonable segmentation on the image after magnification detection to unify the field size and input to the classification prediction module for pathological analysis and prediction of the preprocessed pathological image, and finally display the prediction result on the software system and save the heat map of the diagnosis result to the local for convenient review by doctors. Compared with the way of scanning tissue sections row by row and splicing pictures, the present application occupies less memory and has stronger real-time and continuity.
[0039] 2. The present application mainly analyzes pathological images through a series of algorithm researches such as image validity processing and convolutional neural network prediction model, which can help pathologists to review and diagnose pathological images and greatly accelerate the diagnosis speed of related practitioners, so as to save time for the treatment of patients. BRIEF DESCRIPTION OF DRAWINGS
[0040] Figure 1 is the real-time pathological microscopic image lesion automatic detection method module framework of the present application;
[0041] Figure 2 The pathological image (large intestine hyperplastic polyp image) is processed into a fixed resolution section image under a 2x objective lens;
[0042] Figure 3 is the heat map saved locally after diagnosing (transverse colon) hyperplastic polyp. DETAILED DESCRIPTION
[0043] The present application will be described in detail below in combination with specific implementation cases, and the following implementation cases will help those skilled in the art to further understand the present application, but do not limit the present application in any form.
[0044] As shown in Figure 1 , the microscopic image automatic analysis module includes an image real-time display module 1, an image screening module 2, an image magnification recognition module 3, an image preprocessing module 4, an image classification prediction module 5 and a report generation and review module 6;
[0045] The image real-time display module 1 is used to display the input enlarged pathological microscopic image, and input the pathological microscopic image to the image screening module 2;
[0046] The image screening module 2 is used to screen the pathological microscopic image input by the image real-time display module 1 to obtain a standardized pathological microscopic image, and input the screened pathological microscopic image to the image magnification recognition module 3, the image preprocessing module 4 and the report generation and review module 6;
[0047] An image magnification recognition module 3 is configured to acquire the magnification of the pathological microscopic image input by the image screening module 2, and input the magnification corresponding to the pathological microscopic image to the image preprocessing module 4.
[0048] The image preprocessing module 4 is configured to perform cutting processing on the standardized pathological microscopic image according to the magnification, so as to acquire an image block with consistent field of view after cutting, and input the pathological microscopic image block to the image classification prediction module 5.
[0049] The image classification prediction module 5 is configured to input the image block after cutting to a neural network model for pathological analysis and prediction, and input the pathological analysis and prediction result to the report generation and review module 6.
[0050] The report generation and review module 6 is configured to store the pathological analysis and prediction result, and generate a structured pathological report according to the pathological analysis and prediction result in combination with the standardized pathological microscopic image.
[0051] The acquisition mode of the pathological microscopic image in the image real-time display module 1 includes reading the pathological video pre-stored in the local storage or inputting the pathological image collected by the external microscope.
[0052] Specifically, the microscope is connected with the image collection device, the image collection device collects the pathological image under the microscope to generate a pathological video frame, which can be stored in the local storage or directly input to the image real-time display module 1. Therefore, the pathological microscopic image input in the image real-time display module 1 is all the enlarged image under the microscope.
[0053] The screening processing of the pathological microscopic image by the image screening module 2 includes detecting and removing the black transition frame, overexposed frame, blurred frame and similar frame in the lens switching magnification of all the pathological microscopic images, so as to standardize the images.
[0054] The method of the present application comprises the following steps, as shown in Figure 1
[0055] Step S1, collecting the enlarged pathological microscopic image collected by the microscope and inputting to the image real-time display module 1;
[0056] Step S2, the image screening module 2 performs screening processing on each frame of the pathological microscopic image input by the image real-time display module 1 to acquire the standardized pathological microscopic image, and inputs the screened standardized pathological microscopic image to the image magnification recognition module 3, the image preprocessing module 4 and the report generation and review module 6;
[0057] Step S3, the image magnification recognition module 3 acquires the magnification of the pathological microscopic image input by the image screening module 2, and inputs the magnification of the pathological microscopic image to the image preprocessing module 4;
[0058] Step S4, the image preprocessing module 4 performs cutting processing on the standardized pathological microscopic images according to the magnification, to obtain image blocks with consistent fields of view after cutting, and inputs the pathological microscopic image blocks to the image classification prediction module 5;
[0059] Step S5, the image classification prediction module 5 inputs the cut image blocks into the trained neural network model for pathological analysis and prediction, and inputs the pathological analysis and prediction results to the report generation and review module 6;
[0060] Step S6, the report generation and review module 6 reads and stores the pathological analysis and prediction results, and generates a pathological report according to the pathological analysis and prediction results combined with the standardized pathological microscopic images.
[0061] In step S2, the image screening module 2 screens each frame of pathological microscopic image in the following manner:
[0062] Step S2.1, black transition frame removal:
[0063] Convert each frame of pathological microscopic image from RGB color space to grayscale space, so that the RGB pathological microscopic image is converted into a pathological microscopic grayscale image. When the grayscale ratio D of the pathological microscopic grayscale image is higher than the preset grayscale ratio threshold, the pathological microscopic grayscale image is deleted as a black transition frame. Each frame of pathological microscopic grayscale image that is not deleted is converted from the grayscale space back to the RGB color space to obtain the RGB pathological microscopic image.
[0064] The grayscale ratio D of the pathological microscopic grayscale image is obtained according to the following formula:
[0065] D = N BLACK / N WHITE
[0066] Wherein, N BLACK is the number of pixel points with a grayscale value lower than 50 in the pathological microscopic grayscale image, and N WHITE is the number of pixel points with a grayscale value not lower than 50 in the pathological microscopic grayscale image.
[0067] Step S2.2, overexposed frame removal:
[0068] Obtain the standard deviation of the RGB values of the pixel points in each frame of pathological microscopic image remaining in step S2.1. When the standard deviation of the RGB values of all pixel points in the pathological microscopic image is lower than the preset standard deviation threshold, the pathological microscopic image is deleted as an overexposed frame.
[0069] Step S2.1, blur frame removal:
[0070] Step S2.2, Fourier transform is performed on the remaining frames of the pathological microscopic images to obtain the frequency spectrum information of each frame of the pathological microscopic images, and if the average value of the frequency in the frequency spectrum information of the pathological microscopic image is lower than a preset frequency threshold, the pathological microscopic image is deleted as a blurred frame;
[0071] Step S2.4, similar frame removal:
[0072] Step S2.3, a hash value with a fixed length is obtained for each frame of the remaining pathological microscopic images by using a hash algorithm, and the hash values of two pathological microscopic images are compared, if the difference between the hash values of the two pathological microscopic images is less than a preset difference, the two pathological microscopic images are similar frames, and one of the two similar frames is deleted; the hash values of all pathological microscopic images are compared in pairs until all pathological microscopic images are not similar frames;
[0073] The remaining frames of the pathological microscopic images after the black transition frame, the overexposed frame, the blurred frame and the similar frame removal are used as the standardized pathological microscopic images, and the pixel size of the pathological microscopic images in this example is 5440*3648, and the actual image pixel size will be different due to different acquisition devices;
[0074] In step S3, the image magnification recognition module 3 mainly comprises a trained CNN convolutional neural network, and the CNN convolutional neural network is used to realize four classifications of the pathological microscopic image magnification, and the example includes four classification results of magnification 2 times, magnification 4 times, magnification 10 times and magnification 20 times;
[0075] First, the standardized pathological microscopic images output by the image screening module 2 are down-sampled to obtain pathological microscopic images with a size of 300*300 pixels, and then the 300*300 pixel pathological microscopic images are input into the trained CNN convolutional neural network for magnification classification, and the CNN convolutional neural network outputs four classification results: magnification 2 times, magnification 4 times, magnification 10 times and magnification 20 times. In this example, the CNN convolutional network model is composed of an input layer, a convolutional layer, a pooling layer, a full connection layer and an output layer connected in sequence.
[0076] The function of obtaining the magnification of the microscope objective is that the larger the magnification of the objective used, the smaller the actual field of view range, and in order to unify the field of view range of the image, the window size for subsequent cutting needs to be larger.
[0077] In step S4, the standardized pathological microscopic images are cut according to the magnification to obtain image blocks with consistent fields of view after cutting, and the specific method is as follows:
[0078] Step S4.1, first, the magnification of the pathological microscopic image is obtained and the following judgment is made:
[0079] If the magnification of the pathological microscopic image is 20 times, the pathological microscopic image will not be segmented. Instead, the pathological microscopic image will be treated as an image block and the process will proceed directly to step S4.2.
[0080] Otherwise, a fixed-size sliding window and a preset step size are set according to the magnification of the pathological microscopic image. The pathological microscopic image is traversed by using the sliding window. The image within the sliding window is cut to obtain several cut image blocks, and then the process proceeds to step S4.2.
[0081] The sliding window size corresponding to each frame of pathological microscopic image is determined based on the magnification:
[0082] In this example, when the magnification of the pathological microscopic image is 2x, the size of the sliding window is 418*418; when the magnification of the pathological microscopic image is 4x, the size of the sliding window is 836*836; and when the magnification of the pathological microscopic image is 10x, the size of the sliding window is 2090*2090.
[0083] Step S4.2: Downsample all the image blocks obtained in step S4.1 to obtain pathological microscopic image blocks of 100*100 pixels in size, which are ready to be input into the image classification network model for pathological category prediction.
[0084] In practice, when using a sliding window to cut pathological microscopic images, non-overlapping slide sections are made on the image from left to right and from top to bottom. If there are remaining images after cutting several fixed-size windows, the size of the remaining images is assessed. If the size of the remaining images exceeds a preset value, the remaining images are filled with white background until they match the window size to cut into new image blocks. For example... Figure 2 The image shown illustrates the effect of cutting a row of a colorectal polyp pathology image into 418*418 image blocks. However, when cutting to the bottom and right edge, the remaining content does not meet the full cut size. Therefore, a validity check is performed on the remaining image content. If the valid information accounts for a significant portion, a white background is filled into a window up to the cut size. After each image is cut, it is uniformly downsampled to 100*100 pixel image blocks for input into an image classification network model for pathology category prediction.
[0085] The image classification prediction module 5 uses a trained deep learning neural network model. The 100*100 pixel image blocks cut in step S5 are input into the trained deep learning neural network model. The deep learning neural network model outputs two classification results for each image block: negative and positive.
[0086] In this example, the deep learning neural network model is composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer connected in sequence. The input layer performs image input, where the image is a 100*100 image block cut and down-sampled from a unified field of view. The image can be regarded as a two-dimensional array composed of individual pixel points, each pixel point having its own pixel value. The convolutional layer extracts features from the input image. In a convolutional neural network, there can be one or more convolutional layers. To reduce computation and improve calculation speed, a pooling layer is generated for the convolutional layer, configured to down-sample the feature map to generate a smaller feature map. The fully connected layer is configured to map the feature map obtained by the pooling layer to the sample label space, and finally outputs the classification result through the output layer. The loss function for training the deep learning neural network model is:
[0087] loss(x, class) = weight[class] (-x[class] + log(∑ j exp(x[j])))
[0088] Where: loss() represents the loss; weight represents the weight; x represents the predicted probability; class represents the corresponding category; and j represents the serial number of all categories. The above cross-entropy with weight is used because the proportions of different disease types in the training set are different. Using cross-entropy with weight as the loss function for training makes the classification accuracy of the trained neural network model higher.
[0089] It can be understood that the trained deep learning neural network model mentioned by the image classification prediction module 5 can be a deep learning classification network model or a deep learning target detection network model for image processing. Taking the deep learning classification network model as an example, the model can be a neural network model obtained by adaptively adjusting parameters and training data based on Inception, VGG, ResNet, SqueezeNet, etc. Taking the deep learning target detection network as an example, the model can be a neural network model obtained by adaptively adjusting parameters and training data based on RCNN, SPP Net, Fast RCNN, etc.
[0090] In step S6, the report generation and review module 6 draws a heat map according to the standardized pathological microscopic image output by the image screening module 2 and the classification results of each image block output by the image classification prediction module 5, and obtains the detection result of the tissue according to the heat map:
[0091] If the classification results of all image blocks in the heat map are negative, it indicates that the detection result of the tissue in the heat map is negative.
[0092] Otherwise, it indicates that the detection result of the thermogram is positive.
[0093] Finally, the pathological report of the tissue site is generated according to the detection results of all thermograms.
[0094] The classification results of the image blocks are marked on the corresponding regions of the image blocks in the original standardized pathological microscopic images, and the thermogram is obtained.
[0095] Figure 3 is the thermogram saved locally after diagnosing the transverse colon hyperplastic polyp, and the shaded part in the figure is the image block determined as positive by the model. If the classification results of all image blocks cut from the standardized pathological microscopic images are negative, the detection result of the tissue is negative; otherwise, the detection result of the tissue is positive. Finally, the pathological report is generated according to the detection results of all standardized images and saved to the device of the clinician, which supports review while reducing the time and workload of the clinician in writing the report.
[0096] The system can display the video frame under the microscope external lens in real time, and after performing blur detection, image validity discrimination and similarity detection on each frame, input the neural network for multiple judgment, so as to complete image cutting and input the neural network model to obtain the pathological detection result. The system mainly includes microscopic image real-time display, image screening, image magnification recognition, image preprocessing, image classification prediction, report generation and review modules, which can improve the work efficiency and accuracy of microscopic diagnosis of pathological sections, reduce the work burden of doctors and operators, and at the same time, provide clear diagnostic thermogram for doctors to accurately locate the position of the relevant lesion, further improve the accuracy of diagnosis.
[0097] Through preprocessing and standardization of the video frame, and then inputting into the neural network model for pathological image analysis, it can help the pathologist to quickly review and diagnose the pathological image, and the standardized process can also speed up the diagnosis speed of the relevant practitioners, and strive for time for the treatment of patients.
[0098] Although the specific embodiments of the present application have been described in detail above, the present application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application, and modifications or deformations without creative labor are still within the protection scope of the present application.
Claims
1. A method for automatic lesion detection in real-time pathological microscopic images, characterized in that, Includes the following steps: Step S1: Collect magnified pathological microscopic images with a microscope and input them into the real-time image display module (1); Step S2: The image filtering module (2) filters each frame of pathological microscopic image input by the real-time image display module (1) to obtain standardized pathological microscopic images. Then, the filtered standardized pathological microscopic images are input to the image magnification recognition module (3), the image preprocessing module (4), and the report generation and review module (6), respectively. Step S3: Obtain the magnification of the pathological microscopic image input by the image filtering module (2) through the image magnification recognition module (3), and input the magnification of the pathological microscopic image into the image preprocessing module (4); Step S4: The image preprocessing module (4) cuts the standardized pathological microscopic image according to the magnification to obtain image blocks with consistent field of view after cutting, and inputs the pathological microscopic image blocks into the image classification prediction module (5). Step S5: The image classification and prediction module (5) inputs the cut image blocks into the trained neural network model for pathological analysis and prediction, and inputs the pathological analysis and prediction results into the report generation and review module (6). Step S6, Report Generation and Review Module (6) reads and stores the pathological analysis prediction results, and generates a pathological report based on the pathological analysis prediction results and standardized pathological microscopic images.
2. The method for automatic lesion detection in real-time pathological microscopic images according to claim 1, characterized in that: In step S3, the image magnification recognition module (3) is mainly composed of a CNN convolutional neural network. The CNN convolutional neural network is used to realize the four classifications of the magnification of pathological microscopic images, including magnification of 2 times, magnification of 4 times, magnification of 10 times, and magnification of 20 times.
3. The method for automatic lesion detection in real-time pathological microscopic images according to claim 1, characterized in that: In step S4, the pathological microscopic image is segmented according to the magnification, as follows: Step S4.1: First, obtain the magnification of the pathological microscopic image and make the following judgments: If the magnification of the pathological microscopic image is 20 times, the pathological microscopic image will not be segmented. Instead, the pathological microscopic image will be treated as an image block and the process will proceed directly to step S4.
2. Otherwise, a fixed-size sliding window is set according to the magnification of the pathological microscopic image. The pathological microscopic image is traversed by using the sliding window. The image within the sliding window is cut to obtain several cut image blocks, and then the process proceeds to step S4.
2. The sliding window size corresponding to each frame of pathological microscopic image is determined according to the magnification: When the magnification of the pathological microscopic image is 2x, the sliding window size is h*h, where h represents the preset side length; when the magnification of the pathological microscopic image is 4x, the sliding window size is 2h*2h; when the magnification of the pathological microscopic image is 10x, the sliding window size is 5h*5h. Step S4.2: Downsample all the image blocks obtained in step S4.1 to obtain pathological microscopic image blocks of size 100*100 pixels.
4. The method for automatic lesion detection in real-time pathological microscopic images according to claim 1, characterized in that: The image classification prediction module (5) uses a trained deep learning neural network model. The 100*100 pixel image blocks cut in step S5 are input into the trained deep learning neural network model. The deep learning neural network model outputs two classification results for each image block: negative and positive.
5. The method for automatic lesion detection in real-time pathological microscopic images according to claim 1, characterized in that: In step S6, the report generation and review module (6) draws a heat map based on the standardized pathological microscopic images output by the image screening module (2) and the image block classification results output by the image classification prediction module (5), and obtains the tissue detection results based on the heat map: If the classification results of all image patches in the heatmap are negative, it indicates that the detection result of the tissue in the heatmap is negative. Otherwise, it indicates that the test result of the tissue in the heat map is positive; Finally, a pathological report for the tissue site is generated based on the results of all the thermograms.
6. The method for automatic lesion detection in real-time pathological microscopic images according to claim 1, characterized in that: The method employs a real-time pathological microscopic image automatic analysis system, which includes an image real-time display module (1), an image screening module (2), an image magnification recognition module (3), an image preprocessing module (4), an image classification prediction module (5), and a report generation and review module (6). The real-time image display module (1) is used to display the input pathological microscopic images; The image filtering module (2) is used to filter the pathological microscopic images input by the real-time image display module (1) to obtain standardized pathological microscopic images, and input the filtered pathological microscopic images into the image magnification recognition module (3), the image preprocessing module (4), and the report generation and review module (6). The image magnification recognition module (3) is used to obtain the magnification of the pathological microscopic image input by the image filtering module (2) and input the magnification of the pathological microscopic image to the image preprocessing module (4); The image preprocessing module (4) is used to cut the standardized pathological microscopic image according to the magnification to obtain image blocks with consistent field of view after cutting. The image classification and prediction module (5) is used to input the cut image blocks into the neural network model for pathological analysis and prediction. The report generation and review module (6) is used to store the pathological analysis prediction results and generate a pathological report based on the pathological analysis prediction results and standardized pathological microscopic images.
7. The method for automatic lesion detection in real-time pathological microscopic images according to claim 6, characterized in that: The acquisition methods of pathological microscopic images in the real-time image display module (1) include: reading pathological videos pre-stored in the local memory, or inputting pathological images after they are acquired by an external microscope.
8. The method for automatic lesion detection in real-time pathological microscopic images according to claim 6, characterized in that: The image filtering module (2) performs filtering processing on pathological microscopic images, including detecting and removing black transition frames, overexposed frames, blurred frames and similar frames in all pathological microscopic images to standardize the images.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.
Citation Information
Patent Citations
Optical guidance for surgical, medical, and dental procedures
CN110430809A
Data-efficient multi-scale pathological image classification method
CN116152566A