A fat content analysis device and method for liver pathological slice images

Abstract

The present invention provides a device and method for analyzing fat content in liver pathological slice images. A convolutional neural network connection classifier based on artificial intelligence multi-instance learning is used to grade liver fatty degeneration. Through modules and steps such as image acquisition, identification and screening parameters, and training with a CNN training model, fat content detection, grading, and quantification of liver pathological slice images are achieved, which has a wide range of applications in the field of medical image processing.

Description

Technical Field

[0001] The invention belongs to the field of medical image processing and computer-aided image analysis, and in particular relates to a device and method for grading and quantifying fat content in a digitized image of a liver pathological section. Background Art

[0002] Non-alcoholic fatty liver disease (NAFLD) is the most common chronic liver disease in the world. Among ordinary adults, the prevalence of NAFLD is between 6.3% and 45%, of which 10% to 30% is non-alcoholic steatohepatitis (NASH). With the continuous improvement of the quality of life, the excess nutritional intake has caused the incidence of this disease to show an upward trend.

[0003] Liver pathology sections are the clinical gold standard for pathologists to diagnose whether patients have NASH. Among them, judging the degree of liver steatosis through liver pathology section images requires doctors to have rich diagnostic experience; therefore, generally under the standard clinical diagnosis process, only trained and certified doctors can issue diagnostic reports. In the diagnosis process, due to long-term, large-scale repetitive manual recognition operations of film reading, these experienced doctors face judgment fatigue, resulting in increased deviation and instability of diagnostic results, which will seriously lead to misdiagnosis. Doctors without training and certification in other secondary hospitals are even less able to provide pathology diagnosis services, resulting in a serious lack of medical resources in the institution. At present, the manual and accurate diagnosis of liver pathology section images by doctors alone is unstable and low-throughput, and there is a serious lack of auxiliary tools to assist doctors in objective, high-throughput analysis and diagnosis.

[0004] In recent years, computer vision has been used to detect, identify and segment targets in digital media materials such as images and videos. As a key research direction for the development of medical image processing technology, medical image segmentation tools can assist doctors in making reasonable assessments of patients' pathological sections, which is of great value in assisting doctors' clinical diagnosis and disease assessment as well as subsequent corresponding medication decisions. At present, there are invasive pathological section image diagnosis methods and non-invasive image diagnosis methods for diagnosing the degree of liver fatty degeneration, such as nuclear magnetic resonance spectroscopy (MRI), computed tomography (CT) and Doppler ultrasound. However, liver pathological sections are still the clinical gold standard for diagnosing NASH. At present, neural network model methods in the field of deep learning have been used to analyze medical image information to achieve the grading of fatty degeneration in small areas of the liver, but there is no method for grading and quantifying fat content in panoramic liver pathological section images. Summary of the invention

[0005] In order to objectively evaluate the fat content level in liver pathological sections and to automatically assist in the diagnosis of fatty liver, the present invention proposes an analysis device and method for grading liver fatty degeneration based on a convolutional neural network connection classifier based on artificial intelligence multi-instance learning.

[0006] The present invention discloses an analytical device for grading liver fatty degeneration, which is characterized by comprising:

[0007] (1) Image acquisition and storage module: used to acquire and store full-scan digital images of liver tissue pathological sections, obtain panoramic digital images of the sections, and record them as sample data; and divide the sample data into a training set and a test set for artificial intelligence learning in a conventional ratio of 80%:20%;

[0008] (2) Image recognition and segmentation module: used to set the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and segments the panoramic digitized image into small images according to the above method;

[0009] (3) Image grading and labeling module: the images are classified into four levels: normal, mild, moderate, and severe according to the order of fat content. The small images obtained by segmenting the panoramic digitized images belonging to the training set in module (2) are labeled with the fatty liver disease progression grading. The grading labels set for the segmented small image data set are the same as the grading labels of the panoramic digitized images.

[0010] (4) Neural network training module: Use the convolutional neural network (CNN) pre-trained by Pytorch based on the natural image dataset to train the disease course classification of the small images in the training set in the sample data to obtain the trained CNN model, and use the trained CNN model to predict the disease course classification of the small images in the test set in the sample data to obtain the predicted classification label of the small image;

[0011] (5) Target statistics module: Quantitatively count the four levels of disease course classification of the small images obtained by segmenting a panoramic image in module (4), and obtain the statistics of the four disease course classifications after the panoramic image is segmented into small images;

[0012] (6) Classifier training module: The hierarchical statistical values ​​of the training set samples in module (5) are used to train the classifier. The classifiers used include logistic regression model, support vector machine model, decision tree model and the integrated model of the first three. The trained classifier is obtained and the trained classifier is used to predict the test set samples. The average accuracy of the test set classification is 76%.

[0013] (7) Decision module: A set of graded statistical values ​​obtained in module (5) is input into the classifier trained by module (6), and the result is output as a panoramic image of the graded value of liver fatty degeneration.

[0014] Furthermore, the liver tissue pathological section in module (1) is magnified by a 5-fold objective lens under a microscope, and an image is acquired using a camera.

[0015] Furthermore, in module (2), the size of the small image cut is set to 1024×1024 pixels.

[0016] The second aspect of the present invention discloses a method for grading liver fatty degeneration based on a convolutional neural network connection classifier based on artificial intelligence multi-instance learning, comprising:

[0017] A method for analyzing the grading of liver fatty degeneration, characterized by comprising:

[0018] (1) observing under a microscope using a 5x objective lens magnification and combining with a camera to perform full scan digital image storage of liver tissue pathological sections to obtain a panoramic digital image of the sections, which is recorded as sample data; and dividing the sample data into a training set and a test set for artificial intelligence learning in a conventional ratio of 80%:20%;

[0019] (2) setting the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and setting the cut size of the small image to 1024×1024 pixels, thereby dividing the panoramic digitized image into small images;

[0020] (3) The patients are divided into four levels according to the order of fat content: normal, mild, moderate, and severe. The small images obtained by segmenting the panoramic image belonging to the training set in step (2) are labeled with the fatty liver disease progression grade, and the grade labels set for the segmented small image data set are the same as the grade labels belonging to the panoramic digitized image;

[0021] (4) Use the convolutional neural network (CNN) pre-trained by Pytorch based on the natural image dataset to train the disease course classification of the small images belonging to the training set to obtain the trained CNN model, and use the trained CNN model to predict the disease course classification of the small images belonging to the test set in the sample data to obtain the predicted classification label of the small image;

[0022] (5) performing quantitative statistics on the four levels of disease course classification of the small images obtained by segmenting the panoramic image in step (4) to obtain statistics on the four levels of disease course classification after the panoramic image is segmented into small images;

[0023] (6) The hierarchical statistical values ​​of the training set samples in step (5) are used to train a classifier. The classifiers used include a logistic regression model, a support vector machine model, a decision tree model, and an integrated model of the first three. The trained classifier is obtained and the trained classifier is used to predict the test set samples. The average accuracy of the test set classification is 76%.

[0024] (7) In order to determine the fat content grading value of a panoramic image, a set of graded statistical values ​​obtained in step (5) is input into the classifier trained in step (6), and the result is output as a graded value of liver fatty degeneration in a panoramic image.

[0025] The third aspect of the present invention discloses an analysis device for automatically identifying fat droplets in liver pathological slice images and performing quantitative statistics, characterized in that it comprises:

[0026] (1) Image acquisition and storage module: amplifying liver tissue pathology sections, performing full scan digital image acquisition and storage of liver tissue pathology sections, and obtaining a panoramic digital image of the sections;

[0027] (2) Image recognition and segmentation module: setting the recognition and screening parameters of the cut image so that the computer automatically selects small images with a background blank area not exceeding 20%, and segments the panoramic digitized image into small images;

[0028] (3) Image conversion and threshold processing module: The graphic format of the several small images obtained by module (2) is converted into an 8-bit grayscale image, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binary image;

[0029] (4) Binarized image opening operation processing module: perform opening operation on the binary image obtained by module (3), and the convolution kernel size is set to 3×3;

[0030] (5) Lipid droplet contour detection and recognition module: In order to obtain the pixel points of the lipid droplet target in the image, the binary image obtained by module (4) is used for lipid droplet contour detection and recognition, and a series of lipid droplet target feature contour screening conditions are set. The specific screening conditions are as follows:

[0031] The minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are 100 pixels and 30,000 pixels, respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels;

[0032] The ratio of the convex hull area and the contour area of ​​the lipid droplet target feature is greater than 0.8;

[0033] The matching degree between the convex hull and the contour of the lipid droplet target feature is set to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively;

[0034] The matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is set to be less than 0.35, and the matching degree m2 is calculated as follows: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively;

[0035] (6) Pixel counting module: Binarize all small images obtained by module (5) and count the number of lipid droplet target pixels;

[0036] (7) Debinarization processing module: Debinarize the original image of the small image obtained by module (2), and use the obtained debinarized image to count the number of positive pixel values ​​to obtain the liver tissue area;

[0037] (8) Target statistics module: The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, quantitative indicators such as the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

[0038] Furthermore, the module (1) includes observation under a microscope using a 20x objective lens magnification and full scanning of liver tissue pathology sections in combination with a camera for digital image storage; the image resolution reaches above 0.2738 microns / pixel.

[0039] Furthermore, in the module (2), the size of the small image cut is set to 4096×4096 pixels.

[0040] Furthermore, in the module (3), the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200.

[0041] Furthermore, in the module (7), the binarization threshold is set to 200;

[0042] The fourth aspect of the present invention discloses an analysis method for automatically identifying fat droplets in liver pathological slice images and performing quantitative statistics, characterized in that it includes:

[0043] (1) Observe under a microscope with a 20x objective lens magnification and use a camera to perform full scan digital image storage of liver tissue pathology sections to obtain a panoramic digital image of the section. In order to ensure the accuracy of lipid droplet size quantification, the image resolution should be above 0.2738 μm / pixel;

[0044] (2) Setting the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and setting the cut size of the small image to 4096×4096 pixels, thereby dividing the panoramic digitized image into a number of small images;

[0045] (3) The graphic formats of the several small images obtained in step (2) are converted into 8-bit grayscale images, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binarized image, wherein the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200;

[0046] (4) Perform an opening operation on the binary image obtained in step (3), and set the convolution kernel size to 3×3;

[0047] (5) using the binary image obtained in step (4) for contour detection and recognition of lipid droplets, and setting a series of contour screening conditions for lipid droplet target features;

[0048] (6) According to the method described in step (5), the minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are required to be 100 pixels and 30,000 pixels respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels;

[0049] (7) According to the method described in step (5), the ratio of the convex hull area of ​​the contour of the lipid droplet target feature to the contour area is required to be greater than 0.8;

[0050] (8) According to the method described in step (5), the matching degree between the convex hull of the contour of the lipid droplet target feature and the contour is required to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively;

[0051] (9) According to the method described in step (5), the matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is required to be less than 0.35, and the matching degree m2 is calculated as: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively;

[0052] (10) Binarizing all small images obtained in step (5) and counting the number of lipid droplet target pixels;

[0053] (11) Debinarizing the original image of the small image obtained in step (2), and setting the binarization threshold to 200. Using the debinarized image obtained, the number of pixels with positive values ​​is counted to obtain the liver tissue area;

[0054] (12) The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, quantitative indicators such as the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

[0055] Compared with the prior art, the present invention has achieved the following beneficial effects:

[0056] 1. The present invention uses artificial intelligence methods and image processing related algorithms to determine the degree of fatty degeneration of the liver and to provide objective and accurate quantitative data on the fat content in the liver.

[0057] 2. The present invention is fast, accurate, highly automated, and highly repeatable, and has broad application prospects.

[0058] 3. Compared with manual diagnosis, the present invention is suitable for high-throughput analysis, reduces the misdiagnosis rate, and saves medical resources.

[0059] 4. The technical solution of the present invention can be used for grading and quantifying fat content in panoramic slice images of liver pathology, and is not limited to grading the degree of fatty degeneration in small areas of the liver, and has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 The step of dividing the panoramic image into a plurality of small images according to the present invention;

[0061] Figure 2 These are the 4 different degrees of liver fatty degeneration in the present invention;

[0062] Figure 3 Schematic diagram of output results in Examples 1 and 2 of the present invention;

[0063] Figure 4 Schematic diagram of contour screening in Examples 3 and 4 of the present invention;

[0064] Figure 5is a graph showing the relationship between the percentage of fat droplet area and the grade of liver fatty degeneration in Examples 3 and 4 of the present invention;

[0065] Figure 6 This is a flow chart of the method for grading and quantitatively analyzing liver fat content according to the present invention;

[0066] Figure 7 This is a flow chart of Example 1;

[0067] Figure 8 This is a flow chart of Example 3. DETAILED DESCRIPTION

[0068] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples. The experimental methods in the following examples without specifying specific conditions are carried out according to conventional methods and conditions, or selected according to the product specifications.

[0069] The panoramic digital image described in the present invention is a whole slide image, WSI, or panoramic image for short.

[0070] Example 1

[0071] An analytical device for grading liver fatty degeneration, comprising:

[0072] (1) Image acquisition and storage module: used to acquire and store full-scan digital images of liver tissue pathology sections, obtain panoramic digital images of the sections, and record them as sample data; and divide the sample data into a training set and a test set for artificial intelligence learning in a conventional ratio of 80%:20%; the liver tissue pathology sections are magnified under a microscope using a 5x objective lens, and images are acquired using a camera.

[0073] (2) Image recognition and segmentation module: used to set the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and segments the panoramic digitized image into small images according to the above method ( Figure 1 ), set the small image cutting size to 1024×1024 pixels;

[0074] (3) Image grading and annotation module: The images are divided into four levels: normal, mild, moderate, and severe according to the degree of fat content. The small images obtained by segmenting the panoramic digital images belonging to the training set in module (2) are annotated with the fatty liver disease progression grading ( Figure 2 ), the classification labels set for the segmented small image dataset are the same as the classification labels to which the panoramic digitized image belongs;

[0075] (4) Neural network training module: Use the convolutional neural network (CNN) pre-trained by Pytorch based on the natural image dataset to train the disease course classification of the small images in the training set in the sample data to obtain the trained CNN model, and use the trained CNN model to predict the disease course classification of the small images in the test set in the sample data to obtain the predicted classification label of the small image;

[0076] (5) Target statistics module: Quantitatively count the four levels of disease course classification of the small images obtained by segmenting a panoramic image in module (4), and obtain the statistics of the four disease course classifications after the panoramic image is segmented into small images;

[0077] (6) Classifier training module: The hierarchical statistical values ​​of the training set samples in module (5) are used to train the classifier. The classifiers used include logistic regression model, support vector machine model, decision tree model and the integrated model of the first three. The trained classifier is obtained and the trained classifier is used to predict the test set samples. The average accuracy of the test set classification is 76%.

[0078] (7) Decision module: A set of graded statistical values ​​obtained in module (5) is input into the classifier trained by module (6), and the result is output as a panoramic image of the graded value of liver fatty degeneration.

[0079] According to modules (1)-(5), the WSI of liver steatosis of about 542 liver tissue pathology sections was completed, and the liver steatosis grading values ​​pre-judged by the doctors were used as reference answers to obtain the steatosis grading accuracy of this batch of liver pathology sections. The average accuracy of the four-group grading of the training set of artificial intelligence diagnosis was 85%, and the average accuracy of the four-group grading of the test set was 75% ( Figure 3 ).

[0080] Example 2

[0081] In order to objectively evaluate the fat content in liver pathological sections and to automatically assist in the diagnosis of fatty liver, the present invention proposes an analysis method for grading liver fatty degeneration based on artificial intelligence multi-instance learning convolutional neural network connection classifier. The method consists of two parts: small image course grading analysis and decision system, and its features specifically include the following steps:

[0082] (1) observing under a microscope using a 5x objective lens magnification and combining with a camera to perform full scan digital image storage of liver tissue pathological sections to obtain a panoramic digital image of the sections, which is recorded as sample data; and dividing the sample data into a training set and a test set for artificial intelligence learning in a conventional ratio of 80%:20%;

[0083] (2) Setting the recognition and screening parameters of the cut images so that the computer automatically selects small images with a background blank area not exceeding 20%, and sets the cut size of the small images to 1024×1024 pixels, dividing the panoramic digitized image into small images ( Figure 1 );

[0084] (3) The patients were divided into four levels according to the fat content: normal, mild, moderate, and severe. The small images obtained by segmenting the panorama image belonging to the training set in step (2) were labeled with the fatty liver disease progression classification ( Figure 2 ), the classification labels set for the segmented small image dataset are the same as the classification labels to which the panoramic digitized image belongs;

[0085] (4) Use the convolutional neural network (CNN) pre-trained by Pytorch based on the natural image dataset to train the disease course classification of the small images belonging to the training set to obtain the trained CNN model, and use the trained CNN model to predict the disease course classification of the small images belonging to the test set in the sample data to obtain the predicted classification label of the small image;

[0086] (5) performing quantitative statistics on the four levels of disease course classification of the small images obtained by segmenting the panoramic image in step (4) to obtain statistics on the four levels of disease course classification after the panoramic image is segmented into small images;

[0087] (6) The hierarchical statistical values ​​of the training set samples in step (5) are used to train a classifier. The classifiers used include a logistic regression model, a support vector machine model, a decision tree model, and an integrated model of the first three. The trained classifier is obtained and the trained classifier is used to predict the test set samples. The average accuracy of the test set classification is 76%.

[0088] (7) In order to determine the fat content grading value of a panoramic image, a set of graded statistical values ​​obtained in step (5) is input into the classifier trained in step (6), and the result is output as a graded value of liver fatty degeneration in a panoramic image.

[0089] According to steps (1)-(5), the WSI grading values ​​of liver steatosis of about 542 liver tissue pathological sections were completed, and the liver steatosis grading values ​​pre-judged by the doctors were used as reference answers to obtain the steatosis grading accuracy of this batch of liver pathological sections. The average accuracy of the four-group grading of the training set of artificial intelligence diagnosis was 85%, and the average accuracy of the four-group grading of the test set was 75% ( Figure 3 ).

[0090] Example 3

[0091] An analysis device for automatically identifying fat droplets in liver pathological slice images and performing quantitative statistics, characterized in that it comprises:

[0092] (1) Image acquisition and storage module: amplify the liver tissue pathology section, perform full scan digital image acquisition and storage of the liver tissue pathology section, and obtain a panoramic digital image of the section; the module (1) includes observation under a microscope using a 20x objective lens magnification, and combining a camera to perform full scan digital image storage of the liver tissue pathology section; the image resolution reaches 0.2738 microns / pixel or more.

[0093] (2) Image recognition and segmentation module: Set the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and segments the panoramic digitized image into several small images ( Figure 1 ); Set the small image cropping size to 4096×4096 pixels.

[0094] (3) Image conversion and threshold processing module: The graphic format of the several small images obtained by module (2) is converted into an 8-bit grayscale image, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binarized image; in the module (3), the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200.

[0095] (4) Binarized image opening operation processing module: perform opening operation on the binary image obtained by module (3), and the convolution kernel size is set to 3×3;

[0096] (5) Lipid droplet contour detection and recognition module: In order to obtain the pixel points of the lipid droplet target in the image, the binary image obtained by module (4) is used for lipid droplet contour detection and recognition, and a series of lipid droplet target feature contour screening conditions are set;

[0097] The minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are required to be 100 pixels and 30,000 pixels respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels; after contour screening, the pixel accuracy of the lipid droplet in the image can reach 86.8% ( Figure 4 );

[0098] The ratio of the convex hull area and the contour area of ​​the lipid droplet target feature is required to be greater than 0.8;

[0099] The matching degree between the convex hull and the contour of the lipid droplet target feature is required to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively;

[0100] The matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is required to be less than 0.35, and the matching degree m2 is calculated as follows: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively;

[0101] (6) Pixel counting module: Binarize all small images obtained by module (5) and count the number of lipid droplet target pixels;

[0102] (7) Debinarization processing module: Debinarize the original image of the small image obtained by module (2), and use the obtained debinarized image to count the number of positive pixel values ​​to obtain the liver tissue area; the binarization threshold in the module (7) is set to 200;

[0103] (8) Target statistics module: The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, quantitative indicators such as the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

[0104] (9) The percentage of fat droplet area in approximately 542 liver tissue pathological sections was completed, and the liver fatty degeneration grade values ​​predicted by doctors were used as a reference for grouping, and a relationship diagram between the percentage of fat droplet area and the liver fatty degeneration grade was obtained ( Figure 5 ).

[0105] Example 4

[0106] A method for automatically identifying fat droplets in digitized images of panoramic liver pathology slices and performing quantitative statistics on their size, quantity and location, providing assistance to doctors in quantitatively diagnosing the fat content of pathological slices. Specifically, the following steps are included:

[0107] (1) Observe under a microscope with a 20x objective lens magnification and use a camera to perform full-scan digital image storage of liver tissue pathology sections to obtain a panoramic digital image (whole slide image, WSI) of the section. In order to ensure the accuracy of lipid droplet size quantification, the image resolution should be above 0.2738 μm / pixel;

[0108] (2) Setting the recognition and screening parameters of the cut image so that the computer automatically selects small images with a background blank area not exceeding 20%, and sets the cut size of the small image to 4096×4096 pixels, dividing the panoramic digitized image into several small images ( Figure 1 );

[0109] (3) In order to filter out smaller, irregular non-lipid droplet contour graphics as much as possible, the graphic format of the several small images obtained in step (2) is converted into an 8-bit grayscale image, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binary image, wherein the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200;

[0110] (4) In order to separate the contours of lipid droplets that are too clustered and coherent, and thus improve the accuracy of the number of lipid droplets, an opening operation is performed on the binary image obtained in step (3), and the convolution kernel size is set to 3×3;

[0111] (5) In order to obtain the pixel points of the lipid droplet target in the image, the binary image obtained in step (4) is used for lipid droplet contour detection and recognition, and a series of contour screening conditions for lipid droplet target features are set;

[0112] (6) According to the method described in step (5), the minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are required to be 100 pixels and 30,000 pixels respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels;

[0113] (7) According to the method described in step (5), the ratio of the convex hull area of ​​the contour of the lipid droplet target feature to the contour area is required to be greater than 0.8;

[0114] (8) According to the method described in step (5), the matching degree between the convex hull of the contour of the lipid droplet target feature and the contour is required to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively;

[0115] (9) According to the method described in step (5), the matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is required to be less than 0.35, and the matching degree m2 is calculated as: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively;

[0116] (10) All small images obtained in step (5) are binarized and the number of fat droplet target pixels is counted; after contour screening, the pixel accuracy of fat droplets in the image can reach 86.8% ( Figure 4 );

[0117] (11) In order to obtain the area of ​​liver tissue for calculating the proportion of lipid droplets in liver tissue, the original image of the small image obtained in step (2) is debinarized, and the threshold is set to 200. The number of positive pixel values ​​in the debinarized image is counted to obtain the area of ​​liver tissue;

[0118] (12) The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, quantitative indicators such as the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

[0119] (13) The percentage of fat droplet area in the pathological sections of approximately 542 liver tissues was calculated, and the liver steatosis grade values ​​predicted by the doctors were used as a reference for grouping, thereby obtaining a relationship diagram between the percentage of fat droplet area and the liver steatosis grade ( Figure 5 ).

[0120] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An analysis device for automatically identifying fat droplets in liver pathological slice images and performing quantitative statistics, characterized in that: include: (1) Image acquisition and storage module: amplifying liver tissue pathology sections, performing full scan digital image acquisition and storage of liver tissue pathology sections, and obtaining a panoramic digital image of the sections; (2) Image recognition and segmentation module: setting the recognition and screening parameters of the cut image so that the computer automatically selects small images with a background blank area not exceeding 20%, and segments the panoramic digitized image into small images; (3) Image conversion and threshold processing module: The graphic format of the several small images obtained by module (2) is converted into an 8-bit grayscale image, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binary image; (4) Binarized image opening operation processing module: perform opening operation on the binary image obtained by module (3), and the convolution kernel size is set to 3×3; (5) Lipid droplet contour detection and recognition module: In order to obtain the pixel points of the lipid droplet target in the image, the binary image obtained by module (4) is used for lipid droplet contour detection and recognition, and a series of lipid droplet target feature contour screening conditions are set. The specific screening conditions are as follows: The minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are 100 pixels and 30,000 pixels, respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels; The ratio of the convex hull area and the contour area of ​​the lipid droplet target feature is greater than 0.8; The matching degree between the convex hull and the contour of the lipid droplet target feature is set to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively; The matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is set to be less than 0.35, and the matching degree m2 is calculated as follows: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively; (6) Pixel counting module: Binarize all small images obtained by module (5) and count the number of lipid droplet target pixels; (7) Debinarization processing module: Debinarize the original image of the small image obtained by module (2), and use the obtained debinarized image to count the number of positive pixel values ​​to obtain the liver tissue area; (8) Target statistics module: The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the quantitative indicators of the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

2. The analysis device according to claim 1, wherein The module (1) includes observation under a microscope using a 20-fold objective lens magnification and full scanning of liver tissue pathological sections in combination with a camera for digital image storage; the image resolution reaches above 0.2738 micrometers / pixel.

3. The analysis device according to claim 1, wherein In the module (2), the size of the small image cut is set to 4096×4096 pixels.

4. The analysis device according to claim 1, wherein In the module (3), the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200.

5. The analysis device according to claim 1, wherein In the module (7), the binarization threshold is set to 200.

6. An analysis method for automatically identifying fat droplets in liver pathological slice images and performing quantitative statistics, characterized in that: include: (1) Observe under a microscope with a 20x objective lens magnification and use a camera to perform full scan digital image storage of liver tissue pathology sections to obtain a panoramic digital image of the section. In order to ensure the accuracy of lipid droplet size quantification, the image resolution should be above 0.2738 μm / pixel; (2) Setting the recognition and screening parameters of the cut image so that the computer automatically selects a small image with a background blank area not exceeding 20%, and setting the cut size of the small image to 4096×4096 pixels, thereby dividing the panoramic digitized image into a number of small images; (3) The graphic formats of the several small images obtained in step (2) are converted into 8-bit grayscale images, and adaptive threshold processing and binarization threshold processing are performed respectively. The results of the two are taken as the intersection to obtain a binarized image, wherein the neighborhood size of the adaptive threshold processing is set to 21, the constant is set to 5, and the threshold of the binarization processing is set to 200; (4) Perform an opening operation on the binary image obtained in step (3), and set the convolution kernel size to 3×3; (5) using the binary image obtained in step (4) for contour detection and recognition of lipid droplets, and setting a series of contour screening conditions for lipid droplet target features; (6) According to the method described in step (5), the minimum and maximum values ​​of the contour area of ​​the lipid droplet target feature are required to be 100 pixels and 30,000 pixels respectively, and the length and width of the minimum circumscribed rectangle of the contour are both greater than 10 pixels; (7) According to the method described in step (5), the ratio of the convex hull area of ​​the contour of the lipid droplet target feature to the contour area is required to be greater than 0.8; (8) According to the method described in step (5), the matching degree between the convex hull of the contour of the lipid droplet target feature and the contour is required to be less than 0.5, and the matching degree m1 is calculated as: in A represents the contour, B represents the convex hull of the contour, and are the Hu moments of A and B respectively; (9) According to the method described in step (5), the matching degree between the contour convex hull of the fat droplet target feature and the fat droplet template is required to be less than 0.35, and the matching degree m2 is calculated as: in A represents the outline, B represents the lipid droplet template, and are the Hu moments of A and B respectively; (10) Binarizing all small images obtained in step (5) and counting the number of lipid droplet target pixels; (11) performing a debinarization process on the original image of the small image obtained in step (2), setting the binarization threshold to 200, and using the debinarized image to count the number of positive pixel values ​​to obtain the liver tissue area; (12) The area and number of fat droplets identified in each small image and the area of ​​the liver tissue section are statistically summarized to calculate the percentage of the fat droplet area in the entire liver tissue section to the area of ​​the liver tissue section. This percentage value is used as the fat content in the quantitative pathological section image. At the same time, the individual area of ​​all fat droplets in the entire liver tissue section, the coordinates of the fat droplets, the quantitative indicators of the total number of fat droplets, and the derived numerical indicators between the above quantitative indicators are also obtained.

Images