An endoscope image quality evaluation method based on superpixel segmentation

Abstract

The application discloses an endoscope image quality evaluation method based on superpixel segmentation. Firstly, the effective area of the endoscope image is obtained through threshold binaryzation and morphological calculation. Secondly, superpixel segmentation is performed on the endoscope image. Thirdly, the mean value calculation of the HSV color space is performed on each clustering block after the superpixel segmentation. Finally, the mean value of each superpixel block is compared with the normal image HSV threshold range, and the proportion of poor quality area is obtained, so as to evaluate the quality of the endoscope image as a whole. The endoscope image quality evaluation method is consistent with the subjective evaluation of doctors, has high reliability, can assist doctors in screening high-quality endoscope images, and provides help for the clinical diagnosis of doctors and the provision of excellent data sets for artificial intelligence assisted diagnosis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of endoscopic image quality evaluation, and particularly relates to an endoscopic image quality evaluation method based on superpixel segmentation. BACKGROUND

[0002] In endoscopic images, due to the fact that the images themselves may contain artifacts such as highlights, a large number of bubbles and mucus, and also due to the problem of artificial shooting, some images may have a large range of light spots or shadows, and the movement of the camera may cause motion blur. These artifacts can seriously affect the image quality of the endoscope, and the image quality of the endoscope will not only affect the judgment of the patient's condition by the doctor, but also affect the artificial intelligence assisted diagnosis. Whether it is medical judgment or scientific research, high-quality endoscopic images are needed.

[0003] At present, the quality evaluation of endoscopic images is mainly through subjective evaluation by doctors. A large number of endoscopic images are taken for each diagnosis of a patient, and good pictures need to be selected for diagnosis and treatment. This method requires a lot of time. Zhang et al. (Zhang T, Wang L, Gu J, et al. Design and Implementation of A Real-time Capsule Endoscope Image Assessment Method. The 3 rd International Conference onBioinformatics and Biomedical Engineering.) proposed a method of calculating the gradient field by Sobel operator to evaluate the endoscopic image without reference, but the effect is general, because the correlation between the texture feature difference of the endoscopic image and the image quality is not high. Kamen et al. (Ali Kamen, Shanhui Sun, Shaohua Wan, et al. Automatic tissue differentiation based on confocal endomicroscopic images for intraoperative guidance in neurosurgery. Biomed Res Int., 2016.) calculated the image entropy to select high-quality images, but the experimental results of this method are poor, and the results are quite different from the subjective judgment of doctors, and cannot effectively help evaluate the image quality. SUMMARY

[0004] The application aims to solve the time-consuming and laborious problem that doctors need to manually screen good images from a large number of endoscopic images for diagnosis, and proposes an endoscopic image quality evaluation method with high reliability, which can effectively help doctors evaluate image quality and facilitate the collection of high-quality endoscopic images for diagnosis and screening by evaluating the quality of endoscopic images through the method and obtaining results with small differences from the subjective judgment of doctors.

[0005] In order to achieve the object of the application, the application discloses an endoscopic image quality evaluation method based on superpixel segmentation, characterized in that it comprises the following steps:

[0006] Step 1, pre-processing the image shot by the endoscope, obtaining the required effective image area through threshold binarization and morphological method (generating a mask mask, and processing to obtain the required effective image area);

[0007] Step 2, using linear iterative clustering algorithm to perform SLIC superpixel segmentation on the pre-processed endoscopic image;

[0008] Step 3, for the segmented sub-area, statistics its HSV color space channel mean;

[0009] Step 4, defining the area not within the preset normal threshold range of endoscopic image as a poor quality area, and the poor quality area accounts for more than 25% of the whole image, that is, a poor quality picture, using the method to evaluate the quality of the endoscopic image data provided by the hospital, and comparing the results with the subjective evaluation of doctors.

[0010] Further, step 1 uses threshold binarization to remove the black area in the image, leaving only the effective area of the endoscopic image and the text description area; a circular structural element kernel with a radius of 3 is created, and morphological processing of corrosion and expansion is performed on the remaining image area, and after opening operation, the text area is removed, leaving only the effective area of the endoscopic image, which specifically includes:

[0011] Step 1-1, threshold binarization of the image, setting the pixel value greater than the threshold value 15 to 255 and the pixel value less than the threshold value to 0, for removing the black background information of the image;

[0012] Step 1-2, creating a circular structure kernel B with a radius of 3, and convolving the binarized image A with the kernel B Remove the text information interference of the image through morphological method;

[0013] Step 1-3, morphological erosion operation on the image: using the anchor point of the kernel B, scanning each pixel of the image A, the kernel and the covered binary image are convolved, and the result is 1, then the pixel value is 1;

[0014] Step 1-4, morphological dilation operation is performed on the image: using the anchor point of kernel B, each pixel of image A is scanned, and the kernel is convolved with the binary image it covers. If the result is 0, the pixel value is 0.

[0015] Further, in step 2, the endoscope image is subjected to SLIC superpixel segmentation, the color image is converted into a 5-dimensional feature vector in the CIELAB color space and XY coordinate system, and then a distance metric criterion is constructed for the 5-dimensional feature vector, and the image pixels are locally clustered. A superpixel is a small area composed of a series of pixels that are adjacent in position and similar in color, brightness or texture features.

[0016] Further, step 2 specifically includes:

[0017] Step 2-1, converting the color image into a 5-dimensional feature vector in the CIELAB color space and XY coordinate system;

[0018] Step 2-2, clustering according to the distance criterion constructed from the 5-dimensional feature vector, initializing the seed points, i.e. the cluster centers: according to the set number of superpixels, evenly distributing the seed points in the image;

[0019] Step 2-3, reselecting the seed points in the n*n field of the seed points;

[0020] Step 2-4, assigning a class label to each pixel point in the field around each seed point;

[0021] Step 2-5, distance measurement, including color distance and spatial distance, for each searched pixel point, the distance from it to the seed point is calculated respectively; since each pixel point will be searched by multiple seed points, the seed point corresponding to the minimum value is selected as the cluster center of the pixel point;

[0022] Step 2-6, iterative optimization, each pixel point will be searched by multiple seed points and the distance will be calculated, the smallest distance is selected as the seed center of the pixel point, until the seed center does not change.

[0023] Further, in step 2, 100 seed points are set, the image has N pixel points in total, and K superpixels of the same size are pre-segmented, the distance between adjacent seed points is S=Sqrt(N / K);

[0024] The distance between each pixel point and the seed point is calculated, and the distance measurement method is:

[0025]

[0026]

[0027]

[0028] d c is color distance, d s is spatial distance, S is maximum spatial distance, D is distance metric, x, y are horizontal and vertical coordinates of pixel point, l, a, b are pixel color values.

[0029] Further, step 3 specifically comprises:

[0030] Step 3-1, extracting the endoscope image region after superpixel segmentation, converting each superpixel region to HSV color space, and HSV color space representing hue, saturation and brightness respectively;

[0031] Step 3-2, by testing a large number of high-quality endoscope image datasets, the threshold ranges of H, S and V channels are set to [5, 20], [110, 205] and [110, 220] respectively (the present application obtains the HSV color space threshold range of the excellent image region as H[5, 20], S[110, 205], V[110, 220] by statistically analyzing the HSV color features histogram of a large number of endoscope image regions);

[0032] Step 3-3, calculating the HSV mean value of each superpixel block of the to-be-tested endoscope image.

[0033] Further, in step 3, the method for converting RGB color space to HSV color space is:

[0034]

[0035] wherein, max is the maximum of r, g, b, and min is the minimum of r, g, b;

[0036] The method for calculating the HSV color space mean value of the superpixel block is:

[0037]

[0038]

[0039]

[0040] wherein, H AVG is the hue H mean value, S AVG is the saturation S mean value, V AVG is the brightness V mean value, and k is the number of pixel points.

[0041] Further, step 4 is specifically: when the HSV color space mean value of the to-be-tested image superpixel block is not in the normal image HSV threshold range, the region is considered to be a poor quality region; when the HSV color space mean value of the to-be-tested image superpixel block is in the normal image HSV threshold range, the region is considered to be a normal region; finally, the poor quality region proportion is counted, and when the poor quality region proportion is greater than 25%, the image is considered to be a poor quality endoscope image.

[0042] Further, step 4 specifically includes:

[0043] Step 4-1, the superpixel block HSV channel value exceeding the threshold range is judged to be a poor quality region, and when the poor quality region proportion is greater than 25%, the image is determined to be a low-quality image, which has a great influence on the diagnosis of doctors and artificial intelligence auxiliary diagnosis;

[0044] Step 4-2, the endoscope image quality evaluation method based on superpixel segmentation is tested and experimented on the image data set of multi-center clinical patients, and the experimental results are compared with the results of conventional image quality evaluation based on Sobel gradient operator, entropy calculation and the like;

[0045] Step 4-3, the experimental results are compared with the subjective evaluation of doctors, and the reliability and consistency of the statistical method are counted. Finally, the present application has high reliability, and the prediction result is consistent with the subjective evaluation of doctors, which is of great help to the clinical diagnosis of doctors.

[0046] Compared with the prior art, the present application has the following significant progress: 1) an image quality evaluation method based on superpixel segmentation is proposed, which solves the problem that doctors mainly manually screen high-quality images for diagnosis, which is time-consuming and laborious. The method has high reliability, and the evaluation result is consistent with the subjective evaluation of doctors, which can assist doctors in screening high-quality images for diagnosis and treatment, and provide help for endoscopy; 2) the HSV color space is used to evaluate the quality of the endoscope image, and the color feature of the endoscope image is more obvious than the shape and texture feature. Compared with the RGB color space, the HSV color space is closer to human perception, and the HSV is more sensitive to common brightness distortion, bubble mucus and other artifacts of the endoscope image, and has good identification of image artifacts; 3) compared with the gradient, information entropy, full image color space mean and other commonly used image quality evaluation methods, the present application has better image quality evaluation effect, the superpixel segmentation divides the image into a series of sub-regions composed of pixels with similar features, and the feature statistics of each superpixel block can better reflect the local features of the image, which reflects the quality of the image from the local to the whole, and is more accurate and practical than the average feature calculation of the whole image.

[0047] To more clearly illustrate the functional characteristics and structural parameters of the present application, the following further describes the present application in conjunction with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0049] Figure 1 It is an overall flowchart of an endoscope image quality evaluation method based on superpixel segmentation;

[0050] Figure 2 It is an endoscope image preprocessing diagram;

[0051] Figure 3 It is an endoscope image superpixel segmentation diagram;

[0052] Figure 4 It is a superpixel segmentation block diagram;

[0053] Figure 5 It is a superpixel segmentation block HSV diagram;

[0054] Figure 6 It is an image original data and corresponding HSV diagram;

[0055] Figure 7 It is an image test result diagram. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0057] As shown in the figure, Figure 1 In the embodiment, an endoscope image quality evaluation method based on superpixel segmentation is disclosed, which comprises the following steps:

[0058] First, read the image data, binarize the image with a pixel value of 15 as a threshold, create a circular structural element with a radius of 3 as a kernel, and perform an opening operation on the binarized image region to obtain the ROI region by morphological operation, and remove the text region in the image.

[0059] Next, the image after preprocessing is subjected to SLIC superpixel segmentation, the color image is converted into a 5-dimensional feature vector in the CIELAB color space and XY coordinates, then a distance measurement standard is constructed for the 5-dimensional feature vector, the image pixels are locally clustered, and 100 compact and approximately uniform superpixel regions are generated.

[0060] Then, the HSV color space threshold range of high-quality endoscopic images is obtained through experiments, and the mean value of the HSV color space in each superpixel block of the image to be tested is calculated. The calculated mean value and the HSV threshold of high-quality endoscopic images are compared to calculate the image proportion of poor-quality areas.

[0061] Finally, the image to be tested with a poor-quality area proportion greater than 25% is identified as a low-quality image. The experimental results are compared with the subjective evaluation of doctors, and it is found that the endoscopic image quality evaluation results are highly reliable and consistent with the subjective evaluation of doctors, which is convenient for doctors to diagnose and provides excellent data set for artificial intelligence assisted diagnosis.

[0062] Embodiment

[0063] As shown in Figure 2 , Figure 2 is the endoscopic image preprocessing result image. Through threshold binaryzation processing and morphological operation, the text area and black area of the image are removed, and the endoscopic image area is retained.

[0064] As shown in Figure 3 , Figure 3 is the endoscopic image superpixel segmentation result image. After superpixel segmentation of the endoscopic image, 100 superpixel blocks are generated, each area has similarity and the characteristic characteristics are more representative.

[0065] As shown in Figure 4 , Figure 5 , Figure 6 As shown in Figure 4 , Figure 5 , Figure 6 are the endoscopic image superpixel segmentation block image, block HSV image and excellent image HSV image respectively. It can be found that high-quality and low-quality images have great differences in HSV color space. After superpixel segmentation, the mean value of the HSV color space in each block can be calculated to determine whether it is within the normal image threshold range.

[0066] As shown in Figure 7 , Figure 7 is the quality evaluation result obtained by the endoscopic image quality evaluation method based on superpixel segmentation. It can be found that superpixel segmentation and HSV color space can well evaluate the quality of endoscopic images. The evaluation result is consistent with the subjective evaluation result of doctors, and the reliability is greatly improved compared with Sobel gradient operator and entropy calculation quality evaluation methods. It is suitable for endoscopic image quality evaluation scene and provides convenience for doctors to diagnose patients.

[0067] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.

[0068] While the embodiments of the application have been shown and described herein, it will be understood by those skilled in the art that many changes, modifications, substitutions and alterations to these embodiments can be made without departing from the principles and spirits of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An endoscopic image quality evaluation method based on superpixel segmentation, characterized by, The method comprises the following steps: Step 1, pre-process the image shot by the endoscope, and obtain the required effective image area by threshold binarization and morphological method processing; Step 2, perform SLIC superpixel segmentation on the pre-processed endoscope image by using a linear iterative clustering algorithm; Step 3, for the segmented sub-regions, count the average value of the HSV color space channel; Step 4, define the area not within the preset normal threshold range of the endoscope image as a poor-quality area, and the poor-quality area accounts for more than 25% of the whole image. 2.The endoscopic image quality evaluation method based on superpixel segmentation according to claim 1, characterized in that, Step 1 uses threshold binarization to remove the black area in the image, leaving only the endoscope image effective area and the text description area; a circular structural element kernel with a radius of 3 is created, and morphological processing of erosion and dilation is performed on the remaining image area. After the opening operation, the text area is removed, and only the endoscope image effective area is left, which specifically includes: Step 1-1, threshold binarization processing of the image, setting the pixel value greater than the threshold value 15 to 255 and the pixel value less than the threshold value to 0, for removing the black background information of the image; Step 1-2, create a circular structure core B with a radius of 3, and convolve the binary image A with the core B Remove the image text information interference by morphological method; Step 1-3, morphological erosion operation on the image: using the anchor point of the kernel B, scanning each pixel of the image A, and performing convolution operation on the kernel and the covered binary image. If the result is all 1, the pixel value is 1; Step 1-4, morphological dilation operation on the image: using the anchor point of the kernel B, scanning each pixel of the image A, and performing convolution operation on the kernel and the covered binary image. If the result is all 0, the pixel value is 0. 3.The endoscopic image quality evaluation method based on superpixel segmentation of claim 1, wherein, In step 2, the endoscope image is subjected to SLIC superpixel segmentation, the color image is converted into a 5-dimensional feature vector in the CIELAB color space and XY coordinate system, and then a distance measurement standard is constructed for the 5-dimensional feature vector to locally cluster the image pixels. A superpixel is a small area composed of a series of adjacent pixels with similar color, brightness or texture features.

4. The endoscopic image quality evaluation method based on superpixel segmentation according to claim 1 or 3, characterized in that, Step 2 specifically includes: Step 2-1, converting the color image into a 5-dimensional feature vector in the CIELAB color space and XY coordinate system; Step 2-2, clustering according to the distance standard constructed from the 5-dimensional feature vector, initializing the seed points, i.e. the clustering centers: distributing the seed points evenly in the image according to the set number of superpixels; Step 2-3, reselecting the seed points in the n*n field of the seed points; Step 2-4, assigning a class label to each pixel point in the field around each seed point; Step 2-5, performing distance measurement, including color distance and spatial distance. For each searched pixel point, the distance from it to the seed point is calculated respectively. Since each pixel point is searched by multiple seed points, the seed point corresponding to the minimum value is selected as the clustering center of the pixel point; Step 2-6, iterative optimization. Each pixel point is searched by multiple seed points and the distance is calculated. The minimum distance is selected as the seed center of the pixel point, until the seed center does not change.

5. The endoscopic image quality evaluation method based on superpixel segmentation according to claim 4, characterized in that, In step 2, 100 seed points are set, the image has N pixel points, and K superpixels of the same size are pre-segmented. The distance between adjacent seed points is S=Sqrt(N / K). The distance between each pixel point and the seed point is calculated, and the distance measurement method is: d c For color distance, d s For spatial distance, S is the maximum spatial distance, D is the distance metric, x, y are the horizontal and vertical coordinates of the pixel, and l, a, b are the pixel color values.

6. The endoscopic image quality evaluation method based on superpixel segmentation according to claim 1, characterized in that, Step 3 specifically comprises: Step 3-1, extracting the endoscope image region after superpixel segmentation, and converting each superpixel region to HSV color space, which represents hue, saturation and brightness respectively; Step 3-2, through a large number of tests on high-quality endoscopic image data sets, the threshold ranges of H, S and V channels are set to [5, 20], [110, 205] and [110, 220] respectively; Step 3-3, calculating the HSV mean value of each superpixel block of the test endoscopic image.

7. The endoscopic image quality evaluation method based on superpixel segmentation according to claim 6, characterized in that, In step 3, the method of converting RGB color space to HSV color space is: Wherein, max is the maximum of r, g, b, and min is the minimum of r, g, b; The method for calculating the HSV color space mean value of the superpixel block is: Wherein, H AVG is the hue H mean value, S AVG is the saturation S mean value, V AVG is the brightness V mean value, and k is the number of pixel points. 8.The endoscopic image quality evaluation method based on superpixel segmentation of claim 1, wherein, Step 4 specifically is: when the HSV color space mean value of the test image superpixel block is not within the normal image HSV threshold range, it is considered that the region is a poor quality region; when the HSV color space mean value of the test image superpixel block is within the normal image HSV threshold range, it is considered that the region is a normal region; finally, the proportion of poor quality regions is calculated, and when it is greater than 25%, it is considered to be a poor quality endoscopic image. 9.The endoscopic image quality evaluation method based on superpixel segmentation according to claim 1 or 8, characterized in that, Step 4 specifically comprises: Step 4-1, if the HSV channel value of the superpixel block exceeds the threshold range, it is judged to be a poor quality region, and when the proportion of poor quality regions is greater than 25%, the image is identified as a low-quality image; Step 4-2, the endoscopic image quality evaluation method based on superpixel segmentation is tested and tested in the image data set of multi-center clinical patients, and the experimental results are compared with the results of conventional Sobel gradient operator and entropy image quality evaluation; Step 4-3, compare the experimental results with the subjective evaluation of doctors, and statistically analyze the reliability and consistency of the method.

Images