Endoscope image reflection point identification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of computer vision, and provides an endoscope image reflective point identification method, device and equipment and a storage medium in order to accurately identify a reflective point in an endoscope image, and the method comprises the following steps: segmenting a region of interest through image segmentation, dividing the region of interest into a plurality of pixel sets of interest according to the gradient information of the region of interest, and obtaining a plurality of comparison sets corresponding to each pixel set of interest; for each interested pixel set, calculating a reflective performance parameter and a reflective performance weight of each comparison set corresponding to the interested pixel set; according to the reflective performance parameter and reflective performance weight of each comparison set corresponding to the pixel set of interest, calculating a probability parameter that the pixel set of interest is a pixel set in reflective points, and calculating a probability parameter that the region of interest is a reflective point; and judging whether the region of interest is the reflective point or not according to the probability parameter that the region of interest is the reflective point.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for identifying a specular point in an endoscopic image, an apparatus for identifying a specular point in an endoscopic image, an endoscopic image processing device, and a non-transitory computer-readable storage medium. Background Art

[0002] Medical endoscopes can collect images in the human body cavity to achieve the examination or treatment of lesions. Since the human body cavity generally does not have a natural light source, light needs to be turned on when the endoscope collects images. However, mucus or smooth areas on the inner wall of the human body cavity may cause specular points in the collected images. Specular points will seriously affect the observation of the actual situation at that position, and will also exacerbate the doctor's visual fatigue in the intraoperative endoscope scenario. Therefore, many solutions for filtering specular points in endoscopic images have been proposed in the related art.

[0003] The premise of effectively filtering specular points in endoscopic images is to accurately identify the specular points in the endoscopic images. Since specular points and actual white points are similar in some parameter features, current related techniques such as threshold segmentation and classification are likely to misidentify actual white points in the image as specular points. These actual white points may be lesions, fat particles, cleaning residues (such as food residues), etc. If they are filtered out, it will also affect the observation of the actual situation at the examination or treatment position. Taking the intraoperative endoscope scenario as an example, especially for the target lesion, that is, the lesion targeted by this operation, if it is misidentified as a specular point and processed to be weakened or filtered out, it will lead to the abnormal implementation of the operation, which is completely unacceptable. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method, an apparatus, a device, and a storage medium for identifying a specular point in an endoscopic image, which can accurately identify the specular point in the endoscopic image, avoid misidentifying actual targets such as actual white points as specular points, thereby avoiding misprocessing of actual targets during subsequent processing of specular points in endoscopic images, and ensuring the quality of the processed endoscopic image.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A method for identifying the reflection points of an endoscope image, comprising the following steps: obtaining the original endoscope image; performing grayscale processing on the original endoscope image to obtain a grayscale endoscope image; performing image segmentation on the grayscale endoscope image to segment out the highlighted areas; performing color space conversion processing on the original endoscope image to obtain a converted endoscope image; taking the corresponding areas of the highlighted areas in the grayscale endoscope image in the converted endoscope image as regions of interest, and dividing the regions of interest into multiple sets of interested pixels according to the gradient information of the regions of interest, and obtaining multiple comparison sets corresponding to each set of interested pixels; for each set of interested pixels, calculating the reflection performance parameter and the reflection performance weight of each comparison set corresponding to the set of interested pixels; according to the reflection performance parameter and the reflection performance weight of each comparison set corresponding to the set of interested pixels, calculating the probability parameter that the set of interested pixels is a set of pixels in the reflection points, and calculating the probability parameter that the region of interest is a reflection point; and judging whether the region of interest is a reflection point according to the probability parameter that the region of interest is a reflection point.

[0007] Further, dividing the region of interest into multiple sets of interested pixels according to the gradient information of the region of interest, and obtaining multiple comparison sets corresponding to each set of interested pixels, specifically including: calculating the sum of the gradients of all the pixels in the region of interest; setting the direction of the sum of the gradients as the horizontal direction; dividing each row of pixels in the region of interest into a set of interested pixels; taking the M sets of pixel sets adjacent to the left and the N sets of pixel sets adjacent to the right of each set of interested pixels as the corresponding comparison sets, where the number of pixels in each set of pixel sets is the same as the number of pixels in the corresponding set of interested pixels, and both M and N are positive integers, and M is equal to or differs from N by 1.

[0008] Further, the reflection performance parameter of each comparison set corresponding to the set of interested pixels is:

[0009]

[0010] where L k is the reflection performance parameter of the k-th comparison set corresponding to the set of interested pixels, is the average value of the brightness of all the pixels in the k-th comparison set corresponding to the set of interested pixels, is the average value of the brightness of all the pixels in the set of interested pixels, H k is the average value of the hues of all the pixels in the k-th comparison set corresponding to the set of interested pixels, and H is the average value of the hues of all the pixels in the set of interested pixels, is the average value of the saturation of all the pixels in the k-th comparison set corresponding to the set of interested pixels, The average saturation of all pixels in the set of pixels of interest.

[0011] Furthermore, the specular reflectance performance weight of each comparison set corresponding to the set of pixels of interest is:

[0012]

[0013] where ω k is the specular reflectance performance weight of the k-th comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, C k is the mutation parameter of the k-th comparison set corresponding to the set of pixels of interest, which is the average of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, and C is the mutation parameter of the set of pixels of interest, which is the average of the mutation parameters of all pixels in the set of pixels of interest.

[0014] Furthermore, the mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th comparison set is:

[0015]

[0016] where c i is the mutation parameter of the i-th pixel in the set of pixels of interest or the comparison set, n is the number of pixels within the preset neighborhood range of the i-th pixel, G i,j is the gray value of the j-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set, and G i,j-1 is the gray value of the (j - 1)-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set.

[0017] Furthermore, the probability parameter that the set of pixels of interest is the set of pixels in the specular points is:

[0018]

[0019] where P x is the probability parameter that the x-th set of pixels of interest in the region of interest is the set of pixels in the specular points.

[0020] The probability parameter that the region of interest is a specular point is:

[0021]

[0022] where P is the probability parameter that the region of interest is a specular point, and X is the total number of sets of pixels of interest in the region of interest.

[0023] An identification device for the reflective points of an endoscope image, comprising: an acquisition module for acquiring the original endoscope image; a grayscale processing module for performing grayscale processing on the original endoscope image to obtain a grayscale endoscope image; a segmentation module for performing image segmentation on the grayscale endoscope image to segment out the highlighted area; a conversion processing module for performing color space conversion processing on the original endoscope image to obtain a converted endoscope image; a division module for taking the corresponding area of the highlighted area in the grayscale endoscope image in the converted endoscope image as the region of interest, and dividing the region of interest into a plurality of sets of interested pixels according to the gradient information of the region of interest, and obtaining a plurality of comparison sets corresponding to each set of interested pixels; a first calculation module for calculating, for each set of interested pixels, the reflectivity performance parameter and the reflectivity performance weight of each comparison set corresponding to the set of interested pixels; a second calculation module for calculating, according to the reflectivity performance parameter and the reflectivity performance weight of each comparison set corresponding to the set of interested pixels, the probability parameter that the set of interested pixels is the pixel set in the reflective point, and calculating the probability parameter that the region of interest is the reflective point; a judgment module for judging whether the region of interest is the reflective point according to the probability parameter that the region of interest is the reflective point.

[0024] Further, the division module is specifically configured to: calculate the sum of the gradients of all the pixels in the region of interest; set the direction of the sum of the gradients to the horizontal direction; divide each row of pixels in the region of interest into a set of interested pixels; and take the M sets of pixel sets adjacent to the left and the N sets of pixel sets adjacent to the right of each set of interested pixels as the corresponding comparison sets, where the number of pixels in each set of pixel sets is the same as the number of pixels in the corresponding set of interested pixels, and both M and N are positive integers, and M is equal to or differs from N by 1.

[0025] Further, the reflectivity performance parameter of each comparison set corresponding to the set of interested pixels is:

[0026]

[0027] Where L k is the reflectivity performance parameter of the kth comparison set corresponding to the set of interested pixels, B k is the average value of the brightness of all the pixels in the kth comparison set corresponding to the set of interested pixels, B is the average value of the brightness of all the pixels in the set of interested pixels, is the average value of the hue of all the pixels in the kth comparison set corresponding to the set of interested pixels, is the average hue of all pixels in the set of pixels of interest, is the average saturation of all pixels in the k-th comparison set corresponding to the set of pixels of interest, is the average saturation of all pixels in the set of pixels of interest.

[0028] Further, the specular reflection performance weight of each comparison set corresponding to the set of pixels of interest is:

[0029]

[0030] where ω k is the specular reflection performance weight of the k-th comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, C k is the mutation parameter of the k-th comparison set corresponding to the set of pixels of interest, which is the average of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, and C is the mutation parameter of the set of pixels of interest, which is the average of the mutation parameters of all pixels in the set of pixels of interest.

[0031] Further, the mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th comparison set is:

[0032]

[0033] where c i is the mutation parameter of the i-th pixel in the set of pixels of interest or the comparison set, n is the number of pixels within the preset neighborhood range of the i-th pixel, G i,j is the gray value of the j-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set, G i,j-1 is the gray value of the (j - 1)-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set.

[0034] Further, the probability parameter that the set of pixels of interest is the set of pixels in the specular point is:

[0035]

[0036] where P x is the probability parameter that the x-th set of pixels of interest in the region of interest is the set of pixels in the specular point.

[0037] The probability parameter that the region of interest is a specular point is:

[0038]

[0039] Among them, P is the probability parameter that the region of interest is a specular point, and X is the total number of the set of pixels of interest in the region of interest.

[0040] An endoscopic image processing device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for identifying specular points in endoscopic images is implemented.

[0041] A non-transitory computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the method for identifying specular points in endoscopic images is implemented.

[0042] Advantages of the present invention:

[0043] After image segmentation, the present invention further divides the region of interest into multiple sets of pixels of interest, obtains a comparison set of the sets of pixels of interest, and based on the calculation of the specular performance parameter and the specular performance weight of the comparison set, obtains a judgment basis for whether the region of interest is a specular point. Thus, specular points in endoscopic images can be accurately identified, avoiding misidentifying actual targets such as actual white points as specular points, thereby avoiding misprocessing of actual targets during subsequent processing of specular points in endoscopic images and ensuring the quality of the processed endoscopic images. Description of the drawings

[0044] Figure 1 is a flowchart of the method for identifying specular points in endoscopic images according to an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the division of the set of pixels of interest and its comparison set according to an embodiment of the present invention;

[0046] Figure 3 is a block diagram of the device for identifying specular points in endoscopic images according to an embodiment of the present invention. Detailed implementation manners

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0048] As Figure 1 shown, the method for identifying specular points in endoscopic images according to an embodiment of the present invention includes the following steps:

[0049] S1, obtaining an original endoscopic image.

[0050] In the embodiments of the present invention, the original endoscopic image is the image captured by the endoscope. This image can be a single photo or a frame image in a video, and this image is in RGB mode. The endoscope in the embodiments of the present invention can be an otorhinolaryngological endoscope, a gastroscope, a colonoscope, etc. for examination, or an intraoperative endoscope used during surgery.

[0051] S2. Perform grayscale processing on the original endoscopic image to obtain an endoscopic grayscale image.

[0052] S3. Perform image segmentation on the endoscopic grayscale image to segment out the highlighted area.

[0053] In an embodiment of the present invention, the endoscopic grayscale image can be segmented by using a preset grayscale value as a threshold, and the pixel area greater than the preset grayscale value is segmented out as the highlighted area. In a specific embodiment of the present invention, the preset grayscale value can be 220.

[0054] S4. Perform color space conversion processing on the original endoscopic image to obtain an endoscopic conversion image.

[0055] In an embodiment of the present invention, the original endoscopic image in RGB mode can be converted to the HSB color space to obtain an endoscopic conversion image in HSB mode.

[0056] S5. Use the corresponding area of the highlighted area in the endoscopic grayscale image in the endoscopic conversion image as the region of interest, and divide the region of interest into multiple sets of pixels of interest according to the gradient information of the region of interest, and obtain multiple comparison sets corresponding to each set of pixels of interest.

[0057] Specifically, the sum of the gradients of all pixels in the region of interest can be calculated. As Figure 2 shown, the direction of the sum of the gradients can be set as the horizontal direction. After setting the horizontal direction, it is considered that multiple pixels in the horizontal direction are arranged in a row. In the embodiments of the present invention, each row of pixels in the region of interest is divided into a set of pixels of interest, and the M sets of pixel sets adjacent to the left and the N sets of pixel sets adjacent to the right of each set of pixels of interest are used as the corresponding comparison sets. Among them, the number of pixels in each set of pixel sets is the same as the number of pixels in the corresponding set of pixels of interest, and both M and N are positive integers, and M is equal to or differs from N by 1. As a further more detailed description, each comparison set and the corresponding set of pixels of interest are in the same row; two adjacent comparison sets, or an adjacent comparison set and the set of pixels of interest, do not simultaneously contain a certain same pixel; between two adjacent comparison sets, or between an adjacent comparison set and the set of pixels of interest, they are closely connected without spaced pixels. Figure 2Each small square in it represents a pixel. The dark squares are the pixels within the region of interest, that is, the region within the thick lines is the region of interest. Figure 2 As a simple example, in the illustrated example diagram, there are 4 sets of pixels of interest. These 4 sets of pixels of interest contain 6, 5, 6, and 3 pixels respectively from top to bottom. Figure 2 Taking M = N = 2 as an example, only the set of pixels of interest in the bottom row and its comparison set are marked, and the other rows are similar.

[0058] S6. For each set of pixels of interest, calculate the reflectivity performance parameter and the reflectivity performance weight of each corresponding comparison set.

[0059] In an embodiment of the present invention, the reflectivity performance parameter of each comparison set corresponding to the set of pixels of interest is:

[0060]

[0061] Wherein, L k is the reflectivity performance parameter of the k-th comparison set corresponding to the set of pixels of interest, is the average value of the brightness of all pixels in the k-th comparison set corresponding to the set of pixels of interest, B is the average value of the brightness of all pixels in this set of pixels of interest, is the average value of the hue of all pixels in the k-th comparison set corresponding to the set of pixels of interest, is the average value of the hue of all pixels in this set of pixels of interest, is the average value of the saturation of all pixels in the k-th comparison set corresponding to the set of pixels of interest, is the average value of the saturation of all pixels in this set of pixels of interest.

[0062] It should be explained that for the reflective point, relative to the differences in hue and saturation from the surrounding area, the difference in brightness from the surrounding area is the largest, while for the actual white point, on the contrary, relative to the differences in hue and saturation from the surrounding area, the difference in brightness from the surrounding area is the smallest. Therefore, the reflectivity performance parameter calculated by the above formula can represent the degree to which the comparison pixel pair corresponds to the set of pixels of interest being a set of reflective pixels, that is, the larger the reflectivity performance parameter of the comparison set, the more the comparison set can illustrate that the corresponding set of pixels of interest is the pixel in the reflective point.

[0063] In an embodiment of the present invention, the reflectivity performance weight of each comparison set corresponding to the set of pixels of interest is:

[0064]

[0065] Wherein, ω kis the reflectivity performance weight of the k-th comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, C k is the mutation parameter of the k-th comparison set corresponding to the set of pixels of interest, which is the average value of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest. C is the mutation parameter of the set of pixels of interest, which is the average value of the mutation parameters of all pixels in the set of pixels of interest.

[0066] Among them, the mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th comparison set is:

[0067]

[0068] Among them, c i is the mutation parameter of the i-th pixel in the set of pixels of interest or the comparison set, n is the number of pixels within the preset neighborhood range of the i-th pixel, and n is a preset value. For example, n can be set to 6, indicating taking the 3 pixels on the left and the 3 pixels on the right of the i-th pixel. G i,j is the gray value of the j-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set, G i,j-1 is the gray value of the j - 1-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set.

[0069] It should be noted that the mutation parameter of a pixel can indicate whether the gray change of the pixel relative to the surrounding pixels is obvious, that is, whether it changes slowly or suddenly. Similarly, the mutation parameters of the set of pixels of interest and the comparison set can indicate whether the gray changes of the pixels within a set are obvious. Since the reflection point is caused by lighting, the central illumination is the strongest, and the area near the edge can be considered as a mixture of light and the photographed object. Therefore, the gray level from the center to the edge has a gradual change, and the actual gray level of the white point itself is relatively uniform, and the gray difference from the surrounding objects is relatively large. Therefore, after the image segmentation in step S3, the actual white point can be completely segmented out, while the reflection point may have an outer area remaining outside the highlighted area (region of interest). Therefore, compared with the surrounding area, the mutation parameter of the actual white point has a relatively large difference, while the mutation parameter of the reflection point and the surrounding area has a relatively small difference. Moreover, the mutation parameters of the surrounding area of the actual white point are relatively stable, while the mutation parameters of the surrounding area of the reflection point change greatly. In summary, for the above calculation formula of the reflectivity performance weight, the smaller the difference between the mutation parameter C of the set of pixels of interest and the mutation parameter C k of the corresponding comparison set, the more it indicates that the set of pixels of interest corresponding to the comparison set is the pixel in the reflection point, and the greater the reflectivity performance weight. On the contrary, the reflectivity performance weight is smaller; the variance Vark The larger it is, the more it can indicate that the set of pixels of interest corresponding to the comparison set is the pixels in the specular highlights, and the greater the weight of the specular reflection performance. Conversely, the weight of the specular reflection performance is smaller.

[0070] S7. According to the specular reflection performance parameters and the weights of the specular reflection performance of each comparison set corresponding to the set of pixels of interest, calculate the probability parameter that the set of pixels of interest is the set of pixels in the specular highlights, and calculate the probability parameter that the region of interest is a specular highlight.

[0071] In an embodiment of the present invention, the probability parameter that the set of pixels of interest is the set of pixels in the specular highlights is:

[0072]

[0073] where P x is the probability parameter that the x-th set of pixels of interest in the region of interest is the set of pixels in the specular highlights, and M + N is the total number of comparison sets corresponding to the x-th set of pixels of interest.

[0074] In an embodiment of the present invention, the probability parameter that the region of interest is a specular highlight is the average value of the probability parameters that all sets of pixels of interest in the region of interest are the sets of pixels in the specular highlights. That is, the probability parameter that the region of interest is a specular highlight is:

[0075]

[0076] where P is the probability parameter that the region of interest is a specular highlight, and X is the total number of sets of pixels of interest in the region of interest.

[0077] It should be noted that the probability parameter that the region of interest is a specular highlight in the embodiment of the present invention is only used to represent the magnitude of the probability that the region of interest is a specular highlight. The larger the probability parameter, the more likely the region of interest is a specular highlight. The probability parameter that the region of interest is a specular highlight is not equivalent to the actual probability value.

[0078] S8. Judge whether the region of interest is a specular highlight according to the probability parameter that the region of interest is a specular highlight.

[0079] In an embodiment of the present invention, a comparison value can be preset. When the probability parameter that the region of interest is a specular highlight is greater than the comparison value, it is judged that the region of interest is a specular highlight, and it will be processed when implementing the measures for weakening or filtering the specular highlights subsequently; otherwise, it is judged that the region of interest is not a specular highlight, and it will not be processed when implementing the measures for weakening or filtering the specular highlights subsequently.

[0080] According to the method for identifying the specular points in the endoscopic image according to the embodiments of the present invention, after image segmentation, the region of interest is further divided into a plurality of sets of pixels of interest, and the comparison set of the sets of pixels of interest is obtained. Based on the calculation of the specular performance parameters and the specular performance weights of the comparison set, the judgment basis for whether the region of interest is a specular point is obtained. Thus, the specular points in the endoscopic image can be accurately identified, avoiding misidentifying actual targets such as actual white points as specular points, thereby avoiding misprocessing of actual targets during subsequent processing of specular points in the endoscopic image and ensuring the quality of the processed endoscopic image.

[0081] Corresponding to the method for identifying the specular points in the endoscopic image in the above embodiment, the present invention also proposes an apparatus for identifying the specular points in the endoscopic image.

[0082] As Figure 3 shown, the apparatus for identifying the specular points in the endoscopic image according to the embodiments of the present invention includes: an acquisition module 10, a grayscale processing module 20, a segmentation module 30, a conversion processing module 40, a division module 50, a first calculation module 60, a second calculation module 70, and a judgment module 80. Among them, the acquisition module 10 is used to acquire the original endoscopic image; the grayscale processing module 20 is used to perform grayscale processing on the original endoscopic image to obtain a grayscale endoscopic image; the segmentation module 30 is used to perform image segmentation on the grayscale endoscopic image to segment out the highlighted region; the conversion processing module 40 is used to perform color space conversion processing on the original endoscopic image to obtain a converted endoscopic image; the division module 50 is used to use the corresponding region of the highlighted region in the grayscale endoscopic image in the converted endoscopic image as the region of interest, and divide the region of interest into a plurality of sets of pixels of interest according to the gradient information of the region of interest, and obtain a plurality of comparison sets corresponding to each set of pixels of interest; the first calculation module 60 is used to calculate the specular performance parameters and the specular performance weights of each comparison set corresponding to each set of pixels of interest; the second calculation module 70 is used to calculate the probability parameter of the set of pixels of the set of pixels of interest being a specular point according to the specular performance parameters and the specular performance weights of each comparison set corresponding to the set of pixels of interest, and calculate the probability parameter of the region of interest being a specular point; the judgment module 80 is used to judge whether the region of interest is a specular point according to the probability parameter of the region of interest being a specular point.

[0083] The original endoscopic image in the embodiments of the present invention is an image acquired by an endoscope. This image can be a single photo or a frame image in a video, and this image is in RGB mode. The endoscope in the embodiments of the present invention can be an otorhinolaryngology endoscope, a gastroscope, a colonoscope, etc. for examination, or an intraoperative endoscope used during surgery.

[0084] In an embodiment of the present invention, the segmentation module 30 can perform image segmentation on the endoscopic grayscale image with a preset grayscale value as a threshold, and segment out the pixel regions greater than the preset grayscale value as the highlighted regions. In a specific embodiment of the present invention, the preset grayscale value can be 220.

[0085] In an embodiment of the present invention, the conversion processing module 40 can convert the original endoscopic image in RGB mode to the HSB color space to obtain an endoscopic conversion image in HSB mode.

[0086] In an embodiment of the present invention, the partitioning module 50 can specifically calculate the sum of the gradients of all pixels in the region of interest, as Figure 2 shown, the direction of the sum of the gradients can be set as the horizontal direction. After setting the horizontal direction, it is considered that multiple pixels in the horizontal direction are arranged in a row. In the embodiment of the present invention, the partitioning module 50 divides each row of pixels in the region of interest into an interested pixel set, and takes the M sets of pixel sets adjacent to the left and the N sets of pixel sets adjacent to the right of each interested pixel set as the corresponding comparison sets. Wherein, the number of pixels in each set of pixel sets is the same as the number of pixels in the corresponding interested pixel set, and both M and N are positive integers, and M is equal to or differs from N by 1. As a further more detailed description, each comparison set and the corresponding interested pixel set are in the same row; for two adjacent comparison sets, or for an adjacent comparison set and an interested pixel set, they do not simultaneously contain a certain same pixel; between two adjacent comparison sets, or between an adjacent comparison set and an interested pixel set, they are closely connected without intervening pixels. Figure 2 Each small square in represents a pixel, and the dark squares are the pixels in the region of interest, that is, the region within the thick line is the region of interest. Figure 2 As a simple example, there are 4 interested pixel sets shown in the example figure, and the 4 interested pixel sets contain 6, 5, 6, and 3 pixels from top to bottom in sequence, Figure 2 In, taking M = N = 2 as an example, and only the interested pixel set and its comparison set in the bottom row are marked, and the other rows are similar.

[0087] In an embodiment of the present invention, the reflectivity performance parameter of each comparison set corresponding to the interested pixel set is:

[0088]

[0089] wherein, L k is the reflectivity performance parameter of the kth comparison set corresponding to the interested pixel set, B k is the average value of the lightness of all pixels in the kth comparison set corresponding to the interested pixel set, and B is the average value of the lightness of all pixels in this interested pixel set, is the average hue of all pixels in the k-th comparison set corresponding to the set of pixels of interest, is the average hue of all pixels in the set of pixels of interest, is the average saturation of all pixels in the k-th comparison set corresponding to the set of pixels of interest, is the average saturation of all pixels in the set of pixels of interest.

[0090] It should be noted that for the specular point, compared with the differences in hue and saturation from the surrounding area, the difference in lightness from the surrounding area is the largest, while for the actual white point, on the contrary, compared with the differences in hue and saturation from the surrounding area, the difference in lightness from the surrounding area is the smallest. Therefore, the specular reflectivity performance parameter calculated by the above formula can represent the degree of performance of the set of pixels of interest corresponding to the comparison pixels as a specular pixel set, that is, the larger the specular reflectivity performance parameter of the comparison set, the more it can explain that the corresponding set of pixels of interest is the pixels in the specular point.

[0091] In an embodiment of the present invention, the specular reflectivity performance weight of each comparison set corresponding to the set of pixels of interest is:

[0092]

[0093] where ω k is the specular reflectivity performance weight of the k-th comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, C k is the mutation parameter of the k-th comparison set corresponding to the set of pixels of interest, which is the average of the mutation parameters of all pixels in the k-th comparison set corresponding to the set of pixels of interest, and C is the mutation parameter of this set of pixels of interest, which is the average of the mutation parameters of all pixels in this set of pixels of interest.

[0094] where the mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th comparison set is:

[0095]

[0096] where c i is the mutation parameter of the i-th pixel in the set of pixels of interest or the comparison set, n is the number of pixels within the preset neighborhood range of the i-th pixel, n is a preset value, for example, n can be set to 6, indicating taking 3 pixels on the left and 3 pixels on the right of the i-th pixel. G i,j is the gray value of the j-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set, G i,j-1It is the grayscale value of the (j - 1)-th pixel within the preset neighborhood range of the i-th pixel in the set of pixels of interest or the comparison set.

[0097] It should be noted that the mutation parameter of a pixel can indicate whether the grayscale change of the pixel relative to the surrounding pixels is obvious, that is, whether it changes slowly or suddenly. Similarly, the mutation parameters of the set of pixels of interest and the comparison set can indicate whether the grayscale changes of the pixels within a set are obvious. Since the reflective point is caused by lighting, the central part has the strongest illumination, and the area near the edge can be considered as a mixture of light and the photographed object. Therefore, the grayscale has a gradual change from the center to the edge, and the actual white point itself has a relatively uniform grayscale, with a significantly larger grayscale difference from the surrounding objects. Therefore, after the image segmentation by the segmentation module 30, the actual white point can be completely segmented out, while there may be an outer area of the reflective point remaining outside the highlighted area (region of interest). Therefore, compared with the surrounding area, the mutation parameter of the actual white point has a relatively large difference, while the mutation parameter of the reflective point and the surrounding area has a relatively small difference. Moreover, the mutation parameters of the surrounding area of the actual white point are relatively stable, while the mutation parameters of the surrounding area of the reflective point change greatly. In summary, for the above calculation formula of the reflectivity performance weight, the smaller the difference between the mutation parameter C of the set of pixels of interest and the mutation parameter C of the corresponding comparison set, the more it indicates that the set of pixels of interest corresponding to the comparison set is the pixels in the reflective point, and the greater the reflectivity performance weight. On the contrary, the reflectivity performance weight is smaller; the larger the variance Var k the more it indicates that the set of pixels of interest corresponding to the comparison set is the pixels in the reflective point, and the greater the reflectivity performance weight. On the contrary, the reflectivity performance weight is smaller. k k

[0098] In an embodiment of the present invention, the probability parameter that the set of pixels of interest is the set of pixels in the reflective point is:

[0099]

[0100] where P x is the probability parameter that the x-th set of pixels of interest in the region of interest is the set of pixels in the reflective point, and M + N is the total number of the comparison sets corresponding to the x-th set of pixels of interest.

[0101] In an embodiment of the present invention, the probability parameter that the region of interest is a reflective point is the average value of the probability parameters that all sets of pixels of interest in the region of interest are the sets of pixels in the reflective point. That is, the probability parameter that the region of interest is a reflective point is:

[0102]

[0103] Wherein, P is the probability parameter that the region of interest is a specular point, and X is the total number of pixels of interest in the region of interest.

[0104] It should be noted that the probability parameter that the region of interest is a specular point in the embodiments of the present invention is only used to represent the magnitude of the probability that the region of interest is a specular point. The larger the probability parameter, the more likely the region of interest is a specular point. The probability parameter that the region of interest is a specular point is not equivalent to the actual probability value.

[0105] In an embodiment of the present invention, a comparison value can be preset. When the probability parameter that the region of interest is a specular point is greater than the comparison value, the determination module 80 determines that the region of interest is a specular point, and subsequent measures for weakening or filtering the specular point are taken for processing; otherwise, it is determined that the region of interest is not a specular point, and subsequent measures for weakening or filtering the specular point are not taken for processing.

[0106] According to the endoscopic image specular point recognition device of the embodiments of the present invention, after image segmentation, the region of interest is further divided into a plurality of pixel sets of interest, and a comparison set of the pixel sets of interest is obtained. Based on the calculation of the specular performance parameter and the specular performance weight of the comparison set, a judgment basis for whether the region of interest is a specular point is obtained. Thus, the specular points in the endoscopic image can be accurately recognized, and the misrecognition of actual targets such as actual white points as specular points can be avoided, thereby avoiding the misprocessing of actual targets during subsequent endoscopic image specular point processing and ensuring the quality of the processed endoscopic image.

[0107] Corresponding to the above embodiments, the present invention also proposes an endoscopic image processing device.

[0108] The endoscopic image processing device of the embodiments of the present invention includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the endoscopic image specular point recognition method according to the above embodiments of the present invention can be implemented.

[0109] According to the computer device of the embodiments of the present invention, when the processor executes the computer program stored on the memory, after image segmentation, the region of interest is further divided into a plurality of pixel sets of interest, and a comparison set of the pixel sets of interest is obtained. Based on the calculation of the specular performance parameter and the specular performance weight of the comparison set, a judgment basis for whether the region of interest is a specular point is obtained. Thus, the specular points in the endoscopic image can be accurately recognized, and the misrecognition of actual targets such as actual white points as specular points can be avoided, thereby avoiding the misprocessing of actual targets during subsequent endoscopic image specular point processing and ensuring the quality of the processed endoscopic image.

[0110] Corresponding to the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium.

[0111] A non - transitory computer - readable storage medium according to an embodiment of the present invention, on which a computer program is stored, and when the computer program is executed by a processor, it can implement the method for identifying the reflection points of the endoscopic image according to the above - mentioned embodiments of the present invention.

[0112] According to the non - transitory computer - readable storage medium of the embodiment of the present invention, when a processor executes a computer program stored thereon, after image segmentation, the region of interest is further divided into a plurality of sets of pixels of interest, and a comparison set of the sets of pixels of interest is obtained. Based on the calculation of the reflectivity performance parameters and the reflectivity performance weights of the comparison set, a judgment basis for whether the region of interest is a reflection point is obtained. Thus, the reflection points in the endoscopic image can be accurately identified, avoiding misidentifying actual targets such as actual white points as reflection points, thereby avoiding misprocessing of actual targets during subsequent processing of the reflection points in the endoscopic image and ensuring the quality of the processed endoscopic image.

[0113] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The meaning of "a plurality" is two or more unless otherwise specifically defined.

[0114] In the present invention, unless otherwise clearly specified and limited, the terms such as "mounted", "connected", "connected to", "fixed" and the like should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal connection of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above - mentioned terms in the present invention can be understood according to specific situations.

[0115] In the present invention, unless otherwise clearly specified and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "under" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0116] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0117] Any process or method description represented in a flowchart or described in other ways herein can be understood to represent a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present invention includes additional implementations, where the functions can be executed in a manner that is not in the order shown or discussed, including in a substantially simultaneous manner according to the functions involved or in a reverse order, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0118] The logic and / or steps represented in a flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0119] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0120] Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0121] In addition, in each embodiment of the present invention, each functional unit can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0122] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for identifying reflective points in an endoscope image, characterized in that: The following steps are involved: Acquire the original endoscope image; grayscale the original endoscope image to obtain an endoscope grayscale image; Performing image segmentation on the endoscope grayscale image to segment out a highlighted area; Performing color space conversion processing on the endoscope original image to obtain an endoscope conversion image; Taking the corresponding area of ​​the highlighted area in the endoscope grayscale image in the endoscope conversion image as the region of interest, dividing the region of interest into a plurality of sets of pixels of interest according to gradient information of the region of interest, and obtaining a plurality of comparison sets corresponding to each set of pixels of interest; For each of the pixel-of-interest sets, calculating the reflectivity performance parameter and the reflectivity performance weight of each comparison set corresponding to the pixel-of-interest set; Calculate the probability parameter of the pixel set of interest being a pixel set in a reflective point according to the reflective performance parameter and reflective performance weight of each comparison set corresponding to the pixel set of interest, and calculate the probability parameter of the region of interest being a reflective point; Whether the region of interest is a reflective point is determined according to a probability parameter of the region of interest being a reflective point.

2. The method for identifying reflective points in an endoscope image according to claim 1, characterized in that: Dividing the region of interest into a plurality of sets of pixels of interest according to the gradient information of the region of interest, and acquiring a plurality of comparison sets corresponding to each set of pixels of interest, specifically includes: Calculating the sum of the gradients of all pixels in the region of interest; Setting the direction of the sum of the gradients to be horizontal; Classify each row of pixels in the region of interest as a set of pixels of interest; The M pixel sets adjacent to the left and the N pixel sets adjacent to the right of each pixel set of interest are taken as corresponding comparison sets, wherein the number of pixels in each pixel set is the same as the number of pixels in the corresponding pixel set of interest, M and N are both positive integers, and M and N are equal to or differ by 1.

3. The method for identifying reflective points in an endoscope image according to claim 2, characterized in that: The reflectivity performance parameters of each comparison set corresponding to the set of pixels of interest are: Among them, L k is the reflectivity performance parameter of the kth comparison set corresponding to the pixel set of interest, is the average brightness of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average brightness of all pixels in the set of pixels of interest, is the average value of the hue of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average value of the hue of all pixels in the set of pixels of interest, is the average saturation of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average saturation of all pixels in the set of pixels of interest.

4. The method for identifying reflective points in an endoscope image according to claim 3, characterized in that: The reflectivity performance weight of each comparison set corresponding to the set of pixels of interest is: Among them, ω k is the reflectivity weight of the kth comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the kth alignment set corresponding to the set of pixels of interest, C k is the mutation parameter of the kth comparison set corresponding to the pixel set of interest, is the average value of the mutation parameters of all pixels in the kth comparison set corresponding to the pixel set of interest, and C is the mutation parameter of the pixel set of interest, is the average value of the mutation parameters of all pixels in the pixel set of interest.

5. The method for identifying reflective points in an endoscope image according to claim 4, characterized in that: The mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th alignment set is: Among them, c i is the mutation parameter of the i-th pixel in the pixel set of interest or the comparison set, n is the number of pixels in the preset neighborhood of the i-th pixel, G i,j is the gray value of the jth pixel in the preset neighborhood of the i-th pixel in the pixel set of interest or the comparison set, G i,j-1 It is the gray value of the j-1th pixel in the preset neighborhood of the ith pixel in the pixel set of interest or the comparison set.

6. The method for identifying reflective points in an endoscope image according to claim 5, characterized in that: The probability parameter that the set of pixels of interest is the set of pixels in the reflective point is: Among them, P x is the probability parameter that the xth set of pixels of interest in the region of interest is a set of pixels in the reflective point, The probability parameter of the region of interest being a reflective point is: Wherein, P is a probability parameter of the region of interest being a reflective point, and X is the total number of the set of pixels of interest in the region of interest.

7. A device for identifying reflective points in endoscope images, characterized in that: include: An acquisition module, the acquisition module is used to acquire an original endoscope image; A grayscale processing module, the grayscale processing module is used to perform grayscale processing on the endoscope original image to obtain an endoscope grayscale image; A segmentation module, the segmentation module is used to perform image segmentation on the endoscope grayscale image to segment out a highlighted area; A conversion processing module, the conversion processing module is used to perform color space conversion processing on the endoscope original image to obtain an endoscope conversion image; A division module, the division module is used to take the corresponding area of ​​the highlighted area in the endoscope grayscale image in the endoscope conversion image as the region of interest, and divide the region of interest into a plurality of sets of pixels of interest according to the gradient information of the region of interest, and obtain a plurality of comparison sets corresponding to each set of pixels of interest; A first calculation module, the first calculation module is used to calculate, for each set of pixels of interest, a reflectivity performance parameter and a reflectivity performance weight of each comparison set corresponding to the set of pixels of interest; A second calculation module, the second calculation module is used to calculate the probability parameter that the set of pixels of interest is a set of pixels in a reflective point according to the reflective performance parameter and the reflective performance weight of each comparison set corresponding to the set of pixels of interest, and calculate the probability parameter that the region of interest is a reflective point; A judgment module is used to judge whether the region of interest is a reflective point according to a probability parameter that the region of interest is a reflective point.

8. The device for identifying light reflecting points in endoscope images according to claim 7, characterized in that: The division module is specifically used for: Calculating the sum of the gradients of all pixels in the region of interest; Setting the direction of the sum of the gradients to be horizontal; Classify each row of pixels in the region of interest as a set of pixels of interest; The M pixel sets adjacent to the left and the N pixel sets adjacent to the right of each pixel set of interest are taken as corresponding comparison sets, wherein the number of pixels in each pixel set is the same as the number of pixels in the corresponding pixel set of interest, M and N are both positive integers, and M and N are equal to or differ by 1.

9. The device for identifying light reflecting points in an endoscope image according to claim 8, characterized in that: The reflectivity performance parameters of each comparison set corresponding to the set of pixels of interest are: Among them, L k is the reflectivity performance parameter of the kth comparison set corresponding to the pixel set of interest, is the average brightness of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average brightness of all pixels in the set of pixels of interest, is the average value of the hue of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average value of the hue of all pixels in the set of pixels of interest, is the average saturation of all pixels in the kth comparison set corresponding to the pixel set of interest, is the average saturation of all pixels in the set of pixels of interest.

10. The device for identifying light reflecting points in an endoscope image according to claim 9, characterized in that: The reflectivity performance weight of each comparison set corresponding to the set of pixels of interest is: Among them, ω k is the reflectivity weight of the kth comparison set corresponding to the set of pixels of interest, Var k is the variance of the mutation parameters of all pixels in the kth alignment set corresponding to the set of pixels of interest, C k is the mutation parameter of the kth comparison set corresponding to the pixel set of interest, is the average value of the mutation parameters of all pixels in the kth comparison set corresponding to the pixel set of interest, and C is the mutation parameter of the pixel set of interest, is the average value of the mutation parameters of all pixels in the pixel set of interest.

11. The device for identifying light reflecting points in endoscope images according to claim 10, characterized in that: The mutation parameter of the i-th pixel in the set of pixels of interest or its corresponding k-th alignment set is: Among them, c i is the mutation parameter of the i-th pixel in the pixel set of interest or the comparison set, n is the number of pixels in the preset neighborhood of the i-th pixel, G i,j is the gray value of the jth pixel in the preset neighborhood of the i-th pixel in the pixel set of interest or the comparison set, G i,j-1 It is the gray value of the j-1th pixel in the preset neighborhood of the ith pixel in the pixel set of interest or the comparison set.

12. The device for identifying light reflecting points in an endoscope image according to claim 11, characterized in that: The probability parameter that the set of pixels of interest is the set of pixels in the reflective point is: Among them, P x is the probability parameter that the xth set of pixels of interest in the region of interest is a set of pixels in the reflective point, The probability parameter of the region of interest being a reflective point is: Wherein, P is a probability parameter of the region of interest being a reflective point, and X is the total number of the set of pixels of interest in the region of interest.

13. An endoscopic image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for identifying reflective points in an endoscopic image according to any one of claims 1-6 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying reflective points in an endoscopic image according to any one of claims 1 to 6 is implemented.

Images