Endoscope image processing method and system for improving image quality

Abstract

The invention discloses an endoscope image processing method and system for improving image quality, and relates to the technical field of image processing.The method comprises the steps that a highlight reflection area of an endoscope image is extracted by setting a detection and extraction mechanism of the highlight reflection area; the method comprises the following steps: dividing an image of which a highlight reflection region is extracted into a base layer and a detail layer, performing global illumination adjustment on the base layer, performing detail enhancement on the detail layer, and fusing the enhanced base layer and detail layer to obtain a pre-enhanced image; if the pre-enhanced image extracts an over-high-brightness area, repairing a high-brightness reflection area and outputting the high-brightness reflection area; according to the method, global enhancement of the endoscope image is realized through highlight reflection area detection, image hierarchical processing and highlight reflection area repair, and the visibility and availability of the endoscope image are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an endoscope image processing method and system for improving image quality. Background Art

[0002] Endoscope imaging is a medical diagnosis and treatment procedure, and endoscopes have extensive applications in the examination, diagnosis, and treatment of the esophagus digestive system such as the stomach and intestines. The images collected by endoscopes are usually collected in a narrow lumen. The endoscope images are irradiated only by a unidirectional point light source, which will cause problems such as uneven illumination, low illuminance, and loss of image texture and color information.

[0003] With the development of technology, machine learning and artificial intelligence methods are applied to endoscope image enhancement, which can automatically learn the features and laws of endoscope images and perform image enhancement according to the learned knowledge. However, intelligent methods usually require a large amount of labeled data for training. If the richness of the labeled data is insufficient, the enhancement effect will be poor. Therefore, it is of great significance to design an endoscope image enhancement method that does not rely on a large amount of data. Summary of the Invention

[0004] The purpose of the present invention is to provide an endoscope image processing method and system for improving image quality to solve the deficiencies in the background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: An endoscope image processing method for improving image quality, comprising:

[0006] Obtain an endoscope image, detect whether there is a high-brightness reflection area on the endoscope image. If so, extract the high-brightness reflection area. If not, do not perform any processing, and define the endoscope image that passes the detection as a normal image.

[0007] Divide the normal image into a base layer and a detail layer, perform segmentation on the base layer and adjust the illuminance to obtain an enhanced base layer, and perform detail enhancement on the detail layer to obtain an enhanced detail layer.

[0008] Fuse the enhanced base layer and the enhanced detail layer. If the normal image has not been subjected to high-brightness reflection area extraction, output it as an enhanced image. If the normal image has undergone high-brightness reflection area extraction, output it as a pre-enhanced image and repair the high-brightness reflection area.

[0009] In a new embodiment, the steps of obtaining an endoscope image, detecting whether there is a high-brightness reflection area on the endoscope image, if so, extracting the high-brightness reflection area, and if not, not performing any processing are:

[0010] Obtain an endoscope image, set a light source to irradiate the front of the endoscope image and measure the reflectivity of each pixel point, and set a preset reflectivity threshold.

[0011] Mark the pixel points exceeding the preset reflectivity threshold, and at the same time refine the vascular tissue line, record the pixel points where the vascular line breaks, and perform coordinate matching on the broken pixel points and the pixel points exceeding the preset reflectivity threshold. If there are pixel points with the same coordinates, the same pixel points are used as the highlighted reflection pixel points;

[0012] Define the endoscopic image containing the highlighted reflection pixel points as the highlighted image, and define the endoscopic image without the highlighted reflection area as the normal image;

[0013] Extract the highlighted reflection area in the highlighted image through the edge extraction method to convert the highlighted image into a normal image.

[0014] In a new embodiment, the step of extracting the highlighted reflection area in the highlighted image through the edge extraction method to convert the highlighted image into a normal image is as follows:

[0015] Take the center point of the endoscopic image as the coordinate origin to construct a rectangular coordinate system;

[0016] Define the highlighted reflection pixel points as the target pixel points, and obtain the brightness values of the eight adjacent pixel points of the target pixel points;

[0017] Set a predetermined brightness value. If there are pixel points less than the predetermined brightness value among the eight adjacent pixel points, the target pixel points are defined as the highlighted edge pixel points;

[0018] Connect the adjacent highlighted edge pixel points in sequence until no new highlighted edge pixel points can be found to connect;

[0019] Take the area formed by connecting the highlighted edge pixel points as the highlighted reflection area, segment the highlighted reflection area in the endoscopic image, and convert the highlighted picture into a normal image.

[0020] In a new embodiment, the step of dividing the normal image into a base layer and a detail layer, segmenting the base layer and performing illuminance adjustment to obtain an enhanced base layer, and performing detail enhancement on the detail layer to obtain an enhanced detail layer is as follows:

[0021] Divide the normal image into a base layer image and a detail layer image, where the base layer includes the organ contour, the overall shape and color distribution of the tissue structure, and the detail layer includes the vascular texture and the morphology of the lesion site;

[0022] Define the three-color domain threshold, which includes the bright color domain threshold, the dark color domain threshold, and the extremely dark color domain threshold;

[0023] Copy the base layer image into three copies and number them as base layer p v 、base layer p i and base layer p l;

[0024] Use the three - color domain threshold to segment the base layer p v , the base layer p i and the base layer p l to obtain the three - color domain fragments of the base layer p v , including bright color domain fragments B v , low color domain fragments D v and extremely low color domain fragments U v , and record the connection relationships of each fragment;

[0025] The three - color domain fragments of the base layer p i include bright color domain fragments B i , low color domain fragments D i and extremely low color domain fragments U i , and record the connection relationships of each fragment;

[0026] The three - color domain fragments of the base layer p l include bright color domain fragments B l , low color domain fragments D l and extremely low color domain fragments U l , and record the connection relationships of each fragment;

[0027] Convert the RGB channels of the three - color domain fragments of the base layer p v to HSV channels, and construct a gamma correction function where V out (x, y) is the enhanced V - channel value, V in (x, y) is the original V - channel value, γ v is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the V - channel values of B v , D v and U v to output a predetermined number of enhanced bright color domain fragments E BV , enhanced low color domain fragments E DV and enhanced extremely low color domain fragments E UV as the enhanced three - color domain fragments E V ;

[0028] Convert the RGB channels of the three - color domain fragments of the base layer p i to HSI channels, and construct a gamma correction function where I out (x, y) is the enhanced I - channel value, I in (x, y) is the original I - channel value, γ i is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the I - channel values of B i , D iand U i Perform V-channel value adjustment and output a predetermined number of enhanced bright color gamut fragments E BI 、enhanced low color gamut fragments E DI and enhanced extremely low color gamut fragments E UI as enhanced three-color gamut fragments E I ;

[0029] Convert the RGB channels of the three-color gamut fragments of the base layer p l to HSL channels and construct a gamma correction function where L out (x, y) is the enhanced L-channel value, L in (x, y) is the original L-channel value, γ l is an adjustable gamma parameter, x and y are the coordinate values of pixel points respectively. Based on the gamma correction function, perform L-channel value adjustment on B l 、D l and U l Perform L-channel value adjustment and output a predetermined number of enhanced bright color gamut fragments E BL 、enhanced low color gamut fragments E DL and enhanced extremely low color gamut fragments E UL as enhanced three-color gamut fragments E L ;

[0030] Randomly combine a predetermined number of E V 、E I and E L into a preset number of pre-enhanced base layers based on the connection relationship;

[0031] Convert the preset number of pre-enhanced base layers into grayscale images, obtain the grayscale histograms of the grayscale images, calculate the uniformity indexes of the grayscale histograms, and select the pre-enhanced base layer with the largest uniformity index as the enhanced base layer and output it;

[0032] Obtain the HSV space of the enhanced base layer, and perform detail enhancement on the detail layer based on the HSV space to obtain an enhanced detail layer.

[0033] In a new embodiment, the steps of obtaining the HSV space of the enhanced base layer and performing detail enhancement on the detail layer based on the HSV space to obtain an enhanced detail layer are as follows:

[0034] Obtain the HSV space of the enhanced base layer;

[0035] Use the attention mechanism network to extract features from the edges of the blood vessels and lesion parts of the detail layer to obtain edge features, and use a sharpening filter to sharpen the edge features to obtain enhanced edges;

[0036] Convert the pixel points in the detail layer except for the enhanced edge parts into the HSV space as the HSV space of the detail layer. Preset the weight W1, and fuse the HSV space of the enhanced base layer and the HSV space of the detail layer based on the preset weight to obtain the fused color space;

[0037] Reconstruct the detail layer based on the fused color space to obtain the enhanced detail layer.

[0038] In a new embodiment, the step of fusing the enhanced base layer and the enhanced detail layer is as follows:

[0039] Preset the weights W2 and W3;

[0040] The fusion of the enhanced base layer and the enhanced detail layer based on the preset weights W2 and W2 is expressed as:

[0041] Fusion image color space = W2 * Enhanced base layer color space + W3 * Enhanced detail layer color space

[0042] If the ordinary image has not been segmented for the highlighted reflection area, output the fused image as the enhanced image;

[0043] If the ordinary image has been segmented for the highlighted reflection area, output the image as the pre-enhanced image, repair the highlighted reflection area in the pre-enhanced image, and output the repaired pre-enhanced image as the enhanced image.

[0044] In a new embodiment, the step of repairing the highlighted reflection area in the pre-enhanced image and outputting the repaired pre-enhanced image as the enhanced image is as follows:

[0045] Fuse the extracted highlighted reflection area with the pre-enhanced image;

[0046] Calculate the HSV mean value of the non-highlighted reflection pixel points outside the highlighted reflection area as I avg ;

[0047] Utilize I avg Iteratively adjust the HSV values of the highlighted reflection area, which is expressed as:

[0048] I(x, y) new = I(x,y) old + λ(I avg - I(x, y) old )

[0049] Among them, I(x, y) new represents the HSV value of the highlighted reflection pixel points after iteration, I(x, y) old represents the HSV value of the original highlighted reflection pixel points, and λ is an adjustable parameter;

[0050] Monitor the change of the HSV value of the highlighted reflection area and calculate the standard deviation of the HSV value. Preset the standard deviation value. When the standard deviation of the HSV value of the highlighted reflection area after iteration is less than the preset standard deviation value, terminate the iteration and output the enhanced image.

[0051] The present invention also provides an endoscope image processing system for improving image quality, including:

[0052] Highlight area segmentation module: Obtain the endoscope image, detect whether there is a highlighted reflection area on the endoscope image. If so, extract the highlighted reflection area; if not, do not process it, and define the endoscope image that passes the detection as a normal image.

[0053] Image enhancement module: Connected to the highlight area segmentation module, divide the normal image into a base layer and a detail layer, segment the base layer and perform illuminance adjustment to obtain an enhanced base layer, and perform detail enhancement on the detail layer to obtain an enhanced detail layer.

[0054] Highlight area repair module: Connected to the image enhancement module, fuse the enhanced base layer and the enhanced detail layer. If the normal image has not undergone highlighted reflection area extraction, output it as an enhanced image; if the normal image has undergone highlighted reflection area extraction, output it as a pre-enhanced image and repair the highlighted reflection area.

[0055] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0056] 1. The present invention divides the highlighted reflection area in the endoscope image, divides the segmented image into a base layer and a detail layer, performs gamut segmentation processing on the base layer and enhances different gamut fragments, randomly fuses the enhanced fragments and selects the optimal enhanced base layer to ensure uniform illumination of the base layer image. Based on the color space fusion of the enhanced base layer and the color space of the detail layer, the details of the endoscope image are enhanced on the premise of uniform endoscope image illumination. This method does not rely on a large amount of data training, and only uses the color data and feature data of the endoscope image itself to achieve image enhancement.

[0057] 2. The present invention fuses the enhanced base layer and the enhanced detail layer to obtain a pre-enhanced image, places the extracted highlighted reflection area back into the pre-enhanced image, and iteratively repairs the highlighted reflection area based on the enhanced color space of the endoscope image. By continuously iterating and gradually repairing the color space of the highlighted reflection area, smooth repair of the highlighted reflection area is ensured, and a standard deviation value is set to ensure that there will be no color distortion and detail loss problems in the repaired highlighted reflection area. Description of the Drawings

[0058] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other accompanying drawings can also be obtained based on these drawings.

[0059] Figure 1 It is the flowchart of the method of the present invention;

[0060] Figure 2 It is the system block diagram of the present invention. Specific embodiments

[0061] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, rather than all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0062] Embodiment 1. Please refer to Figure 1 As shown, a method for processing endoscopic images to improve image quality in this embodiment includes:

[0063] S1. Obtain an endoscopic image, detect whether there is a high-brightness reflection area on the endoscopic image. If so, extract the high-brightness reflection area; if not, do not process it, and define the endoscopic image that passes the detection as a normal image;

[0064] S2. Divide the normal image into a base layer and a detail layer, segment the base layer and perform illuminance adjustment to obtain an enhanced base layer, and perform detail enhancement on the detail layer to obtain an enhanced detail layer;

[0065] S3. Fuse the enhanced base layer and the enhanced detail layer. If the normal image has not undergone high-brightness reflection area extraction, output it as an enhanced image; if the normal image has undergone high-brightness reflection area extraction, output it as a pre-enhanced image and repair the high-brightness reflection area;

[0066] As described in the above steps S1 - S3, endoscopic imaging is a medical diagnosis and treatment procedure. Endoscopes have extensive applications in the examination, diagnosis, and treatment of the esophageal digestive system such as the stomach and intestines. The images collected by endoscopes are usually obtained within a narrow lumen. When the endoscopic images are illuminated only by a unidirectional point light source, problems such as uneven illumination, low illuminance, and loss of image texture and color information will occur. With the development of technology, machine learning and artificial intelligence methods are applied to endoscopic image enhancement, which can automatically learn the features and patterns of images and perform image enhancement based on the learned knowledge. However, intelligent methods usually require a large amount of labeled data for training. If the richness of the labeled data is insufficient, the enhancement effect will be poor. In the present invention, the highlighted reflection regions in the endoscopic images are segmented, and the segmented images are divided into a base layer and a detail layer. The base layer is processed by color gamut segmentation and different color gamut fragments are enhanced. The enhanced fragments are randomly fused and the optimal enhanced base layer is selected to ensure uniform illumination of the base layer image. Based on the fusion of the color space of the enhanced base layer and the color space of the detail layer, the details of the endoscopic image are enhanced on the premise of uniform endoscopic image illumination. This method does not rely on a large amount of data training and only uses the color data and feature data of the endoscopic image itself to achieve image enhancement. At the same time, a pre - enhanced image is obtained by fusing the enhanced base layer and the enhanced detail layer. The extracted highlighted reflection region is placed back into the pre - enhanced image, and the color space of the highlighted reflection region is iteratively repaired based on the enhanced color space of the endoscopic image. By continuously iteratively and progressively repairing the color space of the highlighted reflection region, smooth repair of the highlighted reflection region is ensured and a standard deviation value is set to ensure that there will be no color distortion and detail loss in the repaired highlighted reflection region.

[0067] In one embodiment, step S1 of obtaining an endoscopic image, detecting whether there is a highlighted reflection region on the endoscopic image, and extracting the highlighted reflection region if there is, and not performing any processing if there is not, includes:

[0068] S11. Obtain an endoscopic image, set the light source to irradiate the front of the endoscopic image and measure the reflectivity of each pixel point, and set a preset reflectivity threshold;

[0069] S12. Mark the pixel points that exceed the preset reflectivity threshold, and at the same time refine the blood vessel tissue lines, record the pixel points where the blood vessel lines are broken, and perform coordinate matching between the broken pixel points and the pixel points that exceed the preset reflectivity threshold. If there are pixel points with the same coordinates, the same pixel points are used as the highlighted reflection pixel points;

[0070] S13. Define the endoscopic image containing the highlighted reflection pixel points as a highlighted image, and define the endoscopic image without a highlighted reflection region as a normal image;

[0071] S14. Extract the highlighted reflection area in the highlighted image through an edge extraction method to convert the highlighted image into a normal image;

[0072] As described in the above steps S11 - S14, obtain an endoscopic image, give the endoscopic image a virtual light source, measure the reflectivity of each pixel point on the endoscopic image, and at the same time set a preset reflectivity threshold. When the reflectivity of a pixel point exceeds the preset reflection threshold, it will be marked as a quasi-highlight reflection pixel point. At the same time, let the endoscopic image pass through a filter to filter out the simple lines of blood vessels, record the break points of the blood vessel lines, and match the pixel points at the break points of the blood vessel lines with the quasi-highlight reflection pixel points. If the coordinates of the two pixel points are the same, the pixel point at this coordinate is used as the highlight reflection pixel point. Whether the endoscopic image contains a highlighted reflection area is detected through this detection method. Through detection, the endoscopic image is divided into a normal image without a highlighted reflection area and a highlighted image including a highlighted reflection area. The highlighted image is segmented for the highlighted reflection area to make it a normal image for the next step of processing. During the judgment process of the highlighted pixel points, when there are break points in the blood vessel patterns, it indicates that the blood vessels are very likely to pass through the highlighted reflection area, making its details unclear and unable to be filtered out by the filter. The matching of quasi-highlight emission pixel points can further confirm that the break points are highlighted pixel points. At the same time, if the break points of the blood vessel lines are highlighted pixel points, they are basically at the boundary of the highlighted reflection area. Therefore, when all the break points of the blood vessel lines are identified as highlighted pixel points, the connection of the adjacent pixel points of the highlighted edge pixel points will not cause misjudgment and loss of the highlighted reflection area;

[0073] In one embodiment, the step of extracting the highlighted reflection area in the highlighted image through an edge extraction method to convert the highlighted image into a normal image is S14, including:

[0074] S141. Take the center point of the endoscopic image as the coordinate origin to construct a plane rectangular coordinate system;

[0075] S142. Define the highlighted reflection pixel point as the target pixel point, and obtain the brightness values of the eight adjacent pixel points of the target pixel point;

[0076] S143. Set a predetermined brightness value. If there are pixel points among the eight adjacent pixel points that are less than the predetermined brightness value, then define the target pixel point as a highlighted edge pixel point;

[0077] S144. Connect the adjacent highlighted edge pixel points in sequence until no new highlighted edge pixel points can be found to connect;

[0078] S145. Take the area formed by connecting the highlighted edge pixel points as the highlighted reflection area, segment the highlighted reflection area in the endoscopic image, and convert the highlighted picture into a normal image;

[0079] As described in the above steps S141 - S145, a plane rectangular coordinate system is constructed with the center point of the endoscopic image as the origin coordinates to locate the positions of all pixel points. The eight pixel points around all the highlighted reflection pixel points are obtained, and the brightness information of the eight pixel points is collected. A predetermined brightness value is used to distinguish adjacent pixel points. When a certain pixel point is on the edge of the highlighted reflection area, then among the eight pixel points around it, there must be pixel points in the non - highlighted reflection area, and the pixel points in the non - highlighted reflection area must be much smaller in brightness than the highlighted reflection pixel points. Therefore, when it is detected that there are non - highlighted reflection pixel points among the pixel points around the highlighted reflection pixel points, then the highlighted reflection pixel point is determined as a highlighted edge pixel point. Connect all the highlighted reflection pixel points in sequence. The steps are as follows: randomly select a highlighted edge pixel point, and based on the selected highlighted edge pixel point, connect the highlighted edge pixel points in sequence, explore along the continuous path of adjacent pixel points until no new connected highlighted reflection pixel points can be found. Randomly select a pixel point among the unconnected highlighted edge pixel points, and repeat the above steps until all the highlighted edge pixel points are connected. The area enclosed by connecting the highlighted reflection pixel points is defined as the highlighted reflection area, and at the same time, the highlighted reflection area is segmented so that the highlighted image is converted into a normal image for the next step of processing. There may be several highlighted reflection areas in the endoscopic image and each highlighted reflection area exists independently. The sequential connection of the highlighted edge pixel points can reveal all the highlighted reflection areas.

[0080] In one embodiment, the normal image is divided into a base layer and a detail layer. The steps S2 of segmenting the base layer and adjusting the illuminance to obtain an enhanced base layer, and enhancing the details of the detail layer to obtain an enhanced detail layer include:

[0081] S201. Divide the normal image into a base - layer image and a detail - layer image, where the base layer includes the organ outline, the overall shape and color distribution of the tissue structure, and the detail layer includes blood - vessel textures and the shape of the lesion site;

[0082] S202. Define three - color domain thresholds, including a bright color domain threshold, a dark color domain threshold, and an extremely dark color domain threshold;

[0083] S203. Copy the base - layer image into three copies and number them as base - layer p v 、base - layer p i and base - layer p l ;

[0084] S204. Use the three - color domain thresholds to segment base - layer p v 、base - layer p i and base - layer p l , and respectively obtain the three - color domain fragments of base - layer p v including bright color domain fragment Bv and low color gamut fragment D v and extremely low color gamut fragment U v , and record the connection relationships of each fragment;

[0085] S205. The three-color gamut fragments of the base layer p i include bright color gamut fragment B i and low color gamut fragment D i and extremely low color gamut fragment U i , and record the connection relationships of each fragment;

[0086] S206. The three-color gamut fragments of the base layer p l include bright color gamut fragment B l and low color gamut fragment D l and extremely low color gamut fragment U l , and record the connection relationships of each fragment;

[0087] S207. Convert the RGB channels of the three-color gamut fragments of the base layer p v into HSV channels, and construct a gamma correction function where V out (x, y) is the enhanced V-channel value, V in (x, y) is the original V-channel value, γ v is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the V-channel values of B v , D v and U v , and output a predetermined number of enhanced bright color gamut fragments E BV , enhanced low color gamut fragments E DV and enhanced extremely low color gamut fragments E UV as enhanced three-color gamut fragments E V ;

[0088] S208. Convert the RGB channels of the three-color gamut fragments of the base layer p i into HSI channels, and construct a gamma correction function where I out (x, y) is the enhanced I-channel value, I in (x, y) is the original I-channel value, γ i is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the V-channel values of B i , D i and U i , and output a predetermined number of enhanced bright color gamut fragments E BI , enhanced low color gamut fragments E DI and enhanced extremely low color gamut fragments E UI as enhanced three-color gamut fragments EI ;

[0089] S209. Convert the RGB channels of the three-color domain fragments of the base layer p l to the HSL channels, and construct a gamma correction function where L out (x, y) is the enhanced L-channel value, L in (x, y) is the original L-channel value, γ l is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the L-channel values of B l , D l and U l and output a predetermined number of enhanced bright color domain fragments E BL , enhanced low color domain fragments E DL and enhanced extremely low color domain fragments E UL as enhanced three-color domain fragments E L ;

[0090] S210. Randomly combine a predetermined number of E V , E I and E L into a preset number of pre-enhanced base layers based on the connection relationship;

[0091] S211. Convert the preset number of pre-enhanced base layers into grayscale images, obtain the grayscale histograms of the grayscale images, calculate the uniformity indexes of the grayscale histograms, and select the pre-enhanced base layer with the largest uniformity index as the enhanced base layer and output it;

[0092] S212. Obtain the HSV space of the enhanced base layer, and perform detail enhancement on the detail layer based on the HSV space to obtain an enhanced detail layer;

[0093] As described in the above steps S201 - S212, Gaussian filtering is used to smooth the ordinary image to obtain the base layer. The detail layer is obtained by performing a difference operation on the ordinary image and the base layer. Enhancement processing is performed based on the obtained base layer. First, three - color domain thresholds are defined, including a bright color domain threshold, a dark color domain threshold, and an extremely dark color domain threshold. The base layer contains the overall shape and color distribution of the organ contour and tissue structure in the endoscopic image. Its main enhancement direction is to enhance the image illumination and the visibility of the image. The base layer is copied three times. The three base layers are divided into color domain fragments using the three - color domain thresholds. At the same time, the connection relationship is defined using the edge pixel points of the color domain fragments. When dividing the base layer, the newly generated edge pixel points of each fragment are recorded, where the newly generated edge pixel points refer to the pixel points on the newly generated edge of the fragment during the division of the base layer. During fragment fusion, coordinates connected to them are searched based on the coordinates of the edge pixel points. When all the newly generated edge pixel points find the newly generated edge pixel points connected to them, it means that the fragment fusion is completed. Different processing is performed on the color domain fragments of each base layer. The RGB color space of the color domain fragments of the first base layer is converted into the HSV color space, and the V - channel is adjusted through the designed gamma correction function for the V - channel adjustment, where V out (x, y) is the enhanced V - channel value, V in (x, y) is the original V - channel value, γ v is the adjustable gamma parameter, and x and y are the coordinate values of the pixel points respectively. The gamma parameter will be adjusted for different color domain fragments. For example, for the bright color domain fragments, the value of the gamma parameter is greater than 1, and as the brightness increases, the value of the gamma parameter will also increase accordingly. For the low - color domain fragments, the value of the gamma parameter is less than 1 and the lower the brightness, the smaller the value of the gamma parameter. The enhanced color domain fragments of the first base layer are obtained by processing different color domain fragments of the first base layer using different gamma parameters. The RGB color space of the color domain fragments of the second base layer is converted into the HSI color space, and the I - channel value is adjusted through the designed gamma correction function for the I - channel value adjustment. The enhanced color domain fragments of the second base layer are obtained by enhancing each color domain fragment using the adjustable gamma parameter. The RGB color space of the color domain fragments of the third base layer is converted into the HSL color space, and the gamma correction function Adjust the L-channel value, use the adjustable gamma parameter to enhance each color gamut fragment to obtain the enhanced color gamut fragments of the third base layer. Based on the above operations, a predetermined number of first, second, and third base layer color gamut fragments are generated. The first, second, and third base layer color gamut fragments are put into a random pool and randomly fused according to the connection relationship of the edge pixel points to obtain a predetermined number of pre-enhanced base layers. The process is as follows: Extract a fragment from the random pool as the starting point, determine the newborn edge pixel points of the fragment, find other fragments that can be connected based on the newborn edge pixel points, and ensure that the selected fragments are connected to the fragments that have been selected. Gradually spread outwards around the starting point and repeat the above operations until the pre-enhanced base layer is generated. When a preset number of pre-enhanced base layers are generated, end the fragment fusion operation; Convert all the pre-enhanced base layers into grayscale images, calculate the uniformity index of the grayscale histogram of the grayscale images, and select the pre-enhanced base layer with the optimal uniformity index as the enhanced base layer output. The uniformity of the grayscale histogram can determine the uniformity of the base layer brightness. The pre-enhanced base layer image with the optimal uniformity indicates that the adjustment of the gamma correction function for the V, I, and L channels has reached the optimal state.

[0094] In one embodiment, the step S212 of obtaining the HSV space of the enhanced base layer and performing detail enhancement on the detail layer based on the HSV space to obtain the enhanced detail layer includes:

[0095] S2121. Obtain the HSV space of the enhanced base layer;

[0096] S2122. Use the attention mechanism network to extract features from the edges of the blood vessels and lesion sites in the detail layer to obtain edge features, and use a sharpening filter to sharpen the edge features to obtain enhanced edges;

[0097] S2123. Convert the pixel points in the detail layer except for the enhanced edge parts into the HSV space as the HSV space of the detail layer, preset a weight W1, and fuse the HSV space of the enhanced base layer and the HSV space of the detail layer based on the preset weight to obtain a fused color space;

[0098] S2124. Reconstruct the detail layer based on the fused color space to obtain the enhanced detail layer;

[0099] As described in the above steps S2121 - S2124, the enhancement based on the base layer enables the color space of the base layer to meet the requirements of visualization. Then, the color space of the base layer plays an important role in the enhancement of the detail layer. The enhanced base layer is transformed into the HSV space. At the same time, for the detail layer, an attention mechanism network is used to extract features from the detail layer image, focusing on the morphological and texture features of blood vessels and lesion sites. Based on the features extracted from the attention mechanism network, through an edge detection algorithm such as the Canny operator, the edge features of blood vessels and lesion sites are obtained. The Laplacian filter is applied to the obtained edge feature image to enhance the sharpness and contrast of the edges. The pixels except for the edge pixels of blood vessels and lesion sites are transformed into the HSV space. A preset weight is set. Based on the preset weight, the HSV space of the enhanced detail layer and the HSV space transformed from the detail layer are combined to obtain a fused color space. The detail layer is reconstructed based on the fused color space to obtain an enhanced detail layer. The setting of the preset weight value can consider the specific use of the endoscopic image. For example, for an endoscopic image of cancer lesions, if microscopic features such as the cell morphology, atypical nuclei, and cell arrangement of cancerous tissues are required, a larger weight value is set to highlight the display of the detail layer. If global features such as the structure around the cancerous tissue and changes in the mucosal surface need to be observed, a smaller weight value is set to highlight the display of the base layer.

[0100] In one embodiment, step S3 of fusing the enhanced base layer and the enhanced detail layer includes:

[0101] S31. Preset weights W2 and W3;

[0102] S32. The fusion of the enhanced base layer and the enhanced detail layer based on the preset weights W2 and W2 is expressed as:

[0103] Fusion image color space = W2 * Enhanced base layer color space + W3 * Enhanced detail layer color space

[0104] S33. If the ordinary image has not been segmented for the high - light reflection area, the fused image is output as the enhanced image;

[0105] S34. If the ordinary image has been segmented for the high - light reflection area, the image is output as the pre - enhanced image, and the high - light reflection area in the pre - enhanced image is repaired, and the repaired pre - enhanced image is output as the enhanced image;

[0106] As described in the above steps S31 - S34, the enhanced base layer and the enhanced detail layer are obtained based on the above steps. For the fusion of the enhanced base layer and the enhanced detail layer, preset weights W2 and W3 are set, and the enhanced base layer and the enhanced detail layer are fused according to the preset weights. The weights W2 and W3 can be set through expert experience, and the optimal weights can be obtained through experimental tests. The test direction is based on the content and task requirements of the endoscopic image. Neural networks can also be used for training to obtain the optimal weights. The content to be fused is the color space of the enhanced base layer and the enhanced detail layer. The output of the fused image is based on whether the ordinary image has been processed for the highlight reflection area. When the ordinary image has not been subjected to the extraction of the highlight reflection area, the output is the enhanced image. If the ordinary image has been subjected to the extraction of the highlight reflection area, the output is the pre - enhanced image, and after repairing the highlight reflection area, the output is the enhanced image.

[0107] In one embodiment, the step S34 of repairing the highlight reflection area in the pre - enhanced image and outputting the repaired pre - enhanced image as the enhanced image includes:

[0108] S341. Fuse the extracted highlight reflection area with the pre - enhanced image;

[0109] S342. Calculate the HSV mean value of the non - highlight - reflection pixel points outside the highlight reflection area as I avg ;

[0110] S343. Use I avg to iteratively adjust the HSV values of the highlight reflection area, which is expressed as:

[0111] I(x,y) new = I(x,y) old + λ(I avg - I(x,y) old )

[0112] S344. Among them, I(x,y) new represents the HSV value of the iterated highlight reflection pixel points, I(x,y) old represents the HSV value of the original highlight reflection pixel points, and λ is an adjustable parameter;

[0113] S345. Monitor the change of the HSV value of the highlight reflection area and calculate the standard deviation of the HSV value. Preset a standard deviation value. When the standard deviation of the HSV value of the iterated highlight reflection area is less than the preset standard deviation value, terminate the iteration and output the enhanced image;

[0114] As described in the above steps S341 - 345, obtain the pre - enhanced image, place the extracted highlight reflection area back into the pre - enhanced image, calculate the HSV mean value I avg, perform color impact on the high - light reflection area using the HSV mean and the adjustable parameter λ. The steps are as follows: I(x, y) new = I(x, y) old + λ(I avg - I(x, y) old ), where I(x, y) new represents the HSV value of the high - light reflection pixel points after iteration, and I(x, y) old represents the HSV value of the original high - light reflection pixel points. Here, λ, as an adjustable parameter, represents the initial gain coefficient, which is used to control the adjustment range of the HSV value of the high - light reflection area. It can be set as a floating value between 0.1 and 0.5. At the same time, detect the change of the HSV value of the high - light reflection area during the iteration process, and calculate the standard deviation of HSV. When the standard deviation reaches the preset standard deviation, end the iteration to avoid color distortion or detail loss in the high - light reflection area caused by excessive iteration. Among them, the standard deviation value of the snake is used as the threshold for judging whether the iteration ends. Multiple standard endoscope images can be obtained and the standard deviation of the HSV value of their high - light reflection areas can be calculated to obtain a reference value, and this reference value is used as the preset standard deviation value. After the iteration ends, the pre - enhanced image is output as the enhanced image. Its idea is "water flow scouring". The high - light reflection area is regarded as an "isolated island", and the color space outside the high - light reflection area is regarded as "water flow", making the "water flow" continuously scour the "isolated island" to make the "isolated island" return to its original appearance. Based on this idea, the repair of the high - light reflection area can be completed, and the asymptotic repair of the high - light reflection area can be achieved, making the color of the repaired high - light reflection area smooth and highly visible.

[0115] An endoscope image processing system for improving image quality, comprising:

[0116] High - light area segmentation module: Obtain the endoscope image, detect whether there is a high - light reflection area on the endoscope image. If there is, extract the high - light reflection area; if not, do not process it, and define the endoscope image passing the detection as an ordinary image;

[0117] Image enhancement module: Connected to the high - light area segmentation module, divide the ordinary image into a base layer and a detail layer, perform segmentation on the base layer and adjust the illuminance to obtain an enhanced base layer, and perform detail enhancement on the detail layer to obtain an enhanced detail layer;

[0118] High - light area repair module: Connected to the image enhancement module, fuse the enhanced base layer and the enhanced detail layer. If the high - light reflection area is not extracted from the ordinary image, output it as the enhanced image; if the high - light reflection area is extracted from the ordinary image, output it as the pre - enhanced image and repair the high - light reflection area.

[0119] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

Claims

1. An endoscopic image processing method for improving image quality, characterized in that: Obtain an endoscopic image, detect whether there is a high-brightness reflection area on the endoscopic image. If there is, extract the high-brightness reflection area; if not, do not perform any processing, and define the endoscopic image that passes the detection as a normal image; Divide the normal image into a base layer and a detail layer, segment the base layer and adjust the illuminance to obtain an enhanced base layer, and enhance the details of the detail layer to obtain an enhanced detail layer; Fuse the enhanced base layer and the enhanced detail layer. If the normal image has not undergone high-brightness reflection area extraction, output it as an enhanced image; if the normal image has undergone high-brightness reflection area extraction, output it as a pre-enhanced image and repair the high-brightness reflection area.

2. The endoscopic image processing method for improving image quality according to claim 1, wherein: The steps of obtaining an endoscopic image, detecting whether there is a high-brightness reflection area on the endoscopic image. If there is, extract the high-brightness reflection area; if not, do not perform any processing are as follows: Obtain an endoscopic image, set the light source to irradiate the front of the endoscopic image and measure the reflectivity of each pixel point, and set a preset reflectivity threshold; Mark the pixel points that exceed the preset reflectivity threshold, at the same time refine the blood vessel tissue line, record the pixel points where the blood vessel tissue line breaks, and perform coordinate matching on the pixel points where the break occurs and the pixel points that exceed the preset reflectivity threshold. If there are pixel points with the same coordinates, take the same pixel points as high-brightness reflection pixel points; Define the endoscopic image containing high-brightness reflection pixel points as a high-brightness image, and define the endoscopic image without a high-brightness reflection area as a normal image; Extract the high-brightness reflection area in the high-brightness image through an edge extraction method, and convert the high-brightness image into a normal image.

3. An endoscopic image processing method for improving image quality according to claim 2, characterized in that: The steps of extracting the high-brightness reflection area in the high-brightness image through an edge extraction method and converting the high-brightness image into a normal image are as follows: Take the center point of the endoscopic image as the coordinate origin and construct a plane rectangular coordinate system; Define the high-brightness reflection pixel points as target pixel points, and obtain the brightness values of the eight adjacent pixel points of the target pixel points; Set a predetermined brightness value. If there are pixel points among the eight adjacent pixel points that are less than the predetermined brightness value, define the target pixel points as high-brightness edge pixel points; Connect the adjacent high-brightness edge pixel points in sequence until no new high-brightness edge pixel points can be found to connect; Take the area formed by connecting the high-brightness edge pixel points as the high-brightness reflection area, segment the high-brightness reflection area in the endoscopic image, and convert the high-brightness image into a normal image.

4. An endoscopic image processing method for improving image quality according to claim 1, characterized in that: The steps of dividing the normal image into a base layer and a detail layer, segmenting the base layer and adjusting the illuminance to obtain an enhanced base layer, and enhancing the details of the detail layer to obtain an enhanced detail layer are as follows: Divide the normal image into a base layer image and a detail layer image, where the base layer includes the organ contour, the overall morphology and color distribution of the tissue structure, and the detail layer includes blood vessel textures and the morphology of the lesion site; Define three-color domain thresholds, including a bright color domain threshold, a dark color domain threshold, and an extremely dark color domain threshold; Copy the base layer image into three copies, numbered as base layer p v , base layer p i , and base layer p l ; Using a three-color domain threshold for the base layer p v , the base layer p i and the base layer p l are segmented to respectively obtain the three-color domain fragments of the base layer p v , including bright color domain fragments B v , low color domain fragments D v and extremely low color domain fragments U v , and record the connection relationships of each fragment; Base layer p i The three-color domain fragments of i include bright color domain fragments B i , low color domain fragments D i and extremely low color domain fragments U , and record the connection relationships of each fragment; Base layer p l The three-color domain fragments of l include bright color domain fragment B l , low color domain fragment D l , and extremely low color domain fragment U , and record the connection relationships of each fragment; Convert the RGB channels of the three-color domain fragments of the base layer p v to HSV channels and construct a gamma correction function where V out (x, y) is the enhanced V-channel value, and V in (x, y) is the original V-channel value, γ v is an adjustable gamma parameter, and x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the V-channel values of B v , D v and U v and output a predetermined number of enhanced bright color domain fragments E BV , enhanced low color domain fragments E DV and enhanced extremely low color domain fragments E UV as enhanced three-color domain fragments E V ; Convert the RGB channels of the three-color domain fragments of the base layer p i into HSI channels and construct a gamma correction function where I out (x, y) is the enhanced I-channel value, and I in (x, y) is the original I-channel value, γ i is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the V-channel values of B i , D i and U i to output a predetermined number of enhanced bright color domain fragments E BI , enhanced low color domain fragments E DI and enhanced extremely low color domain fragments E UI as enhanced three-color domain fragments E I ; Convert the RGB channels of the three-color domain fragments of the base layer p l into HSL channels and construct a gamma correction function where L out (x, y) is the enhanced L-channel value, and L in (x, y) is the original L-channel value, γ l is an adjustable gamma parameter, x and y are the coordinate values of the pixel points respectively. Based on the gamma correction function, adjust the L-channel values of B l , D l and U l and output a predetermined number of enhanced bright color domain fragments E BL , enhanced low color domain fragments E DL and enhanced extremely low color domain fragments E UL as enhanced three-color domain fragments E L ; Combine a predetermined number of Es V , Es I and Es L into a preset number of pre-enhanced base layers randomly based on the connection relationship; Convert a preset number of pre-enhanced base layers into grayscale images, obtain the grayscale histogram of the grayscale images, calculate the uniformity index of the grayscale histogram, and select the pre-enhanced base layer with the largest uniformity index as the enhanced base layer and output it; Obtain the HSV space of the enhanced base layer, and perform detail enhancement on the detail layer based on the HSV space to obtain the enhanced detail layer.

5. An endoscopic image processing method for improving image quality according to claim 4, characterized in that: The steps of obtaining the HSV space of the enhanced base layer and performing detail enhancement on the detail layer based on the HSV space to obtain the enhanced detail layer are as follows: Obtain the HSV space of the enhanced base layer; Use the attention mechanism network to extract features from the edges of the blood vessels and lesion sites in the detail layer to obtain edge features, and use a sharpening filter to sharpen the edge features to obtain enhanced edges; Convert the pixel points in the detail layer except for the enhanced edge parts into the HSV space as the detail layer HSV space, preset a weight W1, and fuse the HSV space of the enhanced base layer and the detail layer HSV space based on the preset weight to obtain a fused color space; Reconstruct the detail layer based on the fused color space to obtain the enhanced detail layer.

6. An endoscopic image processing method for improving image quality according to claim 1, characterized in that: The steps of fusing the enhanced base layer and the enhanced detail layer are as follows: Preset weights W2 and W3; The fusion of the enhanced base layer and the enhanced detail layer based on the preset weights W2 and W2 is expressed as: Fusion image color space = W2 * enhanced base layer color space + W3 * enhanced detail layer color space If the ordinary image has not been segmented for the high-brightness reflection area, output the fused image as the enhanced image; If the ordinary image has been segmented for the high-brightness reflection area, output the image as the pre-enhanced image, repair the high-brightness reflection area in the pre-enhanced image, and output the repaired pre-enhanced image as the enhanced image.

7. An endoscopic image processing method for improving image quality according to claim 6, characterized in that: The steps of repairing the high-brightness reflection area in the pre-enhanced image and outputting the repaired pre-enhanced image as the enhanced image are as follows: Fuse the extracted high-brightness reflection area with the pre-enhanced image; Calculate the HSV mean value of non-highlight reflective pixels outside the highlighted reflective area as I avg ; Using I avg The HSV value expression for iteratively adjusting the highlight reflection area is expressed as: I(x,y) new = I(x,y) old + λ(I avg - I(x, y) old ) where I(x, y) new represents the HSV value of the highlighted reflection pixel points after iteration, and I(x, y) old represents the HSV value of the original highlighted reflection pixel points, and λ is an adjustable parameter; Monitor the change in the HSV value of the high-brightness reflection area and calculate the standard deviation of the HSV value. Preset the standard deviation value. When the standard deviation of the HSV value of the iterated high-brightness reflection area is less than the preset standard deviation value, terminate the iteration and output the enhanced image.

8. An endoscope image processing system for improving image quality, which is used to implement an endoscope image processing method for improving image quality according to any one of claims 1-7, characterized in that: Highlight area segmentation module: Obtain the endoscope image, detect whether there is a high-brightness reflection area on the endoscope image. If so, extract the high-brightness reflection area. If not, do not process it, and define the endoscope image passed through the detection as an ordinary image; Image enhancement module: Connected to the highlight area segmentation module, divide the ordinary image into a base layer and a detail layer, perform segmentation on the base layer and adjust the illuminance to obtain the enhanced base layer, and perform detail enhancement on the detail layer to obtain the enhanced detail layer; Highlight area repair module: Connected to the image enhancement module, fuse the enhanced base layer and the enhanced detail layer. If the ordinary image has not been extracted for the high-brightness reflection area, output it as the enhanced image. If the ordinary image has been extracted for the high-brightness reflection area, output it as the pre-enhanced image and repair the high-brightness reflection area.

Images