Endoscope image detail enhancement method, device, equipment and medium

Abstract

The invention provides a detail enhancement method and device for an endoscope image, equipment and a medium, and the method comprises the steps: obtaining a brightness image of the endoscope image, and carrying out the N-round iteration processing of the brightness image. In each iteration process, guiding filtering is carried out on the guiding image of the round based on the filtering coefficient of the round, and a filtered image of the round is obtained; and according to the current-round filtering coefficient and the current-round guiding image, performing detail enhancement on the current-round filtering image to obtain a current-round detail-enhanced image. And after N rounds of iteration are finished, determining a result image after the detail enhancement of the endoscope image according to the detail enhancement image of the Nth round. According to the process, multi-round detail enhancement is carried out on the endoscope image in an iteration mode, so that the result image contains more image details, the filtering coefficient adopted by iteration is determined according to the number of pixel points and the pixel value variance of the guide image in the filtering window, the edge retention degree of the image after guide filtering can be improved, and the image quality is improved. And the detail enhancement effect of the image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and medium for enhancing the details of an endoscopic image. Background Art

[0002] Endoscopic images are crucial for the diagnosis of early lesions and recurrence of dysplasia. As an optical camera, endoscopes generate high thermal noise during use, so endoscopic images often need to be processed clearly. The mainstream algorithms for endoscopic image detail enhancement in the industry mainly include: Histogram Equalization (HE) and UnsharPing Mask (USM).

[0003] The HE algorithm achieves image enhancement by remapping the grayscale of the image through the grayscale probability distribution of the input image, but its ability to obtain image details is poor, resulting in the problem of over-enhancement or under-enhancement in the final detail enhancement results of the image. The USM algorithm uses a low-pass filter to filter the input image to generate a low-frequency blurred image, and then determines the high-frequency detail image containing edge features based on the input image and the low-frequency blurred image, and gains the high-frequency detail image to obtain the final detail enhancement result of the image.

[0004] Compared with the HE algorithm, the USM algorithm has a good advantage in image detail extraction, but the additional noise introduced in the filtering process will reduce the edge prominence of the filtered image, and then affect the final detail enhancement effect of the image. In view of this, how to improve the detail enhancement effect of the image is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and medium for enhancing the details of an endoscopic image, which are used to improve the effect of enhancing the details of the image.

[0006] To achieve the aforementioned purpose, the technical solution of the embodiment of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for enhancing details of an endoscopic image, the method comprising:

[0008] In response to the detail enhancement instruction, acquiring a brightness image of the endoscopic image in a color space HSV;

[0009] Perform N rounds of iterative processing on the brightness image; in each round of iteration, perform guided filtering on the guided image of this round based on the filtering coefficient of this round to obtain the filtered image of this round, and determine the detail enhancement image of this round according to the filtering coefficient of this round, the guided image of this round, and the filtered image of this round; wherein, the guided image of the first round is the brightness image, and the guided image of non-first rounds is the detail enhancement image of the previous round; the filtering coefficient of any round is determined according to the number of pixel points and the pixel value variance within the filtering window of the guided image of this round;

[0010] Determine the result image after detail enhancement of the endoscopic image according to the detail enhancement image of the Nth round.

[0011] In some possible embodiments, the filtering coefficient of any round is obtained by the following method:

[0012] For any round of iteration, determine the edge weight according to the number of pixel points and the pixel value variance of this round; wherein, the edge weight is used to constrain the edge region of the image during the guided filtering process;

[0013] Modify the original cost function set for the guided filtering based on the edge weight to obtain the modified cost function;

[0014] Based on the linear regression algorithm, solve the modified cost function according to the pixel values of each pixel point in the guided image of this round to obtain the filtering coefficient of this round.

[0015] In some possible embodiments, the modified cost function of the filtering coefficient of any round is as follows:

[0016]

[0017] wherein, E(a k , b k ) is the modified cost function; is the original cost function; a k and b k are the filtering coefficients of this round, I i is the pixel value of the ith pixel point in the guided image of this round, λ is the regularization parameter, is the edge weight, 丨ω丨 is the number of pixel points in the guided image within the filtering window, is the pixel value variance of the guided image within the filtering window.

[0018] In some possible embodiments, in any two rounds of iteration, the parameter value of the regularization parameter of any round is different from that of the other round, and / or the window size of the filtering window of any round is different from that of the other round.

[0019] In some possible embodiments, the step of performing detail enhancement on the current round filtered image according to the current round filtering coefficient and the current round guiding image to obtain the current round detail enhanced image includes:

[0020] Subtract the pixel values ​​of the pixels at the same coordinates of the current round of guided image and the current round of filtered image to obtain the basic detail image;

[0021] Determine a gain factor according to the current round of filtering coefficients, and use the gain factor to perform noise reduction processing on the basic detail image to obtain a high-frequency detail image;

[0022] The pixel values ​​of the pixel points of the high-frequency detail image and the current round guide image at the same coordinates are added to obtain the current round detail enhanced image.

[0023] In some possible embodiments, the step of acquiring a brightness image of the endoscopic image in a color space HSV includes:

[0024] The endoscopic image is converted from the color space RGB to the color space HSV, and the brightness channel component of the endoscopic image in the color space HSV is used as the brightness image.

[0025] In some possible embodiments, determining the result image after detail enhancement of the endoscopic image according to the detail enhanced image of the Nth round includes:

[0026] The detail-enhanced image is restored from the color space HSV to the color space RGB, and the detail-enhanced image converted to the color space RGB is used as the result image.

[0027] In a second aspect, an embodiment of the present application further provides a device for enhancing details of an endoscopic image, the device comprising:

[0028] An image acquisition unit is configured to acquire a brightness image of the endoscopic image in a color space HSV in response to a detail enhancement instruction;

[0029] The iterative processing unit is configured to perform N rounds of iterative processing on the brightness image; in each round of iteration, the guided image of the current round is subjected to guided filtering based on the filtering coefficient of the current round to obtain the filtered image of the current round, and the filtered image of the current round is subjected to detail enhancement based on the filtering coefficient of the current round and the guiding image of the current round to obtain the detail enhanced image of the current round; wherein the first-round guiding image is the brightness image, and the non-first-round guiding image is the detail enhanced image of the previous round; the filtering coefficient of any round is determined based on the number of pixels and the pixel value variance of the guiding image of the current round in the filtering window;

[0030] The detail enhancement unit is configured to determine a result image of the endoscopic image after detail enhancement according to the detail enhanced image of the Nth round.

[0031] In a third aspect, an embodiment of the present application further provides an endoscope device, including an endoscope head, a processor, and a display screen:

[0032] The endoscope head is configured to: perform an image acquisition operation to acquire an endoscopic image;

[0033] The processor is configured to: in response to the detail enhancement indication, acquire a brightness image of the endoscopic image in a color space HSV;

[0034] The brightness image is iterated N times; in each iteration, the current round of guided images are guided filtered based on the current round of filtering coefficients to obtain the current round of filtered images, and the current round of filtered images are detailed enhanced based on the current round of filtering coefficients and the current round of guided images to obtain the current round of detail enhanced images; wherein the first round of guided images is the brightness image, and the non-first round of guided images is the previous round of detail enhanced images; the filtering coefficients of any round are determined based on the number of pixels and the pixel value variance of the guiding image in the filtering window;

[0035] Determine a result image after detail enhancement of the endoscopic image according to the detail enhancement image of the Nth round;

[0036] The display screen is configured to output and display the result image.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method of any one of the aforementioned first aspects is implemented.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the computer device executes any one of the methods of the first aspect described above.

[0039] In an embodiment of the present application, a luminance image of an endoscopic image is obtained, and the luminance image is iterated N times. In each round of iteration, the guided image of this round is guided filtered based on the filter coefficient of this round to obtain the filtered image of this round; and the filtered image of this round is enhanced in detail according to the filter coefficient of this round and the guided image of this round to obtain the detail enhanced image of this round. The first guided image in the iterative process is a luminance image, and the non-first guided image is a detail enhanced image of the previous round. The filter coefficient of any round is determined based on the number of pixels and the pixel value variance of the guided image of this round in the filter window. After N rounds of iteration, the result image after detail enhancement of the endoscopic image is determined based on the detail enhanced image of the Nth round.

[0040] This iterative method performs multiple rounds of detail enhancement on endoscopic images, so that the final result image contains more image details. The filter coefficient used in each round of iteration is determined according to the number of pixels and pixel value variance of the guide image in the filter window, which can improve the edge retention of the image after guided filtering and enhance the detail enhancement effect of the image.

[0041] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present disclosure. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the USM algorithm provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the effect of traditional guided filtering provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0045] Figure 4 An overall flow chart of the method for enhancing the details of an endoscopic image provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of obtaining a brightness image provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a process flow of an iterative process provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of a filter coefficient obtaining process according to an embodiment of the present application;

[0049] Figure 8 A schematic diagram of a process for obtaining a detail-enhanced image provided in an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of a basic detail image provided in an embodiment of the present application;

[0051] Fig.10 A schematic diagram of a detail-enhanced image provided in an embodiment of the present application;

[0052] Fig.11 A schematic diagram of test results provided in an embodiment of the present application;

[0053] Fig.12A schematic diagram of a process for enhancing the details of an endoscopic image provided in an embodiment of the present application;

[0054] Fig.13 A structural diagram of a device for enhancing details of an endoscopic image provided in an embodiment of the present application;

[0055] Fig.14 A structural diagram of the endoscope device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0057] The terms "first" and "second" in the specification and claims of the present application and the aforementioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0058] Before introducing a method for enhancing details of an endoscopic image provided by an embodiment of the present application, for ease of understanding, the technical background of the embodiment of the present application is first introduced in detail:

[0059] As mentioned above, the mainstream algorithms for endoscopic image detail enhancement in the industry mainly include: histogram equalization algorithm HE and unsharp mask algorithm USM. Compared with the HE algorithm, the USM algorithm has a good advantage in image detail extraction, but the additional noise introduced in the filtering process will reduce the edge prominence of the filtered image, thereby affecting the final detail enhancement effect of the image.

[0060] The USM algorithm uses a low-pass filter to filter the input image to generate a low-frequency blurred image, and then determines the high-frequency detail image containing edge features based on the input image and the low-frequency blurred image, and gains the high-frequency detail image to obtain the final detail enhancement result of the image. Figure 1 As shown in the figure, the USM algorithm pre-filters the input image to generate a filtered image, and then determines the basic detail image containing edge features based on the pixel difference between the input image and the filtered image. Next, the basic detail image is subjected to gain processing to increase image details, and the pixel values ​​of the high-frequency detail image obtained by the gain processing and the input image are summed to obtain the final detail enhancement result of the input image.

[0061] In practical applications, edge presserving filters are often used to filter images such as endoscope images that have high requirements for edge preservation. Common edge presser filtering algorithms are mainly bilateral filtering and guided filtering. The algorithm complexity of bilateral filtering is high and there is a problem of gradient inversion. In practical applications, it is difficult to ensure that the final detail enhancement result of the image can meet medical needs. The guided filtering algorithm lacks consideration of the balance between detail enhancement and noise suppression. The noise introduced during the filtering process will reduce the edge prominence of the filtered image, which will in turn affect the final detail enhancement effect of the image.

[0062] The inventive concept of the embodiment of the present application is explained below in combination with the traditional guided filtering algorithm:

[0063] Guided filtering is an image filtering technique that filters the input image P through a pre-set guide image I, so that the image texture of the final output filtered image Q is similar to the guide image I. The filter function of guided filtering is specifically shown in the following formula (1):

[0064]

[0065] Where I is the guide image, Q is the filtered image, i is the pixel point of the filtered image Q; ω k is the filter window with radius r, where the radius r is an empirical value set manually; a k and b k It is the linear factor in the window, also known as the filter coefficient.

[0066] The role of guided filtering is to search for the filter coefficient (a k , b k ) is the optimal solution, so that the input image P i With the filtered image I iThe pixel difference between them is minimized (that is, the overall difference between the image before and after filtering is minimized). Based on this, the original cost function used to obtain the filter coefficient is defined in the guided filtering algorithm, as shown in the following formula (2):

[0067]

[0068] Among them, e(a k , b k ) is the original cost function, λ is a regularization parameter used to prevent the solution from exceeding the range, and is an artificially set empirical value.

[0069] In the process of guiding filtering the input image, the λ, I shown in the above formula (2) i and P i All are known values. Therefore, by using these known values ​​to perform regression analysis algorithm to solve the above formula (2), the optimal solution of the filter coefficients (ak, bk) that can minimize the overall difference before and after the image filtering can be obtained. The optimal solution can be shown in the following formula (3):

[0070]

[0071] Among them, |ω| is the guide image I in the filter window ω k The number of pixels in To guide the image I in the filter window ω k Variance of pixel values ​​within k To guide the image I in the filter window ω k The mean pixel value within The input image P is in the filter window ω k The mean pixel value within .

[0072] There are differences in the image texture in the filter window. These differences cannot guarantee the accuracy of image edge extraction, which in turn affects the image detail enhancement effect. The original cost function for obtaining the filter coefficient currently introduces a regularization parameter to avoid the solution result exceeding the range (a k The value is too large), but this also ignores the texture differences of the image in different filter windows. For images with rich details such as endoscopic images, this difference will make it difficult for the final filtering result to achieve the effect of image sharpness and edge prominence.

[0073] For example Figure 2 The left side shows the endoscopic image F acquired by the endoscopic device 1 (i.e., the original image), Figure 2 The right side shows the endoscope image F through the traditional guided filtering method. 1 The result image F after detail enhancement 2 .pass Figure 2Comparing the left and right images, we can see that the traditional guided filtering process can improve the image clarity to a certain extent. However, when the image is sharpened, the edge prominence effect is also reduced. That is, image F 2 The clarity of image F 1 Improved, but image F 2 The edges of many capillaries are no longer prominent.

[0074] In order to solve the above-mentioned problems, the invention of the present application is conceived as follows: by acquiring the brightness image of the endoscopic image, and performing N rounds of iterative processing on the brightness image. In each round of iteration, the guided image of this round is guided filtered based on the filter coefficient of this round to obtain the filtered image of this round; and the filtered image of this round is enhanced in detail according to the filter coefficient of this round and the guided image of this round to obtain the detail enhanced image of this round. The first round of guided images in the iterative process is the brightness image, and the non-first round of guided images are the detail enhanced images of the previous round. The filter coefficients of any round are determined according to the number of pixels and the pixel value variance of the guiding image of this round in the filter window. After N rounds of iteration are completed, the result image after detail enhancement of the endoscopic image is determined according to the detail enhanced image of the Nth round.

[0075] This iterative method performs multiple rounds of detail enhancement on endoscopic images, so that the final result image contains more image details. The filter coefficient used in each round of iteration is determined according to the number of pixels and pixel value variance of the guide image in the filter window, which can improve the edge retention of the image after guided filtering and enhance the detail enhancement effect of the image.

[0076] Next, the application scenarios of the embodiments of the present application are explained in terms of adaptability. Figure 3 A schematic diagram of an application scenario provided according to an embodiment of the present application.

[0077] like Figure 3 As shown in FIG. 1 , the application scenario may include, for example, a network 10 , an endoscope device 20 , and a server 30 . After the medical staff obtains an endoscopic image by performing an image acquisition operation through the endoscope device 20 , the endoscope device 20 sends the captured endoscopic image to the server 30 .

[0078] The server 30 has a filtering processing module for enhancing the details of the endoscopic image. After receiving the endoscopic image sent by the endoscopic device 20, the server 30 inputs the endoscopic image into the filtering processing module for detail enhancement processing to obtain a result image after the detail enhancement of the endoscopic image. The result image is sent to the endoscopic device 20 so that the endoscopic device 20 outputs the result image through a display screen for medical personnel to view.

[0079] It should be noted that, in practical applications, the server 30 can be used as an independent processor such as a cloud processor, a processor cluster, etc., which is independent of the endoscope device 20. In addition, the server 30 can also be used as an internal processor of the endoscope device 20, and the specific type of the server 30 is not limited in this application.

[0080] The description in this application only details a single server or terminal device, but those skilled in the art should understand that Figure 3 The server 30 shown is intended to represent the operation of the server involved in the technical solution of the present application. The individual servers are described in detail at least for the convenience of explanation, and do not imply any limitation on the number, type or location of the servers. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concepts of the example embodiments of the present application will not be changed.

[0081] Next, Figure 4 As shown, Figure 4 The overall process of a method for detail enhancement of an endoscopic image provided by the present application is shown, including the following steps:

[0082] Step 401: In response to a detail enhancement instruction, a brightness image of an endoscopic image in a color space HSV is acquired;

[0083] When the medical staff performs an image acquisition operation on the lesion through the endoscopic device, an endoscopic image of the lesion is obtained. At this time, the detail enhancement instruction for the endoscopic image is triggered. During implementation, the endoscopic image acquired by the endoscopic device is obtained, and the endoscopic image is converted from the color space RGB to HSV to determine the brightness image of the endoscopic image according to the brightness channel (V channel) component of the endoscopic image under the color space HSV.

[0084] Figure 5 The process of obtaining the brightness image is shown as an example. When implementing, the RGB value of each pixel in the endoscope image is obtained in advance. Then, for each pixel, the maximum value of the three values ​​of R, G, and B of the pixel is used as the brightness V value of the pixel, thereby obtaining the brightness image of the endoscope device image in the HSV color space.

[0085] It should be understood that the detail enhancement indication can be actively triggered by medical personnel through the endoscope device, for example, a detail enhancement button is set for the endoscope device, and the medical personnel triggers the detail enhancement indication by clicking the button). The detail enhancement indication can also be automatically triggered by the endoscope device, for example, a relevant control strategy is set for the endoscope device, and the detail enhancement indication is automatically triggered when the endoscope device collects the endoscopic image, so as to automatically enhance the details of the collected endoscopic image.

[0086] Step 402: performing N rounds of iterative processing on the brightness image; in each round of iteration, performing guided filtering on the current round of guided image based on the current round of filtering coefficients to obtain the current round of filtered image, and performing detail enhancement on the current round of filtered image based on the current round of filtering coefficients and the current round of guided image to obtain the current round of detail enhanced image; wherein the first round of guided image is the brightness image, and the non-first round of guided image is the previous round of detail enhanced image; the filtering coefficient of any round is determined according to the number of pixels and the pixel value variance of the guiding image of the round in the filtering window;

[0087] In each round of iteration process of the present application embodiment, the specific steps are as follows: Figure 6 As shown, the following steps are included:

[0088] Step 601: obtaining a current round of guiding images, and determining current round filtering coefficients according to the pixel values ​​of each pixel point in the current round of guiding images;

[0089] During the N-round iterative processing, the first-round guide image participating in filtering is the brightness image obtained in the aforementioned step 401. The non-first-round guide image is the detail-enhanced image output by the previous round of iteration.

[0090] Any round of filtering coefficients in the embodiment of the present application is obtained by Figure 7 The process shown obtains:

[0091] Step 701: for any round of iteration, determine the edge weight according to the number of pixels and the pixel value variance of this round; wherein the edge weight is used to constrain the edge area of ​​the image in the guided filtering process;

[0092] As mentioned above, the traditional guided filtering algorithm introduces a regularization parameter into the original cost function for obtaining the filter coefficient to avoid the solution exceeding the range (a k However, this also ignores the texture differences of the image in different filter windows, which makes it difficult for the final filtering result to achieve the effect of image sharpening and edge protrusion.

[0093] Based on this, the embodiment of the present application introduces an edge weight in the original cost function for obtaining the filter coefficient, that is, the aforementioned formula (2). The edge weight is used to constrain the edge size of the image during the guided filtering process, thereby improving the edge preservation of the image. The expression of the edge weight is shown in the following formula (5):

[0094]

[0095] In any round of iteration, Γ is the edge weight of this round, |ω| is the number of pixels in the filter window of the guide image of this round, is the pixel value variance of the current round of guidance image within the filter window.

[0096] Step 702: modifying the original cost function set by the guided filter based on the edge weight to obtain a modified cost function;

[0097] Taking the original cost function shown in the above formula (2) as an example, by substituting the edge weight Γ into the above formula (2) to constrain the edge size of the image during the filtering process, the modified cost function E(a) shown in the following formula (6) is obtained: k ,b k ):

[0098]

[0099] Step 703: Based on the linear regression algorithm, the modified cost function is solved according to the pixel value of each pixel point in the current round of guidance image to obtain the current round of filtering coefficients.

[0100] During implementation, the above formula (6) is solved by regression analysis algorithm to obtain the optimal solution of the filter coefficient that minimizes the overall difference before and after the image is filtered. The optimal solution can be shown in the following formula (7):

[0101]

[0102] Since the guide image and the input image in the embodiment of the present application are the same image, that is, I = P. Therefore, P in the above formula (7) is replaced by i Change to I i , the optimal solution of the filter coefficient can be obtained as shown in the following formula (8):

[0103]

[0104] In order to intuitively reflect the difference between the filter function obtained by the present application and the filter function obtained by solving the original cost function in the traditional guided filtering algorithm, the formula (8) is substituted into the output of the guided filtering, that is, the aforementioned formula (1), and the filter function expression of the following formula (9) is obtained:

[0105]

[0106] Comparing formula (1) with formula (9), we can see that the filter coefficient changes from the original ak to And changed from the original bk to In order to avoid confusion between the filter coefficients obtained in this application and the filter coefficients obtained in the traditional guided filtering algorithm, let

[0107] in, That is, this application modifies the cost function E(a k ,b k ) to obtain ak , That is, this application modifies the cost function E(a k , b k ) to obtain b k . And a k and b k That is, in the traditional guided filtering algorithm, the original cost function e(a k , b k ) to obtain the filter coefficients.

[0108] Step 602: performing guided filtering on the guided image of the current round through a guided filter based on the filtering coefficient of the current round to obtain a filtered image of the current round;

[0109] Step 603: Perform detail enhancement on the filtered image of this round according to the filtering coefficients of this round and the guiding image of this round to obtain the detail enhanced image of this round.

[0110] Specific as Figure 8 As shown, Figure 8 The detailed enhanced image acquisition process of the embodiment of the present application is shown, including the following steps:

[0111] Step 801: Subtract the pixel values ​​of the pixel points of the current round of guided image and the current round of filtered image at the same coordinates to obtain a basic detail image.

[0112] Since the size of the guide image and the filter image is the same, any pixel point I in the guide image i , there is a pixel in the filtered image Q that matches the pixel I i Pixels with the same coordinates Q i . When implementing Fig. 9 As shown, by taking the pixel point I under the same coordinate 1 With Q 1 ,I 2 With Q 2 ……I i With Q i The pixel values ​​of are subtracted to obtain the basic detail image D.

[0113] Step 802: determining a gain factor according to the current round of filtering coefficients, and using the gain factor to perform noise reduction processing on the basic detail image to obtain a high-frequency detail image;

[0114] The function of the gain factor is to reduce the background noise of the image. The gain factor β of the embodiment of the present application is obtained by the following formula (10):

[0115]

[0116] in, That is, the linear coefficient obtained by modifying the cost function of this application, as explained in the above formula (9), That is, it corresponds to the filter coefficient a in the traditional guided filter k .

[0117] Step 803: Add the pixel values ​​of the high-frequency detail image and the pixel values ​​of the current round guide image at the same coordinates to obtain the current round detail enhanced image.

[0118] Since the size of the guide image and the high-frequency detail image L is the same, any pixel I in the guide image I i , there is a pixel point I in the detail enhanced image G i Pixels G with the same coordinates i . When implementing Fig.10 As shown, by taking the pixel point I under the same coordinate 1 With G 1 ,I 2 With G 2 ……I i With G i The pixel values ​​of are added respectively to obtain the detail enhanced image E.

[0119] Step 403: Determine a result image after detail enhancement of the endoscopic image according to the detail enhanced image of the Nth round.

[0120] In an embodiment of the present application, the detail enhanced image output by the last round (i.e., the Nth round) of iterations is restored from the color space HSV to the color space RGB, and the detail enhanced image converted to the color space RGB is used as the result image after detail enhancement of the endoscopic image (i.e., the endoscopic image of the aforementioned step 401).

[0121] It should be noted that the process of guiding filtering the image will reduce the clarity of the non-edge area of ​​the image to a certain extent and highlight the edge of the image. In each round of iteration of the present application, by using the output of the previous round of iteration as the input of the next round of iteration, the ability to obtain non-edge details of the image can be improved while highlighting the edge of the image.

[0122] In addition, as mentioned above, the regularization parameter λ and the radius r of the filter window are both manually set empirical values. In the traditional guided filtering process, there is only one filtering process. In the multi-iteration process of the present application, different regularization parameters λ and, or the radius r of the filter window can be set for each round, so that in any two rounds of iteration, the parameter value of the regularization parameter of any round is different from that of the regularization parameter of another round, and / or the window size of the filter window of any round is different from that of the filter window of another round.

[0123] Although there is no clear value restriction on the radius r and the regularization parameter λ, the industry has a common empirical range for the radius r and the regularization parameter λ when performing guided filtering on images. Since the smaller the radius r, the blurrier the final image content, and the smaller λ, the clearer the final image content, by ensuring that the radius r and the regularization parameter λ of any two rounds in multiple rounds of iterative processing are not exactly the same, the final detail-enhanced image can contain more image details.

[0124] in, Fig.11 The final result image is shown under the condition of setting different radius r and regularization parameter λ in each round. After a large number of tests, it is found that when the number of iterations N is 2, the non-edge details of the image can be retained to the greatest extent and the image edges can be highlighted.

[0125] To facilitate understanding of the technical solution provided in the embodiment of the present application, the above-mentioned image detail enhancement process is described below by taking two rounds of iterative processing of the endoscopic image F as an example. Fig.12 As shown, the following steps are included:

[0126] Step 121: converting the endoscope image F from the RGB color space to the HSV color space;

[0127] Step 122: extract the V channel component of the image in the HSV color space to obtain a lightness image; and use the lightness image as the guide image I of the first round of iteration to enter the first round of iteration;

[0128] Step 123: Obtaining the filter coefficient of this round of iteration according to the pixel value of each pixel point in the current round of guidance image I;

[0129] Step 124: performing guided filtering processing on the guided image I of this round based on the filtering coefficient of this round iteration to obtain the filtered image Q of this round;

[0130] Step 125: Obtain the basic detail image D of this round according to the pixel difference between the current round filtered image Q and the current round guided image I;

[0131] Step 126: determining a corresponding gain factor β according to the filter coefficient of this round of iteration, and using the gain factor β to perform noise reduction processing on the basic detail image D of this round to obtain a high-frequency detail image G of this round;

[0132] Step 127: Obtain the detail enhanced image E output by this round of iteration according to the pixel sum values ​​of the high-frequency detail image G of this round and the guide image I of this round;

[0133] Step 128: Determine whether the current iteration number is less than 2;

[0134] Step 129: If it is not less than 2, the detail enhanced image E obtained in this round is restored from the HSV color space to the RGB color space to obtain the result image F'; if it is less than 2, the detail enhanced image E output in this round of iteration is used as the guide image P to enter the next round of iteration.

[0135] It should be noted that the technical solution of this application is not only applicable to the processing of endoscopic images. After testing, it was found that the detail enhancement process of this application can make the image edge more prominent on the basis of overall image sharpening. The technical solution of this application is applicable to all images with high requirements for edge preservation.

[0136] Based on the same inventive concept, the present application also provides a device for enhancing the details of an endoscopic image, specifically, Fig.13 As shown, including:

[0137] The image acquisition unit 131 is configured to acquire a brightness image of the endoscope image in a color space HSV in response to the detail enhancement instruction;

[0138] The iterative processing unit 132 is configured to perform N rounds of iterative processing on the brightness image; in each round of iteration, the guided image of the current round is subjected to guided filtering based on the filtering coefficient of the current round to obtain the filtered image of the current round, and the filtered image of the current round is subjected to detail enhancement based on the filtering coefficient of the current round and the guiding image of the current round to obtain the detail enhanced image of the current round; wherein the first-round guiding image is the brightness image, and the non-first-round guiding image is the detail enhanced image of the previous round; the filtering coefficient of any round is determined based on the number of pixels and the pixel value variance of the guiding image of the current round in the filtering window;

[0139] The detail enhancement unit 133 is configured to determine a result image after detail enhancement of the endoscopic image according to the detail enhanced image of the Nth round.

[0140] In some possible embodiments, the filter coefficients of any round are obtained in the following manner:

[0141] For any round of iteration, the edge weight is determined according to the number of pixels and the variance of pixel values ​​in this round; wherein the edge weight is used to constrain the edge area of ​​the image in the guided filtering process;

[0142] Modifying the original cost function set by the guided filter based on the edge weight to obtain a modified cost function;

[0143] Based on the linear regression algorithm, the modified cost function is solved according to the pixel value of each pixel point in the current round of guide image to obtain the current round of filter coefficients.

[0144] In some possible embodiments, the modified cost function of any round of filtering coefficients is as follows:

[0145]

[0146] Among them, E(a k ,b k ) is the modified cost function; is the original cost function; a k and b k is the current round filter coefficient, I i is the pixel value of the i-th pixel in the current guide image, λ is the regularization parameter, is the edge weight, |ω| is the number of pixels in the filter window of the current guide image, is the pixel value variance of the current round of guidance image within the filter window.

[0147] In some possible embodiments, in any two rounds of iterations, the parameter value of the regularization parameter of any round is different from that of the other round, and / or the window size of the filtering window of any round is different from that of the other round.

[0148] In some possible embodiments, the detail enhancement of the current round filtered image according to the current round filtering coefficient and the current round guiding image is performed to obtain the current round detail enhanced image, and the iterative processing unit 132 is configured as follows:

[0149] Subtract the pixel values ​​of the pixels at the same coordinates of the current round of guided image and the current round of filtered image to obtain the basic detail image;

[0150] Determine a gain factor according to the current round of filtering coefficients, and use the gain factor to perform noise reduction processing on the basic detail image to obtain a high-frequency detail image;

[0151] The pixel values ​​of the pixel points of the high-frequency detail image and the current round guide image at the same coordinates are added to obtain the current round detail enhanced image.

[0152] In some possible embodiments, the image acquisition unit 131 is configured to:

[0153] The endoscopic image is converted from the color space RGB to the color space HSV, and the brightness channel component of the endoscopic image in the color space HSV is used as the brightness image.

[0154] In some possible embodiments, the step of determining the result image after detail enhancement of the endoscopic image according to the detail enhanced image of the Nth round is performed, and the detail enhancement unit 133 is configured as follows:

[0155] The detail-enhanced image is restored from the color space HSV to the color space RGB, and the detail-enhanced image converted to the color space RGB is used as the result image.

[0156] Based on the same inventive concept, the present application also provides an endoscope device, specifically, Fig.14 As shown, it includes an endoscope lens 141, a processor 142 and a display screen 143:

[0157] The endoscope lens 141 is configured to: perform an image acquisition operation to acquire an endoscopic image;

[0158] The processor 142 is configured to: in response to the detail enhancement indication, acquire a brightness image of the endoscopic image in the color space HSV;

[0159] The brightness image is iterated N times; in each iteration, the current round of guided images are guided filtered based on the current round of filtering coefficients to obtain the current round of filtered images, and the current round of filtered images are detailed enhanced based on the current round of filtering coefficients and the current round of guided images to obtain the current round of detail enhanced images; wherein the first round of guided images is the brightness image, and the non-first round of guided images is the previous round of detail enhanced images; the filtering coefficients of any round are determined based on the number of pixels and the pixel value variance of the guiding image in the filtering window;

[0160] Determine a result image after detail enhancement of the endoscopic image according to the detail enhancement image of the Nth round;

[0161] The display screen 143 is configured to output and display the result image.

[0162] In some possible embodiments, the filter coefficients of any round are obtained in the following manner:

[0163] For any round of iteration, the edge weight is determined according to the number of pixels and the variance of pixel values ​​in this round; wherein the edge weight is used to constrain the edge area of ​​the image in the guided filtering process;

[0164] Modifying the original cost function set by the guided filter based on the edge weight to obtain a modified cost function;

[0165] Based on the linear regression algorithm, the modified cost function is solved according to the pixel value of each pixel point in the current round of guide image to obtain the current round of filter coefficients.

[0166] In some possible embodiments, the modified cost function of any round of filtering coefficients is as follows:

[0167]

[0168] Among them, E(a k ,bk ) is the modified cost function; is the original cost function; a k and b k is the current round filter coefficient, I i is the pixel value of the i-th pixel in the current guide image, λ is the regularization parameter, is the edge weight, |ω| is the number of pixels in the filter window of the current guide image, is the pixel value variance of the current round of guidance image within the filter window.

[0169] In some possible embodiments, in any two rounds of iterations, the parameter value of the regularization parameter of any round is different from that of the other round, and / or the window size of the filtering window of any round is different from that of the other round.

[0170] In some possible embodiments, the processor 142 is configured to perform detail enhancement on the current round filtered image according to the current round filtering coefficient and the current round guide image to obtain the current round detail enhanced image:

[0171] Subtract the pixel values ​​of the pixels at the same coordinates of the current round of guided image and the current round of filtered image to obtain the basic detail image;

[0172] Determine a gain factor according to the current round of filtering coefficients, and use the gain factor to perform noise reduction processing on the basic detail image to obtain a high-frequency detail image;

[0173] The pixel values ​​of the pixel points of the high-frequency detail image and the current round guide image at the same coordinates are added to obtain the current round detail enhanced image.

[0174] In some possible embodiments, to perform the step of acquiring a brightness image of an endoscopic image in a color space HSV, the processor 142 is configured to:

[0175] The endoscopic image is converted from the color space RGB to the color space HSV, and the brightness channel component of the endoscopic image in the color space HSV is used as the brightness image.

[0176] In some possible embodiments, the processor 142 is configured to:

[0177] The detail-enhanced image is restored from the color space HSV to the color space RGB, and the detail-enhanced image converted to the color space RGB is used as the result image.

[0178] The embodiment of the present application further provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned traffic signal control method.

[0179] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0180] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor 202 of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor 202 of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0182] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0183] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detail enhancement of an endoscopic image, characterized in that: The method comprises: In response to the detail enhancement instruction, acquiring a brightness image of the endoscopic image in a color space HSV; The brightness image is iterated N times; in each iteration, the current round of guided images are guided filtered based on the current round of filtering coefficients to obtain the current round of filtered images, and the current round of filtered images are detailed enhanced based on the current round of filtering coefficients and the current round of guided images to obtain the current round of detail enhanced images; wherein the first round of guided images is the brightness image, and the non-first round of guided images is the previous round of detail enhanced images; the filtering coefficients of any round are determined based on the number of pixels and the pixel value variance of the guiding image in the filtering window; The result image after detail enhancement of the endoscopic image is determined according to the detail enhanced image of the Nth round.

2. The method according to claim 1, characterized in that The filter coefficients for any round are obtained in the following way: For any round of iteration, the edge weight is determined according to the number of pixels and the variance of pixel values ​​in this round; wherein the edge weight is used to constrain the edge area of ​​the image in the guided filtering process; Modifying the original cost function set by the guided filter based on the edge weight to obtain a modified cost function; Based on the linear regression algorithm, the modified cost function is solved according to the pixel value of each pixel point in the current round of guide image to obtain the current round of filter coefficients.

3. The method according to claim 2, characterized in that The modified cost function of any round of filtering coefficients is as follows: Among them, E(a k , b k ) is the modified cost function; is the original cost function; a k and b k is the current round filter coefficient, I i is the pixel value of the i-th pixel in the current guide image, λ is the regularization parameter, is the edge weight, |ω| is the number of pixels in the filter window of the current guide image, is the pixel value variance of the current round of guidance image within the filter window.

4. The method according to claim 2 or 3, characterized in that: In any two rounds of iterations, the parameter value of the regularization parameter in any round is different from that in another round, and / or the window size of the filtering window in any round is different from that in another round.

5. The method according to claim 1, characterized in that The step of performing detail enhancement on the current round filtered image according to the current round filtering coefficient and the current round guiding image to obtain the current round detail enhanced image comprises: Subtract the pixel values ​​of the pixels at the same coordinates of the current round of guided image and the current round of filtered image to obtain the basic detail image; Determine a gain factor according to the current round of filtering coefficients, and use the gain factor to perform noise reduction processing on the basic detail image to obtain a high-frequency detail image; The pixel values ​​of the pixel points of the high-frequency detail image and the current round guide image at the same coordinates are added to obtain the current round detail enhanced image.

6. The method according to any one of claims 1 to 3 and 5, characterized in that: The step of obtaining a brightness image of the endoscopic image in a color space HSV includes: The endoscopic image is converted from the color space RGB to the color space HSV, and the brightness channel component of the endoscopic image in the color space HSV is used as the brightness image.

7. The method according to any one of claims 1 to 3 and 5, characterized in that: The step of determining the result image after detail enhancement of the endoscopic image according to the detail enhanced image of the Nth round includes: The detail-enhanced image is restored from the color space HSV to the color space RGB, and the detail-enhanced image converted to the color space RGB is used as the result image.

8. An endoscope device, characterized in that: Includes endoscope lens, processor and display: The endoscope head is configured to: perform an image acquisition operation to acquire an endoscopic image; The processor is configured to: in response to the detail enhancement indication, acquire a brightness image of the endoscopic image in a color space HSV; The brightness image is iterated N times; in each iteration, the current round of guided images are guided filtered based on the current round of filtering coefficients to obtain the current round of filtered images, and the current round of filtered images are detailed enhanced based on the current round of filtering coefficients and the current round of guided images to obtain the current round of detail enhanced images; wherein the first round of guided images is the brightness image, and the non-first round of guided images is the previous round of detail enhanced images; the filtering coefficients of any round are determined based on the number of pixels and the pixel value variance of the guiding image in the filtering window; Determine a result image after detail enhancement of the endoscopic image according to the detail enhancement image of the Nth round; The display screen is configured to output and display the result image.

9. The device according to claim 8, characterized in that The filter coefficients for any round are obtained in the following way: For any round of iteration, the edge weight is determined according to the number of pixels and the variance of pixel values ​​in this round; wherein the edge weight is used to constrain the edge area of ​​the image in the guided filtering process; Modifying the original cost function set by the guided filter based on the edge weight to obtain a modified cost function; Based on the linear regression algorithm, the modified cost function is solved according to the pixel value of each pixel point in the current round of guide image to obtain the current round of filter coefficients.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method according to any one of claims 1 to 7.

Images