Endoscope guided image filtering method based on FPGA (Field Programmable Gate Array)

Abstract

The invention provides an endoscope guide image filtering method based on an FPGA (Field Programmable Gate Array), which comprises the following steps of: copying and expanding an image to obtain a guide image; carrying out accumulation operation on the gray value of the guide image; the gray value of the guide image is squared and then accumulated; calculating the absolute value cumulative sum of the gray value difference between the central point and the surrounding pixel points; calculating a linear weight coefficient of the guide image; calculating a guide filtering output value; calculating the detail layer numerical values of the extended guide image and the guide filtering value of the input image; calculating a weight coefficient; comparing the weight coefficient with an input sharpening degree value, and adjusting the weight coefficient; calculating a final guide image filtering value according to the guide filtering output value, the weight coefficient and the detail layer numerical value; and by adjusting a sharpening degree value, different levels of filtering and detail enhancement effects are observed. According to the method, real-time noise reduction can be realized, a clear image can be generated, original details of the image can be reserved, and edge and detail enhancement of different degrees can be achieved by adjusting the sharpening degree.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscope image processing, and in particular to an endoscope guided image filtering method based on FPGA. Background Art

[0002] Image filtering algorithm is one of the key technologies in computer vision and imaging systems. With the development of endoscope technology, doctors are more concerned about how to make the imaging system clearer without losing the original details. The filtering effect of endoscope on the image plays an extremely important role in the doctor's auxiliary diagnosis and minimally invasive surgery of patients. Through the endoscope, doctors can directly observe the tissue morphology and lesions of the internal organs of the human body.

[0003] Since the clarity of endoscopic images will directly affect the doctor's judgment, and filtering may remove certain detailed information such as nerves and blood vessels, which may cause doctors to analyze lesions incorrectly or increase surgical risks, it is particularly important to retain detailed information while improving clarity through endoscopic filtering.

[0004] FPGA (Field-Programmable Gate Array) devices are a kind of semi-customized circuit in application-specific integrated circuits, and are field-programmable logic arrays. The development of FPGAs in the field of image processing has practical significance. They have advantages such as strong parallel processing capabilities, high flexibility, and short development cycles. Guided image filtering requires a lot of image preprocessing, repeated calculations, and parallel processing, which makes FPGAs very advantageous in dealing with such algorithm problems.

[0005] Current endoscope camera filtering algorithms have several flaws:

[0006] (1) The filtering algorithm used in traditional endoscopes will blur the image while reducing image noise, which means that some detailed information is lost, especially the edges and details of the patient's diseased tissue.

[0007] (2) Traditional endoscope filtering algorithms are not strong enough in suppressing Gaussian noise, resulting in insufficient suppression of the endoscope's own thermal noise and channel noise.

[0008] (3) Traditional endoscopic filtering algorithms may not be able to set the filtering intensity. Once the filtering algorithm is designed, the parameters are customized, and doctors cannot analyze tissue details by changing the filtering degree of each gradient. Summary of the invention

[0009] In order to solve the various problems existing in the prior art, the purpose of the present invention is to provide an endoscope-guided image filtering method based on FPGA, which can reduce noise in real time, generate clear images, retain the original details of the image, and achieve different degrees of edge and detail enhancement by adjusting the degree of sharpening.

[0010] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0011] An endoscope guided image filtering method based on FPGA, the method comprising the following steps:

[0012] Cache the input image; copy and expand the image to obtain a guide image;

[0013] Performing cumulative operation on the grayscale values ​​of the guide image to obtain a cumulative sum;

[0014] The grayscale values ​​of the guide image are squared and then accumulated to obtain the cumulative value of the sum of squares;

[0015] Calculate the absolute cumulative sum of the grayscale value differences between the center point and the surrounding pixels;

[0016] Calculate the linear weight coefficients ak and bk of the guide image respectively;

[0017] Calculate the guided filter output value;

[0018] Calculate the detail layer values ​​of the extended guide image and the guide filter value of the input image;

[0019] Calculate weight coefficients;

[0020] Compare the weight coefficient with the input sharpness value and adjust the weight coefficient;

[0021] Calculate the final guided image filter value according to the guided filter output value, weight coefficient and detail layer value;

[0022] Observe the different levels of filtering and detail enhancement by adjusting the sharpness value.

[0023] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the image is copied and expanded according to the m and n parameters, including:

[0024] The input grayscale image I(x,y) is expanded by m columns on the left and right and n rows on the top and bottom to obtain the guided image I'(x+2m,y+2n), where x is the horizontal coordinate of the image and y is the vertical coordinate.

[0025] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the calculation formula of the cumulative sum sum_I(x, y) is as follows:

[0026]

[0027] The calculation formula for the sum of squares sum_I_square(x,y) is as follows:

[0028]

[0029] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the calculation formula for the absolute value sum_value of the gray value difference between the center pixel and the surrounding pixels is as follows:

[0030]

[0031] According to an FPGA-based endoscope guided image filtering method provided by the present invention, two linear weight coefficients ak and bk of the guided image are calculated, which are expressed as the following formula:

[0032] ak(x,y)=(sum_I_square(x,y)*25-sum_I(x,y)) / (sum_I_square(x,y)*25-sum_I(x,y)+406250)

[0033] bk(x,y)=(1024-ak(x,y))*sum_I*4.

[0034] According to an FPGA-based endoscope guided image filtering method provided by the present invention, a guided filtering output value is calculated and expressed as the following formula:

[0035] y_guide(x,y)=ak*I(x,y)+bk(x,y).

[0036] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the detail layer value is calculated, which is expressed as the following formula:

[0037] y_value(x,y)=I(x,y)-y_guide(x,y).

[0038] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the weight coefficient is calculated and expressed as the following formula:

[0039] y_avg(x,y)=1024+sum_value(x,y)*41.

[0040] According to an FPGA-based endoscope-guided image filtering method provided by the present invention, the adjusting weight coefficient comprises:

[0041] Use the sharpness value α (0-10) as the threshold and limit the weight coefficient:

[0042] If y_avg(x,y)>=1024*α, the weight coefficient is 1024*α; otherwise, y_avg(x,y) is used.

[0043] According to an FPGA-based endoscope guided image filtering method provided by the present invention, the final guided image filtering value is calculated according to the guided filtering output value y_guide(x,y), the weight coefficient y_avg(x,y), and the detail layer value y_value(x,y), which is expressed as the following formula:

[0044] y_new(x,y)=(y_guide(x,y)*1024+y_avg(x,y)*y_value(x,y)) / 1024

[0045] It can be seen that compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. FPGA is used to calculate the guided image filtering value, which is real-time and meets the requirements of the endoscope system for the real-time response speed of the algorithm.

[0047] 2. By setting different sharpness values, you can see different edge enhancement effects, which can better assist doctors in diagnosing lesions and reducing surgical risks.

[0048] 3. FPGA, as a hardware platform, can achieve high-speed parallel computing, thereby significantly improving the processing speed of image filtering. This is especially important for endoscopic image filtering, because endoscopic surgery often needs to be performed in an environment with extremely high real-time requirements.

[0049] 4. The guided filtering algorithm can well preserve the edge and gradient information of the image, avoiding the edge blurring and gradient flipping that may be caused by traditional filtering methods. This is crucial for endoscopic images because edge and gradient information are of great value to doctors' diagnosis and operation.

[0050] 5. By adjusting the sharpness value, the present invention can observe different levels of filtering and detail enhancement effects, thereby meeting the image quality requirements in different surgical scenarios.

[0051] 6. The image filtering module implemented by FPGA can be easily integrated into the entire endoscopic image processing system for system testing and optimization.

[0052] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1It is a flow chart of an embodiment of an endoscope-guided image filtering method based on FPGA of the present invention.

[0054] Figure 2 It is a schematic diagram of an embodiment of an endoscope-guided image filtering method based on FPGA of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] See also Figure 1 and Figure 2 The present invention provides an endoscope guided image filtering method based on FPGA, the method comprising the following steps:

[0058] Step S1, caching the input image; copying and expanding the image to obtain a guide image;

[0059] Step S2, performing an accumulation operation on the grayscale values ​​of the guide image to obtain an accumulated sum;

[0060] Step S3, squaring the grayscale value of the guide image and then accumulating it to obtain a cumulative value of the sum of squares;

[0061] Step S4, calculating the absolute value cumulative sum of the gray value difference between the center point and the surrounding pixels;

[0062] Step S5, respectively calculating the linear weight coefficients ak and bk of the guide image;

[0063] Step S6, calculating the guide filter output value;

[0064] Step S7, calculating the detail layer values ​​of the extended guide image and the guide filter value of the input image;

[0065] Step S8, calculating the weight coefficient;

[0066] Step S9, comparing the weight coefficient with the input sharpening degree value, and adjusting the weight coefficient;

[0067] Step S10, calculating the final guided image filter value according to the guided filter output value, the weight coefficient and the detail layer value;

[0068] Among them, by adjusting the sharpness value, observe different levels of filtering and detail enhancement effects.

[0069] In the above step S1, the input image is cached into 3 lines, and then the image is copied and expanded according to the m and n parameters.

[0070] In the above step S2, the grayscale values ​​of the 5*5 matrix of the guide image are first squared and then accumulated to obtain the cumulative value of the sum of squares.

[0071] In the above step S3, the absolute value cumulative sum of the grayscale value difference between the center point and the surrounding 5x5-1 pixels is calculated, that is, the absolute value of the difference between the center point and the surrounding 5*5-1 pixels is calculated one by one with the center point as a fixed value to obtain the cumulative sum.

[0072] In this embodiment, the image is copied and extended according to the m and n parameters, including:

[0073] The input grayscale image I(x,y) is expanded by m columns on the left and right and n rows on the top and bottom to obtain the guided image I'(x+2m,y+2n), where x is the horizontal coordinate of the image and y is the vertical coordinate.

[0074] In this embodiment, the following intermediate variables are calculated by guiding the image, combining the original image coordinates, using a 5*5 matrix operator and the original image coordinate value as the center point:

[0075] The calculation formula of the cumulative sum sum_I(x,y) is as follows:

[0076]

[0077] The calculation formula of the square sum_I_square(x,y) is as follows:

[0078]

[0079] The calculation formula for the absolute value sum of the grayscale value difference between the center pixel and the surrounding pixels is as follows:

[0080]

[0081] In this embodiment, two linear weight coefficients ak and bk of the guide image are calculated, which are expressed as the following formula:

[0082] ak(x,y)=(sum_I_square(x,y)*25-sum_I(x,y)) / (sum_I_square(x,y)*25-sum_I(x,y)+406250)

[0083] bk(x,y)=(1024-ak(x,y))*sum_I*4.

[0084] In this embodiment, the guided filtering output value is calculated and expressed as the following formula:

[0085] y_guide(x,y)=ak*I(x,y)+bk(x,y).

[0086] In this embodiment, the detail layer value is calculated as follows:

[0087] y_value(x,y)=I(x,y)-y_guide(x,y).

[0088] In this embodiment, the weight coefficient is calculated and magnified to 1024 times, which is expressed as the following formula:

[0089] y_avg(x,y)=1024+sum_value(x,y)*41.

[0090] In this embodiment, the adjusting weight coefficient includes:

[0091] Use the sharpness value α (0-10) as the threshold and limit the weight coefficient:

[0092] If y_avg(x,y)>=1024*α, the weight coefficient is 1024*α; otherwise, y_avg(x,y) is used.

[0093] In this embodiment, the final guide image filter value is calculated according to the guide filter output value y_guide(x,y), the weight coefficient y_avg(x,y), and the detail layer value y_value(x,y), which is expressed as the following formula:

[0094] y_new(x,y)=(y_guide(x,y)*1024+y_avg(x,y)*y_value(x,y)) / 1024

[0095] This embodiment also provides an FPGA-based endoscope-guided image filtering system, including:

[0096] Image cache module, used to cache the input image in 3 lines;

[0097] The extended guided image module is used to copy and expand the image according to the m and n parameters;

[0098] The accumulation and sum module is used to perform accumulation operation on the grayscale values ​​of the 5*5 matrix of the guide image to obtain the accumulated sum.

[0099] The square sum accumulation module is used to first square the grayscale values ​​of the above 5*5 matrix and then accumulate them to obtain the cumulative value of the square sum.

[0100] The absolute value cumulative sum module is used to calculate the absolute value cumulative sum of the grayscale value difference between the center point and the surrounding pixels. That is, taking the center point as a fixed value, the absolute value of the difference between the center point and the surrounding 5*5-1 pixels is calculated one by one to obtain the cumulative sum.

[0101] The linear weight coefficient module is used to calculate the linear weight coefficients ak and bk of the guide image respectively.

[0102] The guided filter value module is used to calculate the guided filter output value.

[0103] The detail layer value module is used to obtain the detail layer value by calculating the guide image and the guide filter value obtained after expanding the input image.

[0104] The weight coefficient calculation module is used to accumulate the absolute value of the difference obtained, add 1 through the fixed coefficient, and the fixed coefficient and 1 are respectively amplified by 1024 times to retain more integer part values.

[0105] According to the obtained weight coefficient, the input sharpness value is magnified 1024 times as the threshold. The two are compared. If it is greater than or equal to the threshold, the threshold is limited to the highest value of the weight coefficient, otherwise the original weight coefficient is retained. Note that this weight coefficient is magnified 1024 times and needs further processing later.

[0106] The guided image filter value output module is used to amplify the obtained guided filter output value module by 1024 times, and multiply the weight coefficient amplified by 1024 times by the detail layer value obtained in step 6. The result obtained after calculation is shifted right by 10 in the FPGA to obtain the required grayscale value, which is the guided image filter value output result.

[0107] In this embodiment, the sharpening degree value is set to 3 according to empirical values, which can achieve a clear image after filtering and a good visual enhancement effect on details.

[0108] If you need to see different levels of filtering and detail enhancement, you can get varying effects by adjusting the sharpness value and repeating steps 7 and 9.

[0109] Since FPGA is a hardware structure based on a lookup table, the above steps need to limit the bit width according to the bit width and precision requirements of the actual image to avoid data overflow and precision value being truncated and discarded.

[0110] In summary, the present invention uses FPGA to calculate the guided image filter value, which is real-time and meets the requirements of the endoscope system for the real-time response speed of the algorithm. By setting different sharpening values, the present invention can make the image clear while retaining more or even enhancing the edge information. The user can also adjust the filter sharpening value α to see the degree of change of the filtering effect and edge enhancement, see different edge enhancement effects, and better assist doctors in diagnosing lesions and reducing surgical risks. In addition, this filtering function has better practical significance in applications such as assisting doctors in diagnosis and intelligent analysis.

[0111] Furthermore, FPGA, as a hardware platform, can achieve high-speed parallel computing, thereby significantly improving the processing speed of image filtering, which is especially important for endoscopic image filtering because endoscopic surgery often needs to be performed in an environment with extremely high real-time requirements.

[0112] Furthermore, the guided filtering algorithm can well preserve the edge and gradient information of the image, avoiding the edge blurring and gradient flipping that may be caused by traditional filtering methods. This is crucial for endoscopic images because the edge and gradient information are of great value to doctors' diagnosis and operation.

[0113] Furthermore, the present invention can observe different levels of filtering and detail enhancement effects by adjusting the sharpness value, thereby meeting image quality requirements in different surgical scenarios.

[0114] Furthermore, the image filtering module implemented by FPGA can be easily integrated into the entire endoscopic image processing system for system testing and optimization.

[0115] It should be noted that a person skilled in the art can understand that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in processor control codes. The above is only a specific implementation of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

[0116] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An FPGA-based endoscope-guided image filtering method, characterized in that: The method comprises the following steps: Cache the input image; copy and expand the image to obtain a guide image; Performing cumulative operation on the grayscale values ​​of the guide image to obtain a cumulative sum; The grayscale values ​​of the guide image are squared and then accumulated to obtain the cumulative value of the sum of squares; Calculate the absolute cumulative sum of the grayscale value differences between the center point and the surrounding pixels; Calculate the linear weight coefficients ak and bk of the guide image respectively; Calculate the guided filter output value; Calculate the detail layer values ​​of the extended guide image and the guide filter value of the input image; Calculate weight coefficients; Compare the weight coefficient with the input sharpness value and adjust the weight coefficient; Calculate the final guided image filter value according to the guided filter output value, weight coefficient and detail layer value; Observe the different levels of filtering and detail enhancement by adjusting the sharpness value.

2. The method according to claim 1, characterized in that The step of copying and extending the image according to the m and n parameters includes: The input grayscale image I(x,y) is expanded by m columns on the left and right and n rows on the top and bottom to obtain the guided image I'(x+2m,y+2n), where x is the horizontal coordinate of the image and y is the vertical coordinate.

3. The method according to claim 2, characterized in that: The calculation formula of the cumulative sum sum_I(x,y) is as follows: The calculation formula for the sum of squares sum_I_square(x,y) is as follows:

4. The method according to claim 2, characterized in that: The calculation formula for the absolute value sum of the grayscale value difference between the center pixel and the surrounding pixels is as follows:

5. The method according to claim 4, characterized in that: Calculate the two linear weight coefficients ak and bk of the guidance image, expressed as the following formula: ak(z,y)=(sum_I_square(x,y)*25-sum_I(x,y)) / (sum_I_square(x,y)*25-sum_I(x,y)+406250) bk(x,y)=(1024-ak(x,y))*sum_I*4.

6. The method according to claim 5, characterized in that: The guided filter output value is calculated and expressed as the following formula: y_guide(x,y)=ak*I(x,y)+bk(x,y).

7. The method according to claim 6, characterized in that: Calculate the detail layer value, expressed as the following formula: y_value(x,y)=I(x,y)-y_guide(x,y).

8. The method according to claim 7, characterized in that: Calculate the weight coefficient, expressed as the following formula: y_avg(x,y)=1024+sum_value(x,y)*41.

9. The method according to claim 8, characterized in that The adjustment weight coefficient includes: Use the sharpness value α (0-10) as the threshold and limit the weight coefficient: If y_avg(x,y)>=1024*α, the weight coefficient is 1024*α; otherwise, y_avg(x,y) is used.

10. The method according to claim 9, characterized in that: According to the guide filter output value y_guide(x,y), the weight coefficient y_avg(x,y), and the detail layer value y_value(x,y), the final guide image filter value is calculated, which is expressed as the following formula: y_new(x,y)=(y_guide(x,y)*1024+y_avg(x,y)*y_value(x,y)) / 1024.

Images