Lens shading correction method and device for endoscopic imaging and endoscope system

Abstract

The present application relates to a lens shading correction method and device for endoscopic imaging, and an endoscope system. Based on the present application, the lens shading correction calibration parameters obtained by calibrating the optical characteristics of the endoscope can be corrected using the brightness information of each frame of the target image imaged by the endoscope to obtain lens shading correction revision parameters that are compatible with the optical characteristics of the endoscope and the actual brightness of the target image in that frame. Thus, by using the lens shading correction revision parameters to perform lens shading correction processing on the target image, the shading defects in the target image can be improved, and the edge overexposure of the target image can be suppressed at the same time.

Description

Technical Field

[0001] The present application relates to medical imaging technology, and in particular to a lens shading correction method for endoscopic imaging, a lens shading correction device for endoscopic imaging, and an endoscope system. Background Art

[0002] Images formed through optical lenses typically have a brightness distribution with bright centers and dark edges. This brightness distribution is due to the optical properties of the lens. Furthermore, based on this brightness distribution, the image often contains shading defects at the edges, such as luma shading caused by uneven brightness and color shading caused by uneven color. Lens shading correction (LSC) can improve shading defects in images by increasing the gain at the edges, and is therefore widely used in image processing for various imaging scenarios.

[0003] Endoscopic images often exhibit the aforementioned shading defects. However, unlike other imaging scenarios, endoscopic imaging of the human body requires supplemental illumination. Furthermore, this supplemental illumination results in images with a dark center and bright edges. In particular, if human tissue, such as fat, appears at the edges of the image, these tissues are likely to form bright areas at the edges. Therefore, applying lens shading correction to such images can easily lead to overexposure at the edges, resulting in a loss of useful image information.

[0004] It can be seen that how to improve the shadow defects in the endoscopic imaging image and at the same time suppress the overexposure of the edge of the image has become a technical problem to be solved in the existing technology. Summary of the Invention

[0005] Embodiments of the present application provide a lens shading correction method for endoscopic imaging, a lens shading correction device for endoscopic imaging, and an endoscope system, which help to improve shading defects in endoscopic imaging images and at the same time suppress edge overexposure of the image.

[0006] In one embodiment of the present application, a lens shading correction method for endoscopic imaging includes:

[0007] acquiring an image of a target imaged through an endoscope;

[0008] Determining, based on brightness information of the target image at each pixel position, parameter correction information for lens shading correction calibration parameters previously obtained based on an endoscope optical characteristic calibration, wherein the lens shading correction calibration parameters are used to represent calibration gains at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gains at each pixel position to suppress edge overexposure of the target image;

[0009] generating a lens shading correction revised parameter based on the lens shading correction calibration parameter and the parameter correction information, wherein the lens shading correction revised parameter is used to represent a target gain obtained after gain adjustment of the calibration gain at each pixel position;

[0010] The lens shading correction revision parameters are used to perform lens shading correction processing on the target image.

[0011] In some examples, optionally, the parameter correction information includes a gain correction factor at each pixel position; the parameter correction information for pre-calibrated lens shading correction calibration parameters is determined based on the brightness information of the target image at each pixel position, including: determining the gain correction factor associated with the brightness information at each pixel position of the target image with the target condition of suppressing edge overexposure of the target image, wherein the gain correction factor of each pixel position is used to implement gain adjustment of the calibration gain at that pixel position, and the target gain at each pixel position is determined based on the calibration gain and the gain correction factor at that pixel position.

[0012] In some examples, optionally, the target condition of suppressing edge overexposure of the target image is used to determine the gain correction factor associated with the brightness information of the target image at each pixel position, including: determining the gain correction factor at the pixel position based on the eccentricity of each pixel position compared to the center position of the image, and the brightness information of the target image at the pixel position; wherein the gain correction factor that meets the target condition has a monotonicity that is opposite to the eccentricity of any pixel position compared to the center position of the image, and the brightness information at the pixel position, and the target gain at each pixel position is determined based on the product of the calibration gain at the pixel position and the gain correction factor.

[0013] In some examples, optionally, determining the gain correction factor at the pixel position based on the eccentric distance of each pixel position compared to the image center position and the brightness information of the target image at the pixel position includes: determining the gain correction factor at the pixel position based on an inverse proportional function value with the eccentric distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

[0014] In some examples, optionally, determining the gain correction factor at the pixel position based on the eccentricity distance of each pixel position compared to the image center position and the brightness information of the target image at the pixel position includes: determining the gain correction factor at the pixel position based on an exponential function value with the eccentricity distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

[0015] In some examples, optionally, determining the gain correction factor at the pixel position based on the eccentric distance of each pixel position compared to the image center position and the brightness information of the target image at the pixel position includes: determining the gain correction factor at the pixel position based on a logarithmic function value with the eccentric distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

[0016] In some examples, optionally, the lens shading correction calibration parameters include a calibration correction coefficient at each pixel position, and the calibration gain at each pixel position is determined by the calibration correction coefficient at the pixel position; generating the lens shading correction revision parameters based on the lens shading correction calibration parameters and the parameter correction information includes: correcting the calibration correction coefficient at each pixel position using the parameter correction information; generating the lens shading correction revision parameters based on the correction result of the calibration correction coefficient at each pixel position; wherein the lens shading correction revision parameters include a target correction coefficient at each pixel position, the target correction coefficient at each pixel position is determined based on the correction result of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is determined by the target correction coefficient at the pixel position.

[0017] In some examples, optionally, the calibration correction coefficient at each pixel position is the sum of a basic value and an incremental value, wherein the incremental value of the calibration correction coefficient at each pixel position is used to characterize the calibration gain at the pixel position; the correcting the calibration correction coefficient at each pixel position using the parameter correction information includes: correcting the incremental value of the calibration correction coefficient at each pixel position using the parameter correction information; generating the lens shading correction revision parameter based on the correction result of the calibration correction coefficient at each pixel position includes: generating the lens shading correction revision parameter based on the correction result of the incremental value of the calibration correction coefficient at each pixel position, wherein the target correction coefficient at each pixel position is the sum of the basic value of the calibration correction coefficient at the pixel position and the correction result of the incremental value of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is characterized by the correction result of the incremental value of the calibration correction coefficient at the pixel position.

[0018] In some examples, optionally, the base value of the calibration correction coefficient at each pixel position is 1, and the numerical portion of the calibration correction coefficient at each pixel position that exceeds 1 is an incremental value used to characterize the calibration gain at the pixel position.

[0019] In another embodiment of the present application, a lens shading correction device for endoscopic imaging includes:

[0020] An image acquisition module, used for acquiring a target image formed by an endoscope;

[0021] a correction estimation module, configured to determine, based on brightness information at each pixel position of the target image, parameter correction information for lens shading correction calibration parameters previously obtained based on an endoscope optical characteristic calibration, wherein the lens shading correction calibration parameters are used to represent a calibration gain at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gain at each pixel position to suppress edge overexposure of the target image;

[0022] a correction execution module, configured to generate a lens shading correction revision parameter based on the lens shading correction calibration parameter and the parameter correction information, wherein the lens shading correction revision parameter is used to represent a target gain obtained after gain adjustment of the calibration gain at each pixel position;

[0023] The correction processing module is used to perform lens shading correction processing on the target image using the lens shading correction revision parameters.

[0024] In some examples, optionally, the parameter correction information includes a gain correction factor at each pixel position; the correction estimation module is specifically used to: determine the gain correction factor associated with the brightness information at each pixel position of the target image with the target condition of suppressing edge overexposure of the target image, wherein the gain correction factor of each pixel position is used to implement gain adjustment of the calibrated gain at the pixel position, and the target gain at each pixel position is determined based on the calibrated gain and the gain correction factor at the pixel position.

[0025] In some examples, optionally, the correction estimation module is specifically used to: determine the gain correction factor at the pixel position based on the eccentricity of each pixel position compared to the image center position and the brightness information of the target image at the pixel position; wherein the gain correction factor that meets the target condition has a monotonicity that is opposite to the eccentricity of any pixel position compared to the image center position and the brightness information at the pixel position, and the target gain at each pixel position is determined based on the product of the calibration gain at the pixel position and the gain correction factor.

[0026] In some examples, optionally, the correction estimation module is specifically used to determine the gain correction factor at any pixel position based on an inverse proportional function value with the eccentricity distance of any pixel position compared to the center position of the image and the brightness information at the pixel position as independent variables.

[0027] In some examples, optionally, the correction estimation module is specifically used to determine the gain correction factor at any pixel position based on an exponential function value with the eccentricity distance of any pixel position compared to the center position of the image and the brightness information at the pixel position as independent variables.

[0028] In some examples, optionally, the correction estimation module is specifically used to determine the gain correction factor at any pixel position based on a logarithmic function value with the eccentricity distance of any pixel position compared to the center position of the image and the brightness information at the pixel position as independent variables.

[0029] In some examples, optionally, the lens shading correction calibration parameters include a calibration correction coefficient at each pixel position, and the calibration gain at each pixel position is determined by the calibration correction coefficient at the pixel position; the correction execution module is specifically used to: use the parameter correction information to correct the calibration correction coefficient at each pixel position; based on the correction result of the calibration correction coefficient at each pixel position, generate the lens shading correction revision parameters; wherein, the lens shading correction revision parameters include a target correction coefficient at each pixel position, the target correction coefficient at each pixel position is determined based on the correction result of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is determined by the target correction coefficient at the pixel position.

[0030] In some examples, optionally, the calibration correction coefficient at each pixel position is the sum of a basic value and an incremental value, wherein the incremental value of the calibration correction coefficient at each pixel position is used to characterize the calibration gain at the pixel position; the correction execution module is specifically used to: use the parameter correction information to correct the incremental value of the calibration correction coefficient at each pixel position; the generating the lens shading correction revision parameter based on the correction result of the calibration correction coefficient at each pixel position includes: generating the lens shading correction revision parameter based on the correction result of the incremental value of the calibration correction coefficient at each pixel position, wherein the target correction coefficient at each pixel position is the sum of the basic value of the calibration correction coefficient at the pixel position and the correction result of the incremental value of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is determined by the correction result of the incremental value of the calibration correction coefficient at the pixel position.

[0031] In some examples, optionally, the base value of the calibration correction coefficient at each pixel position is 1, and the numerical portion of the calibration correction coefficient at each pixel position that exceeds 1 is an incremental value used to characterize the calibration gain at the pixel position.

[0032] In another embodiment of the present application, an endoscope system includes:

[0033] An endoscopic imaging component includes an endoscope and a camera, wherein the camera is used to generate an image of a target imaged through the endoscope;

[0034] An endoscope light source assembly is used to provide fill light brightness around the lens field of view of the endoscope;

[0035] The processor component is used to execute the lens shading correction method described in the above embodiment.

[0036] In another embodiment of the present application, a non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, enable the processor to perform the lens shading correction method described in the aforementioned embodiment.

[0037] Based on the above embodiment, the lens shading correction calibration parameters obtained by calibrating the optical characteristics of the endoscope can be corrected using the brightness information of each frame of the target image imaged by the endoscope to obtain lens shading correction revision parameters that are adapted to the optical characteristics of the endoscope and the actual brightness of the target image of the frame. Therefore, by using the lens shading correction revision parameters to perform lens shading correction processing on the target image, the shadow defects in the target image can be improved, and at the same time, edge overexposure of the target image can be suppressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The following drawings are only provided for schematic illustration and explanation of the present application and do not limit the scope of the present application:

[0039] Figure 1 Schematic diagram of the system architecture of an endoscope system in one embodiment of the present application;

[0040] Figure 2 For example Figure 1 A schematic diagram of the working principle of the endoscope system for lens shading correction in the illustrated embodiment;

[0041] Figure 3 1 is a flow chart of a lens shading correction method for endoscopic imaging according to another embodiment of the present application;

[0042] Figure 4 This is a schematic structural diagram of a lens shading correction device for endoscopic imaging in another embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of this application more clear, the application is further described in detail below with reference to the accompanying drawings and examples.

[0044] Figure 1 Schematic diagram of the system architecture of an endoscope system in one embodiment of the present application. Figure 1 In an embodiment of the present application, an endoscope system may include an endoscopic imaging component 10 , an endoscopic light source component 20 , and a host device 30 .

[0045] The endoscopic imaging assembly 10 includes an endoscope 11 and a camera 12. The endoscope 11 can be inserted into the body through a surgical channel. The camera 12 can be mounted at the distal end of the endoscope 11. While the endoscope 11 is inserted into the body, the camera 12 is configured to generate an image of a target imaged by the endoscope 11. For example, the camera 12 can include a photosensitive element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS).

[0046] The endoscopic light source assembly 20 is used to provide fill light brightness around the lens field of view of the endoscope 11. Therefore, the target image formed by the endoscope 11 may have a brightness distribution with a dark center and bright edges. For example, the endoscopic light source assembly 20 may include any cold light source, and the fill light beam generated by the endoscopic light source assembly 20 may be transmitted to the endoscope 11 via a light pipe.

[0047] The host device 30 includes a processor component 310, which can be used to control the endoscopic imaging component 10 and the endoscopic light source component 20. The processor component 310 can also be used to perform image processing on the target image imaged by the camera 12 through the endoscope 11. The image processing performed by the processor component 310 on the target image includes at least lens shading correction processing, and the image processed by the processor component 310 is used for visual presentation on the display device 40. For example, the processor component 310 may include electrical control elements such as a microcontroller unit (MCU), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), and may also include data processing elements with data processing functions such as a central processing unit (CPU), an image signal processor (ISP), and a graphics processing unit (GPU).

[0048] The host device 30 may further include a storage medium 320 and a memory 330. The storage medium 320 may store instructions for execution by the processor component 310, and the memory 330 may store at least intermediate data generated when the processor component 310 performs image processing. For example, the storage medium 320 may include a non-volatile storage medium such as flash memory, and the memory 330 may include a volatile storage medium such as random access memory (RAM) and its derivative storage media.

[0049] In an embodiment of the present application, in order to improve shadow defects in endoscopic imaging images, the storage medium 320 may also store pre-calibrated lens shading correction calibration parameters, and the lens shading correction calibration parameters are used to characterize the calibration gain at each pixel position.

[0050] For the calibration of the lens shading correction calibration parameters, a position-based lens shading correction calibration algorithm may be selected, or a sample image-based lens shading correction calibration algorithm may be selected.

[0051] For example, a position-based lens shading correction calibration algorithm may include a concentric circle method or a grid method. The concentric circle method can determine the eccentric distance R of each pixel position in the calibration image compared to the set image center, and use an interpolation function to determine the calibration correction coefficient GAIN at each pixel position that is proportional to the eccentric distance R.REF (i, j), where i represents the row coordinate of the pixel position and j represents the column coordinate of the pixel position; the grid rule is to divide the image area into a sparse grid in the middle and dense grid around it, and determine the grid coefficient GAIN in an increasing manner from the center of the image to the edge of the image REF (k) so that the calibration correction coefficient GAIN of the pixel position falling in each network REF (i,j) are all related to the grid coefficient GAIN of the grid REF (k) is the same, where k represents the identity of the grid.

[0052] Whether it is the concentric circle method or the grid method, the calibration correction coefficient GAIN REF (i, j) are determined based on the influence of the optical characteristics of the endoscope 11 on the target image, that is, the calibration correction coefficient GAIN REF The determination process of (i, j) does not introduce environmental factors related to the fill light environment inside the human body. The embodiment of this application is not intended to improve the position-based lens shading correction calibration algorithm. This is because: the tissue structure inside the human body is diverse, and the fill light beam's impact on the illumination within the field of view of the endoscope 11 in an environment with different tissue structures is uncertain. Therefore, it is difficult to determine a universal environmental parameter that can characterize all fill light environments inside the human body. In this case, the calibration correction coefficient GAIN in the lens shading correction calibration parameter is REF The calibration gain at each pixel position represented by (i, j) has a value distribution that is consistent with the bright center and dark edge of the image, that is, the value distribution increases in the direction from the center of the image to the edge of the image.

[0053] For another example, a lens shading correction calibration algorithm based on a sample image may use a calibration image obtained by imaging a monochrome plate such as a white plate through the endoscope 11, and the algorithm may use the pixel information at the center of the calibration image as the reference pixel information I. Cen , combined with the pixel information I at each pixel position pix (i, j) determines the calibration correction coefficient GAIN at the pixel position REF (i, j), see the following expression (1) for details.

[0054]

[0055] The reason for selecting a uniform lighting environment for the monochrome image rather than a fill-light environment simulating the human body is similar to that of the position-based lens shading correction calibration algorithm: the internal tissue structure of the human body is diverse, and the impact of the fill-light beam on the illumination within the field of view of endoscope 11 in environments with different tissue structures is uncertain, making it difficult to simulate a universal fill-light environment within the human body. In this case, the optical characteristics of endoscope 11 itself will cause the calibration image to have a brightness distribution with bright center and dark edges in a uniform lighting environment. As a result, the calibration gain at each pixel position represented by the lens shading correction calibration parameters still has a value distribution consistent with the bright center and dark edges of the image, that is, a value distribution that increases from the center of the image to the edge.

[0056] That is, in the embodiment of the present application, the lens shading correction calibration parameters are obtained based on the optical characteristics of the endoscope 11. This calibration process does not include factors related to the internal fill light environment of the human body. Furthermore, the lens shading correction calibration parameters can also have a parameter format that is compatible with different image formats of the target image 500.

[0057] Specifically, in the embodiments of the present application, the target image 500 may be a grayscale image that does not contain color information, or may be a color image that contains color information, for example, a three-primary color (Red Green Blue, RGB) image or a chromaticity (YUV) image, and the embodiments of the present application are not limited to this.

[0058] If the target image 500 is a grayscale image, the brightness information L(i, j) of the target image 500 at each pixel position can be represented by the grayscale value at each pixel position, and the calibration correction coefficient GAIN at each pixel position is REF (i, j) and target correction coefficient GAIN OBJ (i, j) can be a single-value correction coefficient for the grayscale value;

[0059] If the target image 500 is an RGB image, the brightness information L(i, j) of the target image 500 at each pixel position can be represented by the color information of the three channels R, G, and B at each pixel position, and the calibration correction coefficient GAIN at each pixel position is REF (i, j) and target correction coefficient GAIN OBJ (i, j) may be a coefficient group including correction coefficients for each channel respectively;

[0060] If the target image 500 is a YUV image, the brightness information L(i, j) of the target image 500 at each pixel position can be represented by the Y channel information at each pixel position, and the calibration correction coefficient GAIN at each pixel position is REF(i, j) and target correction coefficient GAIN OBJ (i, j) may be a single-valued correction coefficient for the Y channel information.

[0061] In an embodiment of the present application, since the value distribution of the calibration gain at each pixel position represented by the lens shading correction calibration parameter is applicable to images with bright center and dark edges, the lens shading correction calibration parameter stored in the storage medium 320 can be called and obtained by the processor component 310. However, in order to suppress edge overexposure of the target image with dark center and bright edges, the processor component 310 does not directly use the lens shading correction calibration parameter to perform lens shading correction processing on the target image, but instead corrects the lens shading correction calibration parameter based on the brightness information of the target image, and uses the corrected lens shading correction revised parameter to perform lens shading correction processing on the target image.

[0062] The processor component 310 may modify the lens shading correction calibration parameters once for each target image frame, and the modification process performed on each target image frame may be based on the brightness information at each pixel position in the target image frame. That is, the lens shading correction revised parameters obtained by modifying the lens shading correction calibration parameters for each target image frame further incorporate the brightness information of the target image frame based on the optical characteristics of the endoscope 11. In other words, the lens shading correction revised parameters are parameters adapted to the optical characteristics of the endoscope 11 and the target image of the corresponding frame, rather than being universal parameters for all target images. Therefore, the lens shading correction revised parameters and the parameter modification information used to modify the lens shading correction revised parameters may be stored in the memory 330 as intermediate data when the processor component 310 performs image processing.

[0063] Figure 2 For example Figure 1 Schematic diagram of the working principle of the endoscope system for lens shading correction in the embodiment shown. Figure 2 In an embodiment of the present application, the processor component 310 may also be used to:

[0064] Acquire a target image 500 formed by the endoscope 11;

[0065] Based on the brightness information L(i, j) of the target image 500 at each pixel position, parameter correction information 550 is determined for the lens shading correction calibration parameter 510 obtained in advance based on the optical characteristics calibration of the endoscope 11, wherein the lens shading correction calibration parameter 510 is used to represent the calibration gain ΔGAIN at each pixel position. REF(i, j), and the parameter correction information 550 is used to implement gain adjustment on the calibration gain at each pixel position to suppress edge overexposure of the target image 500;

[0066] Based on the lens shading correction calibration parameters 510 and the parameter correction information 550, the lens shading correction revised parameters 520 are generated, wherein the lens shading correction revised parameters 520 are used to represent the calibration gain ΔGAIN at each pixel position. REF (i, j) The target gain ΔGAIN obtained after gain adjustment OBJ (i,j);

[0067] The lens shading correction revision parameters 520 are used to perform lens shading correction processing on the target image 500 , wherein the corrected image obtained by performing the lens shading correction processing on the target image 500 can be used for visual presentation on the display device 40 .

[0068] Based on the above-mentioned endoscope system, the lens shading correction calibration parameters 510 can be obtained in advance based on the optical characteristics calibration of the endoscope 11, and for each frame of the target image 500 obtained by imaging through the endoscope 11, the lens shading correction calibration parameters 510 can be corrected using the brightness information L(i, j) of the target image 500 at each pixel position to obtain a lens shading correction revision parameter 520 that is adapted to the optical characteristics of the endoscope 11 and the actual brightness of the target image 500. Therefore, by using the lens shading correction revision parameter 520 to perform lens shading correction processing on the target image 500, the shadow defects in the target image 500 can be improved, and the edge overexposure of the target image 500 can be suppressed at the same time.

[0069] In the embodiment of the present application, as described above, the lens shading correction calibration parameter 510 may include the calibration correction coefficient GAIN at each pixel position. REF (i,j), then:

[0070] Calibration gain ΔGAIN at each pixel position REF (i, j) can be obtained by the calibration correction coefficient GAIN at the pixel position REF (i, j) determines, for example, the calibration correction coefficient GAIN at each pixel location REF (i, j) can be the sum of the basic value and the incremental value, wherein the basic value is preferably 1, and the incremental value can be considered as the calibration correction coefficient GAIN REF For the numerical portion of (i, j) exceeding 1, see the following expression (2).

[0071] ΔGAIN REF (i,j)=GAIN REF(i,j)-1 expression (2)

[0072] Moreover, no matter how the base value and the incremental value are divided, the calibration correction coefficient GAIN at each pixel position REF The incremental value of (i, j) can be used to represent the calibration gain ΔGAIN at the pixel position. REF (i,j).

[0073] Thus, the processor component 310 can use the parameter correction information 550 to correct the calibration correction coefficient GAIN at each pixel position in the lens shading correction calibration parameter 510. REF (i, j) to implement the calibration gain ΔGAIN at each pixel position REF (i, j) gain adjustment. As shown in the above expression (2), the calibration gain ΔGAIN REF (i,j) can be regarded as the calibration correction coefficient GAIN REF (i, j) is the part that actually generates the gain. Therefore, the processor component 310 uses the parameter correction information 550 to calibrate the correction coefficient GAIN REF The correction of (i,j) can be the calibration correction coefficient GAIN REF (i, j) is used to represent the calibration gain ΔGAIN REF The correction of the incremental value of (i, j). Compared with the calibration correction coefficient GAIN REF The overall value of (i, j) is corrected by correcting the calibration coefficient GAIN REF The gain adjustment implemented by the incremental value in (i, j) can have higher adjustment accuracy.

[0074] Furthermore, the processor component 310 can be based on the calibration correction coefficient GAIN at each pixel position. REF The correction result of (i, j) (for example, the calibration correction coefficient GAIN at each pixel position) REF (i, j) is used to represent the calibration gain ΔGAIN REF (i, j)) to generate the lens shading correction revised parameters 520, where the calibration correction coefficient GAIN at each pixel position is REF The correction result of (i, j) (for example, the calibration correction coefficient GAIN REF (i, j) is used to represent the calibration gain ΔGAIN REF The correction result of the incremental value of (i, j) is used to represent the target gain ΔGAIN at the pixel position OBJ (i,j).

[0075] Based on the above processing method, the lens shading correction revised parameter 520 may include the target correction coefficient GAIN at each pixel position. OBJ (i, j), the target correction coefficient GAIN at each pixel position OBJ (i, j) can be based on the calibration correction coefficient GAIN at the pixel position REF The correction result of (i, j) is determined, and the target gain ΔGAIN at each pixel position is OBJ (i, j) is the target correction coefficient GAIN at the pixel position OBJ (i, j) is determined. For example, the target correction coefficient GAIN at each pixel position OBJ (i, j) can be the calibration correction coefficient GAIN of the pixel position REF The base value of (i, j) (preferably 1) and the correction result of the incremental value of the calibration correction coefficient at the pixel position (i.e. the target gain ΔGAIN OBJ (i, j)), see the following expression (3) taking the basic value preferably as 1 as an example.

[0076] GAIN OBJ (i, j)=ΔGAIN OBJ (i,j)+1 expression (3)

[0077] In the embodiment of the present application, in order to facilitate the calibration correction coefficient GAIN in the lens shading correction calibration parameter 510 REF (i, j) and the target correction coefficient GAIN in the lens shading correction revised parameter 520 oBJ (i, j) for the bitmap conversion, the parameter correction information 550 may include the gain correction factor ρ(i, j) at each pixel position, for example, the target gain ΔGAIN at each pixel position OBJ (i, j) is based on the calibration gain ΔGAIN at the pixel position REF It is determined by multiplying (i, j) by the gain correction factor ρ(i, j), as shown in the following expression (4).

[0078] ΔGAIN OBJ (i, j) = ρ(i, j) × ΔGAIN REF (i, j) expression (4)

[0079] Furthermore, by substituting Expression (4) into Expression (3) mentioned above, the following Expression (5) can be obtained.

[0080] GAIN OBJ (i, j) = ρ(i, j) × ΔGAIN REF (i, j)+1 expression (5)

[0081] In the case where the parameter correction information 550 includes the gain correction factor ρ(i, j) at each pixel position, the processor component 310 may be specifically configured to determine the parameter correction information 550 based on the brightness information L(i, j) at each pixel position of the target image 500 as follows:

[0082] With the target condition of suppressing edge overexposure of the target image 500, the gain correction factor p(i, j) associated with the brightness information L(i, j) at each pixel position of the target image 500 is determined, wherein the gain correction factor ρ(i, j) at each pixel position is used to implement gain adjustment of the calibration gain ΔGAINREF(i, j) at the pixel position, for example, by correcting the calibration correction coefficient GAINREF(i, j) to implement gain adjustment of the calibration gain ΔGAINREF(i, j), and the target gain ΔGAINoBJ(i, j) at each pixel position is determined based on the calibration gain at the pixel position and the gain correction factor ρ(i, j), for example, the target gain at each pixel position can be determined by the target correction coefficient GAINoBJ(i, j) at the pixel position.

[0083] Specifically, the target condition can be associated with the eccentric distance D(i, j) of each pixel position compared to the image center position, and the brightness information L(i, j) of the target image 500 at the pixel position. If the target gain ΔGAINoBJ(i, j) at each pixel position is determined based on the product of the calibration gain ΔGAINREF(i, j) at the pixel position and the gain correction factor ρ(i, j), then the constraint effect of the target condition can be such that:

[0084] The larger the eccentricity distance D(i, j) of each pixel position compared to the image center position, and / or the larger the brightness information L(i, j) of the target image 500 at the pixel position, the smaller the gain correction factor ρ(i, j) is, so that the calibration gain at the bright pixel position close to the edge can be reduced, thereby avoiding edge overexposure;

[0085] The smaller the eccentric distance D(i, j) of each pixel position compared to the center position of the image, and / or the smaller the brightness information L(i, j) of the target image 500 at the pixel position, the larger the gain correction factor ρ(i, j), so that the calibration gain at the low-brightness pixel position close to the center part can remain unchanged or further increase, thereby ensuring the correction effect of shadow defects.

[0086] In this case, when determining the gain correction factor ρ(i, j) associated with the brightness information L(i, j) at each pixel position of the target image 500, the processor component 310 may be specifically configured as follows:

[0087] Determine a gain correction factor ρ(i, j) at each pixel position based on an eccentricity distance D(i, j) of each pixel position relative to the image center position and brightness information L(i, j) of the target image 500 at the pixel position;

[0088] Among them, the gain correction factor ρ(i, j) that meets the above-mentioned target conditions has the monotonicity that is opposite to the eccentricity distance D(i, j) of any pixel position compared to the image center position and the brightness information L(i, j) at the pixel position.

[0089] That is, the gain correction factor ρ(i, j) associated with the luminance information L(i, j) at each pixel position of the target image 500 can be expressed as the following expression (6).

[0090]

[0091] Among them, F(*) in expression (6) can be regarded as an objective function with the eccentric distance D(i, j) and the brightness information L(i, j) as input variables, and the objective function can be constrained by the aforementioned objective conditions. In addition, in expression (3), the eccentric distance D(i, j) and the brightness information L(i, j) are expressed in reciprocal form, which is intended to express that the gain correction factor ρ(i, j) as the output value is opposite to the monotonicity of these two input variables, rather than intending to limit the eccentric distance D(i, j) and the brightness information L(i, j) to exist in the reciprocal form in the objective function.

[0092] For example, the above-mentioned expression (6) can be concretely expressed as the following expression (7).

[0093]

[0094] Based on the meaning of expression (7), it can be considered that the processor component 300 determines the gain correction factor ρ(i, j) at the pixel position based on the inverse proportional function value with the eccentric distance D(i, j) of any pixel position compared to the image center position and the brightness information L(i, j) of the target image 500 at the pixel position as independent variables, wherein m in expression (3) is a pre-set non-zero inverse proportional coefficient.

[0095] For another example, the above expression (6) can also be concretely expressed as the following expression (8).

[0096] ρ(i, j) = e -m×L(i,j)×D(i，j) Expression (8)

[0097] Based on the meaning of expression (8), it can be considered that the processor component 300 determines the gain correction factor ρ(i, j) at the pixel position based on the exponential function value with the eccentric distance D(i, j) of any pixel position compared to the image center position and the brightness information L(i, j) of the target image 500 at the pixel position as independent variables, where m in expression (4) is a pre-set non-zero adjustment factor.

[0098] For another example, the above expression (6) can also be concretely expressed as the following expression (9).

[0099] ρ(i, j) = log(-m × L(i, j) × D(i, j) + n) Expression (9)

[0100] Based on the meaning of expression (9), it can be considered that the processor component 300 determines the gain correction factor ρ(i, j) at the pixel position based on the logarithmic function value with the eccentric distance D(i, j) of any pixel position compared to the image center position and the brightness information L(i, j) of the target image 500 at the pixel position as independent variables, wherein m and n in expression (5) are pre-set non-zero adjustment factors, and n≤10.

[0101] In addition, in the embodiment of the present application, if the lens shading correction revision parameter 520 includes the target correction coefficient GAIN at each pixel position OBJ (i, j), then, when the processor component 310 uses the lens shading correction revision parameter 520 to perform lens shading correction on the target image 500, it can specifically use the target correction coefficient GAINoBJ(i, j) at each pixel position in the lens shading correction revision parameter 520 to correct the brightness information L(i, j) of the target image 500 at the pixel position. For example, the corrected brightness L'(i, j) of the brightness information L(i, j) at each pixel position after correction can be determined based on the product of the brightness information L(i, j) at the pixel position and the target correction coefficient GAINoBJ(i, j), see the following expression (10).

[0102] L′(i, j)=L(i, j)×GAIN OBJ (i, j) expression (10)

[0103] The processing process of the processor component 310 in the above embodiment can be regarded as a lens shading correction method for endoscopic imaging, and the lens shading correction method can also be applied to other execution entities besides the processor component 310 in the endoscope system.

[0104] Figure 3FIG2 is a flow chart of a lens shading correction method for endoscopic imaging in another embodiment of the present application. Figure 3 In an embodiment of the present application, a lens shading correction method for endoscopic imaging may include:

[0105] S310: Acquire a target image formed through an endoscope.

[0106] S330: Based on the brightness information of the target image at each pixel position, determine the parameter correction information of the lens shading correction calibration parameters obtained in advance based on the optical characteristics calibration of the endoscope, wherein the lens shading correction calibration parameters are used to characterize the calibration gain at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gain at each pixel position with the goal of suppressing edge overexposure of the target image.

[0107] S350: Generate lens shading correction revision parameters based on the lens shading correction calibration parameters and the parameter correction information, wherein the lens shading correction revision parameters are used to represent the target gain obtained after the calibration gain at each pixel position is gain-adjusted.

[0108] S370: Perform lens shading correction processing on the target image using the lens shading correction revision parameters.

[0109] Based on the above process, the lens shading correction calibration parameters obtained by calibrating the optical characteristics of the endoscope can be corrected using the brightness information of each frame of the target image imaged by the endoscope to obtain lens shading correction revision parameters that are adapted to the optical characteristics of the endoscope and the actual brightness of the target image of the frame. Therefore, the lens shading correction revision parameters can be used to perform lens shading correction processing on the target image, which can not only improve the shadow defects in the target image, but also simultaneously suppress edge overexposure of the target image.

[0110] In the embodiments of the present application, Figure 3 The lens shading correction calibration parameters used in the lens shading correction method shown may include calibration correction coefficients at each pixel position. In this case, the parameter correction information determined in S330 may include a gain correction factor at each pixel position, and S330 may specifically include:

[0111] With the goal of suppressing edge overexposure of a target image, a gain correction factor associated with brightness information at each pixel position of the target image is determined, wherein the gain correction factor at each pixel position is used to implement gain adjustment of the calibration gain at that pixel position, and the target gain at each pixel position is determined based on the calibration gain at that pixel position and the gain correction factor.

[0112] As a preferred embodiment, the process of determining the gain correction factor associated with the brightness information at each pixel position of the target image in S330 may specifically include:

[0113] Determining a gain correction factor at each pixel position based on an eccentricity distance of each pixel position compared to an image center position and brightness information of the target image at the pixel position;

[0114] Wherein, referring to expression (6) described above, the gain correction factor that meets the target condition has a monotonicity that is opposite to the eccentric distance of any pixel position compared to the image center position and the brightness information at the pixel position, and, referring to expression (4) described above, the target gain at each pixel position represented by the lens shading correction revision parameter obtained in S350 can be determined based on the product of the calibration gain at the pixel position and the gain correction factor.

[0115] For example, as described in Expression (7) above, S330 can determine the gain correction factor at the pixel position based on the inverse proportional function value with the eccentricity distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

[0116] For another example, as described in Expression (8) above, S330 can determine the gain correction factor at the pixel position based on the exponential function value with the eccentricity distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

[0117] For another example, as described in expression (9) above, S330 can determine the gain correction factor at the pixel position based on the logarithmic function value with the eccentric distance of any pixel position compared to the center position of the image and the brightness information at the pixel position as independent variables.

[0118] In the embodiments of the present application, if Figure 3 The lens shading correction calibration parameters used in the lens shading correction method shown may include calibration correction coefficients at each pixel position. Then, S350 may specifically include:

[0119] The calibration correction coefficient at each pixel position is corrected using the parameter correction information. For example, if the relationship between the calibration gain and the calibration correction coefficient at each pixel position is as shown in Expression (2) described above, that is, the calibration correction coefficient at each pixel position is the sum of the base value and the incremental value, and the incremental value of the calibration correction coefficient at each pixel position is used to represent the calibration gain at the pixel position, then the incremental value of the calibration correction coefficient at each pixel position is corrected using the parameter correction information.

[0120] Based on the correction results of the calibration correction coefficients at each pixel position, the lens shading correction revision parameters are generated. For example, if the calibration correction coefficient at each pixel position is the sum of the basic value and the incremental value, then the lens shading correction revision parameters are generated based on the correction results of the incremental value of the calibration correction coefficients at each pixel position.

[0121] The lens shading correction revision parameters include a target correction coefficient at each pixel position, the target correction coefficient at each pixel position being determined based on a correction result of the calibration correction coefficient at that pixel position, and the correction result of the target correction coefficient at each pixel position being used to represent the target gain at that pixel position. For example, if, as in Expression (3) or (5) described above, the target correction coefficient at each pixel position is the sum of the base value of the calibration correction coefficient at that pixel position and the correction result of the incremental value of the calibration correction coefficient at that pixel position, then the target gain at each pixel position is represented by the correction result of the incremental value of the calibration correction coefficient at that pixel position.

[0122] In addition, if the lens shading correction revision parameters include a target correction coefficient at each pixel position, then S370 may specifically include: using the target correction coefficient at each pixel position in the lens shading correction revision parameters to correct the brightness information of the target image at that pixel position. For example, as described in expression (10) above, the corrected brightness of the brightness information at each pixel position can be determined based on the product of the brightness information at that pixel position and the target correction coefficient.

[0123] Figure 4 This is a schematic diagram of the structure of a lens shading correction device for endoscopic imaging in another embodiment of the present application. Figure 4 In an embodiment of the present application, a lens shading correction device for endoscopic imaging may include:

[0124] The image acquisition module 410 is used to acquire a target image formed by an endoscope.

[0125] The correction estimation module 430 is used to determine parameter correction information for the lens shading correction calibration parameters obtained in advance based on the optical characteristics calibration of the endoscope based on the brightness information of the target image at each pixel position, wherein the lens shading correction calibration parameters are used to characterize the calibration gain at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gain at each pixel position with the goal of suppressing edge overexposure of the target image.

[0126] The correction execution module 450 is used to generate lens shading correction revision parameters based on the lens shading correction calibration parameters and the parameter correction information, wherein the lens shading correction revision parameters are used to represent the target gain obtained after the calibration gain at each pixel position is gain-adjusted.

[0127] The correction processing module 470 is used to perform lens shading correction processing on the target image using the lens shading correction revision parameters.

[0128] Based on the above-mentioned device, the lens shading correction calibration parameters obtained by calibrating the optical characteristics of the endoscope can be corrected using the brightness information of each frame of the target image imaged by the endoscope to obtain lens shading correction revision parameters that are adapted to the optical characteristics of the endoscope and the actual brightness of the target image of the frame. Therefore, by using the lens shading correction revision parameters to perform lens shading correction processing on the target image, the shadow defects in the target image can be improved, and at the same time, edge overexposure of the target image can be suppressed.

[0129] For the specific forms of the lens shading correction calibration parameters, parameter correction information and lens shading correction revision parameters, as well as the specific functions of the correction estimation module 430, the correction execution module 450 and the correction processing module 470, please refer to the previous description of S330, S350 and S370 in the lens shading correction, which will not be repeated here.

[0130] In another embodiment of the present application, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores instructions. When the instructions are executed by a processor, the processor executes the lens shading correction method described above.

[0131] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A lens shading correction method for endoscopic imaging, characterized in that: include: acquiring an image of a target imaged through an endoscope; Determining, based on brightness information of the target image at each pixel position, parameter correction information for lens shading correction calibration parameters previously obtained based on an optical characteristic calibration of an endoscope, wherein the lens shading correction calibration parameters are used to characterize calibration gains at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gains at each pixel position to suppress edge overexposure of the target image, and the parameter correction information includes a gain correction factor at each pixel position; generating a lens shading correction revised parameter based on the lens shading correction calibration parameter and the parameter correction information, wherein the lens shading correction revised parameter is used to represent a target gain obtained after gain adjustment of the calibration gain at each pixel position; Performing lens shading correction processing on the target image using the lens shading correction revision parameters; in: Determining parameter correction information for pre-calibrated lens shading correction calibration parameters based on brightness information of the target image at each pixel position includes: determining a gain correction factor associated with the brightness information at each pixel position of the target image, with suppressing edge overexposure of the target image as a target condition, wherein the gain correction factor at each pixel position is used to implement gain adjustment of the calibration gain at the pixel position, and the target gain at each pixel position is determined based on the calibration gain at the pixel position and the gain correction factor; The method of determining a gain correction factor associated with brightness information at each pixel position of the target image with the target condition of suppressing edge overexposure of the target image includes: determining the gain correction factor at each pixel position based on the eccentricity of each pixel position compared to the image center position and the brightness information of the target image at the pixel position; wherein the gain correction factor that meets the target condition has a monotonicity that is opposite to the eccentricity of any pixel position compared to the image center position and the brightness information at the pixel position, and the target gain at each pixel position is determined based on the product of a calibration gain at the pixel position and the gain correction factor.

2. The lens shading correction method according to claim 1, wherein: The step of determining a gain correction factor at each pixel position based on an eccentric distance of each pixel position compared to an image center position and brightness information of the target image at the pixel position includes: The gain correction factor at any pixel position is determined based on an inverse proportional function value having an eccentric distance of the pixel position from the image center position and brightness information at the pixel position as independent variables.

3. The lens shading correction method according to claim 1, wherein: The step of determining a gain correction factor at each pixel position based on an eccentric distance of each pixel position compared to an image center position and brightness information of the target image at the pixel position includes: The gain correction factor at any pixel position is determined based on an exponential function value with the eccentricity distance of the pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

4. The lens shading correction method according to claim 1, wherein: The step of determining a gain correction factor at each pixel position based on an eccentric distance of each pixel position compared to an image center position and brightness information of the target image at the pixel position includes: The gain correction factor at any pixel position is determined based on a logarithmic function value with the eccentricity distance of any pixel position compared to the image center position and the brightness information at the pixel position as independent variables.

5. The lens shading correction method according to claim 1, wherein: The lens shading correction calibration parameters include a calibration correction coefficient at each pixel position, and the calibration gain at each pixel position is determined by the calibration correction coefficient at the pixel position; The generating of lens shading correction revised parameters based on the lens shading correction calibration parameters and the parameter correction information includes: Correcting the calibration correction coefficient at each pixel position using the parameter correction information; Generating the lens shading correction revised parameters based on the correction results of the calibration correction coefficients at each pixel position; Among them, the lens shading correction revision parameters include a target correction coefficient at each pixel position, the target correction coefficient at each pixel position is determined based on the correction result of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is determined by the target correction coefficient at the pixel position.

6. The lens shading correction method according to claim 5, wherein: The calibration correction coefficient at each pixel position is the sum of the basic value and the incremental value, wherein the incremental value of the calibration correction coefficient at each pixel position is used to represent the calibration gain at the pixel position; The correcting the calibration correction coefficient at each pixel position by using the parameter correction information includes: correcting the incremental value of the calibration correction coefficient at each pixel position by using the parameter correction information; The method of generating the lens shading correction revision parameters based on the correction results of the calibration correction coefficients at each pixel position includes: generating the lens shading correction revision parameters based on the correction results of the incremental values ​​of the calibration correction coefficients at each pixel position, wherein the target correction coefficient at each pixel position is the sum of the base value of the calibration correction coefficient at the pixel position and the correction result of the incremental value of the calibration correction coefficient at the pixel position, and the target gain at each pixel position is represented by the correction result of the incremental value of the calibration correction coefficient at the pixel position.

7. The lens shading correction method according to claim 6, wherein: The base value of the calibration correction coefficient at each pixel position is 1, and the portion of the calibration correction coefficient at each pixel position that exceeds 1 is an incremental value used to characterize the calibration gain at the pixel position.

8. A lens shading correction device for endoscopic imaging, characterized in that: include: An image acquisition module, used for acquiring a target image formed by an endoscope; a correction estimation module, configured to determine, based on brightness information of the target image at each pixel position, parameter correction information for lens shading correction calibration parameters previously obtained based on an endoscope optical characteristic calibration, wherein the lens shading correction calibration parameters are used to represent calibration gains at each pixel position, and the parameter correction information is used to implement gain adjustment on the calibration gains at each pixel position to suppress edge overexposure of the target image, and the parameter correction information includes a gain correction factor at each pixel position; a correction execution module, configured to generate a lens shading correction revision parameter based on the lens shading correction calibration parameter and the parameter correction information, wherein the lens shading correction revision parameter is used to represent a target gain obtained after gain adjustment of the calibration gain at each pixel position; a correction processing module, configured to perform lens shading correction processing on the target image using the lens shading correction revision parameters; in: The correction estimation module is specifically configured to: determine a gain correction factor associated with brightness information at each pixel position of the target image, with suppressing edge overexposure of the target image as a target condition, wherein the gain correction factor at each pixel position is used to implement gain adjustment of a calibration gain at the pixel position, and the target gain at each pixel position is determined based on the calibration gain at the pixel position and the gain correction factor; The correction estimation module is specifically used to determine the gain correction factor at each pixel position based on the eccentricity of each pixel position compared to the image center position and the brightness information of the target image at the pixel position; wherein the gain correction factor that meets the target condition has a monotonicity that is opposite to the eccentricity of any pixel position compared to the image center position and the brightness information at the pixel position, and the target gain at each pixel position is determined based on the product of the calibration gain at the pixel position and the gain correction factor.

9. An endoscope system, characterized in that: include: An endoscopic imaging component includes an endoscope and a camera, wherein the camera is used to generate an image of a target imaged through the endoscope; An endoscope light source assembly is used to provide fill light brightness around the lens field of view of the endoscope; A processor component, configured to execute the lens shading correction method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform the lens shading correction method according to any one of claims 1 to 7.

Images