Imaging picture shadow correction method and device, camera host and endoscope equipment

Abstract

The invention relates to the technical field of image processing, and discloses an imaging picture shadow correction method and device, a camera host and endoscope equipment, and the method comprises the steps: obtaining a correction parameter corresponding to an optical sight glass, and determining the correction parameter according to an optical center corresponding to the optical sight glass and a corresponding preset brightness attenuation model, the optical center and the preset brightness attenuation model are obtained by performing brightness calibration on the optical sight glass; after an imaging picture is collected through the optical sight glass, shadow correction is conducted on the imaging picture through the correction parameters; and displaying the imaging picture after shadow correction. Compared with an existing method adopting fixed correction parameters, the method can perform shadow correction after performing brightness calibration on the optical sight glass to obtain the corresponding correction parameters, and the correction effect is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, camera host and endoscope for correcting shadows in an image. Background Technology

[0002] As a medical device used for diagnosis and treatment, the quality of an endoscope's imaging directly affects the accuracy of a doctor's diagnosis and the safety of surgery. Endoscopic equipment typically supports the replacement of different optical endoscopes to meet diverse clinical needs. Because different optical endoscopes have different optical design parameters and bayonet optical zoom magnification, their imaging characteristics vary considerably, particularly in terms of uneven image brightness distribution, with higher brightness in the central area and obvious vignetting (shadowing) at the edges. Therefore, shadow correction is generally required after changing the optical endoscope.

[0003] In current methods of shadow correction, fixed correction parameters are generally used for the captured image. However, since different optical lenses have different optical design parameters and mount optical zoom magnification, using a fixed correction parameter for all corrections will result in poor correction results. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, camera host, and endoscope for correcting shadows in an image, aiming to solve the technical problem that existing methods for correcting shadows in an image generally use fixed correction parameters, resulting in poor correction effects.

[0005] To achieve the above objectives, this application provides a method for shadow correction in an image, the method comprising: Obtain the correction parameters corresponding to the optical sight mirror. The correction parameters are determined based on the optical center of the optical sight mirror and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are both obtained by calibrating the brightness of the optical sight mirror. After the image is acquired through the optical lens, the image is shaded using the correction parameters. Display the image after shadow correction.

[0006] In one embodiment, before the step of obtaining the correction parameters corresponding to the optical sight lens, the method further includes: Acquire a calibration image, which is an image of a preset white balance object captured by the optical lens; The optical center corresponding to the optical sight is obtained based on the calibration image, and a preset brightness attenuation model corresponding to the optical sight is obtained based on the optical center and the calibration image. The correction parameters corresponding to the optical lens are determined based on the optical center and the preset brightness attenuation model.

[0007] In one embodiment, the step of obtaining the optical center corresponding to the optical lens based on the calibration image includes: Obtain the grayscale image corresponding to the calibration image, and determine the total brightness of pixels in each row and the total brightness of pixels in each column of the grayscale image; A row pixel brightness curve is constructed based on the sum of the pixel brightness of each row, and a column pixel brightness curve is constructed based on the sum of the pixel brightness of each column. The row pixel brightness curves and column pixel brightness curves are filtered respectively, and the filtered row pixel brightness curves and column pixel brightness curves are fitted respectively. The row brightness peak is determined based on the fitted row pixel brightness curve, the column brightness peak is determined based on the fitted column pixel brightness curve, and the optical center corresponding to the optical lens is obtained based on the row brightness peak and the column brightness peak.

[0008] In one embodiment, the step of obtaining a preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image includes: Determine the relative distance between the optical center and each pixel in the calibration image; Obtain the pixel grayscale value of each pixel in the calibration image; Based on the relative distances and pixel gray values, the initial brightness attenuation model is nonlinearly fitted to obtain the calibration parameters corresponding to the optical lens. A preset brightness attenuation model is obtained based on the calibration parameters and the initial brightness attenuation model.

[0009] In one embodiment, the step of performing nonlinear fitting on the initial brightness attenuation model based on each of the relative distances and each of the pixel grayscale values ​​to obtain the calibration parameters corresponding to the optical lens includes: The gray values ​​of each pixel are filtered according to a preset gray value range to obtain valid pixel gray values; The calibration parameters corresponding to the optical lens are obtained by performing nonlinear fitting on the initial brightness attenuation model based on the relative distances and the effective pixel gray values.

[0010] In one embodiment, the step of determining the correction parameters corresponding to the optical lens based on the optical center and the preset brightness attenuation model includes: Substitute each of the relative distances into the aforementioned preset brightness attenuation model to obtain the pixel grayscale value of each pixel; The pixel gray value of the optical center is determined from the pixel gray values ​​of each of the aforementioned pixels; The correction coefficient of each pixel is obtained based on the pixel gray value of the optical center and the pixel gray value of each pixel; The correction parameters corresponding to the optical lens are obtained based on the correction coefficients of each pixel.

[0011] In one embodiment, the step of displaying the shadow-corrected image includes: Obtain the image bit depth of the image and determine the truncation limit range based on the image bit depth; The shadow-corrected image is truncated based on the truncation limitation range; Display the image after truncation and restriction.

[0012] Furthermore, to achieve the above objectives, this application also proposes an image shadow correction device, the device comprising: The parameter acquisition module is used to acquire the correction parameters corresponding to the optical sight lens. The correction parameters are determined based on the optical center of the optical sight lens and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are both obtained by calibrating the brightness of the optical sight lens. The shadow correction module is used to correct the shadows of the image captured by the optical sight lens using the correction parameters. The image display module is used to display the image after shadow correction.

[0013] In one embodiment, the parameter acquisition module is further configured to acquire a calibration image, which is an image obtained by taking a picture of a preset white balance object through the optical lens; obtain the optical center corresponding to the optical lens based on the calibration image, and obtain a preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image; and determine the correction parameters corresponding to the optical lens according to the optical center and the preset brightness attenuation model.

[0014] In one embodiment, the parameter acquisition module is further configured to acquire a grayscale image corresponding to the calibration image, and determine the sum of pixel brightness in each row and the sum of pixel brightness in each column of the grayscale image; construct a row pixel brightness curve based on the sum of pixel brightness in each row, and construct a column pixel brightness curve based on the sum of pixel brightness in each column; filter the row pixel brightness curve and the column pixel brightness curve respectively, and fit the filtered row pixel brightness curve and the filtered column pixel brightness curve respectively; determine the row brightness peak value based on the fitted row pixel brightness curve, determine the column brightness peak value based on the fitted column pixel brightness curve, and obtain the optical center corresponding to the optical lens based on the row brightness peak value and the column brightness peak value.

[0015] In one embodiment, the parameter acquisition module is further configured to determine the relative distance between the optical center and each pixel in the calibration image; acquire the pixel grayscale value of each pixel in the calibration image; perform nonlinear fitting on the initial brightness attenuation model based on each relative distance and each pixel grayscale value to obtain the calibration parameters corresponding to the optical lens; and obtain a preset brightness attenuation model based on the calibration parameters and the initial brightness attenuation model.

[0016] In one embodiment, the parameter acquisition module is further configured to filter the gray values ​​of each pixel according to a preset gray value range to obtain effective pixel gray values; and to perform nonlinear fitting on the initial brightness attenuation model based on the relative distances and the effective pixel gray values ​​to obtain the calibration parameters corresponding to the optical lens.

[0017] In one embodiment, the parameter acquisition module is further configured to substitute each of the relative distances into the aforementioned preset brightness attenuation model to obtain the pixel grayscale value of each of the pixels; determine the pixel grayscale value of the optical center from the pixel grayscale values ​​of each of the pixels; obtain the correction coefficient of each of the pixels based on the pixel grayscale value of the optical center and the pixel grayscale values ​​of each of the pixels; and obtain the correction parameter corresponding to the optical lens based on the correction coefficient of each of the pixels.

[0018] In one embodiment, the image display module is further configured to acquire the image bit depth of the image and determine a truncation limit range based on the image bit depth; truncate the shadow-corrected image based on the truncation limit range; and display the truncation-limited image.

[0019] In addition, to achieve the above objectives, this application also proposes a storage medium storing an image shadow correction program, which, when executed by a processor, implements the steps of the image shadow correction method described above.

[0020] Furthermore, to achieve the above objectives, this application also proposes a camera host, which includes: a memory, a processor, and an imaging image shadow correction program stored in the memory and executable on the processor. When the imaging image shadow correction program is executed by the processor, it implements the steps of the imaging image shadow correction method described above.

[0021] Furthermore, to achieve the above objectives, this application also proposes an endoscope device, which includes: at least one optical mirror, a camera, and a camera host as described above, wherein the camera host is electrically connected to the camera, and the camera is adapted to be connected to the optical mirror.

[0022] This application provides a method, apparatus, camera host, and endoscope for correcting shadows in an imaging image. The method includes: acquiring correction parameters corresponding to an optical mirror, wherein the correction parameters are determined based on the optical center of the optical mirror and a preset brightness attenuation model, wherein the optical center and the preset brightness attenuation model are obtained by calibrating the brightness of the optical mirror; after acquiring an imaging image through the optical mirror, performing shadow correction on the imaging image using the correction parameters; and displaying the shadow-corrected imaging image.

[0023] This application allows for brightness calibration of the optical sight mirror before use to obtain its optical center and a preset brightness attenuation model. Then, the correction parameters for the optical sight mirror are determined based on the optical center and the preset brightness attenuation model. Furthermore, during use, after the camera acquires an image through the optical sight mirror, the correction parameters can be used to correct shadows on the image before displaying the corrected image. Therefore, compared to existing methods using fixed correction parameters, this application improves the correction effect by calibrating the optical sight mirror to obtain the corresponding correction parameters before performing shadow correction. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the camera host structure in the hardware operating environment involved in the embodiments of this application; Figure 2This is a flowchart illustrating the first embodiment of the image shadow correction method of this application; Figure 3 This is a flowchart illustrating the second embodiment of the image shadow correction method of this application; Figure 4 This is a schematic diagram of the calibration image in the second embodiment of the imaging image shadow correction method of this application; Figure 5 This is a schematic diagram of the up-pixel brightness curve in the second embodiment of the image shadow correction method of this application; Figure 6 This is a schematic diagram of the pixel brightness curve in the second embodiment of the image shadow correction method of this application; Figure 7 This is a schematic diagram of the optical center in the second embodiment of the image shadow correction method of this application; Figure 8 This is a flowchart illustrating the third embodiment of the image shadow correction method of this application; Figure 9 This is a structural block diagram of the first embodiment of the image shadow correction device of this application.

[0027] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0028] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0029] Reference Figure 1 , Figure 1 This is a schematic diagram of the camera host structure in the hardware operating environment involved in the embodiments of this application.

[0030] like Figure 1As shown, the camera host may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include an interface for connecting to a display screen; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. In this application, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0031] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the camera host and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0032] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an image shadow correction program.

[0033] exist Figure 1 In the endoscope shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the endoscope calls the imaging image shadow correction program stored in the memory 1005 through the processor 1001 and executes the imaging image shadow correction method provided in the embodiments of this application.

[0034] It should be noted that, as a medical device used for diagnosis and treatment, the image quality of an endoscope directly affects the accuracy of diagnosis and the safety of surgery. Endoscopic equipment typically supports the replacement of different optical endoscopes to meet diverse clinical needs. Because different optical endoscopes have different optical design parameters and bayonet optical zoom magnification, their imaging characteristics vary considerably, particularly in terms of uneven image brightness distribution, with higher brightness in the central area and obvious vignetting (shadowing) at the edges. Therefore, shadow correction is generally required after changing the optical endoscope.

[0035] In current methods of shadow correction, fixed correction parameters are generally used for the captured image. However, since different optical lenses have different optical design parameters and mount optical zoom magnification, using a fixed correction parameter for all corrections will result in poor correction results.

[0036] Therefore, to address the aforementioned shortcomings, this embodiment provides an image shadow correction method. Before use, the brightness of the optical lens is calibrated to obtain its optical center and a preset brightness attenuation model. Then, correction parameters for the optical lens are determined based on the optical center and the preset brightness attenuation model. During use, after the camera acquires the image through the optical lens, the correction parameters are used to correct the shadows in the image, and the corrected image is then displayed. Therefore, compared to existing methods using fixed correction parameters, this embodiment improves the correction effect by calibrating the optical lens to obtain the corresponding correction parameters before performing shadow correction.

[0037] For ease of understanding, the following is combined with Figures 2 to 9 The shadow correction method for imaging images provided in the embodiments of this application will be described in detail.

[0038] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image shadow correction method of this application. The first embodiment of the image shadow correction method of this application is presented as follows: Figure 2 As shown, in this embodiment, the specific method includes: Step S10: Obtain the correction parameters corresponding to the optical sight mirror. The correction parameters are determined based on the optical center of the optical sight mirror and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are obtained by calibrating the brightness of the optical sight mirror.

[0039] It is understood that the method of this embodiment can be applied to any device with data processing, program execution, and shadow correction capabilities, such as the camera host of an endoscope device, and this embodiment does not impose any limitations on this. However, for ease of understanding, this embodiment uses the camera host as the execution subject to describe this embodiment and the following embodiments.

[0040] It is also understood that, in this embodiment, the endoscopic device may include: a camera, a monitor, an optical sight, and a camera host. The camera host can be electrically connected to the camera that captures the image and the monitor that displays the image. The camera is also connected to an optical sight. In actual use, medical staff can place the camera in a suitable position and capture the patient's lesion. After receiving the image, the camera host can perform shadow correction on it and transmit the corrected image to the monitor for display so that medical staff can observe it. Furthermore, since the structures of different lesion sites vary, different types of optical endoscopes are used for easier observation. In this embodiment, the aforementioned optical endoscope can be any type, such as an arthroscope, hysteroscope, laparoscope, etc., and this embodiment does not impose any limitations on it. Therefore, the imaging image in this embodiment can be the image captured by a camera after the optical endoscope is attached.

[0041] It should be understood that the aforementioned correction parameters can be parameters used for shadow correction that are matched to the currently used optical sight. In this embodiment, the camera host can be calibrated for different optical sights before leaving the factory to obtain the corresponding correction parameters and store them in the camera host. Then, when the user uses the camera, the camera host can directly select the corresponding correction parameters according to the connected optical sight.

[0042] As another implementation, considering that the optical sight used by the user in actual use may not be the optical sight calibrated at the factory, and therefore the correction parameters of the optical sight may not be stored inside, the camera host in this embodiment can also support calibration to obtain the above-mentioned correction parameters before the user uses it.

[0043] It should be emphasized that, whether the calibration is performed before leaving the factory or after the user uses it, in this embodiment, in order to obtain the correction parameters corresponding to the optical sight lens, the optical center and the preset brightness attenuation model of the optical sight lens can be obtained by performing brightness calibration on the optical sight lens, and then the corresponding correction parameters can be determined based on the optical center and the preset brightness attenuation model.

[0044] It should be noted that the aforementioned optical center can be the point with the highest brightness in the image obtained through the optical lens. The aforementioned preset brightness attenuation model can be a model of the relationship between the distance of each pixel in the image obtained through the optical lens to the optical center and its brightness. In this embodiment, the aforementioned preset brightness attenuation model can characterize the attenuation relationship of the brightness of each pixel in the optical lens with the distance between the optical centers.

[0045] It should also be noted that the aforementioned brightness calibration can be a process used to obtain the brightness attenuation relationship of the optical center and each pixel as a function of the distance between the optical centers. Therefore, before use, the camera host can first perform brightness calibration on the currently connected optical lens to obtain the optical center and a preset brightness attenuation model. Then, based on the optical center and the preset brightness attenuation model, the camera host determines and saves the correction parameters for the optical lens. Thus, during use, the camera host can directly obtain the corresponding correction parameters for the optical lens.

[0046] Step S20: After acquiring the image through the optical lens, the image is shaded using the correction parameters.

[0047] It is understood that the aforementioned image can be the image captured by the camera through the optical lens. In this embodiment, after capturing the aforementioned image, the camera can transmit it to the camera host. After obtaining the image, the camera host can use the acquired correction parameters to perform shadow correction on the image, thereby making the image visually more uniform and improving display quality.

[0048] Step S30: Display the image after shadow correction.

[0049] It should be understood that after the camera host has completed the shadow correction of the image, the shadow-corrected image can be displayed on the monitor for the user to view.

[0050] Therefore, this embodiment allows for brightness calibration of the optical sight mirror before use to obtain its optical center and a preset brightness attenuation model. Then, the correction parameters for the optical sight mirror are determined based on the optical center and the preset brightness attenuation model. During use, when the camera host obtains the image captured by the optical sight mirror, the correction parameters can be used to correct shadows on the image, and then the corrected image can be displayed. Thus, compared to existing methods using fixed correction parameters, this embodiment improves the correction effect by calibrating the optical sight mirror to obtain the corresponding correction parameters before performing shadow correction.

[0051] Furthermore, in order to achieve brightness calibration, in this embodiment, before the step of obtaining the correction parameters corresponding to the optical lens, the following steps are also included: Step S01: Obtain a calibration image, which is an image obtained by taking a picture of a preset white balance object through the optical lens.

[0052] It should be noted that the aforementioned preset white balance object can be any object that can uniformly reflect white, such as a white balance cap. This embodiment uses a white balance cap for illustration. The aforementioned calibration image can be an image of the preset white balance object taken through the optical lens.

[0053] In actual use, whether before leaving the factory or during subsequent calibration before user use, when the user plugs a new optical lens into the camera host, the camera with the optical lens can be inserted into the white balance cap. Then, after the user triggers a button (such as the shadow correction button), the camera will take a picture of the scene inside the white balance cap, obtain a calibration image and transmit it to the camera host. The camera host can then obtain the calibration image and begin calibration.

[0054] Step S02: Obtain the optical center corresponding to the optical lens based on the calibration image, and obtain the preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image.

[0055] Understandably, after obtaining the calibration image, the image can be identified to determine the pixel with the highest brightness, and the position of this pixel can be used as the optical center of the optical lens. Then, based on the optical center and the brightness of each pixel in the calibration image, the brightness attenuation relationship between the optical center and each pixel can be determined. Based on this attenuation relationship, the preset brightness attenuation model corresponding to the optical lens can be obtained.

[0056] Step S03: Determine the correction parameters corresponding to the optical lens based on the optical center and the preset brightness attenuation model.

[0057] After obtaining the optical center and the preset brightness attenuation model, the correction parameters required for shadow correction of each pixel in the optical lens can be obtained. Then, shadow correction can be performed on each pixel based on these correction parameters.

[0058] This embodiment allows for brightness calibration of the optical sight mirror before use to obtain its optical center and a preset brightness attenuation model. Then, based on the optical center and the preset brightness attenuation model, the correction parameters for the optical sight mirror are determined. During use, when the camera receives the image captured by the optical sight mirror, the correction parameters can be used to correct shadows on the image before displaying the corrected image. Therefore, compared to existing methods using fixed correction parameters, this embodiment improves the correction effect by calibrating the optical sight mirror to obtain the corresponding correction parameters before performing shadow correction.

[0059] Furthermore, since the corresponding calibration parameters can be set according to different optical lenses in this embodiment, it can be adapted to multiple models of endoscope systems and supports rapid lens replacement and recalibration. For interchangeable optical lens systems with different mounts and focal length magnifications, this invention can quickly reconstruct the optical center and brightness model after replacing the optical lens, realizing a rapid calibration and efficient correction workflow.

[0060] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the image shadow correction method of this application. Based on the first embodiment described above, a second embodiment of the image shadow correction method of this application is proposed.

[0061] like Figure 3 As shown, in order to accurately obtain the optical center of the optical sight lens, in this embodiment, the step of obtaining the optical center corresponding to the optical sight lens based on the calibration image includes: Step S021: Obtain the grayscale image corresponding to the calibration image, and determine the sum of the pixel brightness in each row and the sum of the pixel brightness in each column of the grayscale image.

[0062] It should be noted that the grayscale image mentioned above can be an image represented by grayscale values. (See reference...) Figure 4 , Figure 4 This is a schematic diagram of the calibration image in the second embodiment of the image shadow correction method of this application. Figure 4 As shown, the shooting range supported by the optical lens in this embodiment is circular. Of course, this embodiment is only for illustrative purposes and is not limited to a circular shape.

[0063] In this embodiment, after the camera host obtains the calibration image, since the camera generally captures a red-green-blue (RGB) image, it can be converted to obtain the corresponding grayscale image in order to facilitate the subsequent determination of brightness.

[0064] In this embodiment, when converting the calibration image to a grayscale image, each channel of each pixel in the calibration image can be converted and merged according to a preset weighted merging formula to obtain the grayscale value corresponding to that pixel, thus obtaining a grayscale image. Specifically, the preset weights can be set according to the actual situation. For example, in this embodiment, the preset weighted merging formula of 0.299R+0.587G+0.114B can be used for merging, but there is no limitation on this.

[0065] It should also be noted that the above-mentioned total row pixel brightness can be the sum of the brightness values ​​of all pixels in a row, and the above-mentioned total column pixel brightness can be the sum of the brightness values ​​of all pixels in a column.

[0066] In practical use, once the camera host obtains the grayscale image, it can obtain the brightness value of each pixel. Then, it can determine the sum of the pixel brightness values ​​of each row of pixels as the total brightness of that row, and the sum of the pixel brightness values ​​of each column of pixels as the total brightness of that column.

[0067] Step S022: Construct a row pixel brightness curve based on the sum of the pixel brightness of each row, and construct a column pixel brightness curve based on the sum of the pixel brightness of each column.

[0068] Understandably, the row pixel brightness curves described above can be obtained by summing the brightness of each row of pixels, and these row pixel brightness curves can reflect the brightness variation trend of the calibration image in the horizontal direction. Similarly, the column pixel brightness curves described above can be obtained by summing the brightness of each column of pixels, and these column pixel brightness curves can reflect the brightness variation trend of the calibration image in the vertical direction.

[0069] For ease of understanding, please refer to Figure 5 as well as Figure 6 , Figure 5 This is a schematic diagram of the downpixel brightness curve in the second embodiment of the image shadow correction method of this application. Figure 6 This is a schematic diagram of the pixel brightness curve in the second embodiment of the image shadow correction method of this application.

[0070] like Figure 5 As shown, the horizontal axis of the row pixel brightness curve represents the number of pixels in a row of the grayscale image (Row Index), and the vertical axis represents the sum of row pixel brightness (Sum of Row Brightness). Figure 6 As shown, the horizontal axis of the column pixel brightness curve represents the number of columns of pixels in the grayscale image (Row Index), and the vertical axis represents the sum of the column pixel brightness in that row (Sum of Row Brightness).

[0071] according to Figure 5 as well as Figure 6 It is easy to see that the brightness is higher closer to the center of the grayscale image and lower closer to the edge of the optical lens. The image outside the optical lens appears black and has a brightness value of 0 because it is blocked by the optical lens.

[0072] Step S023: Filter the row pixel brightness curve and the column pixel brightness curve respectively, and fit the filtered row pixel brightness curve and the filtered column pixel brightness curve respectively.

[0073] It should be understood that, in order to reduce the impact of noise and improve the accuracy of subsequent results, in this embodiment, after the camera host obtains the row pixel brightness curve and the column pixel brightness curve, the row pixel brightness curve and the column pixel brightness curve can be filtered respectively, specifically, Gaussian filtering can be performed.

[0074] It should be emphasized that the size and standard deviation of the Gaussian filter kernel can be selected according to actual needs when performing Gaussian filtering, and this embodiment does not impose any restrictions on this.

[0075] After smoothing the row and column pixel brightness curves using Gaussian filtering, this embodiment further fits the filtered row and column pixel brightness curves to improve the accuracy of optical center determination. Specifically, Gaussian curves can be used for fitting, that is, Gaussian curves are used to fit both the filtered row and column pixel brightness curves respectively. Since Gaussian curves are used for fitting in this embodiment, the subsequently obtained optical center can be at the sub-pixel level. Furthermore, because the shape of the Gaussian curve is single-peaked, it can better describe the brightness variation and accurately determine the position of the optical center.

[0076] Step S024: Determine the row brightness peak based on the fitted row pixel brightness curve, determine the column brightness peak based on the fitted column pixel brightness curve, and obtain the optical center corresponding to the optical lens based on the row brightness peak and the column brightness peak.

[0077] It should also be understood that the aforementioned row brightness peak can be the peak value of the sum of the brightness of each row of pixels in the row pixel brightness curve, and the aforementioned column brightness peak can be the peak value of the sum of the brightness of each column of pixels in the row pixel brightness curve. Continuing based on... Figure 5 as well as Figure 6 As shown, the value of the vertical coordinate corresponding to the position of the dashed line can be taken as the peak brightness. Furthermore, after determining the row and column peak brightness, since the horizontal coordinate of the curve is the pixel coordinate, the value of the horizontal coordinate below the row peak brightness in the row pixel brightness curve can be directly used as the vertical coordinate of the optical center (i.e., Figure 5 (Yc), directly using the value of the horizontal coordinate below the row brightness peak in the column pixel brightness curve as the horizontal coordinate of the optical center (i.e. Figure 6 (Xc), and thus the corresponding optical center can be determined, denoted as (Xc, Yc), referring to Figure 7 , Figure 7 This is a schematic diagram of the optical center in the second embodiment of the image shadow correction method of this application, namely... Figure 7 Point O is the optical center mentioned above.

[0078] It should also be emphasized that, since Gaussian filtering and smooth fitting are used in this embodiment, the interference of image noise on center positioning can be effectively eliminated. It does not rely on manual marking or optical mirror specifications and is applicable to different types of endoscope systems, thus improving robustness.

[0079] Reference Figure 8 , Figure 8 This is a flowchart illustrating the third embodiment of the image shadow correction method of this application. Based on the above embodiments, the third embodiment of the image shadow correction method of this application is proposed.

[0080] To obtain the aforementioned preset brightness attenuation model, such as Figure 8 As shown, in this embodiment, the step of obtaining the preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image includes: Step S025: Determine the relative distance between the optical center and each pixel in the calibration image.

[0081] It should be noted that the aforementioned relative distance can be the distance between the optical center and each pixel in the calibration image. For ease of subsequent standardization, the relative distance in this embodiment can be a normalized distance, specifically the distance from each pixel in the calibration image to the optical center after normalization.

[0082] In actual use, after obtaining the optical center, the camera host can use the optical center as a reference to determine the distance between each pixel in the calibration image and the optical center. After normalization processing, the obtained result is used as the above relative distance. For ease of subsequent explanation, the relative distance between the i-th pixel and the optical center is denoted as ri.

[0083] Step S026: Obtain the pixel grayscale value of each pixel in the calibration image; Step S027: Perform nonlinear fitting on the initial brightness attenuation model based on the relative distances and pixel gray values ​​to obtain the calibration parameters corresponding to the optical lens.

[0084] It is understood that the aforementioned pixel grayscale values ​​can be the grayscale values ​​corresponding to each pixel in the calibration image. Since the calibration image has been converted into a grayscale image in this embodiment, the pixel grayscale values ​​of each pixel can be obtained directly from the grayscale image. For ease of subsequent explanation, the pixel grayscale value of the i-th pixel is denoted as Li.

[0085] It is also understandable that after obtaining the pixel grayscale value of each pixel, the brightness attenuation relationship between the optical center and each pixel can be determined by nonlinear fitting based on the relative distance between the optical center and each pixel and the corresponding pixel grayscale value.

[0086] It is important to emphasize that the aforementioned initial brightness attenuation model can be a model characterizing the attenuation law of brightness. In this embodiment, the initial brightness attenuation model can be constructed based on the fourth power of cosine law. This is because research has shown that in lens shading, the brightness attenuation trend of the target point conforms to the fourth power of cosine law. For the same optical lens module, its imaging brightness only changes with the imaging angle between the imaging point and the optical axis. Furthermore, its trend is: proportional to the fourth power of the cosine value of the imaging angle, and the proportionality coefficient is determined by the lens diameter and focal length of the optical lens.

[0087] Therefore, based on this, since the image plane brightness varies with the incident angle... The decay pattern is as follows: If we assume that the distance between a point on the image plane and the optical center is r, and the focal length is f, then the following relationship holds (1):

[0088] And due to the following relations (2) and (3):

[0089]

[0090] Therefore, the following relation (4) exists:

[0091] The following relationship (5) is obtained by conversion:

[0092] The following relationship (6) is obtained by conversion:

[0093] The following relation (7) can then be obtained:

[0094] To facilitate the modeling of the normalized radius, the following relationship (8) is defined:

[0095] in The geometric scale factor of the image plane can be determined by the lens focal length, image plane size and pixel normalization scale. It can be used to establish the mapping relationship between the imaging angle and the normalized radius of the image plane, and thus the following relationship (9) can be obtained:

[0096] Substituting the above relation (9) into the fourth power of cosine law, we can obtain the following relation (10):

[0097] By further introducing the luminance amplitude factor A and the offset term, the above initial luminance attenuation model can be obtained, specifically:

[0098] Furthermore, among them Let A be the pixel grayscale value of the pixel at a distance r from the optical center, and let A be the brightness amplitude factor, which can be used to control the brightness amplitude of the optical center region. The brightness decay index reflects the rate at which brightness decreases with increasing distance. The offset term is used to compensate for background brightness and system noise, ensuring that the edges maintain a reasonable brightness level.

[0099] Therefore, in this embodiment, after the camera host obtains the pixel grayscale value of each pixel, the relative distance and the corresponding pixel grayscale value can be substituted into the above-mentioned initial brightness attenuation model. A nonlinear fit is performed with r to obtain the calibration parameters corresponding to the optical lens. These calibration parameters may include the aforementioned brightness amplitude factor A and brightness attenuation index. And the offset term.

[0100] Step S028: Obtain a preset brightness attenuation model based on the calibration parameters and the initial brightness attenuation model.

[0101] After obtaining the above calibration parameters, the calibration parameters can be substituted into the above initial brightness attenuation model to obtain the preset brightness attenuation model.

[0102] Furthermore, after obtaining the preset brightness attenuation model, in order to obtain the correction parameters, in this embodiment, the step of determining the correction parameters corresponding to the optical lens based on the optical center and the preset brightness attenuation model includes: Step S031: Substitute the relative distances into the preset brightness attenuation model to obtain the pixel grayscale value of each pixel.

[0103] It should be noted that, since the preset brightness attenuation model in this embodiment is a model of the attenuation relationship between distance and the corresponding brightness value, after obtaining the above-mentioned preset brightness attenuation model, each relative distance can be substituted into the above-mentioned preset attenuation model to obtain the brightness value of each pixel, and this brightness value is used as the pixel grayscale value of that pixel. For ease of subsequent understanding, that is... The relative distance between the pixel and the optical center The corresponding pixel grayscale value is denoted as .

[0104] Step S032: Determine the pixel gray value of the optical center from the pixel gray values ​​of each pixel.

[0105] Understandably, since the relative distance between the optical centers is 0, r=0 can be directly substituted into the above-mentioned preset brightness attenuation model to obtain the pixel gray value of the optical center, denoted as L(0).

[0106] Step S033: Obtain the correction coefficient of each pixel based on the pixel grayscale value of the optical center and the pixel grayscale value of each pixel; Step S034: Obtain the correction parameters corresponding to the optical lens based on the correction coefficients of each pixel.

[0107] It should be understood that the aforementioned correction coefficients can be coefficients for shadow correction of each pixel. In this embodiment, after obtaining the pixel grayscale value of each pixel... And the pixel grayscale value L(0) at the optical center can be used to calculate the correction coefficient for each pixel by dividing by the value, i.e. .

[0108] After obtaining the correction coefficient for each pixel, a correction coefficient matrix can be constructed, which can include the correction coefficient for each pixel. This correction coefficient matrix can then be used as the correction parameter for the optical lens.

[0109] Therefore, during actual calibration, after obtaining the calibration parameters, the camera host can divide the pixel value of each pixel by the calibration coefficient corresponding to that pixel in the calibration parameters after receiving the image, thereby completing the shadow correction.

[0110] It should be emphasized that, considering that the image is an RGB image, the camera host in this embodiment can perform division by channel when performing correction, that is, obtain the R value, G value and B value of each pixel in the image, and divide the R value, G value and B value by the correction coefficient corresponding to the pixel to perform correction.

[0111] It should also be emphasized that, since the initial brightness attenuation model is constructed based on the fourth power of cosine in this embodiment, the model has few parameters, fast convergence, high accuracy, and strong anti-interference ability, and is suitable for different optical lens modules and working distances.

[0112] Furthermore, considering that not all pixel grayscale values ​​in the calibration image can be nonlinearly fitted, because the presence of edge black areas or outlier values ​​may affect the fitting result, in this embodiment, the step of performing nonlinear fitting on the initial brightness attenuation model based on the relative distances and pixel grayscale values ​​to obtain the calibration parameters corresponding to the optical lens includes: Step S0271: Filter the gray values ​​of each pixel according to the preset gray value range to obtain the effective pixel gray values; Step S0272: Perform nonlinear fitting on the initial brightness attenuation model based on the relative distances and the effective pixel gray values ​​to obtain the calibration parameters corresponding to the optical lens.

[0113] It should be noted that the aforementioned preset grayscale value range can be used to filter out edge black areas or highlight outliers. For example, in this embodiment, the aforementioned preset grayscale value range can be described as 0.05 to 0.95. It should also be noted that the aforementioned effective pixel grayscale values ​​can be grayscale values ​​that meet the fitting requirements after filtering.

[0114] In practical use, after the camera host obtains the pixel grayscale values ​​of each pixel in the calibration image, it can determine whether each pixel grayscale value is between 0.05 and 0.95, specifically 0.05 < pixel grayscale value < 0.95. If it is within this range, it indicates that the pixel is not an edge black area or a highlight anomaly, and thus the pixel grayscale value of that pixel can be considered a valid pixel grayscale value. If it is not within this range, it indicates that the pixel is an edge black area or a highlight anomaly, and thus it is discarded.

[0115] After obtaining the grayscale values ​​of each effective pixel, the initial brightness attenuation model can be nonlinearly fitted based on the relative distance between the pixels corresponding to the effective pixel grayscale value and the effective pixel grayscale value, thereby obtaining more accurate calibration parameters.

[0116] Furthermore, considering that after shadow correction, the use of a division method may lead to over-enhancement of some pixels, resulting in a decrease in image quality, in this embodiment, the step of displaying the shadow-corrected image includes: Step S31: Obtain the image bit depth of the image and determine the truncation limit range based on the image bit depth.

[0117] It should be noted that the image bit depth mentioned above can be the number of bits used to represent the color information of a single pixel in the image. For example, 8-bit can represent that each pixel has 256 (i.e., ...) bits. There are 10-bit colors, for example, 10-bit can represent 1024 colors per pixel (i.e., ... (Colors, etc.) The image bit depth in this embodiment can be set according to the actual situation, and this embodiment does not impose any restrictions on it.

[0118] It should also be noted that the aforementioned truncation limit range can be used to ensure that the image bit depth of the pixel is within the allowed range. In this embodiment, it can be the image bit depth minus 1. For example, 8-bit contains 256 colors, so the aforementioned truncation limit range can be 0 to 255. Or, for example, 10-bit contains 1024 colors, so the aforementioned truncation limit range can be 0 to 1023, and so on. Specifically, the truncation limit range can be determined based on the determined image bit depth.

[0119] Step S32: Based on the truncation limitation range, truncate and limit the image after shadow correction; Step S33: Display the image after truncation and restriction.

[0120] After obtaining the truncation limit range, it can be determined whether each pixel in the shadow-corrected image is within the truncation limit range. If it is, it means that the pixel has not been over-enhanced, and its pixel value can be retained. If it is not, it means that the pixel has been over-enhanced, and its pixel value can be truncated to the end of the truncation limit range, which can also be understood as adjusting the pixel value to the end of the truncation limit range.

[0121] For example, if in an 8-bit image after shadow correction, there is a pixel with a pixel value of 265, since 265 is not between 0 and 255, the pixel value of that pixel can be adjusted from 265 to 255, thereby avoiding over-enhancement that affects image quality.

[0122] Furthermore, this application also proposes a storage medium storing an image shadow correction program, which, when executed by a processor, implements the steps of the image shadow correction method described above.

[0123] In addition, refer to Figure 9 , Figure 9 This is a structural block diagram of the first embodiment of the image shadow correction device of this application; as shown... Figure 9 As shown in the embodiments of this application, an image shadow correction device is also proposed, the device comprising: The parameter acquisition module 901 is used to acquire the correction parameters corresponding to the optical sight mirror. The correction parameters are determined based on the optical center of the optical sight mirror and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are both obtained by calibrating the brightness of the optical sight mirror. The shadow correction module 902 is used to correct the shadow of the image image after it is acquired by the optical sight lens using the correction parameters. The image display module 903 is used to display the image after shadow correction.

[0124] This embodiment allows for brightness calibration of the optical sight mirror before use to obtain its optical center and a preset brightness attenuation model. Then, based on the optical center and the preset brightness attenuation model, the correction parameters for the optical sight mirror are determined. During use, when the camera receives the image captured by the optical sight mirror, the correction parameters can be used to correct shadows on the image before displaying the corrected image. Therefore, compared to existing methods using fixed correction parameters, this embodiment improves the correction effect by calibrating the optical sight mirror to obtain the corresponding correction parameters before performing shadow correction.

[0125] In one implementation, the parameter acquisition module 901 is further configured to acquire a calibration image, which is an image obtained by taking a picture of a preset white balance object through the optical lens; obtain the optical center corresponding to the optical lens based on the calibration image, and obtain a preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image; and determine the correction parameters corresponding to the optical lens according to the optical center and the preset brightness attenuation model.

[0126] Based on the first embodiment of the imaging image shadow correction device described in this application, a second embodiment of the imaging image shadow correction device of this application is proposed.

[0127] In this embodiment, the parameter acquisition module 901 is further configured to acquire the grayscale image corresponding to the calibration image, and determine the sum of pixel brightness in each row and the sum of pixel brightness in each column of the grayscale image; construct a row pixel brightness curve based on the sum of pixel brightness in each row, and construct a column pixel brightness curve based on the sum of pixel brightness in each column; filter the row pixel brightness curve and the column pixel brightness curve respectively, and fit the filtered row pixel brightness curve and the filtered column pixel brightness curve respectively; determine the row brightness peak value based on the fitted row pixel brightness curve, determine the column brightness peak value based on the fitted column pixel brightness curve, and obtain the optical center corresponding to the optical lens based on the row brightness peak value and the column brightness peak value.

[0128] In one implementation, the parameter acquisition module 901 is further configured to determine the relative distance between the optical center and each pixel in the calibration image; acquire the pixel grayscale value of each pixel in the calibration image; perform nonlinear fitting on the initial brightness attenuation model based on each relative distance and each pixel grayscale value to obtain the calibration parameters corresponding to the optical lens; and obtain a preset brightness attenuation model based on the calibration parameters and the initial brightness attenuation model.

[0129] Based on the above embodiments of the imaging image shadow correction device of this application, a third embodiment of the imaging image shadow correction device of this application is proposed.

[0130] In this embodiment, the parameter acquisition module 901 is further configured to filter the gray values ​​of each pixel according to a preset gray value range to obtain effective pixel gray values; and to perform nonlinear fitting on the initial brightness attenuation model based on the relative distances and the effective pixel gray values ​​to obtain the calibration parameters corresponding to the optical lens.

[0131] In one implementation, the parameter acquisition module 901 is further configured to substitute each of the relative distances into the aforementioned preset brightness attenuation model to obtain the pixel grayscale value of each of the pixels; determine the pixel grayscale value of the optical center from the pixel grayscale values ​​of each of the pixels; obtain the correction coefficient of each of the pixels based on the pixel grayscale value of the optical center and the pixel grayscale values ​​of each of the pixels; and obtain the correction parameter corresponding to the optical lens based on the correction coefficient of each of the pixels.

[0132] In one implementation, the image display module 903 is further configured to acquire the image bit depth of the image and determine a truncation limit range based on the image bit depth; truncate the shadow-corrected image based on the truncation limit range; and display the truncated image.

[0133] Other embodiments or specific implementations of the imaging image shadow correction device described in this application can be found in the above-described method embodiments, and will not be repeated here.

[0134] In addition, to achieve the above objectives, this application embodiment also provides a storage medium storing an imaging image shadow correction program, which, when executed by a processor, implements the steps of the imaging image shadow correction method described above.

[0135] In addition, to achieve the above objectives, this application also provides an endoscope device, which includes: at least one optical mirror, a camera, and a camera host as described above, wherein the camera host is electrically connected to the camera, and the camera is adapted to be connected to the optical mirror.

[0136] Other embodiments or specific implementations of the endoscopic device described in this application can be found in the above-described method embodiments, and will not be repeated here.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0138] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0140] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for correcting shadows in an image, characterized in that, The method includes: Obtain the correction parameters corresponding to the optical sight mirror. The correction parameters are determined based on the optical center of the optical sight mirror and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are both obtained by calibrating the brightness of the optical sight mirror. After the image is acquired through the optical lens, the image is shaded using the correction parameters. Display the image after shadow correction.

2. The method as described in claim 1, characterized in that, Before the step of obtaining the correction parameters corresponding to the optical sight lens, the method further includes: Acquire a calibration image, which is an image obtained by taking a picture of a preset white balance object through the optical lens; The optical center corresponding to the optical sight is obtained based on the calibration image, and a preset brightness attenuation model corresponding to the optical sight is obtained based on the optical center and the calibration image. The correction parameters corresponding to the optical lens are determined based on the optical center and the preset brightness attenuation model.

3. The method as described in claim 2, characterized in that, The step of obtaining the optical center corresponding to the optical sight based on the calibration image includes: Obtain the grayscale image corresponding to the calibration image, and determine the total brightness of pixels in each row and the total brightness of pixels in each column of the grayscale image; A row pixel brightness curve is constructed based on the sum of the pixel brightness of each row, and a column pixel brightness curve is constructed based on the sum of the pixel brightness of each column. The row pixel brightness curves and column pixel brightness curves are filtered respectively, and the filtered row pixel brightness curves and column pixel brightness curves are fitted respectively. The row brightness peak is determined based on the fitted row pixel brightness curve, the column brightness peak is determined based on the fitted column pixel brightness curve, and the optical center corresponding to the optical lens is obtained based on the row brightness peak and the column brightness peak.

4. The method as described in claim 2, characterized in that, The step of obtaining the preset brightness attenuation model corresponding to the optical lens based on the optical center and the calibration image includes: Determine the relative distance between the optical center and each pixel in the calibration image; Obtain the pixel grayscale value of each pixel in the calibration image; Based on the relative distances and pixel gray values, the initial brightness attenuation model is nonlinearly fitted to obtain the calibration parameters corresponding to the optical lens. A preset brightness attenuation model is obtained based on the calibration parameters and the initial brightness attenuation model.

5. The method as described in claim 4, characterized in that, The step of performing nonlinear fitting on the initial brightness attenuation model based on the relative distances and pixel grayscale values ​​to obtain the calibration parameters corresponding to the optical lens includes: The gray values ​​of each pixel are filtered according to a preset gray value range to obtain valid pixel gray values; The calibration parameters corresponding to the optical lens are obtained by performing nonlinear fitting on the initial brightness attenuation model based on the relative distances and the effective pixel gray values.

6. The method as described in claim 4, characterized in that, The step of determining the correction parameters corresponding to the optical lens based on the optical center and the preset brightness attenuation model includes: Substitute each of the relative distances into the aforementioned preset brightness attenuation model to obtain the pixel grayscale value of each pixel; The pixel gray value of the optical center is determined from the pixel gray values ​​of each of the aforementioned pixels; The correction coefficient of each pixel is obtained based on the pixel gray value of the optical center and the pixel gray value of each pixel; The correction parameters corresponding to the optical lens are obtained based on the correction coefficients of each pixel.

7. The method as described in claim 1, characterized in that, The step of displaying the shadow-corrected image includes: Obtain the image bit depth of the image and determine the truncation limit range based on the image bit depth; The shadow-corrected image is truncated based on the truncation limitation range; Display the image after truncation and restriction.

8. A device for correcting shadows in an image, characterized in that, The device includes: The parameter acquisition module is used to acquire the correction parameters corresponding to the optical sight lens. The correction parameters are determined based on the optical center of the optical sight lens and the corresponding preset brightness attenuation model. The optical center and the preset brightness attenuation model are both obtained by calibrating the brightness of the optical sight lens. The shadow correction module is used to correct the shadows of the image captured by the optical sight lens using the correction parameters. The image display module is used to display the image after shadow correction.

9. A camera host, characterized in that, The camera host includes: a memory, a processor, and an image shadow correction program stored in the memory and executable on the processor. When the image shadow correction program is executed by the processor, it implements the steps of the image shadow correction method as described in any one of claims 1 to 7.

10. An endoscopic device, characterized in that, The endoscope device includes: at least one optical sight mirror, a camera, and a camera host as described in claim 9, wherein the camera host is electrically connected to the camera, and the camera is adapted to be connected to the optical sight mirror.

Images