Methods to repair glare or shadows in images
By converting images to a color space based on lightness or brightness and utilizing erosion, dilation operations, and diffusion kernel repair algorithms, the problems of glare and shadow identification and repair in endoscopic images are solved, improving image quality and ensuring the accuracy of endoscopic examinations and surgeries.
Patent Information
- Application Number
- CN202210980011.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-16
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-08-16
AI Technical Summary
In the RGB color space, existing technologies struggle to effectively identify and repair glare and shadow areas in endoscopic images, resulting in poor image quality and affecting the effectiveness of examinations and surgeries.
The image is converted to a color space with a lightness or brightness dimension, such as HSV or HSL. Glare or shadow areas are identified through erosion and dilation operations, and iterative repair is performed using a diffusion kernel. The repair algorithm performs the repair operation in the RGB color space.
It enables precise identification and repair of glare or shadow areas in endoscopic images, improving image quality and ensuring the accuracy of examinations and surgeries.
Smart Images

Figure CN117670728B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, computer program, and computer-readable medium for repairing glare or shadows in an image. In particular, it refers to an invention that first represents the original image I in a color space mode with a lightness dimension or luminance dimension (e.g., HSV or HSL), so that the same lightness dimension or luminance dimension can be used as a threshold to accurately identify and repair glare or shadow areas. Background Technology
[0002] When performing medical endoscopy or endoscopic surgery, the body is a dark environment, and the light source comes from the endoscope. Therefore, during external training, the training prosthesis is usually placed in a dark box to simulate the internal environment. However, if the light source comes from one or more directions, due to the principle of optical reflection, when it is blocked by internal tissues or surgical instruments or illuminated by highly reflective body fluids, glare (light spots) may form on the captured image, or shadows may form at the blocked areas. Therefore, in endoscopic photography, it is necessary to repair the endoscopic image to avoid errors in image interpretation.
[0003] To present a complete image, discontinuous areas in the image are repaired using image inpainting techniques. A common method is to detect differences in pixel brightness values in the RGB color space to identify defective areas, and then use image inpainting techniques to repair these defective areas.
[0004] Chinese Invention Patent Application CN112967166A discloses "An Automatic Image Watermark Recognition and Processing Method and System Based on OpenCV". The method involves: establishing an image shadow calculation model to find the watermark region; image classification preprocessing; watermark mask shadow extraction, extracting the original watermark image using a shadow template, and then extracting the brightest pixels after grayscale operations from as many images as possible based on the shadow coverage area (note that RGB values are saved here) until the extracted pixel values no longer change, indicating successful extraction; and finally removing the watermark from the original watermark image.
[0005] PCT international patent application WO2022001615A1 discloses "METHODANDSYSTEMFORAUTOMATICALLY REMOVING GLARE REGIONS". This application receives at least one camera preview frame including at least one glare region, then detects the at least one glare region in the field of view of the at least one camera preview frame based on a comparison of the pixel brightness of the at least one camera preview frame with a predetermined threshold of the brightness, and removes the at least one glare region from the at least one camera preview frame.
[0006] Chinese Invention Patent Application CN109993714A discloses "A method for removing colored glare". The method involves inputting an original image containing colored glare, calculating the dark channel image of the original image, calculating the atmospheric light value of the image using the dark channel image, calculating the transmittance of the image using the dark channel image, and finally inputting the original image, atmospheric light value, and transmittance into a calculation formula to remove the glare present in the image.
[0007] US Patent Application No. US08730356B2 discloses a "System and method for automatic flash removal from images". This application states that glare is typically caused by direct reflection from a flash unit. Images are acquired from at least two different locations (possibly simultaneously) and from one or more cameras. Glare patterns are identified by recognizing areas of complete overexposure or known test glare patterns. The images are then processed to remove the glare patterns by subtracting the glare patterns from at least one image and replacing the corresponding data from related images that are not similarly affected.
[0008] Chinese Invention Patent Application No. CN109712084B discloses an "Image Restoration Method, Image Restoration System, and Flat Panel Detector". This application performs connected component analysis on a first image, extracts all defective pixels in the first image, and classifies the defective pixels into isolated point defects, cluster defects, and bad line defects based on the size and shape of the connected components. Then, a second image is output, and the second image is restored according to the type of defective pixels.
[0009] In the aforementioned cases, discontinuous image areas were detected and repaired using the RGB color space. However, in the RGB color space, it is not easy to find thresholds for the R, G, and B values that are related to brightness or luminance, which may result in poor image quality after repair.
[0010] For endoscopic images, the interpretation of these images can affect the effectiveness of examinations and surgeries, therefore the quality of endoscopic image reconstruction needs to be improved. Summary of the Invention
[0011] To address the difficulty in identifying glare and shadow areas in captured images using the RGB color space, particularly in restoring the quality of endoscopic images, this invention utilizes a color space with either a lightness or brightness dimension for image identification. Accordingly, this invention proposes a method for restoring glare or shadows in images, comprising:
[0012] An input original image is represented in a color space mode with a lightness or luminance dimension, and this original image is defined as I. If the original image I has glare, one or more first target regions in the original image I with a lightness or luminance dimension greater than a first preset value are searched; or, if the original image I has shadows, one or more second target regions in the original image I with a lightness or luminance dimension less than a second preset value are searched. One or more erosion operations are performed on the aforementioned first or second target regions to obtain a predicted restoration region, making the original image I a first image, and this first image is defined as S. erode Then compare the original image I with the first image S. erode Matrix multiplication yields a second image, defined as I1, which retains the predicted restoration area. Converting this second image I1 back to RGB color space mode yields a restoration area mask, defined as I... mask Mask I for the repaired area mask Execute a repair algorithm to mask the repaired area. mask The location of the area to be repaired should be set as a mask area to be repaired, defined as Ω. A boundary of the area is defined as... The repair algorithm starts from the boundary of the region. A diffusion process is performed to fill the mask area Ω to be repaired, thereby removing glare or shadows; after one or more iterations, a repaired image with glare or shadows removed is output, and this repaired image is defined as I. inpaint .
[0013] Furthermore, the aforementioned color space model includes any one or similar color space with a lightness or brightness dimension, such as: a color space with hue, saturation, and lightness (HSL); a color space with hue, saturation, and value (HSV); a CIELAB color space; or a color space with luminance, chroma, and chroma (YUV). The conversion of any color space to a color space with lightness is the same operation as described in the embodiments; therefore, this application extends to, but is not limited to, the various color spaces with lightness or brightness described above.
[0014] Furthermore, in the HSV color space mode, firstly, one or more candidate regions in the original image I whose saturation dimension does not reach a set value are searched. Then, the first target region or the second target region is searched from the aforementioned candidate regions. The saturation dimension ranges from 0 to 255, the glare detection setting value is between 150 and 255, and the shadow detection setting value is between 0 and 100. In a dark environment, the saturation dimension is set to 170 for glare detection and 20 for shadow detection.
[0015] Furthermore, the brightness dimension or the luminance dimension ranges from 0 to 255, the first preset value is between 200 and 255 (referring to glare), and the second preset value is between 0 and 50 (referring to shadow).
[0016] Furthermore, the erosion operation removes light from the first or second target region using matrix pixels, thereby scanning each pixel in the first or second target region to remove small and isolated glare or shadows. The matrix pixels can be any matrix shape, such as circular, square, or elliptical, depending on the shape of the glare or shadow; preferably, square matrices such as 3×3, 5×5, or 7×7 are used.
[0017] Furthermore, when the predicted restoration area is a predicted glare area, the region of the second image I1 with a brightness dimension or luminance dimension greater than a third preset value is subjected to one or more dilate operations to expand the predicted restoration area. When the predicted restoration area is a predicted shadow area, the region of the second image I1 with a brightness dimension or luminance dimension less than a fourth preset value is subjected to one or more dilate operations to expand the predicted restoration area; thus, the second image I1 becomes a third image, which is defined as S. dilate Then the third image S dilate Convert back to RGB color space mode to become the mask for the repaired area. mask In the HSV color space mode, with a range of 0 to 255, when the predicted repair area is a predicted glare area, the third preset value of the lightness dimension or brightness dimension is 210; when the predicted repair area is a predicted shadow area, the fourth preset value of the lightness dimension or brightness dimension is 15.
[0018] Furthermore, the dilation operation expands the matrix pixels, thereby scanning each pixel in the predicted repair area to compensate for the area lost due to the erosion operation. The matrix pixels can adopt any matrix shape, such as circular, square, or elliptical matrices, depending on the shape of the glare or shadow. Preferably, square matrices such as 3×3, 5×5, or 7×7 are used.
[0019] Furthermore, when the predicted repair area is the predicted glare area, the repair area mask I is applied. mask The initial RGB pixel values are set to be higher than the average RGB pixel values of the original image I. When the predicted restoration area is a predicted shadow area, the mask I of the restoration area is set. mask The initial RGB pixel values are set to be lower than the average RGB pixel values of the original image I. Preferably, when repairing glare, the mask I of the repair area is... mask The initial RGB pixel values are set to 1.75 times the average RGB pixel values of the original image I; when repairing shadows, the mask I of the repair area is... mask The initial RGB pixel values are set to 0.2 times the average RGB pixel values of the original image I.
[0020] Furthermore, the repair algorithm removes the mask I from the repair area. mask The color information of the image is used to obtain the initialized mask region Ω to be repaired. This mask region Ω is then divided into many smaller repair regions, which are the boundaries of the region. The diffusion kernel is used to repeatedly convolve the image, and the values obtained by multiplying the neighboring pixels with the diffusion kernel matrix are added together to fill the gaps. This diffusion process gradually fills the mask area Ω to be repaired.
[0021] Furthermore, this repair domain For boundaries that are more than one pixel thick, check if they belong to the inpainting domain. The number of iterations for each inpainting region can be independently controlled by checking whether the number of pixels did not exceed a threshold during the previous iteration; alternatively, the number of iterations can be specified. During the iteration process, inpainting regions of different sizes can be used as needed. Integrating different repair domains with diffusion cores Advantages of diffusion nuclei.
[0022] Furthermore, the diffusion kernel is a Gaussian kernel, which is used to convolve the mask region Ω to be repaired, i.e., the kernel weight is 0 and the weighted average of the pixel neighborhood is calculated. Alternatively, the diffusion kernel is a weighted average kernel, which is used to convolve the mask region Ω to be repaired, i.e., the kernel weight is 0 and the weighted average of the surrounding pixels is calculated.
[0023] Furthermore, when the original image I has both glare and shadows, the first target region and the second target region are searched to remove the glare and shadows; when the original image I only has shadows, the original image I is negative-processed to make the shadows become glare and then the glare is removed, and then the repaired image I is... inpaint Perform negative processing.
[0024] The present invention further proposes a computer program that, when installed on a computer, can execute the aforementioned method for repairing glare or shadows in images.
[0025] The present invention further proposes a computer-readable medium that stores the aforementioned computer program.
[0026] The following effects can be achieved based on the above technical features:
[0027] 1. This invention represents the original image in a color space mode with a brightness dimension or a luminance dimension, and can identify glare or shadows using only a brightness or luminance threshold to find the predicted repair area.
[0028] 2. After detecting the first target area or the second target area, the pixels in the first target area or the second target area can be scanned one by one through erosion operation to remove small and isolated glare or shadows, so as to appropriately delete noise.
[0029] 3. The first image I1 obtained through erosion operation can be expanded again by setting the preset values of the brightness dimension or luminance dimension to make up for the glare or shadow areas that may be missing after the erosion operation.
[0030] 4. When the predicted repair area is the predicted glare area, the repair area mask I mask The initial RGB pixel values are usually higher than the average RGB pixel values of the overall original image I, which will affect the repair area mask I. mask The initial RGB pixel values are set higher than the average RGB pixel values of the original image I, typically set to 1.75 times and close to the pixel values around the glare areas of the original image I. This can accelerate the iterative calculation speed of the mask area Ω to be repaired. Similarly, when the predicted repair area is a predicted shadow area, the repair area mask I... mask The initial RGB pixel values are usually lower than the average RGB pixel values of the overall original image I, which will affect the repair area mask I. mask The initial RGB pixel values are set to be lower than the average RGB pixel values of the original image I, usually set to 0.2 times and close to the pixel values around the shadow area of the original image I, which can accelerate the iterative calculation speed of the mask area Ω to be repaired.
[0031] 5. The algorithm of this invention divides the mask region Ω to be repaired into many smaller boundary repair domains. For example, a boundary with a thickness of more than one pixel is used, and a diffusion kernel is repeatedly convolved. The values of the neighboring pixels multiplied by the diffusion kernel matrix are added together and then used to fill in the mask area Ω to be repaired through the diffusion process.
[0032] 6. The diffusion kernel can be, for example, a Gaussian kernel or a weighted average kernel, and during the iteration process, the repair domain of different sizes can be changed as needed. With different diffusion nuclei to integrate different repair domains Advantages of diffusion nuclei. Attached Figure Description
[0033] Figure 1 This is a flowchart illustrating the process of repairing glare in an image, as described in an embodiment of the present invention.
[0034] Figure 2 This is a schematic diagram of the original image with glare and shadow in an embodiment of the present invention.
[0035] Figure 3 This is a schematic diagram of detecting the predicted restoration area of the original image and converting it into a restoration area mask in an embodiment of the present invention, wherein the predicted restoration area is the predicted glare area.
[0036] Figure 4A This is a schematic diagram of the restored image after removing glare from the original image in an embodiment of the present invention.
[0037] Figure 4B This is a schematic diagram of the restored image after removing glare and shadows from the original image in an embodiment of the present invention.
[0038] Figure 5A This is a schematic diagram of a Gaussian kernel used for the diffusion kernel of the mask area to be repaired, as shown in an embodiment of the present invention.
[0039] Figure 5B This is a schematic diagram of a weighted average kernel used for the diffusion kernel of the mask area to be repaired in an embodiment of the present invention.
[0040] Figure 5C This is a schematic diagram illustrating how the iterative process changes the size of the repair domain in the mask area to be repaired and uses different diffusion kernels in an embodiment of the present invention.
[0041] Figure 5D This is a schematic diagram illustrating how the iterative process changes the size of the repair domain in the mask area to be repaired and uses different diffusion kernels in an embodiment of the present invention.
[0042] Figure 6 In this embodiment of the invention, the original image has both glare and shadow. After repairing the glare, the process of repairing the shadow continues.
[0043] Figure 7 In this embodiment of the invention, if the original image only has shadows, a flowchart is used to repair the shadows in the image after performing negative processing on the original image.
[0044] Figure labeling: I - Original image; 10 - Transparent acrylic tube; 20 - Simulated endoscope tube; A - Glare area; Imask - Glare area mask; A1 - Predict glare area; I inpaint -Repair image; B-Shadow area. Detailed Implementation
[0045] The following embodiments are merely illustrative of the method, computer program, and computer-readable medium for repairing glare or shadows in images according to the present invention, and are not intended to limit the present invention.
[0046] See Figure 1 and Figure 2 As shown, an original image I exhibits a glare area A and a shadow area B due to light source reflection. The original image I is then input into a processor. Taking training for endoscopic examination or endoscopic surgery as an example, a transparent acrylic tube 10 is used to simulate internal tubes such as the airway and digestive tract for easier observation. The transparent acrylic tube 10 is placed in a dark box, and a simulated endoscope tube 20 is inserted into the transparent acrylic tube 10 for training. Due to the use of the transparent acrylic tube 10, at least at the transparent acrylic tube 10, a glare area A is generated due to the light source, and a shadow area B is generated due to the obstruction of the simulated endoscope tube 20 or its wall. This simulates the glare and shadows formed in the internal tubes during actual endoscopic examination or surgery.
[0047] See Figures 1 to 3 As shown, the processor represents the original image I in a color space mode with a brightness dimension or a luminance dimension. This embodiment uses the HSV (Hue, Saturation, Value) color space as an example, which includes a hue dimension, a saturation dimension, and a value dimension. The HSV color space is also called the HSB color space, where B represents brightness. The processor searches for more than one candidate region in the original image I whose saturation dimension does not reach a set value, thereby filtering out glare candidate regions that are not oversaturated. The glare detection set value is set between 150 and 255, with a range of 0 to 255. Glare is particularly prone to occur in dark environments when given a light source from a single direction. In dark environments, the saturation dimension set value can be set to 170. Within the aforementioned glare candidate regions, a first target region with a brightness dimension or luminance dimension greater than a first preset value is searched, ranging from 0 to 255. The first preset value is between 200 and 255. An erosion operation is performed on the first target region at least once to obtain a predicted restoration region A1, making the original image I the first image S. erodeThe erosion operation removes glare from the first target region by scanning each pixel in the first target region to remove small and isolated glare. The matrix pixels can be any matrix shape, such as circular, square, or elliptical, depending on the shape of the glare. Preferably, square matrices such as 3×3, 5×5, or 7×7 are used. Then, the original image I and the first image S are compared... erode The matrix is multiplied to obtain a second image I1, which retains the predicted restoration area A1. In this way, by converting the original image I to the HSV color space mode, the glare area A can be identified using only a value or brightness threshold.
[0048] At this point, the second image I1 can be converted back to RGB color space mode to obtain the repair area mask I. mask Alternatively, the region in the second image I1 whose brightness or luminance dimension is greater than the third preset value can be dilated more than once to expand the predicted restoration region A1, making the second image I1 become the third image S. dilate Then the third image S dilate Convert back to RGB color space mode to become the repair area mask I mask The value ranges from 0 to 255. In this embodiment, it is used to repair glare area A. The third preset value of the brightness dimension or luminance dimension can be set to 210. The dilation operation can be performed by expanding the matrix pixels. In this way, the pixels of the predicted repair area A1 are scanned one by one to compensate for the area lost due to the erosion operation. The matrix pixels can also adopt any matrix shape such as circular matrix, square matrix, elliptical matrix, etc., which can be determined according to the shape of the glare. Preferred square matrices such as 3×3, 5×5, and 7×7 can be used.
[0049] Since the predicted repair area in this embodiment is the predicted glare area, the repair area mask I mask The initial RGB pixel values are usually higher than the average RGB pixel values of the overall original image I, therefore the repair area mask I can be used. mask The initial RGB pixel values are set higher than the average RGB pixel values of the original image I. For example, they are set to be higher than or equal to 1.75 times the pixel values around the glare region A of the original image I, which can improve the iterative convergence speed of the subsequent restoration algorithm. Then, the restoration area mask I... mask Execute the repair algorithm to repair the mask I of the area. mask The location corresponding to the predicted repair area A1 is set as the mask area to be repaired Ω, and the area boundary is set. The repair algorithm is based on the region boundary. A diffusion process is performed to fill the mask area Ω to be repaired, thereby removing glare. After one or more iterations, a repaired image I with glare removed is output. inpaint Image restoration I inpaint like Figure 4A As shown, in the repaired image I inpaint In the image, the transparent acrylic tube 10 after glare removal and the simulated endoscope tube 20 inside are visible. The restored image I at this time... inpaint There is still a shaded area B.
[0050] Specifically, the repair algorithm is to clear the mask in the repair area. mask The color information of the image is used to obtain the initial mask region Ω to be repaired. The mask region Ω is then divided into many smaller repair regions, which are the boundaries of the region. The mask area Ω to be repaired is gradually filled by repeatedly convolving with a diffusion kernel, and the values obtained by multiplying neighboring pixels with the diffusion kernel matrix are added together to fill in the gaps. You can select a boundary with a thickness of more than one pixel and check if it belongs to the inscription domain. Whether the pixel did not exceed a threshold during the previous iteration can be used to independently control the number of iterations for each repair domain, or the number of iterations can be specified.
[0051] See Figure 5A As shown, the diffusion kernel can be a Gaussian kernel. The Gaussian kernel is used to convolve the mask region Ω to be repaired, i.e., the kernel weight is 0, and the weighted average of the pixel neighborhood is calculated iteratively. See also... Figure 5B As shown, alternatively, the diffusion kernel can be a weighted average kernel, which is used to convolve the mask region Ω to be repaired. This involves setting the kernel weight to 0 and iterating through the weighted average of the surrounding pixels. See also... Figure 5C and Figure 5D As shown, during the iteration process, the repair domain can be changed to one or more different sizes as needed. With different diffusion nuclei to integrate different repair domains Advantages of diffusion nuclei.
[0052] The above embodiments are for the purpose of removing the glare area A of the original image I to achieve accurate image restoration during training for endoscopic examination or endoscopic surgery.
[0053] See Figure 6 As shown, after removing the glare area A from the original image I, the shadow area B from the original image I is further removed. The resulting restored image I after glare removal is shown. inpaintAs a new original image I, it is also represented in HSV color space mode. The processor searches for more than one candidate region in the new original image I whose saturation dimension does not reach the set value, thereby filtering out shadow candidate regions that are not oversaturated, with a range of 0 to 255. Since the saturation of shadows is low, the saturation setting value for detecting shadows can be set between 0 and 100. In dark environments, the saturation dimension setting value can be set to 20. Then, in the above shadow candidate regions, a second target region in the new original image I with a brightness dimension less than the second preset value is searched, with a range of 0 to 255. The second preset value is between 0 and 50. Then, the erosion operation, dilation operation, and repair algorithm as described in the previous embodiment are executed to remove the shadow region B, thereby obtaining a new repaired image I after removing glare and shadows. inpaint New restored images I inpaint like Figure 4B As shown, in the new restored image I inpaint The image displays a transparent acrylic tube 10 after glare and shadow removal, and a simulated endoscope tube 20 inside. In the dilation calculation step, areas of the second image I1 with a brightness or luminance dimension less than a fourth preset value are dilated more than once to expand the predicted repair area. When repairing shadows, the fourth preset value for the brightness dimension is 15. During the execution of the repair algorithm, since the predicted repair area is the predicted shadow area, the repair area mask I... mask The initial RGB pixel values are usually lower than the average RGB pixel values of the entire original image I, therefore the repair area mask I can be used. mask Setting the initial RGB pixel values to be lower than the average RGB pixel values of the original image I, for example, setting them to 0.2 times the pixel values around the shadow area of the original image I, can improve the iterative convergence speed of the subsequent repair algorithm.
[0054] See Figure 7 As shown, when the original image I only has shadows, the processor can first perform negative processing on the original image I to turn the shadows into glare and then repair the glare. After performing negative processing again, the repaired image I is output. inpaint .
[0055] It should be further clarified that the embodiments of the present invention are not limited to operating in the HSV color space. They can also be any color space with hue, saturation, and lightness (HSL), or the CIELAB color space, or the YUV color space with luminance, chroma, and chroma (YUV), or similar color spaces with lightness or brightness dimensions. Input raw image formats include, for example, CIE RGB, CIE XYZ (Yxy), CIE Yuv (uvw, YUCV, UVW), CIE U*V*W*, CIE L*a*b*, CIEL*u*v*, NTSC RGB, NTSC YIQ→NTSC TV broadcasting, NTSC Y CbCr→CCIR-601 video standard, IHS, LHS, VHS→perceptual color space, and various other image formats that can be converted to RGB or the aforementioned HSL, CIELAB, and YUV formats. Users can set the threshold for filtering glare or shadows based on the original image format and the selected color space with a lightness or luminance dimension. Therefore, as long as the method of this invention can be performed, any image format or color space with a lightness or luminance dimension is covered by this invention. Any color space can be converted to a color space with lightness, and the operation mode described in the embodiments belongs to the same mode. Therefore, this invention applies to, but is not limited to, the various color spaces with lightness or luminance described above.
[0056] The computer program for the above-mentioned method of repairing glare or shadows in images can be stored in the cloud for download or stored in computer-readable media.
[0057] Based on the above description of the embodiments, one can fully understand the operation, use, and effects of the present invention. However, the above embodiments are only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Simple equivalent changes and modifications made in accordance with the claims and description of the invention are all within the scope of the present invention.
Claims
1. A method for repairing glare or shadows in an image, characterized in that, Including: An input original image is represented in a color space mode with a lightness dimension or a brightness dimension, and the original image is defined as I; wherein, the aforementioned color space mode includes a color space HSV with hue, saturation, and lightness; When the original image I has glare, search for one or more first target regions in the original image I whose lightness or brightness dimension is greater than a first preset value; or, when the original image I has shadows, search for one or more second target regions in the original image I whose lightness or brightness dimension is less than a second preset value; wherein, in the HSV color space mode, first search for one or more candidate regions in the original image I whose saturation dimension does not reach a set value, and then search for the first target region or the second target region from the aforementioned candidate regions; the saturation dimension is in the range of 0 to 255, the set value for detecting glare is between 150 and 255, and the set value for detecting shadows is between 0 and 100; Perform one or more erosion operations on the first or second target region to obtain a predicted restoration region, making the original image I a first image, which is defined as S. erode Then compare the original image I with the first image S. erode A second image is obtained by matrix multiplication, which is defined as I1, and the second image I1 retains the predicted restoration area; When the predicted restoration area is a predicted glare area, the region of the second image I1 with a brightness dimension or luminance dimension greater than a third preset value is subjected to one or more dilation operations to expand the predicted restoration area; when the predicted restoration area is a predicted shadow area, the region of the second image I1 with a brightness dimension or luminance dimension less than a fourth preset value is subjected to one or more dilation operations to expand the predicted restoration area; thus, the second image I1 becomes a third image, which is defined as S. dilate Then the third image S dilate Convert back to RGB color space mode to become the mask for the repaired area. mask The erosion and dilation operations are both performed using matrix pixels. The matrix shape of the matrix pixels is determined from circular, square, and elliptical matrices based on the shape of the glare or shadow. The dilation operation scans the pixels of the predicted repair area one by one using matrix pixels to compensate for the area lost due to the erosion operation. Mask I for the repaired area mask Execute a repair algorithm to mask the repaired area. mask The location of the area to be repaired should be set as a mask area to be repaired, defined as Ω. A boundary of the area is defined as... Ω, the repair algorithm is based on the boundary of this region. The process involves a diffusion step Ω to fill the mask area to be repaired, thereby removing glare or shadows. After one or more iterations, a repaired image with glare or shadows removed is output, and this repaired image is defined as I. inpaint .
2. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, In a dark environment, the saturation dimension is set to 170 for glare detection and the saturation dimension is set to 20 for shadow detection.
3. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, In the HSV color space mode, the lightness dimension or the brightness dimension ranges from 0 to 255, the first preset value is between 200 and 255, and the second preset value is between 0 and 50.
4. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, In the HSV color space mode, with a range of 0 to 255, when the predicted repair area is a predicted glare area, the third preset value of the lightness dimension or brightness dimension is 210; when the predicted repair area is a predicted shadow area, the fourth preset value of the lightness dimension or brightness dimension is 15.
5. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, When the predicted repair area is the predicted glare area, mask the repair area I. mask The initial RGB pixel values are set to be higher than the average RGB pixel values of the original image I. When the predicted restoration area is a predicted shadow area, the mask I of the restoration area is set. mask The initial RGB pixel values are set to be lower than the average RGB pixel values of the original image I.
6. The method for repairing glare or shadows in an image as described in claim 5, characterized in that, When fixing glare, mask the area to be fixed. mask The initial RGB pixel values are set to 1.75 times the average RGB pixel values of the original image I; when repairing shadows, the mask I of the repair area is... mask The initial RGB pixel values are set to 0.2 times the average RGB pixel values of the original image I.
7. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, The repair algorithm removes the mask in the repair area. mask The color information of the image is used to obtain the initialized mask region Ω to be repaired. This mask region Ω is then divided into many smaller repair regions, which are the boundaries of the region. Ω is then repeatedly convolved using a diffusion kernel, and the values obtained by multiplying neighboring pixels with the diffusion kernel matrix are added back to fill the mask area Ω through a diffusion process.
8. The method for repairing glare or shadows in an image as described in claim 7, characterized in that, This repair area Ω represents a boundary of more than one pixel thickness. The iteration of each restoration region is independently controlled by checking whether the pixels belonging to this restoration region did not exceed a threshold during the previous iteration. The number of Ω iterations; or, specify the number of iterations.
9. The method for repairing glare or shadows in an image as described in claim 8, characterized in that, During the iteration process, the repair domain of different sizes was changed as needed. Ω and diffusion core to integrate different repair domains Advantages of Ω and diffused nuclei.
10. The method for repairing glare or shadows in an image as described in claim 7, characterized in that, The diffusion kernel is a Gaussian kernel. The Gaussian kernel is used to convolve the mask region Ω to be repaired, that is, the kernel weight is 0 and the weighted average of the pixel neighborhood is calculated.
11. The method for repairing glare or shadows in an image as described in claim 7, characterized in that, The diffusion kernel is a weighted average kernel. The mask region Ω to be repaired is convolved with the weighted average kernel, that is, the kernel weight is 0 and the weighted average of the surrounding pixels is calculated.
12. The method for repairing glare or shadows in an image as described in claim 1, characterized in that, When the original image I has both glare and shadows, the first target region and the second target region are searched to remove the glare and shadows; when the original image I only has shadows, the original image I is negative-processed to make the shadows into glare and then the glare is removed, and then the repaired image I is processed... inpaint Perform negative processing.
Citation Information
Patent Citations
An image restoration method, an image restoration system, and a flat panel detector
CN109712084B
Method for removing colored glare
CN109993714A
OpenCV-based picture watermark automatic identification processing method and system
CN112967166A
System and method for automatic flash removal from images
US8730356B2
Method and system for automatically removing glare regions
WO2022001615A1
