A method for removing the shadow of an image

Abstract

An embodiment of the present application discloses a method for removing the shadow of an image, which relates to the technical field of image processing and is used to accurately eliminate the cross-striped shadow of an image. The method includes: obtaining an original image with a cross-striped shadow; converting the original image into a grayscale image and performing low-pass filtering on the grayscale image to determine a first light field map and a second light field map; wherein, the first light field map is used to characterize the brightness information of each pixel point of the original image under the condition of removing the interference of the cross-striped shadow, and the second light field map is used to characterize the brightness information of each pixel point of the original image under the condition of retaining the interference of the cross-striped shadow; according to the first light field map and the second light field map, determining a shadow compensation matrix, the shadow compensation matrix is used to record the shadow compensation coefficients corresponding to each pixel point in the original image; based on the shadow compensation matrix, performing shadow correction processing on the original image to obtain an image with the cross-striped shadow removed.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a method for removing shadows from an image. Background Art

[0002] With the growth of network bandwidth and the rise of multimedia applications, images, as a kind of multimedia information, are increasingly used to express content and carry information. However, when acquiring images, due to the influence of various conditions, such as line aging, electromagnetic interference, transmission failure, or excessive light in the shooting area, the acquired images have a phenomenon of quality degradation. For example, the horizontal stripe shadow of the image is one of them. It is a quality degradation phenomenon caused by imaging conditions, which will cause the amount of information reflected by the target to be missing or interfered, reduce the accuracy of image interpretation, and affect various quantitative analysis and applications of the image. Therefore, how to eliminate the horizontal stripe shadow in the image is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiment of the present application provides a method for removing shadows from an image, which is used to remove horizontal shadows from an image.

[0004] In a first aspect, an embodiment of the present application provides a method for removing shadows from an image, the method comprising: acquiring an original image with horizontal stripe shadows; converting the original image into a grayscale image, and performing low-pass filtering on the grayscale image to determine a first light field image and a second light field image; wherein the first light field image is used to characterize the brightness information of each pixel of the original image when the interference of the horizontal stripe shadows is removed, and the second light field image is used to characterize the brightness information of each pixel of the original image when the interference of the horizontal stripe shadows is retained; determining a shadow compensation matrix based on the first light field image and the second light field image, the shadow compensation matrix being used to record shadow compensation coefficients corresponding to each pixel in the original image; and performing shadow correction processing on the original image based on the shadow compensation matrix to obtain an image with the horizontal stripe shadows eliminated.

[0005] The technical solution provided by the embodiments of the present application at least brings the following beneficial effects: Since the gray value of a pixel in a grayscale image can reflect the brightness information, after converting the original image into a grayscale image, the first light field image and the second light field image can be determined based on the grayscale image. Among them, the first light field image is used to characterize the brightness information of each pixel of the original image under the condition of removing the interference of the horizontal stripe shadow, and the second light field image is used to characterize the brightness information of each pixel of the original image under the condition of retaining the interference of the horizontal stripe shadow. In this way, based on the first light field image and the second light field image, the interference of the horizontal stripe shadow on the brightness of each pixel in the original image can be determined, and then the shadow compensation matrix for recording the shadow compensation coefficients corresponding to each pixel in the original image can be determined. The shadow compensation coefficient is used to correct the brightness of the pixel in the original image from the brightness under the interference of the horizontal stripe shadow to the brightness without the interference of the horizontal stripe shadow. Thus, based on the shadow compensation matrix, the original image is subjected to shadow correction processing to obtain an image with the horizontal stripe shadow removed. It can be seen that the technical solution provided by the embodiments of the present application can effectively and accurately remove the horizontal stripe shadow in the image.

[0006] In some embodiments, determining the first light field image based on the grayscale image includes: determining the height and width of the first filtering operator and the height and width of the second filtering operator; wherein, the height of the first filtering operator is greater than the width of the first filtering operator, and the height of the second filtering operator is less than the width of the second filtering operator; based on the first filtering operator, performing low-pass filtering on the grayscale image to obtain a processed grayscale image; based on the second filtering operator, performing low-pass filtering on the processed grayscale image to obtain the first light field image.

[0007] It should be understood that the shadow area in the image is the area with lower brightness. There is a large difference in brightness between the horizontal stripe shadow area and the non-horizontal stripe shadow area in the image. From the longitudinal view of the image, the brightness information of the horizontal stripe shadow area in the image is abrupt compared with the brightness information of its adjacent area. Therefore, performing low-pass filtering on the grayscale image based on the first filtering operator with a height greater than the width is to smooth the brightness information of the grayscale image longitudinally to eliminate the interference of the horizontal stripe shadow. Then, on this basis, performing low-pass filtering on the processed grayscale image based on the second filtering operator with a width greater than the height is to smooth the brightness information of the processed grayscale image transversely to further eliminate the interference of the horizontal stripe shadow. In this way, the finally obtained first light field image can be used to characterize the brightness information of each pixel of the original image under the condition of removing the interference of the horizontal stripe shadow.

[0008] In some embodiments, determining the height and width of the first filtering operator and the height and width of the second filtering operator includes: based on the original image, determining the height of the original image and the number of horizontal stripe shadows; calculating the ratio between the height of the original image and the number of horizontal stripe shadows; and based on the ratio, determining the height and width of the first filtering operator and the height and width of the second filtering operator.

[0009] It should be understood that the ratio between the height of the original image and the number of horizontal stripe shadows can reflect the maximum average height of the horizontal stripe shadows. Based on this, it is possible to determine the height and width of the first filtering operator and the height and width of the second filtering operator according to the maximum average height of the horizontal stripe shadows, so that the settings of the first filtering operator and the second filtering operator are adapted to the distribution of the horizontal stripe shadows in the original image, and thus the first light field map determined based on the first filtering operator and the second filtering operator is more accurate.

[0010] In some embodiments, determining the second light field map based on the grayscale image includes: determining the height and width of the third filtering operator; wherein the height of the third filtering operator is less than the width of the third filtering operator; and based on the third filtering operator, performing low-pass filtering on the grayscale image to obtain the second light field map.

[0011] It should be understood that the shadow area in the image is the area with lower brightness. There is a large difference in brightness between the horizontal stripe shadow area and the non-horizontal stripe shadow area in the image. Looking at the image longitudinally, the brightness information of the horizontal stripe shadow area in the image is mutated compared to the brightness information of its adjacent areas. Therefore, performing low-pass filtering on the grayscale image based on the third filtering operator with a width greater than the height is to smooth the brightness information of the grayscale image in the horizontal direction. However, this horizontal smoothing process cannot eliminate the interference of the horizontal stripe shadows. Therefore, the finally obtained second light field map can be used to characterize the brightness information of each pixel in the original image while retaining the interference of the horizontal stripe shadows.

[0012] In some embodiments, determining the height and width of the third filtering operator includes: based on the original image, determining the height of the original image and the number of horizontal stripe shadows; calculating the ratio between the height of the original image and the number of horizontal stripe shadows; and based on the ratio, determining the height and width of the third filtering operator.

[0013] It should be understood that the ratio between the height of the original image and the number of horizontal stripe shadows can reflect the maximum average height of the horizontal stripe shadows. Based on this, it is possible to determine the height and width of the third filtering operator according to the maximum average height of the horizontal stripe shadows, so that the setting of the third filtering operator is more adapted to the distribution of the horizontal stripe shadows in the original image, and thus the second light field map determined based on the third filtering operator is more accurate.

[0014] In some embodiments, determining the shadow compensation matrix according to the first light field map and the second light field map includes: modifying the pixel values of the pixel points with pixel value 0 in the second light field map to 1 to obtain a modified second light field map; performing a point division operation on the first light field map and the modified second light field map to obtain the shadow compensation matrix.

[0015] It should be understood that in order to avoid division-by-zero errors in the above point division process, the modified second light field map does not include points with pixel value 0. Modifying the pixel values of the pixel points with pixel value 0 in the second light field map to 1 as the modified second light field map can avoid the situation of division by zero. The pixel values of the pixel points in the first light field map are used to represent the brightness information of the pixel points under the condition of eliminating the interference of horizontal stripe shadows, and the pixel values of the pixel points in the modified second light field map are used to represent the brightness information of the pixel points under the condition of retaining the interference of horizontal stripe shadows. Therefore, performing a point division process on the first light field map and the second light field map can obtain the ratio between the brightness information of each pixel point under the condition of eliminating the interference of horizontal stripe shadows and the brightness information of the pixel point under the condition of retaining the interference of horizontal stripe shadows, and this ratio is the shadow compensation coefficient.

[0016] In some embodiments, performing shadow correction processing on the original image based on the shadow compensation matrix to obtain an image with horizontal stripe shadows eliminated includes: multiplying the pixel values of each color channel of each pixel point in the original image by the shadow compensation coefficient corresponding to the pixel point to obtain the corrected pixel values of each color channel of each pixel point in the original image; performing a range limitation process on the corrected pixel values of each color channel of each pixel point in the original image to determine the image with horizontal stripe shadows eliminated, and the range limitation process is used to limit the pixel values of each color channel of the pixel point within a preset value range.

[0017] It should be understood that when performing shadow correction processing on the original image, since the corrected pixel values of each color channel of each pixel point are obtained by multiplying the pixel values of each color channel of each pixel point by the shadow compensation coefficient corresponding to the pixel point, the corrected pixel values may exceed the displayable range of the image. For example, the pixel value range of each channel in an RGB image should be kept between 0 and 255, so the displayable range of its pixel values is 0 - 255. Therefore, it is necessary to perform a range limitation on the corrected pixel values so that the corrected image can be displayed normally.

[0018] In some embodiments, performing a range limitation process on the corrected pixel values of each color channel of each pixel point in the original image to determine the image with horizontal stripe shadows eliminated includes: based on the constraint conditions of the preset value range, using a clipping function to process the corrected pixel values of each color channel of each pixel point in the original image to determine the image with horizontal stripe shadows eliminated.

[0019] It should be understood that by using a clipping function, the corrected pixel values outside the preset value range can be forced to be corrected to the endpoint values of the preset value range closest to the corrected pixel values, so as to limit the corrected pixel values of each color channel of the pixel points within the preset value range, so that the corrected image can be normally displayed.

[0020] In some embodiments, the above method further includes: fusing the original image with the image eliminating the horizontal stripe shadow to obtain a fused image.

[0021] It should be understood that considering that there is a certain loss of details in the image eliminating the horizontal stripe shadow compared with the original image, the original image can be fused with the image eliminating the horizontal stripe shadow to obtain a fused image, so that the details of the original image are still retained after the elimination of the horizontal stripe shadow.

[0022] In some embodiments, fusing the original image with the image eliminating the horizontal stripe shadow to obtain a fused image includes: performing weighted summation of the pixel values of each color channel for the pixel points at the same position in the original image and the image eliminating the horizontal stripe shadow to generate a fused image; wherein, the sum of the weight coefficients corresponding to the target color channel of the pixel points in the original image and the weight coefficients corresponding to the target color channel of the pixel points at the same position in the image eliminating the horizontal stripe shadow is 1, and the target color channel is any color channel in the color space; when the pixel value of the target color channel of the pixel points in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel points in the original image and its corresponding weight coefficient are positively correlated.

[0023] It should be understood that when the pixel value of the target color channel of the pixel points in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel points in the original image and its corresponding weight coefficient are positively correlated. In this way, the larger the pixel value of the target color channel of the pixel points in the original image, the smaller the probability that the pixel points are in the horizontal stripe shadow area. Therefore, its corresponding weight coefficient is larger, so that more information of the pixel points in the original image can be retained in the fused image, thus achieving the purpose of retaining the detail features of the original image.

[0024] In some embodiments, the pixel value of the target color channel of the pixel points in the original image and its corresponding weight coefficient satisfy the following relationship:

[0025]

[0026] Wherein, res represents the pixel value of the target color channel of a pixel point in the original image, wet represents the weight coefficient corresponding to the target color channel of a pixel point in the original image, thr1 represents the first weight value, thr2 represents the second weight value, thr1 is less than thr2, C1 represents the first pixel value, and C2 represents the second pixel value.

[0027] In a second aspect, the present application provides an apparatus for removing stripes shadow from an image, including: an image acquisition module, configured to acquire an original image with stripes shadow; a light field determination module, configured to convert the original image into a grayscale image and perform a low-pass filtering process on the grayscale image to determine a first light field map and a second light field map; wherein, the first light field map is used to characterize the brightness information of each pixel point in the original image without the interference of the stripes shadow, and the second light field map is used to characterize the brightness information of each pixel point in the original image with the interference of the stripes shadow retained; a shadow elimination module, configured to determine a shadow compensation matrix according to the first light field map and the second light field map, where the shadow compensation matrix is used to record the shadow compensation coefficients corresponding to each pixel point in the original image; and the above-mentioned shadow elimination module is further configured to perform a shadow correction process on the original image based on the shadow compensation matrix to obtain an image with the stripes shadow removed.

[0028] In some embodiments, the above-mentioned light field determination module is specifically configured to determine the height and width of a first filtering operator and the height and width of a second filtering operator; wherein, the height of the first filtering operator is greater than the width of the first filtering operator, and the height of the second filtering operator is less than the width of the second filtering operator; perform a low-pass filtering process on the grayscale image based on the first filtering operator to obtain a processed grayscale image; and perform a low-pass filtering process on the processed grayscale image based on the second filtering operator to obtain the first light field map.

[0029] In some embodiments, the above-mentioned light field determination module is specifically configured to determine the height of the original image and the number of stripes shadows based on the original image; calculate the ratio between the height of the original image and the number of stripes shadows; and determine the height and width of the first filtering operator and the height and width of the second filtering operator based on the ratio.

[0030] In some embodiments, the above-mentioned light field determination module is specifically configured to determine the height and width of a third filtering operator; wherein, the height of the third filtering operator is less than the width of the third filtering operator; perform a low-pass filtering process on the grayscale image based on the third filtering operator to obtain the second light field map.

[0031] In some embodiments, the above-mentioned light field determination module is specifically configured to determine the height of the original image and the number of stripes shadows based on the original image; calculate the ratio between the height of the original image and the number of stripes shadows; and determine the height and width of the third filtering operator based on the ratio.

[0032] In some embodiments, the above-mentioned shadow elimination module is specifically configured to modify the pixel values of the pixel points with pixel value 0 in the second light field map to 1 to obtain a corrected second light field map; perform a point division operation on the first light field map and the corrected second light field map to obtain a shadow compensation matrix.

[0033] In some embodiments, the above-mentioned shadow elimination module is specifically configured to multiply the pixel values of each color channel of each pixel point in the original image by the shadow compensation coefficient corresponding to the pixel point to obtain the corrected pixel values of each color channel of each pixel point in the original image; perform a range limitation process on the corrected pixel values of each color channel of each pixel point in the original image to determine an image with horizontal stripe shadows eliminated, and the range limitation process is used to limit the pixel values of each color channel of the pixel point within a preset value range.

[0034] In some embodiments, the above-mentioned shadow elimination module is specifically configured to process the corrected pixel values of each color channel of each pixel point in the original image by using a clipping function based on the constraint conditions of the preset value range to determine an image with horizontal stripe shadows eliminated.

[0035] In some embodiments, the above-mentioned shadow elimination module is further configured to perform a fusion process on the original image and the image with horizontal stripe shadows eliminated to obtain a fused image.

[0036] In some embodiments, the above-mentioned shadow elimination module is specifically configured to perform a weighted sum of the pixel values of each color channel on the pixel points at the same position in the original image and the image with horizontal stripe shadows eliminated to generate a fused image; wherein, the sum of the weight coefficients corresponding to the target color channel of the pixel point in the original image and the weight coefficients corresponding to the target color channel of the pixel point at the same position in the image with horizontal stripe shadows eliminated is 1, and the target color channel is any one color channel in the color space; when the pixel value of the target color channel of the pixel point in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient are positively correlated.

[0037] In some embodiments, the pixel value of the target color channel of the pixel point in the above-mentioned original image and its corresponding weight coefficient satisfy the following relationship:

[0038]

[0039] Wherein, res represents the pixel value of the target color channel of the pixel point in the original image, wet represents the weight coefficient corresponding to the target color channel of the pixel point in the original image, thr1 represents the first weight value, thr2 represents the second weight value, thr1 is less than thr2, C1 represents the first pixel value, and C2 represents the second pixel value.

[0040] In a third aspect, the present application provides a device for removing shadows from an image, including: a memory and a processor; the memory and the processor are coupled; the memory is used for storing computer program code, and the computer program code includes computer instructions; wherein, when the processor executes the computer instructions, the device for removing shadows from the image is caused to execute the task processing method as described in the first aspect and any of its possible design manners.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium includes computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the method provided in the first aspect and its possible implementation manners.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product including computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the method provided in the above-mentioned first aspect and its possible implementation manners.

[0043] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium. Among them, the computer-readable storage medium may be packaged together with the processor of the controller, or may be packaged separately from the processor of the controller, and the present application does not limit this.

[0044] For the specific descriptions of the second aspect to the fifth aspect and their various implementation manners in the present application, reference may be made to the detailed descriptions in the first aspect and its various implementation manners. For the beneficial effects of the second aspect to the fifth aspect and their various implementation manners, reference may be made to the analysis of the beneficial effects in the first aspect and its various implementation manners, and details are not described herein again. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 FIG. is a schematic diagram of the progressive exposure of a sensor according to some embodiments;

[0046] Figure 2 FIG. is a flowchart of a method for removing shadows from an image according to some embodiments Figure 1 ;

[0047] Figure 3 FIG. is a flowchart of a method for removing shadows from an image according to some embodiments Figure 2 ;

[0048] Figure 4 FIG. is a schematic diagram of the process of image convolution operation according to some embodiments;

[0049] Figure 5 FIG. is a flowchart of a method for removing shadows from an image according to some embodiments Figure 3 ;

[0050] Figure 6 Flow schematic of a method for removing shadow of an image according to some embodiments Figure 4 ;

[0051] Figure 7 Schematic diagram of a weight coefficient curve of pixel values according to some embodiments;

[0052] Figure 8 Structural schematic of a device for eliminating shadow of an image according to some embodiments Figure 1 ;

[0053] Figure 9 Structural schematic of another device for eliminating shadow of an image according to some embodiments Figure 2 。 Detailed implementation manners

[0054] In this article, the term "and / or" only describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. These three situations.

[0055] Terms such as "first" and "second" in the specification and drawings of this application are used to distinguish different objects or different processes for the same object, rather than to describe a specific order of the object.

[0056] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include other unlisted steps or units, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0057] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0058] In the description of this application, unless otherwise specified, the meaning of "a plurality of" refers to two or more.

[0059] For the convenience of understanding, relevant concepts involved in this application are briefly introduced first.

[0060] Horizontal stripe shadow: A type of stripe noise, characterized mainly by its directionality and penetrability. In an image with horizontal stripe shadows, there is one or more groups of stripes that run from one end of the image to the other. If there are multiple groups of stripes, the directions of the individual stripes are the same and parallel to each other. It is not difficult to understand that due to the penetrability of the horizontal stripe shadows, a relatively large portion of the content in the image will be blocked, reducing the interpretation accuracy of the image.

[0061] Optical camera: An optical camera includes an optical system and an image sensor. Among them, the optical system refers to a system composed of various optical elements such as lenses, mirrors, prisms, and apertures arranged in a certain order (which can also be called a lens). The image sensor can be a charge coupled device image sensor (CCD), a super CCD, or a complementary metal oxide semiconductor (CMOS) sensor, etc.

[0062] Exposure: Refers to the process by which light reaches the photosensitive medium.

[0063] CMOS sensor: CMOS sensors usually use rolling shutter exposure, which can also be called progressive exposure. Figure 1 Is a schematic diagram of progressive exposure. As Figure 1 Shown, taking a CMOS sensor with a photosensitive unit array of 6 rows and 6 columns as an example, the first row, the second row, and the third row, etc. can all be exposed sequentially at different exposure start time points ( Figure 1 Taking the row of photosensitive units filled with black as an example to show the row that is currently being exposed).

[0064] Gray level: Brightness is a value indicating the light and dark of an image, that is, the color depth of points in a black and white image, generally ranging from 0 to 255, with white being 255 and black being 0. The area between white and black is divided into several levels according to a logarithmic relationship, and each level is called a gray level. In one example, the gray level is divided into 256 levels. The gray level value refers to the brightness of a single pixel point, and the larger the gray level value, the brighter it is.

[0065] High-frequency information: Refers to areas where the brightness or gray level of an image changes drastically, such as the image contour.

[0066] Low-frequency information: Refers to areas where the brightness or gray level of an image changes smoothly, such as areas of continuous color blocks.

[0067] Low-pass filtering: An image processing method that can enhance the low-frequency information in an image and suppress the high-frequency information in the image.

[0068] Color space: Also known as color space, color model or color system, it is used to simplify color specifications in a generally acceptable way under certain specific standards. In essence, a color space is a specification of a coordinate system and subspaces, and each color in the system is represented by a single point. Commonly used color spaces include: RGB (Red-Green-Blue) space, CMY (Cyan-Magenta-Yellow) space, and HSI (Hue-Saturation-Intensity) space, etc.

[0069] YUV: A color encoding method. In the YUV image color space, Y represents the brightness component, U represents the hue component, and V represents the saturation component. YUV belongs to the color and brightness separation color space. The brightness component Y in the YUV image color space is separated from the hue component U and the saturation component V. If there is only the brightness component Y in the image, the image is a black and white grayscale image.

[0070] The above is an introduction to some concepts involved in the embodiments of the present application, which will not be repeated below.

[0071] As described in the background art, when acquiring an image, due to the influence of various conditions, horizontal stripes and shadows may appear in the acquired image, thereby reducing the interpretation accuracy of the image and affecting various quantitative analysis and applications of the image. Therefore, how to eliminate horizontal stripes and shadows in the image is a technical problem that needs to be solved urgently.

[0072] In this regard, an embodiment of the present application provides a shadow removal method for an image, which obtains brightness information of an original image based on a grayscale image of the original image, and determines a first light field image and a second light field image based on the brightness information included in the grayscale image. The first light field image is used to characterize the brightness information of each pixel of the original image when the interference of the horizontal stripe shadow is removed, and the second light field image is used to characterize the brightness information of each pixel of the original image when the interference of the horizontal stripe shadow is retained. In this way, based on the first light field image and the second light field image, the interference of the horizontal stripe shadow on the brightness of each pixel in the original image can be determined, and then the shadow compensation matrix for recording the shadow compensation coefficient corresponding to each pixel in the original image can be determined. The shadow compensation coefficient is used to correct the brightness of the pixel in the original image from the brightness under the interference of the horizontal stripe shadow to the brightness under the interference of the horizontal stripe shadow. Thus, based on the shadow compensation matrix, the original image is subjected to shadow correction processing to obtain an image with the horizontal stripe shadow eliminated. It can be seen that the technical solution provided by the embodiment of the present application can effectively and accurately remove the horizontal stripe shadow in the image.

[0073] The shadow removal method for images provided by the embodiments of the present application can be executed by a shadow removal device for images. For example, the shadow removal device for images can be a server. For another example, the shadow removal device for images can be an electronic chip with image processing capabilities, such as a GPU (Graphics Processing Unit); for another example, the shadow removal device for images can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, etc. Optionally, the shadow removal device for images can have a shooting function to obtain in real time an image of the cross-striped shadow to be processed. Optionally, the shadow removal device for images can also have a storage function to store the image after processing the cross-striped shadow or the image of the unprocessed cross-striped shadow. Optionally, the shadow removal device for images can also be connected to the Internet. The present application does not impose special restrictions on the specific form of the shadow removal device for images. In the following, the shadow removal device for images being a server is taken as an example for illustration.

[0074] The shadow removal method for images provided by the embodiments of the present application can be applied to different cross-striped removal scenarios. For example, cross-striped shadow removal from infrared images, cross-striped shadow removal from remote sensing images, removal of moiré fringes in images, etc. The present application does not impose specific restrictions on the application scenarios of the shadow removal method for images.

[0075] For ease of understanding, the following specifically introduces the shadow removal method for images provided by the present application with reference to the accompanying drawings.

[0076] The present application proposes a shadow removal method for images, which is applied to the field of image processing technology, such as Figure 2 As shown, taking the execution entity of the shadow removal method for images as a server as an example, the shadow removal method for images includes the following steps S101 to step S104:

[0077] S101. Obtain an original image with a cross-striped shadow.

[0078] In some embodiments, if the server establishes a data connection with the camera or the server is integrated with the camera, the server obtains the image information to be processed transmitted in real time by the camera, and obtains an original image with a cross-striped shadow from the image information to be processed.

[0079] In some other embodiments, the server establishes a connection with the storage device or the server is integrated with the storage device. If the storage device contains the image information to be processed, the server obtains the original image with horizontal stripe shadows from the image information to be processed.

[0080] In some embodiments, considering that not all the images included in the above-mentioned image information to be processed have image horizontal stripes, the images included in the above-mentioned image information to be processed can be detected for horizontal stripe shadows to identify whether there are horizontal stripe shadows in the images, so as to obtain the original images with horizontal stripe shadows.

[0081] Optionally, the server performs horizontal stripe shadow detection on all the images included in the image information to be processed to ensure that all the original images with horizontal stripe shadows are obtained.

[0082] Optionally, the server performs horizontal stripe shadow detection on some of the images included in the image information to be processed to save the computing amount. For example, when the server establishes a data connection with the camera or the server is integrated with the camera, only a few frames of the images captured by the camera may occasionally have horizontal stripe shadows. Therefore, it can be considered not to perform horizontal stripe shadow detection on each image, but to perform horizontal stripe shadow detection on the images captured by the camera every preset time or preset number of image frames, thereby saving the computing amount of the server. Another example is that when the server establishes a data connection with the camera or the server is integrated with the camera, if the image sensor used by the camera is a CMOS sensor, when there is an artificial light source in the captured area and the ambient light brightness in the captured area exceeds the ambient light brightness threshold, a horizontal stripe shadow detection is performed on the images captured by the camera. It should be understood that the artificial light source works according to a certain alternating current frequency, and when the ambient light brightness is too high, the exposure time of the CMOS sensor is also short, and it may occur that the exposure time of the CMOS sensor is less than an integer multiple of half of the alternating current period, thus causing horizontal stripe shadows in the images captured by the camera. Therefore, a horizontal stripe shadow detection can be performed when there is an artificial light source in the captured area and the ambient light brightness in the captured area exceeds the ambient light brightness threshold.

[0083] It should be noted that the above detection conditions for horizontal stripe shadows are only exemplary descriptions, and the specific detection conditions may be related to the acquisition method of the image information to be processed. The present application does not specifically limit the conditions for the server to perform horizontal stripe shadow detection on some of the images included in the image information to be processed.

[0084] In some embodiments, the above-mentioned detection of horizontal stripe shadows in the image can be based on manual judgment of whether there are horizontal stripe shadows in the image, or based on a horizontal stripe recognition algorithm, such as identifying whether there are horizontal stripe shadows in the image according to threshold segmentation, image frequency domain analysis, or image filtering.

[0085] Taking image frequency domain analysis as an example, in some examples, the steps for identifying horizontal stripes based on image frequency domain analysis are as follows:

[0086] S1011. Obtain the grayscale image of the image to be identified.

[0087] S1012. Perform a two-dimensional Fourier transform on the grayscale image of the image to be identified to obtain the frequency spectrum of the image to be identified.

[0088] S1013. Project the frequency spectrum of the image to be identified in the X-axis direction and the Y-axis direction.

[0089] Among them, the X-axis direction is perpendicular to the Y-axis direction, the X-axis direction is parallel to the width of the image, and the Y-axis direction is parallel to the height of the image.

[0090] S1014. Calculate the variance of the values of the frequency spectrum of the image to be identified in the X-axis direction and the variance of the values in the Y-axis direction.

[0091] S1015. If the ratio of the variance of the pixel values of the frequency spectrum of the image to be identified in the X-axis direction to the variance of the pixel values in the Y-axis direction is greater than a preset threshold, determine whether there is a horizontal stripe shadow in the image to be identified.

[0092] It should be understood that if there is a horizontal stripe shadow in the image to be identified, that is, there is horizontal stripe noise, then in the frequency spectrum of the image to be identified, there is a peak after projection on the X-axis, and the projection effect on the Y-axis is relatively smooth. Therefore, it is possible to determine whether there is a horizontal stripe shadow based on the relationship between the variance of the values of the frequency spectrum of the image to be identified in the X-axis direction and the variance of the values in the Y-axis direction.

[0093] It should be noted that the above algorithm for determining whether there is a horizontal stripe shadow in an image through image frequency domain analysis is only an example, and the present application does not make specific limitations on this.

[0094] S102. Convert the original image into a grayscale image and perform low-pass filtering on the grayscale image to determine the first light field image and the second light field image.

[0095] Among them, the first light field image is used to represent the brightness information of each pixel point of the original image under the condition of removing the interference of horizontal stripe shadows, and the second light field image is used to represent the brightness information of each pixel point of the original image under the condition of retaining the interference of horizontal stripe shadows.

[0096] In some embodiments, the original image is converted into a grayscale image according to the image format of the original image. Exemplarily, the single image channel of the original image can be used as the grayscale image of the original image, or the original image can be converted into a grayscale image according to the mapping relationship between the respective color components of the pixel points of the original image and the grayscale values.

[0097] For example, for an image in YUV format, its Y component is the luminance component. The image channel corresponding to the Y component can be used as a grayscale image, or the grayscale value corresponding to the image can be determined according to the mapping relationship between the Y component, U component, and V component of each pixel and the grayscale value of that pixel.

[0098] For another example, for an infrared image, since the infrared image itself is a single-channel image, the infrared image itself can be used as a grayscale image. For another example, for an image in RGB format, the single-channel image corresponding to the R component, G component, or B component can be used as the grayscale image of the image, or the grayscale value corresponding to the image can be determined according to the mapping relationship between the R component, G component, and B component of each pixel and the grayscale value of that pixel. Taking the original image as an RGB image as an example, if the R component of the i-th pixel of the original image is R i and the G component is G i and the B component is B i , and the corresponding grayscale value is I i , then the grayscale value of this pixel can satisfy the following relationship:

[0099] I i = R i *0.299 + G i *0.587 + B i *0.114

[0100] Alternatively, the grayscale value of this pixel can satisfy the following relationship:

[0101] I i = (R i + G i + B i ) / 3

[0102] Alternatively, the grayscale value of this pixel can satisfy the following relationship:

[0103] I i = (30 * R i + 59 * G i + 11 * B i ) / 100

[0104] In addition, there may be other mapping relationships between the respective color components and the grayscale value of this RGB pixel. It should be understood that the above methods for determining the grayscale image are only examples, and the present application does not specifically limit the algorithm for converting the original image into a grayscale image.

[0105] In some embodiments, before converting the original image into a grayscale image or after converting the original image into a grayscale image, preprocessing may also be performed on the original image itself or on the grayscale image of the original image. Exemplarily, smoothing filtering may be performed on the original image or the grayscale image of the original image, such as median filtering, bilateral filtering, Gaussian filtering, mean filtering, etc., so as to remove image noise before determining the first light field map and the second light field map, thereby improving the accuracy of determining the first light field map and the second light field map. It should be understood that the present application does not specifically limit the algorithm for image filtering.

[0106] In some embodiments, as Figure 3 shown, the above determination of the first light field map based on the grayscale image may be specifically implemented as the following steps S1021a to S1023a:

[0107] S1021a. Determine the height and width of the first filtering operator and the height and width of the second filtering operator.

[0108] Among them, the height of the first filtering operator is greater than the width of the first filtering operator, and the height of the second filtering operator is less than the width of the second filtering operator. It should be understood that since the height of the first filtering operator is greater than the width of the first filtering operator, when filtering the image according to the first filtering operator, filtering in the longitudinal direction of the image is mainly achieved; since the height of the second filtering operator is less than the width of the second filtering operator, when filtering the image according to the second filtering operator, filtering in the transverse direction of the image is mainly achieved.

[0109] In some examples, step S1021a may be specifically implemented as: based on the original image, determine the height of the original image and the number of horizontal stripe shadows; calculate the ratio between the height of the original image and the number of horizontal stripe shadows; based on the ratio, determine the height and width of the first filtering operator and the height and width of the second filtering operator.

[0110] Exemplarily, if the height of the original image is PicHigt and the number of horizontal stripe shadows of the original image is FlicNum, where the number of horizontal stripe shadows of the original image may be marked by a staff member or obtained according to a relevant stripe noise detection algorithm, and the present application does not limit this, then the height and width of the first filtering operator and the height and width of the second filtering operator satisfy the following relationship:

[0111] KrlHigt_1 = k 11 *PicHigt / FlicNum + b 11

[0112] KrlLen_1 = k 12 *KrlHigt_1 + b 12

[0113] KrlLen_2 = k21 *PicHigt / FlicNum + b 21

[0114] KrlHigt_2 = k 22 *KrlLen_2 + b 22

[0115] Wherein, the height of the first filtering operator is KrlHigt_1, the width of the first filtering operator is KrlLen_1, the height of the second filtering operator is KrlHigt_2, the width of the second filtering operator is KrlLen_2, k 11 、k 12 、k 21 、k 22 、b 11 、b 12 、b 21 、and b 22 are all constants.

[0116] For example, when k 11 takes 1, b 11 takes 40, k 12 takes 0.2, b 12 takes 0, k 21 takes 1, b 21 takes 40, k 22 takes 0.2, and b 22 takes 0, if the height of the original image is 1440 and the number of horizontal stripe shadows is 4, then the height of the first filtering operator is (1 * 1440 / 4 + 40), which is 400, and the width is (0.2 * 400 + 0), which is 80; the width of the second filtering operator is (1 * 1440 / 4 + 40), which is 400, and the height is (0.2 * 400 + 0), which is 80.

[0117] It should be understood that the ratio between the height of the original image and the number of horizontal stripe shadows can reflect the maximum average height of the horizontal stripe shadows. Based on this, it is possible to determine the height and width of the first filtering operator and the height and width of the second filtering operator according to the maximum average height of the horizontal stripe shadows, so that the settings of the first filtering operator and the second filtering operator are adapted to the horizontal stripe shadow distribution of the original image, thereby making the first light field map determined based on the first filtering operator and the second filtering operator more accurate.

[0118] S1022a. Based on the first filtering operator, perform low-pass filtering on the grayscale image to obtain the processed grayscale image.

[0119] Among them, low-pass filtering can enhance the low-frequency information in the image and suppress the high-frequency information in the image. Commonly used low-pass filtering methods include mean filtering, Gaussian filtering, median filtering, etc., and this application does not make specific restrictions on this. Taking the mean filtering of a grayscale image as an example, an implementation method of low-pass filtering is exemplarily given as follows:

[0120] For example, if the first filtering operator is denoted as kernel_1, the height of the first filtering operator is KrlHigt_1, and the width of the first filtering operator is KrlLen_1, then when performing mean filtering on the grayscale image according to the first filtering operator, the first filtering operator can be expressed as:

[0121]

[0122] Furthermore, after performing mean filtering processing on the grayscale image according to the first filtering operator, the processed grayscale image satisfies the following relationship:

[0123]

[0124] Among them, pic represents the grayscale image of the original image, ResTmp represents the processed grayscale image, and kernel_1 represents the first filtering operator.

[0125] It can be seen that the process of performing mean filtering on the grayscale image according to the first filtering operator is actually a process of performing convolution operation on the image with the first filtering operator as a convolution kernel. To further illustrate the process of performing convolution operation on the image according to the convolution kernel, a schematic diagram of the process of image convolution operation is exemplarily given as follows.

[0126] As Figure 4 shown, the original image 100 is a pixel value matrix with a resolution of 8*8, and the convolution kernel 101 is a numerical matrix of 3*3. The anchor point of the convolution kernel is set as the center point of the 3*3 matrix, that is, the numerical point 0.2 in the second row and second column of the convolution kernel 101. Then the process of performing convolution operation on the original image 100 according to the convolution kernel 101 is as follows: Place the anchor point 0.2 of the convolution kernel 101 at the position of a pixel point of the original image 100. At the same time, the other values in the convolution kernel 101 coincide with the pixels in the neighborhood of this pixel; Multiply the values in the convolution kernel 101 by the corresponding pixel values at the coincident positions in the original image 100, and add the products; Place the obtained result on the pixel corresponding to the anchor point; Traverse all pixel points of the original image 100.

[0127] Taking the example that the anchor point 0.2 of the convolution kernel 101 is placed at the pixel point in the third row and third column of the original image 100, and the convolution kernel 101 coincides with the shaded part of the original image 100, the corresponding convolution result is (65*0.1 + 98*0.1 + 123*0.1 + 65*0.1 + 96*0.2 + 115*0.1 + 63*0.1 + 91*0.1 + 107*0.1), which is 91.9. Considering that the convolution result image 102 is a filtered image, the convolution result 91.9 needs to be rounded, that is, 92 is taken, corresponding to the value at the second row and second column of the convolution result image 102. And so on until all pixel points of the original image 100 are traversed. Further, to make the height and width of the original image 100 and the convolution result image 102 consistent, pixel points can also be filled around the original image 100 before the convolution operation. For example, the original image 100 with a resolution of 8*8 is symmetrically filled into an image with a resolution of 10*10, so that the height and width of the convolution result image after the filled image undergoes convolution operation are still 8*8 for image alignment. The specific algorithm for filling the image pixel points in this application is not limited.

[0128] It should be understood that the shaded area in the image is the area with lower brightness. There is a large difference in brightness between the horizontal stripe shaded area and the non-horizontal stripe shaded area in the image. Looking at the image vertically, the brightness information of the horizontal stripe shaded area in the image is mutated compared to the brightness information of its adjacent areas. Therefore, based on the first filtering operator with a height greater than the width, performing low-pass filtering on the grayscale image is to perform smoothing processing on the brightness information of the grayscale image vertically to eliminate the interference of the horizontal stripe shadows.

[0129] S1023a. Based on the second filtering operator, perform low-pass filtering on the processed grayscale image to obtain the first light field image.

[0130] It should be understood that common low-pass filtering methods include mean filtering, Gaussian filtering, median filtering, etc., and this application does not make specific limitations on this. Taking the example of performing mean filtering on the processed grayscale image based on the second filtering operator below, an implementation method of low-pass filtering is exemplarily given.

[0131] For example, if the second filtering operator is denoted as kernel_2, and the height of the second filtering operator is KrlHigt_2 and the width of the second filtering operator is KrlLen_2, then when performing mean filtering on the processed grayscale image according to the second filtering operator, the second filtering operator can be expressed as:

[0132]

[0133] Further, after performing mean filtering on the processed grayscale image according to the second filtering operator, the obtained first light field image satisfies the following relationship:

[0134]

[0135] Among them, ResTmp represents the processed grayscale image, kernel_2 represents the second filtering operator, and LgtFlicClr represents the first light field image.

[0136] It should be understood that performing low-pass filtering on the processed grayscale image based on the second filtering operator with a width greater than the height is to perform smoothing processing on the luminance information of the processed grayscale image in the horizontal direction to further eliminate the interference of crosswise shadow. In this way, the finally obtained first light field image can be used to characterize the luminance information of each pixel point of the original image under the condition of removing the interference of crosswise shadow.

[0137] In some embodiments, as Figure 5 shown, the determination of the second light field image based on the grayscale image can be specifically implemented as the following steps S1021b to step S1022b:

[0138] S1021b. Determine the height and width of the third filtering operator.

[0139] Among them, the height of the third filtering operator is less than the width of the third filtering operator. It should be understood that the height of the third filtering operator is less than the width of the second filtering operator. Therefore, when filtering the image according to the third filtering operator, the filtering in the horizontal direction of the image is mainly realized.

[0140] In some examples, step S1021b can be specifically implemented as: based on the original image, determine the height of the original image and the number of crosswise shadows; calculate the ratio between the height of the original image and the number of crosswise shadows; based on the ratio, determine the height and width of the third filtering operator.

[0141] Exemplarily, if the height of the original image is PicHigt and the number of crosswise shadows of the original image is FlicNum, then the height and width of the third filtering operator satisfy the following relationship:

[0142] KrlLen_3 = k 31 *PicHigt / FlicNum + b 31

[0143] KrlHigt_3 = k 32 *KrlLen_3 + b 32

[0144] Among them, KrlHigt_3 represents the height of the third filtering operator, KrlLen_3 represents the width of the third filtering operator, k 31 、k 32 、b 31 、and b 32 are all constants.

[0145] For example, k 31 Take 1, b 31 Take 40, k 32 Take 0.2, b 32 Take 0. If the height of the original image is 1440 and the number of horizontal stripe shadows is 4, then the width of the third filtering operator is (1 * 1440 / 4 + 40), which is 400, and the height is (0.2 * 400 + 0), which is 80.

[0146] It should be understood that the ratio between the height of the original image and the number of horizontal stripe shadows can reflect the maximum average height of the horizontal stripe shadows. Based on this, it is possible to determine the height and width of the third filtering operator according to the maximum average height of the horizontal stripe shadows, so that the setting of the third filtering operator is more adapted to the distribution of the horizontal stripe shadows of the original image, thereby making the second light field map determined based on the third filtering operator more accurate.

[0147] S1022b. Based on the third filtering operator, perform low-pass filtering on the grayscale image to obtain the second light field map.

[0148] It should be understood that there are various implementation methods for the algorithm of image low-pass filtering, such as mean filtering, Gaussian filtering, median filtering, etc. This application does not make specific limitations on this. Taking the mean filtering of the grayscale image as an example, an implementation method of low-pass filtering is exemplarily given below.

[0149] For example, if the third filtering operator is denoted as kernel_3, and the height of the third filtering operator is KrlHigt_3 and the width is KrlLen_3, then when performing mean filtering on the grayscale image according to the third filtering operator, the third filtering operator can be expressed as:

[0150]

[0151] Furthermore, after performing mean filtering on the grayscale image according to the third filtering operator, the second light field map obtained satisfies the following relationship:

[0152]

[0153] Among them, pic represents the grayscale image of the original image, kernel_3 represents the third filtering operator, and LgtFlic represents the second light field map.

[0154] It should be understood that the shaded areas in the image are the areas with lower brightness. There is a significant difference in brightness between the horizontal striped shaded areas and the non-horizontal striped shaded areas in the image. Looking at the image vertically, the brightness information of the horizontal striped shaded areas in the image is mutated compared to the brightness information of their adjacent areas. Therefore, based on the third filtering operator with a width greater than the height, performing low-pass filtering on the grayscale image is to perform smoothing processing of the brightness information of the grayscale image horizontally. However, this horizontal smoothing process cannot eliminate the interference of the horizontal stripes. Therefore, the finally obtained second light field map can be used to represent the brightness information of each pixel in the original image while retaining the interference of the horizontal stripes.

[0155] It should be noted that in the method shown in this application, the first light field map can be determined first, and then the second light field map can be determined. Or the second light field map can be determined first, and then the first light field map can be determined. It is also possible to determine the first light field map and the second light field map simultaneously. This application does not make specific restrictions on this. In addition, when performing low-pass filtering with the first filtering operator, the second filtering operator, or the third filtering operator, pixel filling should be performed on the image to be filtered before each filtering, so that the heights and widths of the first light field map and the second light field map determined after filtering are the same. This application does not make restrictions on the specific algorithm for image pixel filling.

[0156] S103. Determine a shadow compensation matrix according to the first light field map and the second light field map.

[0157] Among them, the shadow compensation matrix is used to record the shadow compensation coefficients corresponding to each pixel point in the original image.

[0158] As a possible implementation, modify the pixel value of the pixel point with a pixel value of 0 in the second light field map to 1 to obtain a modified second light field map; perform a point division operation on the first light field map and the modified second light field map to obtain a shadow compensation matrix.

[0159] Among them, the first light field map, the second light field map, and the shadow compensation matrix have the same size.

[0160] Performing a point division operation on the first light field map and the modified second light field map means that for each pixel point in the first light field map matrix, divide the pixel value of this pixel point by the pixel value of the pixel point in the second light field map matrix that is in the same position as this pixel point, and use the result after the division operation as the shadow compensation coefficient at the same position on the shadow compensation matrix.

[0161] Exemplarily, the shadow compensation matrix satisfies the following relationship:

[0162] Cgain = LgtFlicClr. / LgtFlic_cor

[0163] Among them, Cgain represents the shadow compensation matrix, LgtFlicClr represents the first light field map, and LgtFlic_cor represents the corrected second light field map.

[0164] It should be understood that in order to avoid the division-by-zero error in the above division process, the corrected second light field map does not include points with a pixel value of 0. Instead, the pixel value of the pixel point with a pixel value of 0 in the second light field map is modified to 1 as the corrected second light field map, thus avoiding the situation of division by zero. The pixel value of the pixel point in the first light field map is used to represent the brightness information of the pixel point under the condition of eliminating the cross-stripe shadow interference, and the pixel value of the pixel point in the corrected second light field map is used to represent the brightness information of the pixel point under the condition of retaining the cross-stripe shadow interference. Therefore, by performing the division operation on the first light field map and the second light field map, the ratio between the brightness information of each pixel point under the condition of eliminating the cross-stripe shadow interference and the brightness information of the pixel point under the condition of retaining the cross-stripe shadow interference can be obtained, and this ratio is the shadow compensation coefficient.

[0165] S104. Based on the shadow compensation matrix, perform shadow correction processing on the original image to obtain an image with cross-stripe shadows eliminated. In one possible implementation, referring to Figure 6 , step S104 is specifically implemented as the following steps S1041 to S1042:

[0166] S1041. Multiply the pixel values of each color channel of each pixel point in the original image by the shadow compensation coefficient corresponding to the pixel point to obtain the corrected pixel values of each color channel of each pixel point in the original image.

[0167] Among them, the size of the original image is the same as that of the shadow compensation matrix.

[0168] Taking the original image as an RGB image as an example, the server can implement step S1041 according to the following formula:

[0169] R_0′ = R_0.*Cgain

[0170] G_0′ = G_0.*Cgain

[0171] B_0′ = B_0.*Cgain

[0172] Among them, Cgain represents the shadow compensation matrix, R_0 represents the pixel value matrix of the R component of the RGB image, G_0 represents the pixel value matrix of the G component of the RGB image, B_0 represents the pixel value matrix of the B component of the RGB image, R_0′ represents the corrected pixel value matrix of the R component of the RGB image, G_0′ represents the corrected pixel value matrix of the G component of the RGB image, and B_0′ represents the corrected pixel value matrix of the B component of the RGB image.

[0173] The symbol ".*" in the formula represents matrix dot multiplication operation. That is, for each color component of the RGB image, for each pixel point in the pixel value matrix of this color component, multiply the pixel value of this pixel point by the shadow compensation coefficient at the same position in the shadow compensation matrix, and use the result after the multiplication operation as the corrected pixel value at the same position in the corrected pixel value matrix of this color component.

[0174] S1042. Perform range limitation processing on the corrected pixel values of each color channel of each pixel point in the original image to determine the image with the horizontal stripe shadow removed. The range limitation processing is used to limit the pixel values of each color channel of the pixel point within a preset value range.

[0175] As a possible implementation manner, based on the constraint conditions of the preset value range, use the clipping function to process the corrected pixel values of each color channel of each pixel point in the original image to determine the image with the horizontal stripe shadow removed.

[0176] For example, if the preset value range of the corrected pixel values in the original image is set to (m, n), where m is less than n, the clipping function will force the corrected pixel values greater than m to be modified to m, and the corrected pixel values less than n to be modified to n.

[0177] Still taking the original image as an RGB image as an example, the server can limit the preset value range of the corrected pixel values of each color component of this image within [0 - 255] according to the following formula:

[0178] R′ = Clip(R_0′, 0, 255)

[0179] G′ = Clip(G_0′, 0, 255)

[0180] B′ = Clip(B_0′, 0, 255)

[0181] Wherein, R_0′ represents the corrected pixel value matrix of the R component of the RGB image, G_0′ represents the corrected pixel value matrix of the G component of the RGB image, B_0′ represents the corrected pixel value matrix of the B component of the RGB image, R′ represents the corrected pixel value matrix of the R component of the RGB image after range limitation, G′ represents the corrected pixel value matrix of the G component of the RGB image after range limitation, B′ represents the corrected pixel value matrix of the B component of the RGB image after range limitation, and Clip represents the clipping function.

[0182] As another possible implementation manner, based on the constraint conditions of the preset value range, map the corrected pixel values of each color channel of each pixel point in the original image proportionally to the pixel values within the preset value range.

[0183] Exemplarily, if the preset value range of the corrected pixel values in the original image is set to (m, n), where m is less than n, then for each color component of the original image, the corrected pixel values after range limitation of the color component satisfy the following relationship:

[0184]

[0185] where p is the minimum value of all the corrected pixel values of the color component, q is the maximum value of all the corrected pixel values of the color component, x i is the pixel value of the i-th corrected pixel point of the color component, and y i is the pixel value of the i-th corrected pixel point of the color component after range limitation. Based on this, it is possible to limit the corrected pixel values of each color component within the preset value range.

[0186] It should be understood that when performing shadow correction processing on the original image, since the corrected pixel values of each color channel of each pixel point are obtained by multiplying the pixel values of each color channel of each pixel point by the shadow compensation coefficient corresponding to the pixel point, the corrected pixel values may exceed the displayable range of the image. For example, in an RGB image, the pixel value range of each channel should be maintained between 0 and 255, so the displayable range of its pixel values is 0 - 255. Therefore, it is necessary to limit the range of the corrected pixel values so that the corrected image can be displayed normally. It should be noted that the above specific algorithm for range limitation is only an example, and this application does not limit it.

[0187] In addition, considering that there is a certain amount of detail loss in the image with the horizontal stripe shadow removed compared to the original image, it can be further fused with the original image to improve the effect of horizontal stripe shadow removal. Exemplarily, after step S104, the following step S105 may further be included:

[0188] S105. Fuse the original image and the image with the horizontal stripe shadow removed to obtain a fused image.

[0189] In some embodiments, step S105 is specifically implemented as: in the original image and the image with the horizontal stripe shadow removed, perform weighted summation of the pixel values of each color channel for the pixel points at the same position to generate a fused image.

[0190] Among them, the sum of the weight coefficients corresponding to the target color channels of the pixel points in the original image and the weight coefficients corresponding to the target color channels of the pixel points at the same position in the image with the horizontal stripe shadow removed is 1. The target color channel is any color channel in the color space. When the pixel value of the target color channel of the pixel point in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient are positively correlated, where the first pixel value is less than the second pixel value.

[0191] Exemplarily, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient may satisfy the following relationship:

[0192]

[0193] Among them, res represents the pixel value of the target color channel of the pixel point in the original image, wet represents the weight coefficient corresponding to the target color channel of the pixel point in the original image, thr1 represents the first weight value, thr2 represents the second weight value, thr1 is less than thr2, C1 represents the first pixel value, and C2 represents the second pixel value.

[0194] As a specific example, when the first weight value thr1 takes 0, the second weight value takes 1, the first pixel value takes 20, and the second pixel value takes 220, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient may satisfy the following relationship:

[0195]

[0196] At this time, the weight coefficient curve graph of the pixel values in the original image is as shown in Figure 7 shown. Among them, Figure 7 the horizontal axis in represents the pixel value of the target color channel in the original image, and the vertical axis represents the weight coefficient corresponding to the pixel value of the target color channel in the original image.

[0197] Taking the original image as an RGB image and the target channel as the R channel as an example, if the pixel value of a certain pixel point on the R channel in the original image is 5, then the brightness of this pixel point is relatively low. Therefore, it is very likely to be a pixel point in the horizontal stripe shadow area. Therefore, when fusing the images, the pixel value of the R channel of this pixel point in the original image is not considered, and the weight is taken as 0. Correspondingly, the weight of the pixel value of the R channel of the pixel point at the same position in the original image after removing the horizontal stripe shadow is taken as 1. If the pixel value of the R channel of the pixel point at the same position is 120, then the pixel value of the R channel of the pixel point at the same position in the fused image is (0 * 5 + 1 * 120), that is, 120.

[0198] Alternatively, if the pixel value of a certain pixel point on the R channel in the original image is 100, then the pixel value weight is (0.8 * 100 / 200), that is, 0.4. Since the brightness of this pixel point is balanced, it may be located at the edge of the crosswise shadow area or in the non-crosswise shadow area. It is not difficult to understand that during image fusion, the larger the pixel value of the R channel of this pixel point in the original image, the brighter this point, and the greater the probability of being in the non-crosswise shadow area, so the weight is greater. The smaller the pixel value of the R channel of this pixel point, the darker this point, and the greater the probability of being at the edge of the crosswise shadow area, so the weight is smaller. In addition, the pixel value weight of the R channel of the pixel point at the same position in the original image after removing the crosswise shadow is taken as (1 - 0.4), that is, 0.6. If the pixel value of the R channel of the pixel point at the same position is 120, then the pixel value of the R channel of the pixel point at the same position in the fused image is (0.4 * 100 + 0.6 * 120), that is, 112.

[0199] Alternatively, if the pixel value of a certain pixel point on the R channel in the original image is 250, then the brightness of this pixel point is relatively high. Therefore, it is very likely not to be a pixel point in the crosswise shadow area. Therefore, during image fusion, more consideration is given to the pixel value of the R channel of this pixel point in the original image, and the weight is taken as 0.8. Correspondingly, the pixel value weight of the R channel of the pixel point at the same position in the original image after removing the crosswise shadow is taken as 0.2. If the pixel value of the R channel of the pixel point at the same position is 200, then the pixel value of the R channel of the pixel point at the same position in the fused image is (0.8 * 250 + 0.2 * 200), that is, 240.

[0200] It should be understood that when the pixel value of the target color channel of a pixel point in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel point in the original image is positively correlated with its corresponding weight coefficient. In this way, the larger the pixel value of the target color channel of a pixel point in the original image, the smaller the probability that this pixel point is in the crosswise shadow area. Therefore, its corresponding weight coefficient is larger, so that more information of this pixel point in the original image can be retained in the fused image, so as to achieve the purpose of retaining the detailed features of the original image.

[0201] In addition, Figure 2The technical solutions shown bring at least the following beneficial effects: Since the gray values of the pixel points in the grayscale image can reflect the luminance information, after converting the original image into a grayscale image, the first light field image and the second light field image can be determined based on the grayscale image. Among them, the first light field image is used to characterize the luminance information of each pixel point in the original image without the interference of horizontal stripe shadows, and the second light field image is used to characterize the luminance information of each pixel point in the original image with the interference of horizontal stripe shadows retained. In this way, based on the first light field image and the second light field image, the interference of the horizontal stripe shadows on the luminance of each pixel point in the original image can be determined, and then the shadow compensation matrix for recording the shadow compensation coefficients corresponding to each pixel point in the original image can be determined. The shadow compensation coefficient is used to correct the luminance of the pixel points in the original image from the luminance under the interference of horizontal stripe shadows to the luminance without the interference of horizontal stripe shadows. Thus, based on the shadow compensation matrix, shadow correction processing is performed on the original image to obtain an image with horizontal stripe shadows removed. It can be seen that the technical solutions provided in the embodiments of the present application can effectively and accurately remove horizontal stripe shadows from the image.

[0202] It can be seen that the above mainly introduces the solutions provided in the embodiments of the present application from the perspective of methods. To implement the above functions, the embodiments of the present application provide the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0203] As Figure 8 shown, the embodiments of the present application provide a device for removing shadows from an image, which is used to execute Figure 1 the method for removing shadows from an image shown. The device 300 for removing shadows from an image includes: an image acquisition module 301, a light field determination module 302, and a shadow elimination module 303.

[0204] The image acquisition module 301 is configured to acquire an original image with horizontal stripe shadows;

[0205] The light field determination module 302 is configured to convert the original image into a grayscale image, and perform low-pass filtering on the grayscale image to determine the first light field image and the second light field image; wherein, the first light field image is used to characterize the luminance information of each pixel point in the original image without the interference of horizontal stripe shadows, and the second light field image is used to characterize the luminance information of each pixel point in the original image with the interference of horizontal stripe shadows retained;

[0206] The shadow elimination module 303 is configured to determine a shadow compensation matrix according to the first light field map and the second light field map, where the shadow compensation matrix is used to record the shadow compensation coefficients corresponding to each pixel point in the original image;

[0207] The shadow elimination module 303 is further configured to perform shadow correction processing on the original image based on the shadow compensation matrix to obtain an image with horizontal stripe shadows eliminated.

[0208] In some embodiments, the above-mentioned light field determination module 302 is specifically configured to determine the height and width of the first filtering operator and the height and width of the second filtering operator; wherein, the height of the first filtering operator is greater than the width of the first filtering operator, and the height of the second filtering operator is less than the width of the second filtering operator; perform low-pass filtering on the grayscale image based on the first filtering operator to obtain a processed grayscale image; perform low-pass filtering on the processed grayscale image based on the second filtering operator to obtain the first light field map;

[0209] In some embodiments, the above-mentioned light field determination module 302 is specifically configured to determine the height of the original image and the number of horizontal stripe shadows based on the original image; calculate the ratio between the height of the original image and the number of horizontal stripe shadows; determine the height and width of the first filtering operator and the height and width of the second filtering operator based on the ratio;

[0210] In some embodiments, the above-mentioned light field determination module 302 is specifically configured to determine the height and width of the third filtering operator; wherein, the height of the third filtering operator is less than the width of the third filtering operator; perform low-pass filtering on the grayscale image based on the third filtering operator to obtain the second light field map.

[0211] In some embodiments, the above-mentioned light field determination module 302 is specifically configured to determine the height of the original image and the number of horizontal stripe shadows based on the original image; calculate the ratio between the height of the original image and the number of horizontal stripe shadows; determine the height and width of the third filtering operator based on the ratio;

[0212] In some embodiments, the above-mentioned shadow elimination module 303 is specifically configured to modify the pixel value of the pixel point with a pixel value of 0 in the second light field map to 1 to obtain a modified second light field map; perform a point division operation on the first light field map and the modified second light field map to obtain a shadow compensation matrix;

[0213] In some embodiments, the above-mentioned shadow elimination module 303 is specifically configured to multiply the pixel values of each color channel of each pixel point in the original image by the shadow compensation coefficient corresponding to the pixel point to obtain the corrected pixel values of each color channel of each pixel point in the original image; perform a range limitation process on the corrected pixel values of each color channel of each pixel point in the original image to determine an image with horizontal stripe shadows eliminated, and the range limitation process is used to limit the pixel values of each color channel of the pixel point within a preset value range;

[0214] In some embodiments, the above-mentioned shadow elimination module 303 is specifically configured to process the corrected pixel values of each color channel of each pixel point in the original image by using a clipping function based on the constraint conditions of a preset value range to determine an image with horizontal stripe shadows eliminated;

[0215] In some embodiments, the above-mentioned shadow elimination module 303 is further configured to perform a fusion process on the original image and the image with horizontal stripe shadows eliminated to obtain a fused image;

[0216] In some embodiments, the above-mentioned shadow elimination module 303 is specifically configured to perform a weighted sum of the pixel values of each color channel on the pixel points at the same position in the original image and the image with horizontal stripe shadows eliminated to generate a fused image; wherein, the sum of the weight coefficients corresponding to the target color channel of the pixel point in the original image and the weight coefficient corresponding to the target color channel of the pixel point at the same position in the image with horizontal stripe shadows eliminated is 1, and the target color channel is any color channel in the color space; when the pixel value of the target color channel of the pixel point in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient are positively correlated;

[0217] In some embodiments, the pixel value of the target color channel of the pixel point in the above-mentioned original image and its corresponding weight coefficient satisfy the following relationship:

[0218]

[0219] wherein, res represents the pixel value of the target color channel of the pixel point in the original image, wet represents the weight coefficient corresponding to the target color channel of the pixel point in the original image, thr1 represents the first weight value, thr2 represents the second weight value, thr1 is less than thr2, C1 represents the first pixel value, and C2 represents the second pixel value.

[0220] It should be noted that Figure 8The division of the modules is schematic and is only a logical function division. In actual implementation, there may be other division methods. For example, two or more functions can also be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module.

[0221] When the functions of the above integrated module are implemented in the form of hardware, an embodiment of the present application provides another possible structural schematic diagram of the shadow removal device for the image involved in the above embodiment. As Figure 9 shown, the shadow removal device 400 for the image includes: a processor 402 and a bus 404. Optionally, the shadow removal device for the image may further include a memory 401; optionally, the shadow removal device for the image may further include a communication interface 403.

[0222] The processor 402 can be used to implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 402 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 402 can also be a combination that implements a computing function, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0223] The communication interface 403 is used to connect to other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc.

[0224] The memory 401 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It can also be an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium, or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0225] As a possible implementation, the memory 401 can exist independently of the processor 402. The memory 401 can be connected to the processor 402 through the bus 404 and is used to store instructions or program codes. When the processor 402 calls and executes the instructions or program codes stored in the memory 401, the method for removing the shadow of the image provided by the embodiments of the present application can be implemented.

[0226] In another possible implementation, the memory 401 can also be integrated with the processor 402.

[0227] The bus 404 can be an extended industry standard architecture (EISA) bus or the like. The bus 404 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 9 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0228] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the image shadow removal device is divided into different functional modules to complete all or part of the functions described above.

[0229] The embodiments of the present application also provide a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by computer instructions instructing relevant hardware. The program can be stored in the above computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be the memory in any of the foregoing embodiments. The above computer-readable storage medium can also be an external storage device of the above image shadow removal device, such as a plug-in hard disk equipped on the above image shadow removal device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the above computer-readable storage medium can also include both the internal storage unit of the above image shadow removal device and the external storage device. The above computer-readable storage medium is used to store the above computer program and other programs and data required by the above image shadow removal device. The above computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0230] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program. When the computer program product runs on a computer, the computer is caused to execute any one of the shadow removal methods of images provided in the above embodiments.

[0231] Although the present application has been described in conjunction with various embodiments herein, however, in the process of implementing the claimed present application, those skilled in the art can understand and realize other variations of the disclosed embodiments by viewing the drawings, the disclosure content, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality of cases. A single processor or other unit can implement several functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0232] Although the present application has been described in conjunction with specific features and their embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the present application. Accordingly, the present specification and the drawings are merely exemplary descriptions of the present application defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

[0233] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for removing shadows from an image, characterized in that, The method includes: Obtaining an original image with cross-striped shadows; Converting the original image into a grayscale image and performing low-pass filtering on the grayscale image to determine a first light field map and a second light field map; wherein, the first light field map is used to characterize the brightness information of each pixel point in the original image without the interference of the cross-striped shadows, and the second light field map is used to characterize the brightness information of each pixel point in the original image with the interference of the cross-striped shadows retained; Determining a shadow compensation matrix according to the first light field map and the second light field map, where the shadow compensation matrix is used to record the shadow compensation coefficients corresponding to each pixel point in the original image; Performing shadow correction processing on the original image based on the shadow compensation matrix to obtain an image with the cross-striped shadows eliminated.

2. The method according to claim 1, wherein Determining the first light field map based on the grayscale image, including: Determining the height and width of a first filtering operator and the height and width of a second filtering operator; wherein, the height of the first filtering operator is greater than the width of the first filtering operator, and the height of the second filtering operator is less than the width of the second filtering operator; Performing low-pass filtering on the grayscale image based on the first filtering operator to obtain a processed grayscale image; Performing low-pass filtering on the processed grayscale image based on the second filtering operator to obtain the first light field map.

3. The method according to claim 2, wherein The determining the height and width of the first filtering operator and the height and width of the second filtering operator includes: Based on the original image, determining the height of the original image and the number of the cross-striped shadows; Calculating the ratio between the height of the original image and the number of the cross-striped shadows; Based on the ratio, determining the height and width of the first filtering operator and the height and width of the second filtering operator.

4. The method according to claim 1, wherein Determining the second light field map based on the grayscale image, including: Determining the height and width of a third filtering operator; wherein, the height of the third filtering operator is less than the width of the third filtering operator; Performing low-pass filtering on the grayscale image based on the third filtering operator to obtain the second light field map.

5. The method according to claim 4, characterized in that The determining the height and width of the third filtering operator includes: Based on the original image, determining the height of the original image and the number of the cross-striped shadows; Calculating the ratio between the height of the original image and the number of the cross-striped shadows; Based on the ratio, determining the height and width of the third filtering operator.

6. The method according to claim 1, wherein The determining the shadow compensation matrix according to the first light field map and the second light field map includes: Modifying the pixel values of the pixel points with pixel values of 0 in the second light field map to 1 to obtain a modified second light field map; Performing a point division operation on the first light field map and the modified second light field map to obtain the shadow compensation matrix.

7. The method according to claim 6, characterized in that, The performing shadow correction processing on the original image based on the shadow compensation matrix to obtain an image with the cross-striped shadows eliminated includes: Multiplying the pixel values of each color channel of each pixel point in the original image by the shadow compensation coefficient corresponding to the pixel point to obtain the corrected pixel values of each color channel of each pixel point in the original image; Perform range limiting processing on the corrected pixel values of each color channel of each pixel point in the original image to determine the image with the horizontal stripe shadows eliminated, where the range limiting processing is used to limit the pixel values of each color channel of a pixel point within a preset value range.

8. The method according to claim 7, wherein The performing range limiting processing on the corrected pixel values of each color channel of each pixel point in the original image to determine the image with the horizontal stripe shadows eliminated includes: Based on the constraint conditions of the preset value range, use a clipping function to process the corrected pixel values of each color channel of each pixel point in the original image to determine the image with the horizontal stripe shadows eliminated.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Fuse the original image and the image with the horizontal stripe shadows eliminated to obtain a fused image.

10. The method according to claim 9, wherein The fusing the original image and the image with the horizontal stripe shadows eliminated to obtain a fused image includes: In the original image and the image with the horizontal stripe shadows eliminated, perform weighted summation of the pixel values of each color channel for pixel points at the same position to generate the fused image; wherein, the sum of the weight coefficients corresponding to the target color channel of a pixel point in the original image and the weight coefficients corresponding to the target color channel of a pixel point at the same position in the image with the horizontal stripe shadows eliminated is 1, and the target color channel is any one color channel in the color space; When the pixel value of the target color channel of a pixel point in the original image is greater than or equal to the first pixel value and less than or equal to the second pixel value, the pixel value of the target color channel of the pixel point in the original image and its corresponding weight coefficient are positively correlated.

11. The method according to claim 10, characterized in that, The pixel value of the target color channel of a pixel point in the original image and its corresponding weight coefficient satisfy the following relationship: where res represents the pixel value of the target color channel of a pixel point in the original image, wet represents the weight coefficient corresponding to the target color channel of a pixel point in the original image, thr1 represents the first weight value, thr2 represents the second weight value, thr1 is less than thr2, C1 represents the first pixel value, and C2 represents the second pixel value.

Images