Gradient direction-based camera spot detection method and apparatus

Abstract

The application relates to a camera stain detection method and device based on gradient direction, which comprises the following steps: extracting a brightness map of an image from an image collected by a camera; calculating a horizontal gradient difference and a vertical gradient difference of the brightness map respectively to obtain a horizontal gradient map and a vertical gradient map; performing stain screening through the horizontal gradient map and the vertical gradient map to obtain a stain screening map, wherein corresponding values of pixels in the stain screening map include preset stain values and non-pre-set stain values; and determining pixels corresponding to the preset stain values in the stain screening map as stains in the image. According to the scheme, pixels possibly being stains are screened based on gradient direction, and a brightness contrast screening method is designed to accurately find stains in order to exclude the influence of other light sources or halos on stain detection. Finally, a final stain area is determined through filtering and connected region screening.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for detecting blemishes in a camera device based on gradient direction. Background Technology

[0002] For various camera devices or components, such as handheld cameras, mobile phone cameras, and network cameras, after production, there may be stains on the lens or image sensor of the camera device or component, which need to be inspected.

[0003] Current spot detection methods typically acquire images that include image enhancement and brightness segmentation, and then locate the spot using brightness or edge information.

[0004] One method for detecting blemishes is to extract the brightness information of the image, then enhance the image based on homomorphic filtering to find the first blemish candidate region. By using edge fitting, an ideal brightness map is fitted, and the difference between the ideal brightness map and the original brightness map is used to find the second blemish candidate region. Finally, the location of the blemish is located based on the enhancement and fitting results.

[0005] However, the above-mentioned spot detection method has two problems: First, the image enhancement method based on homomorphic filtering can easily amplify the halo caused by the camera lens as well, especially during production line operation, where spot detection is usually performed using a uniform light-emitting plate. In this case, the halo caused by the lens will be more obvious, making it difficult to distinguish between spots and halos. Second, the scheme uses surface fitting, and finds the candidate position of spot by subtracting the fitted brightness map from the original brightness map. However, if the selection of the control points required for fitting is not good, the fitted image will not be an ideal brightness map. For example, if the control points already include spots, the fitted brightness map will be uneven, which can easily lead to missed or false spot detections.

[0006] Another method for detecting blemishes is to first binarize the image, then divide the image into several blocks, then compare the current pixel brightness value of all blocks with the average pixel brightness value within the block to determine whether the current pixel belongs to a blemish pixel, and finally draw the final blemish detection map by connecting all blemish pixels in the entire image.

[0007] This spot detection method also has two problems: First, judging spot pixels based on local blocks has very poor accuracy because the brightness of the lens is not uniform, so the brightness of pixels within a local block will definitely be different. It is difficult to distinguish normal pixels from spot pixels by using threshold judgment, which easily leads to missed detections or false detections. Second, if the spot area in the image is very large and the block cannot include the entire spot, the method can only detect the edge part of the large spot, and cannot detect the middle part of the spot. It may mistakenly identify a spot area as multiple small spots. Summary of the Invention

[0008] The existing technology has two main drawbacks. First, blemishes are easily missed (some blemishes are not identified) because the brightness change of blemishes is not very obvious, and edge detection methods are more likely to miss them. Second, blemishes are easily misdetected (for example, halos are detected as blemishes). Because cameras, especially small aperture cameras, have obvious halos on their lenses and may also have vignetting, it is easy to detect some vignetting or halos as blemishes.

[0009] To address the problems existing in the spot detection process of the prior art, this application, with reference to the optical imaging principle of a lens, designs a spot detection scheme for a camera device based on gradient direction. Pixels that may be spots are screened based on gradient direction. In order to eliminate the influence of other light sources or halos on spot detection, a method of screening by brightness comparison is also designed to accurately find spots. Finally, the final spot area is determined by filtering and screening of connected regions.

[0010] According to a first aspect of this application, a method for detecting blemishes in a camera device based on gradient direction is provided, characterized in that it includes:

[0011] Extract the brightness map of the image from the image captured by the camera device;

[0012] The lateral gradient difference and the longitudinal gradient difference are calculated on the brightness map to obtain the lateral gradient map and the longitudinal gradient map.

[0013] A blemish screening map is obtained by performing blemish screening using the horizontal gradient map and the vertical gradient map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values; and

[0014] The pixels whose values ​​in the blemish screening image correspond to the preset blemish value are identified as blemishes in the image.

[0015] According to a second aspect of this application, a stain detection device for a camera based on gradient direction is provided, characterized in that it comprises:

[0016] An extraction module is used to extract the brightness map of the image from the image captured by the camera device;

[0017] The gradient map calculation module is used to calculate the horizontal gradient difference and the vertical gradient difference of the brightness map to obtain the horizontal gradient map and the vertical gradient map.

[0018] The module is configured to perform blemish screening using the horizontal gradient map and the vertical gradient map to obtain a blemish screening map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values; and

[0019] The blemish determination module is used to determine the pixels in the blemish screening image that have a value corresponding to the preset blemish value as blemishes in the image.

[0020] According to a third aspect of this application, an electronic device is provided, comprising:

[0021] Processor; and

[0022] A memory storing computer instructions that, when executed by the processor, cause the processor to perform the method described in the first aspect.

[0023] According to a fourth aspect of this application, a non-transitory computer storage medium is provided, which stores a computer program that, when executed by a plurality of processors, causes the processors to perform the method described in the first aspect.

[0024] This application provides a stain detection scheme for camera devices based on gradient direction. When the camera device faces a uniformly emitting plate, the captured image has the characteristic that the brightness gradually decreases from the center to the edge. By calculating the horizontal and vertical gradients of the image's brightness map, the brightness of a pixel in the image is determined based on the calculation results to see if its location matches a preset condition. If the brightness of a pixel does not match its location, the pixel is likely a stain. Furthermore, to eliminate the influence of water ripples, the pre-screened image obtained through gradient calculation can be mean-filtered to fill the range of stains. In addition, to eliminate the influence of bright spots or halos, a brightness threshold can be obtained, and the pre-screened image can be corrected using the brightness threshold to eliminate falsely judged stains with brightness exceeding the threshold. The scheme of this application can solve the problems of missed detection and false detection in existing stain detection technologies. It is simple and convenient to operate and suitable for stain detection in various camera devices or modules. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings, without exceeding the scope of protection claimed by this application.

[0026] Figure 1 This is a schematic diagram of image acquisition for stain detection using a camera device according to an embodiment of this application.

[0027] Figure 2 This is a flowchart of a stain detection method for a camera device based on gradient direction according to an embodiment of this application.

[0028] Figure 3 This is a flowchart of a stain detection method for a camera device based on gradient direction, according to another embodiment of this application.

[0029] Figure 4 This is a flowchart of a stain detection method for a camera device based on gradient direction, according to yet another embodiment of this application.

[0030] Figure 5 The stain map and test result map are based on the scheme of this application.

[0031] Figure 6 This is a schematic diagram of a stain detection device based on a gradient direction camera according to an embodiment of this application.

[0032] Figure 7 This is a schematic diagram of a stain detection device based on a gradient direction camera according to another embodiment of this application.

[0033] Figure 8 This is a schematic diagram of a stain detection device based on a gradient direction camera according to another embodiment of this application.

[0034] Figure 9 This is a structural diagram of an electronic device provided in this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Figure 1 This is a schematic diagram of image acquisition for stain detection using a camera device according to an embodiment of this application. Figure 1As shown, the camera device is a camera, which is facing a uniform light-emitting panel and is as close to the panel as possible so that the image captured by the camera does not contain edge information of the light-emitting panel.

[0037] exist Figure 1 In the image acquisition scenario shown, the brightness of the image captured by the camera follows a pattern of decreasing from the center to the edges. This is reflected in the image's brightness map, where the brightness values ​​decrease sequentially from the center to the edges. However, if there are blemishes in the image captured by the camera, the location of the blemishes often does not conform to this pattern. Pixels that do not conform to this pattern are pre-screened as blemishes, and a detailed explanation is given below.

[0038] According to one aspect of this application, a method for detecting stains in a camera device based on gradient direction is provided. Figure 2 This is a flowchart of a stain detection method for a camera device based on gradient direction, according to an embodiment of this application. Figure 2 As shown, the method includes the following steps.

[0039] Step S201: Extract the brightness map of the image from the image captured by the camera device.

[0040] The camera device is arranged according to Figure 1 The image acquisition scene shown is used to acquire images, and a luminance map is extracted from the acquired images, for example, by converting them from YUV (Luminance; Chrominance; Color) format or RGB (Red, Green, Blue) format to grayscale. According to some embodiments, the Y channel of the YUV format can be used as the luminance map, or the maximum value of the three RGB channels can be used as the luminance map imgLum.

[0041] Step S202: Calculate the horizontal gradient difference and the vertical gradient difference of the brightness map to obtain the horizontal gradient map and the vertical gradient map.

[0042] The gradient of the brightness map imgLum is calculated by using the horizontal gradient difference and the vertical gradient difference respectively, to obtain the horizontal gradient map grad_x and the vertical gradient map grad_y.

[0043] According to some embodiments, for any pixel in the luminance map imgLum, the difference between its corresponding luminance value and the luminance value of its horizontally adjacent pixel is calculated. This process is repeated for each pixel in the luminance map imgLum to obtain the horizontal gradient map grad_x. At the same time, for any pixel in the luminance map imgLum, the difference between its corresponding luminance value and the luminance value of its vertically adjacent pixel is calculated. This process is repeated for each pixel in the luminance map imgLum to obtain the vertical gradient map grad_y.

[0044] During the traversal of each pixel in the luminance map imgLum, the direction can be any set direction, such as horizontally to the right and vertically downwards, or horizontally to the right and vertically upwards, etc., and this application does not impose any restrictions on this. Furthermore, horizontally or vertically adjacent pixels can be tightly adjacent pixels, meaning there are no other pixels between them, or they can be spaced apart by a preset number of pixels, such as a one-pixel gap between horizontally or vertically adjacent pixels. For pixels at the edges of the luminance map imgLum, for example, during the traversal of pixels horizontally to the right and vertically downwards, the difference between the corresponding rightmost and bottommost pixels is a preset value, such as 0.

[0045] Thus, step S202 can specifically include:

[0046] Step S2021: Determine the horizontal gradient map based on the difference between the brightness value of any pixel in the brightness map and the brightness values ​​of its horizontally adjacent pixels; and

[0047] Step S2022: Determine the vertical gradient map based on the difference between the brightness value of any pixel in the brightness map and the brightness value of its vertically adjacent pixels.

[0048] Step S203: Perform blemish screening using the horizontal gradient map and the vertical gradient map to obtain a blemish screening map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values.

[0049] The principle behind this step is as follows: when a camera is pointed at a uniformly illuminated panel, the captured image exhibits a characteristic where brightness gradually decreases from the center to the edges. By calculating the horizontal and vertical gradients of the image's brightness map, the results are used to determine whether the brightness of a pixel in the image matches a preset condition. If the brightness of a pixel does not match its location, that pixel is likely a blemish.

[0050] The preset conditions relate to the method of calculating gradients during the calculation of the horizontal and vertical gradient maps. Specifically, when calculating the horizontal gradient map, is it the brightness value on the right minus the brightness value on the left ("right minus left") or the brightness value on the left minus the brightness value on the right ("left minus right")? Similarly, when calculating the vertical gradient map, is it the brightness value on the bottom minus the brightness value on the top ("bottom minus top") or the brightness value on the top minus the brightness value on the bottom ("top minus bottom")? The brightness map is divided into four parts at the center point. The horizontal gradient map includes four parts: top left, bottom left, top right, and bottom right (grad_x_lefttop, grad_x_leftBottom, grad_x_righttop, grad_x_rightBottom). The vertical gradient map also includes four parts: top left, bottom left, top right, and bottom right (grad_y_lefttop, grad_y_leftBottom, grad_y_righttop, grad_y_rightBottom). If the calculation follows the formula "right minus left, bottom minus top",... "In the absence of blemishes, the positive and negative values ​​of the horizontal and vertical gradient plots corresponding to the top left, bottom left, top right, and bottom right parts are (positive-positive), (positive-negative), (negative-positive), and (negative-negative), respectively. If the gradient is calculated by subtracting the left from the right and the bottom from the top, the positive and negative values ​​of the horizontal and vertical gradient plots corresponding to the top left, bottom left, top right, and bottom right parts are (positive-negative), (negative-negative), (positive-positive), and (negative-positive), respectively, in the absence of blemishes. And so on. Those skilled in the art can understand the preset conditions corresponding to other gradient calculation methods, which will not be elaborated in this article."

[0051] For example, by calculating the gradient according to the method of "right minus left, top minus bottom", and judging whether each pixel in the image is a blemish based on the preset conditions of (positive positive), (positive negative), (negative positive), and (negative negative), the following equation can clearly show the judgment process:

[0052]

[0053]

[0054]

[0055]

[0056] In the above equation, `stain_mask1` represents the taint screening map. If a pixel is determined to be a taint, its corresponding value is set to a preset taint value; otherwise, its corresponding value is set to another value, called a non-preset taint value. For example, in the above equation, the preset taint value is 255, and the non-preset taint value is 0. Thus, based on the value corresponding to each pixel, whether it is a preset taint value or a non-preset taint value, the taint screening map can be determined.

[0057] Thus, step S203 can specifically include:

[0058] Step S2031: For any pixel located in the horizontal gradient map and the vertical gradient map, if the value of the pixel in the horizontal gradient map and the vertical gradient map does not meet the preset judgment condition corresponding to the pixel's location, the value corresponding to the pixel is determined as the preset blemish value; and

[0059] Step S2032: Determine the blemish screening map using the preset blemish value.

[0060] Step S204: Pixels whose values ​​in the blemish screening image correspond to the preset blemish value are identified as blemishes in the image.

[0061] After determining the blemish filter map, which contains a value for each pixel (either a preset blemish value or a non-preset blemish value), the pixels in the blemish filter map that correspond to the preset blemish value are identified as blemishes in the image.

[0062] The blemishes obtained through the above steps include blemishes covering any number of pixels. However, some blemishes cover very few pixels, such as a few or dozens, and are basically invisible to the naked eye. These blemishes covering very few pixels can be ignored and are not considered blemishes.

[0063] According to some embodiments, connected component calculations can be performed on the taint screening map. A connected component is an image region consisting of adjacent pixels that all have a preset taint value (e.g., 255). Then, positions in the taint screening map where the size of the connected component is less than a certain threshold (e.g., typically between 50 and 500) are marked as non-preset taint values, such as 0, to obtain the final taint map, stain_map.

[0064] Thus, step S204 may include: for any connected region of the blemish screening map, if the number of pixels with the preset blemish value is greater than a preset threshold, the connected region is determined as a blemish.

[0065] pass Figure 2The method for detecting blemishes in a camera device based on gradient direction is shown. It filters pixels based on gradient direction and can identify pixels that may be blemishes.

[0066] The camera device captures images of the light-emitting panel, and the captured images may contain some gradients, similar to water ripples. To further prevent water ripples in the images and fill in the areas of blemishes, according to another embodiment of the application, in... Figure 2 Based on the method shown, another method for detecting blemishes in camera devices based on gradient direction is provided, such as... Figure 3 As shown. With Figure 2 compared to, Figure 3 Steps S301 to S304 correspond to Figure 2 The difference between steps S201 to S204 is that... Figure 3 The methods also include:

[0067] Step 305: After obtaining the blemish screening image, perform mean filtering on the blemish screening image to obtain a filtered screening image; and

[0068] Step 306: Based on the relationship between the value corresponding to any pixel in the filtered screening image and the preset blemish judgment threshold, the value corresponding to the pixel in the blemish screening image is corrected.

[0069] According to some embodiments, the taint screening image `stain_mask1` is subjected to mean filtering, which involves performing a full-image filtering on all pixels of the image using an average filtering convolution kernel. In some embodiments, a 7x7 filtering kernel can be used. Then, the filtered image `stain_mask1_filter` is segmented for taints using a taint detection threshold to obtain the updated taint screening image `stain_mask2`. The following equation visually illustrates this process:

[0070]

[0071] Here, stainthr represents the taint detection threshold, which can be set according to different products and production requirements. For example, it can be 80, the preset taint value is 255, and the non-preset taint value is 0.

[0072] Thus, step S306 can specifically include: if the value corresponding to any pixel in the filtered screening image is greater than the blemish judgment threshold, the value corresponding to the pixel in the blemish screening image is determined as the preset blemish value.

[0073] Then, by comparing the value of any pixel in the filtered screening image with the blemish judgment threshold, the blemish screening image is corrected before blemish judgment is performed, i.e., step S204 or step S304 is executed again.

[0074] Because of interference from other light sources in the environment or reflections from the light-emitting panel itself, the captured image of a spot detection may contain bright spots or halos. Consequently, bright spots or halos may be mistakenly detected as spots.

[0075] In response to this situation, Figure 2 or Figure 3 Based on the method shown, another method for detecting blemishes in camera devices based on gradient direction is provided, such as... Figure 4 As shown. With Figure 2 compared to, Figure 4 Steps S401 to S404 correspond to Figure 2 The difference between steps S201 to S204 is that... Figure 4 The methods also include:

[0076] Step S405: Determine the portion of the brightness map at a preset ratio as the center brightness map;

[0077] Step S406: Calculate the average brightness of all pixels in the brightness map;

[0078] Step S407: Determine the brightness threshold as the brightness value in the histogram of the center brightness map that has the highest brightness value and the largest number of pixels; and

[0079] Step S408: Based on the relationship between the brightness value of any pixel in the brightness map and the brightness threshold, correct the value corresponding to the pixel in the blemish screening map.

[0080] According to some embodiments, a portion of the center of the luminance image imgLum is first cropped at a predetermined ratio to form the center luminance image imgLum_center. For example, the top-left pixel positions x and y of the center luminance image imgLum_center are 1 / 3 of the length and 1 / 3 of the width of the luminance image imgLum, respectively, and the length and width of the center luminance image imgLum_center are 1 / 3 of the length and 1 / 3 of the width of imgLum, respectively. Those skilled in the art can also crop portions of the center of the luminance image imgLum at other ratios to form the center luminance image; this application does not impose any limitations on this.

[0081] Next, the average brightness of all pixels in the brightness map `imgLum` is calculated, denoted as `imgLum_mean`. A histogram of all pixels in the center brightness map `imgLum_center` is then plotted. This histogram represents the frequency of each pixel's occurrence at different brightness levels within a brightness region, for example, the frequency of each pixel's occurrence at different brightness levels within the brightness region [0, 255]. The brightness level that has the highest frequency in the histogram and is greater than the average brightness level `imgLum_mean` is named the brightness threshold `lumax`. Finally, the brightness threshold `lumax` is used to... Figure 2 or Figure 3 The stain screening image, either stain_mask1 or stain_mask2, is corrected.

[0082] The following equation illustrates the process of correcting the stain screening map stain_mask2 using the brightness threshold lumax:

[0083] stain_mask2(x)=0,if imgLum(x)>(Iumax+lumthr)

[0084] Here, lumthr is a correction factor that can be set according to actual needs. For example, if more bright spots are to be excluded, this value can be set to a smaller value, such as -10.

[0085] By correcting the brightness threshold lumax, falsely identified blemishes with brightness exceeding the center average brightness can be eliminated. Since blemishes are caused by blocking light from the camera sensor, the brightness of a blemish is definitely less than the brightness of the surrounding pixels. If the brightness exceeds the brightness threshold lumax, it should not be considered a blemish area. This eliminates the influence of halos and bright spots, preventing bright spots or halos from being detected as blemishes.

[0086] Figure 4 The method shown can be based on Figure 2 The method shown can also be based on Figure 3 The method shown involves obtaining the stain screening map stain_mask1 or stain_mask2, and then applying a brightness threshold lumax to the image. Figure 2 or Figure 3 The stain screening image, stain_mask1 or stain_mask2, is corrected. After correcting the stain screening image, stain_mask1 or stain_mask2, step S204 or step S304 is executed.

[0087] This application provides a method for detecting blemishes in a camera device based on gradient direction. When the camera device faces a uniformly emitting light plate, the captured image exhibits a characteristic of brightness gradually decreasing from the center to the edges. By calculating the horizontal and vertical gradients of the image's brightness map, the method determines whether the brightness of a pixel in the image matches a preset condition. If the brightness of a pixel does not match its location, the pixel is likely a blemish. Furthermore, to eliminate the influence of water ripples, the pre-screened image obtained through gradient calculation can be mean-filtered to fill in the blemish area. Additionally, to eliminate the influence of bright spots or halos, a brightness threshold can be obtained, and the pre-screened image can be corrected using this threshold to eliminate falsely identified blemishes with brightness exceeding the threshold. This solution solves the problems of missed and false detections in existing blemish detection technologies, is simple and convenient to operate, and is suitable for blemish detection in various camera devices or modules.

[0088] Figure 5 This is based on the stain map and test result map of the scheme in this application. For example... Figure 5 As shown, the solution of this application can detect blemishes in images.

[0089] According to one aspect of this application, a stain detection device for a camera based on gradient direction is provided. Figure 6 This is a schematic diagram of a stain detection device for a camera based on gradient direction, according to an embodiment of this application. Figure 6 As shown, the device includes the following modules.

[0090] Extraction module 601 is used to extract the brightness map of the image from the image acquired by the camera device.

[0091] The camera device is arranged according to Figure 1 The image acquisition scene shown is used to acquire images, and a luminance map is extracted from the acquired images, for example, by converting them from YUV (Luminance; Chrominance; Color) format or RGB (Red, Green, Blue) format to grayscale. According to some embodiments, the Y channel of the YUV format can be used as the luminance map, or the maximum value of the three RGB channels can be used as the luminance map imgLum.

[0092] The gradient map calculation module 602 is used to calculate the horizontal gradient difference and the vertical gradient difference of the brightness map to obtain the horizontal gradient map and the vertical gradient map.

[0093] The gradient of the brightness map imgLum is calculated by using the horizontal gradient difference and the vertical gradient difference respectively, to obtain the horizontal gradient map grad_x and the vertical gradient map grad_y.

[0094] According to some embodiments, for any pixel in the luminance map imgLum, the difference between its corresponding luminance value and the luminance value of its horizontally adjacent pixel is calculated. This process is repeated for each pixel in the luminance map imgLum to obtain the horizontal gradient map grad_x. At the same time, for any pixel in the luminance map imgLum, the difference between its corresponding luminance value and the luminance value of its vertically adjacent pixel is calculated. This process is repeated for each pixel in the luminance map imgLum to obtain the vertical gradient map grad_y.

[0095] During the traversal of each pixel in the luminance map imgLum, the direction can be any set direction, such as horizontally to the right and vertically downwards, or horizontally to the right and vertically upwards, etc., and this application does not impose any restrictions on this. Furthermore, horizontally or vertically adjacent pixels can be tightly adjacent pixels, meaning there are no other pixels between them, or they can be spaced apart by a preset number of pixels, such as a one-pixel gap between horizontally or vertically adjacent pixels. For pixels at the edges of the luminance map imgLum, for example, during the traversal of pixels horizontally to the right and vertically downwards, the difference between the corresponding rightmost and bottommost pixels is a preset value, such as 0.

[0096] Thus, the gradient graph calculation module 602 can specifically include:

[0097] The first determining unit 6021 is configured to determine the horizontal gradient map based on the difference between the brightness value of any pixel in the brightness map and the brightness values ​​of its horizontally adjacent pixels; and

[0098] The second determining unit 6022 is used to determine the vertical gradient map based on the difference between the brightness value of any pixel in the brightness map and the brightness value of its vertically adjacent pixel.

[0099] The module 603 is used to perform blemish screening through the horizontal gradient map and the vertical gradient map to obtain a blemish screening map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values.

[0100] The principle behind this step is as follows: when a camera is pointed at a uniformly illuminated panel, the captured image exhibits a characteristic where brightness gradually decreases from the center to the edges. By calculating the horizontal and vertical gradients of the image's brightness map, the results are used to determine whether the brightness of a pixel in the image matches a preset condition. If the brightness of a pixel does not match its location, that pixel is likely a blemish.

[0101] The preset conditions relate to the method of calculating gradients during the calculation of the horizontal and vertical gradient maps. Specifically, when calculating the horizontal gradient map, is it the brightness value on the right minus the brightness value on the left ("right minus left") or the brightness value on the left minus the brightness value on the right ("left minus right")? Similarly, when calculating the vertical gradient map, is it the brightness value on the bottom minus the brightness value on the top ("bottom minus top") or the brightness value on the top minus the brightness value on the bottom ("top minus bottom")? The brightness map is divided into four parts at the center point. The horizontal gradient map includes four parts: top left, bottom left, top right, and bottom right (grad_x_lefttop, grad_x_leftBottom, grad_x_righttop, grad_x_rightBottom). The vertical gradient map also includes four parts: top left, bottom left, top right, and bottom right (grad_y_lefttop, grad_y_leftBottom, grad_y_righttop, grad_y_rightBottom). If the calculation follows the formula "right minus left, bottom minus top",... "In the absence of blemishes, the positive and negative values ​​of the horizontal and vertical gradient plots corresponding to the top left, bottom left, top right, and bottom right parts are (positive-positive), (positive-negative), (negative-positive), and (negative-negative), respectively. If the gradient is calculated by subtracting the left from the right and the bottom from the top, the positive and negative values ​​of the horizontal and vertical gradient plots corresponding to the top left, bottom left, top right, and bottom right parts are (positive-negative), (negative-negative), (positive-positive), and (negative-positive), respectively, in the absence of blemishes. And so on. Those skilled in the art can understand the preset conditions corresponding to other gradient calculation methods, which will not be elaborated in this article."

[0102] For example, by calculating gradients according to the principle of "right minus left, top minus bottom," and using preset conditions of (positive-positive), (positive-negative), (negative-positive), and (negative-negative), it can determine whether each pixel in the image is a blemish. Based on the value corresponding to each pixel, whether it is a preset blemish value or a non-preset blemish value, a blemish screening map can be determined.

[0103] Thus, obtaining module 603 can specifically include:

[0104] The third determining unit 6031 is configured to, for any pixel located in the horizontal gradient map and the vertical gradient map, determine the value corresponding to the pixel as the preset blemish value if the value of the pixel in the horizontal gradient map and the vertical gradient map does not meet the preset judgment condition corresponding to the pixel's location; and

[0105] The fourth determining unit 6032 is used to determine the blemish screening map based on the preset blemish value.

[0106] The blemish determination module 604 is used to determine the pixels in the blemish screening image whose values ​​are the preset blemish values ​​as blemishes in the image.

[0107] After determining the blemish filter map, which contains a value for each pixel (either a preset blemish value or a non-preset blemish value), the pixels in the blemish filter map that correspond to the preset blemish value are identified as blemishes in the image.

[0108] The blemishes obtained through the above steps include blemishes covering any number of pixels. However, some blemishes cover very few pixels, such as a few or dozens, and are basically invisible to the naked eye. These blemishes covering very few pixels can be ignored and are not considered blemishes.

[0109] According to some embodiments, connected component calculations can be performed on the taint screening map. A connected component is an image region consisting of adjacent pixels that all have a preset taint value (e.g., 255). Then, positions in the taint screening map where the size of the connected component is less than a certain threshold (e.g., typically between 50 and 500) are marked as non-preset taint values, such as 0, to obtain the final taint map, stain_map.

[0110] Thus, the determining module 604 can be used to: for any connected region of the blemish screening map, if the number of pixels with the preset blemish value is greater than a preset threshold, determine the connected region as a blemish.

[0111] pass Figure 6 The image sensor blemish detection device shown is a gradient-direction-based camera device that filters pixels based on gradient direction and can identify pixels that may be blemishes.

[0112] To further prevent water ripples in the image and to fill the area of ​​the blemishes, according to another embodiment of the application, in Figure 6 Based on the device shown, another stain detection device based on gradient direction camera is provided, such as... Figure 7 As shown. With Figure 6 compared to, Figure 7 Modules 701 to 704 correspond to Figure 6 The difference between modules 601 to 604 is that... Figure 7 The device also includes:

[0113] Filtering module 705 is configured to perform mean filtering on the taint screening image after obtaining the taint screening image, to obtain a filtered screening image; and

[0114] The first correction module 706 is used to correct the value corresponding to the pixel in the taint screening image based on the relationship between the value corresponding to any pixel in the filtered screening image and the preset taint judgment threshold.

[0115] According to some embodiments, the taint screening image `stain_mask1` is subjected to mean filtering, which involves performing a full-image filtering on all pixels of the image using an average filtering convolution kernel. In some embodiments, a 7x7 filtering kernel can be used. Then, the filtered image `stain_mask1_filter` is segmented for taints using a taint detection threshold to obtain the updated taint screening image `stain_mask2`.

[0116] Thus, the first correction module 706 can be specifically used to: determine the value corresponding to the pixel in the taint screening image as the preset taint value when the value corresponding to any pixel in the filtered screening image is greater than the taint judgment threshold.

[0117] Because of interference from other light sources in the environment or reflections from the light-emitting panel itself, the captured image of a spot detection may contain bright spots or halos. Consequently, bright spots or halos may be mistakenly detected as spots.

[0118] In response to this situation, Figure 6 or Figure 7 Based on the device shown, another stain detection device based on gradient direction camera is provided, such as... Figure 8 As shown. With Figure 6 compared to, Figure 8 Modules 801 to 804 correspond to Figure 6 The difference between modules 601 to 604 is that... Figure 8 The device also includes:

[0119] The first determining module 805 is used to determine the portion of the brightness map at a preset proportion as the center brightness map;

[0120] The calculation module 806 is used to calculate the average brightness of all pixels in the brightness map;

[0121] The second determining module 807 is used to determine the brightness threshold as the brightness value with the highest number of pixels in the histogram of the center brightness map that is greater than the average brightness value; and

[0122] The second correction module 808 is used to correct the value corresponding to the pixel in the blemish screening map according to the relationship between the brightness value of any pixel in the brightness map and the brightness threshold.

[0123] According to some embodiments, a portion of the center of the luminance image imgLum is first cropped at a predetermined ratio to form the center luminance image imgLum_center. For example, the top-left pixel positions x and y of the center luminance image imgLum_center are 1 / 3 of the length and 1 / 3 of the width of the luminance image imgLum, respectively, and the length and width of the center luminance image imgLum_center are 1 / 3 of the length and 1 / 3 of the width of imgLum, respectively. Those skilled in the art can also crop portions of the center of the luminance image imgLum at other ratios to form the center luminance image; this application does not impose any limitations on this.

[0124] Next, the average brightness of all pixels in the brightness map `imgLum` is calculated, denoted as `imgLum_mean`. A histogram of all pixels in the center brightness map `imgLum_center` is then plotted. This histogram represents the frequency of each pixel's occurrence at different brightness levels within a brightness region, for example, the frequency of each pixel's occurrence at different brightness levels within the brightness region [0, 255]. The brightness level that has the highest frequency in the histogram and is greater than the average brightness level `imgLum_mean` is named the brightness threshold `lumax`. Finally, the brightness threshold `lumax` is used to... Figure 2 or Figure 3 The stain screening image, either stain_mask1 or stain_mask2, is corrected.

[0125] By correcting the brightness threshold lumax, falsely identified blemishes with brightness exceeding the center average brightness can be eliminated. Since blemishes are caused by blocking light from the camera sensor, the brightness of a blemish is definitely less than the brightness of the surrounding pixels. If the brightness exceeds the brightness threshold lumax, it should not be considered a blemish area. This eliminates the influence of halos and bright spots, preventing bright spots or halos from being detected as blemishes.

[0126] This application provides a stain detection device for a camera based on gradient direction. When the camera is facing a uniformly emitting plate, the captured image has a characteristic of brightness gradually decreasing from the center to the edge. By calculating the horizontal and vertical gradients of the image's brightness map, the brightness of a pixel in the image is determined based on the calculation results to see if its location matches preset conditions. If the brightness of a pixel does not match its location, the pixel is likely a stain. Furthermore, to eliminate the influence of water ripples, the pre-screened image obtained through gradient calculation can be mean-filtered to fill the range of stains. In addition, to eliminate the influence of bright spots or halos, a brightness threshold can be obtained, and the pre-screened image can be corrected using the brightness threshold to eliminate falsely judged stains with brightness exceeding the threshold. The solution of this application can solve the problems of missed detection and false detection in existing stain detection technologies. It is simple and convenient to operate and suitable for stain detection in various camera devices or modules.

[0127] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0128] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0130] See Figure 9 , Figure 9 An electronic device is provided, including a processor and a memory. The memory stores computer instructions, which, when executed by the processor, cause the processor to perform the computer instructions to achieve the following: Figure 2 , Figure 3 and Figure 4 The method and its detailed scheme are shown.

[0131] It should be understood that the above-described device embodiments are merely illustrative, and the device disclosed in this invention can be implemented in other ways. For example, the division of units / modules described in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, integrated into another system, or some features may be ignored or not executed.

[0132] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of the present invention can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0133] If the integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor or chip can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the on-chip cache, off-chip memory, and storage can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0134] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer electronic device (which may be a personal computer, server, or network electronic device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0135] This application embodiment also provides a non-transitory computer storage medium storing a computer program, which, when executed by multiple processors, causes the processors to perform actions such as... Figure 2 , Figure 3 and Figure 4 The method and its detailed scheme are shown.

[0136] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, and on the specific implementation methods and application scope of this application, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A gradient direction-based camera stain detection method, characterized by, include: Extract the brightness map of the image from the image captured by the camera device; The lateral gradient difference and the longitudinal gradient difference are calculated on the brightness map to obtain the lateral gradient map and the longitudinal gradient map. A blemish screening map is obtained by performing blemish screening using the horizontal gradient map and the vertical gradient map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values; and The pixels in the blemish screening image whose values ​​correspond to the preset blemish value are identified as blemishes in the image. The step of performing taint screening using the horizontal gradient map and the vertical gradient map to obtain a taint screening map includes: For any pixel located in the horizontal gradient map and the vertical gradient map, if the value of the pixel in the horizontal gradient map and the vertical gradient map does not meet the preset judgment condition corresponding to the location of the pixel, the value corresponding to the pixel is determined as the preset taint value; and the taint screening map is determined by the preset taint value. The method further includes: after obtaining the blemish screening image, performing mean filtering on the blemish screening image to obtain a filtered screening image; and correcting the corresponding value of the pixel in the blemish screening image according to the relationship between the corresponding value of any pixel in the filtered screening image and a preset blemish judgment threshold. The method further includes: determining a portion of the center of the brightness map at a preset proportion as the center brightness map; Calculate the average brightness of all pixels in the brightness map; The brightness threshold is determined by the brightness value in the histogram of the center brightness map that is greater than the average brightness value and has the highest number of pixels. Based on the relationship between the brightness value of any pixel in the brightness map and the brightness threshold, the value corresponding to the pixel in the blemish screening map is corrected.

2. The detection method as described in claim 1, characterized in that, The step of correcting the values ​​corresponding to the pixels in the blemish screening image includes: If the value corresponding to any pixel in the filtered screening image is greater than the blemish judgment threshold, the value corresponding to the pixel in the blemish screening image is determined as the preset blemish value.

3. The detection method as described in claim 1, characterized in that, The step of calculating the horizontal gradient difference and the vertical gradient difference of the brightness map to obtain the horizontal gradient map and the vertical gradient map includes: The horizontal gradient map is determined based on the difference between the brightness value of any pixel in the brightness map and the brightness values ​​of its horizontally adjacent pixels; and The vertical gradient map is determined based on the difference between the brightness value of any pixel in the brightness map and the brightness value of its vertically adjacent pixel.

4. The detection method according to any one of claims 1 to 3, characterized in that, The step of determining the pixels in the blemish screening image whose values ​​correspond to the preset blemish value as blemishes in the image includes: For any connected region in the taint screening map, if the number of pixels with the preset taint value is greater than a preset threshold, the connected region is determined to be a taint.

5. A stain detection device for a camera based on gradient direction, characterized in that, include: An extraction module is used to extract the brightness map of the image from the image captured by the camera device; The gradient map calculation module is used to calculate the horizontal gradient difference and the vertical gradient difference of the brightness map to obtain the horizontal gradient map and the vertical gradient map. The module is configured to perform blemish screening using the horizontal gradient map and the vertical gradient map to obtain a blemish screening map, wherein the values ​​corresponding to pixels in the blemish screening map include preset blemish values ​​and non-preset blemish values; wherein, the step of performing blemish screening using the horizontal gradient map and the vertical gradient map to obtain the blemish screening map includes: For any pixel located in the horizontal gradient map and the vertical gradient map, if the value of the pixel in the horizontal gradient map and the vertical gradient map does not meet the preset judgment condition corresponding to the pixel's location, the value corresponding to the pixel is determined as the preset blemish value; and the blemish screening map is determined using the preset blemish value; and A blemish determination module is used to determine pixels in the blemish screening image that have a value corresponding to the preset blemish value as blemishes in the image. The blemish confirmation module is further configured to: after obtaining the blemish screening image, perform mean filtering on the blemish screening image to obtain a filtered screening image; and correct the corresponding value of the pixel in the blemish screening image according to the relationship between the corresponding value of any pixel in the filtered screening image and a preset blemish judgment threshold. The stain confirmation module is also used to: determine the portion of the brightness map at a preset proportion as the center brightness map; Calculate the average brightness of all pixels in the brightness map; The brightness threshold is determined by the brightness value in the histogram of the center brightness map that is greater than the average brightness value and has the highest number of pixels. Based on the relationship between the brightness value of any pixel in the brightness map and the brightness threshold, the value corresponding to the pixel in the blemish screening map is corrected.

6. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method as described in any one of claims 1 to 4 when executing the computer program in the memory.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images