Camera smudginess detection method and device, computer equipment and storage medium

Abstract

The invention relates to a camera smudginess detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of to-be-detected images acquired by a to-be-detected camera; generating a gray average image and a gradient average image according to the multiple frames of images to be detected; determining a brightness decay image and a brightness enhancement image according to the gray mean image and the gradient mean image; and carrying out smudginess detection according to the brightness decay image and the brightness enhancement image. The smudginess detection is carried out based on multiple frames of to-be-detected images collected in real time, so that the real-time performance of smudginess detection is improved; according to the multi-frame to-be-detected image, the brightness decay image and the brightness enhancement image are determined, and then the smudginess detection is carried out according to the brightness decay image and the brightness enhancement image, so that the smudginess detection accuracy is further improved.

Description

Technical Field

[0001] This application relates to the technical field of image detection, and particularly to a method, device, computer device, and storage medium for detecting camera dirt. Background Art

[0002] As the result of the deep integration of modern automotive industry and information technology, intelligent driving is gradually changing people's travel modes. In an intelligent driving system, environmental perception is one of the core links to ensure the safe operation of a vehicle. As an important sensor, a camera plays an indispensable role in providing visual information. Multiple cameras are set at different positions on the vehicle end. During the process of the vehicle being stationary or moving, a driver can observe the images collected by multiple cameras in the vehicle through a display device, so that the driver can determine the driving environment and driving hazards according to the corresponding images, thereby improving driving safety.

[0003] However, cameras on the vehicle end face special challenges. For example, positions close to the ground are easily affected by dust, rain, and mud, resulting in a decline in image quality. In current related technologies, for the detection of camera dirt, it is mainly based on infrared reflection detection or image analysis under a specific target board. Infrared reflection detection requires prior calibration, and image analysis under a specific target board needs to be tested in a specific environment. Therefore, the current related technologies cannot perform dirt detection in real time, and the dirt detection accuracy is relatively low. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer device, and storage medium for detecting camera dirt in view of the above technical problems.

[0005] In a first aspect, this application provides a method for detecting camera dirt. The method includes: obtaining multiple frames of images to be detected collected by a camera to be detected; generating a grayscale mean map and a gradient mean map according to the multiple frames of images to be detected; determining a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map; and performing dirt detection according to the brightness attenuation map and the brightness enhancement map.

[0006] In one of the embodiments, the obtaining multiple frames of images to be detected collected by the camera to be detected includes: obtaining consecutive frames of original images collected by the camera to be detected; determining the change amount of the grayscale value of each pixel point according to two adjacent frames of the original images; counting the number of pixel points whose grayscale value change amount is greater than a change amount threshold; and if the number of pixel points is greater than a number threshold, using the corresponding original image as an image to be detected.

[0007] In one embodiment, generating a grayscale mean map and a gradient mean map based on multiple frames of the image to be detected includes: converting multiple frames of the image to be detected into multiple grayscale images; converting multiple frames of the image to be detected into multiple gradient images; generating the grayscale mean map based on the multiple grayscale images; and generating the gradient mean map based on the multiple gradient images.

[0008] In one embodiment, determining a brightness attenuation map and a brightness enhancement map based on the grayscale mean map and the gradient mean map includes: performing iterative surface fitting on the grayscale mean map to obtain an ideal grayscale map; performing iterative surface fitting on the gradient mean map to obtain an ideal gradient map; and determining the brightness attenuation map and the brightness enhancement map based on the grayscale mean map, the gradient mean map, the ideal grayscale map, and the ideal gradient map.

[0009] In one embodiment, performing iterative surface fitting on the grayscale mean map to obtain an ideal grayscale map includes: sorting all pixel values in the grayscale mean map from largest to smallest, and selecting a preset proportion of pixel points as first grayscale pixel points; performing surface fitting based on the first grayscale pixel points to obtain a first grayscale surface; selecting, in the grayscale mean map, pixel points with a pixel difference less than a first preset pixel threshold from the first grayscale surface as second grayscale pixel points; performing surface fitting based on the second grayscale pixel points to obtain a second grayscale surface; selecting, in the grayscale mean map, pixel points with a pixel difference less than the first preset pixel threshold from the second grayscale surface as third grayscale pixel points, and performing surface fitting on the third grayscale pixel points until a preset number of iterative surface fitting times is reached to obtain the ideal grayscale map.

[0010] In one embodiment, performing iterative surface fitting on the gradient mean map to obtain an ideal gradient map includes: sorting all pixel values in the gradient mean map from largest to smallest, and selecting a preset proportion of pixel points as first gradient pixel points; performing surface fitting based on the first gradient pixel points to obtain a first gradient surface; selecting, in the gradient mean map, pixel points with a pixel difference less than a second preset pixel threshold from the first gradient surface as second gradient pixel points; performing surface fitting based on the second gradient pixel points to obtain a second gradient surface; selecting, in the gradient mean map, pixel points with a pixel difference less than the second preset pixel threshold from the second gradient surface as third gradient pixel points, and performing surface fitting on the third gradient pixel points until a preset number of iterative surface fitting times is reached to obtain the ideal gradient map.

[0011] In one embodiment, determining the brightness attenuation map and the brightness enhancement map based on the grayscale mean map, the gradient mean map, the ideal grayscale map, and the ideal gradient map includes: determining the brightness attenuation map based on the gradient mean map and the ideal gradient map; determining the brightness enhancement map based on the grayscale mean map, the ideal grayscale map, and the brightness attenuation map.

[0012] In one embodiment, performing stain detection based on the brightness attenuation map and the brightness enhancement map includes: determining, as a stained area, an area in the brightness attenuation map where the pixel value is less than a first threshold; and / or determining, as a stained area, an area in the brightness enhancement map where the pixel value is greater than a second threshold.

[0013] In a second aspect, the present application further provides a camera stain detection device, which includes: an acquisition module for acquiring multiple frames of images to be detected collected by a camera to be measured; a first image generation module for generating a grayscale mean map and a gradient mean map based on the multiple frames of images to be detected; a second image generation module for determining a brightness attenuation map and a brightness enhancement map based on the grayscale mean map and the gradient mean map; and a detection module for performing stain detection based on the brightness attenuation map and the brightness enhancement map.

[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements any one of the camera stain detection methods in the first aspect above.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium. On the computer-readable storage medium, there is stored a computer program, and when the computer program is executed by a processor, it implements any one of the camera stain detection methods in the first aspect above.

[0016] The above camera stain detection method, device, computer device, and storage medium acquire multiple frames of images to be detected collected by a camera to be measured, generate a grayscale mean map and a gradient mean map based on the multiple frames of images to be detected, then determine a brightness attenuation map and a brightness enhancement map based on the grayscale mean map and the gradient mean map, and finally perform stain detection through the brightness attenuation map and the brightness enhancement map. Stain detection is performed based on multiple frames of images to be detected collected in real time, improving the real-time performance of stain detection; determining a brightness attenuation map and a brightness enhancement map based on multiple frames of images to be detected, and then performing stain detection based on the brightness attenuation map and the brightness enhancement map further improves the accuracy of stain detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic flowchart of a camera stain detection method in one embodiment;

[0018] Figure 2 A flowchart showing a method for determining a brightness attenuation map and a brightness enhancement map in an embodiment;

[0019] Figure 3 Multi-frame images to be detected in an embodiment;

[0020] Figure 4 The grayscale mean map in an embodiment;

[0021] Figure 5 The gradient mean map in an embodiment;

[0022] Figure 6 A structural block diagram of a camera dirt detection device in an embodiment;

[0023] Figure 7 The internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] In-vehicle cameras are one of the key sensors in intelligent driving systems. They support various functions by capturing visual information around the vehicle, such as environmental perception, lane keeping assistance, automatic emergency braking, pedestrian detection, and traffic sign recognition. There are various types of in-vehicle cameras, and their installation positions are also different to meet different application requirements. For the types of in-vehicle cameras, they usually include monocular cameras, binocular cameras, and surround-view cameras. For the positions of in-vehicle cameras, they usually include front-view cameras, rear-view cameras, side-view cameras, and underbody cameras.

[0026] For front-view cameras, rear-view cameras, and side-view cameras, they can obtain the environmental information around the vehicle, such as surrounding stone piers, children, non-motor vehicles, etc., and avoid safety accidents caused by the above objects by observing the collected images. The underbody camera can obtain the environmental information at the bottom of the vehicle. The environmental information at the bottom of the vehicle is crucial for the safety during vehicle start-up or parking. Many safety problems occur during vehicle start-up or parking, and the observation of the underbody environment is insufficient. For example, when starting the vehicle, if there are children or small animals under the vehicle and the vehicle is started without proper observation, safety accidents will occur. Cameras installed at different positions on the vehicle end are all relatively close to the ground, with a large amount of dust, and are easily contaminated by rainwater and mud. A dirty camera will affect the image quality when collecting images. Therefore, it is necessary to detect the dirt of the cameras on the vehicle end.

[0027] When detecting dirt on a camera, current related technologies generally involve detecting the intensity of infrared reflection based on infrared reflection, and detecting a decrease in image brightness or resolution based on image analysis in a specific calibration plate and test environment to determine whether there is dirt on the camera. However, for the method based on infrared reflection, calibration is required before each detection; for the method based on image analysis, calibration is also required before each detection, and detection needs to be carried out in a specific test environment. Therefore, current related technologies cannot perform real-time dirt detection, and the dirt detection accuracy is relatively low.

[0028] In one embodiment, as Figure 1 shown, a method for detecting dirt on a camera is provided, including the following steps:

[0029] Step 101, obtain multiple frames of images to be detected collected by the camera to be tested.

[0030] The camera to be tested in this embodiment is not limited to the camera on the vehicle end. Any camera that meets the detection requirements can be used as the camera to be tested. The following takes the camera on the vehicle end as an example for illustration. The camera to be tested can be any camera at any position and of any type on the vehicle end. There are changes in the image content between two adjacent frames of images to be detected in the multiple frames of images to be detected. For example, in the case of vehicle movement, multiple frames of images to be detected are collected.

[0031] Step 102, generate a grayscale mean map and a gradient mean map according to the multiple frames of images to be detected.

[0032] After obtaining the multiple frames of images to be detected, determine the grayscale value corresponding to each pixel point and the gradient value corresponding to each pixel point in each image to be detected. For the grayscale mean map, calculate the grayscale average value corresponding to the same position according to the multiple grayscale values corresponding to the same position in the multiple frames of images to be detected, then calculate the grayscale average value corresponding to each position in turn, and finally generate a grayscale mean map according to the grayscale average value corresponding to each position; wherein, the grayscale mean map represents the grayscale average value corresponding to each pixel position of the multiple frames of images to be detected. For the gradient mean map, calculate the gradient average value corresponding to the same position according to the multiple gradient values corresponding to the same position in the multiple frames of images to be detected, then calculate the gradient average value corresponding to each position in turn, and finally generate a gradient mean map according to the gradient average value corresponding to each position; wherein, the gradient mean map represents the gradient average value corresponding to each pixel position of the multiple frames of images to be detected.

[0033] Step 103, determine a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map.

[0034] Perform surface fitting on the pixels with the top - ranked gray - scale values in a preset number in the gray - scale mean map to obtain an ideal gray - scale map. Here, the ideal gray - scale map is the average gray - scale value at each pixel position in an ideal situation without dirt. Perform surface fitting on the pixels with the top - ranked gradient values in a preset number in the gradient mean map to obtain an ideal gradient map. Here, the ideal gradient map is the average gradient value at each pixel position in an ideal situation without dirt. Determine a brightness attenuation map and a brightness enhancement map based on the gray - scale mean map, the gradient mean map, the ideal gray - scale map, and the ideal gradient map. Among them, the brightness attenuation map represents the attenuation value corresponding to each pixel position of the gradient mean map relative to the ideal gradient map; the brightness enhancement map represents the enhancement value corresponding to each pixel position of the gray - scale mean map relative to the ideal gray - scale map.

[0035] Step 104, perform dirt detection based on the brightness attenuation map and the brightness enhancement map.

[0036] After obtaining the brightness attenuation map and the brightness enhancement map, determine the area where the pixel value in the brightness attenuation map is less than the first threshold as the dirty area; or, determine the area where the pixel value in the brightness enhancement map is greater than the second threshold as the dirty area, thereby completing the dirt detection and determining whether there is dirt on the camera to be tested and the area of the dirt corresponding to the image to be detected.

[0037] In this embodiment, by acquiring multiple frames of images to be detected collected by the camera to be tested, generating a gray - scale mean map and a gradient mean map based on the multiple frames of images to be detected, then determining a brightness attenuation map and a brightness enhancement map based on the gray - scale mean map and the gradient mean map, and finally performing dirt detection through the brightness attenuation map and the brightness enhancement map. Performing dirt detection based on multiple frames of images to be detected collected in real - time improves the real - time performance of dirt detection; determining a brightness attenuation map and a brightness enhancement map based on multiple frames of images to be detected, and then performing dirt detection based on the brightness attenuation map and the brightness enhancement map further improves the accuracy of dirt detection.

[0038] In this embodiment, the multiple frames of images to be detected collected need to have changes, that is, there are changes in the image content between two adjacent frames among the multiple frames of images to be detected. If the vehicle is stationary, that is, each frame of the image to be detected does not change, then dirt detection cannot be performed. The camera dirt detection method provided in this embodiment does not require a fixed calibration board, only requires the vehicle to move, and collect multiple frames of images to be detected when the vehicle is moving, so as to be able to perform dirt detection in real - time.

[0039] In one of the embodiments, acquiring multiple frames of images to be detected specifically includes the following steps:

[0040] Step 1, acquire consecutive frames of original images collected by the camera to be tested.

[0041] While the vehicle is in motion, obtain the original images of consecutive frames captured by the camera to be measured. The original images of consecutive frames are a series of consecutive original images.

[0042] Step 2: Determine the change amount of the gray value of each pixel point according to the original images of two adjacent frames.

[0043] After multiple original images are captured, it is necessary to determine whether there is a change in the image content between two adjacent original images. First, it is necessary to calculate the change amount of the gray value of each pixel point in two adjacent original images. For each original image, first determine the gray value of each pixel point in each original image. For example, the average value of the three RGB channels corresponding to each pixel point can be used as the gray value, or the value of any one of the three RGB channels can be used as the gray value. After obtaining the gray value of each pixel point in each original image, determine the change amount of the gray value of the pixel points at the same position in two adjacent original images. The specific calculation is as follows:

[0044] △I(x, y) = I k (x, y) - I k-1 (x, y)

[0045] where x represents the abscissa of the pixel point in the image, y represents the ordinate of the pixel point in the image, I k represents the gray value of the pixel point in the previous original image among two adjacent original images, and I k-1 represents the gray value of the pixel point in the subsequent original image among two adjacent original images, and △I represents the change amount of the gray value.

[0046] Step 3: Count the number of pixel points whose gray value change amount is greater than the change amount threshold.

[0047] Obtain the change amount of the gray value corresponding to each pixel point, compare the change amount of the gray value corresponding to each pixel point with the change amount threshold, and count the number of pixel points whose gray value change amount is greater than the change amount threshold. Specifically, compare the change amount of the gray value of all points in △I(x, y) with the change amount threshold. If △I(x, y) is greater than the change amount threshold, the number of pixel points is incremented by 1 until the comparison of the change amount of the gray value of all points is completed, and the final number of pixel points is obtained. Among them, the change amount threshold needs to be set according to the actual usage scenario, and no specific limitation is made in this embodiment. Preferably, the change amount threshold can be 40. By using the change amount threshold, the influence of noise on the pixel value can be excluded, and further improve the accuracy of determining whether there is a change in the image content between two adjacent original images.

[0048] Step 4: If the number of pixel points is greater than the number threshold, use the corresponding original image as the image to be detected.

[0049] If the number of pixel points with a gray value change amount greater than the change amount threshold in two original images is greater than the quantity threshold, the corresponding original image is regarded as the image to be detected. Among them, the quantity threshold needs to be set according to the actual usage scenario, which is not specifically limited in this embodiment. Preferably, 10% of the total number of pixel points in the original image can be used as the quantity threshold.

[0050] Specifically, the camera to be tested captures 10 original images. First, the first original image and the second original image are compared. If the number of pixel points with a gray value change amount greater than the change amount threshold in the first original image and the second original image is greater than the quantity threshold, the first original image and the second original image are regarded as the images to be detected. Then, the second original image and the third original image are compared. If the number of pixel points with a gray value change amount greater than the change amount threshold in the second original image and the third original image is greater than the quantity threshold, the third original image is regarded as the image to be detected. Then, the third original image and the fourth original image are compared. If the number of pixel points with a gray value change amount greater than the change amount threshold in the third original image and the fourth original image is less than or equal to the quantity threshold, the fourth original image cannot be regarded as the image to be detected. Then, the third original image and the fifth original image are compared. If the number of pixel points with a gray value change amount greater than the change amount threshold in the third original image and the fifth original image is greater than the quantity threshold, the fifth original image is regarded as the image to be detected. And so on, all the images to be detected are determined.

[0051] By determining multiple images to be detected based on the gray value change amount in multiple consecutive frames of original images, so as to obtain the images to be detected with relatively large changes in image content, the accuracy of dirt detection can be further improved.

[0052] In one embodiment, generating a gray mean value map and a gradient mean value map specifically includes the following steps:

[0053] After accumulating the preset number of frames of images to be detected, a gray mean value map and a gradient mean value map are generated according to multiple frames of images to be detected. Among them, the preset number of frames can be set according to the actual application scenario, which is not specifically limited in this embodiment. Preferably, the preset number of frames can be 200 frames.

[0054] Step 1, convert multiple frames of images to be detected into multiple gray maps.

[0055] Convert multiple frames of images to be detected into multiple grayscale images. Specifically, for each image to be detected, determine the grayscale value of each pixel. The average value of the RGB channels corresponding to each pixel can be used as the grayscale value, or the value of any one of the RGB channels can be used as the grayscale value. Generate a grayscale image corresponding to the image to be detected according to the grayscale value of each pixel in each image to be detected. Convert each image to be detected into a corresponding grayscale image.

[0056] Step 2: Convert multiple frames of images to be detected into multiple gradient images.

[0057] Convert multiple frames of images to be detected into multiple gradient images. Specifically, for each image to be detected, determine the gradient value of each pixel. For example, when calculating the gradient value of the current pixel, first obtain the grayscale values of the two adjacent pixels on the left and right in the same row; then obtain the grayscale values of the two adjacent pixels above and below in the same column; calculate the absolute value of the difference according to the grayscale values of the two adjacent pixels on the left and right to obtain the first absolute value of the difference; calculate the absolute value of the difference according to the grayscale values of the two adjacent pixels above and below to obtain the second absolute value of the difference; finally, sum the first absolute value of the difference and the second absolute value of the difference to obtain the gradient value of the current pixel. Calculate the gradient values of the remaining pixels except the edge pixels in the image to be detected based on the above method, and then set the gradient values of the edge pixels to 0 to generate the gradient image corresponding to the image to be detected. Convert each image to be detected into a corresponding gradient image. The formula for calculating the gradient value of a pixel is as follows:

[0058] ▽k(x, y)

[0059] =|I k (x + 1, y) - I k (x - 1, y)|

[0060] +|I k (x, y + 1) - I k (x, y - 1)|

[0061] where x represents the abscissa of the pixel in the image, y represents the ordinate of the pixel in the image, I k represents the grayscale value of the pixel in the image to be detected, and ▽k represents the gradient value of the pixel.

[0062] Step 3: Generate a grayscale average image according to multiple grayscale images.

[0063] Calculate the grayscale average value corresponding to the same pixel position according to the multiple grayscale values corresponding to the same pixel position in multiple grayscale images, calculate the grayscale average value corresponding to each pixel position in turn, and finally generate a grayscale average image according to the grayscale average value corresponding to each pixel position.

[0064] Step 4: Generate a gradient mean map based on multiple gradient maps.

[0065] Based on the multiple gradient values corresponding to the same pixel position in the multiple gradient maps, calculate the gradient average value of the corresponding pixel position. Calculate the gradient average value corresponding to each pixel position in turn. Finally, generate a gradient mean map based on the gradient average value corresponding to each pixel position.

[0066] Generate a grayscale mean map and a gradient mean map based on multiple images to be detected. The grayscale mean map and the gradient mean map can better reflect the grayscale features and gradient features in the multiple images to be detected, thereby further improving the accuracy of dirt detection.

[0067] In one embodiment, as Figure 2 shown, a method for determining a brightness attenuation map and a brightness enhancement map is provided, including the following steps:

[0068] Step 201: Perform iterative surface fitting on the grayscale mean map to obtain an ideal grayscale map.

[0069] Sort all pixel values in the grayscale mean image from largest to smallest, and select a preset proportion of pixel points as the first grayscale pixel points. Specifically, first count the pixel values of all pixel points in the grayscale mean image, and sort the pixel values of all pixel points in descending order. After the sorting is completed, select the preset proportion of pixel points with higher rankings as the first grayscale pixel points. Among them, the preset proportion can be set according to the actual application scenario, and no specific limitation is made in this embodiment. Perform surface fitting based on the first grayscale pixel points to obtain the first grayscale surface. Among them, surface fitting can be performed by algorithms such as the least squares method, polynomial fitting, and spline interpolation. No specific limitation is made on the curve fitting method in this embodiment. Among them, the first grayscale surface can be an image with the same size as the grayscale mean image. The pixel value of each pixel point in the first grayscale surface is the grayscale average value of each pixel position in the ideal situation without dirt obtained by fitting. Select pixel points in the grayscale mean image whose pixel difference from the first grayscale surface is less than the first preset pixel threshold as the second grayscale pixel points. After obtaining the first grayscale surface by fitting, compare the pixel value of each pixel point in the grayscale mean image with the pixel value of the corresponding pixel position in the first grayscale surface respectively to determine the pixel difference. And select the pixel points in the grayscale mean image whose pixel difference is less than the first preset pixel threshold as the second grayscale pixel points. Perform surface fitting based on the second grayscale pixel points to obtain the second grayscale surface; the second grayscale surface can be an image with the same size as the grayscale mean image. The pixel value of each pixel point in the second grayscale surface is the grayscale average value of each pixel position in the ideal situation without dirt obtained by fitting. Select pixel points in the grayscale mean image whose pixel difference from the second grayscale surface is less than the first preset pixel threshold as the third grayscale pixel points, and perform surface fitting on the third grayscale pixel points until the preset iterative surface fitting times are reached to obtain the ideal grayscale image. After obtaining the second grayscale surface by fitting, continue to compare the pixel value of each pixel point in the grayscale mean image with the pixel value of the corresponding pixel position in the second grayscale surface respectively to determine the pixel difference. And select the pixel points in the grayscale mean image whose pixel difference is less than the first preset pixel threshold as the third grayscale pixel points, and perform curve fitting based on the third grayscale pixel points to obtain the third grayscale surface. Repeat the above steps until the preset iterative surface fitting times are reached to obtain the ideal grayscale image. Among them, the preset iterative surface fitting times need to be set according to the actual application scenario, and no specific limitation is made in this embodiment.

[0070] Specifically, the grayscale mean image is Aver(In), and the ideal grayscale image is Aver(I0). First, in Aver(In), select the top 50% of the pixels with larger pixel values as the starting point for the first curve fitting, and after fitting, obtain a surface S1. Then, according to surface S1, in the Aver(In) distribution, count the pixels whose pixel values are close to those of S1, and perform the second fitting based on the counted pixels to obtain surface S2. Repeat the steps of counting pixels and fitting according to surface Sk to obtain surface Sk+1. When k = 100, stop the loop calculation and output Sk+1 as Aver(I0).

[0071] Step 202: Perform iterative surface fitting on the gradient mean image to obtain the ideal gradient image.

[0072] Sort all pixel values in the gradient mean map from largest to smallest, and select a preset proportion of pixel points as the first gradient pixel points. Specifically, the pixel values of all pixel points in the gradient mean map can be counted first, and the pixel values of all pixel points are sorted in descending order. After the sorting is completed, the preset proportion of pixel points with higher rankings are selected as the first gradient pixel points. Among them, the preset proportion can be set according to the actual application scenario, and no specific limitation is made in this embodiment. Perform surface fitting according to the first gradient pixel points to obtain the first gradient surface. Among them, surface fitting can be performed by algorithms such as the least squares method, polynomial fitting, and spline interpolation. No specific limitation is made on the curve fitting method in this embodiment. Among them, the first gradient surface can be an image with the same size as the gradient mean map. The pixel value of each pixel point in the first gradient surface is the average gradient value of each pixel position in the ideal situation without dirt obtained by fitting. Select pixel points in the gradient mean map whose pixel difference from the first gradient surface is less than the second preset pixel threshold as the second gradient pixel points. After the first gradient surface is obtained by fitting, compare the pixel value of each pixel point in the gradient mean map with the pixel value of the corresponding pixel position in the first gradient surface respectively to determine the pixel difference. And the pixel points in the gradient mean map with pixel differences less than the second preset pixel threshold are used as the second gradient pixel points. Perform surface fitting according to the second gradient pixel points to obtain the second gradient surface; the second gradient surface can be an image with the same size as the gradient mean map. The pixel value of each pixel point in the second gradient surface is the average gradient value of each pixel position in the ideal situation without dirt obtained by fitting. Select pixel points in the gradient mean map whose pixel difference from the second gradient surface is less than the second preset pixel threshold as the third gradient pixel points, and perform surface fitting on the third gradient pixel points until the preset number of iterative surface fitting times is reached to obtain the ideal gradient map. After the second gradient surface is obtained by fitting, continue to compare the pixel value of each pixel point in the gradient mean map with the pixel value of the corresponding pixel position in the second gradient surface respectively to determine the pixel difference. And the pixel points in the gradient mean image with pixel differences less than the second preset pixel threshold are used as the third gradient pixel points, and curve fitting is performed based on the third gradient pixel points to obtain the third gradient surface. Repeat the above steps until the preset number of iterative surface fitting times is reached to obtain the ideal gradient map. Among them, the preset number of iterative surface fitting times needs to be set according to the actual application scenario, and no specific limitation is made in this embodiment.

[0073] Specifically, the gradient mean map is Aver(▽In), and the ideal gradient map is Aver(▽I0). First, in Aver(▽In), select the top 50% of the pixels with larger pixel values as the starting points for the first curve fitting, and after fitting, a surface R1 is obtained. Then, according to the surface R1, in the Aver(▽In) distribution, count the pixels whose pixel values are close to those of R1, and perform the second fitting based on the counted pixels to obtain the surface R2. Repeat the steps of counting pixels and fitting according to the surface Rk to obtain the surface Rk+1. When k = 100, stop the loop calculation and output Rk+1 as Aver(▽I0).

[0074] Step 203: Determine the brightness attenuation map and the brightness enhancement map according to the grayscale mean map, the gradient mean map, the ideal grayscale map, and the ideal gradient map.

[0075] Determine the brightness attenuation map according to the gradient mean map and the ideal gradient map. Specifically, divide the pixel value of each pixel point in the gradient mean map by the pixel value at the corresponding pixel position in the ideal gradient map to obtain the attenuation value of each pixel point. According to the attenuation values of each pixel point, obtain the brightness attenuation map. The specific formula is as follows:

[0076] Alpha = Aver(▽In) / Aver(▽I0)

[0077] Where Alpha is the brightness attenuation map, Aver(▽In) is the gradient mean map, and Aver(▽I0) is the ideal gradient map.

[0078] Determine the brightness enhancement map according to the grayscale mean map, the ideal grayscale map, and the brightness attenuation map. Specifically, multiply the pixel value of each pixel point in the ideal grayscale map by the pixel value at the corresponding pixel position in the brightness attenuation map to obtain the transition value of each pixel point. Subtract the pixel value of each pixel point in the grayscale mean map from the corresponding transition value to obtain the enhancement value of each pixel point. According to the enhancement values of each pixel point, obtain the brightness enhancement map. The specific formula is as follows:

[0079] Beta = Aver(In) - Aver(I0) * Alpha

[0080] Where Alpha is the brightness attenuation map, Beta is the brightness enhancement map, Aver(In) is the grayscale mean map, and Aver(I0) is the ideal grayscale map.

[0081] By determining the ideal grayscale image based on the grayscale mean image and the ideal gradient image based on the gradient mean image, finally, based on the grayscale mean image, the gradient mean image, the ideal grayscale image, and the ideal gradient image, the brightness attenuation image and the brightness enhancement image are determined. The dirt detection of the camera is carried out based on the brightness attenuation image and the brightness enhancement image, thereby improving the accuracy of dirt detection.

[0082] In one embodiment, the dirt detection is carried out according to the brightness attenuation image and the brightness enhancement image, which specifically includes: determining the dirty area as the area where the pixel value in the brightness attenuation image is less than the first threshold; and / or determining the dirty area as the area where the pixel value in the brightness enhancement image is greater than the second threshold.

[0083] After obtaining the brightness attenuation image, the pixel value of each pixel point in the brightness attenuation image is compared with the first threshold, the pixel points with pixel values less than the first threshold are counted, and the corresponding area is used as the dirty area. After obtaining the brightness enhancement image, the pixel value of each pixel point in the brightness enhancement image is compared with the second threshold, the pixel points with pixel values greater than the second threshold are counted, and the corresponding area is used as the dirty area. It can be understood that in this embodiment, the dirty area can be determined only based on the brightness attenuation image, or only based on the brightness enhancement image, or the dirty area can be determined by combining the brightness attenuation image and the brightness enhancement image.

[0084] In this embodiment, the brightness attenuation image and the brightness enhancement image are two images. By comparing the pixel values of each pixel point in the two images with the corresponding thresholds respectively, it can be obtained whether there is dirt on the corresponding pixel points.

[0085] In one embodiment, when a dirty area is detected, a user can be prompted, alarmed, etc., and further, automatic cleaning can be performed. When no dirty area is detected or the dirt does not affect the image quality, there is no need for prompting or alarming.

[0086] In one embodiment, the camera dirt detection method can be applied to a π-mirror camera. The π-mirror camera has a 360° annular field of view, and this π-mirror camera is used for collecting the information of the vehicle bottom environment. By detecting the dirt of the π-mirror camera, the image quality and effect of the π-mirror camera are improved, and further, the safety of the automatic parking function is increased.

[0087] In the embodiment of the present application, the dirt position and dirt degree of the camera can be represented by specific values. If the dirt does not reach a certain degree, the dirt can be ignored; if the dirt exceeds a certain degree, actions such as prompting, alarming, and automatic cleaning can be started. As Figures 3 - 5 shown, Figure 3 are multiple frames of images to be detected, Figure 4 is the grayscale mean image, Figure 5is the gradient mean map. In the grayscale mean map and the gradient mean map, the dirty area can be clearly observed, that is, the dirt position and dirt degree can be easily solved based on the grayscale mean map and the gradient mean map. The camera dirt detection method of this embodiment does not need to set a detection environment for calibrating the camera. It only needs the movement of the vehicle to detect the location and severity of the camera dirt.

[0088] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0089] Based on the same inventive concept, the embodiment of the present application also provides a camera dirt detection device for implementing the camera dirt detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more camera dirt detection device embodiments provided below can refer to the limitations of the camera dirt detection method above, and will not be repeated here.

[0090] In one embodiment, Figure 6 As shown, a camera dirt detection device is provided, comprising: an acquisition module 100, a first image generation module 200, a second image generation module 300 and a detection module 400, wherein:

[0091] The acquisition module 100 is used to acquire multiple frames of images to be detected captured by the camera to be detected.

[0092] The first image generation module 200 is used to generate a grayscale mean image and a gradient mean image according to multiple frames of the image to be detected.

[0093] The second image generation module 300 is used to determine a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map.

[0094] The detection module 400 is used to perform dirt detection according to the brightness attenuation map and the brightness enhancement map.

[0095] The acquisition module 100 is further configured to acquire the continuous-frame original images collected by the camera to be measured; determine the change amount of the gray value of each pixel point according to the original images of two adjacent frames; count the number of pixel points whose gray value change amount is greater than the change amount threshold; if the number of pixel points is greater than the number threshold, use the corresponding original image as the image to be detected.

[0096] The first image generation module 200 is further configured to convert multiple frames of the images to be detected into multiple grayscale images; convert multiple frames of the images to be detected into multiple gradient images; generate the grayscale mean image according to the multiple grayscale images; generate the gradient mean image according to the multiple gradient images.

[0097] The second image generation module 300 is further configured to perform iterative surface fitting on the grayscale mean image to obtain an ideal grayscale image; perform iterative surface fitting on the gradient mean image to obtain an ideal gradient image; determine the brightness attenuation image and the brightness enhancement image according to the grayscale mean image, the gradient mean image, the ideal grayscale image, and the ideal gradient image.

[0098] The second image generation module 300 is further configured to sort all pixel values in the grayscale mean image from large to small, and select pixel points with a preset ratio as the first grayscale pixel points; perform surface fitting according to the first grayscale pixel points to obtain the first grayscale surface; select pixel points in the grayscale mean image whose pixel difference from the first grayscale surface is less than the first preset pixel threshold as the second grayscale pixel points; perform surface fitting according to the second grayscale pixel points to obtain the second grayscale surface; select pixel points in the grayscale mean image whose pixel difference from the second grayscale surface is less than the first preset pixel threshold as the third grayscale pixel points, and perform surface fitting on the third grayscale pixel points until the preset iterative surface fitting times are reached to obtain the ideal grayscale image.

[0099] The second image generation module 300 is further configured to sort all pixel values in the gradient mean image from large to small, and select pixel points with a preset ratio as the first gradient pixel points; perform surface fitting according to the first gradient pixel points to obtain the first gradient surface; select pixel points in the gradient mean image whose pixel difference from the first gradient surface is less than the second preset pixel threshold as the second gradient pixel points; perform surface fitting according to the second gradient pixel points to obtain the second gradient surface; select pixel points in the gradient mean image whose pixel difference from the second gradient surface is less than the second preset pixel threshold as the third gradient pixel points, and perform surface fitting on the third gradient pixel points until the preset iterative surface fitting times are reached to obtain the ideal gradient image.

[0100] The second image generation module 300 is further configured to determine the brightness attenuation map according to the gradient mean map and the ideal gradient map; and determine the brightness enhancement map according to the gray mean map, the ideal gray map, and the brightness attenuation map.

[0101] The detection module 400 is further configured to determine the contaminated area as the area where the pixel value in the brightness attenuation map is less than the first threshold; and / or determine the contaminated area as the area where the pixel value in the brightness enhancement map is greater than the second threshold.

[0102] Each module in the above camera contamination detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0103] In one embodiment, a computer device is provided. The computer device can be an in-vehicle terminal, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a camera contamination detection method.

[0104] Those skilled in the art can understand that Figure 7 the structure shown in

[0105] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0106] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements any one of the camera contamination detection methods in the above embodiments.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0108] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0109] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A camera dirt detection method, characterized in that: The method comprises: Obtain multiple frames of images to be detected captured by the camera to be tested; Generate a grayscale mean map and a gradient mean map according to the multiple frames of the image to be detected; Determine a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map; Dirt detection is performed according to the brightness attenuation map and the brightness enhancement map.

2. The method according to claim 1, characterized in that The step of obtaining multiple frames of images to be detected collected by the camera to be detected comprises: Obtaining continuous frame original images captured by the camera to be tested; Determine the grayscale value change of each pixel according to the original image of two adjacent frames; Counting the number of pixels whose grayscale value change is greater than a change threshold; If the number of pixels is greater than the number threshold, the corresponding original image is used as the image to be detected.

3. The method according to claim 1, characterized in that The step of generating a grayscale mean map and a gradient mean map according to the plurality of frames of the image to be detected comprises: Converting multiple frames of the to-be-detected images into multiple grayscale images; Converting multiple frames of the to-be-detected images into multiple gradient images; Generating the grayscale mean image according to the plurality of grayscale images; The gradient mean map is generated according to the multiple gradient maps.

4. The method according to claim 1, characterized in that: Determining a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map includes: Performing iterative surface fitting on the grayscale mean image to obtain an ideal grayscale image; Performing iterative surface fitting on the gradient mean map to obtain an ideal gradient map; A brightness attenuation map and a brightness enhancement map are determined according to the grayscale mean map, the gradient mean map, the ideal grayscale map, and the ideal gradient map.

5. The method according to claim 4, characterized in that The iterative surface fitting of the grayscale mean image to obtain an ideal grayscale image comprises: Sort all pixel values ​​in the grayscale mean image from large to small, and select a preset proportion of pixel points as first grayscale pixel points; Performing surface fitting according to the first grayscale pixel points to obtain a first grayscale surface; Selecting, in the grayscale mean map, a pixel point whose pixel difference with the first grayscale curved surface is less than a first preset pixel threshold as a second grayscale pixel point; Performing surface fitting according to the second grayscale pixel points to obtain a second grayscale surface; In the grayscale mean map, a pixel point whose pixel difference with the second grayscale surface is less than a first preset pixel threshold is selected as a third grayscale pixel point, and a surface fitting is performed on the third grayscale pixel point until a preset number of iterative surface fitting times is reached to obtain an ideal grayscale map.

6. The method according to claim 4, characterized in that The iterative surface fitting of the gradient mean map to obtain an ideal gradient map comprises: Sort all pixel values ​​in the gradient mean map from large to small, and select a preset proportion of pixel points as first gradient pixel points; Performing surface fitting according to the first gradient pixel points to obtain a first gradient surface; Selecting, in the gradient mean map, a pixel point whose pixel difference with the first gradient surface is less than a second preset pixel threshold as a second gradient pixel point; Performing surface fitting according to the second gradient pixel points to obtain a second gradient surface; In the gradient mean map, pixel points whose pixel difference with the second gradient surface is less than a second preset pixel threshold are selected as third gradient pixel points, and surface fitting is performed on the third gradient pixel points until a preset number of iterative surface fitting times is reached to obtain an ideal gradient map.

7. The method according to claim 4, characterized in that Determining a brightness attenuation map and a brightness enhancement map according to the grayscale mean map, the gradient mean map, the ideal grayscale map, and the ideal gradient map includes: Determining the brightness attenuation map according to the gradient mean map and the ideal gradient map; The brightness enhancement map is determined according to the grayscale mean map, the ideal grayscale map and the brightness attenuation map.

8. The method according to claim 1, characterized in that The performing dirt detection according to the brightness attenuation map and the brightness enhancement map comprises: Determine the area in the brightness attenuation map where the pixel value is less than the first threshold as a dirty area; and / or The area in the brightness enhancement image where the pixel value is greater than the second threshold is determined as a dirty area.

9. A camera dirt detection device, characterized in that: The device comprises: An acquisition module is used to acquire multiple frames of images to be detected captured by the camera to be tested; A first image generation module, used to generate a grayscale mean image and a gradient mean image according to multiple frames of the image to be detected; A second image generation module, used for determining a brightness attenuation map and a brightness enhancement map according to the grayscale mean map and the gradient mean map; A detection module is used to perform dirt detection according to the brightness attenuation map and the brightness enhancement map.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images