Image optimization method of self-closed loop system of vehicle lamp integrated camera

Abstract

The invention discloses an image optimization method for a self-closed loop system of a vehicle lamp integrated camera, and belongs to the technical field of vehicle-mounted visual systems. The method comprises the following steps that S1, modeling is conducted on the light transmission uniformity problem of a car lamp lampshade; s2, performing distortion correction on an image captured by a camera integrated in the vehicle lamp; and S3, performing defect compensation processing on image abnormal pixel points caused by the surface defects of the lampshade of the automobile lamp. The invention provides an image optimization method for a self-closed-loop system of a vehicle lamp integrated camera, which effectively eliminates the influence of a lampshade on the quality of an image captured by the camera, remarkably improves the definition and accuracy of the image, solves the main technical difficulty of integration of the camera and the vehicle lamp, improves the robustness of the system in a driving environment, and improves the user experience. The driving safety is improved, and a favorable guarantee is provided for accurate judgment and decision making of an intelligent driving system.

Description

Technical Field

[0001] The present invention relates to an image optimization method for a self-closed loop system of a vehicle lamp integrated with a camera, belonging to the technical field of vehicle-mounted vision systems. Background Art

[0002] At present, with the rapid development of intelligent driving technology, vehicle lamps not only undertake the lighting function, but also are often integrated with sensor devices such as cameras to provide information required for assisted driving and intelligent navigation. However, as an important part of the vehicle lamp, the characteristics of the lamp cover often have an adverse impact on the image quality captured by the camera.

[0003] In the existing technology, the problem of light transmission uniformity of the lamp cover may cause uneven display of the images captured by the camera. The shape and material of the lamp cover may cause image distortion. Defects on the surface of the lamp cover, such as scratches and bubbles, will form bright or dark spots in the image, seriously affecting the clarity of the camera imaging and the accuracy of image recognition. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide an image optimization method for a self-closed loop system of a vehicle lamp integrated with a camera, which effectively removes the various adverse effects of the lamp cover on the image quality captured by the camera, significantly improves the clarity and accuracy of the image, solves the main technical difficulties in the integration of the camera and the vehicle lamp, improves the robustness of the system in the driving environment, enhances the driving safety, and provides a favorable guarantee for the accurate judgment and decision-making of the intelligent driving system.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] The present invention provides an image optimization method for a self-closed loop system of a vehicle lamp integrated with a camera, which includes the following steps:

[0007] Step S1, model the problem of light transmission uniformity of the vehicle lamp cover;

[0008] Step S2, perform distortion correction on the images captured by the camera integrated inside the vehicle lamp;

[0009] Step S3, perform defect compensation processing on the abnormal pixel points in the images caused by the defects on the surface of the vehicle lamp cover.

[0010] Further, in the step S1, modeling the problem of light transmission uniformity of the vehicle lamp cover specifically includes the following steps:

[0011] Step S11, calculate the transmittance of the ambient light around the vehicle on the vehicle lamp cover based on the light transmittance, thickness variation and surface roughness of the vehicle lamp cover material;

[0012] Step S12: Preprocess the image captured by the camera integrated inside the vehicle lamp;

[0013] Step S13: Perform light compensation on the preprocessed image.

[0014] Furthermore, in the step S11, the calculation formula for the transmittance of the ambient light around the vehicle on the vehicle lamp cover is as follows:

[0015] T(λ,x,y)=T0(λ)×e -α ( λ ) ×d ( x , y )×ρ(x,y);

[0016] Wherein, T(λ,x,y) is the transmittance of the ambient light around the vehicle with wavelength λ at the position (x,y) of the vehicle lamp cover;

[0017] T0(λ) is the initial light transmittance of the lamp cover material;

[0018] α(λ) is the light absorption coefficient of the lamp cover material;

[0019] d(x,y) is the thickness of the lamp cover at the position (x,y);

[0020] ρ(x,y) is the light reflectance of the lamp cover surface at the position (x,y).

[0021] Furthermore, in the step S12, the calculation formula for the preprocessing is as follows:

[0022] I denoised (x,y)=Filter(I raw (x,y));

[0023] Wherein, I raw (x,y) is the original image captured by the camera;

[0024] I denoised (x,y) is the image after preprocessing;

[0025] Filter is the filtering operation.

[0026] Furthermore, in the step S13, the calculation formula for the light compensation is as follows:

[0027]

[0028] Wherein, I corrected (x,y) is the image after light compensation;

[0029] Idenoised (x, y) is the preprocessed image;

[0030] T(λ, x, y) is the transmittance of the ambient light around the vehicle with wavelength λ at the position (x, y) of the headlight lens.

[0031] Further, in the step S2, the calculation formula for the distortion correction is as follows:

[0032] x′ = x + k1x(x 2 + y 2 ) + k2y(x 2 + y 2 ) + p1(3x 2 + y 2 ) + 2p2xy;

[0033] y′ = y + k2x(x 2 + y 2 ) + k1y(x 2 + y 2 ) + p2(x 2 + 3y 2 ) + 2p1xy;

[0034] where, (x, y) are the pixel coordinates of the distorted image in the image captured by the camera;

[0035] (x′, y′) are the pixel coordinates of the image after distortion correction;

[0036] k1 and k2 are the radial distortion coefficients;

[0037] p1 and p2 are the tangential distortion coefficients.

[0038] Further, in the step S3, the abnormal pixel points in the image caused by the surface defects of the headlight lens are subjected to defect compensation processing, which specifically includes the following steps:

[0039] Step S31: Detect and identify the defects in the image captured by the camera, and obtain the coordinates of the abnormal pixel points in the image;

[0040] Step S32: Perform weighted average interpolation processing on the identified abnormal pixel points to obtain the interpolated pixel values;

[0041] Step S33: Smooth the image after interpolation processing to obtain the final image after smoothing processing;

[0042] Further, in the step S31, detecting and identifying the defects in the image captured by the camera and obtaining the coordinates of the abnormal pixel points in the image specifically includes the following steps:

[0043] Step S311: Calculate the local variance of each pixel point in the image captured by the camera;

[0044] Step S312: Based on the local variance of each pixel point and a set local variance threshold, identify the coordinates of abnormal pixel points in the image.

[0045] Furthermore, in the said step S32, the calculation formula of the interpolated pixel value is as follows:

[0046]

[0047] where I′(x,y) is the pixel value after interpolation of the abnormal pixel point;

[0048] N(x,y) is the set of normal pixel points around the abnormal pixel point;

[0049] w(i,j) is the weight;

[0050] I(i,j) is the gray value of the pixel point within the W window;

[0051] σ is the standard deviation of the Gaussian function.

[0052] Furthermore, in the said step S33, the calculation formula of the finally smoothed image is as follows:

[0053] I″(x,y) = (I′ * G)(x,y) = ∑ (i,j) I′(i,j) × G(x - i,y - j);

[0054]

[0055] where I"(x,y) is the finally smoothed image;

[0056] G(x,y) is the Gaussian filter;

[0057] I′(x,y) is the pixel value after interpolation of the abnormal pixel point;

[0058] I′(i,j) is each pixel value in the image I′ after interpolation processing;

[0059] G(x - i,y - j) is the value of the Gaussian filter at the relative position (x - i,y - j);

[0060] σ is the standard deviation of the Gaussian function;

[0061] * is the convolution operation.

[0062] Adopting the above technical solution, the present invention has the following beneficial effects:

[0063] By analyzing the geometric shape, material properties, and optical performance of the lamp cover, a model for the light transmission uniformity problem of the vehicle headlight lamp cover is established, which can accurately describe the influence of the lamp cover on the light propagation in the vehicle's surrounding environment and provide a basis for subsequent image processing. Using the established lamp cover model, the degree of image distortion caused by the lamp cover can be calculated, and the image can be corrected through an inverse transformation method based on polynomial fitting, effectively eliminating the image distortion caused by the shape and material of the lamp cover and making the image more real and accurate. For the possible defects on the surface of the lamp cover, the algorithm detects and identifies abnormal pixel points in the image, interpolates using the information of normal pixel points around the abnormal pixel points, and then smooths the interpolated image, thereby eliminating the influence of the defects on the image, significantly improving the image clarity, and making the image more delicate and natural. The present invention effectively removes various influences of the lamp cover on the image quality captured by the camera, solves the main technical difficulties in the integration of the camera and the vehicle headlight, improves the robustness of the system in the driving environment, enhances the driving safety, and provides a favorable guarantee for the accurate judgment and decision-making of the intelligent driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of the image optimization method for the self-closed loop system of the vehicle headlight integrated camera of the present invention;

[0065] Figure 2 It is a flowchart of modeling the light transmission uniformity problem of the vehicle headlight lamp cover of the present invention;

[0066] Figure 3 It is a flowchart of compensating for defects of abnormal pixel points in the image caused by defects on the surface of the vehicle headlight lamp cover of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] In order to make the content of the present invention easier to be clearly understood, the present invention will be further described in detail below according to specific embodiments in conjunction with the accompanying drawings.

[0068] As Figure 1 shown, this embodiment provides an image optimization method for the self-closed loop system of the vehicle headlight integrated camera, which includes the following steps:

[0069] Step S1, model the light transmission uniformity problem of the vehicle headlight lamp cover;

[0070] Step S2, correct the distortion of the image captured by the camera integrated inside the vehicle headlight;

[0071] Step S3, perform defect compensation processing on abnormal pixel points in the image caused by defects on the surface of the vehicle headlight lamp cover.

[0072] In step S1 of this embodiment, a model is established for the light transmission uniformity problem of the headlight lens. Since the light transmission uniformity problem of the lens can lead to uneven brightness and color distribution in the images captured by the camera, a method for modeling the light transmission uniformity problem of the lens based on physical optics is proposed, which specifically includes the following steps:

[0073] Step S11: Calculate the transmittance of the ambient light around the vehicle on the headlight lens based on the light transmittance, thickness variation, and surface roughness of the headlight lens material to determine the influence of the lens on the propagation of the ambient light around the vehicle;

[0074] Step S12: Since the shape and material of the lens can cause distortion in the images captured by the camera, preprocess the images captured by the camera integrated inside the headlight, that is, perform denoising processing;

[0075] Step S13: Perform light compensation on the preprocessed images to eliminate the problems of uneven image light or color deviation caused by the lens.

[0076] In step S11 of this embodiment, the calculation formula for the transmittance of the ambient light around the vehicle on the headlight lens is as follows:

[0077] T(λ,x,y)=T0(λ)×e -α(λ)×d(x,y) ×ρ(x,y);

[0078] Wherein, T(λ,x,y) is the transmittance of the ambient light around the vehicle with wavelength λ at the position (x,y) of the headlight lens;

[0079] T0(λ) is the initial transmittance of the lens material;

[0080] α(λ) is the light absorption coefficient of the lens material;

[0081] d(x,y) is the thickness of the lens at the position (x,y);

[0082] ρ(x,y) is the light reflectance of the lens surface at the position (x,y).

[0083] In step S12 of this embodiment, methods such as Gaussian filtering or median filtering can be used for preprocessing, and the calculation formula for preprocessing is as follows:

[0084] I denoised (x,y)=Filter(I raw (x,y));

[0085] Wherein, I raw (x,y) is the original image captured by the camera;

[0086] I denoised(x, y) is the pre - processed image;

[0087] Filter is the filtering operation.

[0088] In step S13 of this embodiment, the calculation formula for light compensation is as follows:

[0089]

[0090] where, I corrected (x, y) is the image after light compensation;

[0091] I denoised (x, y) is the pre - processed image;

[0092] T(λ, x, y) is the transmittance of the ambient light around the vehicle with wavelength λ at the position (x, y) of the headlight lens.

[0093] In step S2 of this embodiment, a distortion correction algorithm based on polynomial fitting is proposed, and the calculation formula for distortion correction is as follows:

[0094] x′ = x + k1x(x 2 + y 2 ) + k2y(x 2 + y 2 ) + p1(3x 2 + y 2 ) + 2p2xy;

[0095] y′ = y + k2x(x 2 + y 2 ) + k1y(x 2 + y 2 ) + p2(x 2 + 3y 2 ) + 2p1xy;

[0096] where, (x, y) are the pixel coordinates of the distorted image in the image captured by the camera;

[0097] (x′, y′) are the pixel coordinates of the image after distortion correction;

[0098] k1 and k2 are radial distortion coefficients, which are obtained by shooting calibration patterns with known geometric shapes, such as black - and - white checkerboards, calibration targets, and then solving using the least - squares method;

[0099] p1 and p2 are tangential distortion coefficients, which are obtained by shooting calibration patterns with known geometric shapes, such as black - and - white checkerboards, calibration targets, and then solving using the least - squares method.

[0100] In step S3 of this embodiment, defect compensation processing is performed on the abnormal pixel points in the image caused by the surface defects of the vehicle lamp cover. Defects on the surface of the lamp cover (such as scratches, bubbles, etc.) will cause bright or dark spots to appear in the image. By detecting and identifying the abnormal pixel points in the image, interpolation processing is performed on the image using the information of the normal pixel points around the abnormal pixel points, and then the interpolated image is smoothed, so as to effectively eliminate the influence of the surface defects of the lamp cover on the image. The specific steps are as follows:

[0101] Step S31: Perform defect detection and identification on the image captured by the camera, and obtain the coordinates of the abnormal pixel points in the image;

[0102] Step S32: Perform weighted average interpolation processing on the identified abnormal pixel points to obtain the interpolated pixel values;

[0103] Step S33: In order to further reduce the possible image discontinuity problem after interpolation, perform smoothing processing on the interpolated image to obtain the final smoothed image;

[0104] In step S31 of this embodiment, defect detection and identification are performed on the image captured by the camera, and the coordinates of the abnormal pixel points in the image are obtained. The specific steps are as follows:

[0105] Step S311: Calculate the local variance of each pixel point in the image captured by the camera;

[0106] Step S312: Based on the local variance of each pixel point and the set local variance threshold, identify the coordinates of the abnormal pixel points in the image. Specifically, set the local variance threshold as T. If the local variance V(x, y) > T, then the pixel point (x, y) is considered an abnormal pixel point (i.e., the location of the defect). The local variance threshold T is a value that needs to be calibrated according to the actual production situation.

[0107] In step S311 of this embodiment, the calculation formula of the local variance is as follows:

[0108]

[0109] Among them, V(x, y) is the local variance of the pixel point (x, y) in the image captured by the camera;

[0110] W is the local variance window centered on the pixel point (x, y);

[0111] I(i, j) is the gray value of the pixel point in the W window;

[0112] μ(x, y) is the gray mean value of the pixel points in the W window.

[0113] In step S32 of this embodiment, the information of normal pixel points around the abnormal pixel points is used for interpolation to eliminate the influence of the flaw on the image. The calculation formula of the pixel value after interpolation is as follows:

[0114]

[0115] Wherein, I′(x, y) is the pixel value after interpolation of the abnormal pixel point;

[0116] N(x, y) is the set of normal pixel points around the abnormal pixel point;

[0117] w(i, j) is the weight, which is set according to the distance from the abnormal pixel point (x, y). A common method for setting the weight is to use the Gaussian function;

[0118] I(i, j) is the gray value of the pixel point within the W window;

[0119] σ is the standard deviation of the Gaussian function, which is used to control the attenuation speed of the weight.

[0120] In step S33 of this embodiment, the calculation formula of the final image after smoothing processing is as follows:

[0121] I″(x, y) = (I′ * G)(x, y) = ∑ (i , j) I′(i, j) × G(x - i, y - j);

[0122]

[0123] Wherein, I″(x, y) is the final image after smoothing processing;

[0124] G(x, y) is the Gaussian filter;

[0125] I′(x, y) is the pixel value after interpolation of the abnormal pixel point;

[0126] I′(i, j) is each pixel value in the image I′ after interpolation processing;

[0127] G(x - i, y - j) is the value of the Gaussian filter at the relative position (x - i, y - j);

[0128] σ is the standard deviation of the Gaussian function;

[0129] * is the convolution operation.

[0130] It should be noted that the parameters in the flaw compensation algorithm need to be adjusted according to the specific image and application scenario, such as the size of the local variance window W, the local variance threshold T, the standard deviation σ of the Gaussian function, etc.

[0131] The working principle of the present invention is as follows:

[0132] First, model the problem of light transmission uniformity of the headlight lens, including calculating the transmittance of the ambient light around the vehicle on the headlight lens, preprocessing the images captured by the camera, and performing light compensation on the preprocessed images. Then, perform distortion correction on the images captured by the camera integrated inside the headlight. Next, perform defect compensation processing on the abnormal pixel points in the images caused by the surface defects of the headlight lens, including detecting and identifying the abnormal pixel points in the images, performing interpolation processing on the identified abnormal pixel points, and performing smoothing processing on the images after interpolation processing.

[0133] By analyzing the geometric shape, material properties, and optical performance of the lens, modeling the problem of light transmission uniformity of the headlight lens can accurately describe the influence of the lens on the propagation of ambient light around the vehicle, providing a basis for subsequent image processing. Using the established lens model, the degree of image distortion caused by the lens can be calculated, and the image can be corrected by an inverse transformation method based on polynomial fitting, effectively eliminating the image distortion caused by the lens shape and material, making the image more real and accurate. For the possible defects on the lens surface, the algorithm detects and identifies the abnormal pixel points in the image, uses the information of the normal pixel points around the abnormal pixel points for interpolation processing, and then performs smoothing processing on the image after interpolation processing, thereby eliminating the influence of the defects on the image, significantly improving the clarity of the image, and making the image more delicate and natural. The present invention effectively removes the various influences of the lens on the quality of the images captured by the camera, solves the main technical difficulties in the integration of the camera and the headlight, improves the robustness of the system in the driving environment, enhances the driving safety, and provides a favorable guarantee for the accurate judgment and decision-making of the intelligent driving system.

[0134] The specific embodiments described above further elaborate on the technical problems solved, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An image optimization method for a self-closed loop system of a headlight integrated camera, characterized in that: It includes the following steps: Step S1, modeling the light transmittance uniformity problem of the headlight cover; Step S2, performing distortion correction on the image captured by the camera integrated inside the headlight; Step S3: performing defect compensation processing on abnormal image pixels caused by surface defects of the lampshade.

2. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 1, characterized in that: In the step S1, modeling is performed on the light transmittance uniformity problem of the lampshade of the vehicle lamp, which specifically includes the following steps: Step S11, calculating the transmittance of the ambient light around the vehicle through the headlight cover based on the transmittance, thickness variation and surface roughness of the headlight cover material; Step S12, pre-processing the image captured by the camera integrated inside the headlight; Step S13: Perform light compensation on the preprocessed image.

3. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 2, characterized in that: In step S11, the calculation formula of the transmittance of the ambient light around the vehicle on the lampshade is as follows: T(λ,x,y)=T0(λ)×e -α(λ)×d(x,y) ×ρ(x,y); Where T(λ,x,y) is the transmittance of the ambient light around the vehicle with a wavelength of λ at the position (x,y) of the headlight cover; T0(λ) is the initial light transmittance of the lampshade material; α(λ) is the light absorption coefficient of the lampshade material; d(x,y) is the thickness of the lampshade at position (x,y); ρ(x,y) is the light reflectivity of the lampshade surface at position (x,y).

4. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 3, characterized in that: In step S12, the calculation formula for the preprocessing is as follows: I denoised (x,y)=Filter(I raw (x,y)); Among them, I raw (x, y) is the original image captured by the camera; I denoised (x, y) is the preprocessed image; Filter is a filtering operation.

5. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 4, characterized in that: In step S13, the calculation formula of the light compensation is as follows: Among them, I corrected (x, y) is the image after light compensation; I denoised (x, y) is the preprocessed image; T(λ,x,y) is the transmittance of the vehicle's ambient light with a wavelength of λ at the headlight cover position (x,y).

6. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 5, characterized in that: In step S2, the calculation formula for the distortion correction is as follows: x'=x+k1x(x 2 +y 2 )+k2y(x 2 +y 2 )+p1(3x 2 +y 2 )+2p2xy; y'=y+k2x(x 2 +y 2 )+k1y(x 2 +y 2 )+p2(x 2 +3y 2 )+2p1xy: Where (x, y) is the coordinate of the pixel in the image captured by the camera that produces distortion; (x', y') are the pixel coordinates of the image after distortion correction; k1 and k2 are radial distortion coefficients; p1 and p2 are the tangential distortion coefficients.

7. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 6, characterized in that: In step S3, defect compensation processing is performed on abnormal image pixels caused by surface defects of the lampshade, which specifically includes the following steps: Step S31, performing defect detection and identification on the image captured by the camera to obtain the coordinates of abnormal pixel points in the image; Step S32, performing weighted average interpolation processing on the identified abnormal pixel points to obtain interpolated pixel values; Step S33: Smoothing the interpolated image to obtain a final smoothed image.

8. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 7, characterized in that: In step S31, defect detection and identification are performed on the image captured by the camera to obtain the coordinates of abnormal pixel points in the image, which specifically includes the following steps: Step S311, calculating the local variance of each pixel in the image captured by the camera; Step S312: Based on the local variance of each pixel and a set local variance threshold, the coordinates of abnormal pixels in the image are identified.

9. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 8, characterized in that: In step S32, the calculation formula of the interpolated pixel value is as follows: Among them, I'(x,y) is the pixel value after interpolation of abnormal pixel points; N(x,y) is the set of normal pixels around the abnormal pixel; w(i,j) is the weight; I(i,j) is the gray value of the pixel in the W window; σ is the standard deviation of the Gaussian function.

10. The image optimization method of the self-closed loop system of the vehicle lamp integrated camera according to claim 9, characterized in that: In step S33, the calculation formula of the final image after smoothing is as follows: I″(x,y)=(I′*G)(x,y)=∑ (i,j) I′(i,j)×G(x-i,y-j); Where, I”(x,y) is the final image after smoothing; G(x,y) is a Gaussian filter; I'(x,y) is the pixel value after interpolation of abnormal pixel points; I'(i,j) is the value of each pixel in the interpolated image I'; G(xi,yj) is the value of the Gaussian filter at the relative position (xi,yj); σ is the standard deviation of the Gaussian function; * is the convolution operation.

Images