Intelligent car light control method based on color correction and depth correction

Abstract

The present invention relates to the field of intelligent headlight control technology, specifically providing an intelligent headlight control method based on color correction and depth correction. An optical sensing device is used to collect visual images and depth information of the road surface on which the vehicle is traveling, and color deviation tensors and depth deviation tensors are calculated and processed. The actual ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of view is calculated based on the depth deviation tensor, and the actual ratio is compared with the theoretical ratio. The projection size is adjusted based on the comparison result. The color of the pre-projected image is then corrected by adjusting the coefficients in a polynomial model until the color deviation vector is eliminated, thereby obtaining a projected image. The present invention achieves adaptive adjustment of the headlight illumination range, light intensity, and color, ensuring that the headlight projection is not affected by road surface color. This ensures that the projection remains complete and continuous even on bumpy roads, avoiding problems such as tearing and discontinuity in the projected image.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle light control, and specifically provides an intelligent vehicle light control method based on color correction and depth correction. Background Art

[0002] With people's pursuit of safety and comfort, automobile intelligence has become the future development direction of the automobile manufacturing industry. As one of the most important functions in the driving process of the car, the lighting system also needs to be intelligent to meet the different needs of the driver.

[0003] However, current smart headlights are relatively insular, with only basic functions like on / off switching, brightness adjustment, and angle adjustment. When driving at night, the driver's visibility is reduced, requiring smart headlights to project various projections or signs onto the road surface to alert the driver to pedestrians, lane changes, parking, and other information. However, current projections are significantly affected by the color and smoothness of the road surface, resulting in localized color anomalies, image tearing, and discontinuities in uneven areas. This makes it difficult for the driver to discern the information reflected in the projected image, preventing them from reacting to driving conditions in a timely manner. Therefore, it is necessary to address color interference, maintain image color, and resolve issues such as tearing and discontinuities in the projected image, making it easier for the driver to identify the projected image. Summary of the Invention

[0004] To solve the above problems, the present invention provides an intelligent headlight control method based on color correction and depth correction, which effectively solves the problems of tearing or overlapping of projected images caused by color interference of the projected road surface and uneven potholes in the road surface, making it difficult for the driver to recognize the projected information in a short time.

[0005] The intelligent vehicle light control method based on color correction and depth correction provided by the present invention includes the following steps:

[0006] S1. Collect visual images and depth information of the vehicle's road surface through an optical sensing device, and adaptively adjust the detection frequency of the optical sensing device using a cyclic decision algorithm;

[0007] S2. Obtain a global color fitting plane using color information fitting of each point on the visual image, calibrate points on the global color fitting plane with points on the visual image, and calculate color deviation tensors of corresponding points; identify elements on the visual image, and determine a pattern reflecting the element information as a pre-projection image;

[0008] The depth information of each point on the driving road is used to fit the ideal depth plane, and the points on the ideal depth plane are calibrated with the points on the driving road to calculate the depth deviation tensor of the corresponding points.

[0009] S3. For any line segment AB on the depth ideal plane, there exists a corresponding line segment A′B′ in the driver’s field of view, and the vector similarity constraint is satisfied:

[0010]

[0011] Where S represents the distance between any line segment AB and the driver in the depth direction;

[0012] Calculate the actual ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of view based on the depth deviation tensor. If the actual ratio is less than When , the size of the corresponding area on the pre-projected image is reduced according to the depth deviation tensor; if the actual ratio is greater than When the depth of the pre-projected image is magnified according to the depth deviation tensor, the scale factor of reduction and magnification is calibrated with the coordinate position to obtain the global depth compensation matrix Q, and the pre-projected image is depth corrected according to the depth compensation matrix Q;

[0013] S4, performing color correction on the pre-projected image based on the color deviation tensor, obtaining a polynomial model reflecting color information based on the RGB data of each point on the visual image, and correcting the color of the pre-projected image by adjusting the coefficients in the polynomial model until the color deviation vector is eliminated, thereby obtaining a projected image;

[0014] S5. Projecting the projection image onto the projection plane through the vehicle lights.

[0015] Preferably, the optical perception device uses a binocular camera including an RGB channel and a Depth channel.

[0016] Preferably, the detection frequency of the optical sensing device changes periodically and decays over time within each cycle to reduce energy consumption.

[0017] Preferably, the vehicle light is composed of a plurality of independently controllable light sources, and the optical parameters of each light source are adjustable.

[0018] Preferably, a convolutional neural network is used to recognize elements on the visual image.

[0019] Preferably, the elements in the visual image include vehicles, pedestrians and safety signs.

[0020] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0021] The present invention obtains environmental information around the vehicle through an optical sensing device, calculates the potholes and color interference of the road ahead of the vehicle through an internal algorithm, and plans color correction and depth correction schemes according to the usage scenario. It can realize adaptive adjustment of the illumination range, light intensity and color of the headlights, so that the projection of the headlights will not be interfered by the color of the road surface, ensuring that the projection remains complete and continuous on bumpy roads, avoiding problems such as tearing and discontinuity in the projected image, improving the safety and comfort of night driving, and realizing adaptive control of different roads, weather and vehicle conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an overall framework diagram of the intelligent vehicle light control method provided according to an embodiment of the present invention;

[0023] Figure 2 is a global analysis flow chart of intelligent vehicle light control according to an embodiment of the present invention;

[0024] Figure 3 FIG. 4 is a schematic diagram of a scene of depth correction provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.

[0026] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0027] The intelligent vehicle light control method based on color correction and depth correction provided by the embodiments of the present invention uses a binocular camera as an optical sensing device to collect environmental information around the vehicle. The binocular camera includes RGB channels and Depth channels, which correspond to the image acquisition module and depth acquisition module respectively, to acquire visual images and depth information. The specific control process of the method is as follows:

[0028] S1. Before collecting information, preprocessing is required to keep the relative positions of the headlights, binocular camera and driver unchanged, and calibrate the coordinates of the positions of the driver, headlights and binocular camera. It should be noted that the binocular camera needs to be set in a position to ensure that the field of view can cover the entire area in front of the vehicle. In order to ensure accurate modeling and precise control in the future, use measuring tools to measure the relative position relationship between the driver, headlights and binocular camera, and obtain their coordinate systems with their respective positions as the origin. The driver coordinate system U is established with the driver's eyes as the origin, the camera coordinate system C is established with the optical perception device as the origin, and the projection coordinate system P is established with the headlights as the origin. During the calibration process, the measuring tool is used to measure the six-dimensional posture state of the binocular camera and the driver's eyes relative to the origin of the headlights in detail to obtain the posture transformation matrix of the binocular camera relative to the headlights. And the driver's pose transformation matrix relative to the headlight The two pose transformation matrices described above describe the position and posture of the binocular camera and the driver relative to the headlights. Furthermore, for device calibration, established solutions are used to measure the optical parameters of the binocular camera and headlights and calibrate their internal parameters. This ensures that the binocular camera's image quality matches the real image, and the headlight's projection quality matches the specified projection image, ensuring accurate geometric and optical parameters during illumination and projection. Smart headlights can also be color-corrected by capturing an object of known color and transmitting this color information to the controller to compare the controller's output with the known color. Most devices are already calibrated and corrected upon shipment, so this step can usually be omitted.

[0029] Furthermore, in intelligent headlight systems, headlights are composed of multiple independently controllable light sources, each with adjustable optical parameters. The light-emitting elements can be LED or laser-based, offering high brightness, high efficiency, and long life. Based on different light-emitting principles and projection methods, headlights can be categorized into various types, including matrix LED headlights, digital LED headlights, and laser headlights.

[0030] After coordinate and device calibration, the binocular camera is used to collect visual images and depth information about the vehicle's surroundings, including road type, road conditions, weather conditions, vehicle status, and pedestrian locations. The binocular camera's detection frequency is controlled by the intelligent headlight system controller's cyclic decision-making detection algorithm. The controller periodically sends commands to drive the binocular camera to periodically detect visual images and depth information, then drives the headlights to project the updated, compensated image. This cycle repeats, achieving a uniform projection effect and reducing the detection frequency. This not only eliminates errors between the digital image and the projected display quality, but also eliminates deviations between the digital image captured by the binocular camera and the actual image. Furthermore, to reduce energy consumption, the binocular camera's detection frequency varies periodically, encompassing multiple projection cycles. The projected images across these cycles differ minimally, resulting in a decaying detection frequency over time within the cycle, with the time between consecutive detections increasing. When a significant change in the visual image is detected, the detection frequency is reset, and a new detection cycle begins.

[0031] S2, such as Figure 2 As shown, a full-domain analysis is performed, and the corresponding color processing algorithm is specified according to the accuracy of the binocular camera. The visual image is color processed, and the three color channels R, G, and B of the visual image and the height and width of the projection plane are combined into a five-dimensional feature. The projection plane is the part of the road in front of the vehicle. The entire color gamut plane is fitted by fitting the color gamut values ​​of each color channel, and the outliers are removed to obtain three full-domain color fitting planes as the ideal plane reference. Then, the color deviation tensor of each pixel on the visible image on the R, G, and B channels and the corresponding point on the ideal plane can be calculated. They are the color space information R (red), G (green), and B (blue) representing the projection plane, and the geometric space information H (height) and W (width). The plane composed of (H, W) represents the driver's projection plane. Then, any point on this plane can be represented as [r, g, b] (h,w) , where r represents the specific value of the point in the R dimension, g represents the specific value of the point in the G dimension, b represents the specific value of the point in the B dimension, h represents the specific value of the point in the H dimension, and w represents the specific value of the point in the W dimension. Color processing is to obtain the five-dimensional deviation tensor between the data of each dimension in the plane scene and an ideal projection plane with pure white color. Therefore, based on the collected information, the color gamut values ​​of the three dimensions r, g, and b are fitted to each color gamut plane, and outliers are continuously removed to fit the global color fitting plane. The color information of any point on it is recorded as [r0, g0, b0]. From the collected image, the actual color information [r, g, b] of any point in the global gamut (H, W) can be known. (h,w), the coordinates of the points on the global color fitting plane are matched one by one with the coordinates of the pixels on the actual projected road surface image, and the corresponding difference operation is performed to obtain the color deviation tensor of the pixel point on the image in the three color channels of R, G, and B and the corresponding point on the global color fitting plane:

[0032] ΔC(h,w)=[r0-r,g0-g,b-b0] (h，w) ;

[0033] Among them, r0-r=ΔR, g0-g=ΔG, b-b0=ΔB.

[0034] Perform a depth analysis on the depth of the projection plane, and combine the depth D of the projection plane with the height H and width W of the projection plane to form a three-dimensional feature. According to the binocular cameras with different precision, a corresponding depth analysis algorithm is formulated. Any point in the projection plane is represented by d (h，w) , d represents the specific value of the depth of the point on the projection plane. Depth analysis first fuses the depth information with the spatial geometric data and tensors it. According to the accuracy level of the binocular camera that captures the depth information, a plane fitting algorithm is selected. By fitting and continuously removing outliers, the depth fitting plane is obtained and used as the ideal depth plane. For the depth information d of any point in the whole domain (H, W), (h，w) , and perform a difference operation on the depth information of the corresponding point on the depth ideal plane to obtain the depth deviation tensor of the actual road surface position containing the depth information relative to the depth ideal plane. The model establishment methods of depth processing and color processing are relatively similar, and the obtained color deviation tensor and depth deviation tensor can both be expressed in matrix form.

[0035] Before performing color and depth correction on the pre-projected image, a convolutional neural network (CNN) or other machine learning algorithm is required to make decisions on the collected visual image and identify elements in the visual image. The identified elements include pedestrians, vehicles, safety signs, or other objects in the visual image. The pre-projected image is determined based on the elements identified from the visual image.

[0036] S3. Due to the unevenness of the projection plane, it is necessary to perform depth correction on the pre-projected image based on the depth deviation tensor, such as Figure 3 As shown, l1+l2 is the projection plane on the actual road surface. The l2 area is flat. The deviation of the projection effect mainly comes from the l1 area. Use D(h, w) U The function represents the depth information of the projected road surface area l1, which is incorporated into the driver coordinate system U and expressed as (h, w, D(h, w) U ). A driver's field of view space is obtained by dividing it in the driver's coordinate system U. For any line segment AB on the depth ideal plane, there is a corresponding line segment A′B′ in the driver's field of view space, and the vector similarity constraint is satisfied:

[0037]

[0038] Where S represents the distance between any line segment AB and the driver in the depth direction; the actual ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of view is calculated based on the depth deviation tensor. When the actual ratio is less than When , it indicates that the projected road surface is sunken, and it is necessary to reduce the size of the corresponding area on the pre-projected image according to the depth deviation tensor, that is, to shrink the light in this part and reduce the scattering angle; when the actual ratio is greater than When represents a bump on the projected road surface, the corresponding area on the pre-projected image is magnified based on the depth deviation tensor. This expands the light beam in that area and increases the scattering angle. This ensures that the image seen by the driver, whether on a bump or a depression, is the same size as the image on the ideal depth plane, avoiding overlap or tearing of the projected image at the depression or bump location. The scaling factors for reduction and magnification are calibrated with the coordinate positions to obtain the global depth compensation matrix Q. This depth compensation matrix Q is then used to perform depth correction on the pre-projected image.

[0039] S4. Due to the interference of color or other paint marks on the projection plane, it is necessary to perform color correction on the pre-projected image based on the color deviation tensor. Considering the nonlinear relationship between the conversion between the color value of the pre-projected image and the color compensation value to be corrected, a polynomial model is selected to fit the compensation value dataset and a quadratic polynomial model is established. Its expression is:

[0040] R′=a0+a1x+a2y+a3x 2 +a4y 2 +a5xy;

[0041] G′=b0+b1x+b2y+b3x 2 +b4y 2 +b5xy;

[0042] B′=c0+c1x+c2y+c3x 2 +c4y 2 +c5xy;

[0043] Where R′, G′, and B′ are the values ​​of the compensated red, green, and blue channels, x and y are the position coordinates in the projection plane, and a0, a1, ..., a5, b0, b1, ..., b5, c0, c1, ..., c5 are the coefficients of the polynomial.

[0044] Based on the color deviation tensor, the coefficients of a quadratic polynomial model are determined using the least squares method or other curve fitting methods. The most appropriate coefficient values ​​are selected based on the visual image to accurately fit the polynomial model to the data and compensate for color. The compensation values ​​for each pixel in the pre-projected image form a two-dimensional color compensation matrix P. During the projection process, the coefficients in the two-dimensional color compensation matrix P are used to compensate the pre-projected image to eliminate color deviation and prevent distortion in the projected image.

[0045] The processing logic of the pre-projected image img is:

[0046] simg = P·Q(img);

[0047] Wherein simg represents the final projection image obtained by processing the pre-projection image img.

[0048] S5. Send the projection image simg to the vehicle headlight for projection, and project the projection image onto the projection plane.

[0049] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0050] The above specific embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. An intelligent vehicle light control method based on color correction and depth correction, characterized in that: The following steps are involved: S1. Collect visual images and depth information of the vehicle's road surface through an optical sensing device, and adaptively adjust the detection frequency of the optical sensing device using a cyclic decision algorithm; S2. Obtain a global color fitting plane using color information fitting of each point on the visual image, calibrate points on the global color fitting plane with points on the visual image, and calculate color deviation tensors of corresponding points; identify elements on the visual image, and determine a pattern reflecting the element information as a pre-projection image; The depth information of each point on the driving road is used to fit the ideal depth plane, and the points on the ideal depth plane are calibrated with the points on the driving road to calculate the depth deviation tensor of the corresponding points. S3. For any line segment AB on the depth ideal plane, there exists a corresponding line segment A′B′ in the driver’s field of view, and the vector similarity constraint is satisfied: Where S represents the distance between any line segment AB and the driver in the depth direction; Calculate the actual ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of view based on the depth deviation tensor. If the actual ratio is less than When , the size of the corresponding area on the pre-projected image is reduced according to the depth deviation tensor; if the actual ratio is greater than When the depth of the pre-projected image is magnified according to the depth deviation tensor, the scale factor of reduction and magnification is calibrated with the coordinate position to obtain the global depth compensation matrix Q, and the pre-projected image is depth corrected according to the depth compensation matrix Q; S4, performing color correction on the pre-projected image based on the color deviation tensor, obtaining a polynomial model reflecting color information based on the RGB data of each point on the visual image, and correcting the color of the pre-projected image by adjusting the coefficients in the polynomial model until the color deviation vector is eliminated, thereby obtaining a projected image; S5. Projecting the projection image onto the projection plane through the vehicle lights.

2. The intelligent vehicle light control method based on color correction and depth correction according to claim 1, characterized in that: The optical perception device uses a binocular camera, including RGB channels and Depth channels.

3. The intelligent vehicle light control method based on color correction and depth correction according to claim 1, characterized in that: The detection frequency of the optical sensing device changes periodically and decays over time within each cycle to reduce energy consumption.

4. The intelligent vehicle light control method based on color correction and depth correction according to claim 1, characterized in that: The headlights consist of multiple independently controllable light sources, and the optical parameters of each light source can be adjusted.

5. The intelligent vehicle light control method based on color correction and depth correction according to claim 1, characterized in that: Use convolutional neural networks to recognize elements in visual images.

6. The intelligent vehicle light control method based on color correction and depth correction according to claim 1, characterized in that: Elements in the visual image include vehicles, pedestrians and safety signs.

Images