Intelligent vehicle lamp control method based on color avoidance and depth correction

Abstract

The invention relates to the technical field of intelligent vehicle lamps, and particularly provides an intelligent vehicle lamp control method based on color avoidance and depth correction, and the method comprises the steps: determining a corresponding heterochromatic region on a pre-projection image according to the color deviation tensor of a projection plane and a global color fitting plane, and controlling a vehicle lamp to carry out the color avoidance projection of the heterochromatic region, according to the depth difference tensor of the projection plane and the ideal depth plane, determining a corresponding depth change area on the pre-projection image, and controlling the vehicle lamp to change the projection size to realize depth correction, so as to ensure that the finally projected image is not interfered by the color of the actual road surface; and discontinuous and unclear projection images in the visual field of the driver due to road surface potholes are avoided. According to the invention, the light brightness and the local projection size of the vehicle lamp can be adjusted according to the collected information, so that the projection of the vehicle lamp is not interfered by the road surface color and the road surface unevenness.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle lights, and specifically provides an intelligent vehicle light control method based on color avoidance and depth correction. Background Art

[0002] As people pursue safety and comfort, automobile intelligence has become the future development direction of the automobile manufacturing industry. As one of the important functions in the driving process of the car, the headlight system also needs to be intelligent to meet the different needs of the driver.

[0003] At present, there are intelligent car lighting systems such as ADB adaptive high beam, DLP digital lighting, and Matrix matrix LED lighting on the market. They use sensors such as cameras and radars to sense the surrounding environment and targets, and use controllers to adjust the brightness, angle and shape of the lights to achieve adaptive lighting for different road conditions and scenes. For example, ADB adaptive high beam can automatically dim or turn off the glare area of ​​the opposite or front vehicles while maintaining the high beam state; DLP digital lighting can project various patterns or logos on the road surface to remind the driver to pay attention to pedestrians, lane changes, parking and other information; Matrix matrix LED lighting can control the direction of the lights according to the steering angle of the steering wheel to reduce the blind spot when turning. However, the existing intelligent car lights have a low degree of intelligence and cannot cope with complex roads and different weather conditions. In particular, when the light is projected, the clarity of the projection pattern is poor, and when the projection pattern is projected on an obstruction, it cannot be automatically adjusted to avoid the obstruction, making it difficult for the driver to distinguish the projection pattern.

[0004] Therefore, there is an urgent need for a new type of intelligent car light control method that can plan the projection plan, realize adaptive adjustment of the car light illumination range and light intensity, and improve the projection accuracy. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an intelligent vehicle light control method based on color avoidance and depth correction, which effectively solves the problems that existing vehicle lights cannot be restricted by the color interference of the projected road surface and the unevenness of road potholes.

[0006] The intelligent vehicle light control method based on color avoidance and depth correction provided by the present invention comprises the following steps:

[0007] S1. Collect images and depth information of the vehicle's surrounding environment through an optical sensing device, and use a cyclic decision algorithm to adaptively adjust the detection frequency of the optical sensing device;

[0008] S2. Analyze and process the projection plane:

[0009] Performing color processing on the image to obtain a color deviation tensor from the ideal plane; performing depth analysis on the depth information to obtain a depth deviation tensor from the ideal plane; and identifying the image content, and determining the corresponding pre-projected image according to the identified content;

[0010] S3, analyzing and processing the pre-projected image:

[0011] The pre-projection image is subjected to color avoidance and depth correction according to the color deviation tensor and the depth deviation tensor to obtain a projection image, the projection image is projected onto a projection plane through the vehicle lights, and the corresponding information of the projection image is displayed on a display screen.

[0012] Preferably, the process of color processing is:

[0013] The three color channels of the image, R, G, and B, and the height and width of the projection plane form a five-dimensional feature. H represents the height of the projection plane, and W represents the width of the projection plane. Any point in the projection plane is represented as [r, g, b] (h,w) , r represents the specific value of the point in the R dimension, g represents the specific value of the point in the G dimension, b represents the specific value of the point in the B dimension, h represents the specific value of the point in the H dimension, and w represents the specific value of the point in the W dimension;

[0014] According to the color information of the image, the global color fitting plane is obtained and the [r, g, b] of any point is calculated. (h,w) The color deviation tensor of the corresponding points on the global color fitting plane under the three color channels of R, G, and B.

[0015] Preferably, the deep processing process is:

[0016] The depth of the projection plane, the height and the width of the projection plane form a three-dimensional feature. Any point in the projection plane is represented by d (h,w) , d is the specific value of the depth of the point on the projection plane; the ideal depth plane is obtained by fitting the depth information, and d is calculated for any point (h,w) The depth deviation tensor of the corresponding points on the depth ideal plane.

[0017] Preferably, the process of color avoidance is:

[0018] Traverse all pixels of any frame of pre-projection image, and calculate the color deviation tensor ΔC(x, y) of each pixel of the pre-projection image according to the obtained color deviation tensor of the projection plane, where x and y represent the coordinates of the pixel point on the pre-projection image;

[0019] The color channel threshold is set to ΔT. When |ΔC(x, y)|>ΔT, the rectangular box is expanded in the four directions of upward, downward, left and right with the pixel as the starting point. When the color deviation tensor of the pixel is less than or equal to the color channel threshold, the expansion is stopped. The obtained rectangular box is a heterochromatic area, which is avoided.

[0020] Preferably, the starting pixel is (x1, y1), and the pixel currently expanded to is (x i ,y i ), the judgment process of different color areas is:

[0021] If |ΔC(x2,y i-1 )|≤ΔT, then (x2, y i-1 ) is the upper boundary of the rectangular box;

[0022] If |ΔC(x2,y i+1 )|≤ΔT, then (x2, y i+1 ) is the lower border of the rectangular box;

[0023] If |ΔC(x i-1 , y2)|≤ΔT, then (x i-1 , y2) is the left boundary of the rectangular box;

[0024] If |ΔC(x i+1 , y2)|≤ΔT, then (x i+1 , y2) is the right border of the rectangular box.

[0025] Preferably, the process of depth correction is:

[0026] The driver coordinate system U is established with the driver's eyes as the origin, the camera coordinate system C is established with the optical sensing device as the origin, and the projection coordinate system P is established with the headlights as the origin;

[0027] Determine the pose transformation matrix of U and P based on the spatial position of the driver's eyes and headlights Determine the pose transformation matrix of C and P based on the spatial position of the optical sensing device and the headlight Converting the depth information based on the camera coordinate system acquired by the optical perception device into the depth information based on the driver coordinate system;

[0028] For any line segment AB on the depth ideal plane, there is a corresponding line segment A′B′ in the driver's field of vision, and the vector similarity constraint is satisfied:

[0029]

[0030] Where S represents the distance between any line segment AB and the driver in the depth direction;

[0031] The measured ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of vision is calculated based on the depth deviation tensor. When the measured ratio is greater than When , it indicates that the projected road surface is raised. The size of the corresponding area on the pre-projected image is enlarged according to the depth deviation tensor, the scale coefficients of reduction and enlargement are calibrated with the coordinate position, and the global depth compensation matrix Q is obtained. The pre-projected image is depth corrected according to the depth compensation matrix Q.

[0032] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0033] The present invention obtains environmental information around the vehicle through an optical sensing device, calculates the flatness and reflectivity of the front of the vehicle, that is, the pothole situation and color interference of the road surface, through an internal algorithm, and plans color avoidance and depth correction solutions according to the usage scenario. It can realize adaptive adjustment of the illumination range and light intensity of the headlights, so that the projection of the headlights will not be interfered by the color of the road surface, ensuring that the projection is still complete and continuous on a bumpy road surface, thereby improving the safety and comfort of night driving and realizing adaptive control of different roads, weather and vehicle conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is an overall framework diagram of an intelligent vehicle light control method provided according to an embodiment of the present invention;

[0035] Figure 2 is a global analysis flow chart of intelligent vehicle light control according to an embodiment of the present invention;

[0036] Figure 3 4 is a schematic diagram of a scene of depth correction provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0039] like Figure 1As shown, the intelligent vehicle light control method based on color avoidance and depth correction provided by the embodiment of the present invention uses a binocular camera as an optical perception device to collect environmental information around the vehicle. The binocular camera includes an image acquisition module and a depth acquisition module to acquire image and depth information. The RGB channel resolution is 1920×1080, the field of view angle range is 69°×42°, and the Depth channel resolution is 1280×720, and the field of view angle range is 87°×58°. The specific control steps are as follows:

[0040] S1. Before information collection, the coordinates of the driver, headlights and binocular camera must be calibrated first, and the relative position relationship between the driver, headlights and binocular camera must be measured. The coordinate systems with their respective positions as the origins must be obtained for subsequent accurate modeling and precise control. The driver coordinate system U is established with the driver's eyes as the origin, the camera coordinate system C is established with the optical sensing device as the origin, and the projection coordinate system P is established with the headlights as the origin. During the calibration process, the measurement tool is used to measure in detail the six-dimensional posture state of the binocular camera and the driver's eyes relative to the origin of the headlights, and the posture transformation matrix of the binocular camera relative to the headlights is obtained. And the driver's position transformation matrix relative to the headlight The above two pose transformation matrices describe the position and posture of the binocular camera and the driver relative to the headlights. In addition, in terms of equipment calibration, the optical parameters of the binocular camera and the headlights are measured through the existing scheme, and the internal parameters of the binocular camera and the headlights are calibrated, so that the photography effect of the binocular camera is consistent with the real picture, and the projection effect of the headlights is consistent with the specified projection picture, ensuring the accuracy of the geometric and optical parameters during lighting projection.

[0041] After calibration, the binocular camera collects visible image information and depth information of the vehicle's surrounding environment, including road type, road conditions, weather conditions, vehicle status, pedestrian position, etc., and intercepts the overlapping field of view of 69°×42° of the binocular camera's RGB and Depth as effective information. Since the depth information and image do not completely overlap, it is necessary to intercept the overlapping part as effective information, process and project the effective information part. When projecting, the entire domain of effective information can be selected as the target area, or only the central area of ​​the effective area can be intercepted as the target area. The target area corresponds to the projection plane. The pixel resolution of RGB is 1920×1080, and the pixel resolution of Depth is intercepted to: 1016×521, and the projection plane is determined based on the environmental information. The image and depth information are analyzed and processed respectively, and finally the mapping relationship between the RGB image and the depth image is established according to the RGB and Depth camera external parameter matrices.

[0042] S2, such as Figure 2As shown, a global analysis is performed, and the image is color processed. The three color channels of the image, R, G, and B, and the height and width of the projection plane form a five-dimensional feature. The entire color gamut plane is fitted by fitting the color gamut values ​​of each color channel, and the outliers are removed to obtain three global color fitting planes as ideal plane references. Then, the color deviation tensor of each pixel on the visible image on the R, G, and B channels and the corresponding point on the ideal plane can be calculated. They are the color space information R (red), G (green), and B (blue) representing the projection plane, and the geometric space information H (height) and W (width). The plane composed of (H, W) represents the projection plane of the driver. Then, any point on this plane can be represented as [r, g, b] (h，w) , where r represents the specific value of the point in the R dimension, g represents the specific value of the point in the G dimension, b represents the specific value of the point in the B dimension, h represents the specific value of the point in the H dimension, and w represents the specific value of the point in the W dimension. Color processing is to obtain the five-dimensional deviation tensor between the data of each dimension in the plane scene and an ideal projection plane with pure white color. Therefore, the color gamut values ​​of the three dimensions of r, g, and b are fitted to each color gamut plane based on the collected information, and the outliers are continuously removed to fit the global color fitting plane. The color information of any point on it is recorded as [r0, g0, b0]. From the collected image, it can be known that the actual color information [r, g, b] of any point in the global domain (H, W) (h，w) , the coordinates of the points on the global color fitting plane are matched one by one with the coordinates of the pixels on the actual projected road surface image, and the corresponding difference operation is performed to obtain the color deviation tensor of the pixel points on the image in the three color channels of R, G, and B and the corresponding points on the global color fitting plane:

[0043] ΔC(h,w)=[ΔR(h,w),ΔG(h,w),ΔB(h,w)].

[0044] The depth of the projection plane is analyzed, and the depth D of the projection plane is combined with the height H and width W of the projection plane to form a three-dimensional feature. The corresponding depth analysis algorithm is formulated according to the binocular cameras with different precisions. Any point in the projection plane is represented by d (h,w) , d represents the specific value of the depth of the point on the projection plane. Depth analysis first fuses the depth information with the spatial geometric data and quantizes them. The plane fitting algorithm is selected according to the accuracy level of the binocular camera captured by the depth information. The depth fitting plane is obtained by fitting and continuously removing outliers. It is used as the ideal depth plane. For any point in the whole domain (H, W), the depth information d (h,w), and perform a difference operation on the depth information of the corresponding point on the depth ideal plane to obtain the depth deviation tensor of the actual road surface position containing the depth information relative to the depth ideal plane. The model establishment methods of depth processing and color processing are relatively similar, and the obtained color deviation tensor and depth deviation tensor can both be expressed in matrix form.

[0045] Obtaining the color deviation tensor and the depth deviation tensor requires color avoidance and depth correction of the pre-projected image. It is necessary to use a convolutional neural network (CNN) or other machine learning algorithms to make decisions on the collected images, identify pedestrians, vehicles or other objects in the images, and determine the pre-projected image based on the information recognized from the image.

[0046] S3, a pixel point on the pre-projected image is represented as [r', g', b'] (h，w) , r′ represents the specific value of the pixel point on the pre-projection image in the R dimension, g′ represents the specific value of the pixel point on the pre-projection image in the G dimension, b′ represents the specific value of the pixel point on the pre-projection image in the B dimension, and (h, w) represents the corresponding position of the point on the visible image.

[0047] Since the pixel resolution of the pre-projected image is significantly higher than that of the bias tensor, the bias tensor is upsampled to make the two resolutions equivalent.

[0048] Due to the existence of the color deviation tensor, it is necessary to determine the maximum rectangular frame of the heterochromatic area of ​​the projected road surface according to the threshold of the color channel, and divide the corresponding area in the pre-projected image, and perform projection avoidance on the divided heterochromatic area. Specifically, adjust the light brightness or color. The specific steps for determining the heterochromatic area are as follows:

[0049] Traverse all pixels on the pre-projected image and calculate the color deviation tensor ΔC(x, y) of any pixel, where x and y represent the coordinates of the pixel on the pre-projected image. The color deviation tensor is the color deviation tensor of the pixel on the image under the three color channels of R, G, and B with the corresponding point on the global color fitting plane. The coordinate representation is different, but the coordinate points correspond to each other.

[0050] The color channel threshold is set to ΔT, which is used to determine whether the color deviation of the pixel exceeds the threshold. ΔT can be appropriately designed according to the accuracy of the algorithm.

[0051] When |ΔC(x, y)|>ΔT, the rectangular box is expanded in the four directions of upward, downward, left and right with the pixel as the starting point. When the color deviation tensor of the pixel is less than or equal to the color channel threshold, the expansion is stopped and the obtained rectangular box is a heterochromatic area, which is avoided.

[0052] The following is a specific example of the rectangular frame expansion process: Assume that the starting pixel is (x1, y1) and the pixel currently expanded to is (x i ,y i ), determine whether to continue expanding based on the following example:

[0053] If |ΔC(x2,y i-1 )|≤ΔT, then (x2, y i-1 ) is the upper boundary of the rectangular box;

[0054] If |ΔC(x2,y i+1 )|≤ΔT, then (x2, y i+1 ) is the lower border of the rectangular box;

[0055] If |ΔC(x i-1 , y2)|≤ΔT, then (x i-1 , y2) is the left boundary of the rectangular box;

[0056] If |ΔC(x i+1 , y2)|≤ΔT, then (x i+1 , y2) is the right border of the rectangular box.

[0057] Through the above process, the rectangular box that does not meet the color channel threshold can be determined, and the rectangular box is represented as a heterochromatic area. This process can be used to identify areas with large color deviations in the pre-projected image. Once the heterochromatic area is identified, directional projection measures can be taken, that is, the projection light avoids the heterochromatic area or adjusts the optical parameters to achieve the effect of color avoidance. This can avoid the color interference of the road color on the projected image.

[0058] Depth correction of the pre-projected image mainly includes the following steps:

[0059] The pose transformation matrix of the binocular camera relative to the headlight And the driver's position transformation matrix relative to the headlight Transform the coordinates of any point R, and calibrate the points in the projection coordinate system P, the driver coordinate system U, and the camera coordinate system C. The calibration formula is:

[0060]

[0061]

[0062] Among them, R P Represents any point R in the projected coordinate system P, R C Indicates that any point R is in the camera coordinate system C, R U Represents any point R in the driver's coordinate system U.

[0063] like Figure 3 As shown, l +l2 is the projection plane on the actual road surface. The l2 area is flat. The deviation of the projection effect mainly comes from the l1 area. Use D(h, w) U The function represents the depth information of the projected road surface area l1, which is incorporated into the driver coordinate system U and expressed as (h, w, D(h, w) U ). A driver's field of vision space is obtained by dividing in the driver's coordinate system U. For any line segment AB on the depth ideal plane, there is a corresponding line segment A′B′ in the driver's field of vision space, and the vector similarity constraint is satisfied:

[0064]

[0065] Where S represents the distance between any line segment AB and the driver in the depth direction; the measured ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of view is calculated based on the depth deviation tensor. When , it indicates that the projected road surface is sunken, and it is necessary to reduce the size of the corresponding area on the pre-projected image according to the depth deviation tensor, that is, to shrink the light in this part and reduce the scattering angle; when the measured ratio is greater than When , it indicates that the projected road surface is convex. The size of the corresponding area on the pre-projected image is enlarged according to the depth deviation tensor, that is, the light of this part is expanded and the scattering angle is increased, that is, the image size of the driver on the convex road surface or the concave surface is the same as the image size on the depth ideal plane, avoiding overlapping or tearing of the projected image at the concave and convex positions. The scale coefficients of reduction and enlargement are calibrated with the coordinate position to obtain the global depth compensation matrix Q, and the pre-projected image is depth corrected according to the depth compensation matrix Q.

[0066] Perform color avoidance P() on the heterochromatic area of ​​the pre-projected image img and perform depth correction Q() to obtain the final projected image simg. The logical processing formula of the whole process is:

[0067] simg = Q(MAX(P(img)));

[0068] Send the projection image simg to the headlight for projection, and during the entire projection process, the processor periodically sends commands to drive the binocular camera to periodically detect image information and depth information. After analysis and processing, the projection headlight is driven again to project the updated projection image. This cycle is repeated to improve the projection effect of the projection image without differentiation and attenuate the detection frequency over time, taking energy-saving measures into consideration while ensuring the projection accuracy.

[0069] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. Those skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0070] The above specific implementations of the present invention do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An intelligent vehicle light control method based on color avoidance and depth correction, characterized in that: The following steps are involved: S1. Collect images and depth information of the vehicle's surrounding environment through an optical sensing device, and use a cyclic decision algorithm to adaptively adjust the detection frequency of the optical sensing device; S2. Analyze and process the projection plane: Performing color processing on the image to obtain a color deviation tensor from the ideal plane; performing depth analysis on the depth information to obtain a depth deviation tensor from the ideal plane; and identifying the image content, and determining the corresponding pre-projected image according to the identified content; S3, analyzing and processing the pre-projected image: The pre-projection image is subjected to color avoidance and depth correction according to the color deviation tensor and the depth deviation tensor to obtain a projection image, the projection image is projected onto a projection plane through the vehicle lights, and the corresponding information of the projection image is displayed on a display screen.

2. The intelligent vehicle light control method based on color avoidance and depth correction as claimed in claim 1, characterized in that: The color processing process is: The three color channels of the image, R, G, and B, and the height and width of the projection plane form a five-dimensional feature. H represents the height of the projection plane, and W represents the width of the projection plane. Any point in the projection plane is represented as [r, g, b] (h,w) , r represents the specific value of the point in the R dimension, g represents the specific value of the point in the G dimension, b represents the specific value of the point in the B dimension, h represents the specific value of the point in the H dimension, and w represents the specific value of the point in the W dimension; According to the color information of the image, the global color fitting plane is obtained and the [r, g, b] of any point is calculated. (h,w) The color deviation tensor of the corresponding points on the global color fitting plane under the three color channels of R, G, and B.

3. The intelligent vehicle light control method based on color avoidance and depth correction as claimed in claim 1, characterized in that: The process of deep processing is: The depth of the projection plane, the height and the width of the projection plane form a three-dimensional feature. Any point in the projection plane is represented by d (h,w) , d is the specific value of the depth of the point on the projection plane; the ideal depth plane is obtained by fitting the depth information, and d is calculated for any point (h,w) The depth deviation tensor of the corresponding points on the depth ideal plane.

4. The intelligent vehicle light control method based on color avoidance and depth correction as claimed in claim 1, characterized in that: The process of color avoidance is: Traverse all pixels of any frame of pre-projection image, and calculate the color deviation tensor ΔC(x,y) of each pixel of the pre-projection image according to the obtained color deviation tensor of the projection plane, where x and y represent the coordinates of the pixel point on the pre-projection image; Set the color channel threshold to ΔT. When |ΔC(x,y)|>ΔT, take the pixel as the starting point and expand the rectangular box in the four directions of upward, downward, left and right. When the color deviation tensor of the pixel is less than or equal to the color channel threshold, stop expanding. The obtained rectangular box is a heterochromatic area, which is avoided.

5. The intelligent vehicle light control method based on color avoidance and depth correction as claimed in claim 4, characterized in that: The starting pixel is (x1, y1), and the pixel currently expanded to is (x i ,y i ), the judgment process of different color areas is: If |ΔC(x2,y i-1 )|≤ΔT, then (x2,y i-1 ) is the upper boundary of the rectangular box; If |ΔC(x2,y i+1 )|≤ΔT, then (x2,y i+1 ) is the lower border of the rectangular box; If |ΔC(x i-1 ,y2)|≤ΔT, then (x i-1 ,y2) is the left boundary of the rectangular box; If |ΔC(x i+1 ,y2)|≤ΔT, then (x i+1 ,y2) is the right border of the rectangular box.

6. The intelligent vehicle light control method based on color avoidance and depth correction as claimed in claim 1, characterized in that: The process of depth correction is: The driver coordinate system U is established with the driver's eyes as the origin, the camera coordinate system C is established with the optical sensing device as the origin, and the projection coordinate system P is established with the headlights as the origin; Determine the pose transformation matrix of U and P based on the spatial position of the driver's eyes and headlights Determine the pose transformation matrix of C and P based on the spatial position of the optical sensing device and the headlight Converting the depth information based on the camera coordinate system acquired by the optical perception device into the depth information based on the driver coordinate system; For any line segment AB on the depth ideal plane, there is a corresponding line segment A′B′ in the driver's field of vision, and the vector similarity constraint is satisfied: Where S represents the distance between any line segment AB and the driver in the depth direction; The measured ratio of any line segment on the projection plane to the corresponding line segment in the driver's field of vision is calculated based on the depth deviation tensor. When the measured ratio is greater than When , it indicates that the projected road surface is raised. The size of the corresponding area on the pre-projected image is enlarged according to the depth deviation tensor, the scale coefficients of reduction and enlargement are calibrated with the coordinate position, and the global depth compensation matrix Q is obtained. The pre-projected image is depth corrected according to the depth compensation matrix Q.

Images