Vehicle lamp calibration method, device, equipment and computer readable storage medium

Abstract

The present application relates to a method, device, computer equipment and computer-readable storage medium for calibrating a headlight. The method includes: determining a camera projection relationship; the camera projection relationship is used for coordinate conversion between the camera's plane coordinate system and a reference coordinate system; determining a plane conversion relationship of the headlight; the plane conversion relationship is used for coordinate conversion between the camera's plane coordinate system and the headlight's projection surface coordinate system; adjusting and calibrating the plane conversion relationship according to the camera projection relationship to obtain a headlight projection relationship; wherein the headlight projection relationship is used for coordinate conversion between the headlight's projection surface coordinate system and the reference coordinate system. The use of this method can improve processing efficiency and can more accurately control the projection position of the vehicle through the headlight projection relationship.

Description

Technical Field

[0001] The present application relates to the field of vehicle lamp calibration, and in particular to a vehicle lamp calibration method, device, computer equipment and computer-readable storage medium. Background Art

[0002] With the emergence of new technologies, smart car lights have emerged in the field of intelligent driving. With the improvement of car light pixels, car lights are no longer satisfied with basic lighting, but can have many functions such as interaction, prompts, navigation, and assisted driving. These functions need to be realized based on car light calibration.

[0003] The current industry's vehicle headlight calibration is limited by the properties of the lights themselves, such as brightness and clarity, as well as the environment and site. It can only be calibrated in specific scenarios and with specific tools, and the calibration efficiency is low. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and computer-readable storage medium for calibrating vehicle lights that can ensure efficiency in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a method for calibrating a vehicle lamp, comprising:

[0006] Determine a camera projection relationship; the camera projection relationship is used for coordinate conversion between the camera's plane coordinate system and the reference coordinate system;

[0007] Determining a plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between a plane coordinate system of the camera and a projection surface coordinate system of the headlight;

[0008] According to the camera projection relationship, the plane transformation relationship is adjusted and calibrated to obtain the headlight projection relationship;

[0009] The headlight projection relationship is used for coordinate conversion between the projection surface coordinate system of the headlight and the reference coordinate system.

[0010] In one embodiment, determining the camera projection relationship includes:

[0011] Calibrate the extrinsic parameters of the camera based on the coordinates of the calibration points in the camera coordinate system and the coordinates of the calibration points in the reference coordinate system;

[0012] Calibrate the intrinsic parameters of the camera based on the coordinates of the calibration points and the depth values ​​of the calibration points in the plane coordinate system of the camera and the coordinates of the calibration points in the reference coordinate system;

[0013] A camera projection relationship is determined according to the external parameters of the camera and the internal parameters of the camera.

[0014] In one embodiment, the reference coordinate system is a reference system where a reference plane is located; and determining the plane transformation relationship of the vehicle lamp includes:

[0015] Acquiring a projection result of the vehicle light on the reference surface based on the camera, and obtaining a projection image of the vehicle light on the reference surface;

[0016] Determining projection parameters of the headlight based on the headlight projection image;

[0017] A plane conversion relationship between the camera and the headlight is obtained according to the projection parameters of the headlight.

[0018] In one embodiment, the projection result is obtained by projecting the vehicle light based on a preset pattern;

[0019] The determining the projection parameters of the headlight based on the headlight projection image comprises:

[0020] Based on the projection image of the vehicle lamp, the projection parameters of the vehicle lamp are corrected until the corrected projection image of the vehicle lamp matches the preset pattern, thereby obtaining the corrected projection parameters of the vehicle lamp;

[0021] According to the corrected projection parameters of the headlight, the projection parameters of the headlight under the perspective of a pedestrian are determined; wherein the perspective of a pedestrian is a projection perspective in the reference coordinate system.

[0022] In one embodiment, the correcting the projection parameters of the headlight based on the headlight projection image includes:

[0023] Determine the shape deviation ratio between the projected image of the vehicle lamp and the preset image;

[0024] Calibrate the inverse ratio of the negative correlation of the shape deviation ratio;

[0025] The reverse ratio is used to adjust the shape of the projection result, and the shape deviation ratio and the reverse ratio indicate opposite changes in the shape of the projection result at the same position.

[0026] In one embodiment, the method further comprises:

[0027] After determining the projection parameters of each of the headlights, calibrating the shape alignment parameters between the headlights according to the corresponding relationship between the headlights;

[0028] The shape alignment parameters are used to perform shape alignment on the positions of the headlight projection images of the headlights.

[0029] In one embodiment, the method further comprises:

[0030] After determining the projection parameters of each of the headlights, calibrating the grayscale weight of each pixel position in the overlapping area according to the brightness of the headlight projection image of each of the headlights;

[0031] The overlapping area is an intersection area of ​​the headlight projection images of the headlights, and the grayscale weight is used to fuse the brightness of each headlight in the overlapping area.

[0032] In a second aspect, the present application further provides a lamp calibration device, comprising:

[0033] The display and photo module is used to determine the camera projection relationship; the camera projection relationship is used for coordinate conversion between the camera's plane coordinate system and the reference coordinate system;

[0034] The display and photo-taking module is used to determine the plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between the plane coordinate system of the camera and the projection surface coordinate system of the headlight;

[0035] A planning module, used to adjust and calibrate the plane transformation relationship according to the camera projection relationship to obtain a headlight projection relationship;

[0036] The headlight projection relationship is used for coordinate conversion between the projection surface coordinate system of the headlight and the reference coordinate system.

[0037] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of calibrating the vehicle lights in any of the above embodiments are implemented.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of calibrating the vehicle lights in any of the above embodiments are implemented.

[0039] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of calibrating the vehicle lights in any of the above embodiments are implemented.

[0040] The above-mentioned calibration method, device, computer equipment and computer-readable storage medium for headlights determine the camera projection relationship; the camera projection relationship is used for the coordinate conversion between the camera's plane coordinate system and the reference coordinate system, so that the camera's projection can be accurately converted to the reference coordinate system, forming a coordinate system conversion between different dimensions; since the image captured by the camera and the projection image of the headlight belong to different two-dimensional coordinate systems, the plane conversion relationship of the headlight is determined, and a plane conversion relationship between two-dimensional coordinate systems can be formed; finally, according to the camera projection relationship, the plane conversion relationship is adjusted and calibrated to obtain the headlight projection relationship; so that the coordinates between the projection surface coordinate system of the headlight and the reference coordinate system can be converted. Therefore, in the case where the coordinate system relationship between the headlight and the ground cannot be determined, the camera projection relationship and the plane conversion relationship are used as intermediate data to indirectly calibrate the headlight projection relationship, which can be adapted to a variety of calibration tools, the tool has little impact, the efficiency is relatively high, and the concept of the headlight projection relationship is clarified. Through the headlight projection relationship, the direction of the headlight illumination can be dynamically controlled, and the projection position of the headlight in the reference coordinate system and other properties can be configured. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a schematic flow chart of a method for calibrating a vehicle lamp in one embodiment;

[0043] Figure 2 A schematic diagram of coordinate system conversion of a method for calibrating a vehicle lamp in one embodiment;

[0044] Figure 3 A schematic diagram of a process for determining a plane conversion relationship in one embodiment;

[0045] Figure 4 is an application environment diagram of a vehicle lamp calibration method in another embodiment;

[0046] Figure 5 A data flow diagram of image correction in one embodiment;

[0047] Figure 6 is a schematic diagram of the effect of a method for calibrating a vehicle lamp in one embodiment;

[0048] Figure 7 A schematic diagram of the projection result of dual vehicle lights in one embodiment;

[0049] Figure 8 A schematic diagram of the effect of shape alignment parameters in one embodiment;

[0050] Fig. 9 is a mask schematic diagram of grayscale weight in one embodiment;

[0051] Fig.10 A schematic diagram of a process flow of a dual-lamp hard synchronous fusion pathway in one embodiment;

[0052] Fig.11 A schematic diagram of a process of main driver PMF calibration in one embodiment;

[0053] Fig.12 A schematic diagram of a process of PMF calibration for pedestrian perspective in an embodiment;

[0054] Fig.13 is a schematic flow chart of a self-calibration process in one embodiment;

[0055] Fig.14 is a structural block diagram of a calibration device for a vehicle lamp in one embodiment;

[0056] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In an exemplary embodiment, Figure 1 As shown, a method for calibrating a vehicle lamp is provided, comprising the following steps S102 to S106. Among them:

[0059] Step S102, determining the camera projection relationship; the camera projection relationship is used for coordinate conversion between the camera's plane coordinate system and the reference coordinate system.

[0060] The camera projection relationship is a coordinate system mapping relationship for camera settings. The camera projection relationship is used to transform coordinate systems with different numbers of dimensions to form cross-dimensional coordinate transformations. In the case of different numbers of dimensions, the coordinate system with a lower number of dimensions requires additional data dimensions in the camera projection relationship to be mapped to form a transformation from low dimensions to high dimensions.

[0061] The camera projection relationship includes the camera's intrinsic parameters and extrinsic parameters. The camera's intrinsic parameters represent data such as the camera's optical properties and the sensor's physical properties. The camera's extrinsic parameters represent the camera's position in the reference coordinate system, that is, the camera's position in the reference coordinate system and the camera's posture relative to the reference coordinate system, which is determined by the translation and rotation data in the reference coordinate system.

[0062] The plane coordinate system of the camera is a two-dimensional coordinate system designed for the camera. Since the camera has the function of image acquisition, the plane coordinate system of the camera is a coordinate system designed for the image acquired by the camera; the plane coordinate system of the camera can be a pixel coordinate system designed for the image acquired by the camera. The pixel coordinate system includes the upper left corner of the image as the origin, the x-axis horizontally to the right, and the y-axis vertically downward.

[0063] The reference coordinate system is a coordinate system with at least three dimensions. The number of dimensions of the reference coordinate system is greater than the number of dimensions of the camera's plane coordinate system. Since the number of dimensions of the reference coordinate system is greater than the number of dimensions of the screen coordinate system, the position and posture of the camera, the position of the image captured by the camera, the position and posture of the headlights, and the position of the projection of the headlights can be represented in the reference coordinate system. Optionally, the reference coordinate system is a multidimensional coordinate system established with a certain reference object, for example, the reference coordinate system can be a world coordinate system.

[0064] Optionally, the plane coordinate system of the camera includes an X dimension and a Y dimension, and the reference coordinate system includes an X dimension, a Y dimension and a Z dimension, and based on the data of the X dimension and the Y dimension, and using a first camera projection matrix, the positions (x, y) of the X dimension and the Y dimension, and the pixel values ​​(Z) of the positions indicated by the X dimension and the Y dimension are mapped, thereby forming a three-dimensional coordinate point in the reference coordinate system through a depth represented by the pixel value (Z); correspondingly, using a second camera projection matrix, the three-dimensional coordinate point in the reference coordinate system is mapped, and an image with a corresponding pixel value (Z) can be formed at the position (x, y) of the X dimension and the Y dimension, thereby forming an image with a depth represented by the pixel value (Z).

[0065] In an optional embodiment, determining the camera projection relationship includes: fitting based on the coordinates of the calibration point in the plane coordinate system of the camera and the coordinates of the calibration point in the reference coordinate system to obtain the camera projection relationship. The calibration point may be a calibration point acquired by the camera. Thus, the intrinsic connection between the data is found by using the least squares method, neural network, etc. to accurately determine the camera projection relationship.

[0066] In an optional embodiment, determining the camera projection relationship includes: obtaining a preset camera projection relationship from a database according to the vehicle model and the camera identifier, thereby adaptively adjusting the camera projection relationship according to the vehicle model to ensure processing efficiency.

[0067] Step S104, determining the plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between the plane coordinate system of the camera and the projection surface coordinate system of the headlight.

[0068] The plane transformation relationship is the coordinate system mapping relationship for the headlight and camera settings. The plane transformation relationship is used to transform the two-dimensional coordinate systems of different reference objects to transform the coordinate systems corresponding to the headlight and camera. Since the positions of the headlight and the camera are not the same, and their postures or other parameters may also be different, the plane coordinate system of the camera is different from the projection surface coordinate system of the headlight, and mapping is required.

[0069] The plane transformation relationship includes a coordinate transformation method between two two-dimensional coordinate systems; the coordinate transformation method can be a matrix or a function. Optionally, the plane transformation relationship includes a homography matrix from the projection plane of the projector to the image plane, and the homography matrix is ​​obtained by fitting the feature points contained in the plane coordinate system of the camera and the matching points contained in the projection plane coordinate system of the headlight. The feature points and the matching points belong to the points of the same object in different coordinate systems, and the fitting method includes but is not limited to the least squares method, polynomial maximum likelihood estimation and other algorithms.

[0070] Optionally, the plane transformation relationship can be determined by the intrinsic parameters and extrinsic parameters of the camera, as well as the projection parameters of the headlight, so as to improve the accuracy through the corresponding parameters. Optionally, the camera intrinsic parameters can transform a certain feature point in the camera's plane coordinate system to a three-dimensional plane, realizing the coordinate transformation of the camera's plane coordinate system to the camera coordinate system; and based on the extrinsic parameters of the camera and the headlight, the position of the same feature point headlight in the three-dimensional coordinate system of the headlight is determined, and then according to the intrinsic parameters of the headlight, the position of the same feature point headlight in the three-dimensional coordinate system of the headlight is transformed to the projection surface coordinate system of the headlight, so that the feature point is transformed from the camera coordinate system to the three-dimensional coordinate system of the headlight, and then from the three-dimensional coordinate system of the headlight to the projection surface coordinate system of the headlight.

[0071] The projection surface coordinate system is the two-dimensional coordinate system where the projection result of the headlight is located. The projection result of the headlight can be an animation projection result, an interactive entertainment projection result, or an auxiliary and safe driving projection result. Optionally, each headlight has its own projection surface coordinate system, so if there are multiple headlights with projection function, the multiple headlights each have their own projection surface coordinate system.

[0072] Step S106, adjusting and calibrating the plane transformation relationship according to the camera projection relationship to obtain the headlight projection relationship; wherein the headlight projection relationship is used for coordinate transformation between the projection plane coordinate system of the headlight and the reference coordinate system.

[0073] The headlight projection relationship is used to convert the projection result of the headlight into the projection result of the reference coordinate system to form a new dimension of headlight control, so as to more accurately control the position of the projection result in the reference.

[0074] Optionally, the plane conversion matrix between the camera and the headlight is multiplied by the camera projection matrix used to represent the camera projection relationship to obtain a matrix calculation result; the calculation result is used as a conversion matrix used to represent the headlight projection relationship; and the conversion matrix is ​​used as at least one parameter for controlling the headlight. Thus, the plane conversion relationship is adjusted by matrix multiplication. The matrix calculation result is obtained, and then the headlight projection relationship is calibrated by the calculation result to ensure processing efficiency.

[0075] Optionally, the calibration process may refer to adding the projection relationship of the vehicle lights to the projection parameters of the vehicle lights, so that the projection relationship of the vehicle lights is adjustable. Optionally, the calibration process may also refer to recording the projection relationship of the vehicle lights.

[0076] Alternatively, if Figure 2 As shown, the relationship between the light and the ground is not directly obtainable because the light pixel coordinate system of the projection result and the world coordinate system of the ground cannot be directly obtained. Therefore, the camera projection relationship is determined through step S102, that is, the camera-ground relationship, and the conversion relationship between the camera's plane coordinate system (camera pixel coordinate system) and the world coordinate system is clarified. Then, the plane conversion relationship of the headlight is determined through step S104, that is, the camera-camera relationship, and the conversion relationship between the light pixel coordinate system and the camera pixel coordinate system is clarified. Finally, the headlight projection relationship, that is, the light-ground relationship, is indirectly obtained, and the conversion relationship between the light pixel coordinate system and the world coordinate system is clarified.

[0077] In the above-mentioned calibration method of the headlight, the camera projection relationship is determined; the camera projection relationship is used for the coordinate conversion between the plane coordinate system of the camera and the reference coordinate system, so that the camera projection can be accurately converted to the reference coordinate system, forming a coordinate system conversion between different dimensions; because the image captured by the camera and the projection image of the headlight belong to different two-dimensional coordinate systems, the plane conversion relationship of the headlight is determined, and a plane conversion relationship between two-dimensional coordinate systems can be formed; finally, according to the camera projection relationship, the plane conversion relationship is adjusted and calibrated to obtain the headlight projection relationship; so that the coordinates between the projection surface coordinate system of the headlight and the reference coordinate system can be converted. Therefore, in the case where the coordinate system relationship between the headlight and the ground cannot be determined, the camera projection relationship and the plane conversion relationship are used as intermediate data to indirectly calibrate the headlight projection relationship, which can be adapted to a variety of calibration tools, has a small impact on the tools, is relatively efficient, and clarifies the concept of the headlight projection relationship. Through the headlight projection relationship, the direction of the headlight illumination can be dynamically controlled, and the projection position of the headlight in the reference coordinate system and other properties can be configured.

[0078] In an exemplary embodiment, determining the camera projection relationship includes: calibrating the camera's external parameters based on the calibration point coordinates in the camera coordinate system and the calibration point coordinates in the reference coordinate system; calibrating the camera's internal parameters based on the calibration point coordinates and calibration point depth values ​​in the camera's plane coordinate system, and the calibration point coordinates in the reference coordinate system; and determining the camera projection relationship based on the camera's external parameters and the camera's internal parameters.

[0079] The camera coordinate system is a multidimensional coordinate system defined with the camera as a reference. The origin of the camera coordinate system is related to the position of the camera lens, and the direction of the camera coordinate system is related to the direction outside the camera. Optionally, when the camera coordinate system is three-dimensional, the origin of the camera coordinate system is the focus of the camera lens, the x-axis points to the right of the camera, the y-axis points to the top of the camera, and the z-axis points to the front of the camera, and the front of the camera is the direction of the camera's optical axis.

[0080] The calibration point coordinates are the coordinate positions of the same feature point in the corresponding coordinate system. The coordinates of the same calibration point in the camera coordinate system and the reference coordinate system are the same feature point, and because the camera coordinate system and the reference coordinate system are defined based on different reference objects, the calibration point coordinates in the camera coordinate system are different from the calibration point coordinates in the reference coordinate system. Optionally, the calibration point coordinates can be multiple dots on a dot calibration plate, or they can be the corner points of a checkerboard calibration plate.

[0081] The depth value of the calibration point is the image value of the coordinates of the same feature point in the corresponding coordinate system. Since the number of dimensions of the plane coordinate system is less than the number of dimensions of the reference coordinate system, the concept of the calibration point depth value is used to make up for the number of dimensions of the calibration point coordinates in the plane coordinate system to calibrate the intrinsic parameters of the camera.

[0082] The camera's extrinsic parameters are used to convert coordinates in the reference coordinate system into coordinates in the camera coordinate system. The camera's extrinsic parameters include at least translation parameters and rotation parameters, and may also include other data such as distortion coefficients. Optionally, the camera's extrinsic parameters may be in the form of a matrix.

[0083] The intrinsic parameters of the camera are used to convert the coordinates in the camera coordinate system into the coordinates in the camera's plane coordinate system according to the depth value. Optionally, the intrinsic parameters of the camera can be in the form of a matrix. The intrinsic parameters of the camera can be a matrix composed of focal length, principal point, coordinate slope, or other data.

[0084] Optionally, the camera projection relationship may be a matrix multiplication of the camera's external parameters and the camera's internal parameters, or may be formed by adjusting the camera's external parameters and the camera's internal parameters based on other data and combining the adjusted results.

[0085] In an optional embodiment, the external parameters of the camera are calibrated based on the coordinates of the calibration points in the camera coordinate system and the coordinates of the calibration points in the reference coordinate system, including: multiplying the inverse matrix of the calibration point coordinates in the reference coordinate system by the matrix formed by the coordinates of the calibration points in the camera coordinate system of the camera to obtain the external parameters of the camera; and recording the external parameters of the camera.

[0086] In an optional embodiment, based on the calibration point coordinates and calibration point depth values ​​in the camera's plane coordinate system, and the calibration point coordinates in the reference coordinate system, the camera's intrinsic parameters are calibrated, including: multiplying the inverse of the depth value by the camera's extrinsic parameters, and then multiplying it by the calibration point coordinates in the reference coordinate system to obtain an intermediate calculation result; multiplying the inverse matrix of the intermediate calculation result by the calibration point coordinates in the camera's plane coordinate system to obtain the camera's intrinsic parameters; and recording the camera's intrinsic parameters.

[0087] Optionally, the camera projection relationship is expressed as:

[0088] P = K[R t];

[0089] Among them, K is the intrinsic parameter of the camera, and [R t] is the extrinsic parameter of the camera.

[0090] Correspondingly, an expression is first used to determine the external parameter [R t] between the camera and the ground. The corresponding expression is as follows:

[0091]

[0092] Among them, the subscript w stands for world, that is, the world coordinate system, c stands for camera, that is, the camera coordinate system, Xc is the X coordinate in the plane coordinate system, Yc is the Y coordinate in the plane coordinate system, and Zc is the depth value in the plane coordinate system, which corresponds to the coordinate of the calibration point in the Z-axis direction of the camera coordinate system; [Xw, Yw, Zw] is the three-dimensional coordinate of the calibration point in the world coordinate system, and 1 is for homogeneous normalization; [R, T] is the external parameters of the camera, R is the rotation matrix, and T is the translation matrix. The external parameters of the camera can be obtained through the corresponding inverse matrix.

[0093] Next, the relationship between the camera's 2D pixel plane and the 3D world coordinate system is obtained using the expressions for the camera's 3D coordinate system and the world's 3D coordinate system. The expressions are as follows:

[0094]

[0095] Among them, [u,v,1] is the coordinate value of the calibration point in the camera pixel coordinate system; Zc is the coordinate of the calibration point in the Z-axis direction; [Xw,Yw,Zw] is the three-dimensional coordinate of the calibration point in the world coordinate system, 1 is for homogeneous normalization, K is the intrinsic parameter of the camera, which can be obtained through the corresponding inverse matrix.

[0096] In this embodiment, since the dimensions of the camera coordinate system and the reference coordinate system are closer, the camera coordinate system and the reference coordinate system are used for calibration, and the coordinate system conversion process can be omitted, thereby improving the accuracy of the camera's external parameters. At the same time, since the camera's plane coordinate system is related to the camera's image acquisition, it can reflect the camera's internal parameters.

[0097] In an alternative embodiment, Figure 3 As shown, the reference coordinate system is the reference system where the reference plane is located; determining the plane transformation relationship of the headlight includes:

[0098] Step S302: acquiring the projection result of the vehicle headlights on the reference surface based on the camera, and obtaining a projection image of the vehicle headlights on the reference surface.

[0099] The reference surface is a solid surface in the reference coordinate system, and the solid surface can bear the projection of the headlight. Optionally, the reference surface can refer to a projection screen; the reference surface can also refer to the ground. Correspondingly, when the reference surface is the ground, the reference coordinate system is a world coordinate system. Optionally, the reference surface is provided with a grid calibration plate to form calibration points through corner points on the grid calibration plate.

[0100] The projection result is an image displayed on the reference surface. Since the projection result is projected by the headlight, when the headlight projects according to the preset image, the projection result is not the same as the preset image, but has been adjusted according to the projection parameters of the headlight, so the projection result can represent the headlight coordinate system.

[0101] The headlight projection image is obtained by shooting the projection result with a camera, so the headlight projection image and the projection result have corresponding deformation. The deformation of the headlight projection image relative to the projection result is related to the camera's internal and external parameters. Therefore, after determining the camera's internal and external parameters, and the preset image input to the headlight for projection, the headlight projection parameters can be determined based on the headlight projection image.

[0102] Optionally, a preset pattern can be projected onto a reference surface through vehicle lights, and a projection result on the reference surface is captured based on a camera to obtain a vehicle light projection image on the reference surface.

[0103] Step S304: determining projection parameters of the vehicle headlight based on the vehicle headlight projection image.

[0104] The projection parameters of the headlights correspond to the projection surface coordinate system of the headlights. The preset image can be converted into an image in the projection surface coordinate system through the projection parameters of the headlights. The projection parameters include the internal parameters and external parameters of the headlights. The internal parameters of the headlights represent the optical properties of the headlights, and the external parameters of the headlights represent the position and posture of the headlights. Both the internal parameters and the external parameters of the headlights can be in matrix form.

[0105] In an optional embodiment, the projection parameters of the headlights are determined based on the projection image of the headlights, including: after acquiring a calibration image through a camera, and determining the external parameters of the headlights through the coordinates of the calibration points in the calibration image and the reference coordinate system, based on the camera parameters of the camera, converting the projection calibration points in the projection image of the headlights to the reference coordinate system to obtain the projection calibration points of the reference coordinate system; determining the internal parameters of the headlights according to the projection calibration points of the reference coordinate system; the external parameters of the headlights and the internal parameters of the headlights belong to the projection parameters of the headlights.

[0106] Correspondingly, according to the projection parameters of the headlight, the plane transformation relationship between the camera and the headlight is obtained, including: according to the internal and external parameters of the headlight, as well as the internal and external parameters of the camera, the plane transformation relationship between the camera and the headlight is determined. Therefore, the internal and external parameters provide detailed geometric information, which can realize high-precision coordinate transformation and form a coordinate transformation relationship in any dimensional space.

[0107] Step S306, obtaining a plane transformation relationship between the camera and the headlight according to the projection parameters of the headlight.

[0108] In an optional embodiment, the feature points corresponding to the projection result and the projection image of the headlight represent the same calibration point; correspondingly, the plane transformation relationship between the camera and the headlight is obtained according to the projection parameters of the headlight, including: after determining the homography transformation relationship based on the feature points corresponding to the projection result and the projection image of the headlight, adjusting the projection parameters of the headlight, and adjusting the homography transformation relationship according to the adjusted projection parameters of the headlight until the homography transformation relationship meets the convergence condition, and obtaining the plane transformation relationship between the camera and the headlight according to the homography transformation relationship that meets the convergence condition. Thus, through multiple adjustment processes of the projection parameters, the homography transformation relationship is adaptively adjusted to improve accuracy.

[0109] In this embodiment, based on the camera collecting the projection result of the headlight on the reference surface, the headlight projection image on the reference surface is obtained, so the projection result in the headlight projection image and the actual projection result on the reference surface belong to the same object in different coordinate systems, so there are matching feature points between the two. Then, based on the headlight projection image, the projection parameters of the headlight are determined. Since the projection parameters of the headlight provide detailed coordinate system related information, high-precision coordinate transformation can be achieved to more accurately obtain the plane transformation relationship between the camera and the headlight.

[0110] In an optional embodiment, the projection result is obtained by projecting the headlight based on a preset pattern; correspondingly, the projection parameters of the headlight are determined based on the projection image of the headlight, including: based on the projection image of the headlight, correcting the projection parameters of the headlight until the corrected projection image of the headlight matches the preset pattern, and obtaining the corrected projection parameters of the headlight; according to the corrected projection parameters of the headlight, determining the projection parameters of the headlight from the perspective of a pedestrian; wherein the pedestrian perspective is a projection perspective in the reference coordinate system.

[0111] The corrected headlight projection image matches the preset pattern, that is, the corrected headlight projection image has the same projection pattern as the preset pattern, and the deviation ratio between the preset pattern and the projection pattern is less than the preset deviation threshold, and the specification difference between the preset pattern and the projection pattern is within the preset specification range. For example: the corrected headlight projection image has the same projection pattern as the preset pattern, and the deviation ratio between the preset pattern and the projection pattern is less than 30%, and the specification difference between the preset pattern and the projection pattern is less than 10%.

[0112] The pedestrian perspective is the perspective of a person outside the vehicle on the reference plane. When using the pedestrian perspective projection, due to the presence of the pitch angle of the headlight installation, the distortion is obvious, and a single-lamp pedestrian perspective projection correction needs to be performed for each headlight. Alternatively, due to the uncertainty of the external pedestrian perspective, pedestrians may appear from all directions, so for the sake of balance, a bird's-eye view is used.

[0113] like Figure 4 As shown, the projection of the headlights has perspective 1 and perspective 2. Perspective 1 is the main driver's perspective, which can be projected using the vehicle's central control or headlight detection camera, that is, when the vehicle is driving, the main driver looks at the headlights in the same projection direction; Perspective 2 is the bird's-eye view (BEV), which is a bird's-eye view. It is a bird's-eye view, which is aimed at the uncertainty of the external pedestrian's perspective. Therefore, no matter which direction the pedestrian appears, he can see the corrected projection image to balance the viewing conditions in different directions.

[0114] Therefore, the projection of headlights can be used for speed limit signs. After detecting the speed limit sign of the current section, the maximum speed of the current section can be obtained and judged against the vehicle's own body signal. If the vehicle is speeding, the speed limit sign will be projected to remind the driver that the vehicle is currently speeding. The projection of headlights can also be used for distance warning of the vehicle in front: after detecting that the distance to the vehicle in front of the current lane is less than a certain value, the distance to the vehicle in front will be projected to warn the driver to pay attention to the vehicle in front.

[0115] In an optional embodiment, the projection parameters of the headlights include distortion parameters of the headlights; based on the projection image of the headlights, the projection parameters of the headlights are corrected until the corrected projection image of the headlights matches a preset pattern, thereby obtaining the corrected projection parameters of the headlights, including: determining the distortion parameters of the headlights based on the deviation between the projection image of the headlights and the preset pattern; adjusting the projection result of the headlights according to the distortion parameters, and re-executing the above-mentioned step of acquiring the projection result of the headlights on the reference surface based on the camera to obtain the projection image of the headlights on the reference surface, that is, re-executing step S302; when the deviation between the projection image of the headlights and the preset pattern is less than a preset value, the corrected projection image of the headlights matches the preset pattern; determining the corrected projection parameters of the headlights based on the distortion parameters used to obtain the corrected projection image of the headlights.

[0116] Alternatively, if Figure 5 As shown, the distortion parameters include shape deviation ratio, parameters for cropping and scaling, and parameters for edge smoothing. Correspondingly, after determining the projection parameters of the headlights from the perspective of pedestrians, the headlight projection image can be geometrically corrected according to the shape calibration parameters to obtain a shape-corrected image; the shape-corrected image can be adjusted according to the parameters for cropping and scaling to obtain a scaled image; the scaled image can be edge-smoothed according to the parameters for cropping and scaling to form a compensated trapezoidal distortion image; and the compensated trapezoidal distortion image can be used as the corrected headlight projection image.

[0117] Optionally, the projection parameters of the headlights are obtained through Geometric Distortion Correction (GDC). Geometric distortion correction is a process used to correct the image or projection space distortion caused by physical properties of the lens, sensor or projection surface, to ensure accurate representation of the spatial geometry.

[0118] The types of distortion involved in geometric distortion correction include radial distortion and tangential distortion. Radial distortion is caused by the shape of the lens and can be barrel distortion (the image expands in the center area) or pincushion distortion (the edge areas of the image are pulled inward). Tangential distortion occurs when the lens and sensor are not perfectly aligned, causing the image to appear tilted or stretched.

[0119] Optionally, the geometric distortion correction includes a Project Mapping Function (PMF) to achieve correction by mapping points from the distorted image or projection onto a corrected grid based on a mathematical model and its variations.

[0120] Among them, the projection mapping function is a mathematical model used to map points from the distorted image space to the corrected image space. The projection mapping function can be described by a transformation equation or a lookup table (LUT); the projection mapping function includes forward mapping and inverse mapping. Forward mapping refers to mapping from the corrected image space to the distorted image space, that is, projecting the preset image through the car lights to form a projection result; then the projection result is captured by the camera to obtain the car light projection image; inverse mapping refers to mapping from the distorted image space back to the corrected image space, that is, converting the car light projection image into the preset image.

[0121] The mathematical model of the projection mapping function includes: a polynomial model, a calibration matrix, and a lookup table (LUT); wherein the polynomial model can be used to describe image distortion according to the radial distance from the center of the image; the calibration matrix is ​​a matrix that stores the transformation parameters used to correct the image; and the lookup table (LUT) is a pre-calculated table that stores the position of the distorted pixel corresponding to the corrected pixel. The process of correcting the projection mapping function includes but is not limited to the coefficients of the polynomial model, the coefficients of the calibration matrix, and the coefficients in the lookup table (LUT).

[0122] The distortion model is established, including: using the captured data to model the distortion based on a mathematical method (such as a radial distortion model or a polynomial equation).

[0123] Wherein, applying a projection mapping function (PMF) includes: defining the PMF based on a distortion model; and mapping distorted pixel coordinates to corrected coordinates by direct calculation or a lookup table (LUT).

[0124] Correcting the image or projection includes: transforming the image or projection content using a projection mapping function to ensure that the corrected version is aligned with the actual geometric shape.

[0125] In this embodiment, based on the projection image of the headlight, the projection parameters of the headlight are corrected until the corrected projection image of the headlight matches the preset pattern, and the corrected projection parameters of the headlight are obtained. Thus, by gradually correcting based on the projection image of the headlight, the corrected projection parameters can be made more accurate, and then the projection parameters of the headlight from the perspective of the pedestrian are determined based on the corrected projection parameters of the headlight, so as to perform correction according to the perspective of the bird's-eye view.

[0126] In an optional embodiment, based on the headlight projection image, the projection parameters of the headlight are corrected, including: determining a shape deviation ratio between the headlight projection image and a preset image; calibrating a reverse ratio that is negatively correlated with the shape deviation ratio; wherein the reverse ratio is used to adjust the shape of the projection result, and the shape deviation ratio and the reverse ratio indicate opposite changes in the shape of the projection result at the same position.

[0127] The shape deviation ratio indicates the degree of shape distortion between the headlight projection image and the preset image. Optionally, the shape deviation ratio indicates the degree of deviation of the headlight projection image relative to the preset image. Optionally, the shape deviation ratio may be the ratio of deformation of each region of the preset image, or may be a function for calculating the ratio.

[0128] The reverse ratio is used to adjust the projection process of the preset image so that the projection result is closer to the preset pattern. The reverse ratio and the shape deviation ratio are opposite in the direction of adjustment on the shape. Optionally, the reverse ratio is a parameter used to perform shape calibration on the projection result. Optionally, the reverse ratio can be in the form of a function or a matrix; optionally, the shape deviation ratio is a proportional function that gradually increases the deformation with the projection direction; the reverse ratio is a proportional function that gradually decreases the deformation with the projection direction.

[0129] Taking the matrix as an example, the preset image has pixel points (x, y) and pixel points (x+b, y), and the shape deviation ratio is in the form of a matrix; for the matrix composed of the shape deviation ratio, the shape deviation ratio at the pixel point (x, y) is 2, and the shape deviation ratio at the pixel point (x+b, y) is 4. At this time, the corresponding pixel points of the headlight projection image are (2*x, 2*y), and the corresponding pixel points of the headlight projection image are (4*(x+b), 4*y); correspondingly, for the matrix composed of the reverse ratio, the reverse ratio at the pixel point (x, y) is 0.5, and the shape deviation ratio at the pixel point (x+b, y) is 0.25. At this time, the corresponding pixel points of the headlight projection image are (0.5*x, 0.5*y), and the corresponding pixel points of the headlight projection image are (0.25*(x+b), 0.25*y).

[0130] In an optional embodiment, determining the shape deviation ratio between the headlight projection image and the preset image includes: matching feature points in the headlight projection image with feature points in the preset image to obtain each matching point pair; and determining the shape deviation ratio according to the position deviation ratio within each matching point pair. Optionally, the first feature point in the headlight projection image and the second feature point in the preset image form a matching point pair, and the coordinate deviation of the first feature point in the headlight projection image and the coordinate of the second feature point in the preset image are calculated to obtain the coordinate difference between the first feature point and the second feature point; and the shape deviation ratio is obtained by dividing the coordinate difference by the number of coordinates in the preset image. Thus, by determining the coordinate deviation based on the respective coordinate systems of the headlight projection image and the preset image, the shape deviation ratio in the corresponding dimension can be determined according to the dimension of the coordinate system.

[0131] It should be understood that due to the functional properties of the headlights themselves, there is an angle between the headlights and the ground and the wall, which makes the projection information of the headlights produce an image distortion effect of "near small and far large"; in this case, projecting the preset image of the normal matrix through the headlights will make the projection result trapezoidal. In order to suppress this image distortion effect, the projection result needs to be "trapezoidal corrected" to more comprehensively retain the information that the preset image needs to express.

[0132] For example: through the car lights, after projection based on the preset image of 500*500 pixels, the projection result is a trapezoid. After the camera collects the data, it takes a picture to obtain a car light projection image with a lower base of 500 pixels and an upper base of 200 pixels. Correspondingly, after calibrating the corresponding reverse ratio, the corresponding projection result is adjusted according to the reverse ratio, which is similar to projecting a reverse trapezoidal figure with a lower base of 200 pixels and an upper base of 500 pixels. This can reduce the distortion between the projection result obtained by re-projection and the preset pattern.

[0133] In an optional embodiment, the reverse ratio belongs to one of the distortion parameters of the headlight, and the distortion parameter belongs to one of the projection parameters of the headlight; based on the projection image of the headlight, the projection parameters of the headlight are corrected until the corrected projection image of the headlight matches the preset pattern, and the corrected projection parameters of the headlight are obtained, including: based on the deviation between the projection image of the headlight and the preset pattern, the distortion parameters of the headlight are determined; according to the distortion parameters, the projection result of the headlight is adjusted, and the above-mentioned step of acquiring the projection result of the headlight on the reference surface based on the camera to obtain the projection image of the headlight on the reference surface is re-executed, that is, step S302 is re-executed; when the deviation between the projection image of the headlight and the preset pattern is less than the preset value, the corrected projection image of the headlight matches the preset pattern; according to the distortion parameters used to obtain the corrected projection image of the headlight, the corrected projection parameters of the headlight are determined.

[0134] Alternatively, if Figure 5 As shown, the distortion parameters include shape deviation ratio, parameters for cropping and scaling, and parameters for edge smoothing. Correspondingly, after determining the projection parameters of the headlights from the perspective of pedestrians, the headlight projection image can be geometrically corrected according to the shape calibration parameters to obtain a shape-corrected image; the shape-corrected image can be adjusted according to the parameters for cropping and scaling to obtain a scaled image; the scaled image can be edge-smoothed according to the parameters for cropping and scaling to form a compensated trapezoidal distortion image; and the compensated trapezoidal distortion image can be used as the corrected headlight projection image.

[0135] Optionally, the projection parameters of the headlights are obtained through Geometric Distortion Correction (GDC). Geometric distortion correction is a process used to correct the image or projection space distortion caused by physical properties of the lens, sensor or projection surface, to ensure accurate representation of the spatial geometry.

[0136] The types of distortion involved in geometric distortion correction include radial distortion and tangential distortion. Radial distortion is caused by the shape of the lens and can be barrel distortion (the image expands in the center area) or pincushion distortion (the edge areas of the image are pulled inward). Tangential distortion occurs when the lens and sensor are not perfectly aligned, causing the image to appear tilted or stretched.

[0137] Optionally, the geometric distortion correction includes a Project Mapping Function (PMF) to achieve correction by mapping points from the distorted image or projection onto a corrected grid based on a mathematical model and its variations.

[0138] Among them, the projection mapping function is a mathematical model used to map points from the distorted image space to the corrected image space. The projection mapping function can be described by a transformation equation or a lookup table (LUT); the projection mapping function includes forward mapping and inverse mapping. Forward mapping refers to mapping from the corrected image space to the distorted image space, that is, projecting the preset image through the car lights to form a projection result; then the projection result is captured by the camera to obtain the car light projection image; inverse mapping refers to mapping from the distorted image space back to the corrected image space, that is, converting the car light projection image into the preset image.

[0139] The mathematical model of the projection mapping function includes: a polynomial model, a calibration matrix, and a lookup table (LUT); wherein the polynomial model can be used to describe image distortion according to the radial distance from the center of the image; the calibration matrix is ​​a matrix that stores the transformation parameters used to correct the image; and the lookup table (LUT) is a pre-calculated table that stores the position of the distorted pixel corresponding to the corrected pixel. The process of correcting the projection mapping function includes but is not limited to the coefficients of the polynomial model, the coefficients of the calibration matrix, and the coefficients in the lookup table (LUT).

[0140] In an optional embodiment, step S302 is a distortion measurement process, which specifically includes: first capturing a distorted image or projection, and then using a grid or calibration object to measure the deviation between the actual geometry and the expected geometry. Step S302 includes: establishing a distortion model, applying a projection mapping function (PMF) for candidate processing, and correcting the image or projection.

[0141] In this embodiment, the shape deviation ratio between the projected image of the headlight and the preset image is determined; and the inverse ratio that is negatively correlated with the shape deviation ratio is calibrated; thus, the corresponding inverse ratio is calibrated for the projection function of the headlight, so that the deformation of the projection result of the headlight can be relatively small, and the distortion correction of the projection result is achieved through the adjustment of separate parameters.

[0142] In an optional embodiment, the method further includes: after determining the projection parameters of each headlight, calibrating the shape alignment parameters between the headlights according to the corresponding relationship between the headlights; wherein the shape alignment parameters are used to perform shape alignment on the positions of the headlight projection images of each headlight.

[0143] The correspondence of the headlights at least includes the correspondence between the positions and orientations of the headlights, so as to adjust the shape alignment parameters with corresponding relationships for the orientations and reduce the corresponding processing range; optionally, the correspondence of the headlights includes the correspondence between the positions and orientations of the headlights, so as to narrow the number of headlights involved in the corresponding shape alignment parameters; optionally, the correspondence of the headlights also includes the feature point matching relationship of the projection results, so as to determine the displacement changes caused by the feature points belonging to the same object through the feature point matching relationship, and then determine the shape alignment parameters based on the inverse of the displacement change or other inverse proportional data.

[0144] The shape alignment parameter is used to align the projection results of different headlights to form a complete projection result. Optionally, the shape alignment parameter is in the form of a matrix or a function.

[0145] like Figure 6 As shown, after determining the projection parameters of each headlight, when a single headlight is projected through a preset chessboard pattern, a headlight projection image containing a square chessboard can be obtained, rather than a headlight projection image distorted into a trapezoid, so that the projection result of each headlight matches the chessboard.

[0146] On this basis, due to the limitation of the field of view (FOV) of a single headlight, it cannot cover a single lane, nor is it wide enough for the width indicator light blanket of the vehicle body. Therefore, mass production requires multiple headlight modules to make up for the FOV defect. Generally, when there is a corresponding relationship between the headlights, the projection results of multiple headlights have overlapping areas. Figure 7 As shown, in the case where the multiple headlights refer to two headlights, there is a double-headed lamp overlapping area in the projection results of the multiple headlights, and the double-headed lamp overlapping area is a part of the double-headed lamp projection area. The double-headed lamp projection area is the intersection of the projection results of the left-headed lamp projection area and the right-headed lamp projection area, that is, the overlapping area of ​​the projection results; the left-headed lamp projection area and the right-headed lamp projection area are the areas where the left lamp and the right lamp can project respectively.

[0147] When the shape alignment parameter does not exist, the projection result of the two lights is as follows Figure 8As shown in (a) in the figure; when the shape alignment parameter exists, the projection result of the two lights is as follows Figure 8 As shown in (b), Figure 8 In (b), there is a complete preset pattern.

[0148] For example, the input is a picture. After the projection mapping function, the input picture will become two pictures, one for the left light and one for the right light. The two lights project their own pictures and they will be pieced together into one picture. That is, a projection mapping function, after PMF, the left light gets half A, the right light gets half A, and then they are projected separately, and the pictures they get are pieced together into a whole A.

[0149] In this embodiment, after determining the projection parameters of each headlight, multiple headlights are used for projection in view of the field of view (FOV) limitation of a single headlight. In this case, for the headlights that have a corresponding relationship, adjustments are made based on the corresponding relationship to achieve calibration of the corresponding shape alignment parameters, so as to ensure that the projection images of all headlights can be seamlessly connected.

[0150] In an optional embodiment, the method also includes: after determining the projection parameters of each headlight, calibrating the grayscale weight of the overlapping area at each pixel position according to the brightness contained in the headlight projection image of each headlight; wherein the overlapping area is the intersection area of ​​the headlight projection image of the headlight, and the grayscale weight is used to fuse the brightness of each headlight in the overlapping area.

[0151] The overlapping area is the overlapping area of ​​the projection results of at least two vehicle lights. Figure 7 As shown, in the case where the multiple vehicle lights refer to two vehicle lights, there is a double-light overlapping area in the projection results of the multiple vehicle lights.

[0152] Grayscale weight is set for each headlight individually, and grayscale weight is in the form of a mask. Fig. 9 As shown, the grayscale weight of the left light projection result is Fig. 9 In (a), the grayscale weight of the right light projection result is Fig. 9 (b) in the figure.

[0153] Optionally, the grayscale weight of each pixel position in the overlapping area is calibrated according to the brightness contained in the headlight projection image of each headlight, including: calculating the brightness of the fusion interval of the overlapping area according to the relationship between the brightness and grayscale of the light (such as the gamma curve) (the interval can be obtained after the fusion calibration), obtaining the corresponding grayscale weight based on the brightness calculation, and then superimposing the grayscale weights of the left and right lights.

[0154] It should be understood that since the overlapping area of ​​the two lights is the effect of the superposition of the left and right lights, there will be an obvious "bright band" in this area. The human eye is very sensitive to brightness. The appearance of the "bright band" also breaks the integrity of the picture and greatly affects the projection effect of the two lights. Therefore, it is very necessary to adopt an effective method to reduce the brightness of the overlapping area, smoothly transition with the surrounding area, and keep it consistent with the overall brightness of the dual-lamp projection image.

[0155] Optionally, grayscale weight (Alpha) fusion processing is used to generate masks for the left and right lights according to the brightness characteristics of different areas, and then the lights are fused with the masks when projecting, so that the brightness of the entire dual-light projection area is consistent. In addition, the specific brightness fusion function model can also be adjusted based on factors such as pixel transparency, number of parameters, and weight, and is a nonlinear function model.

[0156] In this embodiment, the grayscale weight of each pixel position in the overlapping area is calibrated according to the brightness contained in the headlight projection image of each headlight. Therefore, each pixel in the overlapping area has its own grayscale weight, which can make the brightness of the dual-headlight projection area uniform without bright spots.

[0157] On this basis, the above method can be integrated into a channel, and the functions performed by this channel include: calibration perspective and calibration data; the calibration perspective refers to the calibration of the projection parameters of the main driving perspective (driver) and the pedestrian perspective (bev); and the calibration of related data includes: data calibration of dual-perspective hard-synchronized projection parameters, interactive, game and entertainment projection hard-synchronized data; anti-glare adb, lane line projection hard-synchronized data; dual-light fusion hard-synchronized mask image.

[0158] The headlight calibration method provided in the embodiment of the present application is applied to the hard synchronization path of the smart headlight, and the hard synchronization path of the smart headlight involves the smart AI headlight software architecture of the smart AI headlight software. The smart AI headlight software architecture specifically includes the following modules:

[0159] Body Control Module (BCM): used to receive signals of all body data related to smart headlights, summarize and send them to the smart AI headlight planning module to plan the status of smart AI headlights;

[0160] Planning module: used for the state planning of the overall intelligent AI headlights, such as calibration state, basic auxiliary lighting state, basic animation projection state, interactive entertainment projection state, intelligent assisted driving state, etc. Among them, the calibration state is implemented by this solution.

[0161] Control module: used to control car lights (hundreds of lamp beads, tens of thousands of pixel lights, dozens of levels and millions of pixel lights). After the state planning is completed, the control module generates the projection data required by the intelligent AI car lights. For example, in the calibration state, the control module will get the light projection data during the calibration process and wait for it to be displayed;

[0162] Display module: used for the final display of the headlights. After the control module obtains the data, the display module performs the final processing before sending the data for display, such as how to send it to the headlight end. Before sending it for display, the final data integration will be performed at the same time. For example, the dual-head display will integrate the data of two single heads into one frame for display, reducing the end-to-end delay of the intelligent AI headlights and improving the efficiency of intelligent light control.

[0163] Photo module: As a part of the overall software data acquisition module, the photo module only serves calibration. For this embodiment, the photo module is to send the data required for the light photography calibration to the calibration interface.

[0164] Therefore, this application provides a design that supports the mass production of dual-lamp hard synchronization channels for HD (mega pixels) / DLP (hundreds of thousands of pixels and millions of pixels) smart car light modules. This channel is part of the operation channel of the entire smart AI car light. After the smart car light enters the production line, there is no need to re-run the calibration program or software, and one-click hard synchronization can be achieved directly in the current channel. After the car light leaves the factory, users can also achieve autonomous one-click calibration when the car light module is worn or offset, and the original factory calibration data is inaccurate.

[0165] The steps of the dual-lamp hard synchronous fusion pathway designed in this embodiment are as follows: Fig.10 As shown, it can be divided into the following steps:

[0166] S1, cockpit / central control, buttons, apps, body data and other signals trigger calibration status;

[0167] S2, generate a base map following the current calibration environment;

[0168] S3, main driver PMF calibration. Fig.11 As shown in the figure, the process of main driving PMF calibration includes: reading the data of the left control light of the left base map; sending the photo topic to take a photo to obtain the projection image of the headlight; then, reading the data of the right control light of the right base map to obtain the projection image of the headlight; then, calling the algorithm interface & model to generate the calibration result; finally, the result is reversely verified.

[0169] S4. Main driver PMF calibration verification;

[0170] S5. Pedestrian perspective PMF calibration. Fig.12As shown in the figure, the process of PMF calibration from the pedestrian perspective includes: after reading the main driving calibration data, the parameters of the ground calibration cloth are measured; then, after waiting for the calibration cloth to be laid on the ground, the algorithm interface & model is called to generate the calibration result; finally, the result is reversely verified.

[0171] S6, Pedestrian perspective PMF calibration verification;

[0172] S7, calibration data out-of-bounds verification processing, and calibration results update software configuration;

[0173] The generation of the calibration result contained in the above S5 includes steps S102 to S106, and may also include an embodiment of the calibration method of the vehicle lights related to the above steps S102 to S106.

[0174] In an optional embodiment, the dual-light fusion hard synchronization path of the above-mentioned intelligent AI headlights can be used for technical development and debugging, user-independent calibration after leaving the factory, etc., but for the mass production line of intelligent AI headlights, the above-mentioned calibration still has a certain calibration time, which will have a certain impact on the efficiency of mass production. Therefore, according to the special needs of the above-mentioned mass production, the overall synchronization path is analyzed. Since the same model on the production line has basic characteristics of similar positions, it only needs to be calibrated once for pedestrian perspective correction.

[0175] Correspondingly, such as Fig.13 As shown, the self-calibration process includes two branches; in the first branch, the self-calibration environment of the production line is first built, and then the calibration cloth is laid on the production line; then, the parameters of the production line calibration cloth are measured; photos are taken and the calibration cloth is withdrawn (only for the first time); the pedestrian perspective is self-calibrated multiple times, and when the calibration fails to pass the verification, the calibration is recalibrated until the verification is successful, and the pedestrian perspective self-verification is completed; in the second branch, when the vehicle enters, the main driver's perspective calibration is iteratively performed, and when the calibration fails to pass the verification, the calibration is recalibrated until the verification is successful, and the main driver's perspective self-verification is completed.

[0176] Optionally, the first branch can be performed first, or the second branch can be performed first, that is, the pedestrian perspective is self-calibrated first, and then the main driver's perspective is calibrated; Optionally, the second branch can be performed first, or the first branch can be performed first, that is, the main driver's perspective is calibrated first, and then the pedestrian perspective is self-calibrated. In addition, after the calibration result is updated, if the vehicle enters the calibration scene again, the first branch or the second branch can be re-executed. As a result, subsequent calibrations can use the first calibration data, which can greatly improve the efficiency of production line calibration and reduce costs.

[0177] Based on this, a dual-lamp hard synchronization channel is provided to support the mass production of HD (mega pixels) / DLP (hundreds of thousands of pixels and millions of pixels) smart car light modules. This channel is part of the operation channel of the entire smart AI car light. After the smart car light enters the production line, there is no need to re-run the calibration program or software, and one-click hard synchronization can be achieved directly in the current channel. After the car light leaves the factory, users can also achieve independent one-click calibration when the car light module is worn, offset, and the original factory calibration data is inaccurate.

[0178] Through the calibration relationship between the headlight, camera, and ground, a calibration scheme involving the projection of a fixed size at a fixed position in the projection area is used to realize the projection of fixed-size (configurable) image information at the irradiable position on the ground by the projection device; the dual headlight module fusion scheme in the projection area mainly involves the fusion of projection content and projection brightness. For example, half of the projection content is projected on the left headlight and half on the right headlight. The dual-headlight projection content can be fused to display the overall projection content, and the brightness of the dual-headlight projection area is uniform without bright spots. Moreover, the dual-module self-calibration and hard synchronization fusion scheme for high-pixel headlights (million pixels and above) is suitable for the full-process synchronization of headlights such as development and debugging, user self-adjustment, and mass production lines, involving hardware self-calibration, one-key calibration, one-key hard synchronization, and one-key data automatic synchronization of GDC-PMF distortion correction.

[0179] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0180] Based on the same inventive concept, the embodiment of the present application also provides a lamp calibration device for implementing the lamp calibration method mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of the one or more lamp calibration device embodiments provided below can refer to the limitations of the lamp calibration method above, and will not be repeated here.

[0181] In an exemplary embodiment, Fig.14 As shown, a calibration device for a vehicle lamp is provided, comprising:

[0182] The display and photo module 1402 is used to determine the camera projection relationship; the camera projection relationship is used for coordinate conversion between the camera's plane coordinate system and the reference coordinate system;

[0183] The display and photo module 1402 is used to determine the plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between the plane coordinate system of the camera and the projection surface coordinate system of the headlight;

[0184] A planning module 1404 is used to adjust and calibrate the plane transformation relationship according to the camera projection relationship to obtain a headlight projection relationship;

[0185] The headlight projection relationship is used for coordinate conversion between the projection surface coordinate system of the headlight and the reference coordinate system.

[0186] In one embodiment, the display and photo module 1402 is used to:

[0187] Calibrate the extrinsic parameters of the camera based on the coordinates of the calibration points in the camera coordinate system and the coordinates of the calibration points in the reference coordinate system;

[0188] Calibrate the intrinsic parameters of the camera based on the coordinates of the calibration points and the depth values ​​of the calibration points in the plane coordinate system of the camera and the coordinates of the calibration points in the reference coordinate system;

[0189] A camera projection relationship is determined according to the external parameters of the camera and the internal parameters of the camera.

[0190] In one embodiment, the reference coordinate system is a reference system where the reference plane is located; the display and photo module 1402 is used to:

[0191] Acquiring a projection result of the vehicle light on the reference surface based on the camera, and obtaining a projection image of the vehicle light on the reference surface;

[0192] Determining projection parameters of the headlight based on the headlight projection image;

[0193] A plane conversion relationship between the camera and the headlight is obtained according to the projection parameters of the headlight.

[0194] In one embodiment, the projection result is obtained by projecting the vehicle light based on a preset pattern;

[0195] The display and photo taking module 1402 is used to:

[0196] Based on the projection image of the vehicle lamp, the projection parameters of the vehicle lamp are corrected until the corrected projection image of the vehicle lamp matches the preset pattern, thereby obtaining the corrected projection parameters of the vehicle lamp;

[0197] According to the corrected projection parameters of the headlight, the projection parameters of the headlight under the perspective of a pedestrian are determined; wherein the perspective of a pedestrian is a projection perspective in the reference coordinate system.

[0198] In one embodiment, the planning module 1404 is used to:

[0199] Determine the shape deviation ratio between the projected image of the vehicle lamp and the preset image;

[0200] Calibrate the inverse ratio of the negative correlation of the shape deviation ratio;

[0201] The reverse ratio is used to adjust the shape of the projection result, and the shape deviation ratio and the reverse ratio indicate opposite changes in the shape of the projection result at the same position.

[0202] In one embodiment, the planning module 1404 is used to:

[0203] After determining the projection parameters of each of the headlights, calibrating the shape alignment parameters between the headlights according to the corresponding relationship between the headlights;

[0204] The shape alignment parameters are used to perform shape alignment on the positions of the headlight projection images of the headlights.

[0205] In one embodiment, the planning module 1404 is used to:

[0206] After determining the projection parameters of each of the headlights, calibrating the grayscale weight of each pixel position in the overlapping area according to the brightness of the headlight projection image of each of the headlights;

[0207] The overlapping area is an intersection area of ​​the headlight projection images of the headlights, and the grayscale weight is used to fuse the brightness of each headlight in the overlapping area.

[0208] Each module in the above-mentioned vehicle lamp calibration device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0209] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.15As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for calibrating a headlight is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0210] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0211] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0212] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0213] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0214] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0215] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0216] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0217] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for calibrating a vehicle lamp, characterized in that: The method comprises: Calibrate the external parameters of the camera based on the coordinates of the calibration points in the camera coordinate system and the coordinates of the calibration points in the reference coordinate system; calibrate the internal parameters of the camera based on the coordinates of the calibration points in the plane coordinate system of the camera and the depth values ​​of the calibration points, as well as the coordinates of the calibration points in the reference coordinate system; determine the camera projection relationship according to the external parameters of the camera and the internal parameters of the camera; the camera projection relationship is used for coordinate conversion between the plane coordinate system of the camera and the reference coordinate system; the depth value of the calibration point is the image value of the coordinates of the feature point in the plane coordinate system; Determining a plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between a plane coordinate system of the camera and a projection surface coordinate system of the headlight; According to the camera projection relationship, the plane transformation relationship is adjusted and calibrated to obtain the headlight projection relationship; The headlight projection relationship is used for coordinate conversion between the projection surface coordinate system of the headlight and the reference coordinate system.

2. The method according to claim 1, characterized in that: The calibrating the intrinsic parameters of the camera based on the calibration point coordinates and the calibration point depth values ​​in the plane coordinate system of the camera and the calibration point coordinates in the reference coordinate system includes: The inverse of the depth value is multiplied by the external parameter of the camera, and then multiplied by the coordinates of the calibration point in the reference coordinate system to obtain an intermediate calculation result; The inverse matrix of the intermediate calculation result is multiplied by the coordinates of the calibration point in the plane coordinate system of the camera to obtain the intrinsic parameters of the camera; and the intrinsic parameters of the camera are recorded.

3. The method according to claim 1, characterized in that The reference coordinate system is the reference system where the reference surface is located; The determining of the plane transformation relationship of the vehicle lamp includes: Acquiring a projection result of the vehicle light on the reference surface based on the camera, and obtaining a projection image of the vehicle light on the reference surface; Determining projection parameters of the headlight based on the headlight projection image; A plane conversion relationship between the camera and the headlight is obtained according to the projection parameters of the headlight.

4. The method according to claim 3, characterized in that The projection result is obtained by projecting the vehicle light based on a preset pattern; The determining the projection parameters of the headlight based on the headlight projection image comprises: Based on the projection image of the vehicle lamp, the projection parameters of the vehicle lamp are corrected until the corrected projection image of the vehicle lamp matches the preset pattern, thereby obtaining the corrected projection parameters of the vehicle lamp; According to the corrected projection parameters of the headlight, the projection parameters of the headlight under the perspective of a pedestrian are determined; wherein the perspective of a pedestrian is a projection perspective in the reference coordinate system.

5. The method according to claim 4, characterized in that The correcting the projection parameters of the headlight based on the headlight projection image includes: Determine the shape deviation ratio between the projected image of the vehicle lamp and the preset image; Calibrate the inverse ratio of the negative correlation of the shape deviation ratio; The reverse ratio is used to adjust the shape of the projection result, and the shape deviation ratio and the reverse ratio indicate opposite changes in the shape of the projection result at the same position.

6. The method according to claim 1, characterized in that The method further comprises: After determining the projection parameters of each of the headlights, calibrating the shape alignment parameters between the headlights according to the corresponding relationship between the headlights; The shape alignment parameters are used to perform shape alignment on the positions of the headlight projection images of the headlights.

7. The method according to claim 1, characterized in that The method further comprises: After determining the projection parameters of each of the headlights, calibrating the grayscale weight of each pixel position in the overlapping area according to the brightness of the headlight projection image of each of the headlights; The overlapping area is an intersection area of ​​the headlight projection images of the headlights, and the grayscale weight is used to fuse the brightness of each headlight in the overlapping area.

8. A calibration device for a vehicle lamp, characterized in that: The device comprises: The display and photographing module is used to calibrate the external parameters of the camera based on the coordinates of the calibration points in the camera coordinate system and the coordinates of the calibration points in the reference coordinate system; calibrate the internal parameters of the camera based on the coordinates of the calibration points in the plane coordinate system of the camera and the depth values ​​of the calibration points, as well as the coordinates of the calibration points in the reference coordinate system; determine the camera projection relationship according to the external parameters of the camera and the internal parameters of the camera; the camera projection relationship is used for coordinate conversion between the plane coordinate system of the camera and the reference coordinate system; the depth value of the calibration point is the image value of the coordinates of the feature point in the plane coordinate system; The display and photo-taking module is used to determine the plane transformation relationship of the headlight; the plane transformation relationship is used for coordinate transformation between the plane coordinate system of the camera and the projection surface coordinate system of the headlight; A planning module, used to adjust and calibrate the plane transformation relationship according to the camera projection relationship to obtain a headlight projection relationship; The headlight projection relationship is used for coordinate conversion between the projection surface coordinate system of the headlight and the reference coordinate system.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images