Calibration Method and Related Device for In-vehicle Camera and Vehicle Lamp in ADB Scenario

Abstract

The present invention discloses a calibration method and related device for an in-vehicle camera and vehicle lights in an ADB scenario. By controlling a light zone of the vehicle lights to emit light towards a first calibration surface of a calibration carrier, part of the light passes through through-holes on the first calibration surface and forms first light spots on a second checkerboard on the second calibration surface of the calibration carrier; respectively obtain the first light spot position information and the first through-hole position information of each first light spot and each through-hole in the calibration coordinate system of the calibration carrier; according to the first light spot position information and the first through-hole position information, determine the first light center position information of the light center of the light zone in the calibration coordinate system; according to the first coordinate transformation matrix between the in-vehicle camera coordinate system of the in-vehicle camera and the calibration coordinate system of the calibration carrier and the first light center position information, determine the second light center position information of the light center in the in-vehicle camera coordinate system, so as to determine the target coordinate transformation matrix between the in-vehicle camera coordinate system and the vehicle light coordinate system of the vehicle lights, and to achieve precise control of the high beam lights.

Description

Technical Field

[0001] The present invention relates to the field of calibration technology, and in particular, to a calibration method and related device for a camera and vehicle lights in an ADB scenario. Background Art

[0002] Improper use of high beam headlights of vehicles can easily cause glare and blindness to other drivers, posing certain potential safety hazards. In recent years, with the development of automotive technology and machine vision technology, the application of the Adaptive Driving Beam (ADB) system has become increasingly widespread. In ADB, a sensing sensor is used to sense the position of the vehicle ahead and send the 3D information of relevant obstacles to the control module, which controls the LED light sources of the automotive matrix headlights, thereby avoiding phenomena such as glare and blindness to other road users.

[0003] During the control process of the high beam headlights of a vehicle, the positions of the vehicle's lighting system (i.e., vehicle lights) and the sensing system (i.e., camera) do not coincide. Therefore, it is necessary to accurately map the objects sensed by the sensing system to the minimum control unit (light beads or lighting zones) of the lighting system to form precise control of the lighting system, thereby effectively avoiding the phenomena of glare and blindness caused by the vehicle's high beam headlights to other road users.

[0004] Based on this, how to accurately map the objects sensed by the vehicle's sensing system to the minimum control unit of the lighting system to achieve precise control of the high beam headlights has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of the present invention is to provide a calibration method and related device for a camera and vehicle lights in an ADB scenario, aiming to solve the problem in the prior art that the objects sensed by the vehicle's sensing system cannot be accurately mapped to the minimum control unit of the lighting system to achieve precise control of the high beam headlights.

[0006] To achieve the above objective, the present invention also provides a calibration method for an in-vehicle camera and vehicle lights in an ADB scenario, the method comprising:

[0007] Controlling one lighting zone of the vehicle lights to emit light towards a first calibration surface of a calibration carrier, wherein part of the light passes through through-holes on the first calibration surface and forms a first light spot on a second checkerboard on a second calibration surface of the calibration carrier;

[0008] Respectively obtaining first light spot position information and first through-hole position information of each of the first light spots and each of the through-holes in the calibration coordinate system of the calibration carrier;

[0009] Determine the first optical center position information of the optical center of the lighting zone in the calibration coordinate system according to the first light spot position information and the first through-hole position information;

[0010] Obtain the first coordinate transformation matrix between the vehicle-mounted camera coordinate system of the vehicle-mounted camera and the calibration coordinate system of the calibration carrier;

[0011] Determine the second optical center position information of the optical center in the vehicle-mounted camera coordinate system according to the first coordinate transformation matrix and the first optical center position information;

[0012] Determine the target coordinate transformation matrix between the vehicle-mounted camera coordinate system and the headlight coordinate system of the headlight according to the second optical center position information;

[0013] Wherein, the target coordinate transformation matrix is used for the calibration of the vehicle-mounted camera and the headlight.

[0014] Optionally, the step of respectively obtaining the first light spot position information and the first through-hole position information of each of the first light spots and each of the through-holes in the calibration coordinate system of the calibration carrier specifically includes:

[0015] Obtain the second calibration plane image of the second calibration plane;

[0016] Determine the second light spot position information of each of the first light spots on the second calibration plane in the second checkerboard coordinate system according to the mapping relationship between the pixel points of the second calibration plane image and the second checkerboard coordinate system;

[0017] Determine the first light spot position information of each of the first light spots in the calibration coordinate system according to the second coordinate transformation matrix between the second checkerboard coordinate system and the calibration coordinate system and the second light spot position information of each of the first light spots.

[0018] Optionally, the step of respectively obtaining the first light spot position information and the first through-hole position information of each of the light spots and each of the through-holes in the calibration coordinate system of the calibration carrier specifically includes:

[0019] Obtain the first calibration plane image of the first calibration plane; wherein, the first calibration plane is provided with a first checkerboard; and

[0020] Determine the second through-hole position information of each of the through-holes in the first checkerboard coordinate system according to the mapping relationship between the pixel points of the first calibration plane image and the first checkerboard coordinate system of the first checkerboard;

[0021] Determine the first through-hole position information of each of the through-holes in the calibration coordinate system according to the third coordinate transformation matrix between the first checkerboard coordinate system and the calibration coordinate system and the second through-hole position information.

[0022] Optionally, according to the first light spot position information and the first through hole position information, determining the first light center position information of the light center of the lighting zone in the calibration coordinate system specifically includes:

[0023] Constructing a plurality of light center construction lines according to each of the through holes and the corresponding first light spots; wherein, the through holes and the first light spots are in one-to-one correspondence;

[0024] Solving for the point closest to the plurality of light center construction lines according to the first light spot position information and the first through hole position information, and taking it as the light center of the lighting zone, so as to obtain the first light center position information of the light center in the calibration coordinate system.

[0025] Optionally, the obtaining of the first coordinate transformation matrix between the vehicle-mounted camera coordinate system of the vehicle-mounted camera and the calibration coordinate system of the calibration carrier specifically includes:

[0026] Obtaining a fourth coordinate transformation matrix between the first checkerboard coordinate system of the first checkerboard disposed on the first calibration plane and the calibration coordinate system;

[0027] Determining a fifth coordinate transformation matrix between the first checkerboard coordinate system and the vehicle-mounted camera according to the camera internal parameters of the vehicle-mounted camera, the corner position information of each corner point of the first checkerboard in the first checkerboard coordinate system, and the pixel point position information of each corner point in the first calibration image;

[0028] Determining the first coordinate transformation matrix according to the fourth coordinate transformation matrix and the fifth coordinate transformation matrix.

[0029] Optionally, after determining the second light center position information of the light center in the vehicle-mounted camera coordinate system, the method further includes:

[0030] Obtaining a first calibration plane image of the first calibration plane; the first calibration plane is provided with a first checkerboard, and the first checkerboard forms a plurality of second light spots;

[0031] Determining the first light spot edge position information of each of the second light spots in the first checkerboard coordinate system;

[0032] Obtaining the fifth coordinate transformation matrix and each of the first light spot edge position information, and determining the second light spot edge position information of the second light spots in the vehicle-mounted camera coordinate system;

[0033] Wherein, the fifth coordinate transformation matrix is the coordinate transformation matrix between the first checkerboard coordinate system and the vehicle-mounted camera coordinate system;

[0034] Determine the headlight control angle of the lighting zone of the headlight according to the second light spot edge position information and the second light center position information of the light center.

[0035] Optionally, the method further includes:

[0036] The first light spot edge position information includes: the position information of the left edge, right edge, upper edge and lower edge of the second light spot.

[0037] Optionally, the obtaining of the second calibration plane image of the second calibration plane collected by the auxiliary camera specifically includes:

[0038] Obtain the original second calibration plane image of the second calibration plane collected by the auxiliary camera;

[0039] Obtain the four corner points of the outermost corners of the second checkerboard through the checkerboard corner point algorithm;

[0040] Perform image correction on the original second calibration plane image according to the corner points to obtain the second calibration plane image.

[0041] To achieve the above object, the present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the calibration method of the in-vehicle camera and the headlight in the ADB scenario as described above.

[0042] To achieve the above object, the present invention also provides a terminal, which includes: a processor and a memory; a computer-readable program executable by the processor is stored on the memory; when the processor executes the computer-readable program, the steps in the calibration method of the in-vehicle camera and the headlight in the ADB scenario as described above are implemented.

[0043] The present invention controls a lighting zone of the headlight to emit light towards the first calibration plane of the calibration carrier. Some of the light passes through the through holes on the first calibration plane, forming a first light spot on the second checkerboard of the second calibration plane of the calibration carrier and a second light spot on the first calibration plane. According to the first light spot position information and the first through hole position information of each first light spot and each through hole in the calibration coordinate system of the calibration carrier, determine the first light center position information of the light center of the lighting zone in the calibration coordinate system, and through the first coordinate transformation matrix, determine the second light center position information of the light center in the in-vehicle camera coordinate system, so as to determine the target coordinate transformation matrix between the in-vehicle camera coordinate system and the headlight coordinate system based on the second light center position information. Thus, according to the target coordinate transformation matrix, the objects sensed by the perception system of the vehicle can be accurately mapped to the minimum control unit of the lighting system, realizing precise control of the high beam. Description of the Drawings

[0044] Figure 1 This is an application scenario diagram of the calibration method for in-vehicle cameras and vehicle lights in the ADB scenario provided by the embodiments of the present invention;

[0045] Figure 2 This is a schematic structural diagram of the calibration system for in-vehicle cameras and vehicle lights in the ADB scenario provided by the embodiments of the present invention;

[0046] Figure 3 This is a flowchart of the calibration method for in-vehicle cameras and vehicle lights in the ADB scenario provided by the embodiments of the present invention;

[0047] Figure 4 This is a flowchart of step S302 provided by the embodiments of the present invention;

[0048] Figure 5 This is a schematic diagram of the original image of the second calibration surface provided by the embodiments of the present invention;

[0049] Figure 6 This is a schematic diagram of the image of the second calibration surface provided by the embodiments of the present invention;

[0050] Figure 7 This is a flowchart of step S303 provided by the embodiments of the present invention;

[0051] Figure 8 This is a flowchart of step S304 provided by the embodiments of the present invention;

[0052] Figure 9 This is another flowchart of the calibration method for in-vehicle cameras and vehicle lights in the ADB scenario provided by the embodiments of the present invention;

[0053] Figure 10 This is a schematic structural diagram of the terminal provided by the embodiments of the present invention. Detailed implementation manners

[0054] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] As Figure 1 shown, during the control process of the high beam of the vehicle, the positions of the vehicle's lighting system (i.e., vehicle lights) and the perception system (i.e., in-vehicle camera) do not coincide. Therefore, through a corresponding transformation matrix, the obstacles perceived by the perception system can be accurately mapped to the minimum control unit of the lighting system (i.e., lighting zones or lamp beads), so as to achieve precise control of the lighting system. Therefore, how to accurately map the objects perceived by the vehicle's perception system to the minimum control unit of the lighting system has become a technical problem to be urgently solved.

[0056] Based on this, the present invention provides a calibration method for an in-vehicle camera and vehicle lights in an ADB scenario. This method is applied to a calibration system for an in-vehicle camera and vehicle lights in an ADB scenario, as Figure 2 shown. The calibration system includes: an in-vehicle camera, vehicle lights, a calibration carrier, and an auxiliary camera. As Figure 2 shown, the calibration carrier can be a cube box (and the calibration carrier is made of an opaque material), which includes: a first calibration surface and a second calibration surface. A plurality of through holes and a first checkerboard are provided on the first calibration surface, and a second checkerboard 1 is provided on the second calibration surface.

[0057] Among them, the vehicle lights are composed of several lighting zones, and each lighting zone includes at least one lamp bead. The vehicle lights and the in-vehicle camera are both placed horizontally. It is possible to determine whether the vehicle lights are placed horizontally by the light spot of the vehicle lights on the vertical wall surface, and at the same time, it is also possible to determine whether the in-vehicle camera is placed horizontally by the imaging of the in-vehicle camera.

[0058] In an embodiment of the present invention, as Figure 2 shown, the first checkerboard can only occupy a part of the first calibration surface instead of covering the entire first calibration surface; while the second checkerboard can cover the entire second calibration surface instead of only occupying a part of the second calibration surface. This is to ensure that the light passing through the through holes can form corresponding light spots on the second checkerboard.

[0059] As Figure 3 shown, the calibration method for an in-vehicle camera and vehicle lights in an ADB scenario provided by an embodiment of the present invention includes at least the following steps:

[0060] S301, control one lighting zone of the vehicle lights to emit light towards the first calibration surface of the calibration carrier, and part of the light passes through the through holes on the first calibration surface and forms a first light spot on the second checkerboard of the second calibration surface of the calibration carrier.

[0061] As Figure 2 shown, take one lighting zone (including at least one lamp bead) of the vehicle lights as a light source, control one lighting zone (including at least one lamp bead) of the vehicle lights to emit light, and all the emitted light irradiates the first calibration surface. Part of the light emitted by the lighting zone passes through the through holes on the first calibration surface and irradiates the second calibration surface, forming a first light spot on the second checkerboard of the second calibration surface; another part of the light irradiates the first calibration surface and forms a second light spot on the first checkerboard of the first calibration surface.

[0062] It can be understood that the vehicle lights can be partitioned first to obtain several lighting zones. If a lighting zone includes multiple lamp beads, the optical center of the lighting zone can be the center point of the lighting zone.

[0063] In S302, obtain the first spot position information and the first through-hole position information of each first spot and each through-hole in the calibration coordinate system of the calibration carrier, respectively.

[0064] Among them, the first spot position information may be the position information of the center of the spot of the first spot, which is represented by coordinates in the coordinate system. Similarly, the first through-hole position information may also be the position information of the center of the through-hole, which is represented by coordinates in the coordinate system.

[0065] In the embodiment of the present invention, the calibration coordinate system of the calibration carrier is a 3D coordinate system. The upper left corner vertex of the first calibration surface of the calibration carrier can be used as the origin, with the right direction as the positive x-axis, the downward direction as the positive y-axis, and the direction from the first calibration surface to the second calibration surface as the positive z-axis.

[0066] As Figure 4 shown, step S302 may at least include the following steps:

[0067] S401, obtain the second calibration surface image of the second calibration surface.

[0068] As Figure 2 shown, the second calibration surface can be imaged by an auxiliary camera to obtain the original second calibration surface image (as Figure 5 shown). The original second calibration surface image includes a second checkerboard and a first spot. At the same time, in order to ensure the accuracy of the calibration of the vehicle headlamp and the vehicle-mounted camera, it is necessary to correct the original second calibration surface image collected by the auxiliary camera to obtain the second calibration surface image of the second calibration surface (as Figure 6 shown).

[0069] Specifically, first obtain the original second calibration surface image taken by the auxiliary camera for the second calibration surface; then, through the checkerboard corner point algorithm, obtain the four corner points of the outermost corners of the second checkerboard as the corner points; and correct the original second calibration surface image according to the corner points of the second checkerboard to obtain the second calibration surface image.

[0070] As Figure 5 shown, through the checkerboard corner point algorithm, detect the four outermost corner points of the second checkerboard in the original second calibration surface image, which are A1, B1, C1, and D1, respectively, and are the corner points. Among them, the four corner points A1, B1, C1, and D1 are connected in sequence to form a matrix. Then, determine the homography matrix H for image correction through the four corner points A1, B1, C1, and D1, and use the homography matrix H to correct the original second calibration surface image to obtain the corrected second calibration surface image. As Figure 6 shown, the four outermost corner points (i.e., the corner points) of the second checkerboard in the second calibration surface image are A2, B2, C2, and D2, respectively.

[0071] S402. Determine the second spot position information of each first spot on the second calibration plane in the second checkerboard coordinate system according to the mapping relationship between the pixel points of the second calibration plane image and the second checkerboard coordinate system.

[0072] First, a second checkerboard coordinate system can be established in advance. As Figure 6 shown, taking the corner point A2 in the upper left corner as the origin, establish the second checkerboard coordinate system, with the right direction as the positive x-axis, the downward direction as the positive y-axis, and the direction into the paper as the positive z-axis (not shown in the figure).

[0073] Then, establish the mapping relationship between the pixel points of the second calibration plane image and the second checkerboard coordinate system:

[0074] X = (u - u_l) * L_x / (W - 2 * u_l);

[0075] Y = (v - v_l) * L_y / (H - 2 * v_l);

[0076] Z = 0.

[0077] Where, W and H are the width and height of the second calibration plane image; u_l and v_l are the pixel coordinates of the corner point A2 on the second calibration plane image; L_x is the actual distance between points A2 and B2; L_x is the actual distance between points A2 and D2; u and v are the pixel coordinates of the position to be determined in the second calibration plane image; X and Y are the position coordinates of the position to be determined in the second checkerboard coordinate system.

[0078] In the embodiment of the present invention, the first spots on the second checkerboard of the second calibration plane can be recognized by an existing image recognition algorithm, and the pixel coordinates of each first spot on the second calibration plane image can be determined. Then, according to the mapping relationship between the second calibration plane image and the second checkerboard coordinate system, the second spot position information of each first spot on the second calibration plane in the second checkerboard coordinate system can be determined.

[0079] S403. Determine the first spot position information of each first spot in the calibration coordinate system according to the second coordinate transformation matrix between the second checkerboard coordinate system and the calibration coordinate system, and the second spot position information of each first spot.

[0080] In the embodiment of the present invention, since the second checkerboard is attached to the second calibration plane of the calibration carrier, after determining the calibration coordinate system of the calibration carrier, the second coordinate transformation matrix between the second checkerboard coordinate system and the calibration coordinate system can be obtained.

[0081] On the basis that the second spot position information of each first spot in the second checkerboard coordinate system has been determined through step S402, according to the first coordinate transformation matrix between the second checkerboard coordinate system and the calibration coordinate system, the first spot position information of each first spot in the calibration coordinate system can be determined.

[0082] S404. Obtain the first calibration surface image of the first calibration surface collected by the vehicle-mounted camera.

[0083] Wherein, a first checkerboard is arranged on the first calibration surface, and a second light spot is formed on the first checkerboard.

[0084] Specifically, the vehicle-mounted camera takes an image of the first calibration surface to obtain the original image of the first calibration surface of the first calibration surface, and then corrects the original image of the first calibration surface to obtain the first calibration surface image. In the embodiment of the present invention, the image correction of the original image of the first calibration surface can be implemented by the same or similar method as the image correction of the original image of the second calibration surface, which will not be elaborated herein.

[0085] S405. Determine the second through-hole position information of each through-hole in the first calibration surface coordinate system according to the mapping relationship between the pixel points of the first calibration surface image and the coordinate system of the first checkerboard of the first calibration surface.

[0086] In the embodiment of the present invention, the coordinate system of the first checkerboard of the first checkerboard can be constructed first. For example, taking the upper left corner of the first checkerboard as the origin, the right direction as the positive x, the downward direction as the positive y, and the direction from the first calibration surface to the second calibration surface as the positive z. Then, determine the mapping relationship between the pixels of the first calibration surface image and the coordinate system of the first checkerboard, so as to determine the second through-hole position information of each through-hole on the first calibration surface.

[0087] S406. Determine the first through-hole position information of each through-hole in the calibration coordinate system according to the third coordinate transformation matrix between the first checkerboard coordinate system and the calibration coordinate system and the second through-hole position information.

[0088] In the embodiment of the present invention, the third coordinate transformation matrix between the first checkerboard coordinate system and the calibration coordinate system can be constructed first, and then, through the third coordinate transformation matrix, according to the second through-hole position information of each through-hole, determine the first through-hole position information of each through-hole in the calibration coordinate system.

[0089] It should be noted that steps SX201 - SX203 can be executed first, and then steps SX204 - SX206; or steps SX204 - SX206 can be executed first, and then steps SX201 - SX203; or steps SX201 - SX203 and steps SX204 - SX206 can be executed simultaneously, which is not specifically limited in the embodiment of the present invention.

[0090] S303. Determine the first optical center position information of the optical center of the lighting zone in the calibration coordinate system according to the first light spot position information and the first through-hole position information.

[0091] Such as Figure 7As shown, step S303 may at least include the following steps:

[0092] S701, construct a plurality of optical center construction lines according to each through hole and the corresponding first light spot.

[0093] Since the through holes and the first light spots are in one-to-one correspondence, each through hole has its uniquely corresponding first light spot, forming a through hole - first light spot pair. Through each through hole - first light spot pair, a plurality of optical center construction lines can be constructed.

[0094] S702, solve for the point closest to the plurality of optical center construction lines based on the first light spot position information and the first through hole position information, and use it as the optical center of the lighting partition to obtain the first optical center position information of the optical center in the calibration coordinate system.

[0095] In the embodiment of the present invention, based on the first light spot position information and the first through hole position information, the point closest to the plurality of optical center construction lines can be solved, and this point is used as the optical center of the lighting partition, so as to obtain the optical center position information of the optical center in the calibration coordinate system, that is, the first optical center position information.

[0096] It can be understood that the plurality of optical center construction lines can be extended to determine the intersection points between the optical center construction lines, and the intersection points are used as the optical centers of the lighting partition. However, due to actual errors, the optical centers determined by the intersection point method have certain errors, resulting in inaccurate calibration of the on - vehicle camera and the vehicle lamp. In the embodiment of the present invention, using the point closest to the plurality of optical center construction lines as the optical center of the lighting partition can further improve the calibration accuracy of the on - vehicle camera and the vehicle lamp.

[0097] S304, obtain the first coordinate transformation matrix between the on - vehicle camera coordinate system of the on - vehicle camera and the calibration coordinate system of the calibration carrier.

[0098] As Figure 8 shown, step S304 may be implemented at least through the following steps:

[0099] S801, obtain the fourth coordinate transformation matrix between the first checkerboard coordinate system of the first checkerboard disposed on the first calibration surface and the calibration coordinate system.

[0100] S802, determine the fifth coordinate transformation matrix between the first checkerboard coordinate system and the on - vehicle camera coordinate system of the on - vehicle camera according to the camera internal parameters of the on - vehicle camera, the corner point position information of each corner point of the first checkerboard in the first checkerboard coordinate system, and the pixel point position information of each corner point in the first calibration image.

[0101] In an embodiment of the present invention, first, by performing a checkerboard corner point algorithm on the first calibration plane image, the four corner points at the outermost corners of the first checkerboard can be identified as the corner points of the first checkerboard, so as to determine the corner point position information of the corner points of the first checkerboard in the first checkerboard coordinate system. Then, according to the camera internal parameters of the vehicle-mounted camera, the corner point position information in the first checkerboard coordinate system, and the pixel point position information of each corner point in the first calibration image, the fifth coordinate transformation matrix between the first checkerboard coordinate system and the vehicle-mounted camera coordinate system of the vehicle-mounted camera is determined:

[0102] p = K * Tchess_cam * Pchess;

[0103] where K is the camera internal parameter of the vehicle-mounted camera; Pchess is the corner point position information of the corner points of the first checkerboard in the first checkerboard coordinate system; Tcam_chess is the fifth coordinate transformation matrix from the first checkerboard coordinate system to the vehicle-mounted camera coordinate system; and p is the pixel point position information of each corner point in the first calibration image.

[0104] S803. Determine the first coordinate transformation matrix according to the fourth coordinate transformation matrix and the fifth coordinate transformation matrix.

[0105] Specifically, the first coordinate transformation matrix is:

[0106] Tcal_cam = Tcal_chess * Tchess_cam;

[0107] where Tcal_cam is the first coordinate transformation matrix; Tcal_chess is the fourth coordinate transformation matrix; and Tchess_cam is the fifth coordinate transformation matrix.

[0108] In an embodiment of the present invention, the first coordinate transformation matrix is the coordinate transformation matrix between the calibration coordinate system and the vehicle-mounted camera coordinate system.

[0109] S305. Determine the second optical center position information of the optical center in the vehicle-mounted camera coordinate system according to the first coordinate transformation matrix and the first optical center position information.

[0110] S306. Determine the target coordinate transformation matrix between the vehicle-mounted camera coordinate system and the headlight coordinate system of the headlight according to the second optical center position information.

[0111] Specifically, first, the third optical center position information of the optical center in the headlight coordinate system can be determined according to the lighting zone, and then the target coordinate transformation matrix between the vehicle-mounted camera coordinate system and the headlight coordinate system of the headlight can be determined according to the second optical center position information of the optical center in the vehicle-mounted camera coordinate system and the third optical center position information in the headlight coordinate system.

[0112] In the embodiments of the present invention, the accurate calibration of the in-vehicle camera and the vehicle lamp in the ADB scenario can be achieved through the target coordinate transformation matrix between the in-vehicle camera coordinate system and the vehicle lamp coordinate system of the vehicle lamp, so as to avoid the occurrence of phenomena such as high beam glare.

[0113] In the embodiments of the present invention, after determining the second light center position information of the light center in the in-vehicle camera coordinate system, the vehicle lamp control angle of the light zone can also be determined, such as Figure 9 shown, and it can be achieved at least through the following steps:

[0114] S901, obtain the first calibration plane image of the first calibration plane.

[0115] Among them, a plurality of second light spots are formed on the first checkerboard.

[0116] S902, determine the first light spot edge position information of each second light spot in the first calibration plane image in the first checkerboard coordinate system.

[0117] Among them, the first light spot edge position information includes: the position information of the left edge, right edge, upper edge and lower edge of the second light spot, which are respectively: spot_in_chess_left, spot_in_chess_righ, spot_in_chess_top, spot_in_chess_bot.

[0118] S903, according to the fourth coordinate transformation matrix and each first light spot edge position information, determine the second light spot edge position information of the second light spot in the in-vehicle camera coordinate system.

[0119] In the embodiments of the present invention, according to the fourth coordinate transformation matrix Tchess_cam, the first light spot edge position information of the second light spot in the first checkerboard coordinate system is transformed into the in-vehicle camera coordinate system to obtain the second light spot edge position information of the second light spot in the in-vehicle camera coordinate system, which includes: the position information of the left edge, right edge, upper edge and lower edge of the second light spot, which are respectively spot_in_camera_left, spot_in_camera_top, spot_in_camera_right, spot_in_camera_bot.

[0120] S904, according to the second light spot edge position information and the second light center position information of the light center, determine the vehicle lamp control angle of the light zone of the vehicle lamp.

[0121] Specifically, as follows:

[0122] angle_left = atan((spot_in_camera_left.x - light_point_in_camera.x) / (spot_in_camera_left.z - light_point_in_camera.z));

[0123] angle_top = atan((spot_in_camera_top.y - light_point_in_camera.y) / (spot_in_camera_top.z - light_point_in_camera.z));

[0124] angle_right = atan((spot_in_camera_right.x - light_point_in_camera.x) / (spot_in_camera_right.z - light_point_in_camera.z));

[0125] angle_bot = atan((spot_in_camera_bot.y - light_point_in_camera.y) / (spot_in_camera_bot.z - light_point_in_camera.z));

[0126] Wherein, x represents the x coordinate, y represents the y coordinate, and z represents the z coordinate.

[0127] Through the above solution, the headlight control angles of each light zone of the headlight can be determined to achieve accurate control of the headlight.

[0128] The calibration method of the in-vehicle camera and the headlight in the ADB scenario provided by the embodiment of the present invention controls a light zone of the headlight to emit light towards the first calibration surface of the calibration carrier. Part of the light passes through the through holes on the first calibration surface, forming a first light spot on the second checkerboard of the second calibration surface of the calibration carrier and a second light spot on the first calibration surface. Based on the first light spot position information and the first through hole position information of each first light spot and each through hole in the calibration coordinate system of the calibration carrier, the first light center position information of the light center of the light zone in the calibration coordinate system is determined, and through the first coordinate transformation matrix, the second light center position information of the light center in the in-vehicle camera coordinate system is determined. Thus, based on the second light center position information, the target coordinate transformation matrix between the in-vehicle camera coordinate system and the headlight coordinate system is determined, realizing accurate calibration of the in-vehicle camera and the headlight. Therefore, according to the target coordinate transformation matrix, the objects sensed by the perception system of the vehicle can be accurately mapped to the minimum control unit of the lighting system, achieving precise control of the high beam.

[0129] Based on the calibration method of in-vehicle cameras and vehicle lights in the above ADB scenario, the present invention also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the calibration method of in-vehicle cameras and vehicle lights in the above ADB scenario described in the embodiments.

[0130] Based on the above training method of the image processing model and the image processing method, the present invention also provides a terminal, as Figure 10 shown, which includes at least one processor 30; a display screen 31; and a memory 32, and may further include a communication interface 33 and a bus 34. Among them, the processor 30, the display screen 31, the memory 32, and the communication interface 33 can complete mutual communication through the bus 34. The display screen 31 is set to display a user guidance interface preset in the initial setting mode. The communication interface 33 can transmit information. The processor 30 can call the logical instructions in the memory 32 to execute the steps in the calibration method of in-vehicle cameras and vehicle lights in the above ADB scenario described in the embodiments.

[0131] In addition, when the logical instructions in the above memory 32 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0132] The memory 32, as a computer-readable storage medium, can be set to store software programs and computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 30 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 32, that is, implements the methods in the above embodiments.

[0133] The memory 32 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 32 can include a high-speed random access memory and can also include a non-volatile memory. For example, various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs can also be transient storage media.

[0134] Each embodiment in this application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the terminal and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0135] The terminal and medium provided in the embodiments of this application correspond one by one to the method. Therefore, the terminal and medium also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the terminal and medium will not be elaborated here.

[0136] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including that element.

[0137] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disc, etc.

[0138] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A calibration method for in-vehicle cameras and vehicle lights in an ADB scenario, characterized in that The method includes: Controlling a light-emitting area of the vehicle lamp to emit light towards a first calibration surface of a calibration carrier, wherein part of the light passes through a through hole on the first calibration surface and forms a first light spot on a second checkerboard of a second calibration surface of the calibration carrier; Respectively obtaining first light spot position information and first through hole position information of each of the first light spots and each of the through holes in a calibration coordinate system of the calibration carrier; Determining first light center position information of the light center of the light-emitting area in the calibration coordinate system according to the first light spot position information and the first through hole position information; Obtaining a first coordinate transformation matrix between a vehicle-mounted camera coordinate system of the vehicle-mounted camera and the calibration coordinate system of the calibration carrier; Determining second light center position information of the light center in the vehicle-mounted camera coordinate system according to the first coordinate transformation matrix and the first light center position information; Determining a target coordinate transformation matrix between the vehicle-mounted camera coordinate system and a vehicle lamp coordinate system of the vehicle lamp according to the second light center position information; Wherein, the target coordinate transformation matrix is used for calibration between the vehicle-mounted camera and the vehicle lamp.

2. The method according to claim 1, characterized in that, The step of respectively obtaining first light spot position information and first through hole position information of each of the first light spots and each of the through holes in the calibration coordinate system of the calibration carrier specifically includes: Obtaining a second calibration surface image of the second calibration surface collected by an auxiliary camera; Determining second light spot position information of each of the first light spots on the second calibration surface in a second checkerboard coordinate system according to a mapping relationship between pixel points of the second calibration surface image and a second checkerboard coordinate system of the second checkerboard; Determining first light spot position information of each of the first light spots in the calibration coordinate system according to a second coordinate transformation matrix between the second checkerboard coordinate system and the calibration coordinate system and the second light spot position information of each of the first light spots.

3. The method according to claim 1, wherein The step of respectively obtaining first light spot position information and first through hole position information of each of the light spots and each of the through holes in the calibration coordinate system of the calibration carrier specifically includes: Obtaining a first calibration surface image of the first calibration surface; wherein, a first checkerboard is arranged on the first calibration surface; and Determining second through hole position information of each of the through holes in a first checkerboard coordinate system according to a mapping relationship between pixel points of the first calibration surface image and a first checkerboard coordinate system of the first checkerboard; Determining first through hole position information of each of the through holes in the calibration coordinate system according to a third coordinate transformation matrix between the first checkerboard coordinate system and the calibration coordinate system and the second through hole position information.

4. The method according to claim 1, characterized in that The step of determining first light center position information of the light center of the light-emitting area in the calibration coordinate system according to the first light spot position information and the first through hole position information specifically includes: Constructing a plurality of light center construction lines according to each of the through holes and the corresponding first light spots; wherein, the through holes and the first light spots are in one-to-one correspondence; According to the first light spot position information and the first through hole position information, solve for the point closest to the straight line constructed by the plurality of optical centers as the optical center of the lighting zone, so as to obtain the first optical center position information of the optical center in the calibration coordinate system.

5. The method according to claim 3, characterized in that The obtaining of the first coordinate transformation matrix between the vehicle-mounted camera coordinate system of the vehicle-mounted camera and the calibration coordinate system of the calibration carrier specifically includes: Obtaining a fourth coordinate transformation matrix between the first checkerboard coordinate system of the first checkerboard disposed on the first calibration plane and the calibration coordinate system; Determining a fifth coordinate transformation matrix between the first checkerboard coordinate system and the vehicle-mounted camera according to the camera internal parameters of the vehicle-mounted camera, the corner position information of each corner of the first checkerboard in the first checkerboard coordinate system, and the pixel position information of each corner in the first calibration plane image; Determining the first coordinate transformation matrix according to the fourth coordinate transformation matrix and the fifth coordinate transformation matrix.

6. The method according to claim 3, wherein After determining the second optical center position information of the optical center in the vehicle-mounted camera coordinate system, the method further includes: Obtaining a first calibration plane image of the first calibration plane; the first calibration plane is provided with a first checkerboard, and the first checkerboard forms a plurality of second light spots; Determining the first light spot edge position information of each of the second light spots in the first checkerboard coordinate system; Obtaining the fifth coordinate transformation matrix and the first light spot edge position information of each of the second light spots, and determining the second light spot edge position information of the second light spots in the vehicle-mounted camera coordinate system; Wherein, the fifth coordinate transformation matrix is a coordinate transformation matrix between the first checkerboard coordinate system and the vehicle-mounted camera coordinate system; Determining the headlight control angle of the lighting zone of the headlight according to the second light spot edge position information and the second optical center position information of the optical center.

7. The method according to claim 6, characterized in that, The method further includes: The first light spot edge position information includes: the position information of the left edge, right edge, upper edge and lower edge of the second light spot.

8. The method according to claim 2, wherein The obtaining of the second calibration plane image of the second calibration plane collected by the auxiliary camera specifically includes: Obtaining a second calibration plane original image of the second calibration plane collected by the auxiliary camera; Obtaining the four corner points at the outermost corners of the second checkerboard through the checkerboard corner point algorithm; Performing image correction on the second calibration plane original image according to the corner points to obtain the second calibration plane image.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the calibration method of the vehicle-mounted camera and the headlight in the ADB scenario according to any one of claims 1-8.

10. A terminal, characterized in that, The terminal includes: a processor and a memory; the memory stores a computer-readable program executable by the processor; when the processor executes the computer-readable program, the steps in the calibration method of the vehicle-mounted camera and the headlight in the ADB scenario according to any one of claims 1-8 are implemented.

Images