Method and device for collaborative projection of dual intelligent vehicle lights

Abstract

The application discloses a kind of method and device of the coordination projection of double intelligent car lights, the method includes: establishing world coordinate system, left car light coordinate system and projection picture coordinate system, right car light coordinate system and projection picture coordinate system;Establish one equivalent car light coordinate system of left car light coordinate system and right car light coordinate system;When the world coordinate of car light projection is known, the conversion relationship of pixel coordinate in equivalent car light coordinate system and world coordinate system, the conversion relationship of pixel coordinate on the projection image of left car light and world coordinate system, and the conversion relationship of pixel coordinate on the projection image of right car light and world coordinate system are simultaneously solved, the pixel coordinate on the projection image of left car light and the pixel coordinate on the projection image of right car light are calculated to obtain.This application, by simultaneously solving the conversion relationship of pixel coordinate on the projection image of equivalent car light, left car light and right car light and world coordinate system, makes the intelligent car light of left and right sides can align pixel, and projects target pattern in coordination.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent car lights, in particular to a method and device for cooperative projection of double intelligent car lights. BACKGROUND

[0002] With the pace of entering the era of automatic driving of automobiles becoming faster and faster, various parts of automobiles are developing towards intelligence and networking. An auxiliary driving scheme based on intelligent car lights can become a necessary configuration for information interaction between vehicle-road, person-vehicle and vehicle-vehicle. The intelligent car light technology combining laser projection and car lights will become an important intelligent component necessary for future vehicles. The intelligent car light can project different interactive contents through laser projection according to requirements, and convey information signs that the driver wants to express, such as preparation for lane changing, courtesy to pedestrians, etc. The intelligent car light can also display the current motion state of the vehicle, blind area, weather and other information that is helpful for safe driving, which is conducive to playing an important auxiliary driving function in low visibility, curves, intersections or state changing and other special vehicle condition scenes, greatly reducing the number of accidents in complex traffic situations and improving traffic efficiency.

[0003] At present, the existing technology is basically a single projectable car light, which projects the input image by processing the input projection image and controlling the light source matrix on the car light together with the optical module. However, this technology usually only considers the entire technical process from the image to the projection on the car light, and cannot realize the cooperation of double car lights from the perception input. In actual application, ghosting, incomplete patterns or pattern splicing errors may occur, causing information loss or errors, and even accidents may occur. SUMMARY

[0004] The purpose of the present application is to provide a method and device for cooperative projection of double intelligent car lights, so as to control the left and right intelligent car lights to cooperate with each other, accurately project the same input picture to the preset position, and obtain the desired pattern.

[0005] To achieve the above purpose, the present application provides a method for cooperative projection of double intelligent car lights, comprising:

[0006] Step one, establish a world coordinate system (X W ,Y W ,Z W ), a left car light coordinate system (X l ,Y l ,Z l ) and a projection picture coordinate system (x l ,y l ), a right car light coordinate system (X r ,Y r ,Z r ) and a projection picture coordinate system (xr ,y r );

[0007] Among them, the left headlight coordinate system (X l ,Y l Z l The pixel coordinates (u) on the projected image l ,v l The transformation relationship between the coordinate system and the world coordinate system is as follows:

[0008] [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0009] Where, matrix Q l Represents the pixel coordinates (u) of the projected image l ,v l ) to the left headlight coordinate system (X l ,Y l Z l The transformation matrix of ), matrix P l Represents the coordinate system from the left headlight (X) l ,Y l Z l The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P l Sum matrix Q l The value is calculated by pre-calibrating the coordinates of the left headlight, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0010] Right headlight coordinate system (X) r ,Y r Z r The pixel coordinates (u) on the projected image r ,v r The transformation relationship between the coordinate system and the world coordinate system is as follows:

[0011] [u r v r 1] T =P r - 1Q r-1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0012] wherein matrix Q r represents a transformation matrix of pixel coordinates (u r ,v r ) of the projection picture to the right car lamp coordinate system (X r ,Y r ,Z r ), matrix P r represents a transformation matrix from the right car lamp coordinate system (X r ,Y r ,Z r ) to the world coordinate system, and T represents matrix transposition; values of matrix P r and matrix Q r are calculated by pre-calibrating the right car lamp coordinate, pixel coordinates of the projection picture of the right car lamp and ground projection coordinates;

[0013] Step two, an equivalent car lamp coordinate system (X i ,Y i ,Z i ) of the left car lamp coordinate system and the right car lamp coordinate system is established, pixel coordinates on the projection picture of the equivalent car lamp are (u,v), and a conversion relationship between the pixel coordinates and the world coordinate system is:

[0014]

[0015] wherein G - 1 l is an inverse matrix of matrix Q l ,

[0016] R = R r R l -1 , T i = (T l + T r ) / 2,

[0017] wherein R l and T l are respectively a rotation matrix and a translation vector of the left car lamp, and R r and T r are respectively a rotation matrix and a translation vector of the right car lamp;

[0018] Step three, when the world coordinates of the known car light projection are known, the pixel coordinates of the equivalent car light projection image are calculated as (u, v) according to formula (300), and the pixel coordinates of the left car light projection image (u l ,v l ) and the pixel coordinates of the right car light projection image (u r ,v r ) are calculated according to formula (100) and (200).

[0019] Preferably, the matrix Q has the form:

[0020]

[0021] The matrix P has the form:

[0022]

[0023] The matrix M has the form:

[0024]

[0025] Preferably, step three includes:

[0026] According to formula (300) and (100), we get:

[0027]

[0028] According to formula (300) and (200), we get:

[0029]

[0030] wherein the parameters m 00 i to the parameters m 23 i , the parameters m 00 l to the parameters m 23 l , the parameters m 00 r to the parameters m 23 r are known quantities, and when the world coordinates (x w ,y w ,z w ) of the car light projection are known, the pixel coordinates of the left car light projection image (u l ,v l ) and the pixel coordinates of the right car light projection image (u r ,v r ) are calculated according to the above formula.

[0031] Preferably, the x-axis and y-axis in the left and right car light coordinate systems are the same as the x-axis and y-axis of the projection picture coordinate system, and the z-axis in the world coordinate system is the vertical direction.

[0032] The application also provides a device for coordinated projection of a double intelligent car light, comprising:

[0033] a coordinate system processing unit for establishing a world coordinate system (X W ,Y W ,Z W ), a left car light coordinate system (X l ,Y l ,Z l ), and a projection picture coordinate system (x l ,y l ), a right car light coordinate system (X r ,Y r ,Z r ) and a projection picture coordinate system (x r ,y r );

[0034] wherein the conversion relationship between the pixel coordinates (u l ,v l ) on the projection picture of the left car light coordinate system (X l ,Y l ,Z l ) and the world coordinate system is:

[0035] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0036] wherein the matrix Q l represents the transformation matrix of the pixel coordinates (u l ,v l ) on the projection picture to the left car light coordinate system (X l ,Y l ,Z l ), the matrix P l represents the transformation matrix from the left car light coordinate system (X l ,Y l ,Zl ) to the world coordinate system, T represents matrix transposition; matrix P l and matrix Q l are values calculated by pre-calibrating the left headlamp coordinate, the pixel coordinate of the projection picture of the left headlamp and the ground projection coordinate;

[0037] The conversion relationship between the pixel coordinate (u r ,v r ) on the projection picture of the right headlamp coordinate system (X r ,Y r ,Z r ) and the world coordinate system is:

[0038] [u r v r 1] T =P r - 1Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0039] Wherein, matrix Q r represents the conversion matrix of the pixel coordinate (u r ,v r ) of the projection picture to the right headlamp coordinate system (X r ,Y r ,Z r ), matrix P r represents the conversion matrix from the right headlamp coordinate system (X r ,Y r ,Z r ) to the world coordinate system, T represents matrix transposition; matrix P r and matrix Q r are values calculated by pre-calibrating the right headlamp coordinate, the pixel coordinate of the projection picture of the right headlamp and the ground projection coordinate;

[0040] The equivalent headlamp processing unit is used to establish an equivalent headlamp coordinate system (X i ,Y i ,Z i ) of the left headlamp coordinate system and the right headlamp coordinate system, the pixel coordinate on the projection picture of the equivalent headlamp is (u,v), and the conversion relationship between the pixel coordinate and the world coordinate system is:

[0041]

[0042] wherein G - 1 l is the inverse matrix of matrix Q l ,

[0043] R = R r R l -1 , T i = (T l + T r ) / 2,

[0044] wherein R l and T l are the rotation matrix and translation vector of the left side car light respectively, and R r and T r are the rotation matrix and translation vector of the right side car light respectively;

[0045] The projection pixel coordinate processing unit is configured to, when the world coordinates of the car light projection are known, calculate the pixel coordinates (u, v) on the projection image of the equivalent car light according to formula (300), and then calculate the pixel coordinates (u l , v l ) on the projection image of the left car light and the pixel coordinates (u r , v r ) on the projection image of the right car light according to formula (100) and (200) simultaneously.

[0046] Preferably, the matrix Q has the form:

[0047]

[0048] The matrix P has the form:

[0049]

[0050] The matrix M has the form:

[0051]

[0052] Preferably, the projection pixel coordinate processing unit is configured to:

[0053] According to formula (300) and (100) simultaneously:

[0054]

[0055] According to formula (300) and (200) simultaneously:

[0056]

[0057] Wherein, parameter m 00 i to parameter m 23 i , parameter m 00 l to parameter m 23 l , parameter m 00 r to parameter m 23 r are known quantities, when the world coordinates (x w , y w , z w ) projected by the car lights are known, the pixel coordinates (u l , v l ) on the projection image of the left car light and the pixel coordinates (u r , v r ) on the projection image of the right car light are calculated by the above formula.

[0058] Preferably, the x-axis and y-axis in the left car light coordinate system and the right car light coordinate system are the same as the x-axis and y-axis of the projection picture coordinate system, and the z-axis in the world coordinate system is the vertical direction.

[0059] In the embodiment of the present application, when the world coordinates projected by the car lights are known, the pixel coordinates on the projection image of the left car light and the pixel coordinates on the projection image of the right car light are calculated by simultaneously solving the conversion relationship between the pixel coordinates in the equivalent car light coordinate system and the world coordinate system, the conversion relationship between the pixel coordinates on the projection image of the left car light and the world coordinate system, and the conversion relationship between the pixel coordinates on the projection image of the right car light and the world coordinate system, so that the left and right intelligent car lights can align the pixels and project the target pattern cooperatively, avoiding the problems of projection image deformation or ghosting, and without the need to move or rotate the hardware, avoiding the inconvenience of moving or rotating the car lights in real conditions. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The flowchart of the method for cooperative projection of the double intelligent car lights provided in the embodiment of the present application.

[0061] Figure 2 The schematic diagram of the three coordinate systems included in the intelligent car light system of the embodiment of the present application.

[0062] Figure 3 The combined projection diagram of the double car lights of the embodiment of the present application.

[0063] Figure 4 The schematic diagram of the projection image required to be input by the left and right intelligent car lights respectively in the embodiment of the present application. DETAILED DESCRIPTION

[0064] In the drawings, the same or similar components or components having the same or similar functions are denoted by the same or similar reference numerals. The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0065] In the description of the present application, the terms "center", "longitudinal", "transverse", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or components referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the scope of protection of the present application.

[0066] The input of the vehicle lamp is a matrix of data from the vehicle end perception fusion module, which is an image that needs to be projected by the vehicle lamp, the value is (u p ×v p ) (representing the size of the picture), and the value of each number is a pixel value.

[0067] The present application first needs to calibrate two vehicle lamps separately to obtain their respective projection images to the coordinate system of the earth. Based on this situation, it is assumed that there is an equivalent vehicle lamp that can produce the same effect as the two side vehicle lamps, located between the two side vehicle lamps, the position and conversion relationship of the equivalent vehicle lamp can be calculated, and the relative conversion relationship between the combined vehicle lamp and the two side vehicle lamps can be obtained respectively, so as to calculate the pixel values (u p l ×v p l ) and (u p r ×v p r ) of the inputs of the two side vehicle lamps respectively.

[0068] Figure 1 A flowchart of a method for cooperative projection of a double intelligent vehicle lamp is shown. As shown in the figure, the method comprises the following steps: Figure 1

[0069] Step 101, establishing a world coordinate system (X W ,Y W ,Z W ), a left vehicle lamp coordinate system (X l ,Y l ,Z l ) and a projection picture coordinate system (x l ,y l ), a right vehicle lamp coordinate system (X r ,Y r ,Z r ​) and the projection picture coordinate system (x r ,y r ).

[0070] The conversion relationship between the pixel coordinates (u l ,v l ) on the projection picture of the left headlamp coordinate system (X l ,Y l ,Z l ) and the world coordinate system is:

[0071] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (1)

[0072] The matrix Q l represents the conversion matrix of the pixel coordinates (u l ,v l ) on the projection picture to the left headlamp coordinate system (X l ,Y l ,Z l ), the matrix P l represents the conversion matrix from the left headlamp coordinate system (X l ,Y l ,Z l ) to the world coordinate system, and T represents the matrix transposition; the values of the matrix P l and the matrix Q l are calculated by pre-calibrating the left headlamp coordinate, the pixel coordinates of the projection picture of the left headlamp and the ground projection coordinate;

[0073] The conversion relationship between the pixel coordinates (u r ,v r ) on the projection picture of the right headlamp coordinate system (X r ,Y r ,Z r ) and the world coordinate system is:

[0074] [u r v r 1] T =P r - 1Qr -1 [x w y w z w 1] T =M r [x w y w z w 1] T (2)

[0075] wherein matrix Q r represents a transformation matrix of pixel coordinates (u r ,v r ) of the projection picture to the right car light coordinate system (X r ,Y r ,Z r ), matrix P r represents a transformation matrix from the right car light coordinate system (X r ,Y r ,Z r ) to the world coordinate system, and T represents matrix transposition; values of matrix P r and matrix Q r are calculated by pre-calibrating the right car light coordinate, pixel coordinates of the projection picture of the right car light and ground projection coordinates.

[0076] Step 102, an equivalent car light coordinate system (X i ,Y i ,Z i ) of the left car light coordinate system and the right car light coordinate system is established.

[0077] pixel coordinates on the projection image of the equivalent car light are (u,v), and a conversion relationship between the pixel coordinates and the world coordinate system is as follows:

[0078]

[0079] wherein G - 1 l is an inverse matrix of matrix Q l ,

[0080] R=R r R l -1 , T i =(T l +T r ) / 2,

[0081] wherein R l and T l are respectively a rotation matrix and a translation vector of the left car light, R r and T ris the rotation matrix and translation vector of the right side car light, respectively;

[0082] At step 103, when the world coordinates of the car light projection are known, the conversion relationship between the pixel coordinates in the equivalent car light coordinate system and the world coordinate system, the conversion relationship between the pixel coordinates on the projection image of the left car light and the world coordinate system, and the conversion relationship between the pixel coordinates on the projection image of the right car light and the world coordinate system are combined to calculate the pixel coordinates on the projection image of the left car light and the pixel coordinates on the projection image of the right car light.

[0083] Specifically, the pixel coordinates on the projection image of the equivalent car light are calculated as (u, v) according to formula (3), and then the pixel coordinates (u l ,v l ) on the projection image of the left car light and the pixel coordinates (u r ,v r ) on the projection image of the right car light are calculated by combining formula (1) and (2).

[0084] In an embodiment, the matrix Q has the form:

[0085]

[0086] The matrix P has the form:

[0087]

[0088] The matrix M has the form:

[0089]

[0090] In an embodiment, step 103 comprises:

[0091] Combining formula (3) and (1) gives:

[0092]

[0093] Combining formula (3) and (2) gives:

[0094]

[0095] wherein the parameters m 00 i to the parameters m 23 i , the parameters m 00 l to the parameters m 23 l , the parameters m 00 r to the parameters m 23 r are known quantities, and when the world coordinates (xw ,y w ,z w When calculating the pixel coordinates (u) on the projected image of the left headlight using the above formula, the pixel coordinates (u) are obtained. l ,v l ) and the pixel coordinates (u) on the projected image of the right headlight r ,v r ).

[0096] In one implementation, the x-axis and y-axis in the left and right headlight coordinate systems are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis in the world coordinate system is vertical.

[0097] The technical solution involved in this patent will be described in detail below. It should be noted that the following description is for illustrative purposes and not for limiting purposes.

[0098] 1. Single-sided headlight situation

[0099] 1.1 Principle

[0100] Figure 2 This diagram illustrates the three coordinate systems included in the intelligent vehicle lighting system according to an embodiment of the present invention. Figure 2 As shown, the intelligent vehicle lighting system includes three coordinate systems: the world coordinate system X... W Y W Z w Headlight coordinate system X C Y C Z C And the projected image coordinate system xy. In this application scenario, the world coordinate system X... W Y W Z W Set as a geodetic coordinate system to represent the three-dimensional position of an object in the real world; hereinafter, it will be referred to as the geodetic coordinate system. Headlight coordinate system X C Y C Z C The first coordinate system is a three-dimensional coordinate system with the center position of the single-sided headlight as its origin. The second coordinate system, xy, is a two-dimensional coordinate system with the center position of the input projected image pixel of the single-sided headlight as its origin. For example... Figure 1 The input projection image center point (C) x C y The “projected image” can be understood as the input image to be projected, which will be referred to as the input image in the following text. Its pixels will be referred to as the input image pixels in the following text, and will be represented by (u,v).

[0101] The skilled in the art can understand that the conversion relationship from the vehicle lamp position to the preset road surface projection position is fixed. The position of the vehicle lamp in the geodetic coordinate system can be obtained in the following manner: first, the position of the vehicle lamp in the self-vehicle coordinate system is obtained, and then the position of the vehicle lamp in the geodetic coordinate system is obtained according to the conversion relationship between the self-vehicle coordinate system and the geodetic coordinate system. In view of this, the conversion relationship of inputting the pixel (u, v) of the projection image to the preset road surface projection position is also fixed, which can be obtained by single-side vehicle lamp calibration, as described in the following formula (1), and thus formula (2) can be obtained:

[0102]

[0103] In the formula, the matrix Q represents the parameter relationship of the input projection image pixel (u, v) to the vehicle lamp coordinate system X C Y C Z C , which is the intrinsic parameter of the single-side vehicle lamp; the matrix P represents the parameter matrix from the vehicle lamp coordinate system to the road surface geodetic coordinate, which is the extrinsic parameter of the single-side vehicle lamp; and T represents the matrix transpose. The values of the matrix P and the matrix Q can be calculated by pre-calibrating the vehicle lamp coordinate, the pixel coordinate of the projection image of the vehicle lamp, and the ground projection coordinate (world coordinate of the projection).

[0104] In an example, the Q and P matrices are obtained through the following calibration process, Figure 3 which is a schematic diagram of the joint projection of the double vehicle lamps of the embodiment of the present application. In combination with FIG. 1, the calibration process includes the following steps: Figure 3

[0105] S1, prepare the vehicle to be calibrated and a camera. Select a suitable environment, place the camera and the vehicle adjacent to each other, and place the calibration board with the checkerboard on the ground in the area that can be completely illuminated by the vehicle lamp and is within the camera shooting angle. The "suitable environment" can be understood as an environment that is sufficient to fix the camera and the vehicle to be calibrated, and can clearly see the vehicle lamp projection area and the camera shooting area, and is sufficient to operate the subsequent steps.

[0106] S2, first turn off the left and right vehicle lamps, and use the fixed camera to shoot the picture containing the calibration board.

[0107] S3, use the single-side vehicle lamp to be calibrated to project the picture containing the calibration board, and the camera simultaneously shoots the picture containing the projection of the calibration board.

[0108] S4, change the pose of the calibration board, and repeat S2 and S3.

[0109] S5, perform camera calibration using the pictures of the calibration board obtained in S4, obtain the camera intrinsic parameter matrix G cam , and mark the feature points of the calibration board in the image.​

[0110] From the conversion relationship, for a point (x q ,y q ,z q ) on the chessboard in the earth coordinate system, a corresponding pixel (u c ,v c ) can be found on the picture taken by the camera, and the relationship between the camera image and the real chessboard can be described as formula (3):

[0111] (x q ,y q ,z q ) = P cam G cam (u c ,v c ) (3)

[0112] In the formula, G cam is the camera intrinsic matrix, and P cam is the camera extrinsic matrix, both of which are known quantities (obtained from S5).

[0113] The first picture taken by the camera containing the pixel (u c ,v c ) is used as the first input projection image mentioned above, and the pixel (u c ,v c ) is the first input projection image pixel at this time. When the vehicle lamp is projected, the relationship between the first input projection image pixel (u c ,v c ) and the pixel (x w ,y w ,z w ) projected onto the road surface can be described as formula (4):

[0114] (x w ,y e ,z e ) = P lighy *G light (u c ,v c ) (4)

[0115] In the formula, P light is the extrinsic matrix of the single vehicle lamp, and G light is the intrinsic matrix of the single vehicle lamp.

[0116] When the position of the calibration board is changed, the second picture taken by the camera containing the pixel (u ′ c ,v ′ c ) is used as the input projection image mentioned above, and the pixel (u′ c ,v ′ c )This time is the second input projection image pixel, when the car light projection, this time can get the second input projection image pixel (u ′ c ,v ′ c )And the pixel (x w ,y w ,z w )Projected to the road surface can be described as formula (5):

[0117] (x w ,y w ,z w )=P light *G light (u ′ c ,v ′ c ) (5)

[0118] In the above formula (3)-(5), the unknown quantities include P light And G light , while other variables are known variables. Therefore, by transforming the pose of the calibration board, a plurality of sets of image features (u c ,v c )And the corresponding geodetic coordinates (x w ,y w ,z w )Can be obtained, and finally the car light internal and external parameter matrix P light And G light Can be calculated by the least square method, see formula (2) of the left and right car lights.

[0119] According to formula (2), the conversion relationship between the pixel coordinates (u l ,v l )On the projection picture of the left car light coordinate system (X l ,Y l ,Z l )And the world coordinate system can be obtained as:

[0120] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [xw y w z w 1] T (100)

[0121] wherein matrix Q l represents the transformation matrix of pixel coordinates (u l ,v l ) of the projection picture to the left car light coordinate system (X l ,Y l ,Z l ), and matrix P l represents the transformation matrix from the left car light coordinate system (X l ,Y l ,Z l ) to the world coordinate system;

[0122] The conversion relationship between pixel coordinates (u r ,v r ) on the projection picture of the right car light coordinate system (X r ,Y r ,Z r ) and the world coordinate system is:

[0123] [u r v r 1] T = P r -1 Q r -1 [x w y w z w 1] T = M r [x w y w z w 1] T (200)

[0124] wherein matrix Q r represents the transformation matrix of pixel coordinates (u r ,v r ) of the projection picture to the right car light coordinate system (X r ,Y r ,Z r ), and matrix P r represents the transformation matrix from the right car light coordinate system (X r ,Y r ,Z r ) to the world coordinate system.

[0125] 2. Joint projection of double car lights

[0126] 2.1 Relative relationship of double car lights

[0127] Since the transformation from the car light coordinate system to the earth coordinate system is a rigid body transformation, the relationship can be obtained from equation (2) as follows equation (6):

[0128] X C = RX+T=P -1 X (6)

[0129] In the equation, X C represents the car light coordinate system X C Y C Z C , X represents the earth coordinate system X W Y W Z W , R represents the rotation matrix from the car light coordinate system to the earth coordinate system, and T represents the translation matrix from the car light coordinate system to the earth coordinate system.

[0130] According to the inner and outer parameter matrices P light and G light obtained in the single-side car light calibration process of the above-mentioned embodiment, P light can be decomposed into composed of rotation and translation.

[0131] Given that R l and T l are the rotation matrix and translation vector obtained by separate calibration of the left side car light, R r and T r are the rotation matrix and translation vector obtained by separate calibration of the right side car light, and the relationship between the two can be described as follows equation (7):

[0132]

[0133] In the equation, X l and X r represent the coordinates of the same object in the left side car light coordinate system and the right side car light coordinate system in the earth coordinate system, respectively, and X W represents the coordinates of the object in the earth coordinate system, then equation (7) can be transformed into the following equation (8):

[0134] X r = RX+T=P r R l -1 (X l - T l )+T r = R r R l -1 X l +T r -R r Rl -1 T l (8)

[0135] where R l and R r are set as unitary orthogonal matrices, then R r R l -1 = R r R l T and the following equation (9) is obtained:

[0136]

[0137] where R and T are the relative rotation matrix and translation vector between the left and right side car lights, respectively. R in equation (9).

[0138] 2.2 Equivalent car light i

[0139] where R l , T l , R r , and T r are the rotation matrix and translation vector from the earth coordinate system to each car light coordinate system. Assuming that the equivalent car light i has the same intrinsic parameters as the left light, the extrinsic parameters of the equivalent car light are described as the following equation (10) to achieve the ideal projection effect:

[0140]

[0141] where R i and T i represent the rotation matrix and translation vector of the equivalent car light i relative to the earth coordinate system. The subscript i represents Identical. The projection effect obtained when the left and right car lights are projected simultaneously can be equivalent to the projection effect of a car light that is projected alone. In this case, the car light is equivalent to an equivalent car light.

[0142] The conversion relationship between the image coordinate system of the equivalent car light i and the earth coordinate system is obtained as shown in the following equation (300):

[0143]

[0144] where G -1 l is the inverse matrix of the intrinsic parameter matrix of the equivalent car light, M i is the intrinsic and extrinsic parameter conversion matrix of the equivalent car light, and M is equivalent to multiplying the intrinsic and extrinsic parameter matrices together using the associative law of multiplication.

[0145] 2.3 Pixel coordinate values of the projection images of the two side car lights

[0146] When the projection pixel coordinate value of the equivalent car light is known, the projection pixel coordinate value that should be input by the left and right car lights can be obtained from the relative position relationship.

[0147] From the simultaneous equations of the equivalent car light and the left car light, the following relationship (12) is obtained:

[0148]

[0149] In the formula, M l is the intrinsic-extrinsic parameter conversion matrix of the left car light, (u, v) is the projection pixel coordinate value of the known equivalent car light, and (u l ,v l ) represents the projection pixel coordinate value of the left car light to be solved. The value of (u l ,v l ) can be obtained through formula (12).

[0150] From the simultaneous equations of the equivalent car light and the right car light, the following relationship (13) is obtained:

[0151]

[0152] In the formula, M r is the intrinsic-extrinsic parameter conversion matrix of the right car light, (u, v) is the projection pixel coordinate value of the known equivalent car light, and (u r ,v r ) represents the projection pixel coordinate value of the right car light to be solved. The value of (u l ,v l ) can be obtained through formula (13).

[0153] From the simultaneous equations of the equivalent car light and the left car light, for the pixel (u, v) in the projection image (x p ×y p ) of the equivalent car light, the following formula (14) is obtained:

[0154]

[0155] In the formula, parameters m 00 i to m 23 i are elements of the intrinsic-extrinsic parameter conversion matrix M i of the equivalent car light, and parameters m 00 l to m 23 l are elements of the intrinsic-extrinsic parameter conversion matrix M l of the left car light, all of which are known quantities. When the world coordinates (x w ,y w ,zw When calculating the pixel coordinates (u) on the projected image of the left headlight using the above formula, the pixel coordinates (u) are obtained. l ,v l ).

[0156] From the combined equation of the equivalent headlight and the right headlight, for the projected image (x) of the equivalent headlight p ×y p The pixels (u,v) in the matrix can be represented by the following equation (15):

[0157]

[0158] In the formula, parameter m 00 i To parameter m 23 i M is the intrinsic and extrinsic parameter transformation matrix of the equivalent vehicle headlights. i The element, parameter m 00 r To parameter m 23 r M is the intrinsic and extrinsic parameter transformation matrix of the right headlight. r All elements are known quantities. When the world coordinates (x, y) of the headlight projection are known... w ,y w ,z w When calculating the right headlight's projected image using the above formula, the pixel coordinates (u) are obtained. r ,v r ).

[0159] Based on the above model, in certain situations where accurate positioning of the projected pattern is required, such as when there are obstacles on both sides, it is necessary to project the boundary lines that the vehicle can pass through, such as... Figure 4 As shown. At this point, through perception and decision-making algorithms, the true position of the boundary line that needs to be projected can be calculated, that is, all the true [x] can be obtained. w y w z w 1] T This invention can calculate the pattern that needs to be used as input, i.e., all pixel values ​​of the equivalent vehicle headlight. Then, using the P matrix, the position of the boundary line in the projection image that each of the left and right vehicle headlights needs to be input is calculated. Finally, the template is scaled and processed to serve as the input pattern.

[0160] Since this invention refines the implementation scheme of intelligent vehicle lights, the image to be projected is known. Through the construction and calculation of this model, the intelligent vehicle lights on the left and right sides can align pixels and collaboratively project the target pattern, avoiding the problem of image distortion or ghosting. At the same time, it does not require moving or rotating hardware, avoiding the inconvenience of moving or rotating vehicle lights under real conditions (e.g., the vehicle lights are already fixed).

[0161] The application further provides a device for cooperative projection of double intelligent vehicle lamps, which is used for realizing the method described in any of the above embodiments and the embodiments thereof.

[0162] a coordinate system processing unit, which is configured to establish a world coordinate system (X W ,Y W ,Z W ), a left vehicle lamp coordinate system (X l ,Y l ,Z l ) and a projection picture coordinate system (x l ,y l ), and a right vehicle lamp coordinate system (X r ,Y r ,Z r ) and a projection picture coordinate system (x r ,y r );

[0163] wherein the conversion relationship between the pixel coordinates (u l ,v l ) on the projection picture of the left vehicle lamp coordinate system (X l ,Y l ,Z l ) and the world coordinate system is as follows:

[0164] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0165] wherein the matrix Q l represents a transformation matrix of the pixel coordinates (u l ,v l ) on the projection picture to the left vehicle lamp coordinate system (X l ,Y l ,Z l ), the matrix P l represents a transformation matrix from the left vehicle lamp coordinate system (X l ,Y l ,Z l ) to the world coordinate system, T represents matrix transposition, and the matrix P l and the matrix Ql The value is calculated by pre-calibrating the coordinates of the left headlight, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0166] Right headlight coordinate system (X) r ,Y r Z r The pixel coordinates (u) on the projected image r ,v r The transformation relationship between the coordinate system and the world coordinate system is as follows:

[0167] [u r v r 1] T =P r - 1Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0168] Where, matrix Q r Represents the pixel coordinates (u) of the projected image r ,v r ) to the right headlight coordinate system (X r ,Y r Z r The transformation matrix of ), matrix P r Represents the coordinate system from the right headlight (X) r ,Y r Z r The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P r Sum matrix Q r The value is calculated by pre-calibrating the coordinates of the right headlight, the pixel coordinates of the right headlight's projected image, and the ground projection coordinates;

[0169] An equivalent headlight processing unit is used to establish an equivalent headlight coordinate system (X) that integrates the left headlight coordinate system and the right headlight coordinate system. i ,Y i Z i The pixel coordinates on the projected image of the equivalent headlights are (u,v), and the transformation relationship between pixel coordinates and the world coordinate system is:

[0170]

[0171] Among them, G -1l is the inverse matrix of the matrix Q, l

[0172] R = R r R l -1 , T i = (T l + T r ) / 2,

[0173] wherein R l and T l are the rotation matrix and translation vector of the left side headlamp respectively, and R r and T r are the rotation matrix and translation vector of the right side headlamp respectively;

[0174] the projection pixel coordinate processing unit is configured to, when the world coordinates of the headlamp projection are known, calculate the pixel coordinates (u, v) on the projection image of the equivalent headlamp according to formula (300), and then calculate the pixel coordinates (u l , v l ) on the projection image of the left headlamp and the pixel coordinates (u r , v r ) on the projection image of the right headlamp according to formulae (100) and (200) respectively.

[0175] In an embodiment, the matrix Q has the form:

[0176]

[0177] The matrix P has the form:

[0178]

[0179] The matrix M has the form:

[0180]

[0181] In an embodiment, the projection pixel coordinate processing unit is configured to:

[0182] According to formulae (300) and (100), we have:

[0183]

[0184] According to formulae (300) and (200), we have:

[0185]

[0186] wherein the parameters m 00 i to m 23 ​i , parameter m 00 l to parameter m 23 l , parameter m 00 r to parameter m 23 r are known quantities, when the world coordinates (x w ,y w ,z w ) of the projected car light are known, the pixel coordinates (u l ,v l ) on the projected image of the left car light and the pixel coordinates (u r ,v r ) on the projected image of the right car light are calculated by the above formula.

[0187] In an embodiment, the x-axis and y-axis in the left car light coordinate system and the right car light coordinate system are the same as the x-axis and y-axis of the projection picture coordinate system, and the z-axis in the world coordinate system is the vertical direction.

[0188] Since the implementation scheme of the intelligent car light is refined, the image to be projected is known, and the left and right intelligent car lights can align the pixels and project the target pattern cooperatively through the establishment and calculation of the model, the problem of deformation or ghosting of the projected image is avoided, and the inconvenience of moving or rotating the hardware in the real condition (for example, the car light is fixed) is avoided.

[0189] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit it. Those skilled in the art should understand that the technical solutions described in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for collaborative projection of dual intelligent vehicle lights, comprising: Step 1: Establish the world coordinate system (X) W ,Y W Z W ), left headlight coordinate system (X) l ,Y l Z l ) and the coordinate system of the projected image (x l ,y l ), Right headlight coordinate system (X) r ,Y r Z r ) and the coordinate system of the projected image (x r ,y r ); Among them, the left headlight coordinate system (X l ,Y l Z l The pixel coordinates (u) on the projected image l ,v l The transformation relationship between the coordinate system and the world coordinate system is as follows: [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100) Where, matrix Q l Represents the pixel coordinates (u) of the projected image l ,v l ) to the left headlight coordinate system (X l ,Y l Z l The transformation matrix of ), matrix P l Represents the coordinate system from the left headlight (X) l ,Y l Z l The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P l Sum matrix Q l The value is calculated by pre-calibrating the coordinates of the left headlight, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates; Right headlight coordinate system (X) r ,Y r Z r The pixel coordinates (u) on the projected image r ,v r The transformation relationship between the coordinate system and the world coordinate system is as follows: [u r v r 1] T =P r -1 Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200) Where, matrix Q r Represents the pixel coordinates (u) of the projected image r ,v r ) to the right headlight coordinate system (X r ,Y r Z r The transformation matrix of ), matrix P r Represents the coordinate system from the right headlight (X) r ,Y r Z r The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P r Sum matrix Q r The value is calculated by pre-calibrating the coordinates of the right headlight, the pixel coordinates of the right headlight's projected image, and the ground projection coordinates; Step 2: Establish an equivalent headlight coordinate system (X) based on the left headlight coordinate system and the right headlight coordinate system. i ,Y i Z i The pixel coordinates on the projected image of the equivalent headlights are (u,v), and the transformation relationship between pixel coordinates and the world coordinate system is: Among them, G -1 l For matrix Q l The inverse matrix, Among them, R l and T l R represents the rotation matrix and translation vector of the left headlight, respectively. r and T r These are the rotation matrix and translation vector of the right headlight, respectively; Step 3: Given the world coordinates of the headlight projection, calculate the pixel coordinates (u,v) on the equivalent headlight projection image according to equation (300). Then, combine equations (100) and (200) to calculate the pixel coordinates (u,v) on the left headlight projection image. l ,v l ) and the pixel coordinates (u) on the projected image of the right headlight r ,v r ).

2. The method for collaborative projection of dual intelligent vehicle lights as described in claim 1, characterized in that, The matrix Q is in the form of: The matrix P is in the form of: The matrix M is in the form of:

3. The method for collaborative projection of dual intelligent vehicle lights as described in claim 2, characterized in that, Step three includes: Combining equations (300) and (100), we get: Combining equations (300) and (200), we get: Wherein, parameter m 00 i To parameter m 23 i Parameter m 00 l To parameter m 23 l Parameter m 00 r To parameter m 23 r All are known quantities. When the world coordinates (x, y) of the headlight projection are known... w ,y w ,z w When calculating the pixel coordinates (u) on the projected image of the left headlight using the above formula, the pixel coordinates (u) are obtained. l ,v l ) and the pixel coordinates (u) on the projected image of the right headlight r ,v r ).

4. The method for collaborative projection of dual intelligent vehicle lights as described in claim 1, characterized in that, The x and y axes in the left and right headlight coordinate systems are in the same direction as the x and y axes in the projected image coordinate system, while the z axis in the world coordinate system is vertical.

5. A device for coordinated projection of dual intelligent vehicle lights, comprising: Coordinate system processing unit, used to establish the world coordinate system (X). W ,Y W Z W ), left headlight coordinate system (X) l ,Y l Z l ) and the coordinate system of the projected image (x l ,y l ), Right headlight coordinate system (X) r ,Y r Z r ) and the coordinate system of the projected image (x r ,y r ); Among them, the left headlight coordinate system (X l ,Y l Z l The pixel coordinates (u) on the projected image l ,v l The transformation relationship between the coordinate system and the world coordinate system is as follows: [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100) Where, matrix Q l Represents the pixel coordinates (u) of the projected image l ,v l ) to the left headlight coordinate system (X l ,Y l Z l The transformation matrix of ), matrix P l Represents the coordinate system from the left headlight (X) l ,Y l Z l The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P l Sum matrix Q l The value is calculated by pre-calibrating the coordinates of the left headlight, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates; Right headlight coordinate system (X) r ,Y r Z r The pixel coordinates (u) on the projected image r ,v r The transformation relationship between the coordinate system and the world coordinate system is as follows: [u r v r 1] T =P r -1 Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200) Where, matrix Q r Represents the pixel coordinates (u) of the projected image r ,v r ) to the right headlight coordinate system (X r ,Y r Z r The transformation matrix of ), matrix P r Represents the coordinate system from the right headlight (X) r ,Y r Z r The transformation matrix from the world coordinate system to the world coordinate system, where T represents the matrix transpose; matrix P r Sum matrix Q r The value is calculated by pre-calibrating the coordinates of the right headlight, the pixel coordinates of the right headlight's projected image, and the ground projection coordinates; An equivalent headlight processing unit is used to establish an equivalent headlight coordinate system (X) that integrates the left headlight coordinate system and the right headlight coordinate system. i ,Y i Z i The pixel coordinates on the projected image of the equivalent headlights are (u,v), and the transformation relationship between pixel coordinates and the world coordinate system is: Among them, G -1 l For matrix Q l The inverse matrix, Among them, R l and T l R represents the rotation matrix and translation vector of the left headlight, respectively. r and T r These are the rotation matrix and translation vector of the right headlight, respectively; The projection pixel coordinate processing unit is used to calculate the pixel coordinates (u,v) on the equivalent projection image of the headlight according to equation (300) when the world coordinates of the headlight projection are known, and then calculate the pixel coordinates (u,v) on the projection image of the left headlight by combining equations (100) and (200). l ,v l ) and the pixel coordinates (u) on the projected image of the right headlight r ,v r ).

6. The device for coordinated projection of dual intelligent vehicle lights as described in claim 5, characterized in that, The matrix Q is in the form of: The matrix P is in the form of: The matrix M is in the form of:

7. The device for coordinated projection of dual intelligent vehicle lights as described in claim 6, characterized in that, The projected pixel coordinate processing unit is used for: Combining equations (300) and (100), we get: Combining equations (300) and (200), we get: Wherein, parameter m 00 i To parameter m 23 i Parameter m 00 l To parameter m 23 l Parameter m 00 r To parameter m 23 r All are known quantities. When the world coordinates (x, y) of the headlight projection are known... w ,y w ,z w When calculating the pixel coordinates (u) on the projected image of the left headlight using the above formula, the pixel coordinates (u) are obtained. l ,v l ) and the pixel coordinates (u) on the projected image of the right headlight r ,v r ).

8. The device for coordinated projection of dual intelligent vehicle lights as described in claim 5, characterized in that, The x and y axes in the left and right headlight coordinate systems are in the same direction as the x and y axes in the projected image coordinate system, while the z axis in the world coordinate system is vertical.

Images