Cooperative projection method and device for double intelligent vehicle lamps

Abstract

The invention discloses a collaborative projection method and device for double intelligent vehicle lamps. The method comprises the following steps: establishing a world coordinate system, a left vehicle lamp coordinate system, a projection picture coordinate system, a right vehicle lamp coordinate system and a projection picture coordinate system; establishing an equivalent vehicle lamp coordinate system of the left vehicle lamp coordinate system and the right vehicle lamp coordinate system; when the world coordinate of the car lamp projection is known, the conversion relation between the pixel coordinate in the equivalent car lamp coordinate system and the world coordinate system, the conversion relation between the pixel coordinate on the projection image of the left car lamp and the world coordinate system and the conversion relation between the pixel coordinate on the projection image of the right car lamp and the world coordinate system are combined; pixel coordinates on the projection image of the left vehicle lamp and pixel coordinates on the projection image of the right vehicle lamp are obtained through calculation. According to the method, the conversion relation between the pixel coordinates on the projection images of the equivalent vehicle lamp, the left vehicle lamp and the right vehicle lamp and the world coordinate system is simultaneously established, so that the intelligent vehicle lamps on the left side and the right side can align the pixels and cooperatively project the target pattern.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle lamps, and in particular to a method and device for collaborative projection of dual intelligent vehicle lamps. Background Art

[0002] As the pace of entering the era of autonomous driving is accelerating, all parts of the car are moving towards intelligence and networking. An assisted driving solution based on smart lights can become a necessary configuration for information exchange between cars and roads, people and cars, and cars and cars. Smart light technology that combines laser projection with lights will become an important smart component that must be installed in future vehicles. Smart lights can project different interactive content through laser projection according to needs, and convey the information that the driver wants to express, such as preparing to change lanes, giving way to pedestrians, etc. Smart lights can also display various information that helps safe driving, such as the current state of the vehicle, blind spots, weather, etc. This is conducive to playing an important assisted driving function in special vehicle conditions such as low visibility, curves, intersections or state changes, greatly reducing the number of accidents in complex traffic conditions and improving traffic efficiency.

[0003] The existing technology is basically a single projectable headlight, which processes the input projection image and controls the light source matrix on the headlight together with the optical module to project the input image. However, this technology usually only considers the entire technical process from image to projection on the headlight, and cannot achieve the coordinated projection of dual headlights from sensing input. In actual application, problems such as ghosting, incomplete patterns or pattern splicing errors may occur, resulting in information loss or errors, and may even cause accidents. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for the collaborative projection of dual smart headlights, so as to control the left and right smart headlights to cooperate with each other, accurately project the same input image to a preset position, and obtain a desired pattern.

[0005] To achieve the above object, the present invention provides a method for collaborative projection of dual intelligent vehicle lights, comprising:

[0006] Step 1: Establish the world coordinate system (X W ,Y W ,Z W ), left headlight coordinate system (X l ,Y l ,Z l ) and the projection screen coordinate system (x l ,y l ), right headlight coordinate system (X r ,Y r ,Z r ) and the projection screen coordinate system (xr ,y r );

[0007] Among them, the left headlight coordinate system (X l ,Y l ,Z l ) on the projection screen (u l ,v l ) and the world coordinate system is as follows:

[0008] [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0009] Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l ,v l ) to the left car light coordinate system (X l ,Y l ,Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l ,Y l ,Z l ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Q l The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0010] Right headlight coordinate system (X r ,Y r ,Z r ) on the projection screen (u r ,v r ) and the world coordinate system is as follows:

[0011] [u r v r 1] T =P r - 1Q r-1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0012] Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r ,v r ) to the right headlight coordinate system (X r ,Y r ,Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r ,Y r ,Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates;

[0013] Step 2: Establish an equivalent headlight coordinate system (X i ,Y i ,Z i ), the pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is:

[0014]

[0015] Among them, G - 1 l The matrix Q l The inverse matrix of

[0016] R=R r R l -1 , T i =(T l +T r ) / 2,

[0017] Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T r They are the rotation matrix and translation vector of the right headlight respectively;

[0018] Step 3: When the world coordinates of the headlight projection are known, the pixel coordinates of the equivalent headlight on the projection image are calculated according to equation (300) as (u, v), and then the pixel coordinates of the left headlight on the projection image are calculated by combining equations (100) and (200) as (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0019] Preferably, the matrix Q is in the form of:

[0020]

[0021] The matrix P is of the form:

[0022]

[0023] The matrix M has the form:

[0024]

[0025] Preferably, step three includes:

[0026] Combining equations (300) and (100) yields:

[0027]

[0028] Combining equations (300) and (200) yields:

[0029]

[0030] Among them, the parameter m 00 i To parameter m 23 i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (x w ,y w ,z w ), the pixel coordinates (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0031] Preferably, the x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

[0032] The present invention also provides a device for collaborative projection of dual intelligent vehicle lights, comprising:

[0033] Coordinate system processing unit, used to establish the world coordinate system (X W ,Y W ,Z W ), left headlight coordinate system (X l ,Y l ,Z l ) and the projection screen coordinate system (x l ,y l ), right headlight coordinate system (X r ,Y r ,Z r ) and the projection screen coordinate system (x r ,y r );

[0034] Among them, the left headlight coordinate system (X l ,Y l ,Z l ) on the projection screen (u l ,v l ) and the world coordinate system is as follows:

[0035] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0036] Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l ,v l ) to the left car light coordinate system (X l ,Y l ,Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l ,Y l ,Zl ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Q l The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0037] Right headlight coordinate system (X r ,Y r ,Z r ) on the projection screen (u r ,v r ) and the world coordinate system is as follows:

[0038] [u r v r 1] T =P r - 1Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0039] Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r ,v r ) to the right headlight coordinate system (X r ,Y r ,Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r ,Y r ,Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates;

[0040] The equivalent car light processing unit is used to establish an equivalent car light coordinate system (X i ,Y i ,Z i ), the pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is:

[0041]

[0042] Among them, G - 1 l The matrix Q l The inverse matrix of

[0043] R=R r R l -1 , T i =(T l +T r ) / 2,

[0044] Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T r They are the rotation matrix and translation vector of the right headlight respectively;

[0045] The projection pixel coordinate processing unit is used to calculate the pixel coordinates (u, v) on the projection image of the equivalent headlight according to equation (300) when the world coordinates of the headlight projection are known, and then calculate the pixel coordinates (u, v) on the projection image of the left headlight by combining equations (100) and (200). l ,v l ) and the pixel coordinates (u r ,v r ).

[0046] Preferably, the matrix Q is in the form of:

[0047]

[0048] The matrix P is of the form:

[0049]

[0050] The matrix M has the form:

[0051]

[0052] Preferably, the projection pixel coordinate processing unit is used for:

[0053] Combining equations (300) and (100) yields:

[0054]

[0055] Combining equations (300) and (200) yields:

[0056]

[0057] Among them, the parameter m 00 i To parameter m 23 i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (x w ,y w ,z w ), the pixel coordinates (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0058] Preferably, the x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

[0059] In an embodiment of the present invention, by establishing equivalent headlights, when the world coordinates of the headlight projection are known, the conversion relationship between the pixel coordinates in the equivalent headlight coordinate system and the world coordinate system, the conversion relationship between the pixel coordinates on the projection image of the left headlight and the world coordinate system, and the conversion relationship between the pixel coordinates on the projection image of the right headlight and the world coordinate system are jointly calculated to obtain the pixel coordinates on the projection image of the left headlight and the pixel coordinates on the projection image of the right headlight, so that the smart headlights on the left and right sides can align pixels and collaboratively project the target pattern to avoid the problem of projection image deformation or ghosting. At the same time, there is no need to move or rotate the hardware, avoiding the inconvenience of moving and rotating the headlights under real conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic flow chart of a method for collaborative projection of dual intelligent vehicle lights provided in an embodiment of the present invention.

[0061] Figure 2 Schematic diagram of three coordinate systems included in the intelligent vehicle lighting system according to an embodiment of the present invention.

[0062] Figure 3 Schematic diagram of joint projection of dual headlights according to an embodiment of the present invention.

[0063] Figure 4 This is a schematic diagram of projection images that need to be input to the smart headlights on the left and right sides of an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0065] In the description of the present invention, the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0066] The input of the car lights is assumed to be the data of an image that needs to be projected by the car lights from the vehicle-side perception fusion module, and the value is (u p ×v p )(representing the image size), each number is a pixel value.

[0067] The present invention first needs to calibrate the two headlights separately to obtain the transformation relationship between their respective projected images and the geodetic coordinate system. Based on this situation, it is assumed that there is an equivalent headlight that can produce the same effect as the headlights on both sides, located between the headlights on both sides, the position and transformation relationship of the equivalent headlight can be calculated, and the relative transformation relationship between the combined headlight and the headlights on both sides can be obtained respectively, so as to calculate the pixel values ​​(u p l ×v p l ) and (u p r ×v p r ).

[0068] Figure 1 A schematic diagram of a process flow of a method for collaborative projection of dual intelligent vehicle lights provided by an embodiment of the present invention is shown. Figure 1 As shown, the method includes:

[0069] Step 101, establish the world coordinate system (X W ,Y W ,Z W ), left headlight coordinate system (X l ,Y l ,Z l ) and the projection screen coordinate system (x l ,y l ), right headlight coordinate system (X r ,Y r ,Z r) and the projection screen coordinate system (x r ,y r ).

[0070] Among them, the left headlight coordinate system (X l ,Y l ,Z l ) on the projection screen (u l ,v l ) and the world coordinate system is as follows:

[0071] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (1)

[0072] Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l ,v l ) to the left car light coordinate system (X l ,Y l ,Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l ,Y l ,Z l ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Q l The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0073] Right headlight coordinate system (X r ,Y r ,Z r ) on the projection screen (u r ,v r ) and the world coordinate system is as follows:

[0074] [u r v r 1] T =P r - 1Qr -1 [x w y w z w 1] T =M r [x w y w z w 1] T (2)

[0075] Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r ,v r ) to the right headlight coordinate system (X r ,Y r ,Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r ,Y r ,Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates.

[0076] Step 102: Establish an equivalent headlight coordinate system (X i ,Y i ,Z i ).

[0077] The pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is:

[0078]

[0079] Among them, G - 1 l The matrix Q l The inverse matrix of

[0080] R=R r R l -1 , T i =(T l +T r ) / 2,

[0081] Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T rThey are the rotation matrix and translation vector of the right headlight respectively;

[0082] Step 103, when the world coordinates of the headlight projection are known, the conversion relationship between the pixel coordinates in the equivalent headlight coordinate system and the world coordinate system, the conversion relationship between the pixel coordinates on the projection image of the left headlight and the world coordinate system, and the conversion relationship between the pixel coordinates on the projection image of the right headlight and the world coordinate system are combined to calculate the pixel coordinates on the projection image of the left headlight and the pixel coordinates on the projection image of the right headlight.

[0083] Specifically, the pixel coordinates of the equivalent headlight on the projection image are calculated as (u, v) according to equation (3), and the pixel coordinates of the left headlight on the projection image are calculated by combining equations (1) and (2) to obtain (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0084] In one embodiment, the matrix Q is of the form:

[0085]

[0086] The matrix P is of the form:

[0087]

[0088] The matrix M has the form:

[0089]

[0090] In one embodiment, step 103 includes:

[0091] Combining equations (3) and (1) we get:

[0092]

[0093] Combining equations (3) and (2) we get:

[0094]

[0095] Among them, the parameter m 00 i To parameter m 23 i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (xw ,y w ,z w ), the pixel coordinates (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0096] In one implementation, the x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

[0097] The following is a detailed introduction to the technical solution involved in this patent. It should be noted that the following introduction is for the purpose of illustration rather than limitation.

[0098] 1. Single-side headlight situation

[0099] 1.1 Principle

[0100] Figure 2 FIG. 1 is a schematic diagram showing three coordinate systems included in the intelligent vehicle lighting system according to an embodiment of the present invention. Figure 2 As shown, the intelligent headlight system includes three coordinate systems: world coordinate system X W Y W Z w 、Car light coordinate system X C Y C Z C , and the projection screen coordinate system xy. In this application scenario, the world coordinate system X W Y W Z W Set to the geodetic coordinate system, which is used to represent the three-dimensional position of objects in the real world. It is referred to as the geodetic coordinate system in the following text. C Y C Z C The projection image coordinate system xy is a two-dimensional coordinate system, which takes the center position of the input projection image pixel of the single-side headlight as the origin. Figure 1 The center point of the input projection image (C x ,C y ), the “projection screen” can be understood as the input image to be projected, hereinafter referred to as the input image, and its pixels are hereinafter referred to as input image pixels, represented by (u, v).

[0101] Those skilled in the art can understand that there is a fixed conversion relationship from the position of the headlight to the preset road projection position. The position of the headlight in the geodetic coordinate system can be obtained by first obtaining the position of the headlight in the vehicle coordinate system, and then obtaining the position of the headlight in the geodetic coordinate system based on the conversion relationship between the vehicle coordinate system and the geodetic coordinate system. In view of this, there is also a fixed conversion relationship from the pixel (u, v) of the input projection image to the preset road projection position, which can be obtained by unilateral headlight calibration, as described in the following formula (1), from which formula (2) can be obtained:

[0102]

[0103] Where, the matrix Q represents the pixel (u, v) of the input projection image to the light coordinate system X C Y C Z C The parameter relationship in is the internal parameter of the single-side headlight; the matrix P represents the parameter matrix from the headlight coordinate system to the road geodetic coordinate system, which is the external parameter of the single-side headlight; T represents the matrix transpose. The values ​​of the matrix P and the matrix Q can be calculated by pre-calibrating the headlight coordinates, the pixel coordinates of the projected image of the headlight, and the ground projection coordinates (projected world coordinates).

[0104] In one example, the Q and P matrices are obtained by the following calibration process, Figure 3 FIG. 1 is a schematic diagram of a combined projection of dual headlights according to an embodiment of the present invention. Figure 3 As shown in FIG. 1 , the calibration process includes:

[0105] S1, prepare the vehicle to be calibrated and a camera. Choose a suitable environment, place the camera and the vehicle next to each other, and lay the calibration board with a checkerboard pattern on the ground in an area that can be fully illuminated by the headlights and within the camera's field of view. Among them, "suitable environment" can be understood as an environment that is sufficient to fix the camera and the vehicle to be calibrated, and the area projected by the headlights and the area captured by the camera can be clearly seen, and it is sufficient to operate the subsequent steps.

[0106] S2, first turn off the left and right lights, and use the fixed camera to take a picture containing the calibration plate.

[0107] S3, uses the calibrated single-side vehicle light to project a picture containing the calibration plate, and the camera simultaneously shoots the picture with the calibration plate projection on one side.

[0108] S4, change the position and posture of the calibration plate and repeat S2 and S3.

[0109] S5, use the image of the calibration plate obtained in S4 to calibrate the camera and obtain the camera internal parameter matrix G cam , and mark the feature points of the calibration plate in the image.

[0110] From the conversion relationship, we can know that for a point (x q ,y q ,z q ), the corresponding pixel (u c ,v c ), then the relationship between the camera image and the real chessboard can be described as formula (3):

[0111] (x q ,y q ,z q )=P cam G cam (u c ,v c ) (3)

[0112] In the formula, G cam is the camera intrinsic parameter matrix, P cam is the camera extrinsic parameter matrix, all of which are known quantities (obtained from S5).

[0113] The camera takes a picture with pixels (u c ,v c ) is used as the first input projection image mentioned above, and the pixel (u c ,v c ) is the first input projection image pixel. When projected by car lights, the first input projection image pixel (u c ,v c ) and the pixel projected onto the road surface (x w ,y w ,z w ) can be described as formula (4):

[0114] (x w ,y e ,z e )=P lighy *G light (u c ,v c ) (4)

[0115] Where P light is the external parameter matrix of the single-side headlight, G light is the internal parameter matrix of the single-side headlight.

[0116] When the calibration plate position is changed, the camera captures the image containing pixels (u ′ c ,v ′ c ) is used as the input projection image mentioned above, and the pixel (u′ c ,v ′ c ) is the second input projection image pixel. When projected by car lights, the second input projection image pixel (u ′ c ,v ′ c ) and the pixel projected onto the road surface (x w ,y w ,z w ) can be described as formula (5):

[0117] (x w ,y w ,z w )=P light *G light (u ′ c ,v ′ c ) (5)

[0118] In the above formulas (3)-(5), the unknown quantities include P light and G light , and the other variables are known variables. Therefore, by changing the position and posture of the calibration plate, multiple sets of image features (u c ,v c ) and the corresponding geodetic coordinates (x w ,y w ,z w ), and finally the internal and external parameter matrix P of the headlight can be calculated by the least squares rule light and G light , see formula (2) for left and right headlights.

[0119] According to formula (2), the left headlight coordinate system (X l ,Y l ,Z l ) on the projection screen (u l ,v l ) and the world coordinate system is as follows:

[0120] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [xw y w z w 1] T (100)

[0121] Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l ,v l ) to the left car light coordinate system (X l ,Y l ,Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l ,Y l ,Z l ) to the world coordinate system;

[0122] Right headlight coordinate system (X r ,Y r ,Z r ) on the projection screen (u r ,v r ) and the world coordinate system is as follows:

[0123] [u r v r 1] T = P r -1 Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0124] Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r ,v r ) to the right headlight coordinate system (X r ,Y r ,Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r ,Y r ,Z r ) to the world coordinate system.

[0125] 2. Joint projection of dual headlights

[0126] 2.1 Relative relationship between dual headlights

[0127] Since the transformation from the vehicle light coordinate system to the earth coordinate system is a rigid body transformation, the relationship obtained from equation (2) is as follows:

[0128] X C =RX+T=P -1 X (6)

[0129] In the formula, X C Represents the light coordinate system X C Y C Z C , X represents the geodetic coordinate system X W Y W Z W , R represents the rotation matrix from the vehicle light coordinate system to the earth coordinate system, and T represents the translation matrix from the vehicle light coordinate system to the earth coordinate system.

[0130] According to the above embodiment, the internal and external parameter matrix P of the single-side roof obtained in the process of single-side lamp calibration light and G light , P light can be decomposed into It consists of rotation and translation.

[0131] Known R l and T l are the rotation matrix and translation vector of the left headlight after separate calibration, R r and T r are the rotation matrix and translation vector obtained by calibrating the right headlight separately, and the relationship between them can be described as follows (7):

[0132]

[0133] In the formula, X l and X r They represent the coordinates of the same object in the earth coordinate system in the left headlight coordinate system and the right headlight coordinate system, respectively. W represents the coordinates of the object in the geodetic coordinate system, then equation (7) can be transformed into the following equation (8):

[0134] X r =R r R l -1 (X l -T l )+T r =R r R l -1 X l +T r -R r Rl -1 T l (8)

[0135] In the formula, R l and R r are all set to unit orthogonal matrices, then R r R l -1 =R r R l T , and then get the following formula (9):

[0136]

[0137] Where R and T are the relative rotation matrix and translation vector between the left and right headlights, respectively. (9) R in Eq.

[0138] 2.2 Equivalent headlights

[0139] Known R l ,T l and R r ,T r are the rotation matrix and translation vector from the earth coordinate system to the coordinate system of each headlight. Assuming that the equivalent headlight i has the same internal parameters as the left headlight, in order to achieve the ideal projection effect, the external parameters of the equivalent headlight are described as follows (10):

[0140]

[0141] In the formula, R i and T i They represent the rotation matrix and translation vector of the equivalent headlight i relative to the earth coordinate system. The subscript i stands for Identical, which means that the projection effect obtained when the left and right headlights are projected at the same time is equivalent to the effect of only one headlight projected. In this case, the headlight is equivalent to an equivalent headlight.

[0142] Thus, the conversion relationship between the image coordinate system and the earth coordinate system of the equivalent vehicle light i is shown in the following equation (300):

[0143]

[0144] In the formula, G -1 l is the inverse matrix of the equivalent headlight internal parameter matrix, M i is the internal and external parameter conversion matrix of the equivalent headlight, and the M matrix is ​​equivalent to multiplying the internal and external parameter matrices together using the multiplication associative law.

[0145] 2.3 Pixel coordinate values ​​of the projected images of the headlights on both sides

[0146] When the projection pixel coordinate values ​​of the equivalent headlights are known, the projection pixel coordinate values ​​that should be input for the headlights on both sides can be obtained from the relative position relationship.

[0147] From the combination of the equivalent headlight and the left headlight, we get the following relationship (12):

[0148]

[0149] Where M l is the internal and external parameter conversion matrix of the left headlight, (u, v) is the projection pixel value coordinate value of the known equivalent headlight, (u l ,v l ) represents the projection pixel coordinate value of the left headlight to be determined. By using formula (12), (u l ,v l ) value.

[0150] From the combination of the equivalent headlight and the right headlight, we get the following relationship (13):

[0151]

[0152] Where M r is the internal and external parameter conversion matrix of the right headlight, (u, v) is the projection pixel value coordinate value of the known equivalent headlight, (u r ,v r ) represents the projection pixel coordinate value of the right-hand vehicle light to be determined. By using formula (13), (u l ,v l ) value.

[0153] From the combination of the equivalent headlight and the left headlight, the projection image of the equivalent headlight (x p ×y p ), we can get the following equation (14):

[0154]

[0155] In the formula, the parameter m 00 i To parameter m 23 i is the internal and external parameter conversion matrix M of the equivalent headlight i The element of parameter m 00 l To parameter m 23 l is the internal and external parameter conversion matrix M of the left headlight l The elements of are all known quantities. When the world coordinates (x w ,y w ,zw ), the pixel coordinates (u l ,v l ).

[0156] From the combination of the equivalent headlight and the right headlight, the projection image of the equivalent headlight (x p ×y p ), we can get the following equation (15):

[0157]

[0158] In the formula, the parameter m 00 i To parameter m 23 i is the internal and external parameter conversion matrix M of the equivalent headlight i The element of parameter m 00 r To parameter m 23 r is the internal and external parameter conversion matrix M of the right headlight r The elements of are all known quantities. When the world coordinates (x w ,y w ,z w ), the pixel coordinates (u r ,v r ).

[0159] Based on the above model, in some cases where the projection pattern position needs to be accurate, for example, there are obstacles on both sides and it is necessary to project the boundary line that the vehicle can pass through, such as Figure 4 At this time, through the perception and decision algorithm, the real position of the boundary line to be projected can be calculated, that is, all the real [x w y w z w 1] T , the present invention can obtain the pattern required as input, that is, all pixel values ​​of the equivalent headlights, by calculation. Then, the positions of the boundary lines in the projection images required for the left and right headlights are calculated through the P matrix, and then the template is scaled and processed as the input pattern.

[0160] Since the present invention will refine the implementation scheme of the smart car lights, the image to be projected is known. Through the construction and calculation of this model, the smart car lights on the left and right sides can align pixels and collaboratively project the target pattern to avoid the problem of projection image deformation or ghosting. At the same time, there is no need to move or rotate the hardware, avoiding the inconvenience of moving and rotating the car lights under real conditions (for example, the car lights are already fixed).

[0161] The present invention further provides a device for collaborative projection of dual intelligent vehicle lights, which is used to implement the method described in any of the above embodiments and implementation methods thereof, and the device includes:

[0162] Coordinate system processing unit, used to establish the world coordinate system (X W ,Y W ,Z W ), left headlight coordinate system (X l ,Y l ,Z l ) and the projection screen coordinate system (x l ,y l ), right headlight coordinate system (X r ,Y r ,Z r ) and the projection screen coordinate system (x r ,y r );

[0163] Among them, the left headlight coordinate system (X l ,Y l ,Z l ) on the projection screen (u l ,v l ) and the world coordinate system is as follows:

[0164] [u l v l 1] T =P l - 1Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100)

[0165] Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l ,v l ) to the left car light coordinate system (X l ,Y l ,Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l ,Y l ,Z l ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Ql The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates;

[0166] Right headlight coordinate system (X r ,Y r ,Z r ) on the projection screen (u r ,v r ) and the world coordinate system is as follows:

[0167] [u r v r 1] T =P r - 1Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200)

[0168] Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r ,v r ) to the right headlight coordinate system (X r ,Y r ,Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r ,Y r ,Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates;

[0169] The equivalent car light processing unit is used to establish an equivalent car light coordinate system (X i ,Y i ,Z i ), the pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is:

[0170]

[0171] Among them, G -1l The matrix Q l The inverse matrix of

[0172] R=R r R l -1 , T i =(T l +T r ) / 2,

[0173] Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T r They are the rotation matrix and translation vector of the right headlight respectively;

[0174] The projection pixel coordinate processing unit is used to calculate the pixel coordinates (u, v) on the projection image of the equivalent headlight according to equation (300) when the world coordinates of the headlight projection are known, and then calculate the pixel coordinates (u, v) on the projection image of the left headlight by combining equations (100) and (200). l ,v l ) and the pixel coordinates (u r ,v r ).

[0175] In one embodiment, the matrix Q is of the form:

[0176]

[0177] The matrix P is of the form:

[0178]

[0179] The matrix M has the form:

[0180]

[0181] In one embodiment, the projection pixel coordinate processing unit is used to:

[0182] Combining equations (300) and (100) yields:

[0183]

[0184] Combining equations (300) and (200) yields:

[0185]

[0186] Among them, the parameter m 00 i To parameter m 23i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (x w ,y w ,z w ), the pixel coordinates (u l ,v l ) and the pixel coordinates (u r ,v r ).

[0187] In one implementation, the x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

[0188] Since the present invention will refine the implementation scheme of the smart car lights, the image to be projected is known. Through the construction and calculation of this model, the smart car lights on the left and right sides can align pixels and collaboratively project the target pattern to avoid the problem of projection image deformation or ghosting. At the same time, there is no need to move or rotate the hardware, avoiding the inconvenience of moving and rotating the car lights under real conditions (for example, the car lights are already fixed).

[0189] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for collaborative projection of dual intelligent vehicle lights, comprising: Step 1: Establish the world coordinate system (X W , Y W , Z W ), left headlight coordinate system (X l , Y l , Z l ) and the projection screen coordinate system (x l ,y l )、right headlight coordinate system (X r , Y r , Z r ) and the projection screen coordinate system (x r ,y r ); Among them, the left headlight coordinate system (X l , Y l , Z l ) on the projection screen (u l , v l ) and the world coordinate system is as follows: [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100) Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l , v l ) to the left car light coordinate system (X l , Y l , Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l , Y l , Z l ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Q l The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates; Right headlight coordinate system (X r , Y r , Z r ) on the projection screen (u r , v r ) and the world coordinate system is as follows: [u r v r 1] T =P r -1 Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200) Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r , v r ) to the right headlight coordinate system (X r , Y r , Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r , Y r , Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates; Step 2: Establish an equivalent headlight coordinate system (X i , Y i , Z i ), the pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is: Among them, G -1 l The matrix Q l The inverse matrix of R=R r R l -1 ,T i =(T l +T r ) / 2, Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T r They are the rotation matrix and translation vector of the right headlight respectively; Step 3: When the world coordinates of the headlight projection are known, the pixel coordinates of the equivalent headlight on the projection image are calculated according to equation (300) as (u, v), and then the pixel coordinates of the left headlight on the projection image are calculated by combining equations (100) and (200) as (u l , v l ) and the pixel coordinates (u r , v r ).

2. The method for collaborative projection of dual intelligent vehicle lights as claimed in claim 1, characterized in that: The matrix Q has the form: The matrix P is of the form: The matrix M has the form:

3. The method for collaborative projection of dual intelligent vehicle lights as claimed in claim 2, characterized in that: Step three includes: Combining equations (300) and (100) yields: Combining equations (300) and (200) yields: Among them, the parameter m 00 i To parameter m 23 i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (x w ,y w ,z w ), the pixel coordinates (u l , v l ) and the pixel coordinates (u r , v r ).

4. The method for collaborative projection of dual intelligent vehicle lights as claimed in claim 1, characterized in that: The x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

5. A device for collaborative projection of dual intelligent vehicle lights, comprising: Coordinate system processing unit, used to establish the world coordinate system (X W , Y W , Z W ), left headlight coordinate system (X l , Y l , Z l ) and the projection screen coordinate system (x l ,y l )、right headlight coordinate system (X r , Y r , Z r ) and the projection screen coordinate system (x r ,y r ); Among them, the left headlight coordinate system (X l , Y l , Z l ) on the projection screen (u l , v l ) and the world coordinate system is as follows: [u l v l 1] T =P l -1 Q l -1 [x w y w z w 1] T =M l [x w y w z w 1] T (100) Among them, the matrix Q l Represents the pixel coordinates of the projected image (u l , v l ) to the left car light coordinate system (X l , Y l , Z l )’s transformation matrix, matrix P l Represents the coordinate system of the left headlight (X l , Y l , Z l ) to the world coordinate system, T represents the matrix transpose; the matrix P l and the matrix Q l The value of is calculated by pre-calibrating the left headlight coordinates, the pixel coordinates of the projection image of the left headlight, and the ground projection coordinates; Right headlight coordinate system (X r , Y r , Z r ) on the projection screen (u r , v r ) and the world coordinate system is as follows: [u r v r 1] T =P r -1 Q r -1 [x w y w z w 1] T =M r [x w y w z w 1] T (200) Among them, the matrix Q r Represents the pixel coordinates of the projected image (u r , v r ) to the right headlight coordinate system (X r , Y r , Z r )’s transformation matrix, matrix P r Represents the coordinate system of the right headlight (X r , Y r , Z r ) to the world coordinate system, T represents the matrix transpose; the matrix P r and the matrix Q r The value of is calculated by pre-calibrating the right headlight coordinates, the pixel coordinates of the projection image of the right headlight, and the ground projection coordinates; The equivalent car light processing unit is used to establish an equivalent car light coordinate system (X i , Y i , Z i ), the pixel coordinates on the projection image of the equivalent headlight are (u, v), and the conversion relationship between the pixel coordinates and the world coordinate system is: Among them, G -1 l The matrix Q l The inverse matrix of R=R r R l -1 ,T i =(T l +T r ) / 2, Among them, R l and T l are the rotation matrix and translation vector of the left headlight, R r and T r They are the rotation matrix and translation vector of the right headlight respectively; The projection pixel coordinate processing unit is used to calculate the pixel coordinates (u, v) on the projection image of the equivalent headlight according to equation (300) when the world coordinates of the headlight projection are known, and then calculate the pixel coordinates (u, v) on the projection image of the left headlight by combining equations (100) and (200). l , v l ) and the pixel coordinates (u r , v r ).

6. The device for collaborative projection of dual intelligent vehicle lights as claimed in claim 5, characterized in that: The matrix Q has the form: The matrix P is of the form: The matrix M has the form:

7. The device for collaborative projection of dual intelligent vehicle lights as claimed in claim 6, characterized in that: The projected pixel coordinate processing unit is used to: Combining equations (300) and (100) yields: Combining equations (300) and (200) yields: Among them, the parameter m 00 i To parameter m 23 i , parameter m 00 l To parameter m 23 l , parameter m 00 r To parameter m 23 r are all known quantities. When the world coordinates of the headlight projection (x w ,y w ,z w ), the pixel coordinates (u l ,v l ) and the pixel coordinates (u r ,v r ).

8. The device for collaborative projection of dual intelligent vehicle lights as claimed in claim 5, characterized in that: The x-axis and y-axis in the left headlight coordinate system and the right headlight coordinate system are in the same direction as the x-axis and y-axis in the projection screen coordinate system, and the z-axis direction in the world coordinate system is the vertical direction.

Images