Method, device, storage medium and vehicle for vehicle positioning

Abstract

The application relates to a method for vehicle positioning, which comprises the following steps: receiving an image of a positioning label taken by a camera, wherein the camera is arranged on a vehicle to be positioned, and the positioning label comprises four predetermined positioning points; establishing an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane where the positioning label is located; determining, for each of the positioning points, a first coordinate in a world coordinate system and a second coordinate in an image coordinate system; and determining a third transformation matrix from the world coordinate system to a vehicle coordinate system based on a first transformation matrix from the world coordinate system to the intermediate coordinate system, a second transformation matrix from a vehicle coordinate system to a camera coordinate system, a mapping relationship from the image coordinate system to the camera coordinate system, and the first coordinate and the second coordinate of the four positioning points, so as to position the vehicle. The application also relates to an apparatus for vehicle positioning, a computer storage medium and a vehicle.

Description

Technical Field

[0001] This invention relates to the field of vehicle positioning, and more specifically, to a method, apparatus, computer storage medium, and vehicle for vehicle positioning. Background Technology

[0002] With the continuous development of technologies such as autonomous driving and assisted driving, the requirements for vehicle positioning are becoming increasingly stringent. Existing vehicle positioning technologies sometimes struggle to meet these requirements. For example, GPS-based vehicle positioning technology struggles to provide accurate positioning with a deviation within 10cm; while IMU-based vehicle positioning technology has significant cumulative errors on non-horizontal planes (e.g., slopes), making it difficult to provide accurate positioning results.

[0003] In battery swapping station scenarios, it is often necessary to accurately locate vehicles within 10cm on the slope of the station to prepare for subsequent battery swapping operations. Therefore, an improved technical solution for vehicle positioning is desired. Summary of the Invention

[0004] According to one aspect of the present invention, a method for vehicle positioning is provided. The method includes: receiving an image of a positioning tag captured by a camera, wherein the camera is mounted on the vehicle to be positioned, and the positioning tag includes four predetermined positioning points; establishing an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane in which the positioning tag is located; for each of the positioning points, determining a first coordinate in a world coordinate system and a second coordinate in an image coordinate system; and determining a third transformation matrix T3 from the world coordinate system to the vehicle coordinate system based on a first transformation matrix T1 from the world coordinate system to the intermediate coordinate system, a second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system, a mapping relationship g from the image coordinate system to the camera coordinate system, and the first and second coordinates of the four positioning points, thereby positioning the vehicle.

[0005] As a supplement or replacement to the above scheme, in the above method, the first coordinates are determined by the following steps: obtaining the identification information of the positioning tag from the image; and using the identification information to determine the first coordinates of each of the positioning points in the world coordinate system.

[0006] As a supplement or replacement to the above scheme, in the above method, the step of determining the first coordinates using the identification information includes: obtaining the third coordinates of a positioning reference point associated with the positioning tag in the world coordinate system using the identification information; and determining the first coordinates of each of the positioning points in the world coordinate system based on the third coordinates of the positioning reference point and the positional relationship between the positioning reference point and each of the positioning points. Wherein, the positional relationship between the positioning reference point and each of the positioning points is predetermined.

[0007] As a supplement or replacement to the above scheme, the above method also includes nonlinear optimization of the determined third transformation matrix T3 through the following steps: determining the reprojection error δ of the i-th positioning point in the positioning tag. i , i = 1, 2, 3, 4; calculate the first loss function of the positioning tag. And nonlinear optimization of the third transformation matrix F3 is performed by minimizing the first loss function Loss.

[0008] As a supplement or replacement to the above scheme, in the above method, the reprojection error δ i It is a reprojection error based on the surface angle error:

[0009] i = 1, 2, 3, 4.

[0010] in, It is the first coordinate of the i-th positioning point in the world coordinate system. Let (R, t) be the second coordinate of the i-th positioning point in the image coordinate system, and (R, t) be the fifth transformation matrix T5 from the world coordinate system to the camera coordinate system. It is the mapping of the i-th positioning point from the second coordinate in the image coordinate system to the incident light ray of the camera, where || represents the modulo operation and ∧ represents the outer product operation.

[0011] As a supplement or replacement to the above scheme, the method further includes nonlinear optimization of the determined third transformation matrix T3 through the following steps: capturing images of at least two positioning tags using the camera; determining the reprojection error of n positioning points from the at least two positioning tags; and calculating the second loss function of the at least two positioning tags. Where, δ i w is the reprojection error of the i-th positioning point. i The weighting coefficient w is the weighting coefficient of the i-th positioning point. iThe area of ​​the image captured associated with the location tag from which the i-th location point originates; and the nonlinear optimization of the third transformation matrix T3 by minimizing the second loss function Loss′.

[0012] As a supplement or replacement to the above scheme, the above method further includes nonlinear optimization of the determined third transformation matrix T3 through the following steps: receiving images of at least two positioning tags captured by at least two cameras, wherein the at least two cameras are both mounted on the vehicle; determining the reprojection error of the positioning points from the at least two positioning tags; calculating the pose constraint error of the at least two cameras; and performing nonlinear optimization of the third transformation matrix T3 using the pose constraint error.

[0013] According to another aspect of the present invention, an apparatus for vehicle positioning is provided. The apparatus includes a receiving device and a computing device. The receiving device is used to receive images of positioning tags captured by a camera, wherein the camera is mounted on the vehicle to be positioned, and the positioning tag includes four predetermined positioning points. The computing device is used to perform the following operations: establishing an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane in which the positioning tag is located; for each of the positioning points, determining a first coordinate in a world coordinate system and a second coordinate in an image coordinate system; and determining a third transformation matrix T3 from the world coordinate system to the vehicle coordinate system based on a first transformation matrix T1 from the world coordinate system to the intermediate coordinate system, a second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system, a mapping relationship g from the image coordinate system to the camera coordinate system, and the first and second coordinates of the four positioning points, thereby positioning the vehicle.

[0014] As a supplement or replacement to the above solution, in the above device, the first coordinates are determined by the following steps: obtaining the identification information of the positioning tag from the image, and using the identification information to determine the first coordinates of each of the positioning points in the world coordinate system.

[0015] As a supplement or replacement to the above scheme, in the above device, the step of determining the first coordinates using the identification information includes: obtaining the third coordinates of a positioning reference point associated with the positioning tag in the world coordinate system using the identification information; and determining the first coordinates of each of the positioning points in the world coordinate system based on the third coordinates of the positioning reference point and the positional relationship between the positioning reference point and each of the positioning points, wherein the positional relationship between the positioning reference point and each of the positioning points is predetermined.

[0016] As a supplement or replacement to the above scheme, the above device further includes: a first calibration device for calibrating the second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system; and a second calibration device for calibrating the mapping relationship g from the image coordinate system to the camera coordinate system.

[0017] As a supplement or replacement to the above solution, the device further includes a first optimization device. The first optimization device is used to perform the following operation: determine the reprojection error δ of the i-th positioning point in the positioning tag. i , i = 1, 2, 3, 4; calculate the first loss function of the positioning tag. And nonlinear optimization of the third transformation matrix T3 is performed by minimizing the first loss function Loss.

[0018] As a supplement or replacement to the above solution, in the above device, the reprojection error δ i It is a reprojection error based on the surface angle error:

[0019] i = 1, 2, 3, 4.

[0020] in, It is the first coordinate of the i-th positioning point in the world coordinate system. Let (R, t) be the second coordinate of the i-th positioning point in the image coordinate system, and (R, t) be the fifth transformation matrix T5 from the world coordinate system to the camera coordinate system. It is the mapping of the i-th positioning point from the second coordinate in the image coordinate system to the incident light ray of the camera, where || represents the modulo operation and ∧ represents the outer product operation.

[0021] As a supplement or replacement to the above solution, the device further includes a second optimization device. The second optimization device performs the following operations: acquiring images of at least two positioning tags using the camera; determining the reprojection error of n positioning points from the at least two positioning tags; and calculating a second loss function for the at least two positioning tags. Where, δ i w is the reprojection error of the i-th positioning point. i The weighting coefficient w is the weighting coefficient of the i-th positioning point. i The area of ​​the image captured associated with the location tag from which the i-th location point originates; and the nonlinear optimization of the third transformation matrix T3 by minimizing the second loss function Loss′.

[0022] As a supplement or replacement to the above solution, the above device also includes a third optimization device. The third optimization device is used to perform the following operations: receive images of at least two positioning tags captured by at least two cameras, wherein the at least two cameras are both mounted on the vehicle; determine the reprojection error of positioning points from the at least two positioning tags; calculate the pose constraint error of the at least two cameras; and perform nonlinear optimization of the third transformation matrix T3 using the pose constraint error.

[0023] According to another aspect of the present invention, a computer storage medium is provided, the medium including instructions that, when executed, perform the method described above.

[0024] According to another aspect of the invention, a vehicle is provided, the vehicle including a camera and the device as described above. The camera is mounted on the vehicle and is used to capture images of a location tag. Attached Figure Description

[0025] The objects and advantages of the invention will become more fully clear from the following detailed description taken in conjunction with the accompanying drawings, wherein the same or similar elements are indicated by the same reference numerals.

[0026] Figure 1 A flowchart illustrating a method 1000 for vehicle positioning according to an embodiment of the present invention is shown;

[0027] Figure 2 An intermediate coordinate system for vehicle positioning according to an embodiment of the present invention is shown;

[0028] Figure 3 The transformation relationships between coordinate systems according to an embodiment of the present invention are shown;

[0029] Figure 4 A reprojection error based on surface angle error is shown according to an embodiment of the present invention;

[0030] Figure 5 A schematic diagram of a device 5000 for vehicle positioning according to an embodiment of the present invention is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it.

[0032] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0033] Although exemplary embodiments are described as using multiple units to perform exemplary processes, it should be understood that these exemplary processes may also be performed by one or more modules.

[0034] In the context of this invention, the terms "vehicle," "automobile," or other similar terms include motor vehicles in general, such as passenger cars (including SUVs, buses, trucks, etc.), various commercial vehicles, ships, aircraft, etc., and include hybrid electric vehicles, electric vehicles, plug-in hybrid electric vehicles, etc. A hybrid electric vehicle is a vehicle having two or more power sources, such as a gasoline-powered and an electric vehicle.

[0035] Furthermore, in the context of this invention, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Additionally, unless otherwise specifically indicated, in the context of this invention, the terms "comprising," "including," and similar expressions are intended to indicate non-exclusive inclusion.

[0036] Furthermore, in the context of this paper, "world coordinate system" refers to the three-dimensional absolute coordinate system representing the system. The coordinates of the "location points in the location tags" discussed in detail below in the world coordinate system are predetermined. "Vehicle coordinate system" refers to a three-dimensional coordinate system established with a point in the vehicle as the origin (e.g., the vehicle's center of mass). "Camera coordinate system" refers to a three-dimensional coordinate system established with a point in the camera as the origin (e.g., the camera's optical center). "Image coordinate system" refers to a two-dimensional coordinate system established on the image plane with a point in the image (e.g., the intersection of the camera's optical axis and the image plane) as the origin.

[0037] In the following, technical solutions for vehicle positioning according to various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0038] Figure 1 A flowchart illustrating a method 1000 for vehicle positioning according to an embodiment of the present invention is shown. Figure 1As shown, in order to locate a vehicle in a battery swapping station, in step S110, an image of a positioning tag captured by a camera is received. The camera can be mounted on the vehicle to be located, and the positioning tag can be placed in the environmental features where the vehicle is located, such as on the wall or pillar of the battery swapping station. The captured image of the positioning tag includes four pre-determined positioning points; specifically, the first coordinate P of these four positioning points in the world coordinate system... w It was predetermined.

[0039] In the context of this invention, "location tag" is intended to mean a tag with location functionality, wherein there is a point that has been pre-located in the world coordinate system, such as the AprilTag tag, the ARTag tag based on the augmented reality reference marking system, the ArUco Marker binary square marker, or any suitable location tag.

[0040] In step S120, an intermediate coordinate system is established. The xy plane of the intermediate coordinate system is the plane containing the positioning tags, such that for each point on the positioning tags, z = 0 in the intermediate coordinate system. Furthermore, the origin of the intermediate coordinate system can be any point on the positioning tags, and the z-axis of the intermediate coordinate system can point in the direction of the camera.

[0041] For example, Figure 2 An intermediate coordinate system for vehicle positioning according to an embodiment of the present invention is shown. The xy plane of the intermediate coordinate system is the plane where the positioning tag 210 is located, and the origin O of the intermediate coordinate system is shown. t Let A be one endpoint of the positioning tag 210. Extend the side length AD of the positioning tag 210 as the x-axis of the intermediate coordinate system, and extend the side length AB of the positioning tag 210 as the y-axis of the intermediate coordinate system. Optionally, the z-axis of the intermediate coordinate system points in the direction of the camera.

[0042] In step S130, for each positioning point, its first coordinate P in the world coordinate system is determined. w And the second coordinate P in the image coordinate system img The second coordinate system P for each positioning point. img It can be determined directly from the captured image.

[0043] The first coordinate P of each positioning point wThe identification information (such as an identification code) of the positioning tag can be determined as follows: The identification information is obtained from the image of the positioning tag; the identification information is sent to a cloud server, and the third coordinates of the positioning reference point associated with the positioning tag (e.g., a positioning reference point near the positioning tag) in the world coordinate system are received from the server; based on the third coordinates of the positioning reference point and the positional relationship between the positioning reference point and each positioning point, the first coordinate P of each positioning point in the world coordinate system is determined. w The positional relationship between the positioning reference point and each positioning point can be pre-stored in the vehicle's memory or obtained from a cloud server. Furthermore, this positional relationship can be the same for each battery swapping station; therefore, by obtaining the coordinates of the positioning reference point for each station, the first coordinate P of each positioning point can be obtained through simple calculation. w As an alternative to the above method, the first coordinate P of each positioning point in the world coordinate system is... w Alternatively, it can be obtained directly from the cloud server through the identification information in the location tag.

[0044] In step S140, based on the first transformation matrix T1 from the world coordinate system to the intermediate coordinate system, the second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system, the mapping relationship g from the image coordinate system to the camera coordinate system, and the first coordinates P of the four positioning points determined in step S130... w Second coordinate P img The third transformation matrix T3 from the world coordinate system to the vehicle coordinate system is determined to locate the vehicle.

[0045] The second transformation matrix T2 can be calibrated using the camera's extrinsic parameters. The mapping relationship g from the image coordinate system to the camera coordinate system will transform the second coordinate P in the image coordinate system. img Transform into coordinates P in the camera coordinate system cam The mapping relationship g can be calibrated using the camera's internal parameters.

[0046] In one embodiment, the third transformation matrix T3 from the world coordinate system to the vehicle coordinate system is calculated using the following method. The camera is a fisheye camera.

[0047] First, for the i-th positioning point, set the first coordinate... Transformed to the vehicle coordinate system via T1 and T4

[0048]

[0049] Where T4 is the fourth transformation matrix from the intermediate coordinate system to the vehicle coordinate system, R1 and R4 are the rotation matrices of T1 and T4 respectively, and t1 and t4 are the translation vectors of T1 and T4 respectively.

[0050] Then, based on the fisheye camera model, the location point can be calculated using the inverse imaging model. Corresponding direction:

[0051]

[0052] Assume Q i =(m i n i k i ),because λ∈R, therefore,

[0053]

[0054] Substituting equations 1 and 2 into equation 3, we get:

[0055]

[0056] definition:

[0057] 0 = (0, 0, 0) T X = (r 11 r 21 , t 11 r 12 r 22 , t 21 r 13 r 23 , t 31 ) T Substituting this into equation 4, we get:

[0058]

[0059]

[0060] Therefore, Equation 5 can be written as:

[0061] A i X = 0 (Equation 6),

[0062] in,

[0063]

[0064] When the number of positioning points is 4, four A's can be stacked. i ,get:

[0065] AX = 0, where, A∈R 12，12 (Equation 8)

[0066] SVD decomposition can be performed on matrix A to obtain linear solutions to A, thus yielding the third transformation matrix T3 from the world coordinate system to the vehicle coordinate system. Due to the establishment of the intermediate coordinate system, the solution value corresponding to the third transformation matrix T3 can be quickly determined from multiple solutions of A, reducing computational complexity and error rate.

[0067] Therefore, by using location tags in the environment, high-precision real-time vehicle positioning can be achieved. This positioning performance is not affected by whether the vehicle is level or not, meaning that high-precision real-time positioning can be achieved even when the vehicle is on a non-level surface such as a slope at a battery swapping station.

[0068] In the context of this invention, the transformation relationships between the coordinate systems are as follows: Figure 3 As shown. World coordinate system x w y w z w To the intermediate coordinate system x t y t z t The transformation relationship can be represented by the first transformation matrix T1, in the vehicle coordinate system x car y car z car To the camera coordinate system x cam y cam z cam The transformation relationship can be represented by the second transformation matrix T2, in the world coordinate system x w y w z w To the vehicle coordinate system x car y car z car The transformation relationship can be represented by the third transformation matrix T3, in the camera coordinate system x cam y cam z cam To the image coordinate system x img y img z img The transformation relationship can be represented by the mapping relationship g, with the intermediate coordinate system x t y t z t To the vehicle coordinate system x car y car z car The transformation relationship can be represented by the fourth transformation matrix T4, and the world coordinate system x w y w z w To the camera coordinate system x cam y cam z cam The transformation relationship can be represented by the fifth transformation matrix T5.

[0069] In one embodiment, the third transformation matrix T3 determined in step S104 can be nonlinearly optimized to obtain more accurate vehicle positioning. Specifically, the loss function Loss of the positioning tag can be calculated and minimized to achieve nonlinear optimization. The loss function Loss can be determined by the following formula:

[0070]

[0071] Where, δ i It is the reprojection error of the i-th positioning point, i = 1, 2, 3, 4.

[0072] In one embodiment, the reprojection error δ i It is a reprojection error based on the surface angle error, such as Figure 4 As shown. For a point, A represents the ray incident direction derived from the second coordinate in the image coordinate system using the fisheye camera model, and A' represents the ray incident direction calculated from the first coordinate in the world coordinate system using various transformation matrices. Then δ1 is the reprojection error of this point based on the surface angle error. Similarly, for another point, B represents the ray incident direction derived from the second coordinate in the image coordinate system using the fisheye camera model, and B' represents the ray incident direction calculated from the first coordinate in the world coordinate system using various transformation matrices. Then δ2 is the reprojection error of this point based on the surface angle error. Therefore, for the i-th positioning point, its reprojection error based on the surface angle error can be expressed as:

[0073] i = 1, 2, 3, 4.

[0074] in, It is the first coordinate of the i-th positioning point in the world coordinate system. The second coordinate of the i-th positioning point in the image coordinate system is (R,t), and (R,t) is the fifth transformation matrix T5 from the world coordinate system to the camera coordinate system. img ) is the mapping of the point from the second coordinate in the image coordinate system to the incident light ray of the camera obtained by the camera model. |||| represents the modulo operation, and ^ represents the outer product operation.

[0075] Compared to traditional reprojection error based on plane offset error, reprojection error based on surface angle error can more reasonably and accurately reflect the actual reprojection error of the incident light.

[0076] In one embodiment, the same camera may capture images of multiple location tags. The third transformation matrix T3 determined in step S104 can be nonlinearly optimized using n location points from these multiple location tags to obtain more accurate vehicle positioning. Specifically, the reprojection error δ of these n location points is first determined. i (i = 1, ..., n). Here, the reprojection error δ i This could be the reprojection error based on the surface angle error, as mentioned earlier. Furthermore, the weighting coefficient w for these n positioning points is calculated. i (i = 1, ..., n). Here, the weighting coefficient w for the i-th location point. i The area S of the image captured relative to the location tag where the i-th location point is located. i ,For example, Calculate the loss function Loss′ for these multiple labels:

[0077]

[0078] Finally, the third transformation matrix T3 determined in step S104 is nonlinearly optimized by minimizing the second loss function Loss′.

[0079] For a rectangular positioning label, if its four positioning points are located at the four endpoints of the rectangle, and the coordinates of the four positioning points on the captured image are (x0, y0), (x1, y1), (x2, y2), and (x3, y3), then the area S of the captured image of the positioning label is:

[0080] S=0.5*|x0*y1-x1*y0+x1*y2-x2*y1+x2*y3-x3*y2+x3*y0-x0*y3|.

[0081] It is important to note that "multiple location tags" here refers to at least two location tags. In other words, the method described above can optimize the vehicle's location results by using images of at least two location tags captured by the same camera.

[0082] In one embodiment, multiple cameras may be installed on the vehicle. The third transformation matrix T3 determined in step S104 can be nonlinearly optimized by fusing the localization results from these multiple cameras to obtain more accurate vehicle localization. Specifically, firstly, the multiple cameras installed on the vehicle each capture an image of at least one localization tag; for example, each camera captures an image of one localization tag. Secondly, the reprojection error of the localization points from each of the aforementioned localization tags is determined. Here, the reprojection error can be the reprojection error based on the surface angle error mentioned above. Thirdly, the pose constraint error of these multiple cameras is calculated. Finally, the calculated pose constraint error is used to nonlinearly optimize the third transformation matrix T3 determined in step S104. It should be noted that the term "multiple cameras" refers to at least two cameras. That is, the above method can fuse images of localization tags captured by at least two cameras on the vehicle to be localized to optimize the vehicle localization result. The term "pose constraint error" refers to the error obtained based on the pose constraint relationship between multiple cameras.

[0083] Figure 5 A schematic diagram of a vehicle positioning device 5000 according to an embodiment of the present invention is shown. The device 5000 includes a receiving device 510 and a computing device 520. The receiving device 510 is disposed on the vehicle and is used to capture images of positioning tags. Here, the positioning tags can be disposed in environmental features where the vehicle to be located is located, for example, on the wall or pillar of a battery swapping station where the vehicle to be located is located. The captured images of the positioning tags include four predetermined positioning points; specifically, the first coordinate P of these four positioning points in the world coordinate system... w It was predetermined.

[0084] The computing device 520 is used to establish an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane on which the captured positioning tag is located, such that the z=0 of each point on the positioning tag in the intermediate coordinate system. Furthermore, the origin of the intermediate coordinate system can be any point on the positioning tag, and the z-axis of the intermediate coordinate system can point in the direction of the camera.

[0085] The computing device 520 is also used to determine, for each positioning point, its first coordinate P in the world coordinate system. w And the second coordinate P in the image coordinate system img .

[0086] Wherein, the second coordinate system P of each positioning point img It can be determined directly from the captured image.

[0087] The first coordinate P of each positioning point wThe location can be determined as follows: Identification information (such as an identification code) of the location tag is obtained from its image; the identification information is sent to a cloud server, and the third coordinates of the location reference point associated with the location tag (e.g., a location reference point near the location tag) in the world coordinate system are received from the server; based on the third coordinates of the location reference point and the positional relationship between the location reference point and each location point, the first coordinate P of each location point in the world coordinate system is determined. w .

[0088] The positional relationship between the positioning reference point and each positioning point can be pre-stored in the vehicle's memory or obtained from a cloud server. Furthermore, this positional relationship can be the same for each battery swapping station; therefore, by obtaining the coordinates of the positioning reference point for each station, the first coordinate P of each positioning point can be obtained through simple calculation. w As an alternative to the above method, the first coordinate P of each positioning point in the world coordinate system is... w Alternatively, it can be obtained directly from the cloud server through the identification information in the location tag.

[0089] The computing device 520 is also used to determine the first coordinates P of the four positioning points determined in the computing device 520 based on the first transformation matrix T1 from the world coordinate system to the intermediate coordinate system, the second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system, the mapping relationship g from the image coordinate system to the camera coordinate system, and the first coordinates P of the four positioning points determined in the computing device 520. w Second coordinate P img The third transformation matrix T3 from the world coordinate system to the vehicle coordinate system is determined to locate the vehicle. By introducing an intermediate coordinate system, the complexity of calculating the third transformation matrix T3 is reduced, and the error rate is decreased.

[0090] Although Figure 5 As not shown, the device 5000 for vehicle positioning may further include a first calibration device and a second calibration device. The first calibration device is used to calibrate a second transformation matrix T2 from the vehicle coordinate system to the camera coordinate system using the camera's extrinsic parameters; the second calibration device is used to calibrate the mapping relationship g from the image coordinate system to the camera coordinate system using the camera's intrinsic parameters.

[0091] Although Figure 5 Not shown, the device 5000 for vehicle positioning may further include a first optimization unit. The first optimization unit performs nonlinear optimization on the third transformation matrix T3 determined in the computing unit 520 by minimizing the loss function Loss of the positioning tag. The loss function Loss of the positioning tag is:

[0092]

[0093] δ iIt is the reprojection error of the i-th positioning point among the four positioning points of the positioning tag, i = 1, 2, 3, 4.

[0094] As mentioned above, the reprojection error δ i It can be Figure 4 The reprojection error based on surface angle error is shown in the figure:

[0095] i = 1, 2, 3, 4.

[0096] in, It is the first coordinate of the i-th positioning point in the world coordinate system. It is the second coordinate of the i-th positioning point in the image coordinate system, (R,t) is the fifth transformation matrix T5 from the world coordinate system to the camera coordinate system, g(P img ) represents the mapping from the second coordinate of the i-th positioning point in the image coordinate system to the incident ray of the camera, where || represents the modulo operation and ^ represents the outer product operation. Compared with the traditional reprojection error based on plane offset error, the reprojection error based on surface angle error can more reasonably and accurately reflect the actual reprojection error of the incident ray.

[0097] The device 5000 for vehicle positioning may further include a second optimization device. The second optimization device may nonlinearly optimize the third transformation matrix T3 using the loss function Loss′ of images of multiple (i.e., at least two) positioning tags captured by the same camera. Specifically, it determines the reprojection error δ of n positioning points on these multiple positioning tags. i (i = 1, ..., n). Furthermore, calculate the weighting coefficient w for these n locations. i (i = 1, ..., n). Here, the weighting coefficient w for the i-th location point. i The area S of the image captured relative to the location tag where the i-th location point is located. i ,For example, Then, calculate the loss function Loss′ for these multiple labels:

[0098]

[0099] Finally, the third transformation matrix T3 determined in the computing device 520 is nonlinearly optimized by minimizing the second loss function Loss′.

[0100] The device 5000 for vehicle positioning may further include a third optimization unit. This third optimization unit performs nonlinear optimization on the third transformation matrix T3 determined in the computing unit 520 by fusing positioning results from multiple cameras on the vehicle, thereby obtaining more accurate vehicle positioning. Specifically, multiple cameras installed on the vehicle capture images of at least one positioning tag, for example, each camera captures an image of one positioning tag. Then, the reprojection error of the positioning points from the multiple positioning tags is determined. Here, the reprojection error can be the reprojection error based on the surface angle error mentioned above. Next, the pose constraint error of these multiple cameras is calculated. Finally, the calculated pose constraint error is used to perform nonlinear optimization on the third transformation matrix T3 determined in the computing unit 520. The third optimization unit can be used to optimize the vehicle positioning results by fusing images of positioning tags captured by multiple cameras.

[0101] Furthermore, it should be understood that the vehicle positioning device 5000 according to the foregoing embodiments of the present invention can be incorporated into a vehicle. The vehicle may also include a camera for capturing images of the positioning tags.

[0102] Furthermore, it should be understood that the vehicle positioning method according to the foregoing embodiments of the present invention can be implemented by a computer program. For example, when a computer storage medium (e.g., a USB flash drive) storing the computer program is connected to a computer, running the program on the computer storage medium can execute the vehicle positioning method of one or more embodiments of the present invention. Examples of computer storage media include, but are not limited to, ROM, RAM, optical discs, magnetic tapes, floppy disks, flash drives, smart cards, and optical data storage devices. The computer storage medium can also be distributed across a networked computer system, enabling distributed storage and implementation of the computer storage medium, for example, via in-vehicle telecommunications services or a Controller Area Network (CAN).

[0103] In summary, the vehicle positioning solution proposed in this invention achieves real-time, high-precision vehicle positioning using positioning tags, unaffected by factors such as non-horizontal scenes. By establishing an intermediate coordinate system in the positioning calculation, the technical solution of this invention reduces the complexity and error rate of positioning calculations. Furthermore, the technical solution of this invention can also consider nonlinear optimization based on reprojection errors, thereby further increasing the positioning accuracy. Such nonlinear optimization can take into account scenarios where the same camera captures multiple positioning tags, or multiple cameras on the same vehicle capture positioning tags separately, allowing the positioning accuracy to be further improved according to different scenarios.

[0104] Although the foregoing specification describes only some embodiments of the invention, those skilled in the art will understand that the invention can be implemented in many other forms without departing from its spirit and scope. Therefore, the examples and embodiments shown are to be considered illustrative rather than restrictive, and the invention may encompass various modifications and substitutions without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for vehicle positioning, characterized in that, The method includes: Receive an image of a positioning tag captured by a camera, wherein the camera is mounted on the vehicle to be located, and the positioning tag includes four predetermined positioning points; Establish an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane where the positioning label is located, such that z=0 for each point on the positioning label in the intermediate coordinate system; For each of the positioning points, determine the first coordinate in the world coordinate system and the second coordinate in the image coordinate system; and Based on the first transformation matrix from the world coordinate system to the intermediate coordinate system The second transformation matrix from the vehicle coordinate system to the camera coordinate system The mapping relationship between the image coordinate system and the camera coordinate system The third transformation matrix from the world coordinate system to the vehicle coordinate system is determined by the first and second coordinates of the four positioning points. This allows for the location of the vehicle.

2. The method for vehicle positioning according to claim 1, wherein, The first coordinate is determined by the following steps: Obtain the identification information of the positioning tag from the image; and The identification information is used to determine the first coordinates of each of the positioning points in the world coordinate system.

3. The method for vehicle positioning according to claim 2, wherein, The step of determining the first coordinates using the identification information includes: Using the identification information, obtain the third coordinates of the positioning reference point associated with the positioning tag in the world coordinate system; and The first coordinates of each of the positioning points in the world coordinate system are determined based on the third coordinates of the positioning reference point and the positional relationship between the positioning reference point and each of the positioning points. The positional relationship between the positioning reference point and each of the positioning points is predetermined.

4. The method for vehicle positioning according to claim 1 further includes the following steps for determining the third transformation matrix. Perform nonlinear optimization: Determine the first of the location tags Reprojection error of each positioning point , =1, 2, 3, 4; Calculate the first loss function of the positioning tag. : ;as well as By minimizing the first loss function To the third transformation matrix Perform nonlinear optimization.

5. The method for vehicle positioning according to claim 4, wherein, The reprojection error It is a reprojection error based on the surface angle error: ， =1，2，3，4 in, It is the first The first coordinate of the positioning point in the world coordinate system. The first one is the one mentioned The second coordinate of each positioning point in the image coordinate system. It is the fifth transformation matrix from the world coordinate system to the camera coordinate system. , The first one is the one mentioned The mapping of each positioning point from the second coordinate in the image coordinate system to the incident light ray from the camera. || represents the modulo operation, and ^ represents the outer product operation.

6. The method for vehicle positioning according to claim 1, further comprising the following steps: adjusting the determined third transformation matrix... Perform nonlinear optimization: The camera is used to capture images of at least two location tags; Determine from the at least two location tags Reprojection error of each positioning point; Calculate the second loss function of the at least two positioning tags. : ， in, It is the first Reprojection error of each positioning point The first one is the one mentioned The weighting coefficients for each location point, the weighting coefficients Related to the first The area of ​​the image captured from the location tag from which each location point originates; as well as By minimizing the second loss function To the third transformation matrix Perform nonlinear optimization.

7. The method for vehicle positioning according to claim 1 further includes nonlinear optimization of the determined third transformation matrix through the following step: Receive images of at least two location tags captured by at least two cameras, wherein, The at least two cameras are both mounted on the vehicle; Determine the reprojection error of the positioning points from the at least two positioning tags; Calculate the pose constraint error of the at least two cameras; as well as The third transformation matrix is ​​adjusted using the pose constraint error. Perform nonlinear optimization.

8. A device for vehicle positioning, characterized in that, The device includes: A receiving device for receiving images of a positioning tag captured by a camera, wherein the camera is mounted on the vehicle to be located, and the positioning tag includes four predetermined positioning points. A computing device for performing the following operations: Establish an intermediate coordinate system, wherein the xy plane of the intermediate coordinate system is the plane where the positioning label is located, such that z=0 for each point on the positioning label in the intermediate coordinate system; For each of the positioning points, determine the first coordinate in the world coordinate system and the second coordinate in the image coordinate system; Based on the first transformation matrix from the world coordinate system to the intermediate coordinate system The second transformation matrix from the vehicle coordinate system to the camera coordinate system The mapping relationship between the image coordinate system and the camera coordinate system The third transformation matrix from the world coordinate system to the vehicle coordinate system is determined by the first and second coordinates of the four positioning points. This allows for the location of the vehicle.

9. The device for vehicle positioning according to claim 8, wherein, The first coordinate is determined by the following steps: Obtain the identification information of the positioning tag from the image, and The identification information is used to determine the first coordinates of each of the positioning points in the world coordinate system.

10. The device for vehicle positioning according to claim 9, wherein, The step of determining the first coordinates using the identification information includes: Using the identification information, obtain the third coordinates of the positioning reference point associated with the positioning tag in the world coordinate system; and The first coordinates of each of the positioning points in the world coordinate system are determined based on the third coordinates of the positioning reference point and the positional relationship between the positioning reference point and each of the positioning points. The positional relationship between the positioning reference point and each of the positioning points is predetermined.

11. The device for vehicle positioning according to claim 8, wherein, Also includes: A first calibration device is used to calibrate the second transformation matrix from the vehicle coordinate system to the camera coordinate system. ;as well as The second calibration device is used to calibrate the mapping relationship between the image coordinate system and the camera coordinate system. .

12. The device for vehicle positioning according to claim 8, further comprising a first optimization device, the first optimization device being configured to perform the following operations: Determine the first of the location tags Reprojection error of each positioning point , =1, 2, 3, 4; Calculate the first loss function of the positioning tag. : ;as well as By minimizing the first loss function To the third transformation matrix Perform nonlinear optimization.

13. The device for vehicle positioning according to claim 12, wherein, The reprojection error It is a reprojection error based on the surface angle error: ， =1，2，3，4 in, It is the first The first coordinate of the positioning point in the world coordinate system. The first one is the one mentioned The second coordinate of each positioning point in the image coordinate system. It is the fifth transformation matrix from the world coordinate system to the camera coordinate system. , The first one is the one mentioned The mapping of each positioning point from the second coordinate in the image coordinate system to the incident light ray from the camera. || represents the modulo operation, and ^ represents the outer product operation.

14. The device for vehicle positioning according to claim 8, further comprising a second optimization device, the second optimization device being configured to perform the following operations: The camera is used to capture images of at least two location tags; Determine from the at least two location tags Reprojection error of each positioning point; Calculate the second loss function of the at least two positioning tags. : ， in, It is the first Reprojection error of each positioning point The first one is the one mentioned The weighting coefficients for each location point, the weighting coefficients Related to the first The area of ​​the image captured from the location tag from which each location point originates; as well as By minimizing the second loss function To the third transformation matrix Perform nonlinear optimization.

15. The device for vehicle positioning according to claim 8, further comprising a third optimization device, the third optimization device being configured to perform the following operations: Receive images of at least two location tags captured by at least two cameras, wherein, The at least two cameras are both mounted on the vehicle; Determine the reprojection error of the positioning points from the at least two positioning tags; Calculate the pose constraint error of the at least two cameras; as well as The third transformation matrix is ​​adjusted using the pose constraint error. Perform nonlinear optimization.

16. A computer storage medium, characterized in that, Includes instructions that, when executed, perform the method for vehicle positioning as described in any one of claims 1 to 7.

17. A vehicle, characterized in that, include: A camera, which is mounted on the vehicle and used to capture images of the location tags; The device for vehicle positioning as described in any one of claims 8 to 15.

Images