Vehicle body coordinate system calibration method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device and storage medium for calibrating a vehicle body coordinate system. First, a vehicle body coordinate system is constructed based on the key points of the vehicle body; then, the first type of coordinate information of the key points of the vehicle body is determined, that is, the coordinate information of the key points of the vehicle body in the world coordinate system is determined; finally, according to the first type of coordinate information of the key points of the vehicle body, the pose information of the vehicle body coordinate system in the world coordinate system is determined. This calibration method does not require the introduction of a calibration room and does not limit the site, which can greatly save the calibration cost. Moreover, this calibration method can be carried out automatically, improving the calibration accuracy and accuracy of the vehicle body coordinate system.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and particularly to a method, device, electronic device and storage medium for calibrating a vehicle body coordinate system. Background Art

[0002] In an autonomous vehicle, it is often necessary to use a variety of sensors to obtain and perceive information around the vehicle, and then use these perception results to assist the vehicle in making decisions and motion planning. At this time, it is necessary to know the pose information of the sensors and the vehicle body coordinate system in the world coordinate system, and then obtain the relative pose information between the sensors and the vehicle body coordinate system through a pose transformation method.

[0003] For the vehicle body, calibrating the vehicle body coordinate system means determining the pose information of the vehicle body coordinate system in the world coordinate system. In the prior art, when calibrating the vehicle body coordinate system, a calibration room is usually used, that is, the world coordinate system is defined in the calibration room. After the vehicle is driven to a specified position in the calibration room, the pose information of the vehicle in the calibration room is determined, that is, the calibration of the vehicle body coordinate system is achieved.

[0004] However, the above method for calibrating the vehicle body coordinate system requires the use of a calibration room, which has strong restrictions on the site, resulting in a high calibration cost. Summary of the Invention

[0005] The present invention provides a method, device, electronic device and storage medium for calibrating a vehicle body coordinate system to solve the defects existing in the prior art.

[0006] The present invention provides a method for calibrating a vehicle body coordinate system, including:

[0007] Based on the key points of the vehicle body, a vehicle body coordinate system is constructed, and first-type coordinate information of the key points of the vehicle body is determined, where the first-type coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system;

[0008] Based on the first-type coordinate information, the pose information of the vehicle body coordinate system in the world coordinate system is determined.

[0009] According to the method for calibrating a vehicle body coordinate system provided by the present invention, the determining the first-type coordinate information of the key points of the vehicle body includes:

[0010] Obtain a plurality of perspective images including the key points of the vehicle body and a calibration board;

[0011] Based on the plurality of perspective images, in combination with a multi-view geometry method, the first-type coordinate information is determined.

[0012] A method for calibrating a vehicle body coordinate system provided by the present invention, wherein the multiple perspective images are obtained by an image acquisition device taking pictures at multiple positions, and the perspective images correspond to the positions one by one;

[0013] Correspondingly, determining the first type of coordinate information based on the multiple perspective images and in combination with the multi-view geometry method includes:

[0014] Determining the first type of pose information of the image acquisition device when taking the multiple perspective images, and the second type of coordinate information of the vehicle body key points; the first type of pose information is the pose information of the image acquisition device in the world coordinate system, and the second type of coordinate information is the coordinate information of the vehicle body key points in the multiple perspective images;

[0015] Based on the first type of pose information and the second type of coordinate information, determining the relationship between the first type of coordinate information and the second type of coordinate information;

[0016] Based on the relationship between the first type of coordinate information and the second type of coordinate information, and the second type of coordinate information, determining the first type of coordinate information.

[0017] A method for calibrating a vehicle body coordinate system provided by the present invention, wherein the relationship between the first type of coordinate information and the second type of coordinate information is represented by a system of equations containing multiple equations;

[0018] Correspondingly, determining the first type of coordinate information based on the relationship between the first type of coordinate information and the second type of coordinate information includes:

[0019] Based on the second type of coordinate information and the least squares method, solving the system of equations to obtain the first type of coordinate information.

[0020] A method for calibrating a vehicle body coordinate system provided by the present invention, wherein both the first type of coordinate information and the second type of coordinate information are homogeneous coordinate information.

[0021] A method for calibrating a vehicle body coordinate system provided by the present invention, wherein determining the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information includes:

[0022] Determining the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points; the third type of coordinate information is the coordinate information of the vehicle body key points in the vehicle body coordinate system;

[0023] Based on the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points, determining the pose information of the vehicle body coordinate system in the world coordinate system.

[0024] According to a vehicle body coordinate system calibration method provided by the present invention, based on the first type of coordinate information, the pose information of the vehicle body coordinate system in the world coordinate system is determined. After that, it further includes:

[0025] Based on the pose information of the vehicle body coordinate system in the world coordinate system and the second type of pose information of a sensing device for sensing the surrounding information of the vehicle, the third type of pose information of the sensing device is determined;

[0026] Wherein, the second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0027] According to a vehicle body coordinate system calibration method provided by the present invention, the vehicle body key points include the center points of each wheel of the vehicle;

[0028] Correspondingly, constructing a vehicle body coordinate system based on the vehicle body key points of the vehicle includes:

[0029] Taking the axle center of the front-side wheel or the rear-side wheel among the respective wheels as the coordinate origin, taking the axle axis direction of the front-side wheel or the rear-side wheel as the first axis, taking the vehicle body direction of the vehicle as the second axis, and taking the direction perpendicular to the plane where the vehicle is located as the third axis, to construct the vehicle body coordinate system.

[0030] The present invention also provides a vehicle body coordinate system calibration device, including:

[0031] A coordinate information determination module, configured to construct a vehicle body coordinate system based on the vehicle body key points of the vehicle and determine the first type of coordinate information of the vehicle body key points, where the first type of coordinate information is the coordinate information of the vehicle body key points in the world coordinate system;

[0032] A calibration module, configured to determine the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned vehicle body coordinate system calibration methods are implemented.

[0034] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned vehicle body coordinate system calibration methods are implemented.

[0035] The present invention also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the vehicle body coordinate system calibration method as described in any one of the above.

[0036] The vehicle body coordinate system calibration method, device, electronic device and storage medium provided by the present invention first construct a vehicle body coordinate system based on the key points of the vehicle body; then determine the first type of coordinate information of the key points of the vehicle body, that is, determine the coordinate information of the key points of the vehicle body in the world coordinate system; finally, determine the pose information of the vehicle body coordinate system in the world coordinate system according to the first type of coordinate information of the key points of the vehicle body. This calibration method does not need to introduce a calibration room and does not require a limited site, which can greatly save the calibration cost. Moreover, this calibration method can be carried out automatically, improving the calibration accuracy and accuracy of the vehicle body coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description can also be obtained by those of ordinary skill in the art without creative efforts based on these drawings.

[0038] Figure 1 is a schematic flowchart of the vehicle body coordinate system calibration method provided by the present invention;

[0039] Figure 2 is a schematic diagram of the vehicle body coordinate system provided by the present invention;

[0040] Figure 3 is a schematic structural diagram of the vehicle body coordinate system calibration device provided by the present invention;

[0041] Figure 4 is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0043] In the prior art, when calibrating the vehicle body coordinate system, the vehicle is usually directly driven to a specified position in the calibration room, which has strong restrictions on the site and results in a high calibration cost. Therefore, an embodiment of the present invention provides a vehicle body coordinate system calibration method.

[0044] Figure 1 The flowchart of a method for calibrating a vehicle body coordinate system provided in an embodiment of the present invention is shown as follows Figure 1 As shown, the method includes:

[0045] S1. Based on the key points of the vehicle body, construct a vehicle body coordinate system and determine the first type of coordinate information of the key points of the vehicle body, where the first type of coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system;

[0046] S2. Based on the first type of coordinate information, determine the pose information of the vehicle body coordinate system in the world coordinate system.

[0047] Specifically, in the method for calibrating a vehicle body coordinate system provided in an embodiment of the present invention, calibrating the vehicle body coordinate system refers to determining the pose information of the vehicle body coordinate system in the world coordinate system. The execution subject of this method is a vehicle body coordinate system calibration device, which can be configured in a server. The server can be a local server or a cloud server. The local server can specifically be a computer, a tablet computer, a smart phone, etc. The present invention does not make specific limitations in this regard.

[0048] First, step S1 is executed. A vehicle body coordinate system is constructed through the key points of the vehicle body. Among them, the key points of the vehicle body refer to the position points on the vehicle body that can be used to identify the vehicle body. The key points of the vehicle body can be selected as needed. For example, the center points of each wheel of the vehicle can be selected as the key points of the vehicle body, or other position points on the vehicle body that play an identification role can also be selected, such as the door handles of the vehicle, the center of gravity of the vehicle body, etc. No specific limitations are made here.

[0049] The number of key points of the vehicle body can be multiple, and each key point of the vehicle body is not on the same straight line. Thus, a vehicle body coordinate system can be constructed through the key points of the vehicle body. The vehicle body coordinate system refers to a coordinate system constructed on the vehicle body with a certain point on the vehicle body as the coordinate origin. The vehicle body coordinate system can be a right-handed coordinate system and a three-dimensional coordinate system. The coordinate origin of the vehicle body coordinate system can be any point on the vehicle body. Each coordinate axis of the vehicle body coordinate system can be a straight line where any point on the vehicle body except the coordinate origin of the vehicle body coordinate system and the coordinate origin of the vehicle body coordinate system are located, and it is necessary to ensure that each coordinate axis of the vehicle body coordinate system is perpendicular to each other. For example, the coordinate origin of the vehicle body coordinate system can be set at the center of gravity of the vehicle body, the direction parallel to the wheel axle direction of the front wheel is used as the x-axis direction, the direction parallel to the vehicle body direction is used as the y-axis direction, and the direction parallel to the perpendicular line of the vehicle plane is used as the z-axis direction. The coordinate origin of the vehicle body coordinate system can also be set at other key points of the vehicle body. The present invention does not make specific limitations in this regard.

[0050] After determining the key body points, the first type of coordinate information of the key body points can also be determined. The first type of coordinate information refers to the coordinate information of the key body points in the world coordinate system and can be represented by coordinate points. Among them, the world coordinate system refers to a coordinate system constructed with a certain point other than the vehicle body as the coordinate origin. The world coordinate system can also be a right-handed coordinate system and a three-dimensional coordinate system. The coordinate origin of the world coordinate system can be any point other than the vehicle body. Each coordinate axis of the world coordinate system can be a straight line passing through any point other than the vehicle body and the coordinate origin of the world coordinate system and the coordinate origin of the world coordinate system. It is necessary to ensure that each coordinate axis of the vehicle body coordinate system is perpendicular to each other. The world coordinate system can be constructed as needed. For example, any point on the earth's surface can be used as the coordinate origin, and three mutually perpendicular coordinate axes passing through the coordinate origin can be selected to construct the world coordinate system. The world coordinate system can also be constructed on a calibration board that is in a relatively static state with the vehicle.

[0051] In the embodiments of the present invention, when calibrating the vehicle body coordinate system, the vehicle is usually in a stationary state, and at this time, the calibration board is also in a stationary state. The calibration board can be set near the vehicle. For example, it can be set in front of the vehicle, and the distance between the calibration board and the vehicle can be set as needed. The calibration board can be a two-dimensional checkerboard board. The calibration board can be vertically set perpendicular to the ground. The vertex of the calibration board can be used as the coordinate origin of the world coordinate system. The horizontal direction of the calibration board can be used as the x-axis direction, the vertical direction of the calibration board can be used as the y-axis direction, and the direction perpendicular to the calibration board can be used as the z-axis direction to construct the world coordinate system. The direction perpendicular to the calibration board can also be used as the x-axis direction, the horizontal direction of the calibration board can be used as the y-axis direction, and the vertical direction of the calibration board can be used as the z-axis direction to construct the world coordinate system. In addition, other position points on the calibration board can be used as the coordinate origin of the world coordinate system, such as the center point of the calibration board, etc. The embodiments of the present invention do not make specific limitations in this regard.

[0052] The first type of coordinate information of the key body points can be obtained either through actual measurement or automatically through an intelligent recognition algorithm. The embodiments of the present invention do not make specific limitations in this regard.

[0053] Then, step S2 is executed. According to the first type of coordinate information of the key body points, that is, the coordinate information of the key body points in the world coordinate system, the vehicle body coordinate system is calibrated. Calibrating the vehicle body coordinate system means determining the pose information of the vehicle body coordinate system in the world coordinate system. Since the coordinate information of the key body points in the world coordinate system has been determined, and since the vehicle body coordinate system is constructed through the key body points, the pose information of the vehicle body coordinate system in the world coordinate system can be determined according to the coordinate information of the key body points in the world coordinate system.

[0054] In the embodiments of the present invention, the pose information can be characterized by a rotation matrix and a translation vector, that is, the pose information of the vehicle body coordinate system in the world coordinate system can be characterized by the relationship between the coordinate information of any point on the vehicle body in the vehicle body coordinate system and its coordinate information in the world coordinate system. For example, for any point p on the vehicle body, the coordinate information p of p in the world coordinate system w and the coordinate information p of p in the vehicle body coordinate system c The relationship between them can be expressed as:

[0055] p w = R c p c + t c

[0056] Among them, R c is the rotation matrix of the vehicle body coordinate system, and t c is the translation vector of the vehicle body coordinate system.

[0057] Therefore, the determination of the pose information of the vehicle body coordinate system in the world coordinate system can be equivalent to the determination of the rotation matrix and translation vector of the vehicle body coordinate system, both of which can be determined by the coordinate information of the vehicle body key points in the world coordinate system, and no specific limitation is made here.

[0058] In the vehicle body coordinate system calibration method provided in the embodiments of the present invention, first, a vehicle body coordinate system is constructed based on the vehicle body key points of the vehicle; then, the first type of coordinate information of the vehicle body key points is determined, that is, the coordinate information of the vehicle body key points in the world coordinate system is determined; finally, the vehicle body coordinate system is calibrated according to the first type of coordinate information of the vehicle body key points. This calibration method does not require the introduction of a calibration room and does not need to limit the site, which can greatly save the calibration cost. Moreover, this calibration method can be carried out automatically, improving the calibration accuracy and accuracy of the vehicle body coordinate system.

[0059] Based on the above embodiments, in the vehicle body coordinate system calibration method provided in the embodiments of the present invention, the determination of the first type of coordinate information of the vehicle body key points includes:

[0060] Obtain multiple perspective images including the vehicle body key points and a calibration board;

[0061] Based on the multiple perspective images, combined with the multi-view geometry method, determine the first type of coordinate information.

[0062] Specifically, in the embodiments of the present invention, when determining the coordinate information of the vehicle body key points in the world coordinate system, the multi-view geometry method can be used for determination.

[0063] First, multiple perspective images including vehicle body key points and a calibration board can be obtained. Here, the multiple perspective images can be captured at multiple positions by an image acquisition device with a movable position. One perspective image can be captured at each position. The image acquisition device can be mounted on a moving auxiliary trolley, and the auxiliary trolley can travel around the vehicle. The image acquisition device can also be held by a user, and multiple perspective images can be captured by the user walking around the vehicle. The image acquisition device can be a camera or a camera, and no specific limitation is made here.

[0064] Each perspective image needs to include vehicle body key points and a calibration board. The calibration board is arranged around the vehicle, and a world coordinate system is constructed on the calibration board.

[0065] Then, based on the multiple perspective images and in combination with the multi-view geometry method, the first type of coordinate information of the vehicle body key points, that is, the coordinate information of the vehicle body key points in the world coordinate system, is determined. Among them, the multi-view geometry method refers to using a geometric method to recover a three-dimensional object from multiple two-dimensional images, that is, three-dimensional reconstruction, which is mainly applied in computer vision. Three-dimensional reconstruction, that is, how to recover the three-dimensional geometric structure information of an object from images taken from different viewpoints of a stationary object.

[0066] Since each perspective image contains vehicle body key points and a calibration board, the vehicle body key points and the calibration board both have corresponding coordinate information in the image coordinate system of the corresponding perspective image. Among them, the image coordinate system is a two-dimensional coordinate system. Furthermore, by using the multi-view geometry method, the coordinate information of the vehicle body key points in the world coordinate system can be recovered from multiple perspective images.

[0067] In the embodiment of the present invention, by determining multiple perspective images and in combination with the multi-view geometry method, the automatic determination of the first type of coordinate information of the vehicle body key points can be realized, the positioning accuracy of the vehicle body key points can be improved, and further the calibration accuracy of the vehicle body coordinate system can be ensured.

[0068] Based on the above embodiment, in the vehicle body coordinate system calibration method provided in the embodiment of the present invention, the multiple perspective images are captured at multiple positions by an image acquisition device, and the perspective images correspond to the positions one by one;

[0069] Correspondingly, the determining the first type of coordinate information based on the multiple perspective images and in combination with the multi-view geometry method includes:

[0070] Determine the first type of pose information of the image acquisition device when capturing the multiple perspective images, and the second type of coordinate information of the vehicle body key points; the first type of pose information is the pose information of the image acquisition device in the world coordinate system, and the second type of coordinate information is the coordinate information of the vehicle body key points in the multiple perspective images;

[0071] Based on the first type of pose information and the second type of coordinate information, determine the relationship between the first type of coordinate information and the second type of coordinate information;

[0072] Based on the relationship between the first type of coordinate information and the second type of coordinate information, and the second type of coordinate information, determine the first type of coordinate information.

[0073] Specifically, in the embodiments of the present invention, since multiple perspective images can be obtained by the image acquisition device capturing images at multiple positions, and one perspective image is captured at each position. Therefore, when combining the multi-view geometry method to determine the first type of coordinate information of the vehicle body key points, the first type of pose information of the image acquisition device, that is, the pose information of the image acquisition device in the world coordinate system, can be determined first when the image acquisition device captures multiple perspective images. That is, it is necessary to determine the pose information of the image acquisition device in the world coordinate system when capturing each view image, and the number of view images captured by the image acquisition device is equal to the number of pose information of the image acquisition device in the world coordinate system to be obtained.

[0074] Since each perspective image contains a calibration board, the first type of pose information of the image acquisition device when each perspective image is captured can be calculated respectively by using the camera pose calibration algorithm based on the calibration board. If the image acquisition device captures n (n≥2) perspective images at n positions. Then n first type of pose information of the image acquisition device can be obtained, which can be respectively expressed as [R i ,t i , [R i ,t i , …, [R i ,t i , …, [R i ,t i . Among them, in the embodiments of the present invention, the camera pose calibration algorithm based on the calibration board is not specifically limited.

[0075] For the i-th first type of pose information, it can be used to represent the coordinate information p w of any point p in the world coordinate system and the coordinate information p i in the image coordinate system of the i-th perspective image respectively, and satisfy the relational expression:

[0076] p w =Ri p i +t i

[0077] wherein, R i is the rotation matrix of the image coordinate system of the i-th perspective image, and t i is the translation vector of the image coordinate system of the i-th perspective image.

[0078] In addition, it is also necessary to determine the second type of coordinate information of the vehicle body key points, that is, the coordinate information of the vehicle body key points in multiple perspective images. The coordinate information of the vehicle body key points in multiple perspective images is the coordinate information of the vehicle body key points in the image coordinate system of each perspective image.

[0079] Here, methods such as key point detection algorithms or manual annotation can be used to determine the coordinate information of the vehicle body key points in each perspective image, and this coordinate information is pixel coordinates.

[0080] Then, based on the first type of pose information of the image acquisition device and the second type of coordinate information of the vehicle body key points, the relationship between the first type of coordinate information and the second type of coordinate information of the vehicle body key points can be determined, that is, the relationship between the coordinate information of the vehicle body key points in the world coordinate system and the coordinate information in the image coordinate system.

[0081] For example, the above relationship can be expressed as:

[0082]

[0083] wherein, p k is the first type of coordinate information of the vehicle body key points, that is, the coordinate information of the vehicle body key points in the world coordinate system, and p i is the second type of coordinate information of the vehicle body key points, that is, the coordinate information of the vehicle body key points in the i-th perspective image, is the rotation matrix of the image coordinate system of the i-th perspective image, is the translation vector of the image coordinate system of the i-th perspective image.

[0084] K i is the internal parameter of the image acquisition device when shooting the i-th perspective image, which can be calibrated by the internal parameter calibration method of the camera. The internal parameter calibration method of the camera will not be elaborated here.

[0085] Finally, based on the above relationship obtained and the second type of coordinate information, the first type of coordinate information of the vehicle body key points can be determined, that is, p k is determined. Since p i is a two-dimensional coordinate with two degrees of freedom, p kis a three-dimensional coordinate with 3 degrees of freedom. Therefore, when n ≥ 2, formula (1) can obtain 4 equations and can be successfully solved to obtain p k .

[0086] It should be noted that the image acquisition device used in the embodiments of the present invention is an ideal camera model without image distortion. For an image acquisition device with distortion, its distortion coefficient can be calibrated through the camera calibration method, and then the distortion coefficient can be used to undistort each perspective image, or the distortion model of each perspective image can be brought into the subsequent calculation process, which will not be elaborated here.

[0087] In the embodiments of the present invention, by determining the first type of pose information and the second type of coordinate information, the relationship between the first type of coordinate information and the second type of coordinate information is obtained, and the first type of coordinate information of the vehicle body key points is determined in combination with this relationship. The method is simple and easy to implement, greatly reducing the complexity of vehicle body coordinate system calibration and improving the calibration efficiency. Moreover, this method only requires a calibration board and an image acquisition device to achieve the calibration of the vehicle body coordinate system.

[0088] Based on the above embodiments, in the vehicle body coordinate system calibration method provided in the embodiments of the present invention, the relationship between the first type of coordinate information and the second type of coordinate information is represented based on a system of equations containing multiple equations;

[0089] Correspondingly, determining the first type of coordinate information based on the relationship between the first type of coordinate information and the second type of coordinate information includes:

[0090] Solving the system of equations based on the second type of coordinate information and the least squares method to obtain the first type of coordinate information.

[0091] Specifically, as can be seen from the above embodiments, the relationship between the first type of coordinate information and the second type of coordinate information can be represented by formula (1). However, due to factors such as measurement errors, the equations in the system of equations may not hold simultaneously, resulting in the inability to solve for p k , for which the second type of coordinate information p of the vehicle body key points can be used i and the least squares method to solve the above system of equations, that is, through the system of equations, a least squares problem can be constructed, that is:

[0092]

[0093] where J is the cost function of the least squares method.

[0094] By minimizing the cost function, p can be obtained k .

[0095] In the embodiment of the present invention, by solving the system of equations in the least squares method, the situation where the solution cannot be obtained due to factors such as measurement errors can be avoided, ensuring the correct solution of the first type of coordinate information of the vehicle body key points.

[0096] Based on the above embodiment, in the vehicle body coordinate system calibration method provided in the embodiment of the present invention, both the first type of coordinate information and the second type of coordinate information are homogeneous coordinate information.

[0097] Specifically, since the first type of coordinate information is three-dimensional coordinates and the second type of coordinate information is two-dimensional coordinates, for the convenience of subsequent calculations, both the first type of coordinate information and the second type of coordinate information are expressed as homogeneous coordinate information. That is, it is ensured that the value of the degree of freedom z in the first type of coordinate information is 1, and at the same time, the degree of freedom z is added to the second type of coordinate information, and the value of this degree of freedom z is also 1.

[0098] Based on the above embodiment, in the vehicle body coordinate system calibration method provided in the embodiment of the present invention, determining the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information includes:

[0099] Determine the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points; the third type of coordinate information is the coordinate information of the vehicle body key points in the vehicle body coordinate system;

[0100] Based on the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points, determine the second type of pose information of the vehicle body coordinate system; the second type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system.

[0101] Specifically, in the embodiment of the present invention, when calibrating the vehicle body coordinate system through the first type of coordinate information, the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points can be determined first, that is, the relationship between the coordinate information of the vehicle body key points in the world coordinate system and their coordinate information in the vehicle body coordinate system is determined. This relationship can be represented by the rotation matrix and translation vector of the vehicle body coordinate system. The rotation matrix of the vehicle body coordinate system can be represented as The translation vector of the vehicle body coordinate system can be represented as t c = o c . Since the coordinate information of the vehicle body key points in the world coordinate system and the coordinate information of the vehicle body key points in the vehicle body coordinate system are both known, the rotation matrix of the vehicle body coordinate system and the translation vector of the vehicle body coordinate system can be solved, that is, the second type of pose information of the vehicle body coordinate system is obtained, which is also the pose information of the vehicle body coordinate system in the world coordinate system.

[0102] In the embodiments of the present invention, by using the relationship between the first type of coordinate information and the third type of coordinate information of the key points of the vehicle body, the pose information of the vehicle body coordinate system in the world coordinate system can be determined, enabling fast calibration and improving the calibration efficiency.

[0103] Based on the above embodiments, in the vehicle body coordinate system calibration method provided in the embodiments of the present invention, after determining the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information, the method further includes:

[0104] Determining the third type of pose information of the sensing device based on the pose information of the vehicle body coordinate system in the world coordinate system and the second type of pose information of the sensing device for sensing the surrounding information of the vehicle;

[0105] Wherein, the second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0106] Specifically, in the embodiments of the present invention, after calibrating the vehicle body coordinate system, the pose information of the vehicle body coordinate system in the world coordinate system is obtained. The sensing device coordinate system for sensing the surrounding information of the vehicle can also be calibrated to determine the second type of pose information of the sensing device, that is, to determine the pose information of the sensing device in the world coordinate system. The sensing device can be various sensors, and the second type of pose information of the sensing device can be represented by the first rotation matrix R s of the sensing device coordinate system and the first translation vector t s of the sensing device coordinate system, which is used to characterize the relationship between the coordinate information p w of any point p in the world coordinate system and the coordinate information p s in the sensing device coordinate system: p w =R s p s +t s .

[0107] Here, the second type of pose information of the sensing device can be calculated by the Zhang Zhengyou calibration method, and specific limitations are not provided here.

[0108] Furthermore, the third type of pose information of the sensing device, that is, the pose information of the sensing device in the vehicle body coordinate system, can be determined by the pose information of the vehicle body coordinate system in the world coordinate system and the second type of pose information of the sensing device.

[0109] The third type of pose information of the sensing device can be represented by the second rotation matrix of the sensing device coordinate system and the second translation vector of the sensing device coordinate system, which is used to characterize the coordinate information pc and the coordinate information p in the coordinate system of the sensing device s satisfy the relational expression:

[0110] wherein, and can be calculated by the following formula:

[0111]

[0112]

[0113] In the embodiments of the present invention, the third type of pose information of the sensing device for sensing the surrounding information of the vehicle can be further determined, and then the decision-making and motion planning of the vehicle can be assisted through the sensing results of the sensing device.

[0114] On the basis of the above embodiments, in the vehicle body coordinate system calibration method provided in the embodiments of the present invention, the vehicle body key points include the center points of the wheels of the vehicle;

[0115] Correspondingly, constructing a vehicle body coordinate system based on the vehicle body key points of the vehicle includes:

[0116] Taking the wheel axle center of the front side wheel or the rear side wheel among the wheels as the coordinate origin, taking the wheel axle axis direction of the front side wheel or the rear side wheel as the first axis direction, taking the vehicle body direction of the vehicle as the second axis, and taking the direction perpendicular to the plane where the vehicle is located as the third axis, to construct the vehicle body coordinate system.

[0117] Specifically, in the embodiments of the present invention, the vehicle body key points may include the center points of the wheels of the vehicle. Each wheel includes a front side wheel and a rear side wheel. The front side wheels include a left front side wheel and a right front side wheel, and the rear side wheels include a left rear side wheel and a right rear side wheel. Among them, the front side refers to the front side of the vehicle head, and the rear side refers to the rear side of the vehicle tail. As Figure 2 shown, the first type of coordinate information of the left front side wheel is p lf , and the first type of coordinate information of the right front side wheel is p rf , the first type of coordinate information of the left rear side wheel is p lb , and the first type of coordinate information of the right rear side wheel is p rb .

[0118] Further, when constructing a vehicle body coordinate system according to the vehicle body key points of the vehicle, the wheel axle center of the front side wheel or the rear side wheel can be used as the coordinate origin, the wheel axle axis direction of the front side wheel or the rear side wheel can be used as the first axis, the vehicle body direction of the vehicle can be used as the second axis, and the direction perpendicular to the plane where the vehicle is located can be used as the third axis to construct the vehicle body coordinate system. Among them, the first axis can be the x-axis, the second axis can be the y-axis, and the third axis can be the z-axis.Figure 2 Taking the wheel axle center of the rear side wheel as the coordinate origin, the first type of coordinate information of this coordinate origin can be expressed as o c , and there are:

[0119]

[0120] The x-axis direction can be expressed as:

[0121]

[0122] The y-axis direction can be expressed as:

[0123]

[0124] where p f On the straight line p lf p rf , and there are:

[0125]

[0126] The z-axis direction can be expressed as:

[0127]

[0128] Figure 2 Among them, c 1 , c 2 , …, c n are respectively the coordinate information of the positions where the image acquisition device captures images of each perspective. A world coordinate system is constructed on the checkerboard, and this world coordinate system can be expressed as o-xyz.

[0129] In the embodiment of the present invention, selecting the wheel axle center of the front side wheel or the rear side wheel of the vehicle as the coordinate origin can make the obtained vehicle body coordinate system easier to construct.

[0130] In summary, the vehicle body coordinate system calibration method provided in the embodiment of the present invention has the characteristics of low requirements for the calibration site, high flexibility, and low cost compared with the existing vehicle body coordinate system calibration methods. In the extreme case, only one calibration board and an additional image acquisition device are required to complete the calibration of the vehicle body coordinate system.

[0131] As Figure 3 shown, on the basis of the above embodiment, the embodiment of the present invention provides a vehicle body coordinate system calibration device, including:

[0132] A coordinate information determination module 31, configured to construct a vehicle body coordinate system based on the body key points of the vehicle and determine the first type of coordinate information of the body key points, where the first type of coordinate information is the coordinate information of the body key points in the world coordinate system;

[0133] The calibration module 32 is configured to determine the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information.

[0134] Based on the above embodiments, in the vehicle body coordinate system calibration device provided in the embodiments of the present invention, the coordinate information determination module is specifically configured to:

[0135] Obtain multiple perspective images including the vehicle body key points and the calibration board;

[0136] Based on the multiple perspective images and in combination with the multi-view geometry method, determine the first type of coordinate information.

[0137] Based on the above embodiments, in the vehicle body coordinate system calibration device provided in the embodiments of the present invention, the multiple perspective images are obtained by the image acquisition device taking pictures at multiple positions, and the perspective images correspond to the positions one by one;

[0138] Correspondingly, the coordinate information determination module is further specifically configured to:

[0139] Determine the first type of pose information of the image acquisition device when taking the multiple perspective images, and the second type of coordinate information of the vehicle body key points; the first type of pose information is the pose information of the image acquisition device in the world coordinate system, and the second type of coordinate information is the coordinate information of the vehicle body key points in the multiple perspective images;

[0140] Based on the first type of pose information and the second type of coordinate information, determine the relationship between the first type of coordinate information and the second type of coordinate information;

[0141] Based on the relationship between the first type of coordinate information and the second type of coordinate information, and the second type of coordinate information, determine the first type of coordinate information.

[0142] Based on the above embodiments, in the vehicle body coordinate system calibration device provided in the embodiments of the present invention, the relationship between the first type of coordinate information and the second type of coordinate information is represented by a system of equations including multiple equations;

[0143] Correspondingly, the coordinate information determination module is further specifically configured to:

[0144] Based on the second type of coordinate information and the least squares method, solve the system of equations to obtain the first type of coordinate information.

[0145] Based on the above embodiments, in the vehicle body coordinate system calibration device provided in the embodiments of the present invention, both the first type of coordinate information and the second type of coordinate information are homogeneous coordinate information.

[0146] Based on the above embodiments, the vehicle body coordinate system calibration device provided in the embodiments of the present invention, the calibration module is specifically used for:

[0147] Determine the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points; the third type of coordinate information is the coordinate information of the vehicle body key points in the vehicle body coordinate system;

[0148] Based on the relationship between the first type of coordinate information and the third type of coordinate information of the vehicle body key points, determine the pose information of the vehicle body coordinate system in the world coordinate system.

[0149] Based on the above embodiments, the vehicle body coordinate system calibration device provided in the embodiments of the present invention further includes a pose information determination module for:

[0150] Based on the pose information of the vehicle body coordinate system in the world coordinate system and the second type of pose information of the sensing device for sensing the surrounding information of the vehicle, determine the third type of pose information of the sensing device;

[0151] Wherein, the second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0152] Based on the above embodiments, in the vehicle body coordinate system calibration device provided in the embodiments of the present invention, the vehicle body key points include the center points of the wheels of the vehicle;

[0153] Correspondingly, the coordinate information determination module is further used for:

[0154] Taking the axle center of the front wheel or the rear wheel among the wheels as the coordinate origin, taking the axle axis direction of the front wheel or the rear wheel as the first axis, taking the vehicle body direction of the vehicle as the second axis, and taking the direction perpendicular to the plane where the vehicle is located as the third axis, construct the vehicle body coordinate system.

[0155] Specifically, the functions of the modules in the vehicle body coordinate system calibration device provided in the embodiments of the present invention correspond one-to-one to the operation processes of the steps in the above method embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and the embodiments of the present invention will not be elaborated herein.

[0156] Figure 4 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 4As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logical instructions in the memory 430 to execute the vehicle body coordinate system calibration method provided in each of the above embodiments. The method includes: based on the key points of the vehicle body, constructing a vehicle body coordinate system, and determining the first type of coordinate information of the key points of the vehicle body, where the first type of coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system; based on the first type of coordinate information, determining the pose information of the vehicle body coordinate system in the world coordinate system.

[0157] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software function units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0158] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the vehicle body coordinate system calibration method provided in each of the above embodiments. The method includes: based on the key points of the vehicle body, constructing a vehicle body coordinate system, and determining the first type of coordinate information of the key points of the vehicle body, where the first type of coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system; based on the first type of coordinate information, determining the pose information of the vehicle body coordinate system in the world coordinate system.

[0159] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the vehicle body coordinate system calibration method provided in the above-mentioned embodiments. The method includes: based on the key points of the vehicle body, constructing a vehicle body coordinate system, and determining the first type of coordinate information of the key points of the vehicle body, where the first type of coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system; based on the first type of coordinate information, determining the pose information of the vehicle body coordinate system in the world coordinate system.

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0161] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating a vehicle body coordinate system, characterized in that, it includes: Based on the key points of the vehicle body, a vehicle body coordinate system is constructed, and the first type of coordinate information of the key points of the vehicle body is determined. The first type of coordinate information is the coordinate information of the key points of the vehicle body in the world coordinate system; the vehicle body coordinate system refers to a coordinate system constructed on the vehicle body with a certain point on the vehicle body as the coordinate origin; Based on the first type of coordinate information, the pose information of the vehicle body coordinate system in the world coordinate system is determined; The determining the first type of coordinate information of the key points of the vehicle body includes: Obtaining multiple perspective images including the key points of the vehicle body and a calibration board; the world coordinate system is constructed on the calibration board; Based on the multiple perspective images, combined with the multi-view geometry method, the first type of coordinate information is determined; The multiple perspective images are obtained by the image acquisition device taking pictures at multiple positions, and the perspective images correspond to the positions one by one; Correspondingly, the determining the first type of coordinate information based on the multiple perspective images and combining the multi-view geometry method includes: Determining the first type of pose information of the image acquisition device when taking the multiple perspective images, and the second type of coordinate information of the key points of the vehicle body; Based on the first type of pose information and the second type of coordinate information, determining the relationship between the first type of coordinate information and the second type of coordinate information; Based on the relationship between the first type of coordinate information and the second type of coordinate information, and the second type of coordinate information, determining the first type of coordinate information; The first type of pose information is the pose information of the image acquisition device in the world coordinate system, and the second type of coordinate information is the coordinate information of the key points of the vehicle body in the multiple perspective images.

2. The method for calibrating a vehicle body coordinate system according to claim 1, characterized in that, The relationship between the first type of coordinate information and the second type of coordinate information is represented by a system of equations including multiple equations; Correspondingly, the determining the first type of coordinate information based on the relationship between the first type of coordinate information and the second type of coordinate information includes: Based on the second type of coordinate information and the least squares method, solving the system of equations to obtain the first type of coordinate information.

3. The method for calibrating a vehicle body coordinate system according to claim 1, characterized in that, Both the first type of coordinate information and the second type of coordinate information are homogeneous coordinate information.

4. The method for calibrating a vehicle body coordinate system according to any one of claims 1-3, characterized in that, The determining the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information includes: Determining the relationship between the first type of coordinate information and the third type of coordinate information of the key points of the vehicle body; the third type of coordinate information is the coordinate information of the key points of the vehicle body in the vehicle body coordinate system; Based on the relationship between the first type of coordinate information and the third type of coordinate information of the key points of the vehicle body, determining the pose information of the vehicle body coordinate system in the world coordinate system.

5. The method for calibrating a vehicle body coordinate system according to any one of claims 1-3, characterized in that, Based on the first type of coordinate information, determining the pose information of the vehicle body coordinate system in the world coordinate system, and then further including: Based on the pose information of the vehicle body coordinate system in the world coordinate system and the second type of pose information of the sensing device for sensing the surrounding information of the vehicle, determining the third type of pose information of the sensing device; Wherein, the second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

6. The vehicle body coordinate system calibration method according to any one of claims 1-3, characterized in that the vehicle body key points include the center points of each wheel of the vehicle; Correspondingly, the constructing the vehicle body coordinate system based on the vehicle body key points includes: Taking the wheel axle center of the front-side wheel or the rear-side wheel among the wheels as the coordinate origin, taking the wheel axle axis direction of the front-side wheel or the rear-side wheel as the first axis, taking the vehicle body direction of the vehicle as the second axis, and taking the direction perpendicular to the plane where the vehicle is located as the third axis, to construct the vehicle body coordinate system.

7. A vehicle body coordinate system calibration device, characterized in that including: A coordinate information determination module, configured to construct a vehicle body coordinate system based on vehicle body key points of a vehicle, and determine first type of coordinate information of the vehicle body key points, where the first type of coordinate information is the coordinate information of the vehicle body key points in the world coordinate system; the vehicle body coordinate system refers to a coordinate system constructed on the vehicle body with a certain point on the vehicle body as the coordinate origin; A calibration module, configured to determine the pose information of the vehicle body coordinate system in the world coordinate system based on the first type of coordinate information; The determining the first type of coordinate information of the vehicle body key points includes: Obtaining a plurality of perspective images including the vehicle body key points and a calibration board; the world coordinate system is constructed on the calibration board; Based on the plurality of perspective images, combining with the multi-view geometry method, determining the first type of coordinate information; The plurality of perspective images are obtained by the image acquisition device shooting at multiple positions, and the perspective images correspond to the positions one by one; Correspondingly, the determining the first type of coordinate information based on the plurality of perspective images and combining with the multi-view geometry method includes: Determining first type of pose information of the image acquisition device when shooting the plurality of perspective images, and second type of coordinate information of the vehicle body key points; Based on the first type of pose information and the second type of coordinate information, determining the relationship between the first type of coordinate information and the second type of coordinate information; Based on the relationship between the first type of coordinate information and the second type of coordinate information, and the second type of coordinate information, determining the first type of coordinate information; The first type of pose information is the pose information of the image acquisition device in the world coordinate system, and the second type of coordinate information is the coordinate information of the vehicle body key points in the plurality of perspective images.

8. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that When the processor executes the program, the steps of the vehicle body coordinate system calibration method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the vehicle body coordinate system calibration method according to any one of claims 1 to 6 are implemented.

Images