Vehicle body coordinate system pose determination method and device, electronic equipment and storage medium

Abstract

The present application relates to the technical field of automatic driving, and provides a vehicle body coordinate system pose determination method and device, electronic equipment and a storage medium, the method does not need to introduce a calibration room, does not need to limit the site, and can greatly save the cost. Moreover, the method can be automatically performed, distance constraints are introduced, the first type of pose information of the vehicle body coordinate system is determined by increasing the solving mode of the constraints, and the accuracy and precision of the first type of pose information are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a vehicle body coordinate system pose determination method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In an autonomous vehicle, it is often necessary to use multiple sensors to acquire and perceive information around the vehicle, and then use these perception results to assist the vehicle in decision-making and motion planning. To achieve the application of the perception results, the pose information of the sensor and the vehicle body coordinate system in the world coordinate system needs to be known, and the relative pose information of the sensor and the vehicle body coordinate system is obtained through the pose transformation method.

[0003] In determining the pose information of the vehicle body coordinate system in the world coordinate system, the existing technology usually adopts a calibration room, that is, the world coordinate system is defined in the calibration room, and after driving the vehicle to a specified position in the calibration room, the pose information of the vehicle in the calibration room can be determined, that is, the pose information of the vehicle body coordinate system in the world coordinate system is determined.

[0004] However, the above-mentioned vehicle body coordinate system pose determination method needs to use a calibration room, which has a strong restriction on the site, resulting in high calibration cost. SUMMARY

[0005] The present application provides a vehicle body coordinate system pose determination method, device, electronic equipment and storage medium to solve the defects in the prior art.

[0006] The present application provides a vehicle body coordinate system pose determination method, comprising:

[0007] Based on the distance constraint between each vehicle body key point of the vehicle, the first type of coordinate information of the vehicle body key point is determined; the first type of coordinate information is the coordinate information of the vehicle body key point in the vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key point;

[0008] The second type of coordinate information of the vehicle body key point is determined, and the first type of pose information of the vehicle body coordinate system is determined based on the first type of coordinate information and the second type of coordinate information; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; the second type of coordinate information is the coordinate information of the vehicle body key point in the world coordinate system.

[0009] According to the vehicle body coordinate system pose determination method provided by the present application, the first type of pose information is represented based on the rotation matrix parameter and the translation vector parameter of the vehicle body coordinate system;

[0010] Correspondingly, the first type of pose information of the vehicle body coordinate system is determined based on the first type of coordinate information and the second type of coordinate information, including:

[0011] Based on the first type of coordinate information, the second type of coordinate information, the rotation matrix parameter and the translation vector parameter, a plurality of coordinate conversion equations between the vehicle body coordinate system and the world coordinate system are determined.

[0012] Solving the equation group constructed by the plurality of coordinate conversion equations, the value of the rotation matrix parameter and the value of the translation vector parameter are obtained.

[0013] According to the vehicle body coordinate system pose determination method provided by the application, the equation group constructed by the plurality of coordinate conversion equations is solved, and the value of the rotation matrix parameter and the value of the translation vector parameter are obtained, including:

[0014] Based on the equation group, a loss function is constructed; the loss function can be used to represent the error of the coordinate conversion relationship of the coordinate system corresponding to the first type of coordinate information and the second type of coordinate information;

[0015] The rotation matrix parameter and the translation vector parameter are solved to minimize the loss function, and the value of the rotation matrix parameter and the value of the translation vector parameter are obtained.

[0016] According to the vehicle body coordinate system pose determination method provided by the application, the distance constraint includes that the distance information between the vehicle body key points in the world coordinate system is equal to the distance information between the vehicle body key points in the vehicle body coordinate system, and both are constant values.

[0017] According to the vehicle body coordinate system pose determination method provided by the application, the vehicle body key points include the center points of each wheel of the vehicle.

[0018] Correspondingly, the distance information includes the distance information between each wheel of the vehicle in the length direction and the distance information between each wheel of the vehicle in the width direction.

[0019] According to the vehicle body coordinate system pose determination method provided by the application, the second type of coordinate information of the vehicle body key points is determined, including:

[0020] A plurality of perspective images containing the vehicle body key points and the calibration board are acquired.

[0021] Based on the plurality of perspective images, the first type of coordinate information is determined by combining the multi-view geometry method.

[0022] According to the vehicle body coordinate system pose determination method provided in the application, the first type of pose information of the vehicle body coordinate system is determined based on the first type of coordinate information and the second type of coordinate information, and then the following steps are further included:

[0023] The third type of pose information of the sensing device is determined based on the first type of pose information and the second type of pose information of the sensing device used for sensing the surrounding information of the vehicle.

[0024] The second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0025] The application further provides a vehicle body coordinate system pose determination device, which comprises:

[0026] The coordinate information determination module is configured to determine the first type of coordinate information of the vehicle body key points based on the distance constraints between the vehicle body key points.

[0027] The pose information determination module is configured to determine the second type of coordinate information of the vehicle body key points, and determine the first type of pose information of the vehicle body coordinate system based on the first type of coordinate information and the second type of coordinate information.

[0028] The application further provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0029] The application further provides a non-transitory computer readable storage medium, which stores a computer program executable on a processor to implement the vehicle body coordinate system pose determination method.

[0030] The application further provides a computer program product, which comprises a computer program executable on a processor to implement the vehicle body coordinate system pose determination method.

[0031] The vehicle body coordinate system pose determination method, device, electronic equipment and storage medium provided by the application first determine first type coordinate information of each vehicle body key point based on distance constraints between the vehicle body key points; the first type coordinate information is coordinate information of each vehicle body key point in a vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points; then second type coordinate information of each vehicle body key point is determined, and first type pose information of the vehicle body coordinate system is determined based on the first type coordinate information and the second type coordinate information; the first type pose information is pose information of the vehicle body coordinate system in a world coordinate system; and the second type coordinate information is coordinate information of each vehicle body key point in the world coordinate system. The method does not need to introduce a calibration room and does not need to limit the site, and can greatly save the cost. Moreover, the method can be automatically performed, distance constraints are introduced, the first type pose information of the vehicle body coordinate system is determined by increasing the solving mode of the constraints, and the accuracy and precision of the first type pose information are improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description can also obtain other drawings without creative labor for those skilled in the art.

[0033] Figure 1 FIG. 1 is a flow diagram of the vehicle body coordinate system pose determination method provided by the application;

[0034] Figure 2 FIG. 2 is a schematic diagram of the vehicle body coordinate system provided by the application;

[0035] Figure 3 FIG. 3 is a structural diagram of the vehicle body coordinate system pose determination device provided by the application;

[0036] Figure 4 FIG. 4 is a structural diagram of the electronic equipment provided by the application. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the application will be described clearly and completely below with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0038] In the prior art, when determining the pose information of the vehicle body coordinate system, the vehicle is usually directly driven to a specified position in the calibration room, which has strong limitations on the site and results in high cost. Therefore, the embodiment of the present application provides a vehicle body coordinate system pose determination method.

[0039] Figure 1 A flowchart of the vehicle body coordinate system pose determination method provided in the embodiment of the present application is shown in Figure 1 The method comprises the following steps.

[0040] S1, based on the distance constraints between the vehicle body key points of the vehicle, determining the first type of coordinate information of the vehicle body key points; the first type of coordinate information is the coordinate information of the vehicle body key points in the vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points;

[0041] S2, determining the second type of coordinate information of the vehicle body key points, and based on the first type of coordinate information and the second type of coordinate information, determining the first type of pose information of the vehicle body coordinate system; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; the second type of coordinate information is the coordinate information of the vehicle body key points in the world coordinate system.

[0042] Specifically, the vehicle body coordinate system pose determination method provided in the embodiment of the present application has a vehicle body coordinate system pose determination device as an execution subject, which can be configured in a server. The server can be a local server or a cloud server. The local server can be a computer, etc. The embodiment of the present application does not make specific limitations on this.

[0043] Firstly, step S1 is performed to determine the first type of coordinate information of the vehicle body key points, i.e. the coordinate information of the vehicle body key points in the vehicle body coordinate system, through the distance constraints between the vehicle body key points of the vehicle. The vehicle body key points of the vehicle refer to the position points on the vehicle body that can be used to identify the vehicle body. The vehicle body key points can be selected as needed, for example, the center points of the wheels of the vehicle can be selected as the vehicle body key points, or other position points on the vehicle body that can be used for identification, such as the door handles of the vehicle, the center of gravity of the vehicle body, etc. The present application does not make specific limitations here.

[0044] The number of body key points can be multiple, and each body key point is not on the same straight line, and then the body coordinate system can be constructed through the body key points. The body coordinate system refers to a coordinate system constructed on the vehicle body with a certain point on the vehicle body as the coordinate origin. The body coordinate system can be a right-handed coordinate system and a three-dimensional coordinate system, and the coordinate origin of the body coordinate system can be any point on the vehicle body, and each coordinate axis of the body coordinate system can be a straight line passing through any point on the vehicle body other than the coordinate origin of the body coordinate system and the coordinate origin of the body coordinate system. It needs to be ensured that each coordinate axis of the body coordinate system is perpendicular to each other. For example, the coordinate origin of the body coordinate system can be set at the center of gravity of the vehicle body, the direction parallel to the axle direction of the front wheel can be set as the x-axis direction, the direction parallel to the vehicle body direction can be set as the y-axis direction, and the direction parallel to the vertical line of the plane where the vehicle is located can be set as the z-axis direction. The coordinate origin of the body coordinate system can also be set at other body key points, which is not limited in the embodiments of the present application.

[0045] Since the body coordinate system is constructed based on each body key point, each body key point is usually at the coordinate origin or on the coordinate axis, and the first type of coordinate information of each body key point is usually 0 or at least one of x, y and z is 0.

[0046] The distance constraint between each body key point refers to the distance information between each body key point satisfying the invariability of the vehicle body structure. For example, the distance constraint can be that the distance information between each body key point is a constant value. Then, by combining the distance constraint between each body key point with the construction principle of the body coordinate system, the first type of coordinate information of each body key point can be determined.

[0047] Then, step S2 is performed to determine the second type of coordinate information of each body key point, which refers to the coordinate information of each body key point in the world coordinate system.

[0048] The world coordinate system refers to a coordinate system constructed with a certain point other than the vehicle body as the coordinate origin. The world coordinate system can also be a right-handed coordinate system and a three-dimensional coordinate system, and the coordinate origin of the world coordinate system can be any point other than the vehicle body, and each coordinate axis of the world coordinate system can be a straight line passing through any point other than the vehicle body and the coordinate origin of the world coordinate system. It needs to be ensured that each coordinate axis of the body coordinate system is perpendicular to each other. The world coordinate system can be constructed as needed, for example, a certain point on the earth's surface can be selected as the coordinate origin, and three coordinate axes passing through the coordinate origin and perpendicular to each other can be selected to construct the world coordinate system. The world coordinate system can also be constructed on a calibration board that is in a relatively static state with the vehicle.

[0049] In the embodiment of the present application, when the first type of pose information of the vehicle body coordinate system is determined, the vehicle is usually in a stationary state, and the calibration board is also in a stationary state at this time. The calibration board can be arranged near the vehicle, for example, can be arranged in front of the vehicle, and the distance between the vehicle and the calibration board can be set as needed. The calibration board can be a two-dimensional checkerboard, and the calibration board can be vertically arranged perpendicular to the ground. The top point of the calibration board can be taken as the coordinate origin of the world coordinate system, the horizontal direction of the calibration board can be taken as the x-axis direction, the vertical direction of the calibration board can be taken as the y-axis direction, and the direction perpendicular to the calibration board can be taken as the z-axis direction to construct the world coordinate system. In addition, the direction perpendicular to the calibration board can be taken as the x-axis direction, the horizontal direction of the calibration board can be taken as the y-axis direction, and the vertical direction of the calibration board can be taken as the z-axis direction to construct the world coordinate system. In addition, other position points on the calibration board can also be taken as the coordinate origin of the world coordinate system, such as the center point of the calibration board, and the like, which are not limited in the embodiment of the present application.

[0050] The second type of coordinate information of the vehicle body key points can be obtained by actually measuring the total station, ruler, laser and other devices, or can be obtained automatically by intelligent recognition algorithm, which is not limited in the embodiment of the present application.

[0051] After the second type of coordinate information of each vehicle body key point is determined, the first type of pose information of the vehicle body coordinate system can be determined by combining the first type of coordinate information and the second type of coordinate information of each vehicle body key point.

[0052] It can be understood that there is a coordinate system coordinate conversion relationship between the world coordinate system and the vehicle body coordinate system, which can be represented by unknown first type of pose parameters. The first type of coordinate information and the second type of coordinate information can be associated through the coordinate system coordinate conversion relationship, and then the first type of pose information can be obtained by solving the coordinate system coordinate conversion relationship.

[0053] However, generally, in order to ensure that the first type of pose information can be solved, the selected vehicle body key points need to be sufficient, which may lead to that the association established by the first type of coordinate information and the second type of coordinate information does not match the unknown quantity, and the unknown quantity has multiple values, which will greatly reduce the accuracy of the obtained first type of pose information. Therefore, in the embodiment of the present application, the number of values of the unknown quantity is reduced by minimizing the error of the coordinate system coordinate conversion relationship, so that the unknown quantity can take only one value, that is, the optimal first type of pose information can be obtained. Since the coordinate system coordinate conversion relationship is an equation, the error is the difference between the left and right sides of the equation.

[0054] The vehicle body coordinate system pose determination method provided in the embodiments of the present application first determines first type coordinate information of each vehicle body key point based on distance constraints between the vehicle body key points; the first type coordinate information is coordinate information of each vehicle body key point in a vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points; then second type coordinate information of each vehicle body key point is determined, and first type pose information of the vehicle body coordinate system is determined based on the first type coordinate information and the second type coordinate information; the first type pose information is pose information of the vehicle body coordinate system in a world coordinate system; and the second type coordinate information is coordinate information of each vehicle body key point in the world coordinate system. This method does not need to introduce a calibration room and does not need to limit the site, and can greatly save costs. Moreover, this method can be automatically performed, distance constraints are introduced, the first type pose information of the vehicle body coordinate system is determined by increasing the solving mode of the constraints, and the accuracy and precision of the first type pose information are improved.

[0055] On the basis of the above embodiments, the vehicle body coordinate system pose determination method provided in the embodiments of the present application, the first type pose information is represented based on a rotation matrix parameter and a translation vector parameter of the vehicle body coordinate system; accordingly,

[0056] The first type pose information of the vehicle body coordinate system is determined based on the first type coordinate information and the second type coordinate information, including:

[0057] Based on the first type coordinate information, the second type coordinate information, the rotation matrix parameter and the translation vector parameter, a plurality of coordinate conversion equations between the vehicle body coordinate system and the world coordinate system are determined;

[0058] An equation group constructed by the plurality of coordinate conversion equations is solved to obtain a value of the rotation matrix parameter and a value of the translation vector parameter.

[0059] Specifically, in the embodiments of the present application, the first type pose information can be represented by a rotation matrix parameter and a translation vector parameter of the vehicle body coordinate system. Further, when determining the first type pose information of the vehicle body coordinate system, a plurality of coordinate conversion equations between the vehicle body coordinate system and the world coordinate system can be determined according to the first type coordinate information, the second type coordinate information, the rotation matrix parameter and the translation vector parameter. Each coordinate conversion equation is an association established by the first type coordinate information and the second type coordinate information through a coordinate system coordinate conversion relationship, and the number of coordinate conversion equations is related to the number of vehicle body key points and the dimensions of the first type coordinate information and the second type coordinate information. For example, if the first type coordinate information and the second type coordinate information are both three-dimensional, the number of coordinate conversion equations can be the product of the number of vehicle body key points and 3.

[0060] Then, the obtained multiple coordinate conversion equations can be combined to obtain an equation group, and the equation group can be solved to obtain the value of the rotation matrix parameter and the value of the translation vector parameter.

[0061] It can be understood that, since the first type of coordinate information is determined based on the distance constraints between the body key points, in the case that the distance information in the distance constraints is unknown, the distance information can also be obtained by solving the equation group.

[0062] In the embodiment of the application, the value of the rotation matrix parameter and the value of the translation vector parameter can be determined by a least square method of constructing an equation group, and then the first type of pose information can be obtained, which can ensure that the first type of pose information is the optimal one among all possible results.

[0063] On the basis of the above-mentioned embodiment, the vehicle body coordinate system pose determination method provided in the embodiment of the application comprises:

[0064] Based on the equation group, a loss function is constructed.

[0065] The rotation matrix parameter and the translation vector parameter are solved to obtain the value of the rotation matrix parameter and the value of the translation vector parameter.

[0066] Specifically, in the embodiment of the application, a loss function can be constructed according to the equation group, which can be obtained by subtracting the right side from the left side of all equations in the equation group. That is, the loss function can be used to represent the error of the coordinate conversion relationship between the first type of coordinate information of all body key points and the second type of coordinate information.

[0067] Then, the rotation matrix parameter and the translation vector parameter in the equation group are solved to obtain the value of the rotation matrix parameter and the value of the translation vector parameter.

[0068] In the embodiment of the application, a specific solving method of the first type of pose information is given, so that the value of the rotation matrix parameter and the value of the translation vector parameter satisfy the target of minimizing the loss function, that is, the optimal first type of pose information is obtained.

[0069] On the basis of the above-mentioned embodiment, the vehicle body coordinate system pose determination method provided in the embodiment of the application, the distance constraints comprise that the distance information between the body key points in the world coordinate system is equal to the distance information between the body key points in the vehicle body coordinate system, and both are constant values.

[0070] Specifically, in this embodiment of the invention, the distance constraint can be that the distance information of each key point of the vehicle body in the world coordinate system is equal to the distance information of each key point of the vehicle body in the vehicle coordinate system, and both are constant values. This constant value can be a known quantity or an unknown quantity. When the constant value is an unknown quantity, the process of solving for the first type of pose information can also obtain this constant value.

[0071] In this embodiment of the invention, the introduction of this distance constraint can further improve the accuracy and rationality of the first type of pose information.

[0072] Based on the above embodiments, the vehicle body coordinate system pose determination method provided in the embodiments of the present invention includes the center points of each wheel of the vehicle as each key point of the vehicle body.

[0073] Accordingly, the distance information includes the distance information between each wheel of the vehicle in the length direction and the distance information between each wheel of the vehicle in the width direction.

[0074] Specifically, in this embodiment of the invention, key points of the vehicle body may include the center points of each wheel. Each wheel includes front wheels and rear wheels. The front wheels include the left front wheel and the right front wheel, and the rear wheels include the left rear wheel and the right rear wheel. Here, "front" refers to the side facing forward of the vehicle, and "rear" refers to the side facing backward. Figure 2 As shown, the second type of coordinate information for the left front wheel is p. lf The second type of coordinate information for the right front wheel is p. rf The second type of coordinate information for the left rear wheel is p. lb The second type of coordinate information for the right rear wheel is p. rb .

[0075] Furthermore, when constructing a vehicle body coordinate system based on key points of the vehicle body, the origin can be set at the axle center of the front or rear wheel, the axle axis of the front or rear wheel can be used as the first axis, the vehicle body direction as the second axis, and the direction perpendicular to the plane on which the vehicle is located as the third axis. The first axis can be x... c Axis, by p lb Point to p rb The second axis can be y c The axis passes through the origin o of the coordinate system. c And it intersects the midpoint of the line connecting the center point of the left front wheel and the center point of the right rear wheel. The third axis can be z. c The axis, and also perpendicular to x. c axis and y c The axis is pointing upwards. Figure 2 The origin of the coordinate system is the center of the axle of the rear wheel. The second type of coordinate information of this origin can be represented as o.c .

[0076] Correspondingly, the distance information of the distance constraint can include distance information between each wheel of the vehicle in the length direction and distance information between each wheel of the vehicle in the width direction.

[0077] The distance information l between each wheel of the vehicle in the length direction can be expressed as:

[0078]

[0079] The distance information d between each wheel of the vehicle in the width direction can be expressed as:

[0080]

[0081] In the embodiment of the application, the wheel shaft center of the front side wheel or the rear side wheel of the vehicle is selected as the coordinate origin, so that the obtained vehicle body coordinate system is easier to construct. At the same time, the distance constraint is easier to represent.

[0082] On the basis of the above embodiment, the vehicle body coordinate system pose determination method provided in the embodiment of the application, the rotation matrix parameter of the vehicle body coordinate system can be expressed as R c , R c is a 3x3 matrix, and the degree of freedom is 3; the translation vector parameter can be expressed as t c , t c is a column vector with a length of 3, and the degree of freedom is also 3. Therefore, the degree of freedom of the first type of pose information is 6, that is, there are 6 unknown quantities.

[0083] The coordinate information of the center point of the left front side wheel, the center point of the left rear side wheel, the center point of the right front side wheel and the center point of the right rear side wheel in the vehicle body coordinate system, that is, the first type of coordinate information, are respectively: Combined with the distance constraint between the center point of the left front side wheel, the center point of the left rear side wheel, the center point of the right front side wheel and the center point of the right rear side wheel, there is:

[0084] The first type of coordinate information of the center point of the left front side wheel:

[0085] The first type of coordinate information of the center point of the left rear side wheel:

[0086] The first type of coordinate information of the center point of the right front side wheel:

[0087] The first type of coordinate information of the center point of the right rear side wheel:

[0088] The coordinate conversion relationship between the world coordinate system and the vehicle body coordinate system can be expressed as:

[0089] p w = R c p c + t c

[0090] wherein p w is the second type coordinate information of any point p on the vehicle body, i.e., the coordinate information of p in the world coordinate system, and p c is the first type coordinate information of p, i.e., the coordinate information of p in the vehicle body coordinate system.

[0091] The plurality of coordinate conversion equations form the following equation group:

[0092]

[0093] The above equation group contains eight unknowns, including three in R c , three in t c , and l and d. However, since the first type coordinate information and the second type coordinate information are both three-dimensional, there are a total of 12 equations in the above equation group, which is an over-constrained problem, and each unknown has multiple values. Therefore, in the embodiments of the present application, the above equation group can be solved by the least square method, i.e., the following loss function L is constructed:

[0094]

[0095] wherein a is a constant, which can be 1 or 1 / 4, or other values, which are not limited here. The loss function can represent the error between the coordinate information in the world coordinate system corresponding to the first type coordinate information determined by the distance constraint between the key points of the vehicle body and the second type coordinate information.

[0096] By minimizing the above loss function L using an optimization method, the first type pose information of the vehicle body coordinate system, i.e., R c and t c , can be finally solved, and l and d can also be obtained.

[0097] On the basis of the above embodiments, the vehicle body coordinate system pose determination method provided in the embodiments of the present application comprises:

[0098] Obtaining a plurality of perspective images containing the key points of the vehicle body and the calibration plate;

[0099] Determining the first type coordinate information based on the plurality of perspective images in combination with a multi-view geometry method.

[0100] Specifically, in the embodiment of the present application, when determining the coordinate information of the body key point in the world coordinate system, a multi-view geometry method can be used for determination.

[0101] Firstly, a plurality of perspective images containing the body key point and the calibration board can be acquired. Here, the plurality of perspective images can be obtained by a position movable image acquisition device at a plurality of positions, and each position can obtain a perspective image. The image acquisition device can be mounted on a mobile auxiliary trolley, which can travel around the vehicle. The image acquisition device can also be handheld by a user, and the plurality of perspective images can be obtained by the user walking around the vehicle. The image acquisition device can be a camera or a camera, which is not limited here.

[0102] Each perspective image needs to include the body key point and the calibration board, and the calibration board is arranged around the vehicle, and the world coordinate system is constructed on the calibration board.

[0103] Then, according to the plurality of perspective images, the second type of coordinate information of the body key point, i.e. the coordinate information of the body key point in the world coordinate system, is determined by combining the multi-view geometry method. The multi-view geometry method refers to using a geometric method to recover a three-dimensional object from a plurality of two-dimensional images, i.e. three-dimensional reconstruction, which is mainly applied in computer vision. Three-dimensional reconstruction refers to how to recover the three-dimensional geometric structure information of an object from the images of the object at different viewpoints.

[0104] Since each perspective image contains the body key point and the calibration board, the body key point and the calibration board have corresponding coordinate information in the image coordinate system of the corresponding perspective image. The image coordinate system is a two-dimensional coordinate system. Further, by using the multi-view geometry method, the coordinate information of the body key point in the world coordinate system can be recovered from the plurality of perspective images.

[0105] In the embodiment of the present application, by determining the plurality of perspective images and combining the multi-view geometry method, the second type of coordinate information of the body key point can be automatically determined, the positioning accuracy of the body key point can be improved, and the accuracy of the first type of pose information of the body coordinate system can be ensured.

[0106] On the basis of the above embodiment, the body coordinate system pose determination method provided in the embodiment of the present application, the plurality of perspective images are obtained based on the image acquisition device at a plurality of positions, and the perspective image corresponds to the position one by one.

[0107] Correspondingly, the second type of coordinate information is determined based on the plurality of perspective images and combined with the multi-view geometry method, which comprises:

[0108] The pose information of the image acquisition device in the world coordinate system when capturing the multiple viewpoint images is determined, as well as the third type of coordinate information of the key points of the vehicle body; the third type of coordinate information is the coordinate information of the key points of the vehicle body in the multiple viewpoint images;

[0109] Based on the pose information of the image acquisition device in the world coordinate system and the third type of coordinate information, the relationship between the second type of coordinate information and the third type of coordinate information is determined;

[0110] Based on the relationship between the second type of coordinate information and the third type of coordinate information, and the third type of coordinate information, the second type of coordinate information is determined.

[0111] Specifically, in this embodiment of the invention, since multiple viewpoint images can be captured by an image acquisition device at multiple locations, with each location capturing one viewpoint image, when determining the second type of coordinate information of key points on the vehicle body using a multi-view geometry method, the pose information of the image acquisition device in the world coordinate system when capturing multiple viewpoint images can be determined first. That is, it is necessary to determine the pose information of the image acquisition device in the world coordinate system when capturing each viewpoint image, and the number of viewpoint images captured by the image acquisition device is equal to the number of pose information points of the image acquisition device in the world coordinate system that need to be obtained.

[0112] Since each viewpoint image contains a calibration board, a camera pose calibration algorithm based on the calibration board can be used to calculate the first-type pose information of the image acquisition device when each viewpoint image was captured. If the image acquisition device captures n viewpoint images at n (n≥2) positions, then n first-type pose information of the image acquisition device can be obtained, which can be represented as follows: In this embodiment of the invention, no specific limitations are made on the camera pose calibration algorithm based on the calibration board.

[0113] The j-th pose information of the image acquisition device in the world coordinate system can be used to represent the coordinate information of any point p in the world coordinate system. w The coordinate information p of the j-th viewpoint image in the image coordinate system ij The relationship between them satisfies the following relation:

[0114]

[0115] in, Let be the rotation matrix of the image coordinate system of the j-th viewpoint image. Let be the translation vector of the image coordinate system for the j-th viewpoint image.

[0116] In addition, the third type of coordinate information of the body key point needs to be determined, i.e., the coordinate information of the body key point in multiple view images. The coordinate information of the body key point in multiple view images is the coordinate information of the body key point in the image coordinate system of each view image.

[0117] Here, the coordinate information of the body key point in each view image can be determined by using a key point detection algorithm or manual labeling, and the coordinate information is a pixel coordinate.

[0118] Then, the relationship between the second type of coordinate information and the third type of coordinate information of the body key point can be determined by the pose information of the image acquisition device in the world coordinate system and the third type of coordinate information of the body key point, i.e., the relationship between the coordinate information of the body key point in the world coordinate system and the coordinate information in the image coordinate system.

[0119] For example, the above relationship can be expressed as:

[0120]

[0121] where p k is the second type of coordinate information of the body key point, i.e., the coordinate information of the body key point in the world coordinate system, p i is the third type of coordinate information of the body key point, i.e., the coordinate information of the body key point in the i-th view image, is the rotation matrix of the image coordinate system of the i-th view image, is the translation vector of the image coordinate system of the i-th view image.

[0122] K i is the intrinsic parameter of the image acquisition device when the i-th view image is taken, which can be calibrated by the intrinsic parameter calibration method of the camera, and the intrinsic parameter calibration method of the camera is not described here.

[0123] Finally, the second type of coordinate information of the body key point can be determined according to the above relationship and the third type of coordinate information, i.e., p k is determined. Since p i is a two-dimensional coordinate containing two degrees of freedom, and p k is a three-dimensional coordinate with three degrees of freedom, when n≥2, the above formula can obtain four equations, and p k can be successfully solved.

[0124] It should be noted that the image acquisition device used in the embodiment of the present application is an ideal camera model, and there is no image distortion. For the image acquisition device with distortion, the distortion coefficient can be calibrated by a camera calibration method, and then the distortion coefficient is used to perform anti-distortion on each view angle image, or the distortion model of each view angle image is brought into the subsequent calculation process, which will not be described here.

[0125] In the embodiment of the present application, the relationship between the second type of coordinate information and the third type of coordinate information is obtained by determining the pose information of the image acquisition device in the world coordinate system and the third type of coordinate information, and the second type of coordinate information of the key point of the vehicle body is determined in combination with the relationship. The method is simple and easy to implement, greatly reduces the complexity of determining the first type of pose information of the vehicle body coordinate system, and improves the determination efficiency. Moreover, the method only needs one calibration plate and one image acquisition device to determine the first type of pose information of the vehicle body coordinate system.

[0126] On the basis of the above-mentioned embodiment, the relationship between the second type of coordinate information and the third type of coordinate information is represented based on an equation set containing a plurality of equations in the vehicle body coordinate system pose determination method provided in the embodiment of the present application.

[0127] Correspondingly, based on the relationship between the second type of coordinate information and the third type of coordinate information, and the third type of coordinate information, the second type of coordinate information is determined, which comprises:

[0128] Based on the third type of coordinate information, the equation set is solved to obtain the second type of coordinate information.

[0129] Specifically, as known from the above-mentioned embodiment, the relationship between the second type of coordinate information and the third type of coordinate information can be represented by the above-mentioned formula. However, due to the measurement error and other factors, each equation in the equation set may not be established at the same time, so that p k Therefore, the third type of coordinate information p i of the key point of the vehicle body can be used to solve the above-mentioned equation set, that is, the equation set can be used to construct a least square method problem, that is:

[0130]

[0131] Wherein, J is the cost function of the least square method.

[0132] By minimizing the cost function, p k can be obtained.

[0133] In the embodiment of the present application, by solving the equation set, the situation that cannot be solved due to the measurement error and other factors can be avoided, and the correct solution of the first type of coordinate information of the key point of the vehicle body is ensured.

[0134] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination method provided in the embodiments of the present application, the first type of coordinate information, the second type of coordinate information and the third type of coordinate information are all homogeneous coordinate information.

[0135] Specifically, since the second type of coordinate information is three-dimensional coordinate and the third type of coordinate information is two-dimensional coordinate, for the convenience of subsequent calculation, the second type of coordinate information and the third type of coordinate information are both expressed as homogeneous coordinate information. That is, the value of the degree of freedom z in the second type of coordinate information is ensured to be 1, and the degree of freedom z is added to the third type of coordinate information, and the value of the degree of freedom z is also 1.

[0136] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination method provided in the embodiments of the present application, the first type of coordinate information, the second type of coordinate information and the third type of coordinate information are all homogeneous coordinate information.

[0137] Based on the first type of pose information and the second type of pose information of the sensing device for sensing the surrounding information of the vehicle, the third type of pose information of the sensing device is determined.

[0138] Among them, the second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0139] Specifically, in the embodiments of the present application, after determining the first type of coordinate information of the vehicle body coordinate system, the sensing device coordinate system of the sensing device for sensing the surrounding information of the vehicle can also be calibrated to determine the third type of pose information of the sensing device, that is, to determine the pose information of the sensing device in the world coordinate system. The sensing device can be various sensors, and the third type of pose information of the sensing device can be represented by the first rotation matrix R s and the first translation vector t s of the sensing device coordinate system, which are used to represent the relationship between the coordinate information p w of any point p in the world coordinate system and the coordinate information p s in the sensing device coordinate system: p w =R s p s +t s .

[0140] Here, the third type of pose information of the sensing device can be calculated by Zhang Zhengyou calibration method, which is not limited here.

[0141] Further, the third type of pose information of the sensing device can be determined by the pose information of the vehicle body coordinate system in the world coordinate system and the third type of pose information of the sensing device, i.e., the pose information of the sensing device in the vehicle body coordinate system.

[0142] The third type of pose information of the sensing device can be represented by a second rotation matrix and a second translation vector , which are used to represent the relationship between the coordinate information p c of any point p in the vehicle body coordinate system and the coordinate information p s of the point p in the sensing device coordinate system, and satisfy the following relationship:

[0143] wherein, and can be calculated by the following formula:

[0144]

[0145]

[0146] In the embodiment, the third type of pose information of the sensing device for sensing the surrounding information of the vehicle can be further determined, and the sensing result of the sensing device can be used to assist the vehicle in decision-making and motion planning.

[0147] As shown in Figure 3 , on the basis of the above embodiment, the vehicle body coordinate system pose determination device provided in the embodiment comprises:

[0148] A coordinate information determination module 31 is configured to determine first type of coordinate information of each vehicle body key point based on the distance constraint between the vehicle body key points; the first type of coordinate information is the coordinate information of the vehicle body key points in the vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points.

[0149] A pose information determination module 32 is configured to determine second type of coordinate information of the vehicle body key points, and determine first type of pose information of the vehicle body coordinate system based on the first type of coordinate information and the second type of coordinate information; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; and the second type of coordinate information is the coordinate information of the vehicle body key points in the world coordinate system.

[0150] On the basis of the above embodiment, the vehicle body coordinate system pose determination device provided in the embodiment is characterized in that the first type of pose information is represented based on the rotation matrix parameter and the translation vector parameter of the vehicle body coordinate system; accordingly,

[0151] The pose information determination module is configured to:

[0152] Determine a plurality of coordinate conversion equations between the vehicle body coordinate system and the world coordinate system based on the first type of coordinate information, the second type of coordinate information, the rotation matrix parameter, and the translation vector parameter.

[0153] Solve an equation group constructed by the plurality of coordinate conversion equations to obtain the value of the rotation matrix parameter and the value of the translation vector parameter.

[0154] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination device provided in the embodiments of the present application, the pose information determination module is specifically configured to:

[0155] Construct a loss function based on the equation group; the loss function can be used to represent the error of the coordinate conversion relationship of the coordinate system corresponding to the first type of coordinate information and the second type of coordinate information;

[0156] Solve the rotation matrix parameter and the translation vector parameter to obtain the value of the rotation matrix parameter and the value of the translation vector parameter, with the goal of minimizing the loss function.

[0157] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination device provided in the embodiments of the present application, the distance constraint includes that the distance information of the vehicle body key points in the world coordinate system is equal to the distance information of the vehicle body key points in the vehicle body coordinate system, and both are constant values.

[0158] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination device provided in the embodiments of the present application, the vehicle body key points include the center points of the wheels of the vehicle.

[0159] Correspondingly, the distance information includes the distance information between the wheels of the vehicle in the length direction and the distance information between the wheels of the vehicle in the width direction.

[0160] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination device provided in the embodiments of the present application, the pose information determination module is further specifically configured to:

[0161] Obtain a plurality of perspective images containing the vehicle body key points and a calibration board;

[0162] Determine the first type of coordinate information based on the plurality of perspective images and in combination with a multi-view geometry method.

[0163] On the basis of the above-mentioned embodiments, the vehicle body coordinate system pose determination device provided in the embodiments of the present application, the pose information determination module is further specifically configured to:

[0164] Based on the first type of pose information and the second type of pose information of the sensing device used to perceive the surrounding information of the vehicle, the third type of pose information of the sensing device is determined.

[0165] The second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

[0166] Specifically, the functions of each module in the vehicle coordinate system pose determination device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0167] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute the vehicle body coordinate system pose determination method provided in the above embodiments. The method includes: determining a first type of coordinate information of each vehicle body key point based on the distance constraints between each vehicle body key point; the first type of coordinate information is the coordinate information of each vehicle body key point in the vehicle body coordinate system, which is constructed based on each vehicle body key point; determining a second type of coordinate information of each vehicle body key point, and determining a first type of pose information of the vehicle body coordinate system based on the first type of coordinate information and the second type of coordinate information; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; the second type of coordinate information is the coordinate information of each vehicle body key point in the world coordinate system.

[0168] Further, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0169] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute a vehicle body coordinate system pose determination method, the method comprising: determining first type coordinate information of each vehicle body key point of a vehicle based on distance constraints between the vehicle body key points; the first type coordinate information is coordinate information of the vehicle body key points in a vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points; determining second type coordinate information of the vehicle body key points, and determining first type pose information of the vehicle body coordinate system based on the first type coordinate information and the second type coordinate information; the first type pose information is pose information of the vehicle body coordinate system in a world coordinate system; and the second type coordinate information is coordinate information of the vehicle body key points in the world coordinate system.

[0170] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement a vehicle body coordinate system pose determination method, the method comprising: determining first type coordinate information of each vehicle body key point of a vehicle based on distance constraints between the vehicle body key points; the first type coordinate information is coordinate information of the vehicle body key points in a vehicle body coordinate system, and the vehicle body coordinate system is constructed based on the vehicle body key points; determining second type coordinate information of the vehicle body key points, and determining first type pose information of the vehicle body coordinate system based on the first type coordinate information and the second type coordinate information; the first type pose information is pose information of the vehicle body coordinate system in a world coordinate system; and the second type coordinate information is coordinate information of the vehicle body key points in the world coordinate system.

[0171] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0172] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0173] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining the pose of a vehicle body coordinate system, characterized in that, include: Based on the distance constraints between various key points of the vehicle body, the first type of coordinate information of each key point of the vehicle body is determined; the first type of coordinate information is the coordinate information of each key point of the vehicle body in the vehicle body coordinate system, which is constructed based on each key point of the vehicle body. The second type of coordinate information of each key point of the vehicle body is determined, and based on the first type of coordinate information and the second type of coordinate information, the first type of pose information of the vehicle body coordinate system is determined; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; the second type of coordinate information is the coordinate information of each key point of the vehicle body in the world coordinate system. The distance constraint includes the fact that the distance information between each key point of the vehicle body in the world coordinate system is equal to the distance information between each key point of the vehicle body in the vehicle body coordinate system, and both are constant values; the first type of pose information is represented based on the rotation matrix parameters and translation vector parameters of the vehicle body coordinate system; correspondingly, The step of determining the first type of pose information of the vehicle coordinate system based on the first type of coordinate information and the second type of coordinate information includes: Based on the first type of coordinate information, the second type of coordinate information, the rotation matrix parameters, and the translation vector parameters, multiple coordinate transformation equations between the vehicle coordinate system and the world coordinate system are determined. Based on the system of equations constructed from the multiple coordinate transformation equations, a loss function is constructed; the loss function can be used to characterize the error in the coordinate transformation relationship between the first type of coordinate information and the second type of coordinate information. With the goal of minimizing the loss function, the rotation matrix parameters and the translation vector parameters are solved to obtain the values ​​of the rotation matrix parameters and the translation vector parameters.

2. The method for determining the pose of the vehicle body coordinate system according to claim 1, characterized in that, The key points of the vehicle body include the center points of each wheel of the vehicle; Accordingly, the distance information includes the distance information between each wheel of the vehicle in the length direction and the distance information between each wheel of the vehicle in the width direction.

3. The method for determining the pose of the vehicle body coordinate system according to any one of claims 1-2, characterized in that, The determination of the second type of coordinate information for each key point on the vehicle body includes: Acquire multiple perspective images containing the key points of each vehicle body and the calibration plate; Based on the multiple perspective images, and combined with multi-view geometry methods, the first type of coordinate information is determined.

4. The method for determining the pose of the vehicle body coordinate system according to any one of claims 1-2, characterized in that, The step of determining the first type of pose information of the vehicle coordinate system based on the first type of coordinate information and the second type of coordinate information further includes: Based on the first type of pose information and the second type of pose information of the sensing device used to perceive the surrounding information of the vehicle, the third type of pose information of the sensing device is determined. The second type of pose information is the pose information of the sensing device in the world coordinate system, and the third type of pose information is the pose information of the sensing device in the vehicle body coordinate system.

5. A vehicle body coordinate system pose determination device, characterized in that, include: The coordinate information determination module is used to determine the first type of coordinate information of each key body point based on the distance constraints between each key body point of the vehicle; the first type of coordinate information is the coordinate information of each key body point in the vehicle coordinate system, which is constructed based on each key body point. The pose information determination module is used to determine the second type of coordinate information of each key point of the vehicle body, and to determine the first type of pose information of the vehicle body coordinate system based on the first type of coordinate information and the second type of coordinate information; the first type of pose information is the pose information of the vehicle body coordinate system in the world coordinate system; the second type of coordinate information is the coordinate information of each key point of the vehicle body in the world coordinate system. The distance constraint includes the fact that the distance information between each key point of the vehicle body in the world coordinate system is equal to the distance information between each key point of the vehicle body in the vehicle body coordinate system, and both are constant values; the first type of pose information is represented based on the rotation matrix parameters and translation vector parameters of the vehicle body coordinate system; correspondingly, The pose information determination module is specifically used for: Based on the first type of coordinate information, the second type of coordinate information, the rotation matrix parameters, and the translation vector parameters, multiple coordinate transformation equations between the vehicle coordinate system and the world coordinate system are determined. Based on the system of equations constructed from the multiple coordinate transformation equations, a loss function is constructed; the loss function can be used to characterize the error in the coordinate transformation relationship between the first type of coordinate information and the second type of coordinate information. With the goal of minimizing the loss function, the rotation matrix parameters and the translation vector parameters are solved to obtain the values ​​of the rotation matrix parameters and the translation vector parameters.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the vehicle coordinate system pose determination method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle body coordinate system pose determination method as described in any one of claims 1 to 4.

Images