True value vehicle body coordinate acquisition method and device, equipment, medium and program product

Abstract

The invention discloses a true value vehicle body coordinate obtaining method and device, equipment, a medium and a program product, and relates to the technical field of intelligent driving. The method comprises the following steps of: acquiring marked true value vehicle body coordinates of a plurality of first data frames in a plurality of original data frames, wherein the first data frames are data frames marked with the true value vehicle body coordinates in a vehicle body coordinate system; determining a reference odometer coordinate in an odometer coordinate system according to each vehicle body coordinate marked with the true value and the relative positioning data; according to each reference speedometer coordinate, carrying out coordinate filling on at least one second data frame in the plurality of original data frames to obtain a filling speedometer coordinate of the at least one second data frame, each second data frame being a data frame in the plurality of original data frames, which is not marked with a true value vehicle body coordinate; and converting the filling odometer coordinates of the at least one second data frame into a vehicle body coordinate system to obtain predicted true value vehicle body coordinates. According to the invention, the number of true-value vehicle body coordinates can be expanded, and the labor cost is saved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method and device, equipment, medium, and program product for obtaining true value vehicle body coordinates. Background Art

[0002] When driving a vehicle, by detecting the lane lines on the road, it can be ensured that the vehicle is driving in the correct area and ensure driving safety. At present, detection algorithms are usually used to evaluate lane lines, and the evaluation is based on the true value vehicle coordinates and original data marked in the vehicle coordinate system. However, for the feature points on the lane lines, the number of marked true value vehicle coordinates is relatively small, while the number of original data frames is very large. It takes a lot of manpower costs to manually mark a large number of original data frames. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a method and device, equipment, medium, and program product for obtaining true value vehicle body coordinates, which can expand the number of true value vehicle body coordinates and save labor costs.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for obtaining true value vehicle body coordinates, the method comprising:

[0005] Acquire the annotated true value vehicle body coordinates of a plurality of first data frames among the plurality of original data frames, wherein the first data frames are data frames annotated with the true value vehicle body coordinates in the vehicle body coordinate system;

[0006] Determining a reference odometer coordinate in an odometer coordinate system according to each of the annotated true value vehicle body coordinates and the relative positioning data associated with the plurality of original data frames;

[0007] Filling coordinates of at least one second data frame among the plurality of original data frames according to each of the reference odometer coordinates to obtain filled odometer coordinates of the at least one second data frame, wherein each of the second data frames is a data frame in the plurality of original data frames without annotated true value vehicle body coordinates;

[0008] The filled odometer coordinates of the at least one second data frame are converted to a vehicle body coordinate system to obtain predicted true value vehicle body coordinates of the at least one second data frame.

[0009] Optionally, determining the reference odometer coordinates in the odometer coordinate system according to each of the annotated true value vehicle body coordinates and the relative positioning data associated with the plurality of original data frames includes:

[0010] Generate a first transformation matrix according to a posture relationship between a first timestamp of the first data frame and a first adjacent timestamp in the relative positioning data, where the first transformation matrix is ​​a transformation matrix from a vehicle body coordinate system to an odometer coordinate system, and the first adjacent timestamp is a timestamp in the relative positioning data that is adjacent to the first timestamp;

[0011] Coordinate transformation is performed according to the labeled true value vehicle body coordinates and the first transformation matrix to obtain the reference odometer coordinates.

[0012] Optionally, the first adjacent timestamp includes a first adjacent before timestamp and a first adjacent after timestamp;

[0013] The step of generating a first transformation matrix according to a posture relationship between a first timestamp of the first data frame and a first adjacent timestamp in the relative positioning data comprises:

[0014] Extracting, from the relative positioning data according to the first timestamp, the first adjacent front timestamp and the first front pose parameter data of the first adjacent front timestamp, and the first adjacent rear timestamp and the first rear pose parameter data of the first adjacent rear timestamp;

[0015] A first transformation matrix is ​​calculated based on the first timestamp, the first adjacent previous timestamp, the first adjacent subsequent timestamp, the first previous posture parameter data, and the first subsequent posture parameter data.

[0016] Optionally, calculating a first transformation matrix according to the first timestamp, the first adjacent front timestamp, the first adjacent rear timestamp, the first front pose parameter data, and the first rear pose parameter data includes:

[0017] Calculating a true value interpolation weight according to the first timestamp, the first adjacent previous timestamp, and the first adjacent subsequent timestamp;

[0018] A matrix is ​​constructed according to the true value interpolation weight, the first front posture parameter data and the first rear posture parameter data to obtain the first transformation matrix.

[0019] Optionally, calculating a true value interpolation weight according to the first timestamp, the first adjacent previous timestamp, and the first adjacent subsequent timestamp includes:

[0020] Performing a difference calculation based on the first timestamp and the first adjacent previous timestamp to obtain a first difference;

[0021] Performing a difference calculation based on the first adjacent rear timestamp and the first adjacent front timestamp to obtain a second difference;

[0022] The true value interpolation weight is obtained by performing a ratio calculation based on the first difference and the second difference.

[0023] Optionally, the first front posture parameter data includes a first front position coordinate and a first front posture angle, and the first rear posture parameter data includes a first rear position coordinate and a first rear posture angle;

[0024] The step of constructing a matrix according to the true value interpolation weight, the first front pose parameter data, and the first rear pose parameter data to obtain the first transformation matrix includes:

[0025] Calculating a translation interpolation vector according to the true value interpolation weight, the first front position coordinate, and the first rear position coordinate;

[0026] Calculating a rotation interpolation vector according to the true value interpolation weight, the first front posture angle, and the first rear posture angle;

[0027] Calculating a first rotation matrix according to the rotation interpolation vector;

[0028] A unit matrix is ​​constructed as an initial transformation matrix, the first rotation matrix is ​​filled into the upper right submatrix of the initial transformation matrix, and the translation interpolation vector is filled into the last column of the initial transformation matrix to obtain the first transformation matrix.

[0029] Optionally, calculating a first rotation matrix according to the rotation interpolation vector includes:

[0030] Reading variables of the rotation interpolation vector to obtain roll angle variables, pitch angle variables and yaw angle variables;

[0031] Calculating a first rotation submatrix rotating around a first axis according to the roll angle variable;

[0032] Calculating a second rotation submatrix rotating around a second axis according to the pitch angle variable;

[0033] Calculating a third rotation submatrix rotating around a third axis according to the yaw angle variable;

[0034] Combining the first rotation submatrix, the second rotation submatrix and the third rotation submatrix to obtain the first rotation matrix;

[0035] The first axis, the second axis and the third axis are perpendicular to each other.

[0036] Optionally, performing coordinate transformation according to the annotated true value vehicle body coordinates and the first transformation matrix to obtain the reference odometer coordinates includes:

[0037] Add an element after the labeled true value vehicle body coordinate to obtain a homogeneous coordinate representation;

[0038] Multiplying the first transformation matrix by the homogeneous coordinate representation to obtain a homogeneous transformed coordinate representation;

[0039] The elements of the last dimension represented by the homogeneous transformation coordinates are removed to obtain the reference odometer coordinates.

[0040] Optionally, the performing coordinate filling on at least one second data frame among the plurality of original data frames according to each of the reference odometer coordinates to obtain the filled odometer coordinates of the at least one second data frame includes:

[0041] Matching the timestamp of each first data frame with the timestamp of each original data frame to generate a filled timestamp column;

[0042] Rename the timestamp of each of the original data frames to the filled timestamp column;

[0043] Left-connect the original data frame with the reference odometer coordinates according to the filled timestamp column to obtain a true odometer coordinate column;

[0044] According to each of the reference odometer coordinates in the true odometer coordinate column, the missing values ​​corresponding to at least one second data frame are forward filled to obtain the filled odometer coordinates of the at least one second data frame.

[0045] Optionally, renaming the timestamp of each of the original data frames into the filled timestamp column includes:

[0046] Determine a minimum true value timestamp and a maximum true value timestamp from the timestamps of each of the first data frames;

[0047] Filtering the original data frame interval to be filled from the plurality of original data frames according to the minimum true value timestamp and the maximum true value timestamp;

[0048] Rename the timestamp of the original data frame interval to be filled to the filling timestamp column;

[0049] After forward filling, the filled odometer coordinates of at least one second data frame in the interval of the original data frame to be filled are obtained.

[0050] Optionally, converting the filled odometer coordinates of the at least one second data frame into a vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of the at least one second data frame includes:

[0051] Generate a transformation matrix from the vehicle body coordinate system to the odometer coordinate system according to the posture relationship between the second timestamp of the second data frame and the second adjacent positioning timestamp in the relative positioning data to obtain a second transformation matrix; the second adjacent positioning timestamp is a timestamp in the relative positioning data that is adjacent to the second timestamp;

[0052] Coordinate transformation is performed according to the filled odometer coordinates of the at least one second data frame and the inverse matrix of the second transformation matrix to obtain the predicted true value vehicle body coordinates.

[0053] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for acquiring true value vehicle body coordinates, the device comprising:

[0054] A marked true value acquisition module, used to acquire the marked true value vehicle body coordinates of a plurality of first data frames in the plurality of original data frames, wherein the first data frames are data frames with marked true value vehicle body coordinates in a vehicle body coordinate system;

[0055] A first coordinate conversion module, configured to determine a reference odometer coordinate in an odometer coordinate system according to each of the annotated true value vehicle body coordinates and relative positioning data associated with the plurality of original data frames;

[0056] A coordinate filling module, configured to fill the coordinates of at least one second data frame in the plurality of original data frames according to each of the reference odometer coordinates, to obtain the filled odometer coordinates of the at least one second data frame, wherein each of the second data frames is a data frame in the plurality of original data frames without annotated true value vehicle body coordinates;

[0057] The second coordinate conversion module is used to convert the filled odometer coordinates of the at least one second data frame into a vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of the at least one second data frame.

[0058] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0059] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0060] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a computer program product, which includes a computer program, and the computer program is read and executed by a processor of an electronic device, so that when the processor executes the computer program, the method described in the first aspect is implemented.

[0061] The method and device for acquiring the true value vehicle body coordinates, the electronic device, the storage medium, and the program product proposed in the present application aim at the problem that the related technology needs to consume a lot of manpower to expand the true value vehicle body coordinates so as to achieve the problem that the number of frames of the original data frame is basically consistent with the number of the true value vehicle body coordinates, thereby resulting in the low efficiency of acquiring the true value vehicle body coordinates. In the present application, the annotated true value vehicle body coordinates of multiple first data frames (the first data frame is a data frame annotated with the true value vehicle body coordinates in the vehicle body coordinate system) in multiple original data frames are firstly acquired, so that the multiple original data frames can be effectively distinguished, the annotated first data frame is determined, and the annotated true value vehicle body coordinates are determined. Furthermore, according to each annotated true value vehicle body coordinate and the relative positioning data associated with the multiple original data frames, the reference odometer coordinates in the odometer coordinate system are determined, so that the annotated true value vehicle body coordinates can be converted based on the positioning data of different data frames in the relative positioning data to obtain the reference odometer coordinates. Furthermore, according to each benchmark odometer coordinate, coordinates of at least one second data frame (the second data frame is a data frame in the multiple original data frames that is not labeled with true value vehicle body coordinates) in the multiple original data frames are filled to obtain the filled odometer coordinates of at least one second data frame, and then the filled odometer coordinates of at least one second data frame are converted to the vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of at least one second data frame. In this way, the predicted true value vehicle body coordinates of the second data frame can be obtained without additional labeling. In summary, the present application introduces relative positioning data and conversion between the vehicle coordinate system and the odometer coordinate system to achieve the predicted true value vehicle body coordinates based on the expansion of the existing labeled true value vehicle body coordinates. There is no need to spend manpower to label the data frames that do not have labeled true value vehicle body coordinates. On the basis of being able to expand the number of true value vehicle body coordinates, the acquisition efficiency is improved and the labor cost is saved.

[0062] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of a method for obtaining true value vehicle body coordinates provided in an embodiment of the present application;

[0064] Figure 2 yes Figure 1 Flowchart of step 102;

[0065] Figure 3 yes Figure 2 Flow chart of step 201;

[0066] Figure 4 yes Figure 3 Flowchart of step 302;

[0067] Figure 5 yes Figure 4 Flowchart of step 402;

[0068] Figure 6 yes Figure 1 Flow chart of step 103;

[0069] Figure 7 yes Figure 1 Flow chart of step 104;

[0070] Figure 8 It is a structural schematic diagram of the true value vehicle body coordinate acquisition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0074] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0075] The true value vehicle body coordinate acquisition method provided in the embodiment of the present application can be applied to any end of the terminal and the server end, and can also be software running on the server end or the terminal. The server end can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application or computer program that implements the true value vehicle body coordinate acquisition method, but is not limited to the above forms.

[0076] The true value vehicle body coordinate acquisition method, true value vehicle body coordinate acquisition device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application are specifically explained through the following embodiments. First, the true value vehicle body coordinate acquisition method in the embodiments of the present application is described.

[0077] Please refer to Figure 1 , Figure 1 An optional flow chart of a method for acquiring true value vehicle body coordinates is disclosed. Figure 1 The method may include but is not limited to steps 101 to 104.

[0078] Step 101, obtaining the labeled true value vehicle body coordinates of a plurality of first data frames in a plurality of original data frames;

[0079] Step 102, determining a reference odometer coordinate in an odometer coordinate system according to each annotated true value vehicle body coordinate and relative positioning data associated with a plurality of original data frames;

[0080] Step 103, performing coordinate filling on at least one second data frame among the multiple original data frames according to each reference odometer coordinate, to obtain filled odometer coordinates of at least one second data frame, wherein each second data frame is a data frame without true value vehicle body coordinates marked among the multiple original data frames;

[0081] Step 104 , converting the filled odometer coordinates of at least one second data frame into a vehicle body coordinate system to obtain predicted true value vehicle body coordinates of at least one second data frame.

[0082] Steps 101 to 104 shown in the embodiment of the present application, by introducing relative positioning data and conversion between the vehicle coordinate system and the odometer coordinate system, achieve expansion based on the existing annotated true value vehicle coordinates to obtain predicted true value vehicle coordinates. There is no need to waste manpower to annotate data frames that do not have annotated true value vehicle coordinates. On the basis of being able to expand the number of true value vehicle coordinates, the acquisition efficiency is improved and the labor cost is saved.

[0083] In step 101, the raw data frame refers to the data frame collected by the data acquisition device / data acquisition system. The raw data frame is usually not cleaned, processed or analyzed. In one example, the raw data frame can be obtained from the DB3 recorded by various topic messages sent by the robot operating system (ROS) during the driving process of the vehicle. The raw data frame includes lane line feature points, so the raw data frame can be called a lane line data frame, and the annotated true value vehicle body coordinates are specifically the annotated true value vehicle body coordinates corresponding to the lane line feature points.

[0084] The first data frame is a data frame with true value body coordinates annotated in the body coordinate system. The body coordinate system usually takes the front end or center of gravity of the vehicle as the origin, the X-axis is along the vehicle's forward direction, the Y-axis is perpendicular to the vehicle's travel direction, and the Z-axis usually points upward. The body coordinate system is used to describe the motion state of the vehicle in its own coordinate system. For example, at least one data frame among the multiple original data frames is annotated in the body coordinate system to obtain the annotated true value body coordinates of at least one first data frame.

[0085] In one embodiment, step 101 may include:

[0086] Perform frame extraction on the multiple original data frames according to a preset frame interval to obtain at least two first data frames, where the preset frame interval is between two adjacent first data frames;

[0087] Each first data frame is labeled in the vehicle body coordinate system to obtain the labeled true value vehicle body coordinates of the first data frame.

[0088] In one example, the preset frame interval is 3, and a plurality of original data frames are represented as {Z 1 , Z 2 , Z 3 , Z 4 , Z 5 , Z 6 , Z 7 , Z 8 , Z 9 , Z 10 , Z 11 , Z 12}, then 4 first data frames can be obtained, and are expressed as {Z 1b , Z 4b , Z 7b , Z 10b}. Among them, Z 1b , Z 4b , Z 7b , Z 10b Z 1 The corresponding first data frame, Z 4 The corresponding first data frame, Z 7 The corresponding first data frame, Z 10The corresponding first data frame. 1 , Z 4 , Z 7 , Z 10 Mark in the vehicle coordinate system and get Z 1b The labeled true value of the vehicle body coordinates is (x 1b ,y 1b , z 1b ), Z 4b The labeled true value of the vehicle body coordinates is (x 4b ,y 4b , z 4b ), Z 7b The labeled true value of the vehicle body coordinates is (x 7b ,y 7b , z 7b ), Z 10b The labeled true value of the vehicle body coordinates is (x 10b ,y 10b , z 10b ).

[0089] The benefit of the above embodiment is that it can achieve an even distribution of the multiple first data frames in the multiple original data frames, thereby improving the fairness of labeling.

[0090] It should be noted that, in addition to extracting the first data frame through a preset frame interval as described above, the first data frame may also be obtained by randomly extracting multiple original data frames, or other extraction methods may be used, which is not specifically limited in the present application.

[0091] In step 102, relative positioning data is used to characterize relative poses associated with multiple raw data frames. Relative pose refers to the position and orientation of an object relative to another object. In this embodiment, relative pose can also be understood as the position and orientation of one raw data frame relative to another raw data frame. The so-called association means that there is a relative pose corresponding to the timestamp of the raw data frame in the relative positioning data.

[0092] For example, for multiple original data frames {Z 1 , Z 2 , Z 3 , Z 4 , Z 5 , Z 6 , Z 7 , Z 8 , Z 9 , Z 10 , Z 11 , Z 12}, the relative positioning data includes 1 The relative pose associated with Z 2The relative pose associated with Z 3 The relative pose associated with Z 4 The relative pose associated with Z 5 The relative pose associated with Z 6 The relative pose associated with Z 7 The relative pose associated with Z 8 The relative pose associated with Z 9 The relative pose associated with Z 10 The relative pose associated with Z 11 The relative pose associated with Z 12 The relative pose of the association.

[0093] It should be noted that in three-dimensional space, relative position can be expressed using translation vectors and rotation matrices. Translation vectors represent the displacement of an object on three coordinate axes (x, y, z). Rotation matrices represent the change in the orientation of an object and are used to describe the rotation of an object in three-dimensional space.

[0094] The purpose of executing step 102 is to convert the annotated true value vehicle body coordinates of the first data frame into the odometer coordinate system using the relative poses of multiple original data frames in the relative positioning data to obtain the reference odometer coordinates. The odometer coordinate system is usually a global coordinate system used to describe the position and posture of the vehicle in the environment. The origin of the odometer coordinate system can be the starting point, the Z axis usually points upward, and the X axis usually extends along the forward direction, and the Y axis is perpendicular to the forward direction.

[0095] For example, for Z in the above example 1b The labeled true value of the vehicle body coordinates is (x 1b ,y 1b , z 1b ), Z 4b The labeled true value of the vehicle body coordinates is (x 4b ,y 4b , z 4b ), Z 7b The labeled true value of the vehicle body coordinates is (x 7b ,y 7b , z 7b ), Z 10b The labeled true value of the vehicle body coordinates is (x 10b ,y 10b , z 10b ), after conversion to the odometer coordinate system, we can get Z 1b The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 4b The base odometer coordinates are (x 4j ,y 4j , z4j ), Z 7b The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 10b The base odometer coordinates are (x 10j ,y 10j , z 10j ). The coordinates with the subscript "b" refer to the coordinates marked in the vehicle coordinate system, and the coordinates with the subscript "j" refer to the coordinates converted from the marked true value vehicle coordinates in the odometer coordinate system.

[0096] The following describes the specific process of converting the labeled true value vehicle body coordinates to the odometer coordinate system.

[0097] In one embodiment, referring to Figure 2 , step 102 may include:

[0098] Step 201, generating a first transformation matrix according to a posture relationship between a first timestamp of a first data frame and a first adjacent timestamp in relative positioning data;

[0099] Step 202 , performing coordinate transformation according to the annotated true value vehicle body coordinates and the first transformation matrix to obtain reference odometer coordinates.

[0100] In step 201, the first transformation matrix is ​​a transformation matrix from the vehicle body coordinate system to the odometer coordinate system. The first adjacent timestamp is a timestamp in the relative positioning data that is adjacent to the first timestamp. The first adjacent timestamp includes a first adjacent before timestamp and a first adjacent after timestamp. The first adjacent before timestamp is a timestamp in the relative positioning data that is adjacent to the first timestamp and before the first timestamp. The first adjacent after timestamp is a timestamp in the relative positioning data that is adjacent to the first timestamp and after the first timestamp. The first adjacent before timestamp, the first timestamp, and the first adjacent after timestamp are incremented in sequence.

[0101] It should be noted that the relative positioning data includes multiple continuous relative postures, and the timestamps of the multiple continuous relative postures are incremented. The multiple relative postures at least include the relative postures associated with multiple raw data frames. For example, in combination with the above example, there are 12 raw data frames in total, then the relative positioning data includes the relative postures corresponding to the timestamps of the 12 raw data frames. Alternatively, because the acquisition frequency of the positioning data is usually greater than the acquisition frequency of the raw data frames, the relative positioning data includes not only the relative postures corresponding to the timestamps of the 12 raw data frames, but also the timestamps between the timestamps of two adjacent raw data frames (such as Z 1 Timestamp and Z 2 The timestamp between the timestamps of Z 2 Timestamp and Z 3The relative pose corresponding to the timestamp between the timestamps, etc.

[0102] In one example, the timestamp of the relative positioning data includes t 0 ,t 1 ,t 2 ,t 3 ,t 4 ,t 5 ,t 6 ,t 7 ,t 8 ,t 9 ,t 10 ,t 11 and t 12 If the first timestamp is t 4 , then the first adjacent previous timestamp is t 3 The timestamp of the first adjacent 5 If the first timestamp is t 8 , then the first adjacent previous timestamp is t 7 The timestamp of the first adjacent 9 .

[0103] In step 202, after obtaining the first transformation matrix, the first transformation matrix is ​​used to perform coordinate transformation on the labeled true value vehicle body coordinates to obtain the reference odometer coordinates.

[0104] The benefit of the above-mentioned embodiment of step 201 to step 202 is that the first transformation matrix can be generated for each labeled true value vehicle body coordinate and the coordinate conversion can be performed separately. The processing logic is simple and easy to implement, which helps to improve processing efficiency.

[0105] In one embodiment, referring to Figure 3 , step 201 may include:

[0106] Step 301, extracting, from the relative positioning data according to the first timestamp, the first adjacent front timestamp and the first front pose parameter data of the first adjacent front timestamp, and the first adjacent rear timestamp and the first rear pose parameter data of the first adjacent rear timestamp;

[0107] Step 302, calculating a first transformation matrix according to the first timestamp, the first adjacent previous timestamp, the first adjacent subsequent timestamp, the first previous posture parameter data and the first subsequent posture parameter data.

[0108] In step 301, it can be seen from the above that the relative positioning data includes relative postures corresponding to multiple timestamps, so the parameters required for calculating the transformation matrix can be extracted from the relative positioning data according to the first timestamp, that is, the first adjacent previous timestamp and the first previous posture parameter data, and the first adjacent subsequent timestamp and the first subsequent posture parameter data. For example, in combination with the above example, if the first timestamp is t4 , then extract from the relative positioning data, t 3 and t 3 The corresponding relative position and t 5 and t 5 The corresponding relative position.

[0109] In step 302, a first transformation matrix may be calculated based on the parameters required for calculating the transformation matrix extracted in step 301.

[0110] The benefit of the above-mentioned embodiment of steps 301 to 302 is that by using the first timestamp, the first adjacent front timestamp, the first adjacent rear timestamp, the first front pose parameter data and the first rear pose parameter data as the required parameters for calculating the transformation matrix, the pose relationship between the first timestamp and the first adjacent timestamp can be effectively utilized, thereby improving the calculation accuracy of the transformation matrix.

[0111] The implementation process of the above step 201 (including step 301 to step 302) to step 202 can be shown in the following code:

[0112] {#timestamp is the first timestamp of the first data frame

[0113] #truth_pt_vcs is the point in the vehicle coordinate system

[0114] #truth_pt_vcs=[[x1,y1,z1],[x2,y2,z2],...]

[0115] #Translation vector and rotation angle, t1 <t<t2

[0116] #transforms=[[[t1_x,t1_y,t1_z],[t1_roll,t1_pitch,t1_yaw]],[[t2_x,t2_y,t2_z],[t2_r oll,t2_pitch,t2_yaw]]]

[0117] #time=[t1,t2] timestamps of adjacent frames

[0118] def vehicle_point_to_odom(time,truth_pt_vcs,time,transforms):

[0119] t1 = time[0];

[0120] t2 = time[1];

[0121] t = timestamp;

[0122] translate_1=np.arry(transforms[0][0]);

[0123] rotation_1=np.arry(transforms[0][1]);

[0124] translate_2=np.arry(transforms[1][0]);

[0125] rotation_2=np.arry(transforms[1][1]);

[0126] veh_point=np.arry(truth_pt_vcs);

[0127] # Calculate the transformation matrix

[0128] transform_v2o=calulate_transform(t,t1,t2,translate_1,rotation_1,translate_2,rotation_2);

[0129] odom_point=transform_point(veh_point, transform_v2o)}.

[0130] The above code can be understood as: Get the relationship between the given timestamp t and the adjacent previous and next frame timestamps t1 and t2. Convert the input translation vector and rotation angle into NumPy arrays. Create a NumPy array to store the points in the vehicle coordinate system (veh_points). Use the calculate_transform function to calculate the input parameters (i.e. t, t1, t2, translate_1, rotation_1, translate_2, rotation_2) to obtain the transformation matrix (transform_v2o) from the vehicle coordinate system to the odometer coordinate system. By calling the transform_points function, the input parameters (i.e. veh_point, transform_v2o) are transformed to convert the points in the vehicle coordinate system to the points in the odometer coordinate system (odom_points). Finally, return the converted points in the odometer coordinate system.

[0131] In one embodiment, referring to Figure 4 , step 302 may include:

[0132] Step 401, calculating a true value interpolation weight according to a first timestamp, a first adjacent previous timestamp, and a first adjacent subsequent timestamp;

[0133] Step 402, constructing a matrix according to the true value interpolation weight, the first front pose parameter data and the first rear pose parameter data to obtain a first transformation matrix.

[0134] In step 401, the true value interpolation weight is used to represent the ratio of the gap between the first timestamp and the first adjacent previous timestamp to the gap between the first adjacent subsequent timestamp and the first adjacent previous timestamp.

[0135] In one embodiment, step 401 may include:

[0136] Calculate the difference between the first timestamp and the first adjacent previous timestamp to obtain a first difference;

[0137] Calculate the difference between the first adjacent rear timestamp and the first adjacent front timestamp to obtain a second difference;

[0138] A true value interpolation weight is obtained by calculating a ratio of the first difference to the second difference.

[0139] For example, the true value interpolation weight alpha=t-t1 / t2-t1, where t represents the first timestamp, t1 represents the first adjacent previous timestamp, and t2 represents the first adjacent subsequent timestamp.

[0140] The benefit of the above embodiment is that the true value interpolation weight is determined by first calculating the difference and then calculating the ratio, and the processing logic complexity is low and easy to implement.

[0141] In step 402, the interpolation pose of the first data frame relative to the first front pose parameter data and the first rear pose parameter data is determined mainly by the true value interpolation weight, thereby obtaining a first transformation matrix. The first front pose parameter data includes a first front position coordinate and a first front attitude angle, and the first rear pose parameter data includes a first rear position coordinate and a first rear attitude angle.

[0142] In one embodiment, referring to Figure 5 , step 402 may include:

[0143] Step 501, calculating a translation interpolation vector according to the true value interpolation weight, the first front position coordinate and the first rear position coordinate;

[0144] Step 502, calculating a rotation interpolation vector according to the true value interpolation weight, the first front attitude angle and the first rear attitude angle;

[0145] Step 503, calculating a first rotation matrix according to the rotation interpolation vector;

[0146] Step 504, constructing a unit matrix as an initial transformation matrix, filling the first rotation matrix into the upper right submatrix of the initial transformation matrix, and filling the translation interpolation vector into the last column of the initial transformation matrix to obtain a first transformation matrix.

[0147] The implementation process of the above step 401 (including step 501 to step 504) to step 402 can be shown in the following code:

[0148] {def calculate_transform(t,t1,t2,translation1,ratation1,translation2,ratation2):

[0149] #Calculate the true value interpolation weights.

[0150] alpha = t-t1 / t2-t1;

[0151] #Calculate the translation interpolation vector, translation1 represents the first front position coordinate, and translation2 represents the first back position coordinate.

[0152] translation=(1-alpha)*translation1+alpha*translation2;

[0153] #Calculate the rotation interpolation vector, rotation1 represents the first front attitude angle, rotation2 represents the first rear attitude angle.

[0154] rotation=(1-alpha)*rotation1+alpha*rotation2;

[0155] #Calculate the first rotation matrix, rotation_matrix represents the rotation matrix calculation function.

[0156] R=rotation_matrix(rotation);

[0157] #Build the transformation matrix.

[0158] transform = np.eye(4);

[0159] transform[:3,:3]=R;

[0160] transform[:3,3]=translation;

[0161] return transform}.

[0162] The above code can be understood as: Calculate the true value interpolation weight alpha, which represents the ratio of t to t1 and t2. Perform linear interpolation based on alpha to obtain the translation interpolation vector translation. Also perform linear interpolation based on alpha to obtain the rotation interpolation vector rotation. Use the rotation_matrix function to convert the rotation interpolation vector into the first rotation matrix R. Construct the 4x4 identity matrix np.eye(4) as the basic structure of the transformation matrix. Fill the first rotation matrix R into the upper right 3x3 submatrix of the transformation matrix, and fill the translation vector translation into the last column of the transformation matrix. Return the constructed first transformation matrix.

[0163] It should be noted that the rotation matrix calculation function is used to generate matrices for rotation around the X axis, Y axis and Z axis according to the input rotation interpolation vector. The specific process of constructing the rotation matrix calculation function will be described below and will be omitted here.

[0164] In one embodiment, step 503 may include:

[0165] Read the rotation interpolation vector variable to obtain the roll angle variable, pitch angle variable and yaw angle variable;

[0166] Calculate a first rotation submatrix rotating around a first axis according to the roll angle variable;

[0167] Calculate a second rotation submatrix rotating around a second axis according to the pitch angle variable;

[0168] Calculate a third rotation submatrix rotating around a third axis according to the yaw angle variable;

[0169] The first rotation matrix is ​​obtained by combining the first rotation submatrix, the second rotation submatrix and the third rotation submatrix.

[0170] The first axis, the second axis and the third axis are perpendicular to each other. The first axis is the X-axis, the second axis is the Y-axis, and the third axis is the Z-axis. Specifically, the rotation interpolation vector rotation is parsed by reading the sub-function to obtain the roll angle variable roll, the pitch angle variable pitch and the yaw angle variable yaw. Based on the roll angle variable roll, the first rotation submatrix for rotating around the X-axis can be generated as R_roll. Based on the pitch angle variable pitch, the second rotation submatrix for rotating around the Y-axis can be generated as R_pitch. Based on the yaw angle variable yaw, the third rotation submatrix for rotating around the Z-axis can be generated as R_yaw. Finally, R_roll, R_pitch and R_yaw are combined to obtain the first rotation matrix R.

[0171] The benefit of the above embodiment is that by first calculating the three rotation sub-matrices separately and then combining them to obtain the first rotation matrix, the roll angle on the first axis, the pitch angle on the second axis and the yaw angle on the third axis are fully considered, and the accuracy is high.

[0172] The sub-steps included in the above step 503 can be encapsulated into a rotation matrix calculation function rotation_matrix, and then the first rotation matrix can be obtained by inputting the rotation interpolation vector into the rotation matrix calculation function rotation_matrix. The calculation process of the rotation matrix calculation function rotation_matrix can be shown in the following code:

[0173] {def rotation_matrix(rotation):

[0174] roll, pitch, yaw=rotation;

[0175] R_roll=np.array([[1,0,0],

[0176] [0,np.cos(roll),-np.sin(roll)],

[0177] [0,np.sin(roll),np.cos(roll)]]);

[0178] R_pitch=np.array([[np.cos(pitch),0,np.sin(pitch)],

[0179] [0,1,0],

[0180] [-np.sin(pitch),0,np.cos(pitch)]]);

[0181] R_yaw=np.array([[np.cos(pitch),-np.sin(pitch),0],

[0182] [np.sin(pitch),np.cos(pitch),0],

[0183] [0,0,1]]);

[0184] R=R_roll.dot.(R_pitch).dot(R_yaw);

[0185] Return R}.

[0186] The above code can be understood as: unpacking the variable rotation and dividing it into three variables: roll, pitch, and yaw. Define three two-dimensional arrays, representing the rotation sub-matrices around the X-axis, Y-axis, and Z-axis respectively. R_roll represents the first rotation sub-matrix around the X-axis. R_pitch represents the second rotation sub-matrix around the Y-axis. R_yaw represents the third rotation sub-matrix around the Z-axis. Combine the rotation matrices in the order of first around the X-axis, then around the Y-axis, and finally around the Z-axis, that is, R = R_roll.dot(R_pitch).dot(R_yaw). Return the final first rotation matrix R.

[0187] After obtaining the translation interpolation vector and the rotation interpolation vector, in an example of step 504, a 4x4 identity matrix np.eye(4) is first constructed as the initial transformation matrix transform. Then, the rotation matrix R is filled into the upper right 3x3 submatrix of the initial transformation matrix transform, and the translation interpolation vector translation is filled into the last column of the initial transformation matrix transform to obtain the first transformation matrix.

[0188] The benefit of the above-mentioned embodiment of steps 501 to 504 is that by first calculating the translation interpolation vector and the rotation interpolation vector respectively and then constructing the first transformation matrix, the calculation complexity is reduced and the accuracy is ensured on the basis of being able to calculate the first transformation matrix.

[0189] In one embodiment, step 202 may include:

[0190] Add an element after the labeled true value vehicle body coordinates to obtain homogeneous coordinate representation;

[0191] Multiplying the first transformation matrix and the homogeneous coordinate representation to obtain a homogeneous transformed coordinate representation;

[0192] Remove the elements of the last dimension represented by the homogeneous transformation coordinates to obtain the base odometer coordinates.

[0193] The benefit of this embodiment is that by first adding homogeneous coordinates, then performing coordinate conversion, and then removing the homogeneous coordinates, it is possible to convert the coordinates in the vehicle coordinate system to the coordinates in the odometer coordinate system, and the conversion logic complexity is low and the applicability is high.

[0194] The above-mentioned step of multiplying the first transformation matrix with the homogeneous coordinate representation to obtain the homogeneous transformation coordinate representation may include: performing matrix multiplication operation on the first transformation matrix and the transpose of the homogeneous coordinate representation to obtain an initial transformation coordinate representation; and determining the transpose of the initial transformation coordinate representation as the homogeneous transformation coordinate representation.

[0195] The above steps can be encapsulated into a point coordinate transformation function transform_points, and then the labeled true value vehicle body coordinates and the first transformation matrix are input into the point coordinate transformation function transform_points to obtain the reference odometer coordinates. The calculation process of the point coordinate transformation function transform_points can be shown in the following code:

[0196] {def transform_points(point,transform):

[0197] #Add homogeneous coordinates

[0198] point_homogeneous=np.hasack(points,np.ones((points.shape[0],1)));

[0199] #Multiply, T stands for transpose

[0200] transformed_points=transform.dot(point_homogeneous.T).T;

[0201] #Remove the homogeneous coordinate part, that is, the elements on the last dimension, and keep the coordinates of the first three dimensions

[0202] transformed_points=transformed_points[:,:3];

[0203] return transformed_points}.

[0204] The above code can be understood as: add a 1 after the annotated true value body coordinates to form a homogeneous coordinate representation, which is convenient for matrix transformation. The result of this step is a numpy array with a shape of (n,4). Use numpy's dot method to perform matrix multiplication, multiplying the transformation matrix with the points represented by homogeneous coordinates. First transpose points_homogeneous to meet the requirements of matrix multiplication, then get the result and transpose it back. Remove the homogeneous coordinate part of the transformed point, that is, the elements in the last dimension, and retain the first three-dimensional coordinates. Return the baseline odometer coordinates obtained after the transformation.

[0205] After obtaining the reference odometer coordinates by executing step 102, in step 103, at least one second data frame in the plurality of original data frames is filled with coordinates according to each reference odometer coordinate to obtain filled odometer coordinates of at least one second data frame. Each second data frame is a data frame in the plurality of original data frames without annotated true value vehicle body coordinates.

[0206] In one embodiment, referring to Figure 6 , step 103 may include:

[0207] Step 601, matching the timestamp of each first data frame with the timestamp of each original data frame to generate a filled timestamp column;

[0208] Step 602, rename the timestamp of each original data frame to a fill timestamp column;

[0209] Step 603, left-connect the original data frame with the reference odometer coordinates according to the filled timestamp column to obtain a true odometer coordinate column;

[0210] Step 604 , forward filling the missing values ​​corresponding to at least one second data frame according to each reference odometer coordinate in the true value odometer coordinate column, to obtain the filled odometer coordinates of at least one second data frame.

[0211] In step 601, matching specifically refers to comparing whether two timestamps are the same. For example, the timestamp of the first data frame includes t 0 ,t 3 ,t 7 and t 10 , and the timestamps of each original data frame include t 0 ,t 1 ,t 2 ,t 3 ,t 4 ,t 5 ,t 6 ,t 7 ,t 8 ,t 9 ,t 10 ,t 11 and t 12 , then after matching, we can get t 0 ,t 3 ,t 7 and t 10 is a matching timestamp, and the generated fill timestamp column is {t 0 , t 3 , t 7 , t 10}.

[0212] In step 602, the purpose of renaming the filled timestamp column is to associate the matching timestamp in step 601 with the original timestamp.

[0213] In step 603, a left join is a special type of join that returns all records of the left table (primary table) based on the join condition, even if there are no matching records in the right table (secondary table). When there are no matching records in the right table with the left table, the portion of the right table in the result set will contain missing values ​​(NULL values). This embodiment implements the odometer coordinates corresponding to the first data frame in the true value odometer coordinate column as the reference odometer coordinates of the matching timestamp based on a left join, and the odometer coordinates corresponding to the second data frame in the true value odometer coordinate column are missing values.

[0214] In step 604, after obtaining the true odometer coordinate column, the missing values ​​corresponding to at least one second data frame are forward filled according to each reference odometer coordinate in the true odometer coordinate column to obtain the filled odometer coordinates of at least one second data frame.

[0215] The benefit of the embodiment of the above steps 601 to 604 is that the flexibility and applicability of coordinate filling are ensured by first determining to fill the timestamp column, then performing left connection, and then performing forward filling.

[0216] In one example, the multiple original data frames are {Z 1 , Z 2 , Z 3 , Z 4 , Z 5 , Z 6 , Z 7 , Z 8 , Z 9 , Z 10 , Z 11 , Z 12}, and the data frame with the true value of the vehicle coordinates includes Z 1 , Z 4 , Z 7 , Z 10 , then the first data frame is Z 1b , Z 4b , Z 7b , Z 10b , and the second data frame is Z 2w , Z 3w , Z 5w , Z 6w , Z 8w , Z 9w , Z 11w , Z 12w . Combining the above, we can see that Z 1b The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 4b The base odometer coordinates are (x 4j ,y 4j, z 4j ), Z 7b The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 10b The base odometer coordinates are (x 10j ,y 10j , z 10j ). After filling the coordinates, we can get Z 2w The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 3w The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 5w The base odometer coordinates are (x 4j ,y 4j , z 4j ), Z 6w The base odometer coordinates are (x 4j ,y 4j , z 4j ), Z 8w The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 9w The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 11w The base odometer coordinates are (x 10j ,y 10j , z 10j ), Z 12w The base odometer coordinates are (x 10j ,y 10j , z 10j ).

[0217] In one embodiment, step 602 may include:

[0218] Determine a minimum true value timestamp and a maximum true value timestamp from the timestamps of each first data frame;

[0219] Filtering the interval of the original data frames to be filled from the multiple original data frames according to the minimum true value timestamp and the maximum true value timestamp;

[0220] Rename the timestamp of the original data frame interval to be filled to the filling timestamp column;

[0221] After forward filling, the filled odometer coordinates of at least one second data frame in the original data frame interval to be filled are obtained.

[0222] The benefit of this embodiment is that the frame interval that needs to be filled with odometer coordinates can be well demarcated by the minimum true value timestamp and the maximum true value timestamp. On the basis of improving the filling efficiency, the problem of unstable filling accuracy caused by the timestamp corresponding to the filled odometer coordinates being outside the interval between the minimum true value timestamp and the maximum true value timestamp can be alleviated.

[0223] The implementation process of the above steps 601 to 604 can be shown in the following code:

[0224] {traffic_lane_multi_df1=traffic_lane_multi_df[["timestamp"]];

[0225] traffic_lane_multi_times=traffic_lane_multi_df["timestamp"].to_list();

[0226] truth_data_df1["timestamp0"]=truth_data_df1.apply(lambda x:match_df_time(t raffic_lane_multi_times,x["timestamp"]),axis=1);

[0227] truth_min_time=truth_data_df1["timestamp"].min();

[0228] truth_max_time=truth_data_df1["timestamp"].max();

[0229] traffic_lane_multi_df1=traffic_lane_multi_df1.loc[(traffic_lane_multi_df1[“time stamp”]>=truth_min_time)&(traffic_lane_multi_df1[“timestamp”]<=truth_max_tim e)];

[0230] traffic_lane_multi_df1.rename(columns={“timestamp”:“timestamp0”},inplace=True);

[0231] truth_interpolation_df=traffic_lane_multi_df1.merge(truth_data_df1, on=["timest amp0"], how='left');

[0232] truth_interpolation_df.fillna(method='ffill',inplace=true)}.

[0233] The above code can be understood as follows: First, extract the timestamp information in the traffic_lane_multi_df data frame and convert it into a list. Then, use the match_df_time function to match the timestamp of the first data frame with the timestamp of each original data frame (traffic_lane_multi_times), and generate a fill timestamp column "timestamp0" based on the matching results. Find the minimum value (truth_min_time) and maximum value (truth_max_time) of the timestamp in the first data frame (truth_data_df1), and then filter out the rows in this time interval in traffic_lane_multi_df to obtain the original data frame interval to be filled. Rename the timestamp of the original data frame interval to be filled to the fill timestamp column "timestamp0" to facilitate subsequent merge operations. Finally, based on the common fill timestamp column "timestamp0", the original data frame interval to be filled is left-joined with the first data frame, and the missing values ​​are filled forward to ensure that the second data frame in the original data frame interval to be filled has corresponding filled odometer coordinates.

[0234] In one embodiment, after the filled odometer coordinates of at least one second data frame are obtained in step 103, the filled odometer coordinates can be deduplicated. Specifically, in combination with the above, it can be seen that each original data frame contains lane line feature points, and the second data frame is a data frame without true value vehicle body coordinates labeled in multiple original data frames. Therefore, the filled odometer coordinates of the second data frame may include odometer coordinates of multiple different types of lane line feature points, and these coordinates may be repeated, so the filled odometer coordinates can be deduplicated to avoid subsequent coordinate conversion of repeated coordinates, improve processing efficiency and save processing resources.

[0235] After executing step 103 to obtain the filled odometer coordinates of at least one second data frame, in step 104, the filled odometer coordinates of at least one second data frame are converted to a vehicle body coordinate system to obtain predicted true value vehicle body coordinates of at least one second data frame.

[0236] In one embodiment, step 104 may include: determining predicted true value vehicle body coordinates in a vehicle body coordinate system according to each filled odometer coordinate and relative positioning data associated with a plurality of original data frames. In this way, the filled odometer coordinates of the second data frame are converted to the vehicle body coordinate system by using the relative poses of the plurality of original data frames in the relative positioning data to obtain predicted true value vehicle body coordinates.

[0237] In one embodiment, referring to Figure 7 , step 104 may include:

[0238] Step 701, generating a transformation matrix from a vehicle body coordinate system to an odometer coordinate system according to a posture relationship between a second timestamp of a second data frame and a second adjacent positioning timestamp in the relative positioning data, to obtain a second transformation matrix; the second adjacent positioning timestamp is a timestamp in the relative positioning data that is adjacent to the second timestamp;

[0239] Step 702 , performing coordinate transformation according to the filled odometer coordinates of at least one second data frame and the inverse matrix of the second transformation matrix to obtain predicted true value vehicle body coordinates.

[0240] In step 701, the second transformation matrix is ​​a transformation matrix from the vehicle body coordinate system to the odometer coordinate system. The second adjacent timestamp is a timestamp adjacent to the second timestamp in the relative positioning data. The second adjacent timestamps include a second adjacent front timestamp and a second adjacent rear timestamp. The second adjacent front timestamp is a timestamp adjacent to the second timestamp and before the second timestamp in the relative positioning data. The second adjacent rear timestamp is a timestamp adjacent to the second timestamp and after the second timestamp in the relative positioning data. The second adjacent front timestamp, the second timestamp, and the second adjacent rear timestamp are incremented in sequence.

[0241] The relative positioning data has been introduced in detail above. For details, please refer to the relevant description of step 102 above, which will not be repeated here.

[0242] In one example, the timestamps of the relative positioning data include t0, t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11, and t12. If the second timestamp is t1, the second adjacent before timestamp is t1 and the second adjacent after timestamp is t3. If the second timestamp is t3, the second adjacent before timestamp is t2 and the second adjacent after timestamp is t4.

[0243] In step 702, after obtaining the second transformation matrix, the inverse matrix of the second transformation matrix is ​​used to perform coordinate transformation on the filled odometer coordinates to obtain the predicted true value vehicle body coordinates.

[0244] The benefit of the above-mentioned embodiment of step 701 to step 702 is that the second transformation matrix can be generated for each filled odometer coordinate and the coordinate conversion can be performed separately, and the processing logic is simple and easy to implement, which helps to improve processing efficiency.

[0245] In one embodiment, step 701 may include:

[0246] Extracting, from the relative positioning data according to the second timestamp, second front posture parameter data of the second adjacent front timestamp and the second adjacent rear timestamp and second rear posture parameter data of the second adjacent rear timestamp and the second adjacent rear timestamp;

[0247] A second transformation matrix is ​​calculated based on the second timestamp, the second adjacent front timestamp, the second adjacent rear timestamp, the second front posture parameter data and the second rear posture parameter data.

[0248] Combined with the above, it can be seen that the relative positioning data includes relative poses corresponding to multiple timestamps, so the required parameters for calculating the transformation matrix can be extracted from the relative positioning data according to the first timestamp, that is, the first adjacent front timestamp and the first front pose parameter data, and the first adjacent rear timestamp and the first rear pose parameter data. For example, combined with the above example, if the second timestamp is t1, the relative poses corresponding to t0 and t0, and the relative poses corresponding to t2 and t2 are extracted from the relative positioning data.

[0249] The benefit of the above embodiment is that the posture relationship between the second timestamp and the second adjacent timestamp can be effectively utilized to improve the calculation accuracy of the transformation matrix.

[0250] The implementation process of the above steps 701 to 702 can be shown in the following code:

[0251] {#timestamp is the second timestamp of the second data frame

[0252] #odom_points are points in the odometer (odom) coordinate system

[0253] #odom_points=[[x1,y1,z1],[x2,y2,z2],...]

[0254] #Translation vector and rotation angle, t1 <t<t2

[0255] #transforms=[[[t1_x,t1_y,t1_z],[t1_roll,t1_pitch,t1_yaw]],[[t2_x,t2_y,t2_z],[t2_r oll,t2_pitch,t2_yaw]]]

[0256] #time=[t1,t2] is the timestamp of the adjacent frames

[0257] def odom_point_to_vehicle(time,odom_points,time,transforms):

[0258] t1 = time[0];

[0259] t2 = time[1];

[0260] t = timestamp;

[0261] translate_1=np.arry(transforms[0][0]);

[0262] rotation_1=np.arry(transforms[0][1]);

[0263] translate_2=np.arry(transforms[1][0]);

[0264] rotation_2=np.arry(transforms[1][1]);

[0265] veh_point=np.arry(odom_points);

[0266] # Calculate the transformation matrix

[0267] transform_v2o=calulate_transform(t,t1,t2,translate_1,rotation_1,translate_2,rotation_2);

[0268] #Calculate the inverse matrix of the transformation matrix

[0269] transform_v2o=np.linglg.inv(transform_v2o);

[0270] veh_point=transform_point(odom_points, transform_v2o)}.

[0271] The above code can be understood as: extract the timestamps t1 and t2 and timestamp in the input parameters. Extract the translation vector and rotation angle, translate_1, rotation_1, translate_2, rotation_2 respectively. Convert odom_points to a numpy array. Call the calculate_transform function to calculate the transformation matrix transform_v2o from the vehicle coordinate system to the odometer coordinate system, and then calculate the inverse transformation matrix transform_o2v from the odometer coordinate system to the vehicle coordinate system. Perform coordinate transformation on odom_points to obtain the point veh_points in the vehicle coordinate system. Convert the predicted true value vehicle coordinate veh_points back to a list type and return it.

[0272] It should be noted that the functions used in step 104 and step 102 are basically the same, except that the parameters of the input value function are different. Therefore, the calulate_transform function and the transform_point function are not explained here, and the relevant description of step 102 above can be referred to.

[0273] In one example, combining the above examples, we can get: Z 2w The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 3w The base odometer coordinates are (x 1j ,y 1j , z 1j ), Z 5w The base odometer coordinates are (x 4j ,y 4j , z 4j ), Z 6w The base odometer coordinates are (x 4j ,y 4j , z 4j ), Z 8w The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 9w The base odometer coordinates are (x 7j ,y 7j , z 7j ), Z 11w The base odometer coordinates are (x 10j ,y 10j , z 10j ), Z 12wThe base odometer coordinates are (x 10j ,y 10j , z 10j ). After executing step 104 for these reference odometer coordinates, Z 1b The predicted true value of the vehicle body coordinates is (x 1y ,y 1y , z 1y ), Z 4b The base odometer coordinates are (x 4y ,y 4y , z 4y ), Z 7b The base odometer coordinates are (x 7y ,y 7y , z 7y ), Z 10b The base odometer coordinates are (x 10y ,y 10y , z 10y ). After filling the coordinates, we can get Z 2w The base odometer coordinates are (x 1y ,y 1y , z 1y ), Z 3w The base odometer coordinates are (x 1y ,y 1y , z 1y ), Z 5w The base odometer coordinates are (x 4y ,y 4y , z 4y ), Z 6w The base odometer coordinates are (x 4y ,y 4y , z 4y ), Z 8w The base odometer coordinates are (x 7y ,y 7y , z 7y ), Z 9w The base odometer coordinates are (x 7y ,y 7y , z 7y ), Z 11w The base odometer coordinates are (x 10y ,y 10y , z 10y ), Z 12w The base odometer coordinates are (x 10y ,y 10y , z 10y ). The coordinates with the subscript "y" are the coordinates converted from the filled odometer coordinates in the vehicle coordinate system.

[0274] In summary, the true value vehicle body coordinate acquisition method provided in the present application can save the manpower and cost of labeling, and can also align the number of true value vehicle body coordinates with the number of frames of the original data frame, thereby reducing the false detection rate of algorithm evaluation based on the true value vehicle body coordinates and the original data frame (such as lane line feature point detection, etc.).

[0275] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this specification.

[0276] The present application also discloses a device for acquiring true value vehicle body coordinates. Figure 8 , the device for acquiring the true value vehicle body coordinates includes: a labeled true value acquisition module 801, a first coordinate conversion module 802, a coordinate filling module 803 and a second coordinate conversion module 804. The labeled true value acquisition module 801 is used to acquire the labeled true value vehicle body coordinates of multiple first data frames in multiple original data frames, where the first data frame is a data frame with labeled true value vehicle body coordinates in the vehicle body coordinate system; the first coordinate conversion module 802 is used to determine the reference odometer coordinates in the odometer coordinate system according to each labeled true value vehicle body coordinate and the relative positioning data associated with the multiple original data frames; the coordinate filling module 803 is used to perform coordinate filling on at least one second data frame in the multiple original data frames according to each reference odometer coordinate to obtain the filled odometer coordinates of at least one second data frame, where each second data frame is a data frame without labeled true value vehicle body coordinates in the multiple original data frames; the second coordinate conversion module 804 is used to convert the filled odometer coordinates of at least one second data frame into the vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of at least one second data frame.

[0277] It should be noted that the specific implementation of the true value vehicle body coordinate acquisition device is basically the same as the specific implementation of the true value vehicle body coordinate acquisition method described above, and will not be repeated here.

[0278] The embodiment of the present application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned true value vehicle body coordinate acquisition method when executing the computer program. The electronic device is, for example, a mobile phone, a vehicle, and other devices.

[0279] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the true value vehicle body coordinate acquisition method as described above.

[0280] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0281] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is read and executed by a processor of an electronic device, so that when the processor executes the computer program, the true value vehicle body coordinate acquisition method as described above is implemented.

[0282] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0283] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0284] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0285] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0286] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0287] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0288] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0289] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0290] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0291] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0292] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for obtaining true value vehicle body coordinates, characterized in that: The method comprises: Acquire the annotated true value vehicle body coordinates of a plurality of first data frames among the plurality of original data frames, wherein the first data frames are data frames annotated with the true value vehicle body coordinates in the vehicle body coordinate system; Determining a reference odometer coordinate in an odometer coordinate system according to each of the annotated true value vehicle body coordinates and the relative positioning data associated with the plurality of original data frames; Filling coordinates of at least one second data frame among the plurality of original data frames according to each of the reference odometer coordinates to obtain filled odometer coordinates of the at least one second data frame, wherein each of the second data frames is a data frame without true value vehicle body coordinates marked among the plurality of original data frames; The filled odometer coordinates of the at least one second data frame are converted to a vehicle body coordinate system to obtain predicted true value vehicle body coordinates of the at least one second data frame.

2. The method according to claim 1, characterized in that The step of determining the reference odometer coordinates in the odometer coordinate system according to each of the annotated true value vehicle body coordinates and the relative positioning data associated with the plurality of original data frames comprises: Generate a first transformation matrix according to a posture relationship between a first timestamp of the first data frame and a first adjacent timestamp in the relative positioning data, where the first transformation matrix is ​​a transformation matrix from a vehicle body coordinate system to an odometer coordinate system, and the first adjacent timestamp is a timestamp in the relative positioning data that is adjacent to the first timestamp; Coordinate transformation is performed according to the labeled true value vehicle body coordinates and the first transformation matrix to obtain the reference odometer coordinates.

3. The method according to claim 2, characterized in that The first adjacent timestamps include a first adjacent front timestamp and a first adjacent rear timestamp; The step of generating a first transformation matrix according to a posture relationship between a first timestamp of the first data frame and a first adjacent timestamp in the relative positioning data comprises: Extracting, from the relative positioning data according to the first timestamp, the first adjacent front timestamp and the first front pose parameter data of the first adjacent front timestamp, and the first adjacent rear timestamp and the first rear pose parameter data of the first adjacent rear timestamp; A first transformation matrix is ​​calculated based on the first timestamp, the first adjacent previous timestamp, the first adjacent subsequent timestamp, the first previous posture parameter data, and the first subsequent posture parameter data.

4. The method according to claim 3, characterized in that The calculating a first transformation matrix according to the first timestamp, the first adjacent front timestamp, the first adjacent rear timestamp, the first front posture parameter data, and the first rear posture parameter data includes: Calculating a true value interpolation weight according to the first timestamp, the first adjacent previous timestamp, and the first adjacent subsequent timestamp; A matrix is ​​constructed according to the true value interpolation weight, the first front posture parameter data and the first rear posture parameter data to obtain the first transformation matrix.

5. The method according to claim 4, characterized in that The calculating the true value interpolation weight according to the first timestamp, the first adjacent previous timestamp and the first adjacent subsequent timestamp includes: Performing a difference calculation based on the first timestamp and the first adjacent previous timestamp to obtain a first difference; Performing a difference calculation based on the first adjacent rear timestamp and the first adjacent front timestamp to obtain a second difference; The true value interpolation weight is obtained by performing a ratio calculation based on the first difference and the second difference.

6. The method according to claim 4, characterized in that The first front posture parameter data includes a first front position coordinate and a first front posture angle, and the first rear posture parameter data includes a first rear position coordinate and a first rear posture angle; The step of constructing a matrix according to the true value interpolation weight, the first front pose parameter data, and the first rear pose parameter data to obtain the first transformation matrix includes: Calculating a translation interpolation vector according to the true value interpolation weight, the first front position coordinate, and the first rear position coordinate; Calculating a rotation interpolation vector according to the true value interpolation weight, the first front posture angle, and the first rear posture angle; Calculating a first rotation matrix according to the rotation interpolation vector; A unit matrix is ​​constructed as an initial transformation matrix, the first rotation matrix is ​​filled into the upper right submatrix of the initial transformation matrix, and the translation interpolation vector is filled into the last column of the initial transformation matrix to obtain the first transformation matrix.

7. The method according to claim 6, characterized in that The step of calculating a first rotation matrix according to the rotation interpolation vector comprises: Reading variables of the rotation interpolation vector to obtain roll angle variables, pitch angle variables and yaw angle variables; Calculating a first rotation submatrix rotating around a first axis according to the roll angle variable; Calculating a second rotation submatrix rotating around a second axis according to the pitch angle variable; Calculating a third rotation submatrix rotating around a third axis according to the yaw angle variable; Combining the first rotation submatrix, the second rotation submatrix and the third rotation submatrix to obtain the first rotation matrix; The first axis, the second axis and the third axis are perpendicular to each other.

8. The method according to claim 2, characterized in that: The step of performing coordinate conversion according to the annotated true value vehicle body coordinates and the first transformation matrix to obtain the reference odometer coordinates includes: Add an element after the labeled true value vehicle body coordinate to obtain a homogeneous coordinate representation; Multiplying the first transformation matrix by the homogeneous coordinate representation to obtain a homogeneous transformed coordinate representation; The elements of the last dimension represented by the homogeneous transformation coordinates are removed to obtain the reference odometer coordinates.

9. The method according to any one of claims 1 to 8, characterized in that: The step of performing coordinate filling on at least one second data frame among the plurality of original data frames according to each of the reference odometer coordinates to obtain the filled odometer coordinates of the at least one second data frame comprises: Matching the timestamp of each first data frame with the timestamp of each original data frame to generate a filled timestamp column; Rename the timestamp of each of the original data frames to the filled timestamp column; Left-connect the original data frame with the reference odometer coordinates according to the filled timestamp column to obtain a true odometer coordinate column; According to each of the reference odometer coordinates in the true odometer coordinate column, the missing values ​​corresponding to at least one second data frame are forward filled to obtain the filled odometer coordinates of the at least one second data frame.

10. The method according to claim 9, characterized in that The step of renaming the timestamp of each of the original data frames into the filled timestamp column comprises: Determine a minimum true value timestamp and a maximum true value timestamp from the timestamps of each of the first data frames; Filtering the original data frame interval to be filled from the plurality of original data frames according to the minimum true value timestamp and the maximum true value timestamp; Rename the timestamp of the original data frame interval to be filled to the filling timestamp column; After forward filling, the filled odometer coordinates of at least one second data frame in the interval of the original data frame to be filled are obtained.

11. The method according to any one of claims 1 to 8, characterized in that: The step of converting the filled odometer coordinates of the at least one second data frame into a vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of the at least one second data frame includes: Generate a transformation matrix from the vehicle body coordinate system to the odometer coordinate system according to the posture relationship between the second timestamp of the second data frame and the second adjacent positioning timestamp in the relative positioning data to obtain a second transformation matrix; the second adjacent positioning timestamp is a timestamp in the relative positioning data that is adjacent to the second timestamp; Coordinate transformation is performed according to the filled odometer coordinates of the at least one second data frame and the inverse matrix of the second transformation matrix to obtain the predicted true value vehicle body coordinates.

12. A device for acquiring true value vehicle body coordinates, characterized in that: The device comprises: A marked true value acquisition module, used to acquire the marked true value vehicle body coordinates of a plurality of first data frames in the plurality of original data frames, wherein the first data frames are data frames with marked true value vehicle body coordinates in a vehicle body coordinate system; A first coordinate conversion module, configured to determine a reference odometer coordinate in an odometer coordinate system according to each of the annotated true value vehicle body coordinates and relative positioning data associated with the plurality of original data frames; A coordinate filling module, configured to fill the coordinates of at least one second data frame in the plurality of original data frames according to each of the reference odometer coordinates, to obtain the filled odometer coordinates of the at least one second data frame, wherein each of the second data frames is a data frame in the plurality of original data frames without annotated true value vehicle body coordinates; The second coordinate conversion module is used to convert the filled odometer coordinates of the at least one second data frame into a vehicle body coordinate system to obtain the predicted true value vehicle body coordinates of the at least one second data frame.

13. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is read and executed by a processor of an electronic device, so that when the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

Images