An object pose estimation method and device, and an autonomous driving device

Abstract

Embodiments of the present application provide a target pose estimation method and device and an automatic driving device. The method comprises: obtaining pose information corresponding to a plurality of target time instants of a target to be processed and a speed estimation result; estimating a pose of the target to be processed at a current time instant according to the pose information corresponding to each target time instant and the speed estimation result; and performing fusion processing on the pose corresponding to each target time instant to obtain a predicted pose of the target to be processed at the current time instant. The plurality of target time instants of the target to be processed are obtained, and a plurality of poses of the target to be processed at the current time instant are estimated according to the pose information and the speed estimation result, and then the plurality of poses are fused to obtain the predicted pose of the target to be processed at the current time instant. The influence of the error of a single pose information or a speed estimation result on the pose estimation of the target to be processed is reduced, and the stability and accuracy of the estimation result of the target pose are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving target perception technology, and in particular to a target pose estimation method, apparatus, and autonomous driving device. Background Technology

[0002] With the rapid development of autonomous driving technology, the application of target pose estimation technology through target detection algorithms is becoming increasingly widespread. Target detection algorithms rely on data collected by sensors. Due to the physical limitations of the sensors, the performance of target detection algorithms will significantly decrease when the target is in a certain area; this area is called the detection algorithm blind zone.

[0003] Current target pose estimation methods require determining whether the target is within the detection algorithm's blind zone. If the target is determined to be outside the blind zone, the sensor data is considered valid, and the estimated target pose is directly output. However, if the target is determined to be within the blind zone, for example, when the target is occluded by an obstacle, the sensor data is considered invalid. In this case, the estimated target pose will be inaccurate, resulting in poor stability and accuracy. Summary of the Invention

[0004] This invention provides a method, apparatus, and autonomous driving device for estimating target pose, which can at least improve the stability and accuracy of the target pose estimation results. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a method for estimating a target pose, the method comprising:

[0006] Acquire the pose information and velocity estimation results of the target to be processed at multiple target times, wherein the multiple target times include the current time and multiple historical times before the current time;

[0007] Based on the pose information and velocity estimation results corresponding to each target moment, the pose of the target to be processed at the current moment is estimated;

[0008] The poses corresponding to each target time are fused to obtain the predicted pose of the target at the current time.

[0009] Optionally, the velocity estimation results include linear velocity estimates and angular velocity estimates;

[0010] The step of estimating the pose of the target at the current moment based on the pose information and velocity estimation results corresponding to each target moment includes:

[0011] Based on the estimated linear velocity value corresponding to each target time, the displacement change of the target from that target time to the current time is obtained by performing integration over time.

[0012] Based on the estimated angular velocity value corresponding to each target moment, the integral is performed over time to obtain the rotational change of the target from that target moment to the current moment;

[0013] Based on the pose information, displacement change, and rotation change corresponding to each target moment, the pose of the target to be processed at the current moment is calculated.

[0014] Optionally, the step of calculating the pose of the target at the current moment based on the pose information, displacement change, and rotation change corresponding to each target moment includes:

[0015] The pose information corresponding to each target moment is transformed into the global coordinate system using the rotation and translation matrix from the local coordinate system to the global coordinate system to obtain the first global pose.

[0016] Based on the first global pose, displacement change, and rotation change corresponding to each target time, the second global pose of the target to be processed at the current time is calculated.

[0017] The second global pose is transformed into the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system, thus obtaining the pose of the target at the current moment.

[0018] Optionally, the pose information includes the center coordinates, orientation angle, and target bounding box size of the target to be processed;

[0019] The step of transforming the pose information corresponding to each target time moment into the global coordinate system using a rotation and translation matrix from the sensor's local coordinate system to the global coordinate system to obtain the first global pose includes:

[0020] Based on the center coordinates, target bounding box size, and orientation angle of the target to be processed at each target time, the coordinates of multiple first key points of the target to be processed are calculated;

[0021] According to the rotation and translation matrix from the local coordinate system to the global coordinate system of the sensor, calculate the multiple first key point coordinates transformed into multiple first global coordinates in the global coordinate system;

[0022] Based on the plurality of first global coordinates and the target bounding box size, the center coordinates and orientation angle of the target to be processed in the global coordinate system are calculated as the first global pose.

[0023] The step of transforming the second global pose to the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system, and obtaining the pose of the target at the current moment, includes:

[0024] Based on the second global pose and the target bounding box size, the coordinates of multiple second key points of the target to be processed are calculated;

[0025] According to the rotation and translation matrix from the global coordinate system to the sensor local coordinate system, calculate the multiple second key point coordinates transformed into multiple second local coordinates in the sensor local coordinate system;

[0026] Based on the multiple second local coordinates corresponding to each target time and the target bounding box size, the center coordinates and orientation angle of the target to be processed in the sensor local coordinate system are calculated as the pose of the target to be processed at the current time.

[0027] Optionally, before the step of fusing the poses corresponding to each target time to obtain the predicted pose of the target at the current time, the method further includes:

[0028] Calculate the confidence level of the target's pose at the current moment;

[0029] The step of fusing the poses corresponding to each target time moment to obtain the predicted pose of the target to be processed at the current time includes:

[0030] Based on the confidence level of the pose of the target at the current time, the poses of each target at each time time are fused to obtain the predicted pose of the target at the current time.

[0031] Optionally, the pose information includes pose confidence, and the velocity estimation result includes velocity confidence.

[0032] The step of calculating the confidence level corresponding to the pose of the target to be processed at the current moment includes:

[0033] The product of multiple velocity confidence scores from the target time to the current time is calculated to obtain the target velocity confidence score.

[0034] The product of the pose confidence score at the target time and the target velocity confidence score is calculated and used as the confidence score of the pose of the target at the current time.

[0035] Optionally, the step of fusing the poses corresponding to each target at the current time based on the confidence level corresponding to the pose of the target to be processed at the current time includes:

[0036] Based on the confidence level, abnormal poses are identified in the poses corresponding to each target time, and the abnormal poses are removed to obtain the target predicted pose.

[0037] The predicted pose of the target is calculated by weighting the corresponding confidence levels to obtain the predicted pose of the target at the current time.

[0038] Optionally, the step of identifying abnormal poses in the pose corresponding to each target time based on the confidence level includes:

[0039] For each parameter of the pose corresponding to each target time, the parameter is weighted and averaged according to the corresponding confidence level to obtain the weighted average value of the parameter, wherein the parameter includes at least one of the center coordinates, orientation angle, and target box size;

[0040] Calculate the difference between each parameter and its corresponding weighted average, and use it as the prediction bias value;

[0041] Based on the relationship between the predicted deviation value and the preset threshold, abnormal poses are determined.

[0042] Optionally, the step of determining the abnormal pose based on the relationship between the predicted deviation value and the preset threshold includes:

[0043] If at least one prediction deviation value is greater than a first preset threshold, the pose corresponding to that prediction deviation value is determined to be an abnormal pose; or,

[0044] According to the preset weights corresponding to each parameter, calculate the weighted sum of the prediction deviation values ​​of each parameter in the pose corresponding to each target time.

[0045] If the weighted sum is greater than the second preset threshold, the pose corresponding to the weighted sum is determined to be an abnormal pose.

[0046] Secondly, embodiments of the present invention provide a target pose estimation device, the device comprising:

[0047] The information acquisition module is used to acquire the pose information and velocity estimation results of the target to be processed at multiple target times, wherein the multiple target times include the current time and multiple historical times before the current time;

[0048] The pose estimation module is used to estimate the pose of the target to be processed at the current moment based on the pose information and velocity estimation results corresponding to each target moment.

[0049] The fusion processing module is used to fuse the poses corresponding to each target time to obtain the predicted pose of the target to be processed at the current time.

[0050] Thirdly, embodiments of the present invention provide an autonomous driving device, the autonomous driving device comprising:

[0051] The processor is configured to execute the target pose estimation method described in the first aspect above to obtain multiple predicted poses of the target to be processed; and to generate a travel path of the autonomous driving device based on the multiple predicted poses.

[0052] Optionally, the autonomous driving device further includes:

[0053] A display for showing a map of the travel path and the environment surrounding the autonomous driving device; and / or,

[0054] A controller is used to control the autonomous driving device to perform autonomous driving based on the travel path.

[0055] Beneficial effects of the embodiments of the present invention:

[0056] In the solution provided by this invention, an electronic device can acquire pose information and velocity estimation results of a target at multiple target times. These multiple target times include the current time and multiple historical times preceding the current time. Based on the pose information and velocity estimation results corresponding to each target time, the pose of the target at the current time can be estimated. Then, the poses corresponding to each target time are fused to obtain the predicted pose of the target at the current time. There is no need to determine whether the target is in the blind zone of the detection algorithm. By acquiring the pose information and velocity estimation results corresponding to multiple target times of the target, and thereby estimating multiple poses of the target at the current time, and then fusing these multiple poses to obtain the predicted pose of the target at the current time, the impact of errors from using a single pose information or velocity estimation result on the pose estimation of the target is reduced, improving the stability and accuracy of the target pose estimation results. Of course, implementing any product or method of this invention does not necessarily require achieving all of the above advantages simultaneously. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0058] Figure 1 A flowchart illustrating a target pose estimation method provided in an embodiment of the present invention;

[0059] Figure 2(a) shows the results based on Figure 1A schematic diagram of the target bounding box of the target to be processed in the embodiment shown;

[0060] Figure 2(b) shows the results based on... Figure 1 A schematic diagram of the orientation angle of the target to be processed in the embodiment shown;

[0061] Figure 3 Based on Figure 1 A flowchart of a target pose estimation method according to the embodiment shown;

[0062] Figure 4 for Figure 1 The illustrated embodiment is a schematic diagram of multiple poses of the target to be processed at the current time, estimated.

[0063] Figure 5 for Figure 1 A schematic diagram of the predicted pose of the target to be processed at the current moment in the illustrated embodiment;

[0064] Figure 6 for Figure 1 A specific flowchart of step S102 in the illustrated embodiment;

[0065] Figure 7 for Figure 6 A specific flowchart of step S603 in the illustrated embodiment;

[0066] Figure 8 for Figure 7 A specific flowchart of step S701 in the illustrated embodiment;

[0067] Figure 9 for Figure 7 A specific flowchart of step S703 in the illustrated embodiment;

[0068] Figure 10 Based on Figure 1 A flowchart illustrating the calculation of confidence level corresponding to pose in the embodiment shown.

[0069] Figure 11 Based on Figure 1 A flowchart illustrating a pose fusion process in the embodiment shown.

[0070] Figure 12 for Figure 11 A specific flowchart of step S1101 in the illustrated embodiment;

[0071] Figure 13 This is a schematic diagram of the structure of a target pose estimation device provided in an embodiment of the present invention;

[0072] Figure 14 This is a schematic diagram of the structure of an autonomous driving device provided in an embodiment of the present invention;

[0073] Figure 15 This is a schematic diagram of another autonomous driving device provided in an embodiment of the present invention;

[0074] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0076] To improve the stability and accuracy of target pose estimation results, embodiments of the present invention provide a target pose estimation method, a target pose estimation device, an autonomous driving device, an electronic device, a computer-readable storage medium, and a computer program product. The target pose estimation method provided by the embodiments of the present invention will be introduced first below.

[0077] The target pose estimation method provided in this invention can be applied to any electronic device that performs pose estimation on a target, such as a robot, unmanned vehicle, autonomous vehicle, driver assistance vehicle, processor, controller, etc., in an autonomous driving system, without specific limitations. This electronic device can perform pose estimation on the target based on sensor data obtained from detecting the target. For clarity, it will be referred to as the electronic device below.

[0078] like Figure 1 As shown, a method for estimating target pose includes:

[0079] S101, Obtain the pose information and velocity estimation results of the target at multiple target times.

[0080] The multiple target times include the current time and multiple historical times preceding the current time.

[0081] S102, based on the pose information and velocity estimation results corresponding to each target time, estimate the pose of the target to be processed at the current time.

[0082] S103, perform fusion processing on the poses corresponding to each target time to obtain the predicted pose of the target to be processed at the current time.

[0083] As can be seen, in the solution provided by this embodiment of the invention, the electronic device can acquire the pose information and velocity estimation results of the target to be processed at multiple target times. These multiple target times include the current time and multiple historical times preceding the current time. Based on the pose information and velocity estimation results corresponding to each target time, the pose of the target to be processed at the current time can be estimated. Then, the poses corresponding to each target time are fused to obtain the predicted pose of the target to be processed at the current time. There is no need to determine whether the target is in the blind zone of the detection algorithm. By acquiring the pose information and velocity estimation results corresponding to multiple target times of the target to be processed, and thereby estimating multiple poses of the target to be processed at the current time, and then fusing these multiple poses to obtain the predicted pose of the target to be processed at the current time, the impact of errors from using a single pose information or velocity estimation result on the pose estimation of the target to be processed is reduced, improving the stability and accuracy of the target pose estimation results.

[0084] In the process of target perception in autonomous driving environments, it is necessary to accurately estimate the poses of targets including motor vehicles, non-motor vehicles, and pedestrians. Pose describes the position, orientation, and size of a target. When identifying targets in the current scene and estimating their poses based on data from target detection sensors, the target pose is generally represented by the center coordinates of the bounding box, the size of the bounding box, and the orientation angle. The bounding box can be a cube with an outer envelope that can identify the target.

[0085] For example, a schematic diagram of the target bounding box of the target to be processed is shown in Figure 2(a), where X, Y, and Z are the X-axis, Y-axis, and Z-axis directions of the target detection sensor coordinate system, respectively, and O is the origin of the sensor coordinate system; the cube formed by the dashed lines is the target's outer envelope, i.e., the target bounding box; (x, y, z) are the center coordinates of the target bounding box; l is the length of the target bounding box, w is the width of the target bounding box, and h is the height of the target bounding box.

[0086] Figure 2(b) shows a schematic diagram of the orientation angle of the target to be processed. X and Y are the X-axis and Y-axis directions of the target detection sensor coordinate system, respectively. The dashed box is the projection of the target box in the XOY plane, l is the length of the target box, w is the width of the target box, and θ is the angle between the orientation of the target to be processed and the Y-axis direction of the sensor coordinate system, which is also the orientation angle.

[0087] In this embodiment of the invention, target detection, target tracking, global localization, and velocity estimation techniques from current target pose estimation methods can be utilized. Multiple poses are then fused to obtain the predicted pose of the target at the current moment. A flowchart of the target pose estimation method in this embodiment of the invention is shown below. Figure 3As shown, it mainly includes acquiring sensor data, target detection, target tracking, global positioning, velocity estimation, and fusion processing.

[0088] Electronic devices can acquire sensor data from target detection sensors in the current scene. The sensor data is used to calculate the pose information of the target to be processed. The sensor data can be laser point cloud, image, or other data, without specific limitations.

[0089] During target detection, electronic devices can identify targets in the current scene based on sensor data. These targets may include motor vehicles, non-motor vehicles, and pedestrians, and their poses can be estimated. The target pose estimation results output by target detection can be represented in the form of a target list. For example, the list of detected targets obtained at time t can be represented as D. t ={D t 1 D t 2 D t n}. D t i The pose information of the target i at time t can include a series of key parameters, such as (x, y, z, h, w, l, θ), where (x, y, z) are the center coordinates of the target box; l is the length of the target box; w is the width of the target box; h is the height of the target box; and θ is the orientation angle of the target.

[0090] In one implementation, the pose information may further include pose confidence, denoted as cof. d This is used to identify the accuracy of the target pose information. When the target to be processed is in the blind zone of the detection algorithm, the target detection will give a lower pose confidence score, and vice versa.

[0091] During target tracking, electronic devices can correlate the target pose estimation results of the same target at different times, summarizing them to form a target sequence information arranged in the time dimension. The electronic device can represent the target tracking results in the form of a target list. For example, the tracked target list obtained from target tracking at time t can be represented as T. t ={T t 1 T t 2 ,…,T t m}, where m represents the number of targets to be processed, and T t iThis represents the tracking sequence result of target i at time t, including the target pose estimation result of the target detection output from time tT to time t in the most recent period, i.e., T. t i ={D t i D t-1 i D t-2 i D t-T i The parameter T is the number of historical cache frames of the target pose estimation results output by the target detection. It can be set according to factors such as the computing power resources of the electronic device and the prediction accuracy requirements, and is not specifically limited here.

[0092] During global positioning, electronic devices can acquire positioning-related sensor data, such as GPS (Global Positioning System) data, and then obtain the pose information of the target in the global coordinate system based on the sensor data. To facilitate the subsequent transformation of pose information between the sensor's local coordinate system and the global coordinate system, a rotation and translation matrix R from the sensor's local coordinate system to the global coordinate system can be used. 4*4 This matrix represents the result of global positioning. By multiplying the coordinates of a point in the local coordinate system of the sensor by this matrix, we can obtain its coordinates in the global coordinate system.

[0093] During velocity estimation, the electronic device can predict the velocity estimation result of the target to be processed based on the target sequence information obtained during target tracking. The velocity estimation result can include linear velocity and angular velocity. The velocity estimation result V of the target to be processed is obtained from the velocity estimation. t i You can use a vector (v) x v y v z ,ω) means, v x v represents the linear velocity of the target in the X-axis direction in the global coordinate system. y v represents the linear velocity of the target object along the Y-axis in the global coordinate system. z Let ω represent the linear velocity of the target in the Z-axis direction in the global coordinate system, and ω represent the angular velocity of the target's orientation angle. Since the target pose estimation results of the same target at different times have been associated during target tracking, the velocity estimation results can be added to the target pose estimation results. Each frame of target pose estimation results corresponds to one velocity estimation result, that is, the target pose estimation result of target i from time tT to time t with velocity estimation results, which can be identified as T. t i ={Dt i D t-1 i D t-2 i D t-T i V t i V t-1 i V t-2 i , ..., V t-T i}

[0094] In one implementation, the velocity estimation result may include a velocity confidence score, which can be denoted as cof. v The confidence level of velocity is used to indicate the accuracy of the velocity estimation result. The confidence level of velocity can be related to the algorithm used for velocity estimation and the confidence level of the pose information corresponding to the velocity estimation result, but no specific limitation is made here.

[0095] In step S101 of this embodiment of the invention, the electronic device can acquire the pose information and velocity estimation results of the target at multiple target times.

[0096] Pose information and velocity estimation results can be used to estimate the pose of the target at the current moment. During the velocity estimation process, the electronic device can obtain the target pose estimation result with velocity estimation results for the target. The electronic device can determine the current moment and multiple historical moments prior to the current moment as target moments, and then obtain the target pose estimation results with velocity estimation results for the target at multiple target moments, which serve as the pose information and velocity estimation results for the target at multiple target moments.

[0097] Since the pose information of the target to be processed can represent the pose of the target at a historical moment when the pose information was acquired, the electronic device can acquire multiple velocity estimation results of the target from that historical moment to the current moment. Using the velocity estimation results corresponding to the target at multiple target moments, the pose change of the target from that historical moment to the current moment can be calculated, and thus the pose of the target at the current moment can be estimated. Therefore, in step S102 above, the electronic device can estimate the pose of the target at the current moment based on the pose information and velocity estimation results corresponding to each target moment.

[0098] In one implementation, since the value obtained by integrating the velocity over time represents the displacement of the moving object during that time period, for each target moment of the target to be processed, the electronic device can integrate the velocity estimation result of the target to be processed from that target moment to the current moment over time to obtain the pose change of the target to be processed from that target moment to the current moment. Then, the pose change is superimposed on the pose information of the target to be processed at that target moment to obtain the estimated pose of the target to be processed at the current moment.

[0099] For example, a schematic diagram of the multiple poses of the target i at the current time estimated by the electronic device is shown below. Figure 4 As shown, the multiple poses of the target i at the current time estimated by the electronic device are P. i ={P0 i P1 i , ..., P T i}, where T represents the total number of estimated poses of target i at the current time. P0 i P1 represents the pose of target i at the current time, estimated at target time 0; i P represents the pose of target i at the current time, estimated at target time 1; T i Let P represent the pose of the target i at the current time, estimated from the target time T. j i It can include a series of key parameters of the target i to be processed, including center coordinates, orientation angle, target box size, etc., which are consistent with the pose information of the target to be processed and can be represented by (x, y, z, h, w, l, θ). Where j∈[0,T].

[0100] After the electronic device estimates multiple poses of the target to be processed at the current time, it can fuse the estimated multiple poses at the current time to obtain the predicted pose of the target to be processed at the current time, which is to perform the above step S103.

[0101] Since both the electronic device and the target may be in motion, the target may be outside or outside the detection algorithm's blind zone at different target times. The pose information obtained during target detection and the velocity estimation results obtained during target estimation may be inaccurate, leading to significant differences in the estimated poses at different target times. Estimating the pose using only a single pose information and velocity estimation result results in a large error. Therefore, the electronic device can fuse the estimated poses for each target time to obtain the predicted pose of the target at the current time. Because the target is mostly outside the detection algorithm's blind zone, and the time spent in the blind zone is short, most pose information and velocity estimation results are relatively accurate. After fusion processing, the predicted pose of the target at the current time is also relatively accurate.

[0102] A schematic diagram of the predicted pose of target i at the current time can be shown as follows: Figure 5 As shown, where P i The multiple poses of the target i at the current time are estimated by the electronic device. By fusing these multiple poses, the predicted pose of the target i at the current time can be obtained.

[0103] By employing the solution provided in this invention, an electronic device can acquire pose information and velocity estimation results of a target at multiple target times, estimate multiple poses of the target at the current time, and then fuse the estimated poses at these multiple target times to obtain the predicted pose of the target at the current time. Without altering existing methods for target pose estimation, there is no need to determine whether the target is in the blind zone of the detection algorithm. The fusion of poses at multiple target times can be performed with relatively low computational cost, reducing the impact of errors from using a single pose information or velocity estimation result on the target pose estimation and improving the stability and accuracy of the target pose estimation results.

[0104] As one embodiment of the present invention, since the motion of the target to be processed may include movement and rotation, the above velocity estimation results may include linear velocity estimates and angular velocity estimates.

[0105] like Figure 6 As shown, the step of estimating the pose of the target at the current moment based on the pose information and velocity estimation results corresponding to each target moment can include:

[0106] S601, based on the estimated linear velocity value corresponding to each target time, perform integration over time to obtain the displacement change of the target from that target time to the current time.

[0107] In order to determine the movement of the target from the target time to the current time, for each target time, the electronic device can integrate the estimated linear velocity of the target from that target time to the current time over time to obtain the displacement change of the target from that target time to the current time.

[0108] In one implementation, it is assumed that the time interval for obtaining the velocity estimation result is Δt. When Δt is small, it can be approximated that the target to be processed moves at a constant speed within the time interval Δt. That is, the displacement change can be obtained by accumulating the product of the linear velocity estimate corresponding to each target moment and Δt.

[0109] S602, based on the estimated angular velocity value corresponding to each target time, perform integration over time to obtain the rotational change of the target from that target time to the current time.

[0110] Accordingly, in order to determine the rotation of the target from the target time to the current time, for each target time, the electronic device can perform an integral operation on the estimated angular velocity of the target from that target time to the current time over time to obtain the change in rotation of the target from that target time to the current time.

[0111] In one implementation, it is assumed that the time interval for obtaining the velocity estimation result is Δt. When Δt is small, it can be approximated that the target to be processed moves at a constant speed within the time interval Δt. That is, the rotational change can be obtained by accumulating the product of the estimated angular velocity value corresponding to each target moment and Δt.

[0112] S603, based on the pose information, displacement change, and rotation change corresponding to each target moment, calculate the pose of the target to be processed at the current moment.

[0113] The pose information of the target at each target time represents the pose of the target at the target time when the pose information is acquired. Therefore, after the electronic device calculates the displacement and rotation changes of the target from the target time to the current time, it can superimpose the displacement and rotation changes onto the pose information of the target at the target time to estimate multiple poses of the target at the current time. Each target time corresponds to an estimated pose at the current time.

[0114] In this embodiment, for each target moment, the electronic device can integrate the estimated linear velocity of the target over time to obtain the displacement change of the target from that target moment to the current moment, and integrate the estimated angular velocity of the target over time to obtain the rotation change of the target from that target moment to the current moment, thereby calculating the pose of the target at the current moment. By calculating the displacement and rotation changes of the target separately, the estimated poses of the target at the current moment are more accurate, which helps to improve the accuracy of the target pose estimation results.

[0115] As one embodiment of the present invention, such as Figure 7 As shown, the steps described above for calculating the pose of the target at the current moment based on the pose information, displacement change, and rotation change corresponding to each target moment can include:

[0116] S701 transforms the pose information corresponding to each target moment into the global coordinate system using the rotation and translation matrix from the local coordinate system to the global coordinate system, thus obtaining the first global pose.

[0117] Since the pose information output by the target detection process is the pose of the target in the local coordinate system of the sensor, and the target may also be in motion as the target moves over time, the pose of the target detection sensor may change, which will cause the local coordinate system of the sensor to change, thus affecting the accuracy of the pose information output by the target detection process.

[0118] Therefore, in order to eliminate the influence of changes in the local coordinate system of the sensor on the estimation of the target pose, the electronic device can transform the pose information corresponding to each target moment into the global coordinate system according to the rotation and translation matrix from the local coordinate system of the sensor to the global coordinate system, and obtain the first global pose.

[0119] For example, within the time interval from historical time tT to the current time t, the target i to be processed has pose information D at time j. j i That is, the pose information corresponding to time j is (x local y local , z local h, w, l, θ local During global positioning, the electronic device acquires the rotation and translation matrix R from the sensor's local coordinate system to the global coordinate system at time j. 4*4 (j), then the electronic device can transform the pose information corresponding to time j into the global coordinate system using the rotation and translation matrix from the sensor's local coordinate system to the global coordinate system, thus obtaining (x). global y global, z global h, w, l, θ global ), thus obtaining the first global pose.

[0120] S702, based on the first global pose, displacement change, and rotation change corresponding to each target time, calculate the second global pose of the target to be processed at the current time.

[0121] For each target time, the first global pose of the target at that target time represents the global pose of the target at that target time. Therefore, after the electronic device calculates the displacement and rotation changes of the target from the target time to the current time, it can add the displacement and rotation changes to the first global pose of the target at that target time, thereby estimating multiple second global poses of the target at the current time. Each target time corresponds to one estimated second global pose.

[0122] In one implementation, the electronic device can calculate the second global pose of the target at the current moment according to the following formula:

[0123]

[0124]

[0125]

[0126]

[0127] The time interval for obtaining the velocity estimation results is Δt. The center coordinates of the target to be processed are obtained through calculation. v is the calculated orientation angle of the target to be processed. x (k) represents the linear velocity of the target at time k along the X-axis, v y (k) represents the linear velocity of the target in the Y-axis direction at time k, v z (k) represents the linear velocity of the target in the Z-axis direction at time k, ω x (k) represents the angular velocity of the target at time k. The size of the target bounding box in the second global pose is consistent with the size of the target bounding box in the first global pose.

[0128] S703, the second global pose is transformed into the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system to obtain the pose of the target to be processed at the current moment.

[0129] During global localization, the electronic device can obtain the rotation and translation matrix from the sensor's local coordinate system to the global coordinate system at the current moment. Its inverse matrix is ​​the rotation and translation matrix from the global coordinate system to the sensor's local coordinate system at the current moment. Then, the second global pose can be transformed back to the sensor's local coordinate system according to the rotation and translation matrix from the global coordinate system to the sensor's local coordinate system to obtain the pose of the target to be processed at the current moment.

[0130] In this embodiment, the electronic device can convert the pose information corresponding to each target moment to the global coordinate system, and then estimate the pose of the target at the current moment in the global coordinate system. This can eliminate the influence of the target detection sensor's own motion on the estimation of the target pose and improve the accuracy of the target pose estimation.

[0131] As one embodiment of the present invention, the above pose information may include the center coordinates, orientation angle, and target bounding box size of the target to be processed.

[0132] like Figure 8 As shown, the steps described above, which transform the pose information corresponding to each target moment into the global coordinate system using a rotation and translation matrix from the sensor's local coordinate system to the global coordinate system, to obtain the first global pose, may include:

[0133] S801, based on the center coordinates, target frame size and orientation angle of the target to be processed at each target time, calculate the coordinates of multiple first key points of the target to be processed.

[0134] To maximize the accuracy of pose prediction, in addition to utilizing the center coordinates of the target, information from other key points can also be used, such as the eight vertices of the target's bounding box. Therefore, the electronic device can calculate the coordinates of multiple first key points of the target based on its center coordinates, bounding box size, and orientation angle at each target moment. Since the center coordinates, bounding box size, and orientation angle of the target are fixed, the coordinates of other points on the bounding box can be calculated based on geometric relationships, serving as the coordinates of the first key points.

[0135] S802, according to the rotation and translation matrix from the local coordinate system to the global coordinate system of the sensor, calculate the multiple first key point coordinates transformed into multiple first global coordinates in the global coordinate system.

[0136] After determining the coordinates of the first key point, the electronic device can multiply each first key point coordinate with the corresponding rotation and translation matrix from the local coordinate system to the global coordinate system to obtain the first global coordinates of the first key point coordinates transformed into the global coordinate system.

[0137] Taking the coordinates (x, y, z) of a first key point of the target i at time j as an example, the electronic device can calculate the first global coordinates of the key point in the global coordinate system according to the following formula:

[0138]

[0139] Where (x, y, z) are the coordinates of the first keypoint, (x', y', z') are the first global coordinates after transformation to the global coordinate system, and R 4*4 (j) is the rotation and translation matrix from the sensor's local coordinate system to the global coordinate system at time j.

[0140] S803, based on the plurality of first global coordinates and the target bounding box size, calculate the center coordinates and orientation angle of the target to be processed in the global coordinate system, as the first global pose.

[0141] Through the above coordinate system transformation, the electronic device can obtain multiple first global coordinates of the target to be processed at each target time. Since the target box size of the target to be processed does not change during the transformation from the local coordinate system to the global coordinate system of the sensor, the electronic device can calculate the center coordinates and orientation angle of the target to be processed based on multiple first global coordinates and the target box size, and use them as the first global pose.

[0142] Correspondingly, such as Figure 9 As shown, the step of transforming the second global pose to the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system to obtain the pose of the target at the current moment may include:

[0143] S901, based on the second global pose and the target bounding box size, calculate the coordinates of multiple second key points of the target to be processed.

[0144] After calculating the second global pose of the target at the current moment based on the first global pose, displacement change, and rotation change corresponding to each target moment, the electronic device may change the center coordinates and orientation angle of the target, but the target bounding box size will not change. In order to utilize as much of the target's pose information as possible, rather than just the information of a single point, the electronic device can calculate the coordinates of multiple second key points of the target based on the second global pose and the target bounding box size. In one embodiment, the coordinates of the second key points can be the coordinates of the eight vertices of the target bounding box, which is not specifically limited here.

[0145] S902, according to the rotation and translation matrix from the global coordinate system to the sensor local coordinate system, calculate the multiple second key point coordinates transformed into multiple second local coordinates in the sensor local coordinate system.

[0146] After determining the coordinates of the second key point, the electronic device can multiply each second key point coordinate with the corresponding rotation and translation matrix from the global coordinate system to the sensor local coordinate system, thereby obtaining multiple second key point coordinates transformed into multiple second local coordinates in the sensor local coordinate system.

[0147] For example, an electronic device can obtain the rotation and translation matrix from the sensor's local coordinate system to the global coordinate system at the current time t, which is R. 4*4 (t), then the rotation and translation matrix from the global coordinate system to the sensor's local coordinate system at the current time t is R. 4*4 (t) -1 The coordinates (x, y) of a second key point of the target i at time t. p y p , z p For example, the electronic device can calculate the coordinates of the key point and transform them into the second local coordinates in the sensor's local coordinate system using the following formula:

[0148]

[0149] Among them, (x p ',y p ',z p ') represents the coordinates of the second key point (x) p y p , z p Transform to the second local coordinate system in the sensor's local coordinate system, R 4*4 (t) -1 Let be the rotation and translation matrix from the global coordinate system to the sensor's local coordinate system at time t.

[0150] S903, based on the plurality of second local coordinates corresponding to each target time and the target box size, calculate the center coordinates and orientation angle of the target to be processed in the sensor local coordinate system, as the pose of the target to be processed at the current time.

[0151] For each target time, since the size of the target bounding box of the target to be processed remains unchanged, the electronic device can calculate the center coordinates and orientation angle of the target to be processed in the sensor local coordinate system based on multiple second local coordinates and the target bounding box size. This is used as the estimated pose of the target to be processed at the current time, thereby obtaining the poses corresponding to multiple target times.

[0152] In this embodiment, the electronic device can determine the coordinates of multiple key points of the target to be processed based on the center coordinates, target box size, and orientation angle of the target to be processed. By transforming the coordinates of multiple key points of the target to be processed between the local coordinate system and the global coordinate system of the sensor, the pose of multiple targets to be processed at the current moment is estimated, making full use of the pose information of the target to be processed, and further improving the accuracy of the target pose estimation result.

[0153] As one embodiment of the present invention, before the step of fusing the poses corresponding to each target time to obtain the predicted pose of the target to be processed at the current time, the method may further include:

[0154] Calculate the confidence level of the target's pose at the current moment.

[0155] Since the target to be processed may be in or outside the blind zone of the detection algorithm at different target times, the accuracy of the pose information and velocity estimation results corresponding to different target times may be different. In order to make reasonable use of the pose information and velocity estimation results with different accuracies, the electronic device can calculate the confidence level of the estimated pose of the target to be processed at the current time, so that the poses with different confidence levels can make appropriate contributions in the process of calculating the predicted pose of the target to be processed at the current time.

[0156] Accordingly, the step of fusing the poses corresponding to each target time to obtain the predicted pose of the target at the current time may include:

[0157] Based on the confidence level of the pose of the target at the current time, the poses of each target at each time time are fused to obtain the predicted pose of the target at the current time.

[0158] If the target is outside the blind zone of the detection algorithm, its pose information and velocity estimation results are relatively accurate, and the confidence level corresponding to its pose at the current moment is higher, thus contributing more to the pose fusion process. Conversely, the pose information and velocity estimation results for targets within the blind zone of the detection algorithm are less accurate, with larger errors, and the confidence level corresponding to the pose is lower, resulting in a smaller contribution to the pose fusion process. Therefore, the electronic device can fuse the estimated poses corresponding to each target moment based on the confidence level of the target's pose at the current moment, thereby obtaining the accurate predicted pose of the target at the current moment.

[0159] In this embodiment, the electronic device can calculate the confidence level corresponding to the pose of the target at the current moment, and based on the confidence level, perform fusion processing on the estimated poses corresponding to each target moment to obtain the predicted pose of the target at the current moment. This allows the target pose information and velocity estimation results outside the blind zone of the detection algorithm to contribute more to the target pose estimation process, further improving the accuracy of the target pose estimation results.

[0160] As one embodiment of the present invention, the pose information may include pose confidence, and the velocity estimation result may include velocity confidence, such as... Figure 10 As shown, the step of calculating the confidence level corresponding to the pose of the target to be processed at the current moment may include:

[0161] S1001, calculate the product of multiple velocity confidence scores from the target time to the current time, which is the target velocity confidence score.

[0162] Since both the pose information output by the target detection process and the velocity estimation result output by the velocity estimation process affect the accuracy of target pose estimation, the influence of the pose confidence level of the pose information and the velocity confidence level of the velocity estimation result can be considered comprehensively. The pose confidence level is used to indicate the accuracy of the pose information, and the velocity confidence level is used to indicate the accuracy of the velocity estimation result.

[0163] In the process of calculating displacement change using velocity estimation results, the accumulated error increases with the increase of calculation time, and the confidence level of the estimated pose of the target at the current moment should decrease accordingly. In order to accurately determine the confidence level of the target pose at the current moment, the electronic device can calculate the product of multiple velocity confidence levels from the target moment to the current moment, and use it as the target velocity confidence level at that target moment.

[0164] S1002, calculate the product of the pose confidence level corresponding to the target time and the target velocity confidence level, and use it as the confidence level of the pose of the target to be processed at the current time.

[0165] After obtaining the pose confidence and target velocity confidence corresponding to the target time, the electronic device can calculate the product of the pose confidence and target velocity confidence corresponding to the target time, which is used as the estimated pose confidence of the target to be processed at the current time.

[0166] As one implementation method, the pose confidence level of target i at target time j obtained in the target detection process is cof d The velocity confidence level at target time k obtained from the velocity estimation process from target time j to current time t is cof v(k), the electronic device can calculate the confidence level corresponding to the pose of the target i at the current time according to the following formula:

[0167]

[0168] Among them, cof p Let cof be the confidence level of the estimated pose of target i at the current time corresponding to target time j. d Let be the pose confidence of target i at target time j. Let k be the target velocity confidence of target i from target time j to current time t, where k∈[j,t].

[0169] In this embodiment, the electronic device can calculate the product of multiple velocity confidence scores from the target time to the current time as the target velocity confidence score, and calculate the product of the pose confidence score corresponding to the target time and the target velocity confidence score as the confidence score corresponding to the pose of the target at the current time. This method can accurately determine the confidence score corresponding to the pose of the target at the current time, thus enabling the estimated confidence score to comprehensively consider the cumulative error of the velocity estimation result and the error of the pose information, making the confidence score calculation more accurate and improving the accuracy of the target pose estimation result.

[0170] As one embodiment of the present invention, such as Figure 11 As shown, the step of fusing the poses of each target at the current time based on the confidence level of the target at the current time may include:

[0171] S1101, based on the confidence level, identify abnormal poses in the poses corresponding to each target time, and remove the abnormal poses to obtain the target predicted pose.

[0172] In the estimated poses corresponding to each target time step, a higher confidence level indicates higher accuracy, and this confidence level can be used for fusion processing to obtain the predicted pose of the target at the current time step. Conversely, a lower confidence level indicates lower accuracy, suggesting the target may be in the blind zone of the detection algorithm, and the pose information and velocity estimation results output by the target detection process may be inaccurate. Confidence levels can be used to determine the prediction deviation of multiple estimated poses. Therefore, to further improve the accuracy of the target pose estimation results, the electronic device can calculate the deviation between the current pose and the multiple estimated poses based on the confidence level of the target's pose at the current time step. This allows for the identification and removal of abnormal poses among the estimated poses corresponding to each target time step, resulting in the predicted target pose.

[0173] S1102, the predicted pose of the target is calculated by weighting the corresponding confidence level to obtain the predicted pose of the target at the current time.

[0174] For the target's predicted pose, the higher the confidence level, the higher the accuracy level, and the greater the contribution to the calculation of the predicted pose. Therefore, electronic devices can calculate the predicted pose of the target according to the corresponding confidence level by weighted averaging, so as to obtain the predicted pose of the target at the current moment.

[0175] Specifically, for each parameter included in the target's predicted pose, the electronic device can calculate the weighted average of the parameter according to the corresponding confidence level, and the weight is the confidence level corresponding to the parameter, thereby obtaining the predicted pose of the target to be processed at the current time.

[0176] In one implementation, taking the X-axis coordinate in the center coordinates of the target's predicted pose as an example, assuming λ k To fuse valid flags, when the pose corresponding to the target at a given time is an abnormal pose, the corresponding λ k The value is 0 when the pose corresponding to the target at any given time is not an abnormal pose, and the corresponding λ is 0. k For a value of 1, the electronic device can calculate the weighted average of the X-axis coordinates in the center coordinate system using the following formula:

[0177]

[0178] Where, x f λ is the weighted average of the X-axis coordinates in the predicted pose center coordinates of each target. k To integrate effective markers, COF k Let x be the confidence level of the predicted pose of the target at historical time k. k The x-axis coordinate is the center coordinate of the target's predicted pose at historical time k.

[0179] In this embodiment, the electronic device can identify and remove abnormal poses in the estimated poses corresponding to each target time based on confidence levels, obtain the target predicted pose, and calculate a weighted average of the target predicted poses according to the corresponding confidence levels to obtain the predicted pose of the target at the current time. This can effectively avoid the influence of the detection algorithm blind zone on the target pose estimation. By using confidence-weighted averaging and abnormal pose identification, the contribution of the estimated poses outside the detection algorithm blind zone to the predicted pose of the target at the current time is increased, further improving the accuracy of the target pose estimation results.

[0180] As one embodiment of the present invention, such as Figure 12As shown, the steps for identifying abnormal poses in the pose corresponding to each target time based on the confidence level may include:

[0181] S1201, for each parameter included in the pose corresponding to each target time, the parameter is weighted and averaged according to the corresponding confidence level to obtain the weighted average value of the parameter.

[0182] The estimated pose of the target to be processed can include multiple parameters, such as center coordinates, orientation angle, and bounding box size. The estimated pose of the target to be processed can be in the form of (x, y, z, h, w, l, θ). For each parameter, the electronic device can calculate a weighted average of the parameter according to its corresponding confidence level, and the weight is the confidence level corresponding to the parameter. The parameter can include at least one of the center coordinates, orientation angle, and bounding box size. Taking the X-axis coordinate in the center coordinate system as an example, the electronic device can calculate the weighted average of the X-axis coordinate in the center coordinate system according to the following formula:

[0183]

[0184] Where, x a cof is the weighted average of the X-axis coordinates in the center coordinates of the poses of the targets to be processed obtained from various estimates. k To estimate the confidence level corresponding to the pose k of the target to be processed, x k The X-axis coordinate is the center coordinate of the estimated pose k of the target to be processed.

[0185] S1202, calculate the difference between each parameter and its corresponding weighted average, as the prediction bias value.

[0186] The weighted average reflects the average magnitude of the corresponding parameter. The difference between the weighted average and the parameter can characterize the accuracy of the parameter. Therefore, for each parameter in the estimated pose of the target to be processed, the electronic device can calculate the difference between each parameter in the estimated pose and the corresponding weighted average as the prediction bias value.

[0187] As one implementation method, the electronic device can calculate the prediction deviation value of the X-axis coordinate in the center coordinates of the estimated pose k of the target to be processed according to the following formula:

[0188] Δx k =x k -x a

[0189] Where, Δx kx is the prediction deviation value of the X-axis coordinate of the center coordinate of the pose of the target to be processed obtained from the estimation at historical time k. k Let x be the X-axis coordinate of the center coordinate of the pose of the target to be processed at the estimated time k in history. a It is the weighted average of the X-axis coordinates of the center coordinates of the poses of the targets to be processed obtained from each estimation.

[0190] S1203, Based on the relationship between the predicted deviation value and the preset threshold, determine the abnormal pose.

[0191] The prediction deviation value can characterize the accuracy of the corresponding parameter. Therefore, the electronic device can determine the relationship between the prediction deviation value of each parameter in the pose corresponding to each target time and the preset threshold. Then, based on this relationship, it can determine whether the accuracy of the parameter is too low. If it is too low, the pose can be identified as an abnormal pose. The preset threshold can be positively correlated with the standard deviation of the parameters of the pose corresponding to each target time, and is not specifically limited here.

[0192] In this embodiment, the electronic device can calculate the difference between each parameter and its corresponding weighted average value in the estimated pose at each target time, using this difference as a prediction deviation value. Based on the relationship between the prediction deviation value and a preset threshold, abnormal poses are determined. By comparing the relationship between each parameter and its corresponding prediction deviation value and the preset threshold, it is possible to accurately determine whether the estimated pose at each target time is an abnormal pose, thereby further improving the accuracy of the subsequently determined predicted pose.

[0193] As one embodiment of the present invention, the step of determining abnormal pose based on the relationship between the predicted deviation value and the preset threshold may include:

[0194] If at least one prediction deviation value is greater than a first preset threshold, the pose corresponding to that prediction deviation value is determined to be an abnormal pose. Alternatively, according to the preset weights corresponding to each parameter, a weighted sum of the prediction deviation values ​​corresponding to each parameter in the pose at each target time is calculated. If the weighted sum is greater than a second preset threshold, the pose corresponding to that weighted sum is determined to be an abnormal pose.

[0195] Depending on the specific target pose estimation scenario, the stringency of the evaluation for the pose at each estimated target time varies. In one case, it is necessary to accurately estimate each parameter of the pose at each estimated target time. If even one parameter is inaccurate, the pose including that parameter is considered inaccurate. The electronic device can then compare the prediction deviation value for each parameter with a first preset threshold. If at least one prediction deviation value is greater than the first preset threshold, the pose corresponding to that prediction deviation value is determined to be an abnormal pose.

[0196] In another scenario, it is only necessary to make an overall judgment on the parameters of the pose estimation corresponding to each target time. Then, the electronic device can calculate the weighted sum of the prediction deviation values ​​of each parameter in the pose estimated for each target time according to the preset weights of each parameter. If the weighted sum is greater than the second preset threshold, the pose corresponding to the weighted sum is determined to be an abnormal pose.

[0197] In this embodiment, the electronic device can individually judge each parameter included in the estimated pose corresponding to each target time moment, or judge the pose as a whole including all parameters, thereby determining whether the corresponding pose is an abnormal pose. The method for determining abnormal poses can be selected according to actual application requirements. Regardless of the method used, abnormal poses in the estimated pose corresponding to each target time moment can be accurately identified, thereby obtaining the target predicted pose and improving the accuracy of the predicted pose.

[0198] Corresponding to the above-described target pose estimation method, this invention also provides a target pose estimation device. The target pose estimation device provided by this invention will be described below.

[0199] like Figure 13 As shown, a target pose estimation device includes:

[0200] The information acquisition module 1301 is used to acquire the pose information and velocity estimation results of the target at multiple target times.

[0201] The multiple target times include the current time and multiple historical times preceding the current time.

[0202] The pose estimation module 1302 is used to estimate the pose of the target to be processed at the current time based on the pose information and velocity estimation results corresponding to each target time.

[0203] The fusion processing module 1303 is used to fuse the poses corresponding to each target time to obtain the predicted pose of the target to be processed at the current time.

[0204] As can be seen, in the solution provided by this embodiment of the invention, the electronic device can acquire the pose information and velocity estimation results of the target to be processed at multiple target times. These multiple target times include the current time and multiple historical times preceding the current time. Based on the pose information and velocity estimation results corresponding to each target time, the pose of the target to be processed at the current time can be estimated. Then, the poses corresponding to each target time are fused to obtain the predicted pose of the target to be processed at the current time. There is no need to determine whether the target is in the blind zone of the detection algorithm. By acquiring the pose information and velocity estimation results corresponding to multiple target times of the target to be processed, and thereby estimating multiple poses of the target to be processed at the current time, and then fusing these multiple poses to obtain the predicted pose of the target to be processed at the current time, the impact of errors from using a single pose information or velocity estimation result on the pose estimation of the target to be processed is reduced, improving the stability and accuracy of the target pose estimation results.

[0205] As one embodiment of the present invention, the above-mentioned velocity estimation results may include linear velocity estimates and angular velocity estimates.

[0206] The pose estimation module 1302 mentioned above may include:

[0207] The displacement integration submodule is used to perform integration over time based on the estimated linear velocity value corresponding to each target time to obtain the displacement change of the target from the target time to the current time.

[0208] The rotation integral submodule is used to perform integration over time based on the estimated angular velocity value corresponding to each target moment to obtain the rotational change of the target from that target moment to the current moment.

[0209] The pose calculation submodule is used to calculate the pose of the target at the current moment based on the pose information, displacement change, and rotation change corresponding to each target moment.

[0210] As one embodiment of the present invention, the above pose calculation submodule may include:

[0211] The first transformation unit is used to transform the pose information corresponding to each target moment into the global coordinate system according to the rotation and translation matrix from the local coordinate system of the sensor to the global coordinate system, so as to obtain the first global pose.

[0212] The pose calculation unit calculates the second global pose of the target at the current moment based on the first global pose, displacement change, and rotation change corresponding to each target moment.

[0213] The second transformation unit is used to transform the second global pose into the sensor local coordinate system according to the rotation and translation matrix from the global coordinate system to the sensor local coordinate system, so as to obtain the pose of the target to be processed at the current time.

[0214] As one embodiment of the present invention, the pose information may include the center coordinates, orientation angle, and target bounding box size of the target to be processed.

[0215] The first conversion unit mentioned above may include:

[0216] The first calculation subunit is used to calculate the coordinates of multiple first key points of the target to be processed based on the center coordinates, target frame size and orientation angle of the target to be processed at each target time.

[0217] The first transformation subunit is used to calculate the multiple first key point coordinates transformed into multiple first global coordinates in the global coordinate system according to the rotation and translation matrix from the local coordinate system to the global coordinate system of the sensor.

[0218] The first acquisition subunit is used to calculate the center coordinates and orientation angle of the target to be processed in the global coordinate system based on the plurality of first global coordinates and the target box size, as the first global pose.

[0219] The second conversion unit mentioned above may include:

[0220] The second calculation subunit is used to calculate the coordinates of multiple second key points of the target to be processed based on the second global pose and the target bounding box size.

[0221] The second transformation subunit is used to calculate the transformation of the coordinates of the plurality of second key points to a plurality of second local coordinates in the sensor local coordinate system according to the rotation and translation matrix from the global coordinate system to the sensor local coordinate system.

[0222] The second acquisition subunit is used to calculate the center coordinates and orientation angle of the target to be processed in the sensor local coordinate system based on the plurality of second local coordinates corresponding to each target time and the target box size, as the pose of the target to be processed at the current time.

[0223] As one embodiment of the present invention, the above-mentioned apparatus may further include:

[0224] The confidence calculation module is used to calculate the confidence of the pose of the target at the current time before the step of fusing the poses corresponding to each target time to obtain the predicted pose of the target at the current time.

[0225] The aforementioned fusion processing module 1303 may include:

[0226] The fusion processing submodule is used to fuse the poses corresponding to each target time based on the confidence level corresponding to the pose of the target at the current time, so as to obtain the predicted pose of the target at the current time.

[0227] As one embodiment of the present invention, the pose information may include pose confidence, and the velocity estimation result may include velocity confidence.

[0228] The confidence calculation module mentioned above may include:

[0229] The target velocity confidence calculation unit is used to calculate the product of multiple velocity confidences from the target time to the current time, which is the target velocity confidence.

[0230] The confidence calculation unit is used to calculate the product of the pose confidence at the target time and the target velocity confidence, which is used as the confidence of the pose of the target to be processed at the current time.

[0231] As one embodiment of the present invention, the above-mentioned fusion processing submodule may include:

[0232] The target prediction pose acquisition unit is used to identify abnormal poses in the poses corresponding to each target time based on the confidence level, and remove the abnormal poses to obtain the target prediction pose.

[0233] The fusion processing unit is used to calculate the predicted pose of the target according to the corresponding confidence level by performing a weighted average calculation to obtain the predicted pose of the target to be processed at the current time.

[0234] As one embodiment of the present invention, the target prediction pose acquisition unit may include:

[0235] The weighted average calculation subunit is used to calculate the weighted average of each parameter in the pose corresponding to each target time according to the corresponding confidence level.

[0236] The parameters include at least one of the following: center coordinates, orientation angle, and target bounding box size.

[0237] The prediction deviation calculation subunit is used to calculate the difference between each parameter and its corresponding weighted average, which is used as the prediction deviation value.

[0238] An abnormal pose determination subunit is used to determine abnormal poses based on the relationship between the predicted deviation value and a preset threshold.

[0239] As one embodiment of the present invention, the above-mentioned abnormal pose determination subunit may include:

[0240] The abnormal pose determination section is used to determine the pose corresponding to the prediction deviation value as an abnormal pose if there is at least one prediction deviation value greater than a first preset threshold; or, according to the preset weights corresponding to each parameter, calculate the weighted sum of the prediction deviation values ​​corresponding to each parameter in the pose at each target time; if the weighted sum value is greater than a second preset threshold, determine the pose corresponding to the weighted sum value as an abnormal pose.

[0241] This invention also provides an autonomous driving device that can determine its own travel path based on the aforementioned target pose estimation method. For example, the autonomous driving device can be a robot, an unmanned vehicle, an autonomous driving vehicle, an assisted driving vehicle, etc., and is not specifically limited thereto.

[0242] like Figure 14 As shown, an autonomous driving device includes:

[0243] The processor 1401 is configured to execute the target pose estimation method described in any of the above embodiments to obtain multiple predicted poses of the target to be processed; and generate the travel path of the autonomous driving device based on the multiple predicted poses.

[0244] As can be seen, in the solution provided by the embodiments of the present invention, the processor can execute the target pose estimation method described in any of the above embodiments to obtain multiple predicted poses of the target to be processed. Then, a travel path for the autonomous driving device can be generated based on the multiple predicted poses. The processor obtains pose information and velocity estimation results corresponding to multiple target times of the target to be processed, and thereby estimates multiple poses of the target to be processed at the current time. Then, the multiple poses are fused to obtain the predicted pose of the target to be processed at the current time. This reduces the impact of errors from using a single pose information or velocity estimation result on the pose estimation of the target to be processed, improving the stability and accuracy of the target pose estimation results. Therefore, the processor can generate an accurate travel path for the autonomous driving device based on multiple predicted poses.

[0245] As one embodiment of the present invention, such as Figure 15 As shown, the above-mentioned autonomous driving device may further include:

[0246] Display 1501 is used to display a map of the travel path and the environment surrounding the autonomous driving device. And / or,

[0247] The controller 1502 is used to control the autonomous driving device to perform autonomous driving based on the travel path.

[0248] Once the autonomous driving device's path is determined, the display can show a map of that path and the surrounding environment for the user to view. Based on this path, the controller can then guide the autonomous driving device to proceed to the target location and perform the corresponding operation. This enables automatic path planning and autonomous driving for the device.

[0249] This invention also provides an electronic device, such as... Figure 16 As shown, it includes a processor 1601, a communication interface 1602, a memory 1603, and a communication bus 1604. The processor 1601, communication interface 1602, and memory 1603 communicate with each other via the communication bus 1604.

[0250] Memory 1603 is used to store computer programs;

[0251] The processor 1601, when executing a program stored in the memory 1603, implements the steps of the method described in any of the above embodiments.

[0252] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0253] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0254] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0255] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0256] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0257] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of the method described in any of the above embodiments.

[0258] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0259] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0260] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for the target pose estimation device, autonomous driving device, electronic device, computer-readable storage medium, and computer program product are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0261] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for estimating the pose of a target, characterized in that, The method includes: Acquire the pose information and velocity estimation results of the target to be processed at multiple target times, wherein the multiple target times include the current time and multiple historical times before the current time; Based on the pose information and velocity estimation results corresponding to each target moment, estimate multiple poses of the target to be processed at the current moment; The multiple poses at the current moment are fused to obtain the predicted pose of the target at the current moment.

2. The method according to claim 1, characterized in that, The velocity estimation results include linear velocity estimates and angular velocity estimates; The step of estimating multiple poses of the target at the current moment based on the pose information and velocity estimation results corresponding to each target moment includes: Based on the estimated linear velocity value corresponding to each target time, the displacement change of the target from that target time to the current time is obtained by performing integration over time. Based on the estimated angular velocity value corresponding to each target moment, the integral is performed over time to obtain the rotational change of the target from that target moment to the current moment; Based on the pose information, displacement change, and rotation change corresponding to each target moment, the pose of the target to be processed at the current moment is calculated.

3. The method according to claim 2, characterized in that, The step of calculating the pose of the target at the current moment based on the pose information, displacement change, and rotation change corresponding to each target moment includes: The pose information corresponding to each target moment is transformed into the global coordinate system using the rotation and translation matrix from the local coordinate system to the global coordinate system to obtain the first global pose. Based on the first global pose, displacement change, and rotation change corresponding to each target time, the second global pose of the target to be processed at the current time is calculated. The second global pose is transformed into the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system, thus obtaining the pose of the target at the current moment.

4. The method according to claim 3, characterized in that, The pose information includes the center coordinates, orientation angle, and target bounding box size of the target to be processed; The step of transforming the pose information corresponding to each target time moment into the global coordinate system using a rotation and translation matrix from the sensor's local coordinate system to the global coordinate system to obtain the first global pose includes: Based on the center coordinates, target bounding box size, and orientation angle of the target to be processed at each target time, the coordinates of multiple first key points of the target to be processed are calculated; According to the rotation and translation matrix from the local coordinate system to the global coordinate system of the sensor, calculate the multiple first key point coordinates transformed into multiple first global coordinates in the global coordinate system; Based on the plurality of first global coordinates and the target bounding box size, the center coordinates and orientation angle of the target to be processed in the global coordinate system are calculated as the first global pose. The step of transforming the second global pose to the sensor local coordinate system using a rotation and translation matrix from the global coordinate system to the sensor local coordinate system, and obtaining the pose of the target at the current moment, includes: Based on the second global pose and the target bounding box size, the coordinates of multiple second key points of the target to be processed are calculated; According to the rotation and translation matrix from the global coordinate system to the sensor local coordinate system, calculate the multiple second key point coordinates transformed into multiple second local coordinates in the sensor local coordinate system; Based on the multiple second local coordinates corresponding to each target time and the target bounding box size, the center coordinates and orientation angle of the target to be processed in the sensor local coordinate system are calculated as the pose of the target to be processed at the current time.

5. The method according to any one of claims 1-4, characterized in that, Before the step of fusing multiple poses at the current moment to obtain the predicted pose of the target at the current moment, the method further includes: Calculate the confidence level of the target's pose at the current moment; The step of fusing the poses corresponding to each target time moment to obtain the predicted pose of the target to be processed at the current time includes: Based on the confidence level of the pose of the target at the current time, the poses of each target at each time time are fused to obtain the predicted pose of the target at the current time.

6. The method according to claim 5, characterized in that, The pose information includes pose confidence, and the velocity estimation result includes velocity confidence. The step of calculating the confidence level corresponding to the pose of the target to be processed at the current moment includes: The product of multiple velocity confidence scores from the target time to the current time is calculated to obtain the target velocity confidence score. The product of the pose confidence score at the target time and the target velocity confidence score is calculated and used as the confidence score of the pose of the target at the current time.

7. The method according to claim 5, characterized in that, The step of fusing the poses of each target at the current time based on the confidence level of the target's pose at the current time includes: Based on the confidence level, abnormal poses are identified in the poses corresponding to each target time, and the abnormal poses are removed to obtain the target predicted pose. The predicted pose of the target is calculated by weighting the corresponding confidence levels to obtain the predicted pose of the target at the current time.

8. The method according to claim 7, characterized in that, The step of identifying abnormal poses in the pose corresponding to each target time based on the confidence level includes: For each parameter of the pose corresponding to each target time, the parameter is weighted and averaged according to the corresponding confidence level to obtain the weighted average value of the parameter, wherein the parameter includes at least one of the center coordinates, orientation angle, and target box size; Calculate the difference between each parameter and its corresponding weighted average, and use it as the prediction bias value; Based on the relationship between the predicted deviation value and the preset threshold, abnormal poses are determined.

9. The method according to claim 8, characterized in that, The step of determining abnormal poses based on the relationship between the predicted deviation value and a preset threshold includes: If at least one prediction deviation value is greater than a first preset threshold, the pose corresponding to that prediction deviation value is determined to be an abnormal pose; or, According to the preset weights corresponding to each parameter, calculate the weighted sum of the prediction deviation values ​​of each parameter in the pose corresponding to each target time. If the weighted sum is greater than the second preset threshold, the pose corresponding to the weighted sum is determined to be an abnormal pose.

10. A target pose estimation device, characterized in that, The device includes: The information acquisition module is used to acquire the pose information and velocity estimation results of the target to be processed at multiple target times, wherein the multiple target times include the current time and multiple historical times before the current time; The pose estimation module is used to estimate multiple poses of the target to be processed at the current time based on the pose information and velocity estimation results corresponding to each target time. The fusion processing module is used to fuse multiple poses at the current moment to obtain the predicted pose of the target to be processed at the current moment.

11. An automatic driving device, characterized in that, The autonomous driving device includes: A processor is configured to execute the target pose estimation method according to any one of claims 1-9 to obtain multiple predicted poses of the target to be processed; and to generate a travel path of the autonomous driving device based on the multiple predicted poses.

12. The automatic driving device according to claim 11, characterized in that, The autonomous driving device also includes: A display for showing a map of the travel path and the environment surrounding the autonomous driving device; and / or, A controller is used to control the autonomous driving device to perform autonomous driving based on the travel path.

Images