A target state estimation method, apparatus, electronic device, and medium

Abstract

This disclosure relates to a target state estimation method, comprising: acquiring a data frame sequence corresponding to multiple time points; determining a first time window in the data frame sequence, the first time window including a set of first state variables to be optimized; determining multiple second time windows from within the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have overlapping data frames, and each second time window has a set of second state variables to be optimized; and obtaining an optimized set of first state variables by simultaneously optimizing the sets of second state variables within the multiple second time windows. The target state estimation method of this disclosure can obtain sufficiently accurate state estimates. Furthermore, a target state estimation device, electronic device, and medium are also proposed.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and particularly to the fields of autonomous driving and data processing technology, specifically to a target state estimation method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the process of target identification or observation, it is usually necessary to accurately estimate the target's state based on the target measurement data obtained by sensors. As the target moves, parameters such as velocity, angle, and acceleration constantly change, making the target's position highly correlated. For example, a crucial aspect of autonomous driving is the real-time estimation of the position, speed, size, and orientation of other vehicles on the road; this technology largely determines the safety factor of autonomous driving. Therefore, to improve the performance of target identification or observation, there is an urgent need to research more superior state estimation methods.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] According to one aspect of this disclosure, a target state estimation method is provided, comprising: acquiring a data frame sequence corresponding to multiple time points; determining a first time window in the data frame sequence, the first time window including a set of first state variables to be optimized; determining a plurality of second time windows from within the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have overlapping data frames, and each second time window has a set of second state variables to be optimized; and obtaining an optimized set of first state variables by simultaneously optimizing the sets of second state variables within the plurality of second time windows.

[0005] According to another aspect of this disclosure, a target state estimation apparatus is provided, comprising: an acquisition unit configured to acquire a sequence of data frames corresponding to multiple time points; a determination unit configured to determine a first time window in the data sequence, the first time window including a set of first state variables to be optimized; a selection unit configured to determine a plurality of second time windows from within the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have overlapping data frames, and each second time window has a set of second state variables to be optimized; and an optimization unit configured to obtain an optimized set of first state variables by simultaneously optimizing the sets of second state variables within the plurality of second time windows.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods described in this disclosure.

[0008] According to one or more embodiments of this disclosure, when optimizing the state variables within a first time window, the global smoothness of the state variables within the first time window is maintained by setting multiple second time windows and simultaneously optimizing the state variables within the second time windows, thereby improving the accuracy of target state estimation.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0011] Figure 1 This is a flowchart illustrating a target state estimation method according to an exemplary embodiment;

[0012] Figure 2 This is a schematic diagram illustrating a time window for target state estimation according to an exemplary embodiment;

[0013] Figure 3 This is a schematic diagram illustrating a truck motion model according to an exemplary embodiment;

[0014] Figure 4 This is a schematic diagram illustrating a motion model of a vehicle comprising only the first component according to an exemplary embodiment;

[0015] Figure 5 This is a structural block diagram illustrating a target state estimation apparatus according to an exemplary embodiment; and

[0016] Figure 6 This is a structural block diagram illustrating an exemplary computing device that can be applied to exemplary embodiments. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0019] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0020] A crucial aspect of autonomous driving is the real-time estimation of the position, speed, size, and orientation of other vehicles on the road, which largely determines the safety factor. While observation models can be used to monitor vehicle speed and position, the data obtained from these models is often affected by noise, leading to discrepancies between the observed data and the actual driving data. Therefore, it is necessary to correct this observed data by estimating the physical state of the vehicle during its movement based on the observed data.

[0021] Typically, state estimation involves estimating a certain state based on corresponding observation data. For example, it might estimate the vehicle's speed state based on observed velocity information, or estimate its position state based on observed coordinates of the vehicle's center point. When estimating the vehicle's physical state, a time window (e.g., a sliding time window) can be constructed to optimize a set of state variables within that window. It can be seen that the optimization effect of this set of state variables within the time window becomes crucial to the safety factor of autonomous driving.

[0022] Therefore, embodiments of this disclosure provide a target state estimation method, comprising: acquiring a data frame sequence corresponding to multiple time points; determining a first time window in the data frame sequence, the first time window including a set of first state variables to be optimized; determining multiple second time windows from within the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have overlapping data frames, and each second time window has a set of second state variables to be optimized; and obtaining an optimized first state variable set by simultaneously optimizing the second state variable sets within the multiple second time windows.

[0023] According to embodiments of this disclosure, when optimizing the state variables within a first time window, by setting multiple second time windows and simultaneously optimizing the state variables within the second time windows, the global smoothness of the state variables within the first time window is maintained, thereby improving the accuracy and robustness of the target state estimation.

[0024] Figure 1 A flowchart of a target state estimation method according to an embodiment of the present disclosure is shown. Figure 1 As shown, in step 110, a data frame sequence corresponding to multiple time points is obtained.

[0025] In embodiments of this disclosure, the target may include a vehicle. Therefore, the data frame sequence may correspond to observations of the target vehicle at multiple moments. For example, the observations may include at least one of the target vehicle's speed, position, and orientation at each moment; and the dimensions of the target vehicle, which may include at least one of length, width, and height.

[0026] In some embodiments, the observations corresponding to the data frame sequence can be obtained based on multiple sensors. For example, the multiple sensors may include at least one of the following: an image acquisition device, a point cloud acquisition device. For example, the image acquisition device may include a variety of devices, such as a visual camera, an infrared camera, a camera for ultraviolet or X-ray imaging, etc. Different devices can provide different detection accuracies and ranges. A visual camera can capture information such as the target's running status in real time. An infrared camera can capture targets in night vision conditions. An ultraviolet or X-ray imaging camera can image targets in various complex environments (nighttime, harsh weather, electromagnetic interference, etc.). The point cloud acquisition device may also include a variety of devices, such as lidar (LiDAR), millimeter-wave radar, ultrasonic sensors, etc. Different devices can provide different detection accuracies and ranges. LiDAR can be used to detect target edges and shape information, thereby enabling target identification and tracking. Millimeter-wave radar can be used to measure the distance to a target using the characteristics of electromagnetic waves. Ultrasonic sensors can be used to measure the distance to a target using the strong directionality of ultrasonic waves. Due to the Doppler effect, radar devices can also measure changes in the speed of moving targets.

[0027] According to some embodiments, multiple sensors can be located on at least one observation vehicle or roadside equipment. For example, while an autonomous vehicle is in motion, multiple sensors can be mounted in front of, behind, or other locations on the vehicle to enable real-time observation of surrounding vehicles. Alternatively, multiple sensors can be located on a roadside device to perform real-time observation of targets such as vehicles and pedestrians passing by the roadside device.

[0028] In some examples, the roadside equipment may include electronic devices and communication devices. The electronic devices and communication devices can be integrated or separate units. The electronic devices can acquire data observed by multiple sensors, perform data processing and calculations to obtain corresponding observations, and then transmit the processing and calculation results to the computing device via the communication device. Optionally, the electronic devices can also be located in the cloud to acquire data observed by multiple sensors on the roadside equipment via the communication device, and obtain corresponding observations through data analysis and calculations.

[0029] According to some embodiments, the target state estimation method of this disclosure can be implemented in a computing device that acquires at least one observation through each sensor. That is, the observations of the target at various times acquired by multiple sensors can be analyzed online or offline by the computing device. The computing device can reside on at least one observation vehicle, on a roadside device, or in the cloud, without limitation.

[0030] According to some embodiments, the observations can be obtained using observation models corresponding to each sensor. For example, the observation model includes at least one of the following: an image-based binocular ranging algorithm, an image-based monocular ranging algorithm, a point cloud-based ranging algorithm, an image and map-based projective ranging algorithm, and a point cloud and map-based projective ranging algorithm.

[0031] In this disclosure, the observation model can analyze and calculate based on the data acquired by the sensors to output the observations at each time point corresponding to the target. Specifically, in some examples, based on the projection ranging algorithm, the coordinates of the center point of the surrounding vehicles and the coordinates of the four corner points of the detection box can be obtained; based on ranging algorithms such as binocular ranging algorithms and monocular ranging algorithms, the coordinates of the center point of the surrounding vehicles and their speed can be obtained.

[0032] In some embodiments, after acquiring observations of the target at various times using multiple sensors, the acquired observations can be preprocessed. For example, abnormal observations can be deleted, usable observations can be retained, and data formats can be standardized, etc., without limitation.

[0033] In step 120, a first time window is determined in the data frame sequence. The first time window includes a set of first state variables to be optimized.

[0034] During driving, a vehicle can use multiple sensors to observe surrounding vehicles in real time, continuously generating observation data, i.e., a sequence of data frames. Based on this observation data, the state variables to be optimized can be determined. In some embodiments, the optimization of the vehicle's physical state can be achieved by constructing a time window.

[0035] Specifically, observations of the target vehicle observed by at least one observation model within a first time window are acquired, and state variables describing the physical state of the target vehicle within that first time window are constructed based on these observations. In some examples, the first time window can be a sliding time window, and the length and sliding step size of the first time window can be arbitrarily set. Of course, the first time window can also be non-sliding; no restrictions are imposed here.

[0036] According to some embodiments, the first set of state variables corresponding to the first time window includes at least one of the following: the target's velocity, position, orientation, and size at each moment within the first time window; the target's velocity, position, orientation, and other state variables at each moment are instantaneous state variables. Additionally, the first set of state variables may also include at least one of the target's average velocity, average position, and average orientation within the first time window.

[0037] Figure 2A schematic diagram of a time window for target state estimation according to an embodiment of the present disclosure is shown. Figure 2 As shown, the first time window includes n states to be optimized, forming the first set of state variables, namely S0, S1, ..., S2. n-1 The first set of state variables may also include at least one of the target's average velocity, average position, and average orientation within the first time window, without limitation.

[0038] For example, the state variables to be optimized in the first time window can be constructed according to formula (1).

[0039]

[0040] Wherein, the state quantity s of the i-th frame in the first time window i For example, it can include the state variables shown in formula (2).

[0041]

[0042] Among them, v i θ i o i These represent the target vehicle's speed magnitude, speed direction, and vehicle orientation, respectively. Additionally, regarding... and The relevant description of step 130 will be explained later.

[0043] According to some embodiments, the target is a vehicle, and the vehicle includes a first component and at least one second component, the second component being rotatable about the first component. Therefore, in some examples, the position of the target may include at least one of the following: the position of the first component, the positions of each of the second components, and the position of the vehicle; the size of the target may include at least one of the following: the size of the first component, the size of each of the second components, and the size of the vehicle. The orientation of the target may include at least one of the following: the orientation of speed, the orientation of the first component, and the orientation of the lane in which the vehicle is located.

[0044] For example, in the embodiment where the target is a vehicle, the target vehicle can be a truck, which includes two parts, namely, the first part of the truck is a tractor and the second part is a trailer, and the pivot (or hinge) structure connecting the tractor and the trailer forms a structural constraint between them.

[0045] In some examples, when the target vehicle is, for example, a truck, the speed magnitude and direction of the target vehicle can be the speed magnitude and direction of the tractor. Additionally, the vehicle orientation is the tractor orientation, and the state quantity s in the i-th frame... i It may also include trailers facing β i ,Right now It is understood that the state variables to be optimized within the time window shown in formulas (1) and (2) are merely exemplary and are not limited here.

[0046] Figure 3 A schematic diagram of a truck motion model according to an embodiment of the present disclosure is shown. (As follows) Figure 3 As shown, the tractor 301 and trailer 302 are connected by a pivot structure 303. In some embodiments, the tractor 301 can be processed based on a motion model of a vehicle containing only the first component, but the motion observation of the trailer imposes constraints on the motion observation of the tractor. The vehicle containing only the first component can be, for example, a unicycle, a conventional four-wheeled vehicle, etc.

[0047] Figure 4 A schematic diagram of a vehicle motion model including only a first component according to an embodiment of the present disclosure is shown. In some examples, the vehicle's velocity direction is distinguished from its heading direction to improve the accuracy of vehicle state estimation. In such cases... Figure 4 In the motion model shown, ο represents the vehicle's orientation (i.e., the direction of its front), and θ represents the direction of the vehicle's velocity. Assume the vehicle is at time t... i Time to t i+1 The velocity v between moments i Therefore, it has the transformation formulas shown in formulas (3) and (4).

[0048] px i+1 =px i +v i ·cosθ i Formula (3)

[0049] py i+1 =py i +v i ·sinθ i Formula (4)

[0050] Among them, px i and py i They represent t respectively i The coordinates of the vehicle's center point at that moment; px i+1 and py i+1 They represent t respectively i+1 The coordinates of the vehicle's center point at time θ; i Indicates t i The angle between the vehicle's velocity direction at a given moment and the x-direction in the reference coordinate system.

[0051] In this disclosure, the reference coordinate system is a coordinate system determined based on the observation vehicle or road testing equipment where multiple sensors are located. For example, when multiple sensors are located on the observation vehicle, the reference coordinate system is used to describe the relationship between the vehicle and objects around it. Depending on the definition, its origin may differ; for instance, the center of gravity can be used as the origin, and the right-handed coordinate system derived from it becomes the reference coordinate system; or, if the reference coordinate system is defined using an IMU (Inertial Measurement Unit), the IMU position can be used as the origin.

[0052] Understandably, any suitable reference coordinate system is possible. For example, the reference coordinate system could also use the lane centerline as the horizontal coordinate axis, the axis deviating from the lane centerline as the vertical coordinate axis, and the axis perpendicular to the lane centerline as the vertical coordinate axis. No restrictions are imposed here.

[0053] As mentioned above, based on the projection ranging algorithm, the coordinates of the four corner points of the vehicle detection box can be obtained, thus achieving vehicle contour detection. Therefore, in situations such as... Figure 4 In the vehicle frame shown, the vector from the vehicle center to the i-th vehicle corner point can be expressed as shown in formula (5).

[0054]

[0055] Where L and W are the length and width of the vehicle, respectively; [δ i ,η i ] represents the offset of the i-th vehicle corner point relative to the vehicle center point in the reference coordinate system, which is a constant for each vehicle corner point; R bw Represents the rotation matrix from the reference coordinate system to the ENU (East-North-Up) coordinate system, where R bw It is represented by formula (6).

[0056]

[0057] Therefore, based on information such as the vehicle's speed, orientation, size, and center point position, it is sufficient to identify a vehicle.

[0058] Continue to refer to Figure 3 In some embodiments, the trailer 302 and the pivot structure 303 typically have the same orientation and can therefore be treated as rigid structures. Additionally, it can be assumed that the pivot structure 303 links to the center position of the contact surface between the tractor 301 and the trailer 302. Once the center point coordinates p0, length L0, and width W0 of the tractor 301 are known, the center point coordinates p1 of the trailer 302 can be obtained, as shown in formulas (7)-(9).

[0059] p1=p0+offset0-offset1 Formula (7)

[0060]

[0061]

[0062] Where L1 is the length of the trailer, L h Let θ be the length of the rotating shaft structure, and o and β be the angles between the tractor and trailer relative to the x-axis of the reference coordinate system, respectively.

[0063] In some examples, the detection frames of both the tractor and trailer can be obtained simultaneously using sensors such as LiDAR. The position of the trailer's detection frame is set from h1 (when i is 1). Figure 3 Chinese h i Move the position to h0 ( Figure 3 The position of h0 is assumed to be another observation of the tractor, which makes the observation of the trailer constrain the observation of the tractor, as shown in formula (10).

[0064]

[0065] The angular velocity of the trailer can be expressed as shown in formula (11):

[0066]

[0067] Where v represents the speed of the tractor. This represents the angular velocity of the trailer. Given the speed, orientation, dimensions, axle length, and position of the tractor unit, various states of a truck can be determined.

[0068] As described above, the target vehicle has been described as having a two-level structure, namely, the target vehicle includes a first component and a second component. In some embodiments, the second component may also be multiple, such as a train, a multi-trailer truck, etc., and its motion model can be referred to the truck model described above, which will not be repeated here.

[0069] exist Figure 1 In step 130, multiple second time windows are determined from the first time window. The number of data frames in the second time window is less than the number of data frames in the first time window. Adjacent second time windows have duplicate data frames, and each second time window has a set of second state variables to be optimized.

[0070] Continue to refer to Figure 2 The first time window includes multiple second time windows, namely second time window 1, second time window 2, ..., second time window (n-m+1). Each second time window has a set of second state variables to be optimized. Figure 2Each second time window includes m state variables, and two adjacent second time windows have (m-1) repeated data frames. It is understood that the number of repeated data frames between adjacent second time windows and the number of state variables in the second state variable set are merely exemplary and are not limited here.

[0071] According to some embodiments, the second set of state variables includes at least one of the following: the target's velocity, position, orientation, and size at each moment within the second time window; and the target's average velocity, average position, and average orientation within the second time window.

[0072] Please refer to the aforementioned formula (1). This represents the average velocity within the first and second time windows. This represents the average velocity within the (nm)th second time window. This represents the average velocity within the (n-m+1)th second time window. This represents the average orientation within the first and second time windows. This represents the average orientation within the (nm)th second time window. This represents the average orientation within the (n-m+1)th second time window.

[0073] exist Figure 1 In step 140, the optimized first state variable set is obtained by simultaneously optimizing the second state variable set within the multiple second time windows.

[0074] According to some embodiments, the optimization is achieved by minimizing a loss function. The loss function is determined based on the target's state variables and observations at each time point within the plurality of second time windows; the observations are obtained through at least one observation model, which is based on at least one sensor.

[0075] Specifically, in some embodiments, the loss function may include at least one of the target's position loss, orientation loss, velocity loss, size loss, and structural constraints. By minimizing the loss function, the state variables of the target at each time step are optimized. For example, when the loss function includes position loss, orientation loss, velocity loss, and size loss, the loss function can be constructed based on formula (12).

[0076] E = E p +E v +E o +E s Formula (12)

[0077] Among them, E p Ev E o and E s These represent position loss, orientation loss, velocity loss, and size loss, respectively. The loss function is determined based on the state variable to be optimized. Specifically, each of the position loss, orientation loss, velocity loss, and size loss can be determined based on the state variable to be optimized, the observation corresponding to that state variable, and other observations that can provide constraints on that state variable.

[0078] In the exemplary scenario according to this disclosure, the speed observation of the target vehicle, the position of the target vehicle, etc., can provide constraints on the speed magnitude and speed direction of the target vehicle; in addition, the speed prior of the target vehicle, the average speed, etc., can also provide constraints on the speed magnitude and speed direction of the target vehicle; lane line direction, speed direction, the target vehicle orientation observed by the lidar sensor, the target vehicle orientation prior, the average orientation, etc., can provide constraints on the vehicle body orientation of the target vehicle; and so on. This will be described in detail below.

[0079] In this disclosure, observations of the target at various times are acquired through multiple sensors, and corresponding loss functions are constructed, realizing the transition from single-sensor recognition to multi-sensor fusion. Therefore, during vehicle operation, the perception results from multiple sensors can be combined to model surrounding vehicles and update their status information in real time. This allows the autonomous driving system to make safe path planning based on these results, thereby avoiding traffic accidents.

[0080] In some embodiments, the loss function includes a smoothing loss of the state quantities determined based on a plurality of second time windows; the smoothing loss is calculated based on the state quantities of the target at each time point in each second time window and the average value of the state quantities of the target in each second time window.

[0081] By using a smoothing loss based on state variables determined by multiple second time windows, the data from the current time and previous and subsequent time points are fully utilized when estimating the state at the current time. This makes the state variable obtained after optimizing the first time window smoother and improves the accuracy of the target state estimation.

[0082] Specifically, according to some embodiments, the loss function includes a velocity loss associated with the velocity of the target. The velocity loss includes a velocity smoothing loss, which is calculated based on the velocity state of the target at each time point within each second time window, the velocity state of the target at each time point within the corresponding second time window, and the average velocity state of the target within the corresponding second time window.

[0083] In some embodiments, when the target is a vehicle, the state quantity includes the speed of the target vehicle at each time point within the second time window. To ensure speed smoothing within the second time window, the speed at each time point within the second time window can be limited to an average value using the speed smoothing loss shown in formula (13).

[0084]

[0085] Among them, w a R represents the weight value corresponding to this speed smoothing loss. bw As described in the above reference formula (6), This represents the average speed within the current second time window.

[0086] It can be noted that a velocity smoothing constraint as shown in Equation (13) can be applied to each second time window.

[0087] According to some embodiments, the velocity smoothing loss is further calculated based on the velocity state quantity of the target at each moment within the first time window and the average velocity state quantity of the target within the first time window.

[0088] In some embodiments, the state quantity includes the speed of the target vehicle at each time point within the first time window. To ensure speed smoothness within the first time window, the speed at each time point within the first time window can be limited to an average value using the speed smoothing loss shown in formula (14).

[0089]

[0090] Among them, w b Here, the weight value corresponding to the speed smoothing loss is... This represents the average velocity within the first time window.

[0091] In some embodiments, the weight value w corresponding to the velocity smoothing loss a and w b The weight value can be determined based on the distance between the target vehicle and the vehicle or roadside equipment where the multiple sensors are located. For example, when the distance is greater than a preset threshold, the weight value is positively correlated with the distance; when the distance is not greater than the preset threshold, the weight value is a fixed value.

[0092] In some embodiments, the weight value w corresponding to the velocity smoothing loss a and w b The speed can be further determined based on the rate of change of the target vehicle's speed, which is calculated according to the speed of the target vehicle at each moment within the sliding time window. Specifically, the weight value for a speed rate of change greater than another preset threshold is less than the weight value for a speed rate of change not greater than that other preset threshold.

[0093] According to some embodiments, the velocity loss further includes a velocity prior loss, which is calculated based on the following: the velocity state quantity at each moment in the overlapping interval between the current first time window and the previous first time window, and the optimized velocity state quantity at each moment in the overlapping interval during the state quantity optimization process for the previous first time window.

[0094] In some embodiments, the first time window is a sliding time window, and its sliding step size is less than the length of the first time window. Then, the velocity loss can be determined based on the velocity prior loss. Specifically, in order to retain the optimization information obtained in the previous optimization at each current time, the velocity prior loss term shown in formula (15) can be used to limit the velocity at each time within the first time window to be close to the velocity after the previous optimization at that time.

[0095]

[0096] in, Let k be the velocity after the last optimization at the current time. Here, k ranges from 0 to n-2, indicating that the sliding step size of the first time window is 1. For v0, v1, ..., v... n-2 Its optimal solution was obtained in the previous optimization (the previous first time window); w p The weight value corresponding to the velocity prior loss.

[0097] In some embodiments, the weight value corresponding to the speed prior loss can be determined based on the distance between the target vehicle and the vehicles or roadside devices where multiple sensors are located. When the distance is greater than a preset threshold, the weight value is positively correlated with the distance; when the distance is not greater than the preset threshold, the weight value is a fixed value.

[0098] According to some embodiments, the velocity loss further includes a velocity residual, which is calculated based on the velocity observations of the target at each moment within a first time window and the velocity state quantities of the target at each moment within the first time window.

[0099] When an observation model provides velocity observations, such as a radar model, the velocity residual loss can be flexibly added to the velocity loss. Assume the observations of the l-th observation model are: The velocity loss term e in formula (16) ov It needs to be added to the velocity loss formula, where This represents the number of models that can provide velocity observations.

[0100]

[0101] In some embodiments, such as radar models, the velocity vectors observed are unreliable; however, the velocity norm can be used. If only the velocity norm is available at this time, then the velocity loss term e ov It can be shown in formula (17).

[0102]

[0103] In summary, the complete velocity loss term can be expressed as shown in formula (18).

[0104]

[0105] Among them, the smoothing constraint term determined based on the second time window It can be multiple, which is equal to the number of second time windows.

[0106] According to some embodiments, the loss function includes an orientation loss associated with the orientation of the target. The orientation loss includes an orientation smoothing loss, which is calculated based on each time step of each second time window, the orientation state of the target at each time step within the corresponding second time window, and the average orientation state of the target within the corresponding second time window.

[0107] Just like velocity loss, orientation loss also has a similar smoothing loss. In some embodiments, the state quantity includes the orientation of the target vehicle at each time point within the second time window. To ensure orientation smoothing within the second time window, the orientation at each time point within the second time window can be constrained to an average value using the orientation smoothing loss shown in Equation (19).

[0108]

[0109] Among them, w c This is the weight value corresponding to the smoothing loss in that direction. This represents the average orientation within the current second time window.

[0110] It is worth noting that the velocity smoothing constraint shown in Equation (19) can be applied to any second time window.

[0111] According to some embodiments, the orientation smoothing loss is further calculated based on the orientation state of the target at each time point within the first time window and the average orientation state of the target within the first time window.

[0112] In some embodiments, the state quantity includes the orientation of the target vehicle at each time point within a first time window. To ensure orientation smoothness within the first time window, the orientation at each time point within the first time window can be constrained to an average value using the orientation smoothing loss shown in formula (20).

[0113]

[0114] Among them, w d Here, the weight values ​​corresponding to the smoothing loss are... The average orientation within the first time window.

[0115] According to some embodiments, the orientation loss further includes an orientation prior loss, which is calculated based on the following: the orientation state quantity at each time point within the overlapping interval of the first time window and the previous first time window, and the optimized orientation state quantity at each time point within the overlapping interval during the state quantity optimization process for the previous first time window.

[0116] In some embodiments, the first time window is a sliding time window, and its sliding step size is less than the length of the first time window. Then, the orientation loss can be determined based on the orientation prior loss. Specifically, in order to retain the optimization information obtained in the previous optimization at each current time, the orientation prior loss term shown in formula (21) can be used to restrict the orientation at each time in the first time window to be close to the orientation optimized at the previous time.

[0117]

[0118] Among them, w p The weight values ​​corresponding to the prior loss are as follows. This represents the orientation optimized in the previous iteration at the current moment (assuming the sliding step size of the first time window is 1).

[0119] In some embodiments, the state quantity includes the orientation of the target at each time step within a first time window. In this case, the orientation loss may include orientation residuals, which are calculated based on the orientation state quantity of the target at each time step within the first time window and the orientation observations of the target at each time step within that first time window.

[0120] Specifically, the orientation observations can directly constitute the orientation constraints, so the orientation loss term can be as shown in Equation (22).

[0121]

[0122] in, It is a set of different observation sources. It is the weight corresponding to the l-th observation source, and its calculation method can be shown in the formula (32) of the reference position loss.

[0123] In some embodiments, the orientation observation can be the vehicle body orientation, lane line orientation, or speed direction of the target vehicle observed by at least one observation model. In some embodiments, to optimize orientation, when no reliable orientation observation is provided, the vehicle should travel along the lane, in which case the lane line heading can be regarded as an orientation observation with a fixed variance; furthermore, the speed direction can also be regarded as an orientation observation, and the higher the speed, the smaller the difference between the speed direction and the vehicle orientation.

[0124] In some examples, for velocity-based orientation observations, the weight λ k It can be calculated according to formula (23).

[0125]

[0126] Among them, w v 'a' and 'a' are hyperparameters.

[0127] According to some embodiments, the loss function includes an orientation loss associated with the orientation of the target; the orientation loss includes an orientation smoothing loss, which is calculated based on each moment of each second time window, the orientation state quantity of the target at each moment within the corresponding second time window, and the average orientation state quantity of the target within the corresponding second time window. The orientation loss further includes an orientation residual, which includes a first component orientation residual and / or a second component orientation residual; the first component orientation residual is calculated based on the orientation state quantity of the first component at each moment within the first time window and the orientation observation quantity of the first component at each moment within the first time window; the second component orientation residual is calculated based on the orientation state quantity of the second component at each moment within the first time window and the orientation observation quantity of the second component at each moment within the first time window.

[0128] In some embodiments, the target vehicle is, for example, a truck, a vehicle comprising a first component and a second component, such as... Figure 3 As shown. The first and second components can form a structural constraint between them via a pivot structure (or hinge). The state quantities include the orientation of the first component at each moment within the sliding time window and the orientation of the second component at each moment within the sliding time window.

[0129] Therefore, in some embodiments, the orientation loss can be based on the first component orientation residual and the second component orientation residual, wherein the first component orientation residual is calculated based on the orientation of the first component at each moment within the sliding time window and the observed orientation value of the first component at each moment within the sliding time window, and the second component orientation residual is calculated based on the orientation of the second component at each moment within the sliding time window and the observed orientation value of the second component at each moment within the sliding time window. The first component orientation residual and the second component orientation residual can be referred to the above description and will not be repeated here.

[0130] In some embodiments, the orientation observation value of the first component is the orientation of the first component, the lane line orientation, or the velocity direction of the first component as observed by at least one observation model, and the orientation observation value of the second component is the orientation of the second component, the lane line orientation, or the velocity direction of the second component as observed by the at least one observation model.

[0131] In some embodiments, when the target vehicle is a vehicle including a first component and a second component, the state quantity includes the average orientation of the first component within a sliding time window. Therefore, the orientation loss may include the orientation smoothing loss of the first component, which is calculated based on the orientation state quantity of the first component at each time in each second time window, the orientation state quantity of the first component at each time in the corresponding second time window, and the average orientation state quantity of the first component within the corresponding second time window.

[0132] In some embodiments, when the target vehicle is a vehicle including a first component and a second component, the state quantity includes the average orientation of the first component within the sliding time window. Therefore, the orientation smoothing loss of the first component can also be calculated based on the orientation state quantity of the first component at each time point within the first time window and the average orientation state quantity of the first component within the first time window.

[0133] It is understandable that the orientation loss may also include the orientation smoothing loss of the second component, which will not be elaborated here.

[0134] In some embodiments, the sliding step size of the first time window is smaller than the length of the first time window. Therefore, when the target vehicle is a vehicle including a first component and a second component, the orientation loss may include the orientation prior loss of the first component, which is calculated based on the following: the orientation of the first component at each moment in the overlapping area of ​​the first time window and the previous first time window, and the optimized orientation of the first component at each moment in the overlapping area during the state quantity optimization process for the previous first time window.

[0135] According to some embodiments, the loss function includes an orientation loss associated with the orientation of the target; the orientation loss includes an orientation smoothing loss, which is calculated based on each moment of each second time window, the orientation state of the target at each moment within the corresponding second time window, and the average orientation state of the target within the corresponding second time window. The orientation loss further includes angular velocity constraints, which are calculated based on the following: the speed state of the truck at each moment within the first time window, the length of the second component within the first time window, the length of the hinge between the first and second components within the first time window, the orientation state of the first component at each moment within the first time window, and the orientation state of the second component at each moment within the first time window.

[0136] Specifically, the orientation observation of the second component is also subject to a motion constraint as shown in formula (11). The angular velocity loss can then be expressed as shown in formula (24).

[0137]

[0138] Among them, L t and L h The lengths of the first component and the shaft structure are respectively, and their calculation method will be described below in the reference dimensional loss section.

[0139] In summary, the complete orientation loss term can be expressed, for example, as shown in formula (25).

[0140]

[0141] Among them, the smoothing constraint term determined based on the second time window It can be multiple, which is equal to the number of second time windows.

[0142] According to some embodiments, the loss function further includes a position loss associated with the location of the target. The position loss includes at least one reference point residual; the reference point residual includes at least one of the following: a center point residual and a contour corner point residual.

[0143] According to some embodiments, the position includes the position of at least one reference point, which includes at least one of the following: a center point and contour corner points (e.g., the four corner points of a vehicle detection frame). The position loss includes at least one reference point residual, which includes at least one of the following: a center point residual and a contour corner point residual, wherein the center point residual represents the difference between the observations and state quantities for the center point, and the contour corner point residual represents the difference between the observations and state quantities for the contour corner points.

[0144] Specifically, assume that the state variables of the target vehicle are optimized based on the observation data obtained from L observation models, where L is a positive integer. If the center point observation of the l-th observation model is... The center point residual can then be constructed based on the difference between the observations and state variables at the center point. If the l-th observation model also provides profile observations, the profile observations are: The residuals of the contour corner points can then be constructed based on the differences between the observed quantities and state quantities of the contour corner points.

[0145] In some embodiments, the state variables of the center point can be characterized based on velocity to further optimize the velocity state variables through the center point residual. Specifically, when the observed quantities include the center point coordinates of the target vehicle at each moment within the sliding time window, and the state variables include the velocity of the target vehicle at each moment within the sliding time window, the center point residual can be calculated based on the center point coordinates of the target vehicle at each moment within the sliding time window and the velocity of the target vehicle at each moment within the sliding time window.

[0146] Specifically, assume that the state variables of the target vehicle are optimized based on the observation data obtained from L observation models, where L is a positive integer. If the center point observation of the l-th observation model is... The first frame position coordinates corresponding to the target vehicle are determined to be p0. Represents the l-th observation model at t k The residual vector at the center point at time t is shown in formula (26):

[0147]

[0148] in,

[0149]

[0150] v i =[v i cos(θ i ),v i sin(θ i )] T Formula (28)

[0151] In some embodiments, the state variables of the contour corner points can be characterized based on the state variables of the center point, so as to further optimize the center point state variables through the contour corner point residuals. Specifically, when the observed quantities include the contour corner point coordinates of the target vehicle at each moment within the sliding time window, the reference point residuals can be calculated based on the following: the center point coordinates of the target vehicle at the initial moment within the sliding time window, the speed of the target vehicle at each moment within the sliding time window, the contour corner point coordinates of the target vehicle at each moment within the sliding time window, and the corresponding vector from the center point coordinates to the contour corner point coordinates of the target vehicle at each moment within the sliding time window.

[0152] Specifically, if the l-th observation model also provides contour observations, the contour observations are... Then the residual of the contour corner point can be obtained, as shown in formula (29).

[0153]

[0154] in,

[0155]

[0156] Where, φ m This represents the vector from the vehicle's center point to the corner point of the vehicle's outline.

[0157] As mentioned above, in reference Figure 3 In the truck motion model described above, the trailer imposes constraints on the profile observation of the tractor. Therefore, when optimizing the corresponding state variables of the tractor (such as the velocity state variables mentioned above), constraints on the profile observation of the tractor by the trailer can be further introduced based on the reference point residuals described above.

[0158] According to some embodiments, the center point residual and the contour corner residual each have corresponding weights, and the weights are all diagonal matrices; each of the center point residual and the contour corner residual includes a horizontal residual component and a vertical residual component, and the horizontal residual component and the vertical residual component each have corresponding weights.

[0159] In the examples according to this disclosure, when the target is a vehicle, the lateral direction can be a horizontal direction perpendicular to the approximate orientation of the target vehicle; the longitudinal direction can be a horizontal direction parallel to the approximate orientation of the target vehicle. Specifically, the "approximate orientation" may include, for example, the observed vehicle body orientation of the target vehicle, the lane orientation of the lane in which the target vehicle is located (i.e., lane heading), and so on.

[0160] Therefore, according to some embodiments, when the target is a vehicle, the lateral residual component is perpendicular to the lane orientation of the vehicle, and the longitudinal residual component is parallel to the lane heading of the vehicle; or the lateral residual component is perpendicular to the vehicle body orientation, and the longitudinal residual component is parallel to the vehicle body orientation.

[0161] In this disclosure, the estimation of state variables focuses on the lateral and longitudinal directions, and for ease of model tuning, the lateral and longitudinal directions can be decoupled. In some examples, such as when the vehicle orientation or lane orientation observed by radar sensors is known, the residuals in the ENU coordinate system can be expressed through R... bw When the matrix is ​​rotated to the reference coordinate system, the position loss function, which includes the center point residual and the contour corner point residual, can be expressed as Equation (31).

[0162]

[0163] Where ρ(·) is the robust function; The weight matrix (diagonal matrix) assigns different weights to the horizontal and vertical residuals respectively; R bw As described in the above reference formula (6).

[0164] In this disclosure, the robust function ρ(·) can be a robust function based on any suitable loss function, including but not limited to Cauchy (Lorentzian), Charbonnier (pseudo-Huber, L1-L2), Huber, Geman-McClure, smooth truncated quadratic, truncated quadratic, Tukey's biweight, and so on. For example, a convex loss function such as Huber can be chosen to preserve the convex optimization problem. However, convex loss functions may have limited robustness to outliers. Therefore, in some examples, a non-convex loss function can be chosen.

[0165] According to some embodiments, when the lateral variance of one of the center point residual and the contour corner residual is less than a predetermined threshold, the weight of the corresponding lateral residual component is taken as a first fixed value; when the longitudinal variance of one of the center point residual and the contour corner residual is less than a predetermined threshold, the weight of the corresponding longitudinal residual component is taken as a first fixed value.

[0166] In some examples, taking the center point residual as an example, if at least one of the horizontal and vertical center point variance components of the center point variance is less than a corresponding first threshold, the corresponding weights of the horizontal and vertical center point residual components are first fixed values. Furthermore, when at least one of the horizontal and vertical center point variance components is not less than the corresponding first threshold, the weight of at least one of the horizontal and vertical center point residual components is negatively correlated with said at least one of the horizontal and vertical center point variance components.

[0167] In some examples, the contour corner residuals can be similar to the center point residuals mentioned above, that is, the weights corresponding to the contour corner residuals are determined based on the contour corner variance.

[0168] Specifically, the weight matrix is ​​negatively correlated with the variance. Given the horizontal and vertical variances, the weight matrix can be expressed as shown in formula (32):

[0169]

[0170] Among them, w long w lat a and b are hyperparameters. Due to the limitations of the observation model's accuracy, a small variance cannot accurately reflect the true error. Therefore, using formula (32), a fixed weight is used when the variance is below a threshold. In this disclosure, a weighting formula similar to formula (32) can be used for all observation loss terms.

[0171] According to some embodiments, the loss function further includes a size loss associated with the size of the target. The size loss term includes at least one of the following: a prior size loss, and a cumulative size loss optimized at each time step. The prior size loss includes the residual between the size of the target at each current time step and the size of the target optimized during state quantity optimization for the previous first time window; the cumulative size loss includes the sum of all size losses of the target from the initial time step to the previous optimization time step.

[0172] In some embodiments, the sliding step size of the first time window is smaller than the length of the first time window. Therefore, the size loss term may include a size prior loss, which is calculated based on the following: the size of the target at each time step within the overlapping region of the first time window and the previous first time window, and the optimized size of the target at each time step within the overlapping region during the state quantity optimization process for the previous first time window.

[0173] According to some embodiments, the cumulative size loss is calculated using an incremental update method; the observations of the target at each time point are the observations of the target at each time point within a first time window; the state variables of the target at each time point are the state variables of the target at each time point within a first time window; the first time window includes multiple data times, and each time point is at least two of the multiple data times.

[0174] Specifically, the cumulative size loss includes the sum of all size losses of the target from the initial time to the previous optimization time. The initial time is the moment when state quantity optimization first begins, such as the moment when the first frame of data is obtained. The previous optimization time can be, for example, the last moment within the previous first time window. For example, target vehicle contour observation can provide the size information of the target vehicle; therefore, the cumulative size loss can be calculated based on the following: the size loss determined based on the reference point residual for each moment that does not fall within the current first time window but falls within the previous first time window, and the cumulative size loss used in the state quantity optimization process for the previous first time window.

[0175] In some embodiments, the observations include the contour corner coordinates of the target vehicle at each moment within the first time window and the center point coordinates of the target vehicle at each moment within the first time window. The state quantities include the speed of the target vehicle at each moment within the first time window, and the reference point residual corresponding to each moment is calculated according to the following: the center point coordinate observation of the target vehicle at that moment, the contour corner coordinate observation of the target vehicle at that moment, and the corresponding vector from the center point coordinate observation of the target vehicle at that moment to the contour corner coordinate observation, which can be determined according to formula (30).

[0176] Specifically, in the optimization framework, the vehicle body size or the tractor size of a truck is considered a global variable to be optimized. After the current state is updated, the oldest frame is discarded and will not be updated again. Although the state variables outside the sliding window are fixed, they can still provide some information about the global size variable. Specifically, after the i-th frame is discarded, if the contour observation... If available, a new size loss can be generated, as shown in Equation (33).

[0177]

[0178] in, and It is a constant. It is the weight calculated based on the variance in formula (32).

[0179] Since the Laplace distribution can be equivalently represented as the product of the Gaussian distribution and the inverse Gaussian distribution, in some examples, the L2 term of the Huber loss function can be approximated by γ = diag(γ0, γ1), as shown in Equation (34), to have better robustness.

[0180]

[0181] Where δ represents the preset parameter, r i In formula (33)

[0182] The number of size loss terms increases over time. To avoid redundant calculations, in embodiments according to this disclosure, they are combined into one term in an incremental form. Therefore, the loss term at time Ti can be expressed as shown in formula (35).

[0183]

[0184] Among them, A i It can be calculated using the SVD decomposition method, as shown in formulas (36)-(38):

[0185]

[0186] A i =(UΛV T ) T Formula (38)

[0187] in, It is a symmetric matrix, therefore U = V. i It can be shown in formula (39).

[0188]

[0189] In some embodiments, in examples where the objective includes a first component and a second component, such as... Figure 3 The dimensions of the truck model shown, the trailer, and the axle structure connecting the trailer and the tractor can be obtained by observation calculation, as shown in the following formulas (40)-(42).

[0190]

[0191]

[0192]

[0193] Among them, formulas (40)-(42) are the solutions to the optimization problem, as shown in formula (43).

[0194]

[0195] In summary, given the prior losses of L and W, the total size loss term can be expressed as shown in formula (44).

[0196]

[0197] Among them, E s The first term is the cumulative size loss, and the second term is the prior size loss.

[0198] In this disclosure, based on a loss function including at least one of the target's position loss, orientation loss, velocity loss, size loss, and structural constraints, the state variables of the target at each time step can be optimized by minimizing this loss function, thereby obtaining the optimized state variables. In the field of autonomous driving, the method of this disclosure can more accurately update the state information of surrounding vehicles, enabling the autonomous driving system to make safe path planning based on this result, thereby avoiding traffic accidents.

[0199] According to embodiments of this disclosure, such as Figure 5 As shown, a target state estimation device 500 is also provided, comprising: an acquisition unit 510 configured to acquire a data frame sequence corresponding to multiple time points; a determination unit 520 configured to determine a first time window in the data sequence, the first time window including a set of first state variables to be optimized; a selection unit 530 configured to determine multiple second time windows from the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have duplicate data frames, and each second time window has a set of second state variables to be optimized; and an optimization unit 540 configured to obtain an optimized set of first state variables by simultaneously optimizing the sets of second state variables in the multiple second time windows.

[0200] Reference Figure 6 The computing device 2000 will now be described as an example of a hardware device that can be applied to various aspects of this disclosure. The computing device 2000 can be any machine configured to perform processing and / or computation, and can be, but is not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal digital assistant, smartphone, in-vehicle computer, or any combination thereof. The aforementioned target state estimation apparatus can be implemented wholly or at least partially by the computing device 2000 or similar devices or systems.

[0201] The computing device 2000 may include elements (possibly via one or more interfaces) connected to or communicating with the bus 2002. For example, the computing device 2000 may include the bus 2002, one or more processors 2004, one or more input devices 2006, and one or more output devices 2008. The one or more processors 2004 may be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., special-purpose processing chips). The input devices 2006 may be any type of device capable of inputting information to the computing device 2000 and may include, but are not limited to, a mouse, keyboard, touchscreen, microphone, and / or remote control. The output devices 2008 may be any type of device capable of presenting information and may include, but are not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. The computing device 2000 may also include or be connected to a non-transitory storage device 2010. The non-transitory storage device may be any storage device that is non-transitory and capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, optical discs or any other optical media, ROM (read-only memory), RAM (random access memory), cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 2010 may be detachable from an interface. The non-transitory storage device 2010 may have data / programs (including instructions) / code for implementing the methods and steps described above. The computing device 2000 may also include a communication device 2012. The communication device 2012 may be any type of device or system enabling communication with external devices and / or with a network, and may include, but is not limited to, modems, network interface cards, infrared communication devices, wireless communication devices and / or chipsets, such as Bluetooth. TM Devices, 1302.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0202] The computing device 2000 may also include working memory 2014, which may be any type of working memory that can store programs (including instructions) and / or data useful to the operation of the processor 2004, and may include, but is not limited to, random access memory and / or read-only memory devices.

[0203] Software elements (programs) may reside in the working memory 2014, including but not limited to the operating system 2016, one or more application programs 2018, drivers, and / or other data and code. Instructions for performing the above methods and steps may be included in one or more application programs 2018, and the various units of the target state estimation device may be implemented by the processor 2004 reading and executing the instructions of one or more application programs 2018. More specifically, the acquisition unit 510 of the aforementioned target state estimation device may be implemented, for example, by the processor 2004 executing an application program 2018 with instructions to perform step 110. The construction unit 520 of the aforementioned target state estimation device may be implemented, for example, by the processor 2004 executing an application program 2018 with instructions to perform step 120. Furthermore, the optimization unit 530 of the aforementioned target state estimation device may be implemented, for example, by the processor 2004 executing an application program 2018 with instructions to perform step 130. The executable code or source code of the instructions of the software element (program) may be stored in a non-transitory computer-readable storage medium (such as the storage device 2010 described above) and may be stored in the working memory 2014 during execution (possibly for compilation and / or installation). The executable code or source code of the instructions of the software element (program) may also be downloaded from a remote location.

[0204] It should also be understood that various modifications can be made depending on specific requirements. For example, custom hardware can also be used, and / or specific elements can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, some or all of the disclosed methods and apparatus can be implemented by programming hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using logic and algorithms according to this disclosure in assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++).

[0205] It should also be understood that the aforementioned methods can be implemented using a server-client model. For example, the client can receive user input data and send it to the server. Alternatively, the client can receive user input data, perform a portion of the processing described in the aforementioned methods, and send the resulting data to the server. The server can receive data from the client, execute the aforementioned methods or a portion thereof, and return the execution result to the client. The client can receive the execution result from the server and, for example, present it to the user via an output device.

[0206] It should also be understood that the components of computing device 2000 can be distributed across a network. For example, some processing can be performed using one processor, while other processing can be performed simultaneously by another processor located far away from that processor. Other components of computing system 2000 can also be distributed similarly. In this way, computing device 2000 can be interpreted as a distributed computing system that performs processing in multiple locations.

[0207] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A target state estimation method, comprising: Obtain the data frame sequence corresponding to multiple time points; A first time window is determined in the data frame sequence, the first time window including a set of first state variables to be optimized; Multiple second time windows are determined from the first time window. The number of data frames in the second time window is less than the number of data frames in the first time window. Adjacent second time windows have duplicate data frames, and each second time window has a set of second state variables to be optimized. By simultaneously optimizing the set of second state variables within the multiple second time windows, an optimized set of first state variables is obtained. The optimization is achieved by minimizing the loss function; The loss function is determined based on the target's state variables and the target's observations at each time point within the plurality of second time windows; The observations are obtained through at least one observation model, and the at least one observation model is based on at least one sensor. The loss function further includes a location loss associated with the location of the target; The position loss includes at least one reference point residual; The reference point residual includes at least one of the following: center point residual and profile corner point residual.

2. The method as described in claim 1, wherein, The first set of state variables includes at least one of the following: The target's velocity, position, orientation, and size at each moment within the first time window; The target's average velocity, average position, and average orientation within the first time window.

3. The method as described in claim 1, wherein, The second set of state variables includes at least one of the following: The target's velocity, position, orientation, and size at each moment within the second time window; The target's average velocity, average position, and average orientation within the second time window.

4. The method of claim 1, wherein, The loss function includes a smoothing loss of the state quantities determined based on the plurality of second time windows; The smoothing loss is calculated based on the state quantity of the target at each time point in each second time window and the average value of the state quantity of the target in each second time window.

5. The method of claim 1, wherein, The loss function includes a velocity loss associated with the velocity of the target; The velocity loss includes a velocity smoothing loss, which is calculated based on the velocity state of the target at each moment within each second time window, the velocity state of the target at each moment within the corresponding second time window, and the average velocity state of the target within the corresponding second time window.

6. The method of claim 5, wherein, The velocity smoothing loss is further calculated based on the velocity state quantity of the target at each moment within the first time window and the average velocity state quantity of the target within the first time window.

7. The method of claim 5, wherein, The velocity loss further includes a velocity prior loss, which is calculated based on the following: the velocity state quantity at each moment within the overlapping interval of the current first time window and the previous first time window, and the optimized velocity state quantity at each moment within the overlapping interval during the state quantity optimization process for the previous first time window.

8. The method of claim 5, wherein, The velocity loss further includes a velocity residual, which is calculated based on the velocity observations of the target at each moment within the first time window and the velocity state quantities of the target at each moment within the first time window.

9. The method of claim 1, wherein, The loss function includes an orientation loss associated with the orientation of the target; The orientation loss includes an orientation smoothing loss, which is calculated based on the orientation state of the target at each moment in each second time window, the orientation state of the target at each moment in the corresponding second time window, and the average orientation state of the target in the corresponding second time window.

10. The method of claim 9, wherein, The orientation smoothing loss is further calculated based on the orientation state of the target at each time point within the first time window and the average orientation state of the target within the first time window.

11. The method of claim 9, wherein, The orientation loss further includes orientation prior loss, which is calculated based on the following: the orientation state quantity at each moment within the overlapping interval of the first time window and the previous first time window, and the optimized orientation state quantity at each moment within the overlapping interval during the state quantity optimization process for the previous first time window.

12. The method as described in claim 2 or 3, wherein, The target is a vehicle, which includes a first component and at least one second component, the second component being rotatable about the first component; The location of the target includes at least one of the following: the location of the first component, the location of each of the second components, and the location of the vehicle; The dimensions of the target include at least one of the following: the dimensions of the first component, the dimensions of each of the second components, and the dimensions of the vehicle; The orientation of the target includes at least one of the following: the orientation of the speed, the orientation of the first component, and the orientation of the lane in which the vehicle is located.

13. The method of claim 12, wherein, The loss function includes an orientation loss associated with the orientation of the target; The orientation loss includes an orientation smoothing loss, which is calculated based on the orientation state of the target at each moment within each second time window, the orientation state of the target at each moment within the corresponding second time window, and the average orientation state of the target within the corresponding second time window. The orientation loss further includes an orientation residual, which includes a first component orientation residual and / or a second component orientation residual; The orientation residual of the first component is calculated based on the orientation state quantity of the first component at each moment within the first time window and the orientation observation quantity of the first component at each moment within the first time window; The orientation residual of the second component is calculated based on the orientation state quantity of the second component at each moment within the first time window and the orientation observation quantity of the second component at each moment within the first time window.

14. The method of claim 12, wherein, The loss function includes an orientation loss associated with the orientation of the target; The orientation loss includes an orientation smoothing loss, which is calculated based on the orientation state of the target at each moment within each second time window, the orientation state of the target at each moment within the corresponding second time window, and the average orientation state of the target within the corresponding second time window. The orientation loss further includes angular velocity constraints, which are calculated based on the following: the speed state of the truck at each moment within the first time window, the length of the second component within the first time window, the length of the hinge between the first component and the second component within the first time window, the orientation state of the first component at each moment within the first time window, and the orientation state of the second component at each moment within the first time window.

15. The method of claim 1, wherein, The loss function further includes a size loss associated with the size of the target; The size loss term includes at least one of the following: size prior loss, and the optimized cumulative size loss at each time step; The size prior loss includes the residual between the size of the target at each current time and the size of the target that has been optimized during the state quantity optimization process for the previous first time window; The cumulative size loss includes the sum of all size losses of the target from the initial time to the previous optimization time.

16. A target state estimation device, comprising: The acquisition unit is configured to acquire data frame sequences corresponding to multiple time points; The determining unit is configured to: determine a first time window in the data frame sequence, wherein the first time window includes a set of first state variables to be optimized; The selection unit is configured to: determine multiple second time windows from the first time window, wherein the number of data frames in the second time window is less than the number of data frames in the first time window, adjacent second time windows have duplicate data frames, and each second time window has a set of second state variables to be optimized; The optimization unit is configured to: simultaneously optimize the set of second state variables within the plurality of second time windows to obtain an optimized set of first state variables; The optimization is achieved by minimizing the loss function; The loss function is determined based on the target's state variables and the target's observations at each time point within the plurality of second time windows; The observations are obtained through at least one observation model, and the at least one observation model is based on at least one sensor. The loss function further includes a location loss associated with the location of the target; The position loss includes at least one reference point residual; The reference point residual includes at least one of the following: center point residual and profile corner point residual.

17. An electronic device comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-15.

Images