Vehicle state determination method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a vehicle state determination method and device, electronic equipment and a storage medium, wherein the method comprises: determining current detection box information of a target vehicle based on a current frame image; determining a current observation size, a current observation longitudinal distance and a current observation lateral distance of the target vehicle based on the current detection box information; determining a current state quantity of the target vehicle based on a previous state quantity of the target vehicle, the current observation size, the current observation longitudinal distance and the current observation lateral distance, the previous state quantity comprising an optimized state quantity corresponding to a first number of historical frames obtained in advance; and determining a current state of the target vehicle based on the current state quantity. The embodiments of the present disclosure realize simultaneous determination of the position, speed, acceleration and other states of the target vehicle, and can effectively avoid dependence of acceleration on speed and dependence of speed on position relative to the existing filtering method, thereby greatly improving the accuracy and stability of the obtained target vehicle state.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to computer vision technology, and in particular, to a vehicle state determination method and device, electronic equipment and storage medium. BACKGROUND

[0002] In an autonomous driving scenario, it is very important for vehicle driving safety to determine the state (such as position, speed, acceleration, etc.) of a surrounding dynamic vehicle in real time and stably. In related technologies, the state of a surrounding dynamic vehicle is usually determined based on a monocular camera filtering method, such as constructing an EKF (Extended Kalman Filter) filter for position, speed and acceleration respectively from top to bottom (i.e., the output of the position filter is taken as the input of the speed filter, and the output of the speed filter is taken as the input of the acceleration filter) to estimate the vehicle state. However, the EKF filter has a strong dependence of acceleration on speed and speed on position due to the top-down construction, so if the optimized position is jittered or lagged, the jitter and lag of speed and acceleration will be more serious, resulting in poor accuracy of the obtained vehicle state. SUMMARY

[0003] To solve the technical problem of poor accuracy of the vehicle state obtained by the filter, the present disclosure is proposed. Embodiments of the present disclosure provide a vehicle state determination method and device, electronic equipment and storage medium.

[0004] According to an aspect of an embodiment of the present disclosure, a vehicle state determination method is provided, including: determining current bounding box information of a target vehicle based on a current frame image; determining a current observation size, a current observation longitudinal distance and a current observation lateral distance of the target vehicle based on the current bounding box information; determining a current state quantity of the target vehicle based on a previous state quantity of the target vehicle, the current observation size, the current observation longitudinal distance and the current observation lateral distance, the previous state quantity including a first number of historical frames corresponding to optimized state quantities respectively obtained in advance; and determining a current state of the target vehicle based on the current state quantity.

[0005] According to another aspect of the embodiments of the present disclosure, a determination apparatus of a vehicle state is provided, comprising: a first processing module configured to determine current bounding box information of a target vehicle based on a current frame image; a second processing module configured to determine a current observed size, a current observed longitudinal distance and a current observed lateral distance of the target vehicle based on the current bounding box information; a third processing module configured to determine a current state quantity of the target vehicle based on a previous state quantity of the target vehicle, the current observed size, the current observed longitudinal distance and the current observed lateral distance, the previous state quantity comprising a first number of historical frame corresponding optimized state quantities obtained in advance; and a fourth processing module configured to determine a current state of the target vehicle based on the current state quantity.

[0006] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, the storage medium storing a computer program, the computer program being configured to execute the determination method of a vehicle state according to any one of the embodiments of the present disclosure.

[0007] According to yet another aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory configured to store executable instructions of the processor; and the processor configured to read the executable instructions from the memory and execute the instructions to implement the determination method of a vehicle state according to any one of the embodiments of the present disclosure.

[0008] Based on the determination method of a vehicle state, the determination apparatus, the electronic device and the storage medium provided by the embodiments of the present disclosure, the observed size and the observed distance of the target vehicle are estimated based on the bounding box of the target vehicle, and the current state of the current frame is obtained by optimization in combination with the state of the target vehicle in a certain number of historical frames, so that the determination of the position, the speed and the acceleration of the target vehicle is realized at the same time, and the dependence of the acceleration on the speed and the dependence of the speed on the position can be effectively avoided, thereby greatly improving the accuracy and stability of the obtained state of the target vehicle.

[0009] The technical solutions of the present disclosure will be described in further detail below by means of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present disclosure serve to provide a further understanding about the present disclosure. The drawings constitute a part of the specification and are used to explain the present disclosure together with the embodiments of the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally indicate the same components or steps.

[0011] Figure 1is an exemplary application scenario of the method for determining the vehicle state provided by the present disclosure;

[0012] Figure 2 is a flowchart of the method for determining the vehicle state provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of the method for determining the vehicle state provided by another exemplary embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of step 203 provided by an exemplary embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of step 203 provided by another exemplary embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of the principle of determining the distance provided by an exemplary embodiment of the present disclosure;

[0017] Figure 7 is a structural schematic diagram of the device for determining the vehicle state provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8 is a structural schematic diagram of the device for determining the vehicle state provided by another exemplary embodiment of the present disclosure;

[0019] Figure 9 is a structural schematic diagram of an application embodiment of the electronic device of the present disclosure. DETAILED DESCRIPTION

[0020] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part but not all of the embodiments of the present disclosure, and the present disclosure can be implemented in other embodiments without departing from the spirit of the present disclosure.

[0021] It should be noted that: unless otherwise specified, the relative arrangement, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0022] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they represent the inherent logical order between them.

[0023] It should also be understood that in the embodiments of the present disclosure, "multiple" can mean two or more, and "at least one" can mean one, two or more.

[0024] It should also be understood that, for any components, data, or structures mentioned in the embodiments of the present disclosure, one or more can be generally understood in the absence of explicit limitation or in the presence of contrary indications in the preceding or subsequent text.

[0025] In addition, the term "and / or" in the present disclosure is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A existing alone, A and B existing simultaneously, and B existing alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0026] It should also be understood that the description of the embodiments of the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0027] At the same time, it should be understood that, for the convenience of description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0028] The following description of at least one example embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0029] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0030] It should be noted that similar reference numbers and letters refer to similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0031] The embodiments of the present disclosure can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, and the like.

[0032] Electronic devices such as terminal devices, computer systems, servers, and the like can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types. Computer systems / server can be practiced in distributed cloud computing environments with other computer systems coupled via communication networks. The program modules can be stored in the local or remote computer system memory devices.

[0033] Summary of the disclosure

[0034] In the process of implementing the present disclosure, the inventors found that in the automatic driving scene, it is very important to determine the state (such as position, speed, acceleration, etc.) of the surrounding dynamic vehicle in real time and stably for the safety of vehicle driving. In the related art, the state of the surrounding dynamic vehicle is usually determined based on a monocular camera filtering method, such as constructing an EKF (Extended Kalman Filter) filter for position, speed, and acceleration respectively from top to bottom (that is, the output of the position filter is taken as the input of the speed filter, and the output of the speed filter is taken as the input of the acceleration filter), and performing vehicle state estimation. However, the EKF filter is from top to bottom, which leads to strong dependence of acceleration on speed and speed on position. If the optimized position is jittered or lagged, the jitter and lag of speed and acceleration will be more serious, thereby leading to poor accuracy of the obtained vehicle state.

[0035] The vehicle state determination method provided by the embodiments of the present disclosure is not limited to be applied to the automatic driving scene, but can also be applied to other any possible scene according to actual needs, such as an assisted driving scene, without limitation.

[0036] Example overview

[0037] Figure 1 is an exemplary application scene of the vehicle state determination method provided by the present disclosure.

[0038] In an autonomous driving scenario, the images of surrounding target vehicles can be collected by cameras on the vehicle (ego vehicle), and using the vehicle state determination method of the present disclosure, the current detection box information of the observed target vehicle can be determined based on the collected current frame image, the current observation size, the current observation longitudinal distance and the current observation lateral distance of the target vehicle can be determined based on the current detection box information, and then the current state quantity of the target vehicle can be determined based on the preceding state quantity, the current observation size, the current observation longitudinal distance and the current observation lateral distance of the target vehicle, and the current state of the target vehicle can be determined based on the current state quantity. The preceding state quantity includes the optimized state quantity corresponding to each of the first number of historical frames obtained in advance. The current state of the target vehicle can include at least one of the position, speed, acceleration, size and other states of the target vehicle. The present disclosure estimates the observation size and observation distance of the target vehicle through the detection box of the target vehicle, and combines the state of the target vehicle in a certain number of historical frames to obtain the current state of the current frame through optimization, thereby realizing the parallel and simultaneous optimization of the position, speed, acceleration and other states of the target vehicle, effectively reducing the dependency between different states, and effectively avoiding the dependency of acceleration on speed and the dependency of speed on position compared with the existing filtering method, thereby greatly improving the accuracy and stability of the obtained target vehicle state.

[0039] Example method

[0040] Figure 2 is a flowchart of a vehicle state determination method provided by an exemplary embodiment of the present disclosure. The present embodiment can be applied on an electronic device, such as a vehicle-mounted computing platform, as shown in Figure 2 The present embodiment includes the following steps:

[0041] Step 201, determining the current detection box information of the target vehicle based on the current frame image.

[0042] The current frame image can include one or more perspective images, such as the current frame image collected by one or more perspective cameras on the vehicle (ego vehicle) at the current time. The target vehicle is a surrounding vehicle observed in the current frame image, and the number of target vehicles can be one or more. The current detection box information of each target vehicle can include the detection box position of the target vehicle in the current frame image. The specific representation of the detection box position can be set according to actual needs, such as the pixel coordinates of the detection box center point and the length and width of the detection box, or the pixel coordinates of the four corner points of the detection box, etc., and the specific representation is not limited.

[0043] In an optional embodiment, the current detection box information of the target vehicle can include the current full vehicle box information and the current tail box information of the target vehicle.

[0044] In an optional embodiment, the current bounding box information of the target vehicle can include current full vehicle bounding box information and current head bounding box information of the target vehicle.

[0045] In step 202, based on the current bounding box information, a current observation size, a current observation longitudinal distance and a current observation lateral distance of the target vehicle are determined.

[0046] The current observation size can include at least one of an observation height and an observation width of the target vehicle, which can be set according to actual requirements. The observation longitudinal distance represents the distance between the target vehicle and the ego vehicle in the longitudinal direction, and the current observation lateral distance represents the distance between the target vehicle and the ego vehicle in the lateral direction. The longitudinal direction can refer to the length direction of the ego vehicle, and the lateral direction can refer to the width direction of the ego vehicle.

[0047] In an optional embodiment, the distance between the target vehicle and the ego vehicle in the longitudinal direction and the distance between the target vehicle and the ego vehicle in the lateral direction can be represented by the distance between the target vehicle and the camera observing the target vehicle. Alternatively, the distance between the target vehicle and the camera can be converted into the distance between the target vehicle and the preset position of the ego vehicle according to the positional relationship between the camera and the ego vehicle, which can be set according to actual requirements.

[0048] In an optional embodiment, the current observation size can be determined based on the type of the target vehicle. The type of the target vehicle can be extracted from the target vehicle image of the current frame based on the current bounding box information, and then the target vehicle image can be classified to obtain the type.

[0049] In an optional embodiment, the type of the target vehicle can also be obtained together with the current bounding box information of the target vehicle. For example, a target detection model pre-trained to obtain the target detection model is used to perform target detection on the current frame image, and the current bounding box information and the type of the target vehicle are output simultaneously. The specific method can be set according to actual requirements.

[0050] In an optional embodiment, the current observation longitudinal distance and the current observation lateral distance can be determined based on corresponding preset mapping rules, respectively. The preset mapping rules can be set based on the proportional relationship between the image size and the physical space size of the target vehicle in the imaging principle.

[0051] In step 203, based on the preceding state quantity of the target vehicle, the current observation size, the current observation longitudinal distance and the current observation lateral distance, a current state quantity of the target vehicle is determined. The preceding state quantity includes the optimized state quantity corresponding to each of the first number of historical frames obtained in advance.

[0052] The first number can be set according to actual requirements.

[0053] In an optional embodiment, the optimized state quantity corresponding to any historical frame can be the state quantity of the target vehicle corresponding to the historical frame obtained by optimizing the determination process of the vehicle state of the historical frame.

[0054] In an optional embodiment, the optimized state quantity corresponding to any historical frame can also be the state quantity of the target vehicle of each historical frame obtained by optimizing the determination process of the vehicle state of the previous frame of the current frame. That is, the target vehicle state of all frames can be updated at each frame. For example, the current frame is the 10th frame. In addition to updating the current initial state quantity of the 10th frame to obtain the optimized state quantity of the current frame, the optimized state quantity of the target vehicle of the 9th frame obtained by updating the 9th frame can be optimized again to obtain the optimized state quantity corresponding to the 9th frame. Then, the optimized state quantity of the target vehicle corresponding to the 10th frame can be obtained in the optimization process of the 10th frame.

[0055] In an optional embodiment, the number of historical frames can also be limited by a sliding window. For example, the number of historical frames is limited to 10 frames. Then, the determination process of the vehicle state of each frame can determine the optimized state quantity of the target vehicle corresponding to 10 frames (including the current frame and 9 historical frames) when the number of observation frames exceeds 10 frames. The specific setting can be based on actual needs.

[0056] In an optional embodiment, a preset optimization algorithm can be used to optimize the state quantity to obtain the current state quantity of the target vehicle. The preset optimization algorithm can be any implementable optimization algorithm, such as a nonlinear least squares optimization algorithm. The iterative step of optimization is determined by a certain solving algorithm to realize the iterative optimization of the state quantity. The solving algorithm can be set according to actual needs, such as a trust region dogleg algorithm or a Gauss-Newton algorithm, and the like, without limitation.

[0057] In an optional embodiment, the state of the target vehicle at each frame can be optimized in the optimization process. Then, in addition to obtaining the current state quantity of the target vehicle, the optimized state quantity of each historical frame can also be obtained. The specific setting can be based on actual needs.

[0058] In an optional embodiment, the current state quantity of the target vehicle can be represented as a vector of a preset dimension, such as the current state quantity of the target vehicle which can be represented as follows:

[0059]

[0060] wherein, represents the optimized position of the target vehicle, represents the optimized speed of the target vehicle, represents the optimized acceleration of the target vehicle, an optimized height of the target vehicle, an optimized width of the target vehicle. The optimized position of the target vehicle can be the position of the target vehicle in the ego vehicle coordinate system (or the world coordinate system) in the current frame (the t-th frame).

[0061] In step 204, the current state of the target vehicle is determined based on the current state quantity.

[0062] The current state of the target vehicle can include at least one of the position, speed, acceleration, and size of the target vehicle, wherein the size of the target vehicle can include at least one of the height, width, and length. For example, if the target vehicle is in front of or behind the ego vehicle, the size of the target vehicle includes the height and the width; if the target vehicle is on the left side or the right side of the ego vehicle, the size of the target vehicle can include the height and the length; if the target vehicle is on the front left, the front right, the back left, or the back right of the ego vehicle, the size of the target vehicle can include the height, the length, and the width; and the specific setting can be made according to actual needs.

[0063] The vehicle state determination method provided in the embodiment estimates the observed size and the observed distance of the target vehicle through the bounding box of the target vehicle, and combines the state of the target vehicle in a certain number of historical frames to obtain the current state of the current frame through optimization, realizes the parallel and simultaneous optimization of the position, speed, acceleration, and other states of the target vehicle, effectively reduces the dependency between different states, and compared with the existing filtering method, can effectively avoid the dependency of acceleration on speed and the dependency of speed on position, greatly improves the accuracy and stability of the obtained state of the target vehicle.

[0064] Figure 3 is a flowchart of a vehicle state determination method provided by another exemplary embodiment of the disclosure.

[0065] In an optional embodiment, the optimized state quantity corresponding to any frame includes at least one of the position, speed, acceleration, and size of the target vehicle in the frame corresponding to the optimized state quantity; and step 203 can specifically include the following steps:

[0066] In step 2031, the current initial state quantity of the target vehicle is determined based on the preceding state quantity, the current observed size, the current observed longitudinal distance, and the current observed lateral distance.

[0067] The current initial state quantity represents the current unoptimized state quantity of the target vehicle. The current initial state quantity can include at least one of the position, speed, acceleration, and size of the target vehicle before optimization in the current frame.

[0068] In an optional embodiment, the pre-optimization position of the target vehicle in the current frame can be determined based on the current observed size, the current observed longitudinal distance and the current observed lateral distance.

[0069] In an optional embodiment, the pre-optimization speed and acceleration of the target vehicle in the current frame can be determined based on the preceding state quantity in combination with a motion model.

[0070] In an optional embodiment, the pre-optimization size of the target vehicle in the current frame can be determined based on the current observed size, which can be taken as the pre-optimization size.

[0071] In step 2032, the pre-optimization speed and acceleration of the target vehicle in the current frame can be determined based on the preceding state quantity in combination with a motion model.

[0072] In step 2032, the pre-optimization speed and acceleration of the target vehicle in the current frame can be determined based on the preceding state quantity in combination with a motion model.

[0073] In step 2033, the pre-optimization speed and acceleration of the target vehicle in the current frame can be determined based on the preceding state quantity in combination with a motion model.

[0074] In step 2032, the pre-optimization speed and acceleration of the target vehicle in the current frame can be determined based on the preceding state quantity in combination with a motion model.

[0075] Figure 4 FIG. 2 is a flowchart of step 203 according to an example embodiment of the present disclosure.

[0076] In an optional embodiment, step 2032 of optimizing the current initial state quantity based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance by using the preset optimization algorithm to obtain the optimized state quantity corresponding to the current frame comprises:

[0077] In step 20321, a target function is determined based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance, and the target function comprises at least one of a speed and acceleration physical constraint residual function, a position, speed and acceleration physical constraint residual function, a distance constraint residual function, a historical optimization constraint residual function, a size constraint residual function, a speed and acceleration weight constraint residual function and a motion model confidence constraint residual function.

[0078] wherein, the velocity acceleration physical constraint residual function is used to represent the constraint condition that the velocity state and the acceleration state need to satisfy; the position velocity acceleration physical preset residual function is used to represent the constraint condition that the position state, the velocity state and the acceleration state need to satisfy; the distance constraint residual function is used to represent the constraint condition that the current observed longitudinal distance and the current observed lateral distance need to satisfy; the history optimization constraint residual function is used to represent the constraint condition that the predicted state of the history optimization result needs to satisfy; the size constraint residual function is used to represent the constraint condition that the size state needs to satisfy; the velocity acceleration weight constraint residual function is used to represent the constraint condition that the weight of the velocity state and the acceleration state needs to satisfy; and the motion model confidence constraint residual function is used to represent the constraint condition that the confidence of the uniform speed model and the uniform acceleration model needs to satisfy. The purpose of the optimization is to minimize the target function, that is, to minimize the comprehensive residual corresponding to each residual function, so that the optimized state quantity can satisfy the constraint condition corresponding to each residual function, so as to ensure the accuracy and effectiveness of the optimized state quantity.

[0079] In an optional example, the velocity acceleration physical constraint residual function can be represented as follows:

[0080]

[0081] wherein, represents the power of the covariance matrix of the velocity, represents the optimized velocity state of the target vehicle corresponding to the kth frame, represents the optimized velocity state of the target vehicle corresponding to the k-1th frame, represents the optimized acceleration state of the target vehicle corresponding to the kth frame, represents the optimized acceleration state of the target vehicle corresponding to the k-1th frame, and Δt represents the time interval between adjacent two frames. B represents a frame set, which can be set according to actual needs, for example, can include all frames in which the target vehicle can be observed, or can be all frames in a sliding window, and the specific is not limited.

[0082] It should be noted that, since the velocity acceleration physical constraint residual function involves optimization variables including velocity and acceleration, in the optimization process, when the derivative of the residual function with respect to the state quantity is determined, only the derivative of the velocity and the acceleration with respect to the residual function needs to be calculated.

[0083] In an optional example, the position velocity acceleration physical constraint residual function can be represented as follows:

[0084]

[0085] wherein, represents the power of the covariance matrix of the position, represents the optimized position state of the target vehicle corresponding to the kth frame, represents the optimized position state of the target vehicle corresponding to the k-1th frame, and other symbols refer to the foregoing.

[0086] In an optional example, for the motion of the target vehicle, the uniform motion model and the uniform acceleration motion model can be comprehensively considered based on the motion model confidence, and the position-velocity-acceleration physical constraint residual function can be represented as follows:

[0087]

[0088] wherein, trust represents the motion model confidence, and the motion model confidence at this time is the model confidence optimized by the previous optimization process. In this residual function, trust is not an optimization variable.

[0089] Similarly with L1, when deriving, only the derivatives of the residual function with respect to the position, the velocity and the acceleration need to be calculated.

[0090] In an optional example, the distance constraint residual function can be represented as follows:

[0091]

[0092] wherein, represents the power of the covariance matrix of the distance, represents the coordinate of the longitudinal direction in the optimized position state of the target vehicle corresponding to the kth frame, represents the coordinate of the transverse direction in the optimized position state of the target vehicle corresponding to the kth frame, d_x represents the currently observed longitudinal distance, and d_y represents the currently observed transverse distance. Wherein, d_x can be determined based on the observation height or the observation width in the current observation size. Accordingly, the distance constraint residual function can include at least one of L3 based on the observation height and L3 based on the observation width, and can be set according to actual needs. In the distance constraint residual function, the optimization variable is the position.

[0093] In an optional example, the historical optimization constraint residual function can be represented as follows:

[0094]

[0095] wherein represents the power of the covariance matrix of the prediction, T represents transposition, and respectively represent the position, velocity and acceleration of the k-th frame predicted based on the optimized position, velocity and acceleration of the target vehicle in the (k-1)-th frame, combined with the state transition matrix of the motion model. In the historical optimization constraint residual function, the optimization variables are position, velocity and acceleration.

[0096] For example, taking the uniform acceleration motion model as an example, The specific motion model can be determined according to actual conditions.

[0097] In an optional example, the size constraint residual function can be represented as follows:

[0098]

[0099] wherein, represents the power of the covariance matrix of the observation height, represents the power of the covariance matrix of the observation width, represents the optimized height state of the target vehicle in the k-th frame, represents the observation height of the target vehicle in the k-th frame, represents the optimized width state of the target vehicle in the k-th frame, represents the observation width of the target vehicle in the k-th frame. In the size constraint residual function, the optimization variables are height and width.

[0100] In an optional example, the velocity and acceleration weight constraint residual function can be represented as follows:

[0101]

[0102] wherein ω represents a weight, and β is a preset value, In the velocity and acceleration weight constraint residual function, the optimization variables are velocity and acceleration.

[0103] In an optional example, the motion model confidence constraint residual function can be represented as follows:

[0104]

[0105] wherein trust represents the motion model confidence, represents the power of the covariance matrix of the confidence, CA represents the uniform acceleration model, and CV represents the uniform velocity model, represents the total displacement amount (such as the displacement amount of all frames in a sliding window) predicted based on the uniform acceleration model, represents the total displacement amount predicted based on the uniform velocity model, and Δp mearepresents the displacement amount determined based on the optimized position of the first frame (such as the first frame in the sliding window) and the optimized position of the last frame (such as the last frame in the sliding window). For example r represents the total number of frames in the sliding window. In the motion model confidence constraint residual function, the motion model confidence trust is an optimization variable, and only the derivative of the motion model confidence needs to be derived in the optimization process.

[0106] In an optional example, represents the following:

[0107]

[0108] In an optional example, may be represented as follows:

[0109]

[0110] where Δt i represents the interval time between the i-1th frame and the ith frame, and other symbols are described above.

[0111] Step 20322, determining the current iteration step based on the objective function.

[0112] where the current iteration step can include iteration steps corresponding to each state in the state quantity respectively. The current iteration step represents the iteration step of the state quantity in the current optimization process.

[0113] In an optional embodiment, the current iteration step determined based on the objective function can be obtained based on a preset solving algorithm, such as a trust region dogleg algorithm.

[0114] Step 20323, updating the current initial state quantity based on the current iteration step to obtain the optimized state quantity corresponding to the current frame.

[0115] where the current iteration step can include a current frame iteration step corresponding to the current initial state quantity. After the current iteration step is obtained, the optimized state quantity corresponding to the current frame can be determined based on the current initial state quantity and the current frame iteration step.

[0116] In an optional embodiment, the current iteration step can also include historical frame iteration steps corresponding to each historical frame, for re-optimizing the previous state quantity of the historical frame to obtain the optimized state quantity corresponding to each historical frame.

[0117] The embodiment is configured to construct a target function based on at least one of a speed acceleration physical constraint residual function, a position speed acceleration physical constraint residual function, a distance constraint residual function, a history optimization constraint residual function, a size constraint residual function, a speed acceleration weight constraint residual function and a motion model confidence constraint residual function, so that the constraints of the state quantity in the optimization process can be met, and the accuracy and effectiveness of the optimized state quantity can be ensured.

[0118] Figure 5 FIG. 3 is a flowchart of step 203 according to another example embodiment of the present disclosure.

[0119] In an optional embodiment, the current iteration step length includes a current frame iteration step length corresponding to the current initial state quantity and a current model confidence iteration step length; and step 20323 of updating the current initial state quantity based on the current iteration step length to obtain the optimized state quantity corresponding to the current frame includes:

[0120] Step 203231 of updating the current initial state quantity based on the current frame iteration step length to obtain the optimized state quantity corresponding to the current frame.

[0121] The method of the present disclosure further includes:

[0122] Step 3010 of updating the previous model confidence based on the current model confidence iteration step length to obtain the optimized current model confidence, wherein the current model confidence is used to represent the proportion of the uniform motion and the uniform acceleration motion of the target vehicle in the subsequent motion process.

[0123] The previous model confidence is the model confidence obtained in the previous optimization process. In the optimization process, the uniform motion of the target vehicle is described by a uniform motion model, and the uniform acceleration motion is described by a uniform acceleration motion model.

[0124] In an optional embodiment, the current model confidence can be obtained by adding the previous model confidence and the current model confidence iteration step length.

[0125] In an optional embodiment, the model confidence (trust) can also be used as one component of the state quantity, and the iteration step length is solved together with other components. The model confidence can be set according to actual requirements. The constraint of the model confidence is described in L7.

[0126] The embodiment can effectively determine the proportion of the uniform motion and the uniform acceleration motion in the motion process of the target vehicle by iteratively updating the model confidence, and further improve the accuracy of the optimized state quantity.

[0127] In an optional embodiment, the current iteration step length further comprises a historical frame iteration step length corresponding to each historical frame; the method of the present disclosure further comprises:

[0128] Step 3020, updating the previous state quantity based on the historical frame iteration step length corresponding to each historical frame to obtain the optimized state quantity corresponding to each historical frame.

[0129] In the formula, the historical frame iteration step length represents the iteration step length of the target vehicle in the state quantity corresponding to the historical frame, which is used to further optimize the previous state quantity corresponding to the historical frame of the target vehicle and improve the accuracy of the historical frame state of the target vehicle.

[0130] For example, frame sliding is performed through a sliding window, the sliding window includes 10 frames, specifically including a current frame and 9 historical frames, for each target vehicle, it is possible to be observed in one or more frames of the sliding window, then in the optimization process of the current frame, the state quantity of the target vehicle in one or more frames of the sliding window can be optimized to obtain the optimized state quantity corresponding to each frame. In this way, the optimization of the state quantity of each frame comprehensively considers the state quantity of the previous and subsequent frames and the observation, so that a more accurate state quantity of each frame can be further obtained, and the more accurate state quantity can be continuously used as a reference for optimization of subsequent frames, and so on, so that the optimization effect can be effectively improved.

[0131] In the embodiment, the state quantity of the target vehicle of the current frame and the historical frame can be optimized at the same time at each optimization, the optimized state quantity of the current frame can determine the real-time state of the target vehicle, and the optimized state quantity of the historical frame can determine a more accurate state of the target vehicle in the historical frame, participate in the optimization of the subsequent frame, further improve the accuracy of the state quantity of the historical frame, and then when used for the optimization of the subsequent frame, the accuracy of the optimization result of the subsequent frame can be improved.

[0132] In an optional embodiment, step 20321 of determining the target function based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance comprises:

[0133] 1. Determining the speed acceleration physical constraint residual function and the position speed acceleration physical constraint residual function based on the previous state quantity and the current initial state quantity.

[0134] For example, the speed acceleration physical constraint residual function is L1 as described above, and the position speed acceleration physical constraint residual function is L2 as described above, wherein, and The current iteration step length is represented by the position, speed and acceleration in the current initial state quantity and the corresponding current iteration step length, and is solved by a solving algorithm.

[0135] 2. Determine the distance constraint residual function based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance, the current observed lateral distance, and the historical observed longitudinal distance and the historical observed lateral distance corresponding to each historical frame.

[0136] For example, the distance constraint residual function refers to L3 described above. The historical observed longitudinal distance and the historical observed lateral distance are determined and stored in the historical frame observation, and the determination principle is consistent with the current observed longitudinal distance and the current observed lateral distance, which will not be described here.

[0137] Figure 6 is a schematic diagram of the determination principle of the distance provided by an exemplary embodiment of the present disclosure. Wherein C represents the camera optical center, XYZ represents the camera coordinate system, x represents the x direction of the pixel plane, y represents the y direction of the pixel plane, h represents the detection frame height of the target vehicle, H represents the observed height of the target vehicle, is the height of the physical space, a represents the corner point of the target vehicle detection frame in the pixel plane, A represents the point of the target vehicle in the physical space corresponding to a, d_x represents the current observed longitudinal distance of the target vehicle.

[0138] In an optional example, d_x can be determined based on the observed height H, and is represented as follows:

[0139]

[0140] Wherein H represents the observed height of the target vehicle, h represents the pixel height of the target vehicle in the current frame image, f v = f / d y , f represents the camera focal length corresponding to the current frame image, d y represents the physical size of 1 pixel in the y direction on the pixel plane, f v represents the length of the y direction focal length described by pixels.

[0141] In an optional example, d_x is determined based on the observed width, and is represented as follows:

[0142]

[0143] Wherein W represents the observed width of the target vehicle, w represents the pixel width of the target vehicle in the current frame image, f u = f / d x , f represents the camera focal length corresponding to the current frame image, d x represents the physical size of 1 pixel in the x direction on the pixel plane, f u represents the length of the x direction focal length described by pixels.

[0144] d_y is determined based on d_x, and is represented as follows:

[0145]

[0146] wherein, c x represents the coordinate of the x direction of the pixel coordinate system corresponding to the camera optical center C, cb x represents the coordinate of the x direction of the bottom edge midpoint of the target vehicle detection frame.

[0147] In an optional embodiment,

[0148]

[0149] wherein, the meanings of the symbols are as described above.

[0150] 3. Determine a history optimization constraint residual function based on the preceding state quantity, the current initial state quantity, and the state transition matrix of the acceleration model.

[0151] For example, the history optimization constraint residual function is as described above in L4.

[0152] 4. Determine a size constraint residual function based on the preceding state quantity, the current initial state quantity, the current observation size, and the history observation size corresponding to each history frame.

[0153] For example, the size constraint residual function is as described above in L5.

[0154] 5. Determine a speed and acceleration weight constraint residual function based on the preceding state quantity, the current initial state quantity, and a preset weight.

[0155] For example, the speed and acceleration weight constraint residual function is as described above in L6.

[0156] 6. Determine a motion model confidence constraint residual function based on the preceding state quantity, the current initial state quantity, the preceding model confidence, the uniform motion model, and the uniform acceleration motion model.

[0157] For example, the motion model confidence constraint residual function is as described above in L7.

[0158] 7. Determine a target function based on the speed and acceleration physical constraint residual function, the position, speed, and acceleration physical constraint residual function, the distance constraint residual function, the history optimization constraint residual function, the size constraint residual function, the speed and acceleration weight constraint residual function, and the motion model confidence constraint residual function.

[0159] In an optional example, the target function can be represented as follows:

[0160] L = L1 + L2 + L3 + L4 + L5 + L6 + L7

[0161] The embodiment determines a target function based on a speed acceleration physical constraint residual function, a position speed acceleration physical constraint residual function, a distance constraint residual function, a history optimization constraint residual function, a size constraint residual function, a speed acceleration weight constraint residual function and a motion model confidence constraint residual function, so as to solve an iterative step length to realize optimization of state quantity, ensure that the optimized state quantity can meet various constraints, and make the optimized state quantity more smooth. The various constraints contain common constraints of previous and subsequent frames, further improve the timeliness and accuracy of state response, and provide more reliable basis for subsequent decision and control.

[0162] In an optional embodiment, the step 20322 of determining the current iterative step length based on the target function comprises:

[0163] The step 203221 of determining the current iterative step length based on the target function adopts a trust region dog-leg algorithm.

[0164] The trust region dog-leg algorithm is a hybrid method of the Gauss-Newton algorithm and the fastest descent method, and replaces a damping term with a trust region.

[0165] In an optional example, the residual function (or the target function) is represented as F(x)=f(x) 2 , x represents a state quantity, and the specific process of the dog-leg algorithm is as follows:

[0166] 1. Initialize the trust region radius Δ0.

[0167] 2. Solve the gradient g k =J(x k ) T f(x k ), if ||g k ||≤∈1, exit, otherwise continue. If ||f(x k )||≤∈3, exit, otherwise continue. Wherein, T represents transposition, x k represents the state quantity to be optimized in the kth iteration, and J(x k ) represents the first-order partial derivative of f(x k ) with respect to the state quantity.

[0168] 3. If the trust region radius Δ k ≤∈2(||x k ||+∈2), exit the iteration, otherwise continue.

[0169] 4. Calculate the descent direction h gn and h sd according to the Gauss-Newton method and the fastest descent method respectively, and then calculate the iterative step length of the fastest descent method

[0170] 5. According to h gn h sd and Δ k Determine the Dog-Leg step value h dl If ||h dl ||≤∈2(||x k If ||+∈2), then exit the iteration; otherwise, continue.

[0171] If ||h gn ||≤Δ k Then h dl =h gn .

[0172] If ||αh sd ||≥Δ k ,but

[0173] Otherwise h dl =αh sd +β(h gn -αh sd ), where β is based on ||h dl ||=Δ k Sure.

[0174] 6. x new =x k +h dl Calculate the gain ratio L(0)-L(h dl F(x) represents the theoretical decrease value. k )-F(x new () indicates the actual decrease.

[0175] If ρ > 0, the optimized state variables

[0176] If ρ>0.75, Δ k+1 =max(Δ k ,3*||h dl ||).

[0177] If ρ < 0.25,

[0178] Repeat step 2.

[0179] Among them, ∈1, ∈2, and ∈3 can be set according to actual needs and used as the termination conditions of the iteration.

[0180] The specific process of the fastest descent method is as follows:

[0181] 1. Perform a first-order Taylor expansion on F(x).

[0182]

[0183] where x k represents the initial state quantity of the kth iteration, Δx k represents the iteration step length, represents the gradient direction.

[0184] 2. Determine the descending direction as the negative gradient direction.

[0185]

[0186] 3. Determine the appropriate step length.

[0187]

[0188] The Gauss-Newton method solves the iteration step length by the normal equation satisfied by the Jacobian matrix, the iteration step length and the residual (objective function). For example, the current Jacobian matrix is represented as J(x k ), the iteration step length is represented as Δx k , and the residual is represented as f(x k ), which satisfies the following equation:

[0189] J(x k ) T J(x k )△x k =-J(x k ) T f(x k )

[0190] where J(x k ) and f(x k ) are determined quantities, J(x k ) is determined by the first-order partial derivative of the residual with respect to the state quantity x k , and the iteration step length Δx k can be obtained by solving the equation set.

[0191] The current iteration step length is determined by the trust region dogleg algorithm in this embodiment, which can effectively solve the non-convergence situation that may occur in the Gauss-Newton method.

[0192] In an optional embodiment, the current observation size includes a current height and a current width of the target vehicle; the current bounding box information includes current full vehicle bounding box information and current rear vehicle bounding box information; and the current observation size, the current observation longitudinal distance and the current observation lateral distance of the target vehicle are determined based on the current bounding box information in step 202, including:

[0193] Step 2021: Determine the type of the target vehicle based on the current detection box information and the current frame image.

[0194] Specifically, full-vehicle image feature information can be extracted from the current frame image based on the current full-vehicle frame information in the current detection box information, rear-end image feature information can be extracted based on the current rear-end frame information, and the type of the target vehicle can be determined based on the full-vehicle image feature information and / or the rear-end image feature information.

[0195] In one alternative embodiment, the type of the target vehicle can be determined based on a pre-trained vehicle classification model. The network structure of the vehicle classification model can adopt any feasible classification structure, and this disclosure is not limited thereto.

[0196] Step 2022: Determine the current height and current width of the target vehicle based on its type.

[0197] Here, the current height refers to the aforementioned observation height H, and the current width refers to the aforementioned observation width W. Different vehicle types can be pre-configured with corresponding heights and widths. After determining the type of the target vehicle, the current height and current width of the target vehicle can be determined based on this correspondence.

[0198] Step 2023: Determine the current longitudinal distance based on the current height and the current full vehicle frame information; or, determine the current longitudinal distance based on the current width and the current full vehicle frame information.

[0199] The specific principle for determining the current longitudinal distance is explained in the aforementioned d_x, and will not be repeated here.

[0200] Step 2024: Determine the current observation lateral distance based on the current observed longitudinal distance and the current full vehicle frame information.

[0201] The specific principle for determining the current observation lateral distance is described in the aforementioned d. y This will not be elaborated upon here.

[0202] The embodiments or optional examples disclosed above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit it.

[0203] Any of the vehicle status determination methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the vehicle status determination methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the vehicle status determination methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0204] Example apparatus

[0205] Figure 7 is a structural schematic diagram of a determination apparatus of a vehicle state provided by an example embodiment of the present disclosure. The apparatus of this embodiment can be used to implement the corresponding method embodiments of the present disclosure, such as Figure 7 The apparatus shown includes a first processing module 501, a second processing module 502, a third processing module 503, and a fourth processing module 504.

[0206] The first processing module 501 is configured to determine current bounding box information of a target vehicle based on a current frame image.

[0207] The second processing module 502 is configured to determine a current observed size, a current observed longitudinal distance, and a current observed lateral distance of the target vehicle based on the current bounding box information.

[0208] The third processing module 503 is configured to determine a current state quantity of the target vehicle based on a preceding state quantity of the target vehicle, the current observed size, the current observed longitudinal distance, and the current observed lateral distance, the preceding state quantity including an optimized state quantity corresponding to a first number of historical frames obtained in advance.

[0209] The fourth processing module 504 is configured to determine a current state of the target vehicle based on the current state quantity.

[0210] Figure 8 is a structural schematic diagram of a determination apparatus of a vehicle state provided by another example embodiment of the present disclosure.

[0211] In an optional embodiment, the optimized state quantity corresponding to any frame includes at least one of an optimized position, speed, acceleration, and size of the target vehicle at the frame; and the third processing module 503 includes a first processing unit 5031, a second processing unit 5032, and a third processing unit 5033.

[0212] The first processing unit 5031 is configured to determine a current initial state quantity of the target vehicle based on the preceding state quantity, the current observed size, the current observed longitudinal distance, and the current observed lateral distance.

[0213] The second processing unit 5032 is configured to optimize the current initial state quantity based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance, and the current observed lateral distance, using a preset optimization algorithm, to obtain the optimized state quantity corresponding to the current frame.

[0214] The third processing unit 5033 is configured to take the optimized state quantity corresponding to the current frame as the current state quantity of the target vehicle.

[0215] In an optional embodiment, the second processing unit 5032 is specifically configured to:

[0216] determine, based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance, and the current observed lateral distance, a target function, the target function comprising at least one of a velocity-acceleration physical constraint residual function, a position-velocity-acceleration physical constraint residual function, a distance constraint residual function, a history optimization constraint residual function, a size constraint residual function, a velocity-acceleration weight constraint residual function, and a motion model confidence constraint residual function; determine, based on the target function, a current iteration step length; and update, based on the current iteration step length, the current initial state quantity to obtain an optimized state quantity corresponding to the current frame.

[0217] In an optional embodiment, the current iteration step length comprises a current frame iteration step length corresponding to the current initial state quantity and a current model confidence iteration step length; the second processing unit 5032 is specifically configured to: update, based on the current frame iteration step length, the current initial state quantity to obtain the optimized state quantity corresponding to the current frame; and the second processing unit 5032 is further configured to update, based on the current model confidence iteration step length, the preceding model confidence to obtain an optimized current model confidence, the current model confidence being used to represent a proportion of uniform motion and uniform acceleration motion of the target vehicle in the subsequent motion.

[0218] In an optional embodiment, the current iteration step length further comprises a history frame iteration step length corresponding to each of the history frames; and the second processing unit 5032 is further configured to update, based on the history frame iteration step length corresponding to each of the history frames, the preceding state quantity to obtain an optimized state quantity corresponding to each of the history frames.

[0219] In an optional embodiment, the second processing unit 5032 is specifically configured to:

[0220] The speed acceleration physical constraint residual function and the position speed acceleration physical constraint residual function are determined based on the previous state quantity and the current initial state quantity; the distance constraint residual function is determined based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance, the current observed transverse distance, and the historical observed longitudinal distance and the historical observed transverse distance corresponding to each historical frame; the historical optimization constraint residual function is determined based on the previous state quantity, the current initial state quantity, and a state transition matrix of the acceleration model; the size constraint residual function is determined based on the previous state quantity, the current initial state quantity, the current observed size, and the historical observed size corresponding to each historical frame; the speed acceleration weight constraint residual function is determined based on the previous state quantity, the current initial state quantity, and a preset weight; the motion model confidence constraint residual function is determined based on the previous state quantity, the current initial state quantity, the previous model confidence, the uniform motion model, and the uniform acceleration motion model; and the objective function is determined based on the speed acceleration physical constraint residual function, the position speed acceleration physical constraint residual function, the distance constraint residual function, the historical optimization constraint residual function, the size constraint residual function, the speed acceleration weight constraint residual function, and the motion model confidence constraint residual function.

[0221] In an optional embodiment, the second processing unit 5032 is specifically configured to: based on the objective function, determine the current iteration step length by using a trust region dogleg algorithm.

[0222] In an optional embodiment, the current observed size includes a current height and a current width of the target vehicle; the current bounding box information includes current full vehicle bounding box information and current tail bounding box information; and the second processing module 502 includes a first determining unit 5021, a second determining unit 5022, a third determining unit 5023, and a fourth determining unit 5024.

[0223] The first determining unit 5021 is configured to determine the type of the target vehicle based on the current bounding box information and the current frame image.

[0224] The second determining unit 5022 is configured to determine the current height and the current width of the target vehicle based on the type of the target vehicle.

[0225] The third determining unit 5023 is configured to determine the current observed longitudinal distance based on the current height and the current full vehicle bounding box information, or to determine the current observed longitudinal distance based on the current width and the current full vehicle bounding box information.

[0226] The fourth determining unit 5024 is configured to determine the current observed transverse distance based on the current observed longitudinal distance and the current full vehicle bounding box information.

[0227] It should be noted that the specific operations of the modules and units are described in the foregoing method embodiments, which will not be described here.

[0228] The modules or units of the embodiments of the present disclosure can be further divided into finer granularity according to actual needs, and can be specifically set according to actual needs.

[0229] Example electronic device

[0230] The embodiments of the present disclosure also provide an electronic device, comprising: a memory, configured to store a computer program;

[0231] A processor, configured to execute the computer program stored in the memory, and when the computer program is executed, the method for determining the state of the vehicle according to any one of the embodiments of the present disclosure is implemented.

[0232] Figure 9 is a structural schematic diagram of an application embodiment of the electronic device of the present disclosure. In the present embodiment, the electronic device 10 comprises one or more processors 11 and a memory 12.

[0233] The processor 11 can be a central processing unit (CPU) or other forms of processing units having data processing capability and / or instruction execution capability, and can control other components in the electronic device 10 to perform desired functions.

[0234] The memory 12 can comprise one or more computer program products, which can comprise various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can comprise, for example, random access memory (RAM), cache memory and / or the like. The non-volatile memory can comprise, for example, read-only memory (ROM), hard disk, flash memory and / or the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 11 can run the program instructions to implement the methods of the embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components and the like can also be stored in the computer readable storage medium.

[0235] In one example, the electronic device 10 can further comprise an input device 13 and an output device 14, and these components are interconnected by a bus system and / or other forms of connection mechanism (not shown).

[0236] For example, the input device 13 can be the microphone or microphone array described above, for capturing the input signal of the sound source.

[0237] In addition, the input device 13 can also include, for example, a keyboard, a mouse and the like.

[0238] The output device 14 can output various information including the determined distance information, direction information, etc. to the outside. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0239] Of course, in order to simplify, Figure 9 Only some of the components in the electronic device 10 related to the present disclosure are shown in the middle, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 10 can include any other appropriate components according to specific application cases.

[0240] Example computer program product and computer readable storage medium

[0241] In addition to the above-mentioned methods and devices, embodiments of the present disclosure can also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present disclosure described in the above "Exemplary Methods" section of the present specification.

[0242] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0243] In addition, embodiments of the present disclosure can also be a computer readable storage medium, which stores computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present disclosure described in the above "Exemplary Methods" section of the present specification.

[0244] The computer readable storage medium can be any combination of one or more computer readable medium(s). The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0245] The above generally describes the basic principles of the disclosure in conjunction with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the disclosure are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as the various embodiments of the disclosure must have. In addition, the above specific details of the disclosure are only for the purpose of example and for the purpose of understanding, and the above details do not limit the disclosure to the above specific details.

[0246] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be referred to each other. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0247] The block diagrams of the devices, apparatuses, equipment, systems involved in the disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "include but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0248] The methods and apparatus of the present disclosure can be implemented in a number of ways. For example, the methods and apparatus of the present disclosure can be implemented using software, hardware, firmware or any combination of software, hardware, firmware. The order of any steps described above is merely exemplary and the steps of the methods of the present disclosure need not be performed in the order described above unless otherwise specified. Furthermore, in some embodiments, the present disclosure can also be implemented as a program for running on a computer or a processor to implement the methods according to the present disclosure. Thus, the present disclosure also covers a record medium storing the program for executing the methods according to the present disclosure.

[0249] It is also noted that the methods of the present disclosure can be implemented by a computer or processor running a software program to execute the steps of the methods. In addition, the present disclosure covers any possible hardware and / or software combination having these functionalities. It is therefore contemplated to be covered by the scope of the present disclosure that combining computer software and hardware to create the methods of the present disclosure.

[0250] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0251] The above description has been presented for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although various example aspects and embodiments have been discussed above, those of skill in the art will recognize certain modifications, permutations, additions, and sub-combinations thereof.

Claims

1. A method for determining a state of a vehicle, comprising: determining, based on a current frame image, current bounding box information of a target vehicle; determining, based on the current bounding box information, a current observed size, a current observed longitudinal distance and a current observed lateral distance of the target vehicle; determining, based on a previous state quantity of the target vehicle, the current observed size, the current observed longitudinal distance, the current observed lateral distance, a current state quantity of the target vehicle, the previous state quantity comprising an optimized state quantity corresponding to a first number of historical frames obtained in advance respectively; determining, based on the current state quantity, a current state of the target vehicle; wherein the determining, based on the previous state quantity of the target vehicle, the current observed size, the current observed longitudinal distance, the current observed lateral distance, the current state quantity of the target vehicle comprises: determining, based on the previous state quantity, the current observed size, the current observed longitudinal distance and the current observed lateral distance, a current initial state quantity of the target vehicle; optimizing, based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance, the current initial state quantity by using a preset optimization algorithm to obtain an optimized state quantity corresponding to the current frame; taking the optimized state quantity corresponding to the current frame as the current state quantity of the target vehicle.

2. The method of claim 1, wherein, The optimized state quantity corresponding to any frame comprises at least one state of a position, a speed, an acceleration and a size of the target vehicle corresponding to the frame.

3. The method of claim 1, wherein, The optimizing, based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance, the current initial state quantity by using a preset optimization algorithm to obtain an optimized state quantity corresponding to the current frame comprises: determining, based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance, an objective function, the objective function comprising at least one of a speed acceleration physical constraint residual function, a position speed acceleration physical constraint residual function, a distance constraint residual function, a historical optimization constraint residual function, a size constraint residual function, a speed acceleration weight constraint residual function and a motion model confidence constraint residual function; determining a current iteration step length based on the objective function; updating the current initial state quantity based on the current iteration step length to obtain the optimized state quantity corresponding to the current frame.

4. The method of claim 3, wherein, The current iteration step length comprises a current frame iteration step length and a current model confidence iteration step length corresponding to the current initial state quantity. The updating the current initial state quantity based on the current iteration step length to obtain the optimized state quantity corresponding to the current frame comprises: updating the current initial state quantity based on the current frame iteration step length to obtain the optimized state quantity corresponding to the current frame. The method further comprises: The current model confidence is updated based on the current model confidence iteration step, and an optimized current model confidence is obtained, the current model confidence being used to represent a proportion of uniform motion and uniform acceleration motion of the target vehicle in subsequent motion.

5. The method of claim 4, wherein, The current iteration step further includes a historical frame iteration step corresponding to each of the historical frames. The method further includes: The previous state quantity is updated based on the historical frame iteration step corresponding to each of the historical frames, and an optimized state quantity corresponding to each of the historical frames is obtained.

6. The method of claim 3, wherein, The target function is determined based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance, and the current observed transverse distance, including: The speed acceleration physical constraint residual function and the position speed acceleration physical constraint residual function are determined based on the previous state quantity and the current initial state quantity; The distance constraint residual function is determined based on the previous state quantity, the current initial state quantity, the current observed longitudinal distance, the current observed transverse distance, and historical observed longitudinal distance and historical observed transverse distance corresponding to each of the historical frames; The historical optimization constraint residual function is determined based on the previous state quantity, the current initial state quantity, and a state transition matrix of an acceleration model; The size constraint residual function is determined based on the previous state quantity, the current initial state quantity, the current observed size, and historical observed size corresponding to each of the historical frames; The speed acceleration weight constraint residual function is determined based on the previous state quantity, the current initial state quantity, and a preset weight; The motion model confidence constraint residual function is determined based on the previous state quantity, the current initial state quantity, the previous model confidence, a uniform motion model, and a uniform acceleration motion model; The target function is determined based on the speed acceleration physical constraint residual function, the position speed acceleration physical constraint residual function, the distance constraint residual function, the historical optimization constraint residual function, the size constraint residual function, the speed acceleration weight constraint residual function, and the motion model confidence constraint residual function.

7. The method of claim 3, wherein, The current iteration step is determined based on the target function, including: The current iteration step is determined based on the target function by using a trust region dogleg algorithm.

8. The method of claim 1, wherein, The current observed size includes a current height and a current width of the target vehicle; and the current bounding box information includes current full vehicle bounding box information and current tail bounding box information. The current observed size, the current observed longitudinal distance, and the current observed transverse distance of the target vehicle are determined based on the current bounding box information, including: The type of the target vehicle is determined based on the current bounding box information and the current frame image; The current height and the current width of the target vehicle are determined based on the type of the target vehicle; The current observed longitudinal distance is determined based on the current height and the current full vehicle bounding box information, or the current observed longitudinal distance is determined based on the current width and the current full vehicle bounding box information; and The current observed transverse distance is determined based on the current width and the current tail bounding box information. Determine the current observed lateral distance based on the current observed longitudinal distance and the current whole-vehicle bounding box information.

9. A vehicle state determination apparatus, comprising: a first processing module configured to determine current bounding box information of a target vehicle based on a current frame image; a second processing module configured to determine a current observed size, a current observed longitudinal distance and a current observed lateral distance of the target vehicle based on the current bounding box information; a third processing module configured to determine a current state quantity of the target vehicle based on a preceding state quantity of the target vehicle, the current observed size, the current observed longitudinal distance and the current observed lateral distance, the preceding state quantity comprising a first number of historical frame corresponding optimized state quantities obtained in advance; a fourth processing module configured to determine a current state of the target vehicle based on the current state quantity; wherein the third processing module comprises a first processing unit, a second processing unit and a third processing unit; the first processing unit is configured to determine a current initial state quantity of the target vehicle based on the preceding state quantity, the current observed size, the current observed longitudinal distance and the current observed lateral distance; the second processing unit is configured to optimize the current initial state quantity based on the preceding state quantity, the current initial state quantity, the current observed longitudinal distance and the current observed lateral distance by using a preset optimization algorithm to obtain a current frame corresponding optimized state quantity; the third processing unit is configured to take the current frame corresponding optimized state quantity as the current state quantity of the target vehicle.

10. A computer readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the vehicle state determination method of any one of claims 1-8.

11. An electronic device, comprising: a processor; a memory configured to store executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the vehicle state determination method of any one of claims 1-8.

Images