Method for determining feature information of target object, electronic equipment and storage medium

Abstract

The invention discloses a method for determining feature information of a target object, electronic equipment and a storage medium, and belongs to the technical field of intelligent driving. The method comprises the following steps: acquiring first sensing data of a target object around a vehicle at a first moment, wherein the first sensing data is used for indicating a first measurement value of feature information of the target object at the first moment under a bird's-eye view sensing view angle; acquiring state information of the vehicle, second sensing data related to the feature information at the first moment and a predicted value; and determining a true value of the feature information at the first moment according to the state information, the predicted value, the second measurement value and the first measurement value. Through the method, the advantages of perception and image perception can be combined, and the accuracy of determining the feature information of the target object is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a method, an electronic device, and a storage medium for determining characteristic information of a target object. Background Art

[0002] Determining the characteristic information of a target object refers to the process of comprehensively estimating multi-dimensional characteristics such as the shape, size, position, and speed of the target object to obtain accurate and complete target characteristic information.

[0003] Currently, the determination of the characteristic information of a target object mainly uses the BEV (bird’s eye view) depth model solution, and the output result is the detection frame of the target object in the BEV perception perspective.

[0004] However, since the BEV depth model outputs fewer features, it is difficult to continuously track the target object, resulting in large fluctuations in the shape and position of the target object between consecutive frames, and poor accuracy when determining the characteristic information of the target object. Summary of the Invention

[0005] This application provides a method, an electronic device, and a storage medium for determining characteristic information of a target object to solve the technical problems existing in the related art. Specifically, the following technical solutions are included.

[0006] In a first aspect, this application provides a method for determining characteristic information of a target object, the method including: obtaining first perception data of a target object around a vehicle at a first moment, the first perception data being used to indicate a first measurement value of the characteristic information of the target object at the first moment in a bird’s eye view perception perspective; obtaining state information of the vehicle, second perception data related to the characteristic information at the first moment, and a prediction value, the second perception data being used to indicate a second measurement value of the characteristic information at the first moment in an image perception perspective, the prediction value being determined according to a third measurement value of the characteristic information of the target object at a second moment in a bird’s eye view perception perspective, the second moment being earlier than the first moment; determining a true value of the characteristic information at the first moment according to the state information, the prediction value, the second measurement value, and the first measurement value.

[0007] In some possible implementation manners, the determining a true value of the characteristic information at the first moment according to the state information, the prediction value, the second measurement value, and the first measurement value includes: determining displacement information of the vehicle between the first moment and the second moment according to the state information; performing motion compensation on the prediction value according to the displacement information; determining a true value of the characteristic information at the first moment according to the prediction value after motion compensation, the second measurement value, and the first measurement value.

[0008] In some possible embodiments, determining the true value of the feature information at the first moment based on the predicted value after motion compensation, the second measurement value, and the first measurement value includes: constructing a joint optimization function for the true value according to the predicted value after motion compensation and the first credibility of the predicted value, the first measurement value and the second credibility of the first measurement value, the second measurement value and the third credibility of the second measurement value, where the joint optimization function is used to determine at least one estimated value of the true value under different hyperparameter combinations, and the hyperparameter combinations are related to the first credibility, the second credibility, and the third credibility; determining the true value of the feature information at the first moment according to the joint optimization function.

[0009] In some possible embodiments, the joint optimization function includes a first optimization term, a second optimization term, and a third optimization term. Constructing the joint optimization function for the true value according to the predicted value after motion compensation and the first credibility of the predicted value, the first measurement value and the second credibility of the first measurement value, the second measurement value and the third credibility of the second measurement value includes: constructing the first optimization term according to the predicted value after motion compensation, where the first optimization term is used to indicate the difference between the predicted value and the true value; constructing the second optimization term according to the first measurement value, where the second optimization term is used to indicate the difference between the first measurement value and the true value; constructing the third optimization term according to the second measurement value, where the third optimization term is used to indicate the difference between the second measurement value and the true value; constructing the joint optimization function according to the first credibility and the first optimization term, the second credibility and the second optimization term, and the third credibility and the third optimization term.

[0010] In some possible embodiments, determining the true value of the feature information at the first moment according to the joint optimization function includes: determining the minimum value of the joint optimization function under the specified hyperparameter combination and given constraints as the true value at the first moment.

[0011] In some possible embodiments, the second measurement value includes the key measurement values of the key measurement points of the target object and the bounding box measurement values of the bounding box of the target object. Determining the true value of the feature information at the first moment according to the state information, the predicted value, the second measurement value, and the first measurement value includes: if the key measurement value meets a preset condition, determining the true value of the feature information at the first moment according to the state information, the predicted value, the key measurement value, and the first measurement value; or, if the key measurement value does not meet the preset condition, determining the true value of the feature information at the first moment according to the state information, the predicted value, the first measurement value, and the bounding box measurement value.

[0012] In some possible embodiments, the key measurement value includes visibility information for indicating the clarity of the key measurement point, and the preset condition includes: when there are multiple key measurement points, the number of visibility information exceeding the visibility threshold among the multiple visibility information corresponding to the multiple key measurement points reaches a predetermined number.

[0013] In some possible embodiments, the target object includes a target vehicle, and the key measurement points include the wheel contact points of the target vehicle.

[0014] In a second aspect, the present application provides a device for determining feature information of a target object, including: a first acquisition module, configured to acquire first perception data of a target object around a vehicle at a first moment, where the first perception data is used to indicate a first measurement value of the feature information of the target object at the first moment from a bird's-eye view perception perspective; a second acquisition module, configured to acquire state information of the vehicle, second perception data related to the feature information at the first moment, and a predicted value, where the second perception data is used to indicate a second measurement value of the feature information at the first moment from an image perception perspective, and the predicted value is determined according to a third measurement value of the feature information of the target object at a second moment from a bird's-eye view perception perspective, and the second moment is earlier than the first moment; a determination module, configured to determine the true value of the feature information at the first moment according to the state information, the predicted value, the second measurement value, and the first measurement value.

[0015] In some possible embodiments, the determination module is configured to determine displacement information of the vehicle between the first moment and the second moment according to the state information; perform motion compensation on the predicted value according to the displacement information; and determine the true value of the feature information at the first moment according to the predicted value after motion compensation, the second measurement value, and the first measurement value.

[0016] In some possible embodiments, the determining module is configured to construct a joint optimization function of the true value according to the predicted value after motion compensation and the first credibility of the predicted value, the first measurement value and the second credibility of the first measurement value, the second measurement value and the third credibility of the second measurement value, where the joint optimization function is used to determine at least one estimated value of the true value under different hyperparameter combinations, and the hyperparameter combinations are related to the first credibility, the second credibility, and the third credibility; determine the true value of the feature information at the first moment according to the joint optimization function.

[0017] In some possible embodiments, the joint optimization function includes a first optimization term, a second optimization term, and a third optimization term. The determining module is configured to construct the first optimization term according to the predicted value after motion compensation and the first credibility of the predicted value, the first measurement value and the second credibility of the first measurement value. The first optimization term is used to indicate the difference between the predicted value and the true value; construct the second optimization term according to the first measurement value, and the second optimization term is used to indicate the difference between the first measurement value and the true value; construct the third optimization term according to the second measurement value, and the third optimization term is used to indicate the difference between the second measurement value and the true value; construct the joint optimization function according to the first credibility and the first optimization term, the second credibility and the second optimization term, and the third credibility and the third optimization term.

[0018] In some possible embodiments, the determining module is configured to determine the minimum value of the joint optimization function under the specified hyperparameter combination and given constraints as the true value at the first moment.

[0019] In some possible embodiments, the second measurement value includes the key measurement value of the key measurement point of the target object and the bounding box measurement value of the bounding box of the target object. The determining module is configured to, if the key measurement value meets a preset condition, determine the true value of the feature information at the first moment according to the state information, the predicted value, the key measurement value, and the first measurement value; or, if the key measurement value does not meet the preset condition, determine the true value of the feature information at the first moment according to the state information, the predicted value, the first measurement value, and the bounding box measurement value.

[0020] In some possible embodiments, the key measurement value includes visibility information indicating the clarity of the key measurement point, and the preset condition includes: when there are multiple key measurement points, the number of visibility information exceeding the visibility threshold among the multiple visibility information corresponding to the multiple key measurement points reaches a predetermined number.

[0021] In some possible embodiments, the target object includes a target vehicle, and the key measurement points include the wheel contact points of the target vehicle.

[0022] In a third aspect, the present application provides an electronic device for determining characteristic information of a target object, including: a memory storing at least one program instruction for determining characteristic information of the target object; a processor, when the program instruction is executed by the processor, enabling the vehicle to implement the method in the first aspect or any possible embodiment of the first aspect of the present application.

[0023] In a fourth aspect, the present application provides a computer program (product), the computer program (product) including computer programs / instructions, the computer programs / instructions being executed by a processor to enable the vehicle to implement the control method in the first aspect or any possible embodiment of the first aspect of the present application.

[0024] In a fifth aspect, the present application provides a computer-readable storage medium storing program instructions for determining characteristic information of a target object, when the program instructions are executed by one or more processors, enabling the vehicle to implement the method in the first aspect or any possible embodiment of the first aspect of the present application.

[0025] The beneficial effects of the technical solution provided by the present application at least include:

[0026] For the technical solution provided by the present application, on the one hand, the characteristic information of the target object can be perceived from data in different time dimensions through the first measurement values and predicted values of the characteristic information of the target object at different times before and after from the perspective of the bird's-eye view perception. This significantly improves the accuracy when determining the characteristic information of the target object. On the other hand, the second measurement value of the characteristic information of the target object from the perspective of image perception is also combined, improving the information richness of the characteristic information of the target object from the perspective of the bird's-eye view perception. It not only can accurately track the target according to the characteristic information of the target object, which is beneficial to improving the accuracy of target perception, but also can more comprehensively perceive the characteristic information of the target object, which is beneficial to downstream task processing. Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0028] Figure 1 It is a schematic diagram of an implementation scenario provided by an embodiment of the present application;

[0029] Figure 2 is a flowchart of a method for determining characteristic information of a target object provided by an embodiment of the present application;

[0030] Figure 3 is a schematic structural diagram of a device for determining characteristic information of a target object provided by an embodiment of the present application;

[0031] Figure 4 is a schematic structural diagram of an electronic device for determining characteristic information of a target object provided by an embodiment of the present application. Detailed implementation manners

[0032] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0033] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0034] Figure 1 is a schematic diagram of an implementation scenario provided by an embodiment of the present application. Refer to Figure 1 , the implementation scenario provided by the embodiment of the present application may include a vehicle 11 and a control unit 12.

[0035] A variety of sensors for detecting target objects around the vehicle are provided in the vehicle 11, and the variety of sensors can establish a communication connection with the control unit 12 in a wired or wireless manner, so that the control unit 12 can perceive the target objects around the vehicle 11 through the variety of sensors to determine the characteristic information of the target objects around the vehicle 11. Among them, the target objects around the vehicle may include, for example, any other traffic participants such as pedestrians and other vehicles, and the present application does not make any restrictions in this regard.

[0036] Optionally, the control unit 12 may be a single server or a server cluster composed of multiple servers. The variety of sensors may also include any sensors such as lidar and millimeter wave radar that can detect the surrounding environment of the vehicle, and the present application does not make any restrictions in this regard.

[0037] Those skilled in the art should understand that the above-mentioned vehicle 11 and control unit 12 are only examples. Other existing or future vehicles or control units that can be applied to this application should also be included within the protection scope of this application and are hereby incorporated herein by reference.

[0038] Figure 2 FIG. 4 is a flowchart of a method for determining feature information of a target object provided by an embodiment of the present application. This method can be executed, for example, by a control unit, a control module, or a control system installed in a vehicle. The present application makes no limitation in this regard. Refer to Figure 2 , the method for determining feature information of a target object provided by an embodiment of the present application may include the following steps.

[0039] Step S210, obtain first perception data of a target object around the vehicle at a first moment. The first perception data is used to indicate a first measurement value of the feature information of the target object at the first moment in a bird's-eye view (BEV) perception perspective.

[0040] Optionally, the target object around the vehicle may be, for example, other traffic participants such as pedestrians or other vehicles around the vehicle that are on the same road or in the same traffic environment as the vehicle. The feature information of the target object at the first moment can be used, for example, but not limited to indicating the true value of the feature information of the target object in the actual space at the first moment, and may include, for example, but not limited to, spatial extension attributes such as the shape and size of the target object in the actual space (for example, the shape and size of the projection of the target object on the horizontal plane), and state information such as the movement and position of the target object at the first moment (for example, the movement and position of the target object on the horizontal plane).

[0041] In some embodiments, the expression form of the feature information of the target object at the first moment may refer to formula (1):

[0042] X = [x, y, v x , v y , θ, L, W] (1)

[0043] where X is the feature information of the target object at the first moment; x is the abscissa of the center point of the horizontal plane projection of the target object in the vehicle coordinate system in the actual space; y is the ordinate of the center point of the horizontal plane projection of the target object in the vehicle coordinate system in the actual space; v x is the speed of the target object in the abscissa direction of the vehicle coordinate system in the actual space; v yv is the speed of the target object in the vertical coordinate direction of the vehicle coordinate system in the actual space; θ is the rotation angle of the target object relative to the horizontal plane in the actual space (i.e., the rotation angle of the target object in the vertical plane); L is the maximum length of the horizontal plane projection of the target object in the actual space; W is the maximum width of the horizontal plane projection of the target object in the actual space. Among them, the vehicle coordinate system can be, for example, a coordinate system with the vehicle as the coordinate origin, the heading direction of the vehicle as the y-axis, and the direction perpendicular to the vehicle heading as the x-axis.

[0044] Exemplarily, the first perception data of the target object at the first moment can be used for but not limited to indicating the first measurement value of the feature information of the target object at the first moment from the BEV perception perspective. For example, it can be the state and features of the target in the BEV perception perspective obtained after processing and conversion of the feature information of the target object at the first moment. Due to the limitations of the BEV perception perspective, there may be a deviation between the first measurement value and the true value of the feature information of the target object at the first moment.

[0045] In some embodiments, the manifestation form of the first measurement value can refer to, for example, formula (2):

[0046] Y BEV =[x1, y1, θ1, L1, W1] (2)

[0047] Among them, Y BEV is the first measurement value, x1 is the abscissa of the center point of the horizontal plane projection of the target object in the vehicle coordinate system from the BEV perception perspective; y1 is the ordinate of the center point of the horizontal plane projection of the target object in the vehicle coordinate system from the BEV perception perspective; θ1 is the rotation angle of the target object relative to the horizontal plane from the BEV perception perspective; L1 is the maximum length of the horizontal plane projection of the target object from the BEV perception perspective; W1 is the maximum width of the horizontal plane projection of the target object from the BEV perception perspective. The vehicle coordinate system can also be, for example, a coordinate system with the vehicle as the coordinate origin, the heading direction of the vehicle as the y-axis, and the direction perpendicular to the vehicle heading as the x-axis.

[0048] In some embodiments, the feature information of the target object can be obtained, for example, by detecting the target object around the vehicle through any on-vehicle sensors such as visual sensors, radar sensors, and laser sensors. The method of processing and converting the feature information to obtain the first perception data can be adjusted according to the acquisition method of the feature information in the actual application scenario, and the present application does not make any restrictions on this.

[0049] Step S220: Obtain the status information of the vehicle, second perception data related to the feature information at the first moment, and a predicted value. The second perception data is used to indicate the second measurement value of the feature information at the first moment from the perspective of image perception. The predicted value is determined based on the third measurement value of the feature information of the target object at the second moment from the perspective of bird's-eye view perception, where the second moment is earlier than the first moment.

[0050] Exemplarily, the status information of the vehicle may include, for example, but is not limited to, any information related to the motion state of the vehicle, such as the speed, position, mileage information, etc. of the vehicle.

[0051] The second perception data related to the feature information of the target object at the first moment may include, for example, but is not limited to, indicating the second measurement value of the feature information of the target object at the first moment from the perspective of image perception. For example, it may be obtained by detecting the feature information of the target object at the first moment through a vision sensor mounted on the vehicle, including the state and features of the feature information of the target object at the first moment from the perspective of image perception. Similarly, due to the limitations of the image perception perspective, there may also be a deviation between the second measurement value and the true value of the feature information of the target object at the first moment.

[0052] In some embodiments, the expression form of the second measurement value may, for example, refer to Formula (3):

[0053] Y 2D =[u,v,L2,W2] (3)

[0054] where Y 2D is the second measurement value, u is the abscissa of the center point of the target object in the image coordinate system from the perspective of image perception; v is the ordinate of the center point of the target object in the image coordinate system from the perspective of image perception; L2 is the maximum length of the target object in the image coordinate system from the perspective of image perception; W2 is the maximum width of the target object in the image coordinate system from the perspective of image perception. Here, the origin of the image coordinate system may be, for example, the corner point in the upper left corner of the image, the abscissa may be the row direction of the image, and the ordinate may be the column direction of the image, which can be used to describe, for example, but is not limited to, the pixel positions in the image.

[0055] Optionally, the feature information of the target object at the second moment can be used, for example, but not limited to, indicating the true value in the physical space of the feature information of the target state at the second moment. Similarly, it can include, for example, but not limited to, spatial extension attributes such as the shape, size, and position of the target object in the physical space, as well as state information such as the movement and position of the target object at the second moment. The third measurement value of the feature information of the target object at the second moment in the BEV perception perspective can be, for example, the state and features of the target in the BEV perception perspective obtained after processing and transformation of the feature information of the target object at the second moment. Here, the second moment is earlier than the first moment.

[0056] Since the second moment is earlier than the first moment, in view of this, the feature information of the target object at the first moment can be predicted based on the third measurement value of the feature information of the target object at the second moment in the bird's-eye view perception perspective. Similarly, due to the limitations of the prediction method, there may also be a deviation between the predicted value and the true value of the feature information of the target object at the first moment.

[0057] In some embodiments, determining the predicted value of the feature information of the target object at the first moment based on the third measurement value of the feature information of the target object at the second moment in the BEV perception perspective can be carried out, for example, through the following formulas (4) and (5):

[0058]

[0059] Where, is the predicted value of the feature information of the target object at the first moment; is the feature information of the target object at the second moment; A k-1 is the state transition matrix of the feature information of the target object from the second moment to the first moment, used to describe the change law of the feature information of the target object from the second moment to the first moment.

[0060]

[0061] Where, is the deviation between the predicted value and the true value of the feature information of the target object at the first moment; is the deviation between the predicted value and the true value of the feature information of the target object at the second moment; A k-1 is the state transition matrix of the feature information of the target object from the second moment to the first moment; is A k-1 's transpose matrix; Q k is the noise caused by the prediction model during the prediction process. Here, the predicted value of the feature information at the second moment can be determined based on the feature information of other moments earlier than the second moment.

[0062] As described above, the feature information of the target object at the first moment may include the spatial expansion attribute and the state information of the target object. Considering the differences between the spatial expansion attribute and the state information, different prediction models may be used to predict the spatial expansion attribute and the state information of the target object at the first moment respectively. The first predicted value of the spatial expansion attribute of the target object at the first moment is obtained according to the spatial expansion attribute of the target object at the second moment, and the second predicted value of the state information of the target object at the first moment is obtained according to the state information of the target object at the second moment.

[0063] In some embodiments, obtaining the second predicted value of the state information of the target object at the first moment according to the state information of the target object at the second moment may be performed, for example, through the following formula (6):

[0064]

[0065] Wherein, is the second predicted value of the state information of the target object at the first moment; is the state information of the target object at the second moment; is the state transition matrix determined according to the kinematic characteristics of the target object, and is used to describe the change rule of the state information of the target object between the second moment and the first moment. Wherein, For example, it may be determined according to a uniform motion model, and is used to represent the process in which the state information of the target object changes uniformly between the second moment and the first moment; or it may also be determined according to a uniformly accelerated motion model, and is used to represent the process in which the state information of the target object changes uniformly accelerated between the second moment and the first moment; or it is other state transition matrices determined according to the actual application situation, and the present application does not make any restrictions in this regard.

[0066] In some embodiments, obtaining the first predicted value of the spatial expansion attribute of the target object at the first moment according to the spatial expansion attribute of the target object at the second moment may be performed, for example, through the following formula (7):

[0067]

[0068] is the first predicted value of the spatial expansion attribute of the target object at the first moment; is the spatial expansion attribute of the target object at the second moment; is the state transition matrix determined according to the physical characteristics of the spatial expansion attribute of the target object, and is used to describe the change rule of the spatial expansion attribute of the target object between the second moment and the first moment. Wherein, For example, it may include an identity matrix, which is used to represent that the spatial extension attributes of the target object (such as the size and shape of the horizontal plane projection of the target object) have not changed between the second moment and the first moment; or it may be other state transition matrices determined according to the actual application situation, and the present application does not impose any restrictions in this regard.

[0069] Step S230, determine the true value of the feature information at the first moment according to the state information, the predicted value, the second measurement value, and the first measurement value.

[0070] As described above, due to the limitations of the prediction method, the BEV perception perspective, and the image perception perspective, there may be deviations between the predicted value, the first measurement value, and the second measurement value and the true value of the feature information of the target object at the first moment. In view of this, by combining the predicted value, the first measurement value, the second measurement value, and the state information of the vehicle itself, the present application can not only enhance the accuracy of the true value of the feature information of the target object at the first moment, but also integrate the characteristics of different data, complement each other's advantages, and enrich the information dimension when perceiving the feature information of the target object at the first moment.

[0071] Considering that in the actual application scenario, the movement of the vehicle may increase the error between the predicted value and the true value. In view of this, the predicted value can be compensated for movement according to the movement of the vehicle to improve the accuracy of the predicted value. In some embodiments, determining the true value of the feature information at the first moment according to the state information, the predicted value, the second measurement value, and the first measurement value may include, for example: determining the displacement information of the vehicle between the first moment and the second moment according to the state information; compensating the predicted value for movement according to the displacement information; determining the true value of the feature information at the first moment according to the predicted value after movement compensation, the second measurement value, and the first measurement value.

[0072] As described above, the predicted value may include a first predicted value related to the spatial extension attributes of the target object at the first moment, and a second predicted value related to the state information of the target object at the first moment. Considering that in the actual application scenario, the displacement of the vehicle in different planes may have different effects on the first predicted value and the second predicted value. For example, the displacement of the vehicle in the horizontal plane (such as the displacement and rotation in the horizontal plane) mainly affects the second predicted value related to the state information of the target object at the first moment, and the displacement of the vehicle in the vertical plane (such as the displacement and rotation in the vertical plane) mainly affects the first predicted value related to the spatial extension attributes of the target object at the first moment. In view of this, the second predicted value and the first predicted value can be compensated for movement according to the displacement of the vehicle in the horizontal plane and the vertical plane respectively.

[0073] In some embodiments, the displacement of the vehicle in the horizontal plane can be determined by the following formula (8):

[0074]

[0075] Among them, is the displacement information of the vehicle between the second moment and the first moment, is the mileage information of the vehicle at the second moment, is the mileage information of the vehicle at the first moment.

[0076] In some embodiments, motion compensation for the second predicted value according to the displacement of the vehicle on the horizontal plane can be performed, for example, by formula (9) described below:

[0077]

[0078] Among them, is the second predicted value after motion compensation, is the second predicted value before motion compensation (i.e., the second predicted value determined according to the state information of the target object at the second moment), is the displacement of the vehicle on the horizontal plane.

[0079] Also, considering that the change in the spatial expansion attribute of the target object at the first moment is mainly related to the rotation angle of the vehicle relative to the horizontal plane, that is, the rotation angle of the vehicle in the vertical plane. In view of this, in some embodiments, the first predicted value may include, for example, an angle predicted value for predicting the rotation angle of the target object relative to the horizontal plane at the first moment. Motion compensation for the first predicted value according to the displacement of the vehicle in the vertical plane includes, for example: performing motion compensation for the angle predicted value according to the rotation angle of the vehicle in the vertical plane.

[0080] Among them, motion compensation for the angle predicted value according to the rotation angle of the vehicle in the vertical plane can be performed, for example, by formula (10) described below:

[0081] θ′ k = θ k - δ k,k-1 (10)

[0082] Among them, θ′ k is the angle predicted value after motion compensation, θ k is the angle predicted value before motion compensation, δ k,k-1 is the rotation angle of the vehicle in the vertical plane from the second moment to the first moment.

[0083] In some embodiments, determining the true value of the feature information at the first moment based on the predicted value after motion compensation, the second measurement value, and the first measurement value may include, for example: constructing a joint optimization function for the true value based on the predicted value after motion compensation and the first confidence level of the predicted value, the first measurement value and the second confidence level of the first measurement value, and the second measurement value and the third confidence level of the second measurement value, and determining the true value of the feature information at the first moment according to the joint optimization function.

[0084] Among them, the joint optimization function can be used to, but is not limited to, determining at least one estimated value of the true value under different hyperparameter combinations; the hyperparameter combinations are related to the first confidence level, the second confidence level, and the third confidence level. The hyperparameter combination may be, for example, a weight combination of the predicted value, the first measurement value, and the second measurement value determined according to the first confidence level, the second confidence level, and the third confidence level, and is used to adjust the contribution degrees of the predicted value, the first measurement value, and the second measurement value in the joint optimization function.

[0085] Exemplarily, determining the true value of the feature information at the first moment according to the joint optimization function may include, for example: determining the minimum value of the joint optimization function under the specified hyperparameter combination and given constraints as the true value at the first moment. Among them, the specified hyperparameter combination may be, for example, the weights assigned to the predicted value, the first measurement value, and the second measurement value according to the first confidence level of the predicted value at the first moment, the second confidence level of the first measurement value, and the third confidence level of the second measurement value. The criteria for weight assignment may refer to the following principles, for example: the sum of the weights is equal to 1, and the magnitude of the weight is positively correlated with the level of confidence.

[0086] Exemplarily, the joint optimization function may include a first optimization term, a second optimization term, and a third optimization term. Constructing a joint optimization function for the true value based on the predicted value after motion compensation and the first confidence level of the predicted value, the first measurement value and the second confidence level of the first measurement value, and the second measurement value and the third confidence level of the second measurement value may include, for example: constructing a first optimization term according to the predicted value after motion compensation; constructing a second optimization term according to the first measurement value; constructing a third optimization term according to the second measurement value; and constructing a joint optimization function according to the first confidence level and the first optimization term, the second confidence level and the second optimization term, and the third confidence level and the third optimization term.

[0087] In some embodiments, the first optimization term may be constructed, for example, by the following formula (11):

[0088]

[0089] Among them, is the predicted value of the feature information of the target object at the first moment after motion compensation, X is the true value of the feature information of the target object at the first moment, and f1 is the deviation between

[0090] As described above, the feature information of the target object at the first moment may include the shape, size, and position of the projection on the horizontal plane. In view of this, in order to facilitate tracking of the target object at different moments and reduce the amount of calculation, for example, the minimum bounding rectangle of the projection of the target object on the horizontal plane may be used to indicate the shape, size, and position of the projection of the target object on the horizontal plane.

[0091] In some embodiments, the feature information of the target object at the first moment may, for example, include the feature information of the four vertices of the minimum bounding rectangle of the projection of the target object on the horizontal plane at the first moment. For example, reference may be made to formula (12) shown below:

[0092] X = [X FL , X FR , X RL , X RR (12)

[0093] where X is the feature information of the target object at the first moment, X FL is the feature information of the lower left vertex of the minimum bounding rectangle of the horizontal plane projection of the target object at the first moment, X FR is the feature information of the upper left vertex of the minimum bounding rectangle of the horizontal plane projection of the target object at the first moment, X RL is the feature information of the lower right vertex of the minimum bounding rectangle of the horizontal plane projection of the target object at the first moment, X RR is the feature information of the upper right vertex of the minimum bounding rectangle of the horizontal plane projection of the target object at the first moment.

[0094] Similarly, the predicted value of the feature information of the target object at the first moment after motion compensation may also include the predicted values of the four vertices of the minimum bounding rectangle of the projection of the target object on the horizontal plane. For example, reference may be made to formula (13) shown below:

[0095]

[0096] where is the predicted value of the feature information of the target object at the first moment after motion compensation, is the predicted value of the lower left vertex of the minimum bounding rectangle of the horizontal plane projection of the target object after motion compensation at the first moment; is the predicted value of the upper left vertex of the minimum bounding rectangle of the horizontal plane projection of the target object after motion compensation at the first moment; is the predicted value of the lower right vertex of the minimum bounding rectangle of the horizontal plane projection of the target object after motion compensation at the first moment; is the predicted value of the upper right vertex of the minimum bounding rectangle of the horizontal plane projection of the target object after motion compensation at the first moment.

[0097] When the predicted value of the feature information of the target object at the first moment after motion compensation includes the predicted values of the four vertices of the minimum circumscribed rectangle of the horizontal plane projection of the target object, and the feature information of the target object at the first moment includes the feature information of the four vertices of the minimum circumscribed rectangle of the horizontal plane projection of the target object at the first moment, the expression form of the first optimization term can, for example, refer to formula (14) shown below:

[0098]

[0099] Among them, the meaning of each term in formula (14) can refer to formula (12) and formula (13), and the same content will not be repeated here.

[0100] In some embodiments, the second optimization term can be constructed, for example, by formula (15) shown below:

[0101] f2 = 1 / 2{|Y BEV，k -X| 2} (15)

[0102] Among them, Y BEV，k is the first measurement value, x is the true value of the feature information of the target object at the first moment, and F2 is the deviation between Y BEV，k and x.

[0103] The first measurement value can, similarly, also include the predicted values of the four vertices of the minimum circumscribed rectangle of the projection of the target object on the horizontal plane. For example, it can refer to formula (16) shown below:

[0104] Y BEV，k =[Y BEV，k FL , Y BEV，k FR , Y BEV，k RL , Y BEV，k RR (16)

[0105] Among them, Y BEV，k is the first measurement value, Y BEV，k FL is the first measurement value of the lower left vertex of the minimum circumscribed rectangle of the projection of the target object on the horizontal plane, Y BEV，k FR is the first measurement value of the upper left vertex of the minimum circumscribed rectangle of the projection of the target object on the horizontal plane, Y BEV，k RL is the first measurement value of the minimum circumscribed rectangle of the projection of the target object on the horizontal plane, Y BEV，k RRThe first measurement value of the upper right vertex of the minimum circumscribed rectangle of the horizontal plane projection of the target object.

[0106] When the first measurement value includes the first measurement values of the four vertices of the minimum circumscribed rectangle of the horizontal plane projection of the target object, and the feature information of the target object at the first moment includes the feature information of the four vertices of the minimum circumscribed rectangle of the horizontal plane projection of the target object at the first moment, the expression form of the first optimization term can, for example, refer to formula (17) shown below:

[0107] f2 = 1 / 2{|Y BEV，k FL -X FL | 2 +|Y BEV，k FR -X FR | 2 +|Y BEV，k RL -X RL | 2 +|Y BEV，k RR -X RR | 2} (17)

[0108] Among them, the meaning of each term in formula (17) can refer to formula (12) and formula (16), and the same content will not be repeated here.

[0109] In some embodiments, the third optimization term can be constructed, for example, by formula (18) shown below:

[0110] f3 = 1 / 2{|Y 2D，k -X| 2} (18)

[0111] Among them, Y 2D，k is the second measurement value, X is the true value of the feature information of the target object at the first moment, and f3 is the deviation between Y 2D，k and X.

[0112] As described above, the second measurement value can be expressed as a perceptual representation of the feature information of the target object at the first moment in the image coordinate system. In view of this, for the convenience of subsequent data processing, the second measurement value can be transformed into the vehicle coordinate system.

[0113] In some embodiments, the transformation of the second measurement value into the vehicle coordinate system can be performed, for example, by formula (19) shown below:

[0114]

[0115] Wherein, Z is the depth of the pixel, and u and v are the abscissa and ordinate of the second measurement data in the pixel coordinate system respectively, and K, is the internal parameter matrix, external parameter matrix and permutation matrix for visual sensor calibration. x1, y1, and z1 are the x-axis coordinate, y-axis coordinate, and z-axis coordinate of the second vehicle data after being transformed into the vehicle coordinate system respectively.

[0116] Also, considering that problems such as distortion and deformation may occur during the process of transforming the coordinate system of the image, in order to ensure the accuracy of the data, the points in the image that are beneficial for tracking the target can be marked as key measurement points, and the target can be tracked according to the second measurement values of the key measurement points of the target object from the perspective of image perception. For example, when the target object is a target vehicle, the key measurement points can include the wheel grounding points of the target vehicle, which is not only beneficial for tracking the target, but also the height of the wheel grounding points is 0, which can reduce the variables in the coordinate system transformation and improve the data processing speed and accuracy.

[0117] In some embodiments, the second measurement value can include, for example, the key measurement value of the key measurement point of the target object and the bounding box measurement value of the bounding box of the target object. The true value of the feature information at the first moment is determined according to the state information, prediction value, second measurement value and first measurement value. For example, it can include: if the key measurement value meets the preset condition, the true value of the feature information at the first moment is determined according to the state information, prediction value, key measurement value and first measurement value; or, if the key measurement value does not meet the preset condition, the true value of the feature information at the first moment is determined according to the state information, prediction value, first measurement value and bounding box measurement value. In the embodiments of the present application, for different situations where the key measurement value meets or does not meet the preset condition, different methods for determining the true value of the feature information of the target object at the first moment are provided. On the one hand, it can improve the data processing efficiency and accuracy when the key measurement value meets the preset condition, and on the other hand, it can ensure the executability when the key measurement value does not meet the preset condition, greatly improving the flexibility of the method.

[0118] Wherein, the key measurement value can include, for example, visibility information indicating the clarity of the key measurement point, and the preset condition can include, for example: when there are multiple key measurement points, the number of visibility information exceeding the visibility threshold among the multiple visibility information corresponding to the multiple key measurement points reaches a predetermined number.

[0119] On the one hand, the technical solution provided by this application can perceive the characteristic information of the target object from data in different time dimensions through the first measurement values and predicted values of the characteristic information of the target object at different times before and after from the bird's-eye view perception perspective, significantly improving the accuracy when determining the characteristic information of the target object. On the other hand, it also combines the second measurement value of the characteristic information of the target object from the image perception perspective, improving the information richness of the characteristic information of the target object from the bird's-eye view perception perspective. It can not only accurately track the target based on the characteristic information of the target object, which is beneficial to improving the accuracy of target perception, but also more comprehensively perceive the characteristic information of the target object, which is beneficial to downstream task processing.

[0120] In some possible implementation manners, this application provides a device for determining the characteristic information of a target object. Figure 3 It is a schematic structural diagram of the device for determining the characteristic information of the target object provided by the embodiment of this application. Refer to Figure 3 The device for determining the characteristic information of the target object provided by the embodiment of this application includes a first acquisition module 310, a second acquisition module 320, and a determination module 330.

[0121] The first acquisition module 310 is configured to acquire first perception data of a target object around the vehicle at a first moment, where the first perception data is used to indicate a first measurement value of the characteristic information of the target object at the first moment from the bird's-eye view perception perspective.

[0122] The second acquisition module 320 is configured to acquire the state information of the vehicle, second perception data related to the characteristic information at the first moment, and a predicted value, where the second perception data is used to indicate a second measurement value of the characteristic information at the first moment from the image perception perspective, and the predicted value is determined according to a third measurement value of the characteristic information of the target object at a second moment from the bird's-eye view perception perspective, and the second moment is earlier than the first moment.

[0123] The determination module 330 is configured to determine the true value of the characteristic information at the first moment according to the state information, the predicted value, the second measurement value, and the first measurement value.

[0124] In some embodiments, the determination module 330 is configured to determine the displacement information of the vehicle between the first moment and the second moment according to the state information; perform motion compensation on the predicted value according to the displacement information; and determine the true value of the characteristic information at the first moment according to the predicted value after motion compensation, the second measurement value, and the first measurement value.

[0125] In some embodiments, the determining module 330 is configured to construct a joint optimization function of the true value based on the predicted value after motion compensation and the first confidence level of the predicted value, the first measurement value and the second confidence level of the first measurement value, the second measurement value and the third confidence level of the second measurement value. The joint optimization function is used to determine at least one estimated value of the true value under different hyperparameter combinations, and the hyperparameter combinations are related to the first confidence level, the second confidence level, and the third confidence level. The true value of the feature information at the first moment is determined according to the joint optimization function.

[0126] In some embodiments, the joint optimization function includes a first optimization term, a second optimization term, and a third optimization term. According to the predicted value after motion compensation and the first confidence level of the predicted value, the first measurement value and the second confidence level of the first measurement value, the determining module 330 is configured to construct the first optimization term based on the predicted value after motion compensation, and the first optimization term is used to indicate the difference between the predicted value and the true value. The second optimization term is constructed based on the first measurement value, and the second optimization term is used to indicate the difference between the first measurement value and the true value. The third optimization term is constructed based on the second measurement value, and the third optimization term is used to indicate the difference between the second measurement value and the true value. The joint optimization function is constructed based on the first confidence level and the first optimization term, the second confidence level and the second optimization term, and the third confidence level and the third optimization term.

[0127] In some embodiments, the determining module 330 is configured to determine the minimum value of the joint optimization function under the specified hyperparameter combination and the given constraints as the true value at the first moment.

[0128] In some embodiments, the second measurement value includes the key measurement value of the key measurement point of the target object and the bounding box measurement value of the bounding box of the target object. According to the state information, the predicted value, the determining module 330 is configured to, if the key measurement value meets the preset condition, determine the true value of the feature information at the first moment according to the state information, the predicted value, the key measurement value, and the first measurement value; or, if the key measurement value does not meet the preset condition, determine the true value of the feature information at the first moment according to the state information, the predicted value, the first measurement value, and the bounding box measurement value.

[0129] In some embodiments, the key measurement value includes visibility information indicating the clarity of the key measurement point, and the preset condition includes: when there are multiple key measurement points, the number of visibility information exceeding the visibility threshold among the multiple visibility information corresponding to the multiple key measurement points reaches a predetermined number.

[0130] In some embodiments, the target object includes a target vehicle, and the key measurement points include the wheel contact points of the target vehicle.

[0131] It should be understood that the apparatus for determining the feature information of the target object provided in the above embodiments and the method embodiments for determining the feature information of the target object belong to the same concept. For the specific implementation process, please refer to the method embodiments for determining the feature information of the target object.

[0132] In some other possible implementation manners, the present application further provides an electronic device for determining the feature information of a target object. Figure 4 FIG. is a schematic structural diagram of the electronic device for determining the feature information of the target object provided in the embodiments of the present application. Refer to Figure 4 The electronic device for determining the feature information of the target object provided in the embodiments of the present application includes:

[0133] A memory 410, on which at least one program instruction for determining the feature information of the target object is stored.

[0134] A processor 420. When the above program instruction is executed by the processor 420, the vehicle implements the method and the steps of its multiple embodiments described above in conjunction with Figure 2 According to different implementation manners, the processor 420 may be a CPU (central processing unit), a GPU (graphics processing unit), or one or more types of other general and / or special processors, including but not limited to a DSP (digital signal processor), an ASIC (application specific integrated circuit), an FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number thereof may be determined according to actual needs.

[0135] In still some other possible implementation manners, the present application further provides a computer program (product), where the computer program (product) includes computer programs / instructions. When the computer programs / instructions are executed by a processor, the vehicle implements the method and the steps of its multiple embodiments described above in conjunction with Figure 2 The steps described above.

[0136] In still some other possible implementation manners, the present application further provides a computer-readable storage medium, on which program instructions for determining the feature information of the target object are stored. When the program instructions are executed by one or more processors, the vehicle implements the method and the steps of its multiple embodiments described above in conjunction with Figure 2The steps of the described method and its multiple embodiments. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0137] It should be noted that the electronic device in this application can also be referred to as a display device. In addition, the information, data (including but not limited to image data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the ID information, driving information, etc. involved in this application are obtained under full authorization.

[0138] It should also be noted that the terms "first", "second", etc. (if any) in the description and claims of this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0139] The term "and / or" in the embodiments of this application is merely a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone.

[0140] The above description is only for the convenience of those skilled in the art to understand the technical solution of this application and does not limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A method for determining the characteristic information of a target object, characterized in that, The method includes: Obtaining first perception data of a target object around a vehicle at a first moment, where the first perception data is used to indicate a first measurement value of the feature information of the target object at the first moment from a bird's-eye view perception perspective; Obtaining state information of the vehicle, second perception data related to the feature information at the first moment, and a prediction value, where the second perception data is used to indicate a second measurement value of the feature information at the first moment from an image perception perspective, and the prediction value is determined according to a third measurement value of the feature information of the target object at a second moment from a bird's-eye view perception perspective, and the second moment is earlier than the first moment; Determining a true value of the feature information at the first moment according to the state information, the prediction value, the second measurement value, and the first measurement value.

2. The method according to claim 1, characterized in that, The determining the true value of the feature information at the first moment according to the state information, the prediction value, the second measurement value, and the first measurement value includes: Determining displacement information of the vehicle between the first moment and the second moment according to the state information; Performing motion compensation on the prediction value according to the displacement information; Determining the true value of the feature information at the first moment according to the prediction value after motion compensation, the second measurement value, and the first measurement value.

3. The method according to claim 2, wherein The determining the true value of the feature information at the first moment according to the prediction value after motion compensation, the second measurement value, and the first measurement value includes: Constructing a joint optimization function of the true value according to the prediction value after motion compensation and a first credibility of the prediction value, the first measurement value and a second credibility of the first measurement value, the second measurement value and a third credibility of the second measurement value, where the joint optimization function is used to determine at least one estimated value of the true value under different hyperparameter combinations, and the hyperparameter combinations are related to the first credibility, the second credibility, and the third credibility; Determining the true value of the feature information at the first moment according to the joint optimization function.

4. The method according to claim 3, wherein The joint optimization function includes a first optimization term, a second optimization term, and a third optimization term. The constructing the joint optimization function of the true value according to the prediction value after motion compensation and a first credibility of the prediction value, the first measurement value and a second credibility of the first measurement value, the second measurement value and a third credibility of the second measurement value includes: Constructing the first optimization term according to the prediction value after motion compensation, where the first optimization term is used to indicate the difference between the prediction value and the true value; Constructing the second optimization term according to the first measurement value, where the second optimization term is used to indicate the difference between the first measurement value and the true value; Constructing the third optimization term according to the second measurement value, where the third optimization term is used to indicate the difference between the second measurement value and the true value; Constructing the joint optimization function according to the first credibility and the first optimization term, the second credibility and the second optimization term, the third credibility and the third optimization term.

5. The method according to claim 3, wherein The determining the true value of the feature information at the first moment according to the joint optimization function includes: Determine the minimum value of the combined optimization function under the specified hyperparameter combination and given constraints as the true value at the first moment.

6. The method according to any one of claims 1-5, characterized in that, The second measurement value includes the key measurement values of the key measurement points of the target object and the bounding box measurement values of the bounding box of the target object. Determining the true value of the feature information at the first moment according to the state information, the prediction value, the second measurement value, and the first measurement value includes: If the key measurement value meets the preset conditions, determine the true value of the feature information at the first moment according to the state information, the prediction value, the key measurement value, and the first measurement value; or, If the key measurement value does not meet the preset conditions, determine the true value of the feature information at the first moment according to the state information, the prediction value, the first measurement value, and the bounding box measurement value.

7. The method according to claim 6, characterized in that The key measurement value includes visibility information for indicating the clarity of the key measurement point, and the preset conditions include: When there are multiple key measurement points, among the multiple visibility information corresponding to the multiple key measurement points, the number of visibility information exceeding the visibility threshold reaches a predetermined number.

8. The method according to claim 6, wherein The target object includes a target vehicle, and the key measurement point includes the wheel contact point of the target vehicle.

9. An electronic device, characterized in that, Comprising: A memory storing program instructions for determining the feature information of a target object; And A processor, when the program instructions are executed by the processor, enabling the vehicle to implement the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Storing program instructions for determining the feature information of a target object, and when the program instructions are executed by one or more processors, enabling the vehicle to implement the method according to any one of claims 1-8.

Images