Method and apparatus for determining the orientation of a target object

Abstract

This specification provides a method and apparatus for determining the orientation of a target object. The method includes receiving an (i+1)th video frame containing a target object and acquiring the shape, key points, and edge detection boxes of the target object in the (i+1)th video frame; determining a target attribute value of the target object based on the shape and key points; determining a first orientation of the target object in the (i+1)th video frame based on the target attribute value and the edge detection boxes; determining a second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame; and determining the target orientation of the target object in the (i+1)th video frame based on the first orientation and the second orientation.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method for determining the orientation of a target object. One or more embodiments of this specification also relate to a target object orientation determination device, a computing device, and a computer-readable storage medium. Background Technology

[0002] As autonomous driving matures and seeks practical application and mass production, there are higher requirements for the stability, comprehensiveness, and accuracy of vehicle obstacle perception and structured output. In the field of autonomous driving, vehicle orientation calculation is a fundamental task of a perception system and part of vehicle pose estimation. Subsequent trajectory prediction and planning control also rely on orientation calculation. Current autonomous driving solutions mainly rely on LiDAR for vehicle orientation calculation, but in scenarios without LiDAR or in LiDAR blind spots, the vehicle's orientation cannot be accurately identified.

[0003] Therefore, there is an urgent need to provide a target object orientation determination method that can improve the accuracy and stability of vehicle orientation recognition. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method for determining the orientation of a target object. One or more embodiments of this specification also relate to a target object orientation determination apparatus, a computing device, and a computer-readable storage medium, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a method for determining the orientation of a target object is provided, comprising:

[0006] Receive the (i+1)th video frame containing the target object, and obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame;

[0007] The target attribute values ​​of the target object are determined based on the shape and key points of the target object.

[0008] The first orientation of the target object in the (i+1)th video frame is determined based on the target attribute value of the target object and the edge detection box;

[0009] The second orientation of the target object in the (i+1)th video frame is determined based on the target orientation of the target object in the i-th video frame, and the target orientation of the target object in the (i+1)th video frame is determined based on the first orientation and the second orientation.

[0010] According to a second aspect of the embodiments of this specification, a target object orientation determining device is provided, comprising:

[0011] The first video receiving module is configured to receive the (i+1)th video frame containing the target object, and to obtain the shape, key points and edge detection box of the target object in the (i+1)th video frame.

[0012] The first determining module is configured to determine the target attribute value of the target object based on the shape and key points of the target object;

[0013] The second determining module is configured to determine the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box;

[0014] The first target orientation determination module is configured to determine the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame, and to determine the target orientation of the target object in the (i+1)th video frame based on the first orientation and the second orientation.

[0015] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0016] Memory and processor;

[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the target object orientation determination method.

[0018] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the target object orientation determination method.

[0019] This specification provides an embodiment of a method and apparatus for determining the orientation of a target object. The method includes receiving an (i+1)th video frame containing a target object and acquiring the shape, key points, and edge detection boxes of the target object in the (i+1)th video frame; determining target attribute values ​​of the target object based on the shape and key points; determining a first orientation of the target object in the (i+1)th video frame based on the target attribute values ​​and edge detection boxes; determining a second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame; and determining the target orientation of the target object in the (i+1)th video frame based on the first and second orientations. Specifically, the method decomposes the calculation of the target orientation into multiple steps, such as shape estimation, target attribute value calculation, and orientation calculation, allowing for good decoupling and fusion between these steps, ultimately resulting in an accurate and stable target orientation of the target object. Attached Figure Description

[0020] Figure 1 This is an example diagram illustrating a specific application scenario of a target object orientation determination method provided in one embodiment of this specification;

[0021] Figure 2 This is a flowchart illustrating a method for determining the orientation of a target object according to one embodiment of this specification;

[0022] Figure 3 This is a schematic diagram of a target object orientation determination method including a video frame of the target object, provided in one embodiment of this specification;

[0023] Figure 4 This is a schematic diagram of the projection relationship of the target object in the world coordinate system in a target object orientation determination method provided in one embodiment of this specification;

[0024] Figure 5 This is a flowchart illustrating the application of a target object orientation determination method provided in one embodiment of this specification in autonomous driving of vehicles;

[0025] Figure 6 This is a schematic diagram of the structure of a target object orientation determining device according to one embodiment of this specification;

[0026] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0027] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0028] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0030] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0031] Projective geometry: distinct from Euclidean geometry, it is typically used for transformations and calculations between the real world and the image plane.

[0032] Long-tail problem: Due to the limited richness of samples and the performance of models, the field of autonomous driving often faces many corner cases that cause existing systems to make mistakes.

[0033] Interpretability: End-to-end models are typically black boxes, which face the problem of weak interpretability.

[0034] Edge computing: requires real-time computing to be performed on the edge, usually emphasizing high real-time performance, which is different from cloud computing.

[0035] Camera extrinsics are parameters in the world coordinate system, such as the camera's position and rotation direction.

[0036] Kalman filtering: Kalman filtering is an algorithm that uses the state equations of a linear system and the system's input and output observation data to make an optimal estimate of the system state.

[0037] Single-vehicle model: also known as a kinematic autonomous vehicle model, which can predict the vehicle's orientation at the next moment given the current vehicle orientation.

[0038] In the field of autonomous driving, vehicle orientation calculation is a fundamental task of a perception system and part of vehicle pose estimation. Subsequent processes such as vehicle trajectory prediction and planning control also rely on orientation. This specification provides a vision-based method for determining the orientation of a target object, enabling more accurate calculation of vehicle orientation. For applications with LiDAR-equipped vehicles, combining this method with high-precision vision-based orientation calculation can significantly improve the accuracy and stability of orientation calculation. For vehicle applications without LiDAR or in radar blind spots, the vision solution provided in this specification can independently handle orientation calculation and output, making the target object orientation method applicable to various application scenarios and enhancing user experience.

[0039] Based on this, this specification provides a method for determining the orientation of a target object. One or more embodiments of this specification also relate to a target object orientation determination device, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0040] See Figure 1 , Figure 1 The illustration shows an example scenario of a target object orientation determination method provided in one embodiment of this specification.

[0041] Figure 1 The application scenario includes an image acquisition terminal 102, an image receiving terminal 104, and a server 106. Specifically, the image receiving terminal 104 receives an image 'a' containing a vehicle, which is acquired in real time by the image acquisition terminal 102. After receiving image 'a', the image receiving terminal 104 sends image 'a' to the server 106. After receiving image 'a', the server 106 inputs image 'a' into a vehicle model detection model to obtain the vehicle model in image 'a'; inputs image 'a' into a key point detection model to obtain the key points of the vehicle in image 'a'; and inputs image 'a' into a full vehicle detection model to obtain the full vehicle detection bounding box of the vehicle in image 'a'. The vehicle model detection model, key point detection model, and full vehicle detection model can be understood as deep learning models trained using convolutional neural networks.

[0042] In the specific calculation, firstly, the vehicle type in image a is used to determine the range of the vehicle's length and width, i.e., the vehicle's length and width are 1. For example, if the vehicle type is SUV, the length of a compact SUV is generally around 4.4m to 4.65m and the width is around 1.8m, while the length of a large SUV is around 4.7m to 5m and the width is around 1.9m. Therefore, based on the vehicle type, the vehicle's length range is 4.4m to 5m and the vehicle's width range is 1.8m to 1.9m.

[0043] Then, using the key points of the vehicle in image a and the camera extrinsic parameters of image a, the true length and width of the vehicle in image a are calculated, i.e., the vehicle's length and width are 2; where the key points are... Figure 1 Points E, F, D, and G in image a. If image a is the first frame containing the vehicle, then if the vehicle's length and width 2 are within the range of the vehicle's length and width 1, then the vehicle's length and width 2 are taken as the target length and width of the vehicle in image a, i.e. Figure 1 The vehicle's length and width in the image are 3. If image a is a second, third, or fourth frame image following the first frame image containing the vehicle, then the predicted length and width of the vehicle in the current image a are predicted using the actual length and width of the vehicle in the previous frame image. Then, the actual length and width of the vehicle in image a obtained through the vehicle's key points and camera extrinsic parameters, along with the predicted length and width of the vehicle in image a predicted from the vehicle's length and width in the previous frame image, are input into a Kalman filter for fusion and correction to obtain the target length and width of the vehicle in image a. In practical applications, the camera extrinsic parameters corresponding to each frame image change during vehicle movement. Therefore, when calculating the target length and width of the vehicle in an image, it is necessary to iteratively calculate the vehicle's length and width based on the camera extrinsic parameters to obtain the accurate length and width of the vehicle in each frame image, ensuring the accuracy of subsequent calculations of the vehicle's orientation.

[0044] After obtaining the target length and width of the vehicle in image a (i.e., length and width 3), projective geometry is used to project the full vehicle detection box of image a onto a preset coordinate system to obtain the mapping key points of the full vehicle detection box in image a in the preset coordinate system. The preset coordinate system is the world coordinate system of the camera. After determining the target length and width of the vehicle in image a and the mapping key points, the vehicle orientation of image a can be calculated based on the coordinate values ​​of the target length and width of the vehicle in image a and the mapping key points in the world coordinate system. In practical applications, if image a is the first frame image containing the vehicle, then the vehicle orientation is the target vehicle orientation of image a. If there is a previous frame image of image a containing the vehicle, the predicted vehicle orientation of image a can be obtained through a single-vehicle model. Then, the predicted vehicle orientation of image a and the vehicle orientation calculated from the vehicle's length and width and the mapping key points are input into a Kalman filter for processing to obtain the target vehicle orientation of image a.

[0045] The target object orientation determination method provided in the embodiments of this specification is applied to vehicle orientation calculation. It decomposes the vehicle orientation calculation into multiple parts such as vehicle model estimation, length and width calculation, and Kalman filtering. Through decoupling and fusion of different implementation methods, a more accurate and stable vehicle orientation can be obtained. Furthermore, the overall vehicle orientation calculation model only depends on key points, and the computational load itself is small. Therefore, using the target object orientation determination method provided in the embodiments of this specification to calculate the vehicle orientation has significant advantages in terms of accuracy, stability, applicability, and real-time performance.

[0046] See Figure 2 , Figure 2 A flowchart of a method for determining the orientation of a target object according to an embodiment of this specification is shown, which specifically includes the following steps.

[0047] Step 202: Receive the (i+1)th video frame containing the target object, and obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame.

[0048] The target objects include, but are not limited to, two-wheeled vehicles, three-wheeled vehicles, four-wheeled vehicles, or other multi-wheeled vehicles, or logistics vehicles, public service vehicles, medical service vehicles, terminal service vehicles, etc.; in addition, i is a positive integer, for example, if i is 1, then i+1 is 2. In practical applications, the target object orientation determination method provided in the embodiments of this specification can predict the orientation of stationary vehicles or vehicles in motion. For ease of understanding, this specification uses a four-wheeled car in motion as an example to describe this solution in detail.

[0049] Specifically, receiving the (i+1)th video frame containing the target object can be understood as receiving the (i+1)th video frame containing the moving vehicle acquired by the camera, and obtaining the shape, key points, and edge detection box of the target object in the (i+1)th video frame.

[0050] Taking i as 1 as an example, the (i+1)th video frame is the second video frame containing the moving vehicle. After obtaining the second video frame, the shape, key points, and edge detection boxes of the vehicle in the second video frame are obtained. The shape of the vehicle can be understood as the vehicle model, and the edge detection box of the vehicle can be understood as the full vehicle detection box.

[0051] In specific implementation, obtaining the shape, key points, and edge detection box of the target object in the (i+1)th video frame includes:

[0052] The (i+1)th video frame is input into the first recognition model, the second recognition model, and the third recognition model respectively to obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame.

[0053] The first recognition model, the second recognition model, and the third recognition model can be understood as deep learning models trained using convolutional neural networks.

[0054] Using the previous example, inputting the second video frame into the first recognition model yields the vehicle model in the second video frame; inputting the second video frame into the second recognition model yields the key points of the vehicle in the second video frame; inputting the second video frame into the third recognition model yields the full vehicle detection bounding box of the vehicle in the second video frame; the full vehicle detection bounding box can display the range of the vehicle's 3D detection and the vehicle's edge information; the vehicle's key points can display wheel points and headlight points, which have clear physical meaning and texture, and the vehicle's key point information also includes the information of the vehicle's side bounding box.

[0055] In the embodiments of this specification, different recognition models are used to obtain the shape, key points, and edge detection boxes of the target object. For each feature of the target object, a targeted model is used to accurately obtain it, and then they are fused and calculated to obtain the accurate orientation of the target object. This is faster and more accurate than the original large model that directly obtains the orientation of the target object. For example, a key point model is used to obtain the key points of the target object. This key point model has a strong image texture analysis capability and has a great advantage over the original large model in terms of model stability and handling of long-tail problems.

[0056] See Figure 3 , Figure 3 A schematic diagram is shown of a target object orientation determination method comprising a video frame containing a target object, according to an embodiment of this specification.

[0057] Depend on Figure 3 As can be seen from this, the target object in this video frame is a vehicle. Figure 3 Video frames containing vehicles are input into three recognition models to obtain... Figure 3 The full-vehicle detection frame of the vehicle, that is, the rectangular edge detection frame composed of ABCD surrounding the vehicle, is obtained. Figure 3 Key points for vehicles: E, F, G, D, obtain Figure 3 The vehicle models in the text.

[0058] Specifically, before receiving the (i+1)th video frame containing the target object, the process further includes:

[0059] Receive the i-th video frame containing the target object, and obtain the shape, key points, and edge detection box of the target object in the i-th video frame;

[0060] The target attribute values ​​of the target object are determined based on the shape and key points of the target object.

[0061] The target orientation of the target object in the i-th video frame is determined based on the target attribute value of the target object and the edge detection box.

[0062] In this case, the target object in the i-th video frame is the same as the target object in the (i+1)-th video frame.

[0063] In practical applications, the target object orientation determination method is applied to the scenario of vehicle driving. For a moving vehicle, in order to obtain an accurate and stable vehicle orientation, the orientation of the vehicle in the current frame is predicted based on the vehicle orientation of the previous moment of the current frame. Then, the target orientation of the vehicle in the current frame is obtained based on the predicted vehicle orientation and the vehicle orientation calculated through the length, width and key points of the vehicle in the current frame.

[0064] Therefore, when determining the target orientation of a vehicle in the (i+1)th video frame, it is necessary to obtain the target orientation of the vehicle in the ith video frame.

[0065] Continuing with the previous example, if i is still 1, firstly, the first video frame containing the vehicle is received, and the shape, key points, and edge detection boxes of the vehicle in the first video frame are obtained; then, the target attribute value of the vehicle is determined based on the shape and key points of the vehicle; finally, the target orientation of the target object in the first video frame is determined based on the target attribute value of the vehicle and the edge detection boxes.

[0066] The method for obtaining the shape, key points, and edge detection box of the target object in the i-th video frame can be found in the detailed description of the shape, key points, and edge detection box of the target object in the (i+1)-th video frame in the above embodiments, and will not be repeated here.

[0067] In specific implementation, determining the target attribute value of the target object based on its shape and key points includes:

[0068] The first initial attribute value of the target object in the i-th video frame is determined based on the shape of the target object in the i-th video frame;

[0069] Based on the key points of the target object in the i-th video frame and the camera extrinsic parameters of the i-th video frame, determine the second initial attribute value of the target object in the i-th video frame;

[0070] If the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the i-th video frame.

[0071] The attribute value can be understood as the length and width. So, when the target object is a vehicle, the attribute value can be understood as the length and width of the vehicle.

[0072] Specifically, if the i-th video frame is the first video frame containing the target object, and there are no other video frames before the first video frame, then the second initial attribute value of the target object in the i-th video frame is calculated based on the key points of the target object in the i-th video frame and the camera extrinsic parameters of the i-th video frame, and this target attribute value must also be within the first initial attribute value.

[0073] Following the previous example, after obtaining the vehicle model, key points, and full vehicle detection box in the first video frame, the first initial length and width of the vehicle are determined based on the vehicle model, i.e., the length and width range of the vehicle; then, the second initial length and width of the vehicle in the first video frame are calculated based on the key points of the vehicle in the first video frame and the extrinsic parameters of the camera that acquired the video frame; finally, if the second initial length and width are less than or equal to the first initial length and width, the second initial length and width are used as the target length and width of the vehicle in the first video frame.

[0074] In practical applications, the maximum and minimum length and width of a vehicle can be obtained based on its model. These maximum and minimum length and width are then used as the range for the vehicle's length and width. After calculating the vehicle's true length and width based on key points and camera extrinsic parameters, if the true length and width fall within this range, the calculation is accurate and can be used as the target length and width for the vehicle in the first video frame. If the true length and width are not within this range, the calculation is incorrect.

[0075] In the embodiments of this specification, when the i-th video frame is the first video frame containing the target object, the first initial attribute value determined by the shape of the target object is used as a constraint condition, and the second initial attribute value of the vehicle in the i-th video frame calculated based on the key points of the target object and the camera extrinsic parameters of the i-th video frame is used as the target attribute value. Subsequently, the orientation of the target object in the first video frame can be quickly obtained.

[0076] Step 204: Determine the target attribute value of the target object based on the shape and key points of the target object.

[0077] Specifically, determining the target attribute value of the target object based on its shape and key points includes:

[0078] Based on the shape of the target object in the (i+1)th video frame, determine the first initial attribute value of the target object in the (i+1)th video frame;

[0079] Based on the key points of the target object in the (i+1)th video frame and the camera extrinsic parameters of the (i+1)th video frame, determine the second initial attribute value of the target object in the (i+1)th video frame;

[0080] If the second initial attribute value is less than or equal to the first initial attribute value, the third initial attribute value of the target object in the (i+1)th video frame is determined based on the target attribute value of the target object in the i-th video frame.

[0081] The target attribute value of the target object in the (i+1)th video frame is determined based on the second initial attribute value and the third initial attribute value.

[0082] Specifically, the calculation method for the target attribute value of the target object in the (i+1)th video frame is different from the calculation method for the target attribute value of the target object in the i-th video frame in the above embodiment.

[0083] First, the first initial attribute value of the target object in the (i+1)th video frame is determined based on the shape of the target object in the (i+1)th video frame. Then, based on the key points of the target object in the (i+1)th video frame and the camera extrinsic parameters of the target object in the (i+1)th video frame, the second initial attribute value of the target object in the (i+1)th video frame is obtained. The calculation of the first and second initial attribute values ​​of the target object in the (i+1)th video frame is the same as the calculation method for the first and second initial attribute values ​​of the target object in the ith video frame, and will not be repeated here.

[0084] Since the first initial attribute value of the target object in the (i+1)th video frame is a constraint on the second initial attribute value, after obtaining the first and second initial attribute values ​​of the target object in the (i+1)th video frame, it is necessary to determine whether the second initial attribute value of the target object in the (i+1)th video frame is within the first initial attribute value. If so, the third initial attribute value of the target object in the (i+1)th video frame can be determined based on the target attribute value of the target object in the ith video frame. In practical applications, the target attribute value of the target object in the ith video frame is input into a Kalman filter for prediction to obtain the third initial attribute value of the target object in the (i+1)th video frame.

[0085] Finally, the second and third initial attribute values ​​of the target object in the (i+1)th video frame are fused and corrected to obtain the target attribute value of the target object in the (i+1)th video frame.

[0086] The target object orientation determination method described in the embodiments of this specification is applied in a vehicle driving scenario. The camera will acquire images containing vehicles in real time and send them to the server. After receiving the images containing vehicles, the server will predict the length and width of the vehicle in the current video frame based on the length and width of the vehicle in the previous video frame. Then, the predicted length and width of the vehicle, along with the length and width of the vehicle obtained by the key points of the vehicle in the current frame and the camera extrinsic parameters of the current video frame, will be input into a Kalman filter for fusion correction to obtain an accurate and stable length and width of the vehicle in the current video frame.

[0087] Step 206: Determine the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box.

[0088] Specifically, determining the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box includes:

[0089] Map the edge detection box of the target object in the (i+1)th video frame in a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the (i+1)th video frame;

[0090] The first orientation of the target object in the (i+1)th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object.

[0091] The preset coordinate system can be understood as the camera's world coordinate system. In practical applications, when calculating the first orientation of the target object in the (i+1)th video frame, the edge detection box of the target object in the (i+1)th video frame is first mapped in the world coordinate system to obtain the mapping key points of the edge detection box of the target object in the (i+1)th video frame. Then, based on the coordinate values ​​of the mapping key points in the world coordinate system and the target attribute values ​​of the target object in the (i+1)th video frame, the first orientation of the target object in the (i+1)th video frame is calculated.

[0092] See Figure 4 , Figure 4 This diagram illustrates the projection relationship of a target object in the world coordinate system in a target object orientation determination method according to an embodiment of this specification.

[0093] Combination Figure 3 , Figure 3 The rectangle formed by ABCD in the diagram is the edge detection box for the vehicle. Figure 4 In the image, ABC represents the three visible points of the vehicle. Through camera imaging, the vehicle's ABC points are... Figure 4 The three points corresponding to the bottom horizontal line (i.e., the imaging plane) are A', B', and C', and according to... Figure 4 The XOY coordinate system can be used to obtain the specific pixel values ​​of A', B', and C' on the imaging plane, i.e., coordinate values. Based on the coordinate values ​​of A', B', and C' on the imaging plane, as well as AC (vehicle width) and BC (vehicle length), the direction of BC in the XOY coordinate system can be obtained. This direction is the orientation of the vehicle in the video frame.

[0094] In the embodiments of this specification, the orientation of the target object is calculated using geometric modeling. The principle design is not sensitive to visual imaging and is applicable to both pinhole and fisheye cameras. Therefore, it is not sensitive to modules with different parameters and has good mass production characteristics. Furthermore, by utilizing the constraint relationship between the length, width, and orientation of the target object, the orientation of the target object in the video frame can be calculated relatively accurately by using the coordinate values ​​of the mapping key points of the target object's edge detection box in the world coordinate system and the length and width of the target object.

[0095] Specifically, determining the first orientation of the target object in the (i+1)th video frame based on the coordinates of the mapped key points in the preset coordinate system and the target attribute values ​​of the target object includes:

[0096] The first target edge and the second target edge of the target object are determined based on the edge detection box;

[0097] The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object.

[0098] Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the (i+1)th video frame is calculated.

[0099] In this context, the first target edge is the long side of the target object, and the second target edge is the wide side of the target object. In practical applications, the edge detection box based on the target object can determine which side is the length of the target object and which side is the width of the target object. After determining the long and wide sides of the target object, the target attribute value of the target object is used to assign length information to the long side and width information to the wide side. Then, the orientation of the target object is obtained by calculating the coordinates of the mapping key points in the world coordinate system, the length of the long side of the target object, and the width of the wide side.

[0100] In the embodiments of this specification, by utilizing the constraint relationship between the length, width, and orientation of the target object, the orientation of the target object in the video frame can be calculated relatively accurately by using the coordinate values ​​of the mapping key points of the target object's edge detection box in the world coordinate system and the length and width of the target object.

[0101] Step 208: Determine the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame, and determine the target orientation of the target object in the (i+1)th video frame based on the first orientation and the second orientation.

[0102] Specifically, determining the target orientation of the target object in the i-th video frame based on the target attribute value of the target object and the edge detection box includes:

[0103] Map the edge detection box of the target object in the i-th video frame to a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the i-th video frame;

[0104] The first orientation of the target object in the i-th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object.

[0105] When i is 1, the first orientation of the target object in the i-th video frame is taken as the target orientation of the target object in the i-th video frame.

[0106] Specifically, if the i-th video frame is the first video frame containing the target object, the target orientation of the target object in the i-th video frame can be obtained directly by mapping the coordinates of the key points in the world coordinate system and calculating the length and width of the target object. If the i-th video frame is not the first video frame, the target orientation can be obtained by referring to the method for obtaining the target orientation of the target object in the (i+1)-th video frame.

[0107] In the embodiments of this specification, if the i-th video frame is the first video frame containing the target object or a still video frame, there are no constraints related to speed, distance, etc. In this case, the target orientation of the i-th video frame can be quickly obtained by directly mapping the coordinate values ​​of the key points in the world coordinate system and calculating the length and width of the target object.

[0108] The step of determining the first orientation of the target object in the i-th video frame based on the coordinate values ​​of the mapped key points in the preset coordinate system and the target attribute values ​​of the target object includes:

[0109] The first target edge and the second target edge of the target object are determined based on the edge detection box;

[0110] The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object.

[0111] Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the i-th video frame is calculated.

[0112] In the embodiments of this specification, when the i-th video frame is the first video frame or a still video frame, the first orientation of the target object in the i-th video frame can be calculated based on the mapping key points of the edge detection box of the target object in the i-th video frame, the length and width of the target object, and the constraint relationship between the length, width and orientation of the target object can be used to calculate a more accurate orientation of the target object.

[0113] When the video frame is not the first video frame and is not a still video frame, in order to obtain a stable and accurate target orientation, it is necessary to use a single-vehicle model and filters to better calculate and correct the target orientation. The specific implementation method is as follows:

[0114] The step of determining the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame includes:

[0115] Input the target orientation of the target object in the i-th video frame into the bicycle model to obtain the second orientation of the target object in the (i+1)-th video frame.

[0116] In practical applications, a single-vehicle model is pre-built, and the calculated orientation is used as the observation of the Kalman filter. In the single-vehicle model, the vehicle's orientation and velocity direction are consistent. Simultaneously, the temporal ranging difference provides velocity information. Therefore, for vehicles with high speeds, the velocity direction can be more trusted as the orientation, while for slow-moving or stationary vehicles, the orientation calculated by geometric constraints can be more trusted. For moving vehicles, the vehicle's speed can be obtained through temporal ranging or LiDAR. The target orientation of the object in the i-th video frame is input into the single-vehicle model. Through the constraint relationship between the target orientation and the vehicle's speed, the second orientation of the target object in the (i+1)-th video frame can be predicted.

[0117] The first and second orientations of the target object in the (i+1)th video frame are then input into the Kalman filter. The Kalman filter performs weighted fusion of the first and second orientations to obtain the target orientation of the target object in the (i+1)th video frame. In addition, the key point information of the target object in the (i+1)th video frame can also be added to the observations of the Kalman filter. In the orientation calculation, the key points have a positive effect on the orientation calculation.

[0118] In the embodiments of this specification, the target object orientation determination method proposes a new geometric modeling scheme, applicable to vehicles both near and far, as well as stationary and moving vehicles. It has no fundamental limitations and replaces the deep learning model in existing technologies with geometric modeling, offering advantages in terms of computational resources and time consumption, making it suitable for real-time autonomous driving applications. Furthermore, this scheme decomposes the end-to-end deep learning model into multiple sub-models and the new geometric model, increasing redundancy and offering advantages in stability and reducing long-tail problems, thus improving the safety of autonomous driving applications. The corresponding geometric modeling is designed to be insensitive to visual imaging, applicable to pinhole and fisheye cameras, and therefore insensitive to modules with different parameters (e.g., camera extrinsic parameters), exhibiting excellent mass production characteristics. Moreover, the physical modeling scheme provides strong interpretability.

[0119] The following is in conjunction with the appendix Figure 5 Taking the target object orientation determination method provided in this specification as an example of its application in autonomous driving of vehicles, the target object orientation determination method will be further explained, specifically including the following steps.

[0120] Step 502: Receive a video frame containing the vehicle, and obtain the full vehicle detection frame, key points, and vehicle model in the video frame.

[0121] Specifically, the full vehicle detection bounding box, key points, and vehicle model in the video frame are obtained using three deep learning models.

[0122] Step 504: Obtain the initial length and width of the vehicle in the video frame based on the vehicle model.

[0123] Step 506: Under the constraints of the initial length and width of the vehicle, determine the actual length and width of the vehicle based on the key points of the vehicle and the camera extrinsic parameters of the acquired vehicle video frames.

[0124] Step 508: Calculate the initial orientation of the vehicle in the video frame based on the constraints of the vehicle's length and width and the mapping key points of the vehicle's full-vehicle detection box in the world coordinate system.

[0125] Step 510: Build a motion model of the bicycle in advance, obtain the predicted orientation of the vehicle in the video frame, and use a Kalman filter to estimate the initial orientation and the predicted orientation.

[0126] Step 512: After weighted fusion of the initial orientation and the predicted orientation using a Kalman filter, a stable orientation of the vehicle is obtained.

[0127] The target object orientation determination method provided in the embodiments of this specification is applied to autonomous driving of vehicles. By modeling the constraint relationship between the length, width, and orientation of the vehicle, a stable and accurate orientation of the vehicle can be obtained. It is applicable to both stationary and moving vehicles, as well as vehicles at near and far distances, without any theoretical limitations, and has superior overall performance. Furthermore, since the model used is a keypoint model, it has strong image texture and has advantages in model stability and long-tail problems compared to the original large model. Secondly, by decomposing the orientation calculation into multiple parts such as vehicle model estimation, length and width calculation, and Kalman filtering, it can be decoupled and fused in a divide-and-conquer manner, ultimately achieving a robust effect. Finally, the overall vehicle orientation calculation model only depends on keypoints, and its computational load is relatively small. Therefore, the target object orientation determination method provided in the embodiments of this specification has superior performance in terms of accuracy, stability, applicability, and real-time performance.

[0128] Corresponding to the above method embodiments, this specification also provides embodiments of a target object orientation determination device. Figure 6 A schematic diagram of a target object orientation determining device according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:

[0129] The first video receiving module 602 is configured to receive the (i+1)th video frame containing the target object, and to obtain the shape, key points and edge detection box of the target object in the (i+1)th video frame.

[0130] The first determining module 604 is configured to determine the target attribute value of the target object based on the shape and key points of the target object;

[0131] The second determining module 606 is configured to determine the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box;

[0132] The first target orientation determination module 608 is configured to determine the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame, and to determine the target orientation of the target object in the (i+1)th video frame based on the first orientation and the second orientation.

[0133] Optionally, the device further includes:

[0134] The second video receiving module is configured to receive the i-th video frame containing the target object, and to obtain the shape, key points and edge detection box of the target object in the i-th video frame;

[0135] The third determining module is configured to determine the target attribute value of the target object based on the shape and key points of the target object;

[0136] The second target orientation determination module is configured to determine the target orientation of the target object in the i-th video frame based on the target attribute value of the target object and the edge detection box.

[0137] Optionally, the first video receiving module 602 is further configured to:

[0138] The (i+1)th video frame is input into the first recognition model, the second recognition model, and the third recognition model respectively to obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame.

[0139] Optionally, the third determining module is further configured to:

[0140] The first initial attribute value of the target object in the i-th video frame is determined based on the shape of the target object in the i-th video frame;

[0141] Based on the key points of the target object in the i-th video frame and the camera extrinsic parameters of the i-th video frame, determine the second initial attribute value of the target object in the i-th video frame;

[0142] If the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the i-th video frame.

[0143] Optionally, the first determining module 604 is further configured to:

[0144] Based on the shape of the target object in the (i+1)th video frame, determine the first initial attribute value of the target object in the (i+1)th video frame;

[0145] Based on the key points of the target object in the (i+1)th video frame and the camera extrinsic parameters of the (i+1)th video frame, determine the second initial attribute value of the target object in the (i+1)th video frame;

[0146] If the second initial attribute value is less than or equal to the first initial attribute value, the third initial attribute value of the target object in the (i+1)th video frame is determined based on the target attribute value of the target object in the i-th video frame.

[0147] The target attribute value of the target object in the (i+1)th video frame is determined based on the second initial attribute value and the third initial attribute value.

[0148] Optionally, the second determining module 606 is further configured to:

[0149] Map the edge detection box of the target object in the (i+1)th video frame in a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the (i+1)th video frame;

[0150] The first orientation of the target object in the (i+1)th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object.

[0151] Optionally, the second determining module 606 is further configured to:

[0152] The first target edge and the second target edge of the target object are determined based on the edge detection box;

[0153] The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object.

[0154] Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the (i+1)th video frame is calculated.

[0155] Optionally, the second target orientation determination module is further configured to:

[0156] Map the edge detection box of the target object in the i-th video frame to a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the i-th video frame;

[0157] The first orientation of the target object in the i-th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object.

[0158] When i is 1, the first orientation of the target object in the i-th video frame is taken as the target orientation of the target object in the i-th video frame.

[0159] Optionally, the second target orientation determination module is further configured to:

[0160] The first target edge and the second target edge of the target object are determined based on the edge detection box;

[0161] The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object.

[0162] Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the i-th video frame is calculated.

[0163] Optionally, the first target orientation determination module 608 is further configured to:

[0164] Input the target orientation of the target object in the i-th video frame into the bicycle model to obtain the second orientation of the target object in the (i+1)-th video frame.

[0165] One embodiment of this specification implements a target object orientation determination device. By decomposing the calculation of the target orientation into multiple steps such as shape estimation of the target object, calculation of target attribute values, and orientation calculation, the various implementation steps can be well decoupled and integrated, and finally, an accurate and stable target orientation of the target object can be obtained.

[0166] The above is a schematic scheme of a target object orientation determination device according to this embodiment. It should be noted that the technical solution of this target object orientation determination device and the technical solution of the target object orientation determination method described above belong to the same concept. For details not described in detail in the technical solution of the target object orientation determination device, please refer to the description of the technical solution of the target object orientation determination method described above.

[0167] The target object orientation determination device provided in the embodiments of this specification can be applied to autonomous driving scenarios. The orientation of the autonomous vehicle is calculated by various functional modules (such as the first determination module, the second determination module, and the first target orientation determination module) in the embodiments of the specification.

[0168] Of course, these algorithm modules (such as the functional modules mentioned above) will differ depending on the type of autonomous vehicle. For example, different algorithm modules will be involved for logistics vehicles, public service vehicles, medical service vehicles, and terminal service vehicles. The following examples illustrate the algorithm modules for these four types of autonomous vehicles:

[0169] Logistics vehicles refer to vehicles used in logistics scenarios, such as logistics vehicles with automatic sorting functions, logistics vehicles with refrigeration and insulation functions, and logistics vehicles with measurement functions. These logistics vehicles involve different algorithm modules.

[0170] For example, logistics vehicles can be equipped with automated sorting devices that can automatically retrieve, transport, sort, and store goods after the vehicle arrives at its destination. This involves an algorithm module for goods sorting, which mainly implements the logical control of goods retrieval, handling, sorting, and storage.

[0171] For example, in cold chain logistics scenarios, logistics vehicles can also be equipped with refrigeration and insulation devices. These devices can refrigerate or keep warm transported fruits, vegetables, aquatic products, frozen foods, and other perishable foods, maintaining them in a suitable temperature environment and solving the problem of long-distance transportation of perishable foods. This involves an algorithm module for refrigeration and insulation control. This module is mainly used to dynamically and adaptively calculate the appropriate temperature for refrigeration or insulation based on information such as the nature of the food (or item), its perishability, transportation time, current season, and climate. Based on this appropriate temperature, the refrigeration and insulation device is automatically adjusted. This eliminates the need for manual temperature adjustments by transport personnel when transporting different foods or items, freeing them from tedious temperature control and improving the efficiency of refrigerated and insulated transportation.

[0172] For example, in most logistics scenarios, charges are based on package volume and / or weight. However, the number of packages is enormous, and relying solely on couriers to measure their volume and / or weight is highly inefficient and labor-intensive. Therefore, some logistics vehicles are equipped with measuring devices that automatically measure the volume and / or weight of packages and calculate the cost. This involves an algorithm module for package measurement. This module primarily identifies the type of package, determines the measurement method (e.g., volume, weight, or a combination of both), and performs the volume and / or weight measurement based on the chosen method, as well as calculating the cost based on the measurement results.

[0173] Public service vehicles refer to vehicles that provide certain public services, such as fire trucks, de-icing trucks, water trucks, snowplows, garbage trucks, and traffic control vehicles. These public service vehicles involve different algorithm modules.

[0174] For example, the main task of an autonomous fire truck is to carry out reasonable fire extinguishing tasks at the fire scene. This involves an algorithm module for fire extinguishing tasks. This algorithm module needs to realize at least the logic of fire condition identification, fire extinguishing plan planning, and automatic control of fire extinguishing devices.

[0175] For example, the main task of a de-icing truck is to remove ice and snow from the road surface. This involves a de-icing algorithm module, which at least needs to identify the ice and snow conditions on the road surface, formulate a de-icing plan based on the ice and snow conditions, such as which road sections need de-icing, which road sections do not need de-icing, whether to use salting, the amount of salt to be applied, and the logic for automatic control of the de-icing device after determining the de-icing plan.

[0176] Among them, medical service vehicles refer to autonomous vehicles capable of providing one or more medical services. These vehicles can provide medical services such as disinfection, temperature measurement, medication dispensing, and isolation. This involves algorithm modules that provide various self-service medical services. These algorithm modules mainly realize the identification of disinfection needs and the control of disinfection devices so that the disinfection devices can disinfect patients; or the identification of patient positions and the control of temperature measuring devices to automatically place them close to the patient's forehead or other positions to measure the patient's temperature; or the identification of symptoms, the provision of prescriptions based on the diagnosis results, and the identification of medicines / medicine containers, as well as the control of the medication dispensing robot to make it grab medicines for patients according to the prescription, etc.

[0177] Among them, terminal service vehicles refer to self-service autonomous vehicles that can replace some terminal equipment to provide users with certain convenient services. For example, these vehicles can provide users with services such as printing, attendance, scanning, unlocking, payment, and retail.

[0178] For example, in some application scenarios, users often need to go to a specific location to print or scan documents, which is time-consuming and laborious. Therefore, a terminal service vehicle has emerged that can provide printing / scanning services to users. These service vehicles can interconnect with user terminal devices. Users issue print commands through their terminal devices, and the service vehicle responds by automatically printing the required documents and delivering them to the user's location. Users do not need to queue at the printer, greatly improving printing efficiency. Alternatively, it can respond to scanning commands issued by users through their terminal devices, move to the user's location, and the user places the document to be scanned on the service vehicle's scanning tool to complete the scan, eliminating the need to queue at the printer / scanner and saving time and effort. This involves an algorithm module that provides printing / scanning services. This algorithm module needs to at least recognize the interconnection with the user's terminal device, the response to print / scan commands, the user's location, and movement control.

[0179] For example, with the development of new retail businesses, more and more e-commerce companies are using vending machines to deliver goods to office buildings and public areas. However, these vending machines are placed in fixed locations and cannot be moved. Users need to go to the vending machine to purchase the goods they need, which is inconvenient. Therefore, self-driving vehicles that provide retail services have emerged. These vehicles can carry goods automatically and provide corresponding self-service shopping apps or shopping portals. Users can place orders with the self-driving vehicles using their mobile phones or other devices through the apps or shopping portals. The order includes the name and quantity of the goods to be purchased and the user's location. After receiving the order request, the vehicle can determine whether the remaining goods are available and whether the quantity is sufficient. If the goods are available and the quantity is sufficient, the vehicle can automatically move to the user's location and provide the goods, further improving the convenience of shopping, saving users time, and allowing them to focus on more important things. This involves the algorithm modules for providing retail services. These algorithm modules mainly implement logic such as responding to user order requests, order processing, product information maintenance, user location positioning, and payment management.

[0180] See Figure 7 , Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0181] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0182] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0183] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.

[0184] The processor 720 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the target object orientation determination method.

[0185] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the target object orientation determination method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the target object orientation determination method described above.

[0186] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the target object orientation determination method.

[0187] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the target object orientation determination method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the target object orientation determination method described above.

[0188] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0189] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0190] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0191] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0192] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for determining the orientation of a target object, comprising: Receive the (i+1)th video frame containing the target object, and obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame; The target attribute values ​​of the target object are determined based on the shape and key points of the target object. The first orientation of the target object in the (i+1)th video frame is determined based on the target attribute value of the target object and the edge detection box; The second orientation of the target object in the (i+1)th video frame is determined based on the target orientation of the target object in the i-th video frame, and the target orientation of the target object in the (i+1)th video frame is determined based on the first orientation and the second orientation. The determination of the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame includes: inputting the target orientation of the target object in the i-th video frame into a single-vehicle model, and predicting the second orientation of the target object in the (i+1)th video frame through the constraint relationship between the target orientation and the vehicle's movement speed.

2. The target object orientation determination method according to claim 1, further comprising, before receiving the (i+1)th video frame containing the target object: Receive the i-th video frame containing the target object, and obtain the shape, key points, and edge detection box of the target object in the i-th video frame; The target attribute values ​​of the target object are determined based on the shape and key points of the target object. The target orientation of the target object in the i-th video frame is determined based on the target attribute value of the target object and the edge detection box.

3. The target object orientation determination method according to claim 1 or 2, wherein obtaining the shape, key points, and edge detection box of the target object in the (i+1)th video frame includes: The (i+1)th video frame is input into the first recognition model, the second recognition model, and the third recognition model respectively to obtain the shape, key points, and edge detection box of the target object in the (i+1)th video frame.

4. The method for determining the orientation of a target object according to claim 2, wherein determining the target attribute value of the target object based on the shape and key points of the target object includes: The first initial attribute value of the target object in the i-th video frame is determined based on the shape of the target object in the i-th video frame; Based on the key points of the target object in the i-th video frame and the camera extrinsic parameters of the i-th video frame, determine the second initial attribute value of the target object in the i-th video frame; If the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the i-th video frame.

5. The target object orientation determination method according to claim 4, wherein determining the target attribute value of the target object based on the shape and key points of the target object includes: Based on the shape of the target object in the (i+1)th video frame, determine the first initial attribute value of the target object in the (i+1)th video frame; Based on the key points of the target object in the (i+1)th video frame and the camera extrinsic parameters of the (i+1)th video frame, determine the second initial attribute value of the target object in the (i+1)th video frame; If the second initial attribute value is less than or equal to the first initial attribute value, the third initial attribute value of the target object in the (i+1)th video frame is determined based on the target attribute value of the target object in the i-th video frame. The target attribute value of the target object in the (i+1)th video frame is determined based on the second initial attribute value and the third initial attribute value.

6. The target object orientation determination method according to claim 1, wherein determining the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box comprises: Map the edge detection box of the target object in the (i+1)th video frame in a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the (i+1)th video frame; The first orientation of the target object in the (i+1)th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object.

7. The target object orientation determination method according to claim 6, wherein determining the first orientation of the target object in the (i+1)th video frame based on the coordinate values ​​of the mapped key points in the preset coordinate system and the target attribute values ​​of the target object includes: The first target edge and the second target edge of the target object are determined based on the edge detection box; The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object. Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the (i+1)th video frame is calculated.

8. The target object orientation determination method according to claim 2, wherein determining the target orientation of the target object in the i-th video frame based on the target attribute value of the target object and the edge detection box comprises: Map the edge detection box of the target object in the i-th video frame to a preset coordinate system to obtain the mapping key points of the edge detection box of the target object in the i-th video frame; The first orientation of the target object in the i-th video frame is determined based on the coordinate values ​​of the mapping key points in the preset coordinate system and the target attribute values ​​of the target object. When i is 1, the first orientation of the target object in the i-th video frame is taken as the target orientation of the target object in the i-th video frame.

9. The target object orientation determination method according to claim 8, wherein determining the first orientation of the target object in the i-th video frame based on the coordinate values ​​of the mapped key points in the preset coordinate system and the target attribute values ​​of the target object comprises: The first target edge and the second target edge of the target object are determined based on the edge detection box; The edge values ​​of the first target edge and the second target edge are determined based on the target attribute values ​​of the target object. Based on the coordinates of the mapping key points in the preset coordinate system, the edge values ​​of the first target edge and the second target edge, the first orientation of the target object in the i-th video frame is calculated.

10. A target object orientation determining device, comprising: The first video receiving module is configured to receive the (i+1)th video frame containing the target object, and to obtain the shape, key points and edge detection box of the target object in the (i+1)th video frame. The first determining module is configured to determine the target attribute value of the target object based on the shape and key points of the target object; The second determining module is configured to determine the first orientation of the target object in the (i+1)th video frame based on the target attribute value of the target object and the edge detection box; The first target orientation determination module is configured to determine the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame, and to determine the target orientation of the target object in the (i+1)th video frame based on the first orientation and the second orientation. The determination of the second orientation of the target object in the (i+1)th video frame based on the target orientation of the target object in the i-th video frame includes: inputting the target orientation of the target object in the i-th video frame into a single-vehicle model, and predicting the second orientation of the target object in the (i+1)th video frame through the constraint relationship between the target orientation and the vehicle's speed.

11. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the target object orientation determination method according to any one of claims 1-9.

12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the target object orientation determination method according to any one of claims 1-9.

Images