Method, device, electronic device and storage medium for determining vehicle orientation

Abstract

The present disclosure provides a method, device, electronic device, and storage medium for determining a vehicle's heading, wherein the method comprises: acquiring a captured image; detecting the image to determine the position information of each wheel contact point of a target vehicle in the image, and the contact point type corresponding to each wheel contact point; the contact point type is used to indicate the visibility and / or obstruction of the wheel contact point; and determining the vehicle heading information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point. The wheel contact points in the present disclosure belong to the ground when looking down at the target vehicle, and the vehicle heading can be quickly and accurately determined using the relevant information of the wheel contact points on the ground.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent vehicle technology, and in particular to a method, device, electronic device, and storage medium for determining a vehicle orientation. Background Art

[0002] One of the core requirements of assisted driving is to perceive the status of surrounding vehicles. The orientation of surrounding vehicles is crucial information for predicting their next behavior and performing appropriate path planning.

[0003] Related technologies often use point cloud data to directly obtain a vehicle's 3D bounding box, thereby indirectly predicting its orientation. However, lasers are susceptible to rain and snow, resulting in inaccurate point cloud data. Furthermore, lidar has difficulty capturing objects with low reflectivity. These factors hinder the widespread application of point cloud data-based methods.

[0004] Due to their low cost and robustness to adverse weather conditions like rain and snow, cameras are currently a good alternative or supplement to lidar. However, because cameras themselves cannot provide spatial information, predicting the vehicle's orientation from images is difficult. Summary of the Invention

[0005] The embodiments of the present disclosure at least provide a method, device, electronic device, and storage medium for determining a vehicle orientation.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for determining a vehicle orientation, the method comprising:

[0007] Acquire the collected image;

[0008] Detecting the image to determine position information of each wheel contact point of the target vehicle in the image, as well as a contact point type of each wheel contact point; the contact point type is used to indicate visibility and / or obstruction of the wheel contact point;

[0009] Vehicle orientation information of the target vehicle is determined based on the position information of each wheel contact point and the contact point type of each wheel contact point.

[0010] Using the above-described method for determining vehicle orientation, wheel contact points can be detected upon acquiring an image. Vehicle orientation information can then be determined based on the detected location information of each wheel contact point and the contact point type corresponding to each wheel contact point. In this disclosure, the wheel contact points are located on the ground when viewed from above, and information related to the wheel contact points on the ground can be used to quickly and accurately determine the vehicle orientation.

[0011] In a possible implementation, determining the vehicle orientation information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point includes:

[0012] Combining the wheel contact points in pairs to obtain multiple groups of wheel contact points;

[0013] selecting a target group of wheel contact points from the plurality of groups of wheel contact points based on respective contact point types of two wheel contact points included in each group of wheel contact points;

[0014] The vehicle orientation information of the target vehicle is determined based on the position information of the two wheel contact points included in the target group of wheel contact points.

[0015] Here, when the combined multiple groups of wheel contact points are screened, the vehicle orientation information can be determined based on the position information of the two wheel contact points included in the screened target group of wheel contact points, which is simple and efficient.

[0016] In a possible implementation, selecting a target group of wheel contact points from the multiple groups of wheel contact points based on respective contact point types of two wheel contact points included in each group of wheel contact points includes:

[0017] Determine the grounding point score to be set for each grounding point type;

[0018] For each group of wheel contact points, determining a total contact point score of the group of wheel contact points based on the contact point scores of the contact point types of the two wheel contact points included in the group of wheel contact points;

[0019] A target group of wheel contact points is selected from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points.

[0020] Here, different contact point scores can be set for different contact point types, and then the total contact point score corresponding to each group of wheel contact points can be determined. According to the total contact point score, the target group of wheel contact points can be quickly selected from the multiple groups of wheel contact points.

[0021] In one possible implementation, the method further includes:

[0022] Determine the predicted probability of the contact point type for each wheel contact point;

[0023] The selecting a target group of wheel contact points from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points comprises:

[0024] A target group of wheel contact points is selected from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points and the predicted probability of the contact point type of each wheel contact point.

[0025] Here, the predicted probability of the contact point type detection and the total contact point score can be combined to implement the screening of the target group's wheel contact points, so as to better meet the needs of actual scenarios.

[0026] In a possible implementation, selecting a target group of wheel contact points from the multiple groups of wheel contact points based on the total contact point score of each group of wheel contact points includes:

[0027] A group of wheel contact points with the highest total contact point score is selected as the target group of wheel contact points.

[0028] In a possible implementation manner, determining the vehicle orientation information of the target vehicle based on the position information of two wheel contact points included in the target group of wheel contact points includes:

[0029] determining ground coordinate information of the two wheel contact points based on a coordinate conversion relationship between the image coordinate system where the wheel contact points are located and the top view coordinate system where the ground is located, and position information of two wheel contact points included in the target group of wheel contact points in the image;

[0030] determining, based on the ground coordinate information of the two wheel contact points, information about a connection line between the two wheel contact points;

[0031] Based on the connection information, vehicle orientation information of the target vehicle is determined.

[0032] Here, the wheel contact point in the image can be converted to the wheel contact point on the ground, so that the vehicle orientation information can be directly determined on the ground top view, which is simple and efficient.

[0033] In one possible implementation, the coordinate transformation relationship between the image coordinate system of the wheel contact point and the top-view coordinate system of the ground includes a first transformation relationship between the image coordinate system of the wheel contact point and a three-dimensional space coordinate system and a second transformation relationship between the three-dimensional space coordinate system and the top-view coordinate system of the ground.

[0034] Determining ground coordinate information of the two wheel contact points based on a coordinate conversion relationship between an image coordinate system where the wheel contact points are located and a top-view coordinate system where the ground is located, and position information of two wheel contact points included in the target group of wheel contact points in the image, includes:

[0035] determining three-dimensional coordinate information of two wheel contact points included in the target group of wheel contact points based on the first conversion relationship and position information of the two wheel contact points in the image;

[0036] Based on the second conversion relationship and the three-dimensional coordinate information of the two wheel contact points, the ground coordinate information of the two wheel contact points is determined.

[0037] In a possible implementation, the ground contact point type of each wheel ground contact point is determined according to the following steps:

[0038] Using a trained ground point detection neural network to extract features from the image, obtaining a multi-scale feature map; the multi-scale feature map includes feature maps of multiple scales;

[0039] Using a feature map with the smallest scale among the feature maps of the multiple scales as a first feature map for ground point type prediction;

[0040] A ground contact point type of each wheel ground contact point is predicted based on the first feature map.

[0041] Here, the first feature map representing higher-level semantics can be used to predict the grounding point type, which has higher prediction accuracy.

[0042] In one possible implementation, the scales of the feature maps at the multiple scales are gradually reduced, and the feature map at each level is enhanced by the feature map at the next level to obtain an enhanced feature map of the same scale. The position information of each wheel contact point of the target vehicle in the image is determined according to the following steps:

[0043] For each level, based on the feature map of the level and the enhanced feature map corresponding to the feature map of the next level, determine the enhanced feature map corresponding to the level;

[0044] The enhanced feature map corresponding to the feature map with the largest scale is used as the second feature map for position prediction;

[0045] The position information of each wheel contact point in the image is predicted based on the second feature map.

[0046] Here, the position information can be predicted through the second feature map with slightly lower-level semantics, which is conducive to the projection of the subsequent image coordinate system to the top-view coordinate system of the ground, and improves the accuracy and completeness of the projection points.

[0047] In a possible implementation manner, the grounding point type includes one of the following types:

[0048] The ground point is visible, invisible due to image truncation, blocked by other objects, or self-occluded.

[0049] In a second aspect, an embodiment of the present disclosure further provides a device for determining a vehicle orientation, the device comprising:

[0050] An acquisition module, used to acquire the collected image;

[0051] a detection module, configured to detect the image and determine position information of each wheel contact point of the target vehicle in the image, as well as a contact point type of each wheel; the contact point type is used to indicate visibility and / or obstruction of the wheel contact point;

[0052] The determination module is configured to determine the vehicle orientation information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point.

[0053] In a third aspect, an embodiment of the present disclosure further provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for determining the orientation of a vehicle as described in any one of the first aspect and its various embodiments are performed.

[0054] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the method for determining the orientation of a vehicle as described in the first aspect and any of its various embodiments.

[0055] For a description of the effects of the above-mentioned device for determining the vehicle orientation, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned method for determining the vehicle orientation, which will not be repeated here.

[0056] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0058] Figure 1 A flow chart of a method for determining a vehicle orientation provided by an embodiment of the present disclosure is shown;

[0059] Figure 2 A schematic diagram of a device for determining a vehicle orientation provided by an embodiment of the present disclosure is shown;

[0060] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0062] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0063] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0064] Research has found that related technologies often use point cloud data to directly obtain a vehicle's 3D bounding box, thereby indirectly predicting its orientation. However, lasers are susceptible to rain and snow, resulting in inaccurate point cloud data. Furthermore, lidar also struggles to capture objects with low reflectivity. These factors hinder the widespread application of point cloud data-based methods.

[0065] Due to their low cost and robustness to adverse weather conditions like rain and snow, cameras are currently a good alternative or supplement to lidar. However, because cameras themselves cannot provide spatial information, predicting the vehicle's orientation from images is difficult.

[0066] Based on the above research, the present disclosure provides a method, device, electronic device and storage medium for determining vehicle orientation based on wheel contact point detection, and the determined vehicle orientation is more accurate.

[0067] To facilitate understanding of this embodiment, a method for determining vehicle orientation disclosed in an embodiment of the present disclosure is first described in detail. The method for determining vehicle orientation provided in an embodiment of the present disclosure is generally executed by an electronic device with certain computing capabilities, such as a terminal device, a server, or other processing device. The terminal device may be a user equipment (UE), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the method for determining vehicle orientation may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0068] See also Figure 1 FIG. 1 is a flow chart of a method for determining a vehicle orientation provided by an embodiment of the present disclosure, the method comprising steps S101 to S103, wherein:

[0069] S101: Acquire the collected image;

[0070] S102: Detecting the image to determine the position information of each wheel contact point of the target vehicle in the image, as well as the contact point type of each wheel contact point; the contact point type is used to indicate the visibility and / or obstruction of the wheel contact point;

[0071] S103: Determine vehicle orientation information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point.

[0072] To facilitate understanding of the method for determining vehicle orientation provided by the embodiments of the present disclosure, the following briefly introduces the application scenarios of the method. The method for determining vehicle orientation in the embodiments of the present disclosure can be mainly applied to any scenario where it is necessary to determine the posture information of the vehicle orientation, such as the field of autonomous driving, and can also be applied to the field of road monitoring, etc., without specific limitations here. Considering the wide application of autonomous driving, the following examples are mainly used in autonomous driving scenarios.

[0073] Considering that the related technologies that use point cloud data to directly obtain the 3D bounding box of the vehicle and thus indirectly realize the prediction of the vehicle orientation are difficult to be widely used due to various reasons, the embodiments of the present disclosure provide a solution for determining the vehicle orientation based on the wheel contact point detection in the two-dimensional image, and the determined vehicle orientation is more accurate.

[0074] The captured images vary depending on the application scenario. For example, in autonomous driving, the captured images may be vehicle images captured by a camera mounted on the autonomous vehicle. These vehicle images may contain one or more target vehicles, and this disclosure does not impose any specific limitations on this.

[0075] For the captured image, wheel contact point detection can be used to determine the location and type of each wheel contact point. Wheel contact points correspond to the ground contact points of each wheel of the target vehicle. The target vehicle can be a sedan, a truck, or a tricycle, and the number of corresponding wheel contact points varies depending on the vehicle model.

[0076] The disclosed embodiment can detect the position information of all wheel contact points during the wheel contact point detection process. The wheels here include not only wheels visible in the captured image, but also wheels that are not visible in the captured image. The contact point type is used to indicate the visibility and / or occlusion of the wheel contact point. In specific applications, the contact point type here can be various types, such as the contact point is visible, the contact point is invisible due to image truncation, the contact point is invisible due to occlusion by other objects, and the contact point is invisible due to self-occlusion. For wheel contact points that are occluded and invisible, the position of the occluded wheel contact point can be estimated based on relevant information such as the position of the visible wheel contact point.

[0077] In specific applications, the aforementioned wheel contact point detection can be determined using image processing techniques, based on a neural network for contact point detection, or by other methods. Given that neural networks can uncover deeper wheel characteristics, embodiments of the present disclosure employ a neural network approach to implement wheel contact point detection, which offers simplified operation and relatively accurate detection results.

[0078] When the position information of each wheel contact point and the contact point type of each wheel contact point are determined, the vehicle orientation information of the target vehicle may be determined using the position information and the contact point type.

[0079] In the process of determining the vehicle orientation information, the wheel contact points can be first matched, and then the vehicle orientation information can be determined based on the position information of the paired wheel contact points. This can be achieved by the following steps:

[0080] Step 1: Combine the ground contact points of each wheel in pairs to obtain multiple groups of ground contact points of the wheel;

[0081] Step 2: selecting a target group of wheel contact points from the multiple groups of wheel contact points based on the contact point types of the two wheel contact points included in each group of wheel contact points;

[0082] Step 3: Determine the vehicle orientation information of the target vehicle based on the position information of the two wheel contact points included in the target group of wheel contact points.

[0083] Here, according to the principle that two points on a plane can determine a line, the orientation of a vehicle in a top view can be determined by the line connecting the two wheel contact points of the vehicle in the top view. Based on this, the embodiment of the present disclosure can first combine the various wheel contact points in pairs to obtain multiple groups of wheel contact points. Taking into account that different types of contact points have different degrees of influence on the determination of vehicle orientation information, for example, for wheel contact points that are in an obstructed state, it will weaken the determination result of the vehicle orientation information to a certain extent, and for wheel contact points that are visible, it will enhance the determination result of the vehicle orientation information to a certain extent. Therefore, here, the selection of the target group of wheel contact points can be achieved based on the respective contact point types of the two wheel contact points included in each group of wheel contact points, and then the vehicle orientation information can be determined by the position information of the two wheel contact points included in the target group of wheel contact points.

[0084] In the process of selecting the contact points of the target group wheels, the contact points may be selected based on the contact point scores set for each contact point type, and specifically may include the following steps:

[0085] Step 1: Determine the grounding point score set for each grounding point type;

[0086] Step 2: for each group of wheel contact points, based on the contact point scores of the contact point types of the two wheel contact points included in the group, determine the total contact point score corresponding to the group of wheel contact points;

[0087] Step 3: Based on the total contact point score of each group of wheel contact points, select a target group of wheel contact points from multiple groups of wheel contact points.

[0088] Here, each predicted wheel contact point can first be scored based on its contact point type. For example, a visible contact point can be assigned a score of 5, an obscured contact point can be assigned a score of 2, a self-occluded contact point can be assigned a score of 1, and an invisible contact point can be assigned a score of 0. Then, for each group of wheel contact points, a total contact point score can be determined for that group. In other words, the sum of the wheel contact point scores for different combinations can be calculated. These different combinations can include the right rear wheel and the left rear wheel, the right front wheel and the left front wheel, the right rear wheel and the right front wheel, and the left rear wheel and the left front wheel. The target group of wheel contact points can be selected by determining the sum of the scores for each combination.

[0089] Among them, when determining the sum of the scores, the ground contact point scores of the corresponding wheel ground contact points can be directly summed to obtain the sum value, or different weights can be assigned to the corresponding wheel ground contact points, and the sum value can be determined by weighted summation. Other methods can also be used to determine the sum value, and there are no specific restrictions here.

[0090] In a specific application, a group of wheel contact points with the highest total contact point score can be directly selected as the target group of wheel contact points.

[0091] In addition to the above method of selecting the one with the highest score, selection can also be made based on the predicted probability of detecting the corresponding contact point type for each wheel contact point.

[0092] In specific applications, the selection strategy can be set according to different application requirements. For example, if the total contact point scores of two groups of vehicle contact points are similar, the target group of wheel contact points can be further selected based on the product of the predicted probabilities of the two corresponding wheel contact points.

[0093] When determining the target group of wheel contact points, the vehicle orientation information of the target vehicle may be determined based on the position information of the two wheel contact points included in the target group of wheel contact points. This may be achieved by the following steps:

[0094] Step 1: Determine the ground coordinate information of the two wheel contact points based on the coordinate conversion relationship between the image coordinate system of the wheel contact points and the top view coordinate system of the ground, and the position information of the two wheel contact points included in the target group of wheel contact points in the image;

[0095] Step 3: Determine the connection information of the two wheel contact points based on the ground coordinate information of the two wheel contact points;

[0096] Step 4: Based on the connection information, determine the vehicle orientation information of the target vehicle.

[0097] The coordinate transformation relationship here can include a first transformation relationship between the image coordinate system where the wheel contact point is located and the three-dimensional space coordinate system, and a second transformation relationship between the three-dimensional space coordinate system and the top-view coordinate system where the ground is located. In this way, based on the first transformation relationship and the second transformation relationship, the position information of the two wheel contact points included in the target group of wheel contact points can be converted into the ground coordinate information of the two wheel contact points, and then the connection information of the two wheel contact points can be determined according to the ground coordinate information of the two wheel contact points. Such connection information can be used to characterize the vehicle orientation information of the target vehicle.

[0098] In specific applications, the first conversion relationship and the second conversion relationship can be realized by a homography matrix. Taking a camera as an imaging device as an example, since the various parameters of the camera (including internal and external parameters of the camera, height from the ground, and tilt angle with the ground) are known, a homography matrix can be obtained based on the above parameters to project the points on the image plane one-to-one onto the ground. Therefore, the embodiment of the present disclosure only needs to detect the ground contact points of the rear wheel and the front wheel on the same side of the vehicle (or the ground contact points of the right wheel and the left wheel on the same side) in the two-dimensional image, and then project the two wheel ground contact points onto the ground (equivalent to a top view) through the above homography matrix to determine the vehicle orientation in the top view.

[0099] Considering the key role of wheel contact point detection in determining vehicle orientation, the process of detecting the wheel contact point can be described in detail below.

[0100] The method for determining wheel orientation provided by the embodiment of the present disclosure can use a trained neural network to detect the wheel contact point on the collected image. The following will specifically describe the process of detecting position information and contact point type.

[0101] The embodiment of the present disclosure can detect the ground contact point type corresponding to each wheel ground contact point according to the following steps:

[0102] Step 1: Use the trained neural network to extract features from the image to obtain a multi-scale feature map; the multi-scale feature map includes feature maps of multiple scales;

[0103] Step 2: Using the smallest feature map among the multiple scale feature maps as the first feature map for ground point type prediction;

[0104] Step 3: Predict the contact point type of each wheel contact point based on the first feature map.

[0105] Here, as features are extracted layer by layer, the size of the feature maps that can be extracted becomes smaller and smaller, and the smaller and smaller sizes can be used to represent higher-level semantic features. The accuracy of predicting the ground point type using such semantic features is also higher.

[0106] In specific applications, multiple feature extraction layers can be set up to extract relevant feature maps. When the smallest feature map is extracted, it can be used as the first feature map for ground contact point type prediction. This first feature map can be used to predict the ground contact point type corresponding to each wheel ground contact point.

[0107] In addition, when the scales of feature maps at multiple scales decrease gradually, and the feature map at each level is enhanced by the feature map at the next level to obtain an enhanced feature map of the same scale (the same scale as the feature map before enhancement), the embodiment of the present disclosure can predict the position information of the wheel contact point according to the following steps:

[0108] Step 1: For each level, based on the feature map of the level and the enhanced feature map corresponding to the feature map of the next level, determine the enhanced feature map corresponding to the level;

[0109] Step 2: Use the enhanced feature map corresponding to the feature map with the largest scale as the second feature map for position prediction;

[0110] Step 3: Predict the position information of each wheel contact point in the image based on the second feature map.

[0111] Here, we can first determine the enhanced feature map corresponding to each level, and then use the enhanced feature map corresponding to the level of the largest feature map as the second feature map for position prediction. This is mainly because the predicted position information needs to support the subsequent conversion of the image coordinate system to the ground view coordinate system. The larger enhanced feature map retains the strong projection effect of the large size while also having richer features, making the determined position information more accurate.

[0112] Among them, the enhanced feature map corresponding to each level is associated with the feature map of this level and the enhanced feature map of a smaller scale below this level. There is an iterative relationship and mutual influence, which enables the extracted feature information to better express the wheel contact point.

[0113] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0114] Based on the same inventive concept, an embodiment of the present disclosure also provides a device for determining the vehicle orientation corresponding to the method for determining the vehicle orientation. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the method for determining the vehicle orientation in the above-mentioned embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0115] Reference Figure 2 FIG. 1 is a schematic diagram of a device for determining a vehicle orientation provided by an embodiment of the present disclosure, the device comprising: an acquisition module 201, a detection module 202, and a determination module 203; wherein,

[0116] An acquisition module 201 is used to acquire the collected image;

[0117] Detection module 202 is used to detect the image and determine the location information of each wheel contact point of the target vehicle in the image, as well as the contact point type of each wheel contact point; the contact point type is used to indicate the visibility and / or obstruction of the wheel contact point;

[0118] The determination module 203 is configured to determine the vehicle orientation information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point.

[0119] Using the above-described device for determining vehicle orientation, wheel contact points can be detected upon acquiring an image. Vehicle orientation information can then be determined based on the positional information and contact point type of each wheel contact point obtained. The wheel contact points in this disclosure are considered to be on the ground when viewed from above, and information related to the wheel contact points on the ground can be used to quickly and accurately determine vehicle orientation.

[0120] In one possible implementation, the determination module 203 is configured to determine the vehicle orientation information of the target vehicle based on the position information of each wheel contact point and the contact point type of each wheel contact point according to the following steps:

[0121] Combine the ground contact points of each wheel in pairs to obtain multiple groups of ground contact points of the wheel;

[0122] selecting a target group of wheel contact points from the plurality of groups of wheel contact points based on the contact point types of the two wheel contact points included in each group of wheel contact points;

[0123] Vehicle orientation information of the target vehicle is determined based on position information of two wheel contact points included in the target group of wheel contact points.

[0124] In one possible implementation, the determination module 203 is configured to select a target group of wheel contact points from multiple groups of wheel contact points based on the contact point types of the two wheel contact points included in each group of wheel contact points according to the following steps:

[0125] Determine the grounding point score to be set for each grounding point type;

[0126] For each group of wheel contact points, determining a total contact point score of the group of wheel contact points based on the contact point scores of the contact point types of the two wheel contact points included in the group of wheel contact points;

[0127] A target group of wheel contact points is selected from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points.

[0128] In one possible implementation, the determination module 203 is configured to select a target group of wheel contact points from multiple groups of wheel contact points based on the total contact point score of each group of wheel contact points according to the following steps:

[0129] Determine the predicted probability of the contact point type for each wheel contact point;

[0130] A target group of wheel contact points is selected from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points and the predicted probability of the contact point type of each wheel contact point.

[0131] In one possible implementation, the determination module 203 is configured to select a target group of wheel contact points from multiple groups of wheel contact points based on the total contact point score of each group of wheel contact points according to the following steps:

[0132] A group of wheel contact points with the highest total contact point score is selected as the target group of wheel contact points.

[0133] In a possible implementation, the determination module 203 is configured to determine the vehicle orientation information of the target vehicle based on the position information of two wheel contact points included in the target group of wheel contact points according to the following steps:

[0134] Determine the ground coordinate information of the two wheel contact points based on the coordinate conversion relationship between the image coordinate system where the wheel contact points are located and the top view coordinate system where the ground is located, and the position information of the two wheel contact points included in the target group of wheel contact points in the image;

[0135] Determine the connection line information of the two wheel contact points based on the ground coordinate information of the two wheel contact points;

[0136] Based on the connection information, the vehicle orientation information of the target vehicle is determined.

[0137] In one possible implementation, the coordinate transformation relationship between the image coordinate system of the wheel contact point and the top-view coordinate system of the ground includes a first transformation relationship between the image coordinate system of the wheel contact point and the three-dimensional space coordinate system and a second transformation relationship between the three-dimensional space coordinate system and the top-view coordinate system of the ground.

[0138] The determination module 203 is configured to determine the ground coordinate information of the two wheel contact points based on the coordinate conversion relationship between the image coordinate system of the wheel contact points and the top view coordinate system of the ground, and the position information of the two wheel contact points included in the target group of wheel contact points in the image, in accordance with the following steps:

[0139] Determine three-dimensional coordinate information of the two wheel contact points based on the first conversion relationship and position information of the two wheel contact points included in the target group of wheel contact points in the image;

[0140] Based on the second conversion relationship and the three-dimensional coordinate information of the two wheel contact points, the ground coordinate information of the two wheel contact points is determined.

[0141] In one possible implementation, the detection module 202 is configured to determine the contact point type of each wheel contact point according to the following steps:

[0142] The trained ground point detection neural network is used to extract features from the image to obtain a multi-scale feature map; the multi-scale feature map includes feature maps of multiple scales;

[0143] The smallest feature map among the multiple scale feature maps is used as the first feature map for ground point type prediction;

[0144] The ground contact point type of each wheel ground contact point is predicted based on the first feature map.

[0145] In one possible implementation, the scales of the feature maps at multiple scales are gradually reduced, and the feature map at each level is enhanced by the feature map at the next level to obtain an enhanced feature map of the same scale. The detection module 202 is configured to determine the position information of each wheel contact point of the target vehicle in the image according to the following steps:

[0146] For each level, based on the feature map of the level and the enhanced feature map corresponding to the feature map of the next level, determine the enhanced feature map corresponding to the level;

[0147] The enhanced feature map corresponding to the feature map with the largest scale is used as the second feature map for position prediction;

[0148] The position information of each wheel contact point in the image is predicted based on the second feature map.

[0149] In a possible implementation, the grounding point type includes one of the following types:

[0150] The ground point is visible, invisible due to image truncation, blocked by other objects, or self-occluded.

[0151] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0152] The present disclosure also provides an electronic device, such as Figure 3 FIG. 3 is a schematic diagram of the electronic device structure provided by an embodiment of the present disclosure, including: a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions (e.g., Figure 2 When the electronic device is running, the processor 301 communicates with the memory 302 via the bus 303. When the machine-readable instructions are executed by the processor 301, the following processing is performed:

[0153] Acquire the collected image;

[0154] Detecting the image to determine the position information of each wheel contact point of the target vehicle in the image, as well as the contact point type of each wheel contact point; the contact point type is used to indicate the visibility and / or obstruction of the wheel contact point;

[0155] Vehicle orientation information of the target vehicle is determined based on the position information of each wheel contact point and the contact point type of each wheel contact point.

[0156] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the method for determining a vehicle orientation described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0157] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the method for determining the vehicle orientation described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0158] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0159] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0162] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0163] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A method for determining a vehicle orientation, characterized in that: The method comprises: Acquire the collected image; Detecting the image to determine position information of each wheel contact point of the target vehicle in the image, as well as a contact point type of each wheel contact point; the contact point type is used to indicate visibility and / or obstruction of the wheel contact point; Combining the wheel contact points in pairs to obtain multiple groups of wheel contact points; determining a contact point score set for each contact point type; determining, for each group of wheel contact points, a total contact point score for the group of wheel contact points based on the contact point scores of the contact point types of two wheel contact points included in the group of wheel contact points; and selecting a target group of wheel contact points from the multiple groups of wheel contact points based on the total contact point score of each group of wheel contact points; determining ground coordinate information of the two wheel contact points based on a coordinate conversion relationship between an image coordinate system where the wheel contact points are located and a top-view coordinate system where the ground is located, and position information of two wheel contact points included in the target group of wheel contact points in the image; According to the ground coordinate information of the two wheel contact points, the connection information of the two wheel contact points is determined; and based on the connection information, the vehicle orientation information of the target vehicle is determined.

2. The method according to claim 1, characterized in that The method further comprises: Determine the predicted probability of the contact point type for each wheel contact point; The selecting a target group of wheel contact points from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points comprises: A target group of wheel contact points is selected from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points and the predicted probability of the contact point type of each wheel contact point.

3. The method according to claim 1, characterized in that The selecting a target group of wheel contact points from the plurality of groups of wheel contact points based on the total contact point score of each group of wheel contact points comprises: A group of wheel contact points with the highest total contact point score is selected as the target group of wheel contact points.

4. The method according to claim 1, wherein The coordinate conversion relationship between the image coordinate system where the wheel contact point is located and the top view coordinate system where the ground is located includes a first conversion relationship between the image coordinate system where the wheel contact point is located and the three-dimensional space coordinate system and a second conversion relationship between the three-dimensional space coordinate system and the top view coordinate system where the ground is located; Determining ground coordinate information of two wheel contact points included in the target group of wheel contact points based on a coordinate conversion relationship between an image coordinate system where the wheel contact points are located and a top-view coordinate system where the ground is located, and position information in the image of two wheel contact points, includes: determining three-dimensional coordinate information of two wheel contact points included in the target group of wheel contact points based on the first conversion relationship and position information of the two wheel contact points in the image; Based on the second conversion relationship and the three-dimensional coordinate information of the two wheel contact points, the ground coordinate information of the two wheel contact points is determined.

5. The method according to any one of claims 1 to 4, characterized in that: Determine the ground contact point type for each wheel ground contact point as follows: Extracting features from the image using a trained neural network to obtain a multi-scale feature map; the multi-scale feature map includes feature maps at multiple scales; Using a feature map with the smallest scale among the feature maps of the multiple scales as a first feature map for ground point type prediction; A ground contact point type of each wheel ground contact point is predicted based on the first feature map.

6. The method according to claim 5, characterized in that The scales of the feature maps of the multiple scales decrease gradually, and the feature map of each level is enhanced by the feature map of the next level to obtain an enhanced feature map of the same scale; the position information of each wheel contact point of the target vehicle in the image is determined according to the following steps: For each level, based on the feature map of the level and the enhanced feature map corresponding to the feature map of the next level, determine the enhanced feature map corresponding to the level; The enhanced feature map corresponding to the feature map with the largest scale is used as the second feature map for position prediction; The position information of each wheel contact point in the image is predicted based on the second feature map.

7. The method according to any one of claims 1 to 4, characterized in that: The grounding point type includes one of the following types: The ground point is visible, invisible due to image truncation, blocked by other objects, or self-occluded.

8. A device for determining a vehicle orientation, characterized in that: The device comprises: An acquisition module, used to acquire the collected image; a detection module, configured to detect the image and determine position information of each wheel contact point of the target vehicle in the image, as well as a contact point type corresponding to each wheel contact point; the contact point type is used to indicate visibility and / or obstruction of the wheel contact point; a determination module, configured to combine the wheel contact points in pairs to obtain multiple groups of wheel contact points; determining a contact point score set for each contact point type; determining, for each group of wheel contact points, a total contact point score for the group of wheel contact points based on the contact point scores of the contact point types of two wheel contact points included in the group of wheel contact points; and selecting a target group of wheel contact points from the multiple groups of wheel contact points based on the total contact point score of each group of wheel contact points; determining ground coordinate information of the two wheel contact points based on a coordinate conversion relationship between an image coordinate system where the wheel contact points are located and a top-view coordinate system where the ground is located, and position information of two wheel contact points included in the target group of wheel contact points in the image; According to the ground coordinate information of the two wheel contact points, the connection information of the two wheel contact points is determined; and based on the connection information, the vehicle orientation information of the target vehicle is determined.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the method for determining the vehicle orientation as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for determining the vehicle orientation according to any one of claims 1 to 7.

Images