Obstacle orientation determination method, device, system, equipment, medium and product

Abstract

The present invention provides a method, apparatus, system, device, medium, and product for determining the orientation of an obstacle. The method comprises: obtaining a target obstacle image detected in a detection area, the target obstacle image including multiple wheels; when the multiple wheels are wheels of the same vehicle, determining the wheel pair with the largest distance between the multiple wheels; obtaining the contact position of the wheel pair with the largest distance in the target obstacle image; mapping the contact position to a road surface position in a world coordinate system; and determining the orientation angle of the vehicle based on the road surface position in the world coordinate system. In other words, the embodiment of the present invention can quickly determine the orientation of 3D obstacle vehicles in the area near the main vehicle based on wheel detection, solving the technical problem of inaccurate determination of the 3D obstacle orientation due to truncated information of the 3D obstacle in the pixel coordinate system. By using the embodiment of the present invention, the accuracy of the orientation angle of the obstacle vehicle can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of obstacle detection technology, and in particular to a method, apparatus, system, device, computer-readable storage medium, and computer program product for determining the direction of an obstacle. Background Art

[0002] With the gradual maturity of 3D obstacle detection and lidar technology, in related technologies, although the vehicle's orientation (yaw) angle can be determined by obtaining the vehicle's 3D coordinate information from the pixel coordinate system, in camera-based surround view perception tasks, when an obstacle (such as a large truck) passes through the range of the main vehicle's lateral camera, if the complete 3D coordinate information of the large truck in the pixel coordinate system is not obtained, that is, there is truncated information of the obstacle in the pixel coordinate system, this will cause serious errors in the orientation of the 3D obstacle.

[0003] Therefore, how to accurately determine the direction of 3D obstacles is a technical problem that needs to be solved. Summary of the Invention

[0004] The present invention provides a method, apparatus, system, electronic device, computer-readable storage medium, and computer program product for determining the orientation of an obstacle, to at least address the technical problem in the related art of inaccurately determining the orientation of a 3D obstacle due to truncation of 3D obstacle information in a pixel coordinate system. The technical solution of the present invention is as follows:

[0005] According to a first aspect of an embodiment of the present invention, a method for determining an obstacle orientation is provided, comprising:

[0006] Acquire a target obstacle image detected in a detection area, where the target obstacle image includes a plurality of wheels;

[0007] When the plurality of wheels are wheels of the same vehicle, determining a wheel pair having a largest distance between the plurality of wheels;

[0008] Obtaining the ground contact position of the wheel pair with the largest distance in the target obstacle image;

[0009] Mapping the ground contact position to a road surface position in a world coordinate system;

[0010] The heading angle of the vehicle is determined based on the road surface position in the world coordinate system.

[0011] Optionally, determining the wheel pair having the largest distance between the multiple wheels includes:

[0012] Matching the size of each wheel in the plurality of wheels with the size of a wheel template in a template pool;

[0013] If the matching result meets the set threshold, the coordinate information of the wheel meeting the set threshold in pixel coordinates is obtained;

[0014] Calculate the distance between each pair of wheels according to the coordinate information;

[0015] Select the pair of wheels with the largest distance between them.

[0016] Optionally, when the multiple wheels are wheels of the same vehicle, the method further includes:

[0017] performing consistency checking on the dimensions of the plurality of wheels;

[0018] If the consistency check succeeds, performing the step of determining the wheel pair having the largest distance between the plurality of wheels;

[0019] If the consistency check fails, the wheel is completed according to the set strategy.

[0020] Optionally, before performing consistency check on the dimensions of the plurality of wheels, the method further includes:

[0021] determining whether each of the plurality of wheels has a wheel truncation condition;

[0022] If there is no wheel truncation, then executing the step of verifying the consistency of all wheel sizes;

[0023] If there is a wheel cutoff, the wheel is completed according to the set strategy.

[0024] Optionally, the completing the wheel according to a set strategy includes:

[0025] Selecting the truncated wheel with the largest area, matching the truncated wheel with the wheel template in the template pool; completing all truncated wheels according to the matched wheel template; or

[0026] If a complete wheel is selected from the plurality of wheels, then all the truncated wheels are completed according to the selected complete wheel; or

[0027] If two complete wheels are selected from the plurality of wheels, an average value of the wheel areas of the two complete wheels is calculated, and all the truncated wheels are completed based on the average value.

[0028] Optionally, mapping the ground contact position to a road surface position in a world coordinate system includes:

[0029] The ground contact position is mapped to a road surface position in a world coordinate system by using an inverse perspective transformation formula.

[0030] Optionally, determining the heading angle of the vehicle based on the road surface position in the world coordinate system includes:

[0031] Calculating a wheel line of a road surface position of the wheel pair in a world coordinate system;

[0032] The orientation of the wheel line around the direction of gravity is determined as the orientation angle of the vehicle.

[0033] According to a second aspect of an embodiment of the present invention, a method for determining an obstacle orientation is provided, comprising:

[0034] Determining a first heading angle of a current vehicle within the detection area, the first heading angle being determined based on a wheel pair in an image of a target obstacle in the detection area;

[0035] Obtain the second orientation angle of the current vehicle detected within the detection area output by the 3D obstacle detection model;

[0036] determining a difference between the first orientation angle and the second orientation angle;

[0037] If the difference is less than a preset threshold, determining an angle average of the first orientation angle and the second orientation angle;

[0038] The angle average is determined as the heading angle of the current vehicle.

[0039] Optionally, the method further includes:

[0040] If the difference is not less than a preset threshold, obtaining the historical heading angle of the current vehicle;

[0041] Performing curve fitting on the historical heading angles using a random sampling consistency RANSAC verification algorithm to predict a third heading angle of the current vehicle;

[0042] The first orientation angle or the second orientation angle closest to the third orientation angle is determined as the orientation angle of the current vehicle.

[0043] Optionally, determining the first heading angle of the current vehicle includes:

[0044] Acquire a target obstacle image detected in a detection area, where the target obstacle image includes a plurality of wheels;

[0045] When the plurality of wheels are wheels of the same vehicle, determining a wheel pair having a largest distance between the plurality of wheels;

[0046] Obtaining the ground contact position of the wheel pair with the largest distance in the target obstacle image;

[0047] Mapping the ground contact position to a road surface position in a world coordinate system;

[0048] A first heading angle of the vehicle is determined based on a road surface position in the world coordinate system.

[0049] According to a third aspect of an embodiment of the present invention, a device for determining an obstacle direction is provided, comprising:

[0050] A first acquisition module is configured to acquire a target obstacle image detected in a detection area, wherein the target obstacle image includes a plurality of wheels;

[0051] a first determining module, configured to determine, when the plurality of wheels are wheels of the same vehicle, a wheel pair having a largest distance between the plurality of wheels;

[0052] a second acquisition module, configured to acquire the ground contact position of the wheel pair with the largest distance in the target obstacle image;

[0053] A mapping module, configured to map the ground contact position to a road surface position in a world coordinate system;

[0054] The second determining module is configured to determine the heading angle of the vehicle based on the road surface position in the world coordinate system.

[0055] Optionally, the first determining module includes:

[0056] A first matching module is configured to match the size of each wheel in the plurality of wheels with the size of a wheel template in a template pool;

[0057] a third acquisition module, configured to acquire, when the matching result of the first matching module meets a set threshold, coordinate information of the wheel that meets the set threshold in pixel coordinates;

[0058] A first calculation module is used to calculate the distance between each pair of wheels according to the coordinate information;

[0059] The first selection module is used to select a pair of wheels with the largest distance between them.

[0060] Optionally, the device further includes:

[0061] a verification module, configured to perform consistency verification on the dimensions of the plurality of wheels when the plurality of wheels are wheels of the same vehicle;

[0062] The first determining module is further configured to determine the wheel pair having the largest distance between the plurality of wheels when the consistency check by the checking module succeeds;

[0063] The completion module is used to complete the wheel according to a set strategy when the consistency check of the verification module fails.

[0064] Optionally, the device further includes:

[0065] a first judging module, configured to judge whether each of the plurality of wheels has a wheel truncation condition before the checking module performs consistency checking on the dimensions of the plurality of wheels;

[0066] The first determining module is further configured to determine a wheel pair having the largest distance between the multiple wheels when the first judging module determines that no wheel truncation occurs.

[0067] The completion module is further configured to complete the wheel according to a set strategy when the first judgment module determines that a wheel is cut off.

[0068] Optionally, the completion module includes: a second selection module, a second matching module and a first completion module; and / or a third selection module and a second completion module; and / or a fourth selection module and a third completion module; wherein,

[0069] The second selection module is used to select the truncated wheel with the largest area;

[0070] The second matching module is used to match the truncated wheel with the largest area with a wheel template in a template pool;

[0071] The first completion module is configured to complete all truncated wheels according to the wheel template matched by the second matching module;

[0072] The third selection module is configured to select a complete wheel from the plurality of wheels;

[0073] The second completing module is configured to complete all truncated wheels according to the complete wheels selected by the third selecting module;

[0074] The fourth selection module is configured to select two complete wheels from the plurality of wheels;

[0075] The second calculation module is used to calculate the average wheel area of ​​the two complete wheels selected by the fourth selection module, and complete all the truncated wheels according to the average wheel area.

[0076] Optionally, the mapping module is specifically configured to map the ground contact position to a road surface position in a world coordinate system through an inverse perspective transformation formula.

[0077] Optionally, the second determining module includes:

[0078] a third calculation module, configured to calculate a wheel line of the road surface position of the wheel pair in a world coordinate system;

[0079] The orientation angle determination module is used to determine the orientation of the wheel line around the direction of gravity as the orientation angle of the vehicle.

[0080] According to a fourth aspect of an embodiment of the present invention, there is provided a device for determining an obstacle direction, comprising:

[0081] a first determining module, configured to determine a first heading angle of a current vehicle in a detection area, wherein the first heading angle is determined based on a wheel pair in a target obstacle image in the detection area;

[0082] An acquisition module is used to obtain a second orientation angle of the current vehicle detected in the detection area output by the 3D obstacle detection model;

[0083] a second determining module, configured to determine a difference between the first orientation angle and the second orientation angle;

[0084] a third determining module, configured to determine an angle average of the first orientation angle and the second orientation angle when the difference is less than a preset threshold;

[0085] The fourth determining module is configured to determine the angle average as the heading angle of the current vehicle.

[0086] Optionally, the device further includes:

[0087] A second acquisition module is used to acquire a historical heading angle of the current vehicle when the difference is not less than a preset threshold;

[0088] A fitting module, configured to perform curve fitting on the historical heading angles using a random sampling consistency RANSAC verification algorithm to predict a third heading angle of the current vehicle;

[0089] A fifth determining module is configured to determine the first orientation angle or the second orientation angle closest to the third orientation angle as the orientation angle of the current vehicle.

[0090] Optionally, the first determining module includes:

[0091] a third acquisition module, configured to acquire a target obstacle image detected in the detection area, wherein the target obstacle image includes a plurality of wheels;

[0092] a sixth determining module, configured to determine, when the plurality of wheels are wheels of the same vehicle, a wheel pair having a largest distance between the plurality of wheels;

[0093] A fourth acquisition module is configured to acquire the ground contact position of the wheel pair with the largest distance in the target obstacle image;

[0094] A mapping module, configured to map the ground contact position to a road surface position in a world coordinate system;

[0095] A seventh determination module is configured to determine a first heading angle of the vehicle based on a road surface position in the world coordinate system.

[0096] According to a fifth aspect of an embodiment of the present invention, a system for determining an obstacle orientation is provided. The system is applied to a 3D obstacle detection network, and the system includes:

[0097] The 2D obstacle detection module is configured to detect images decoded by the decoder in the 3D obstacle detection network and obtain a target obstacle image in the detection area, the target obstacle image including multiple wheels; if the multiple wheels are wheels of the same vehicle, determine the wheel pair with the largest distance between the multiple wheels; and obtain the contact position of the wheel pair with the largest distance in the target obstacle image;

[0098] A parameter transformation module is used to map the ground contact position to a road surface position in a world coordinate system in the 3D obstacle detection network; and determine the vehicle's heading angle based on the road surface position in the world coordinate system.

[0099] In an embodiment of the present invention, the system is based on a 3D obstacle detection network. That is, the wheel detection network provided in this embodiment is a 2D head separated from the 3D obstacle detection network. Therefore, the changes to the backbone network are relatively small. At the same time, after detecting the wheels, the results of the vehicle detection are verified and supplemented through a post-processing, thereby obtaining a more precise vehicle orientation angle.

[0100] According to a sixth aspect of an embodiment of the present invention, there is provided an electronic device, comprising:

[0101] processor;

[0102] a memory for storing instructions executable by the processor;

[0103] The processor is configured to execute the instructions to implement the method for determining the direction of the obstacle as described above.

[0104] According to a seventh aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method for determining the direction of an obstacle as described above.

[0105] According to an eighth aspect of an embodiment of the present invention, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implements the method for determining the direction of an obstacle as described above.

[0106] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects:

[0107] In an embodiment of the present invention, when the target obstacle image detected in the detection area includes multiple wheels, if the multiple wheels are wheels of the same vehicle, the wheel pair with the largest distance between the multiple wheels is determined; the ground contact position of the wheel pair in the target obstacle image is obtained; the ground contact position is mapped to the road surface position in the world coordinate system; and the heading angle of the vehicle is determined based on the road surface position in the world coordinate system. That is, in an embodiment of the present invention, based on wheel detection, the ground contact position of a pair of wheels with the largest distance among multiple wheels of the same vehicle is obtained, and the wheel connection line of the ground contact position of this pair of wheels is in a straight line to determine the heading angle (i.e., the yaw angle) of the obstacle vehicle. Therefore, based on wheel detection, the embodiment of the present invention can quickly determine the heading of a 3D obstacle vehicle in the area near the main vehicle, solving the technical problem in the related art of inaccurate determination of the 3D obstacle direction due to truncation of 3D obstacle information in the pixel coordinate system. By adopting the embodiment of the present invention, the accuracy of the heading angle of the obstacle vehicle can be effectively improved.

[0108] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the description are used to explain the principles of the present invention, and do not constitute an improper limitation of the present invention.

[0110] Figure 1 This is a flow chart of a method for determining the direction of an obstacle provided by an embodiment of the present invention.

[0111] Figure 2 Schematic diagram of a marked wheel detection frame provided by an embodiment of the present invention.

[0112] Figure 3 It is a schematic diagram of a truncated wheel provided by an embodiment of the present invention.

[0113] Figure 4 It is a schematic diagram of a truncated wheel provided by an embodiment of the present invention.

[0114] Figure 5 This is an example diagram of an application of a method for determining the direction of an obstacle provided by an embodiment of the present invention.

[0115] Figure 6 This is another flow chart of a method for determining an obstacle orientation provided by an embodiment of the present invention.

[0116] Figure 7 This is another flow chart of a method for determining an obstacle direction provided by an embodiment of the present invention.

[0117] Figure 8 This is a block diagram of a device for determining an obstacle direction provided by an embodiment of the present invention.

[0118] Figure 9 This is another block diagram of a device for determining an obstacle direction provided by an embodiment of the present invention.

[0119] Figure 10 This is a block diagram of a system for determining an obstacle orientation provided by an embodiment of the present invention.

[0120] Figure 10A This is an application block diagram of a system for determining the direction of an obstacle provided by an embodiment of the present invention.

[0121] Figure 11 This is a block diagram of an electronic device provided by an embodiment of the present invention.

[0122] Figure 12 This is a block diagram of a device for determining the direction of an obstacle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0123] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0124] It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0125] Figure 1 is a flow chart of a method for determining the direction of an obstacle provided by an embodiment of the present invention, such as Figure 1 As shown, the method for determining the direction of the obstacle includes the following steps:

[0126] Step 101: Acquire a target obstacle image detected in a detection area, where the target obstacle image includes a plurality of wheels;

[0127] Step 102: When the multiple wheels are wheels of the same vehicle, determining a wheel pair having a largest distance between the multiple wheels;

[0128] Step 103: obtaining the contact position of the wheel pair with the largest distance in the target obstacle image;

[0129] Step 104: Mapping the ground contact position to a road surface position in a world coordinate system;

[0130] Step 105: Determine the heading angle of the vehicle based on the road surface position in the world coordinate system.

[0131] The method for determining the direction of an obstacle described in an embodiment of the present invention can be applied to a terminal, etc. The implementation device of the terminal can be an electronic device such as a vehicle-mounted terminal, a main control platform of an autonomous driving vehicle, or a vehicle computer, and is not limited here.

[0132] The following combination Figure 1 , the specific implementation steps of a method for determining the direction of an obstacle provided by an embodiment of the present invention are described in detail.

[0133] In step 101, a target obstacle image detected in a detection area is acquired, where the target obstacle image includes a plurality of wheels.

[0134] In this step, a 2D obstacle detection network (2D head, also called 2D detection head) is separated based on the 3D obstacle detection network (3D head, also called 3D detection head). The 2D head is used to detect the wheels of vehicles in a preset area, that is, the main control platform of the main vehicle can obtain the target obstacle image in the predetermined area near the main vehicle through the camera in the 2D head, such as the vehicle image, etc. The target obstacle image may include multiple wheels, and the multiple wheels may be wheels of the same vehicle or wheels of different vehicles. When the wheels are different wheels of multiple wheels, such as the case where the same vehicle has only one wheel, the system will automatically filter out the case where a vehicle has only one wheel. Afterwards, the multiple wheels of each vehicle in the target obstacle image are marked, and the wheels can be marked by wheel detection frames. Furthermore, the wheel detection frames of each vehicle can be numbered and sorted. For example, the main control platform detects three wheel detection frames of the wheels of a large truck in the predetermined area, such as Figure 2 As shown in Figure 2 A schematic diagram of a wheel detection frame marked according to an embodiment of the present invention is provided. Figure 2 In the figure, the wheel detection frames marking the wheels are respectively illustrated with reference numerals 1, 2 and 3 as examples, etc. For another example, the vehicle detected in this embodiment may also be a car, etc., which is not limited in this embodiment.

[0135] In step 102 , if the multiple wheels are wheels of the same vehicle, a wheel pair having the largest distance between the multiple wheels is determined.

[0136] In this step, after the main control platform obtains the multiple wheels included in the target obstacle image detected in the detection area, it first determines the number of vehicles based on the multiple wheels. If it is determined that the multiple wheels belong to different vehicles and each vehicle includes only one wheel, the operation process is terminated; if the multiple wheels belong to the same vehicle, the wheel pair with the largest distance between the multiple wheels is determined.

[0137] Determining the wheel pair having the largest distance between the multiple wheels includes:

[0138] First, the main control platform matches the size of each wheel in the multiple wheels with the size of the wheel template in the template pool.

[0139] In this step, the purpose of matching the wheel modules in the template pool is to ensure that the multiple wheels detected are those of vehicles in the core area. In other words, this embodiment typically corrects the wheel heading angles of obstacle vehicles in the core area near the autonomous vehicle to avoid errors introduced by obstacle vehicles outside the core area. The wheel size must meet the threshold of the template pool.

[0140] One matching process is as follows: determine the vehicle model, search for the wheel template of the model from the template pool, then calculate the difference between the size of each wheel among multiple wheels and the size of the found wheel template, and judge whether the difference is less than the set threshold. If so, it is considered that the wheel size meets the set threshold and the match is successful; otherwise, it is considered that the match is unsuccessful.

[0141] Secondly, if the matching result meets the set threshold, the coordinate information of the wheel meeting the set threshold in pixel coordinates is obtained.

[0142] In this step, if the matching result meets the set threshold, the pixel coordinate information of the wheel meeting the set threshold is obtained. The process of obtaining the pixel coordinate information of each wheel meeting the set threshold is well known in the art and will not be described in detail here.

[0143] Again, the distance between each pair of wheels is calculated based on the coordinate information.

[0144] In this step, after obtaining the coordinate information of each wheel that meets the set threshold, the calculation formula between the two coordinate points can be used to calculate the distance between the coordinate points of each two wheels. The specific calculation formula is already a well-known technology for those skilled in the art and will not be repeated here.

[0145] Finally, select the pair of wheels with the largest distance between them.

[0146] After calculating the distance between each coordinate point, the wheel pair corresponding to the two coordinate points with the largest distance is selected.

[0147] In step 103, the ground contact position of the wheel pair in the target obstacle image is obtained.

[0148] In this step, after determining the pair of wheels with the largest distance, the main control platform can obtain the coordinate information of the grounding position of this pair of wheels in the target obstacle image, that is, the coordinate information of the wheel landing point in the pixel coordinate system. The method of obtaining it is already a well-known technology for those skilled in the art and will not be repeated here.

[0149] In step 104 , the ground contact position is mapped to a road surface position in a world coordinate system.

[0150] In this step, one mapping method is that the main control platform maps the ground contact position to the road surface position in the world coordinate system through an inverse perspective transformation formula. Of course, in actual applications, it is not limited to this.

[0151] In this step, the wheel contact position (i.e., the wheel contact point) is regressed to the road surface position in the world coordinate system. This relies on the internal and external parameters of the unmanned vehicle camera. Since the wheel contact position is at coordinate z = 0 in the world coordinate system, the wheel position (u, v) in the image is converted to (x, y) in the world coordinate system using the inverse perspective transformation formula. The specific inverse perspective transformation formula is as follows:

[0152]

[0153] Where X0(u,v) and Y0(u,v) represent the road coordinates in the world coordinate system; u and v represent the horizontal and vertical coordinates in the image coordinate system (i.e., pixel coordinate system) mapped to the horizontal and vertical coordinates in the world coordinate system; m and n represent the width and height of the image coordinate system; (Cx, Cy, h) represents the coordinate position of the camera in the world coordinate system. The camera installation position is relatively fixed. Choosing a suitable world coordinate system can make h equal to the height of the camera from the ground; α v Indicates the upper and lower field of view angle range of the camera; α u Indicates the horizontal field of view of the camera. θ indicates the camera's pitch angle.

[0154] Among them, α v and α a The value of can usually be expressed by other internal parameter data, as follows:

[0155]

[0156] Where W and H in the formula represent the length and width of the camera's photosensitive component, respectively, and f is the camera's focal length.

[0157] It should be noted that, for those skilled in the art, the above-mentioned inverse perspective transformation formula is already a well-known technology and will not be described in detail here.

[0158] In step 105 , the heading angle of the vehicle is determined based on the road surface position in the world coordinate system.

[0159] In this step, the wheel line of the wheel pair on the road surface in the world coordinate system is first calculated; and the orientation of the wheel line around the direction of gravity is determined as the orientation angle of the vehicle.

[0160] In other words, based on wheel detection, once the road position of the wheel pair is determined, any two front and rear wheels will be aligned due to the vehicle's rigid structure. Therefore, the wheel line connecting the wheel pair's road position in the world coordinate system is calculated. The orientation of this wheel line around gravity is the vehicle's heading (i.e., the yaw angle). In other words, knowing the road coordinates of the wheel pair allows for an accurate estimation of the vehicle's heading.

[0161] In an embodiment of the present invention, when the target obstacle image detected in the detection area includes multiple wheels, if the multiple wheels are wheels of the same vehicle, the wheel pair with the largest distance between the multiple wheels is determined; the ground contact position of the wheel pair in the target obstacle image is obtained; the ground contact position is mapped to the road surface position in the world coordinate system; and the heading angle of the vehicle is determined based on the road surface position in the world coordinate system. That is, in an embodiment of the present invention, based on wheel detection, the ground contact position of a pair of wheels with the largest distance among multiple wheels of the same vehicle is obtained, and the wheel connection line of the ground contact position of this pair of wheels is in a straight line to determine the heading angle (i.e., the yaw angle) of the obstacle vehicle. Therefore, based on wheel detection, the embodiment of the present invention can quickly determine the heading of a 3D obstacle vehicle in the area near the main vehicle, solving the technical problem in the related art of inaccurate determination of the 3D obstacle direction due to truncation of 3D obstacle information in the pixel coordinate system. By adopting the embodiment of the present invention, the accuracy of the heading angle of the obstacle vehicle can be effectively improved.

[0162] Optionally, in another embodiment, based on the above embodiment, multiple wheels in the target obstacle image are wheels of the same vehicle, and the sizes of the multiple wheels are checked for consistency. If the consistency check is successful, the step of determining the wheel pair with the largest distance between the multiple wheels is executed; if the consistency check fails, the wheels are completed according to the set strategy.

[0163] Among them, there are many ways to perform consistency check on the sizes of the multiple wheels. One is to first calculate the area of ​​each wheel, and then compare whether the areas are equal or approximately equal. If the areas of the multiple wheels are equal or approximately equal, it is considered that the consistency of the multiple wheels is strong, that is, it is confirmed that the consistency check is successful. If the areas of the multiple wheels are not equal, or the area of ​​at least one wheel is not equal to or has a large difference from the areas of the other wheels, the wheel with a smaller area is considered to be a truncated wheel, and the truncated wheel needs to be completed according to the set strategy.

[0164] The purpose of the wheel consistency check in this embodiment is to solve the problem of truncation of some wheels. For example, when there is a slightly truncated wheel among multiple wheels, but when the consistency check of the sizes of the multiple wheels is performed, the consistency check is successful. In this case, this embodiment defaults to not requiring additional processing for the slightly truncated wheel.

[0165] Optionally, in another embodiment, based on the above embodiment, before performing consistency check on the sizes of the multiple wheels, the method may further include: determining whether there is a wheel truncation situation for each of the multiple wheels; if there is no wheel truncation situation, executing the step of checking the consistency of the sizes of all wheels; if there is a wheel truncation situation, completing the wheel according to the set strategy.

[0166] In this embodiment, before performing consistency check on the sizes of the multiple wheels, it is necessary to first determine whether each of the multiple wheels has a wheel truncation. If a wheel truncation exists, it means that the sizes of the multiple wheels are inconsistent and the consistency is relatively poor. The truncated wheels need to be completed according to the set strategy, and then the step of determining the wheel pair with the largest distance between the multiple wheels is executed; if no wheel truncation exists, it means that the sizes of the multiple wheels are relatively consistent, and the step of directly checking the consistency of the sizes of all wheels is executed.

[0167] It is necessary to determine whether each wheel is truncated. There are various ways to do this. For example, the diameters of the wheels of the same vehicle can be compared, with the largest wheel being considered intact and the others being truncated. Alternatively, the area of ​​each wheel can be calculated, and then the areas can be compared to determine the wheel with the largest area as intact and the others with smaller areas as truncated. Alternatively, after calculating the area, if the difference between the areas of the two larger wheels is less than a predetermined value, the wheel can be determined to be intact.

[0168] To understand the complete and cut-off wheels, please also refer to Figures 2 to 4 , as mentioned above Figure 2 As shown, Figure 2In the example of detecting three wheels of a large truck, the three wheels numbered 1 to 3 are all complete wheels, without any truncation. Figure 3 , Figure 3 A schematic diagram of a truncated wheel provided in an embodiment of the present invention, Figure 3 Still taking the large truck as an example, Figure 3 It can be seen that the wheel numbered 1 is a truncated wheel, while the wheels numbered 2 and 3 are complete wheels. Figure 4 , Figure 4 A schematic diagram of another method of cutting off a wheel according to an embodiment of the present invention is provided. Figure 4 Still taking the large truck as an example, Figure 4 It can be seen that the wheels marked 1 and in Table 3 are truncated wheels, while the wheels marked 2 are complete wheels. Figure 3 and Figure 4 The wheel cutting described in the figure is only an example for explanation and is not limited to this in practical applications.

[0169] Furthermore, in another embodiment, based on the above embodiment, the method may also include: determining whether each wheel among the multiple wheels has a wheel truncation; if no truncation exists, entering the wheel consistency check; if the wheel size consistency is strong, then continuing to determine whether the wheel and the wheel template size in the template pool match; if the size meets the threshold of the template pool, the match is successful, and all truncated wheels are completed using the matched wheel template.

[0170] It should be noted that, in the embodiment of the present invention, wheel detection is used because: 1. The size of the wheels is relatively fixed and the features are obvious, and the detection network is easy to design and apply; 2. There must be a grounding point at the wheel of each vehicle, and the coordinate point of the wheel grounding point in the pixel coordinate system can be transformed into the corresponding coordinate point in the world coordinate system through inverse perspective; 3. The wheels of the same vehicle are of the same size, so the completion strategy can be used to complete the problem of partial wheel truncation.

[0171] Furthermore, in another embodiment, based on the above embodiment, the wheels are complemented according to a set strategy, including:

[0172] Selecting the truncated wheel with the largest area, matching the truncated wheel with the wheel template in the template pool; completing all truncated wheels according to the matched wheel template; or

[0173] If a complete wheel is selected from the plurality of wheels, then all the truncated wheels are completed according to the selected complete wheel; or

[0174] If two complete wheels are selected from the plurality of wheels, an average value of the wheel areas of the two complete wheels is calculated, and all the truncated wheels are completed based on the average value.

[0175] In the embodiment of the present invention, the wheel heading angle is not corrected for all obstacle vehicles sensed by the main vehicle. Instead, the heading angle is corrected based on the wheels that match the wheel templates in the template pool. This allows vehicles that are far away and may produce large errors to be filtered out very accurately, ensuring a positive benefit in the sensed vehicle heading.

[0176] See also Figure 5 , is a diagram illustrating an application example of a method for determining an obstacle orientation provided by an embodiment of the present invention, the method comprising:

[0177] Step 501: Acquire a target obstacle image detected in a detection area, where the target obstacle image includes a plurality of wheels.

[0178] Step 502: Determine whether the multiple wheels are wheels of the same vehicle. If yes, execute step 503; otherwise, execute step 512;

[0179] Step 503: performing consistency check on the dimensions of the plurality of wheels; if the consistency check fails, executing step 504, and then executing step 505; if the consistency check succeeds, executing step 505;

[0180] Step 504: completing the wheel according to the set strategy.

[0181] Step 505: Match the wheel size of each of the multiple wheels with the wheel template size in the template pool; if the matching result meets the set threshold, execute step 506; if the matching result does not meet the set threshold, execute step 512:

[0182] Step 506: Obtaining coordinate information of wheels that meet a set threshold in pixel coordinates;

[0183] Step 507: Calculate the distance between each pair of wheels based on the coordinate information;

[0184] Step 508: Select a pair of wheels with the largest distance;

[0185] Step 509: Acquire the contact position of the wheel pair with the largest distance in the target obstacle image;

[0186] Step 510: Mapping the ground contact position to a road surface position in a world coordinate system using an inverse perspective transformation formula;

[0187] Step 511: Determine the heading angle of the vehicle based on the road surface position in the world coordinate system.

[0188] Step 512: End this operation.

[0189] In the embodiment of the present invention, the effect of obtaining the vehicle orientation through wheel detection in the side camera is very robust. Therefore, the embodiment of the present invention can effectively improve the detection accuracy of the vehicle orientation angle in the detection area set near the main vehicle.

[0190] See also Figure 6 , is another flow chart of a method for determining an obstacle orientation provided by an embodiment of the present invention, the method comprising:

[0191] Step 601: Determine a first heading angle of a current vehicle in a detection area, where the first heading angle is a heading angle of the vehicle determined based on a wheel pair in an image of a target obstacle in the detection area.

[0192] Among them, in this step, the process of determining the first heading angle includes: the vehicle's main control platform or vehicle computer obtains a target obstacle image detected in the detection area, and the target obstacle image includes multiple wheels; when the multiple wheels are wheels of the same vehicle, determines the wheel pair with the largest distance between the multiple wheels; obtains the ground contact position of the wheel pair with the largest distance in the target obstacle image; maps the ground contact position to a road surface position in a world coordinate system; and determines the vehicle's heading angle, i.e., the first heading angle, based on the road surface position in the world coordinate system.

[0193] It should be noted that the specific implementation process of each step of determining the first orientation angle by the vehicle's main control platform or vehicle computer is detailed in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0194] Step 602: Obtain a second orientation angle of the current vehicle detected within the detection area output by the 3D obstacle detection model;

[0195] In this step, the vehicle's main control platform or vehicle computer can directly obtain the second orientation angle of the vehicle in the current detection area through the 3D obstacle detection model. The specific process of obtaining the second orientation angle is well known to those skilled in the art and will not be repeated here.

[0196] Step 603: Determine the difference between the first orientation angle and the second orientation angle;

[0197] In this step, the vehicle's main control platform or vehicle computer calculates the difference between the first orientation angle and the second orientation angle using a calculation formula.

[0198] Step 604: If the difference is less than a preset threshold, determining an average value of the first orientation angle and the second orientation angle;

[0199] In this embodiment, the preset threshold is a hyperparameter Thres, and different values ​​are selected for the hyperparameter according to the selection performance of different 3D obstacle detection models.

[0200] Step 605: Adjust the angle average to the current vehicle's heading angle.

[0201] In this step, the vehicle's main control platform or vehicle computer adjusts the angle average value to the current vehicle's heading angle.

[0202] In an embodiment of the present invention, a first heading angle of the current vehicle within a detection area is determined, and a second heading angle of the current vehicle detected within the detection area, as output by a 3D obstacle detection model, is obtained; the difference between the first and second heading angles is determined; if the difference is less than a preset threshold, an average of the first and second heading angles is determined; and the average is adjusted to the heading angle of the current vehicle. In an embodiment of the present invention, the heading angle of the obstacle vehicle determined based on the wheel is integrated with the heading angle of the obstacle vehicle output by the 3D obstacle detection model to adaptively adjust the heading angle of the obstacle vehicle and improve the accuracy of the vehicle's heading angle.

[0203] See also Figure 7 , is another flow chart of a method for determining an obstacle orientation provided by an embodiment of the present invention, the method comprising:

[0204] Step 701: Determine a first heading angle of a current vehicle in a detection area, where the first heading angle is a heading angle of the vehicle determined based on a wheel pair in a target obstacle image in the detection area.

[0205] Step 702: Obtain a second orientation angle of the current vehicle detected within the detection area output by the 3D obstacle detection model.

[0206] Step 703: Determine the difference between the first orientation angle and the second orientation angle.

[0207] Step 704: Determine whether the difference is less than a preset threshold. If so, execute steps 705 and 706; otherwise, execute steps 707 to 709.

[0208] Step 705: Determine the average value of the first orientation angle and the second orientation angle.

[0209] Step 706: Determine the angle average as the heading angle of the current vehicle, and end this operation.

[0210] Step 707: Obtain the historical heading angle of the current vehicle.

[0211] Step 708: Perform curve fitting on the historical heading angles using a random sampling consistency (RANSAC) verification algorithm to predict the third heading angle of the current vehicle.

[0212] In this step, the random sampling consistency (RANSAC) verification algorithm usually selects a first degree curve, that is, a straight line. This is because under normal driving conditions, the vehicle's orientation is relatively fixed. Even when changing lanes, the vehicle's orientation will not change much. Therefore, the vehicle's orientation is stable on a first degree curve. Even if the vehicle turns around, it can be understood as a uniform angular velocity state, and its curve is still a first degree curve. In an embodiment of the present invention, the historical orientation angle of the current vehicle is judged by a random sampling consistency check, which can clearly reflect the predicted value of the historical orientation angle and the current angle, called the third orientation angle. Afterwards, by comparing the third orientation angle with the first orientation angle or the second orientation angle, whichever orientation angle is closest, the closest orientation angle is determined as the orientation angle of the current vehicle. Its purpose is to avoid serious distortion between the determined orientation angle and the orientation angle predicted by the model, thereby affecting the final detection result.

[0213] Step 709: Determine the first orientation angle or the second orientation angle closest to the third orientation angle as the orientation angle of the current vehicle.

[0214] The embodiment of the present invention provides a method for comparing a determined vehicle heading angle with a vehicle heading angle predicted by a model, and fusing the comparison results according to a fusion strategy, thereby improving the accuracy of the vehicle heading angle.

[0215] It should be noted that for the method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0216] See also Figure 8 , is a block diagram of an obstacle orientation determination device provided by an embodiment of the present invention. The device includes: a first acquisition module 801, a first determination module 802, a second acquisition module 803, a mapping module 804 and a second determination module 805, wherein,

[0217] The first acquisition module 801 is configured to acquire a target obstacle image detected in a detection area, wherein the target obstacle image includes a plurality of wheels;

[0218] The first determining module 802 is configured to determine a wheel pair having the largest distance between the multiple wheels when the multiple wheels are wheels of the same vehicle;

[0219] The second acquisition module 803 is configured to acquire the contact position of the wheel pair with the largest distance in the target obstacle image;

[0220] The mapping module 804 is used to map the ground contact position to a road surface position in a world coordinate system;

[0221] The second determining module 805 is configured to determine the heading angle of the vehicle based on the road surface position in the world coordinate system.

[0222] Optionally, in another embodiment, based on the above embodiment, the first determination module includes: a first matching module, a third acquisition module, a first calculation module and a first selection module, wherein:

[0223] The first matching module is configured to match the size of each wheel in the plurality of wheels with the size of a wheel template in the template pool;

[0224] The third acquisition module is configured to acquire, when the matching result of the first matching module meets a set threshold, coordinate information of the wheel that meets the set threshold in pixel coordinates;

[0225] The first calculation module is used to calculate the distance between each pair of wheels according to the coordinate information;

[0226] The first selection module is used to select a pair of wheels with the largest distance between them.

[0227] Optionally, in another embodiment, based on the above embodiment, the device further includes: a verification module and a policy completion module, wherein:

[0228] The verification module is used to perform consistency verification on the dimensions of the multiple wheels when the multiple wheels are wheels of the same vehicle;

[0229] The first determining module is further configured to determine the wheel pair having the largest distance between the plurality of wheels when the consistency check by the checking module succeeds;

[0230] The strategy completion module is used to complete the wheel according to the set strategy when the consistency check of the verification module fails.

[0231] Optionally, in another embodiment, based on the above embodiment, the device further includes: a first judgment module, wherein:

[0232] The first judgment module is configured to judge whether each of the plurality of wheels has a wheel truncation condition before the verification module performs consistency verification on the dimensions of the plurality of wheels;

[0233] The first determining module is further configured to determine a wheel pair having the largest distance between the multiple wheels when the first judging module determines that no wheel truncation occurs.

[0234] The completion module is further configured to complete the wheel according to a set strategy when the first judgment module determines that a wheel is cut off.

[0235] Optionally, in another embodiment, based on the above embodiment, the completion module includes: a second selection module, a second matching module and a first completion module; and / or a third selection module and a second completion module; and / or a fourth selection module and a third completion module; wherein,

[0236] The second selection module is used to select the truncated wheel with the largest area;

[0237] The second matching module is used to match the truncated wheel with the largest area with a wheel template in a template pool;

[0238] The first completion module is configured to complete all truncated wheels according to the wheel template matched by the second matching module;

[0239] The third selection module is configured to select a complete wheel from the plurality of wheels;

[0240] The second completing module is configured to complete all truncated wheels according to the complete wheels selected by the third selecting module;

[0241] The fourth selection module is configured to select two complete wheels from the plurality of wheels;

[0242] The second calculation module is used to calculate the average wheel area of ​​the two complete wheels selected by the fourth selection module, and complete all the truncated wheels according to the average wheel area.

[0243] Optionally, in another embodiment, based on the above embodiment, the mapping module is specifically configured to map the ground contact position to a road surface position in a world coordinate system through an inverse perspective transformation formula.

[0244] Optionally, in another embodiment, based on the above embodiment, the second determination module includes: a third calculation module and a direction angle determination module, wherein:

[0245] The third calculation module is used to calculate the wheel line of the road surface position of the wheel pair in the world coordinate system;

[0246] The orientation angle determination module is used to determine the orientation of the wheel line around the direction of gravity as the orientation angle of the vehicle.

[0247] See also Figure 9 , is another block diagram of an obstacle direction determination device provided by an embodiment of the present invention, the device comprising: a first determination module 901, an acquisition module 902, a second determination module 903, a third determination module 904 and a fourth determination module 905, wherein,

[0248] The first determining module 901 is configured to determine a first heading angle of the current vehicle in the detection area, where the first heading angle is determined based on the wheel pairs in the target obstacle image in the detection area;

[0249] The acquisition module 902 is used to obtain a second orientation angle of the current vehicle detected in the detection area output by the 3D obstacle detection model;

[0250] The second determining module 903 is configured to determine a difference between the first orientation angle and the second orientation angle;

[0251] The third determining module 904 is configured to determine an average of the first orientation angle and the second orientation angle when the difference is less than a preset threshold;

[0252] The fourth determining module 905 is configured to determine the angle average as the heading angle of the current vehicle.

[0253] Optionally, in another embodiment, based on the above embodiment, the device may further include: a second acquisition module, a fitting module and a fifth determination module, wherein:

[0254] The second acquisition module is configured to acquire a historical heading angle of the current vehicle when the difference is not less than a preset threshold;

[0255] The fitting module is used to perform curve fitting on the historical heading angles by using a random sampling consistency RANSAC verification algorithm to predict the third heading angle of the current vehicle;

[0256] The fifth determining module is configured to determine the first orientation angle or the second orientation angle that is closest to the third orientation angle as the orientation angle of the current vehicle.

[0257] Optionally, in another embodiment, based on the above embodiment, the first determining module includes:

[0258] a third acquisition module, configured to acquire a target obstacle image detected in the detection area, wherein the target obstacle image includes a plurality of wheels;

[0259] a sixth determining module, configured to determine, when the plurality of wheels are wheels of the same vehicle, a wheel pair having a largest distance between the plurality of wheels;

[0260] A fourth acquisition module is configured to acquire the ground contact position of the wheel pair with the largest distance in the target obstacle image;

[0261] A mapping module, configured to map the ground contact position to a road surface position in a world coordinate system;

[0262] A seventh determination module is configured to determine a first heading angle of the vehicle based on a road surface position in the world coordinate system.

[0263] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0264] See also Figure 10 An embodiment of the present invention provides a system for determining the direction of an obstacle. The system is based on a 3D obstacle detection network 1000 and includes a 2D obstacle detection module 1001 and a parameter transformation module 1002. The 2D obstacle detection module 1001 and the parameter transformation module 1002 can be located in a 2D detection network, which can also be called a wheel detection network.

[0265] The 2D obstacle detection module 1001 is configured to detect the image decoded by the decoder in the 3D obstacle detection network 1000 and obtain a target obstacle image in the detection area, where the target obstacle image includes multiple wheels. If the multiple wheels are wheels of the same vehicle, the module determines the wheel pair with the largest distance between the multiple wheels and obtains the contact position of the wheel pair with the largest distance in the target obstacle image.

[0266] The parameter transformation module 1002 is configured to map the touchdown position to a road surface position in a world coordinate system in the 3D obstacle detection network; and determine the vehicle's heading angle based on the road surface position in the world coordinate system.

[0267] Among them, the 3D obstacle detection network 1000 is used to detect 3D objects. The goal of detecting 3D objects is usually to find all objects of interest in the scene based on point cloud data, such as vehicles, pedestrians, static obstacles, etc. in the autonomous driving scene. It can include but is not limited to the following modules: image module, feature (backbone) module, decoder, 3D obstacle detection module and 3D rectangular box (3D BBox). The above modules are connected in sequence; among them, 3D rectangular box (3D BBox, 3DBoundingBox), each 3D rectangular box corresponds to an object in the scene. 3D BBox can be represented in many ways. Generally, the most commonly used method is to use the 3D coordinates of the center point, length, width and height, and 3D rotation angle to represent it. For a simpler way, only in-plane rotation can be considered. The 2D obstacle detection network (i.e., wheel detection network) 1003 includes: a 2D obstacle detection module 1001 and a parameter transformation module 1002, wherein the 2D obstacle detection module 1001 is used to detect the image decoded by the decoder in the 3D obstacle detection network, and obtain the target obstacle image of the detection area, wherein the target obstacle image includes multiple wheels; when the multiple wheels are wheels of the same vehicle, determine the wheel pair with the largest distance between the multiple wheels; obtain the ground contact position of the wheel pair with the largest distance in the target obstacle image; the parameter transformation module 1002 is used to map the ground contact position to the road surface position in the world coordinate system of the 3D obstacle detection network, i.e., 3D BBox; determine the orientation angle of the vehicle based on the road surface position in the world coordinate system. The corresponding block diagram is as follows: Figure 10A As shown, Figure 10A This is an application block diagram of a system for determining the direction of an obstacle according to an embodiment of the present invention.

[0268] The 2D obstacle detection network (i.e., wheel detection network) provided in this embodiment is connected after the decoder module in the 3D obstacle detection network. The 2D obstacle detection network can use the simplest YOLO network. Afterwards, the 2D obstacle detection module in the 2D obstacle detection network can detect the image decoded by the decoder, obtain multiple wheels in the target obstacle image of the detection area, and the wheel detection frame of each wheel. The wheel detection frames of the same vehicle can be sorted first, and the distance between any two wheels belonging to the same vehicle can be calculated. The distance is sorted in descending order of distance, and the pair of wheels with a farther distance is found according to the ground position of the vehicle; the parameter transformation module 1002 obtains the orientation angle of the vehicle in the world coordinate system through the inverse perspective transformation formula; at the same time, it is also necessary to determine whether the vehicle in each wheel frame is truncated. If the wheel is truncated, the truncated wheel is completed according to the strategy. This improves the detection accuracy of the vehicle orientation angle in the detection area near the main vehicle.

[0269] Optionally, an embodiment of the present invention further provides an electronic device, including:

[0270] processor;

[0271] a memory for storing instructions executable by the processor;

[0272] The processor is configured to execute the instructions to implement the method for determining the direction of the obstacle as described above.

[0273] Optionally, an embodiment of the present invention further provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can perform the above-described method for determining the direction of an obstacle. Optionally, the computer-readable storage medium can be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0274] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-mentioned method for determining the direction of an obstacle.

[0275] Optionally, an embodiment of the present invention further provides an electronic device, such as Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104, wherein,

[0276] The memory 1103 is used to store computer programs;

[0277] The processor 1101 is configured to implement the above-mentioned method for determining the direction of an obstacle when executing the program stored in the memory 1103 .

[0278] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0279] The communication interface is used for communication between the above terminal and other devices.

[0280] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0281] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0282] In an embodiment, the electronic device 1101 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the method for determining the obstacle orientation shown above.

[0283] In an embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1103 including instructions. The instructions can be executed by the processor 1101 of the electronic device to implement the above-described method for determining the direction of an obstacle. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0284] In an embodiment, a computer program product is further provided. When instructions in the computer program product are executed by the processor 1101 of an electronic device, the electronic device executes the above-mentioned method for determining the direction of an obstacle.

[0285] Figure 12 1 is a block diagram of an apparatus 1200 for determining an obstacle direction according to an embodiment of the present invention. For example, the apparatus 1200 may be provided as a server. Figure 12The apparatus 1200 includes a processing component 1222, which further includes one or more processors, and memory resources represented by a memory 1232 for storing instructions, such as applications, that can be executed by the processing component 1222. The application stored in the memory 1232 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1222 is configured to execute the instructions to perform the above-described method.

[0286] The device 1200 may also include a power supply component 1226 configured to perform power management of the device 1200, a wired or wireless network interface 1250 configured to connect the device 1200 to a network, and an input / output (I / O) interface 1258. The device 1200 may operate based on an operating system stored in the memory 1232, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0287] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0288] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for determining the direction of an obstacle, characterized in that: include: Acquire a target obstacle image detected in a detection area, where the target obstacle image includes a plurality of wheels; When the plurality of wheels are wheels of the same vehicle, determining a wheel pair having a largest distance between the plurality of wheels; Obtaining the ground contact position of the wheel pair with the largest distance in the target obstacle image; Mapping the ground contact position to a road surface position in a world coordinate system; Determining a heading angle of the vehicle based on a road surface position in the world coordinate system; When the plurality of wheels are wheels of the same vehicle, the method further comprises: performing consistency checking on the dimensions of the plurality of wheels; If the consistency check succeeds, performing the step of determining the wheel pair having the largest distance between the plurality of wheels; If the consistency check fails, the wheel is completed according to the set strategy; Before performing consistency check on the dimensions of the plurality of wheels, the method includes: determining whether each of the plurality of wheels has a wheel truncation condition; The determining whether each wheel among the plurality of wheels is in a wheel truncation state includes: Determining the diameters and areas of the multiple wheels, taking the wheel with the largest diameter and the largest area as a complete wheel, and the remaining wheels as wheels to be determined; For each wheel to be judged, respectively calculating the difference between its diameter and area and the diameter and area of ​​the complete wheel; If the difference exceeds a preset threshold, the wheel to be judged is determined to be a truncated wheel; Determining a first heading angle of a current vehicle within the detection area, the first heading angle being determined based on a wheel pair in an image of a target obstacle in the detection area; Obtain the second orientation angle of the current vehicle detected within the detection area output by the 3D obstacle detection model; determining a difference between the first orientation angle and the second orientation angle; If the difference is less than a preset threshold, determining an angle average of the first orientation angle and the second orientation angle; Determining the angle average as the heading angle of the current vehicle; If the difference between the first heading angle and the second heading angle is not less than a preset threshold, obtaining a historical heading angle of the current vehicle; Performing curve fitting on the historical heading angles using a random sampling consistency check algorithm to predict a third heading angle of the current vehicle; The first orientation angle or the second orientation angle closest to the third orientation angle is determined as the orientation angle of the current vehicle.

2. The method for determining the direction of an obstacle according to claim 1, wherein: Determining the wheel pair with the largest distance between the multiple wheels comprises: Matching the size of each wheel in the plurality of wheels with the size of a wheel template in a template pool; If the matching result meets the set threshold, the coordinate information of the wheel meeting the set threshold in pixel coordinates is obtained; Calculate the distance between each pair of wheels according to the coordinate information; Select the pair of wheels with the largest distance between them.

3. The method for determining the direction of an obstacle according to claim 1, wherein: After determining whether each of the plurality of wheels has a wheel truncation condition, the method further includes: If there is no wheel truncation, then executing the step of verifying the consistency of all wheel sizes; If there is a wheel cutoff, the wheel is completed according to the set strategy.

4. The method for determining the direction of an obstacle according to claim 1, wherein: The completing the wheel according to the set strategy includes: Selecting the truncated wheel with the largest area, matching the truncated wheel with the wheel template in the template pool; completing all truncated wheels according to the matched wheel template; or If a complete wheel is selected from the plurality of wheels, then all the truncated wheels are completed according to the selected complete wheel; or If two complete wheels are selected from the plurality of wheels, an average value of the wheel areas of the two complete wheels is calculated, and all the truncated wheels are completed based on the average value.

5. The method for determining the direction of an obstacle according to any one of claims 1 to 4, characterized in that: Mapping the ground contact position to a road surface position in a world coordinate system includes: The ground contact position is mapped to a road surface position in a world coordinate system by using an inverse perspective transformation formula.

6. The method for determining the direction of an obstacle according to any one of claims 1 to 4, characterized in that: Determining the heading angle of the vehicle based on the road surface position in the world coordinate system includes: Calculating a wheel line of a road surface position of the wheel pair in a world coordinate system; The orientation of the wheel line around the direction of gravity is determined as the orientation angle of the vehicle.

Images