Angle determination method and device and electronic equipment

Abstract

The invention discloses an angle determination method and device and electronic equipment. The method comprises the steps that a target image is acquired, and the target image comprises a target object; according to the target image, determining a three-dimensional coordinate of the target object in a predetermined three-dimensional coordinate system, an initial angle of the target object relative to a target reference object, and target parameters related to the target object, the predetermined three-dimensional coordinate system being a coordinate system constructed with the target reference object as an original point; determining a weight coefficient corresponding to the three-dimensional coordinate according to the target parameter; and determining a target angle of the target object relative to the target reference object according to the three-dimensional coordinate, the weight coefficient and the initial angle. According to the invention, the technical problem that the determined angle of the target object relative to the target reference object is inaccurate in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of angle prediction, and in particular to an angle determination method, device and electronic equipment. Background Art

[0002] In unmanned driving technology, the input data of various sensors needs to be processed by the perception module, and the real-time description of the surrounding environment obtained after processing is then input into the planner for behavioral decision-making and planning. Among them, the perception module plays a key role in the unmanned driving system. How to better optimize the algorithm of the perception module has become one of the keys to developing and promoting the progress of unmanned driving technology. In the monocular 3D detection task of the perception module, the input is an RGB image. The algorithm needs to detect all vehicles, pedestrians, cyclists and other targets in the image, and the output is the actual position information, size information and direction information of the target. Among them, the direction information plays a very important role in the detection effect of the target. How to improve the prediction effect of the direction angle has become one of the keys to improve the overall detection effect of the monocular 3D detection task.

[0003] Traditional direction angle prediction generally adopts a direct regression solution, in which the features extracted by the backbone network are directly input into a detection head to regress the observation angle of the corresponding target. The regressed observation angle is then combined with the position information to obtain the final global direction angle. This direct regression solution has the technical problem of inaccurately predicted angles.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide an angle determination method, device and electronic device to at least solve the technical problem in the related art that the angle of a target object determined relative to a target reference object is inaccurate.

[0006] According to one aspect of an embodiment of the present invention, there is provided an angle determination method, comprising: acquiring a target image, wherein the target image includes a target object; determining, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object, wherein the predetermined three-dimensional coordinate system is a coordinate system constructed with the target reference object as the origin; determining, based on the target parameters, a weight coefficient corresponding to the three-dimensional coordinates; and determining, based on the three-dimensional coordinates, the weight coefficient, and the initial angle, a target angle of the target object relative to the target reference object.

[0007] Optionally, determining the weight coefficient corresponding to the three-dimensional coordinates based on the target parameters includes: when the target parameters include a target orientation, determining an accuracy index of the target orientation; and determining the weight coefficient corresponding to the three-dimensional coordinates based on the accuracy index, wherein the accuracy index is proportional to the weight coefficient.

[0008] Optionally, determining the target angle of the target object relative to the target reference object based on the three-dimensional coordinates, the weight coefficient, and the initial angle includes: inverse tangent the x-axis coordinate and the z-axis coordinate in the three-dimensional coordinates to obtain a target inverse tangent value; determining the product of the target inverse tangent value and the weight coefficient; and determining the sum of the product and the initial angle as the target angle.

[0009] Optionally, determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object based on the target image includes: extracting image features of the target image; determining a target anchor frame that selects the target object in the target image based on the image features; determining the three-dimensional coordinates corresponding to a center point of the target anchor frame, the initial angle of the center point relative to the target reference object, and the target parameters related to the target object in the target anchor frame.

[0010] Optionally, before using the target model to perform the following steps: determining, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object, it also includes: obtaining an initial model and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is a loss function term of a three-dimensional coordinate estimation branch, the second loss function term is a loss function term of an angle estimation branch, and the third loss function term is a loss function term of a target parameter estimation branch; based on the loss function, the initial model is trained using sample data to obtain the target model.

[0011] Optionally, before determining the initial angle of the target object relative to the target reference object based on the target image, the method further includes: determining target probabilities corresponding to multiple target angle intervals respectively based on the target image, wherein the corresponding target probabilities represent the probabilities that the predicted angle belongs to the corresponding target angle interval, the predicted angle is the angle of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals respectively is 1, and the target probabilities corresponding to the multiple target angle intervals respectively are not 1; determining the initial angle based on the target probabilities corresponding to the multiple target angle intervals respectively.

[0012] Optionally, determining the weight coefficient corresponding to the three-dimensional coordinate based on the target parameter includes: when there are multiple target parameters, determining sub-weight coefficients corresponding to the multiple target parameters respectively; determining the weight coefficient corresponding to the three-dimensional coordinate based on the multiple target parameters and the sub-weight coefficients corresponding to the multiple target parameters respectively.

[0013] According to one aspect of an embodiment of the present invention, there is provided an angle determination device, comprising: an acquisition module, for acquiring a target image, wherein the target image includes a target object; a first determination module, for determining, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object, wherein the three-dimensional coordinate system based on the three-dimensional coordinates is a coordinate system constructed with the target reference object as the origin; a second determination module, for determining, based on the target parameters, a weight coefficient corresponding to the three-dimensional coordinates; and a third determination module, for determining, based on the three-dimensional coordinates, the weight coefficient, and the initial angle, a target angle of the target object relative to the target reference object.

[0014] According to one aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the angle determination methods described above.

[0015] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned angle determination methods.

[0016] In an embodiment of the present invention, a target image including a target object is obtained. Based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object are determined. Therefore, the weight coefficient corresponding to the three-dimensional coordinates can be determined based on the target parameters. Finally, the purpose of determining the target angle of the target object relative to the target reference object based on the three-dimensional coordinates, the weight coefficient, and the initial angle can be achieved. Since the weight coefficient is taken into account when determining the target angle of the target object relative to the target reference object, the position information, that is, the importance of the three-dimensional coordinates, is taken into account, therefore, the determined target angle can be made more accurate, thereby solving the technical problem of inaccurate angle of the target object determined relative to the target reference object in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flow chart of an angle determination method according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of an angle error indicator based on a wide-angle camera provided in an optional embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an angle error index based on a wide-angle camera after introducing a weight coefficient provided in an optional embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of an average precision AP indicator based on a wide-angle camera provided by an optional embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of an AP index based on a wide-angle camera after introducing a weight coefficient provided in an optional embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of an angle error indicator based on a telephoto camera provided by an optional embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of an angle error index based on a telephoto camera after introducing a weight coefficient provided in an optional embodiment of the present invention;

[0025] Figure 8 is a schematic diagram of an AP indicator based on a telephoto camera provided by an optional embodiment of the present invention;

[0026] Fig. 9is a schematic diagram of an AP index based on a telephoto camera after introducing a weight coefficient provided in an optional embodiment of the present invention;

[0027] Fig.10 is a structural block diagram of an angle determination device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Example 1

[0031] According to an embodiment of the present invention, an embodiment of an angle determination method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of an angle determination method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0033] Step S102, acquiring a target image, wherein the target image includes a target object;

[0034] The method provided in the present application can be applied to multiple scenarios, such as scenarios in the field of vehicle autonomous driving. In this scenario, the direction angle of the target object in the target image is predicted, and the positional relationship between the vehicle and the target object can be accurately determined, thereby achieving the effect of avoiding the target object and preventing traffic accidents.

[0035] When the method provided in this application is used in the field of target detection in autonomous driving, the target object may refer to a person or object that appears next to the vehicle during the driving process, and in a specific solution, may be a specified object. The target image is the image captured by the camera in the vehicle.

[0036] Step S104, determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and target parameters related to the target object based on the target image, wherein the predetermined three-dimensional coordinate system is a coordinate system constructed with the target reference object as the origin;

[0037] In step S104 provided in the present application, in the scenario of vehicle automatic driving, the target reference object is the vehicle, or the camera of the vehicle itself. It is used to determine the angle of the target object relative to the vehicle. It should be noted that the angle facing the front of the vehicle can be set as 0, the angle facing the rear of the vehicle can be set as 180, etc., which is not limited here, and the angle standard can be set according to the actual application and scenario.

[0038] In this step, based on the target image, the three-dimensional coordinates of the target object in the predetermined three-dimensional coordinate system can be determined by methods such as feature extraction and coordinate recognition. Since the predetermined three-dimensional coordinate system is a coordinate system constructed with the target reference object as the origin, the three-dimensional position of the target object relative to the target reference object can be known through the three-dimensional coordinates. Through the three-dimensional position of the target object relative to the target reference object, the angle of the target object relative to the target reference object can be known to a certain extent.

[0039] In this step, the initial angle of the target object relative to the target reference object can be determined based on the target image or by methods such as angle interval prediction based on feature extraction. That is, the angle interval of the target reference object where the target object is located can be directly predicted so as to preliminarily determine the initial angle of the target object relative to the target reference object.

[0040] In this step, target parameters related to the target object may also be determined based on the target image. For example, when the target parameter is the orientation of the target object, the orientation of the target object may be determined based on the target image.

[0041] Step S106, determining a weight coefficient corresponding to the three-dimensional coordinates according to the target parameters;

[0042] In step S106 provided in the present application, a certain weight coefficient is assigned to the three-dimensional coordinates according to the target parameters, that is, the target parameters are related to the three-dimensional coordinates, and the target parameters can reflect the importance of the three-dimensional coordinates to a certain extent, or can reflect the coefficient of the accuracy of the three-dimensional coordinates for the subsequent target angle judgment. Through this operation, the subsequent target angle can be determined more accurately.

[0043] Step S108, determining a target angle of the target object relative to the target reference object according to the three-dimensional coordinates, the weight coefficient, and the initial angle.

[0044] In step S108 provided in the present application, the target angle of the target object relative to the target reference object can be determined based on the three-dimensional coordinates, the weight coefficient, and the initial angle, so as to achieve the purpose of ultimately determining the target angle.

[0045] Through the above steps S102-S108, a target image including a target object is obtained. Based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object are determined. Therefore, the weight coefficient corresponding to the three-dimensional coordinates can be determined based on the target parameters. Finally, the purpose of determining the target angle of the target object relative to the target reference object based on the three-dimensional coordinates, the weight coefficient, and the initial angle can be achieved. Since the weight coefficient is taken into account when determining the target angle of the target object relative to the target reference object, the position information, that is, the importance of the three-dimensional coordinates, is taken into account, therefore, the determined target angle can be made more accurate, thereby solving the technical problem of inaccurate angle of the target object determined relative to the target reference object in the related art.

[0046] As an optional embodiment, determining a weight coefficient corresponding to the three-dimensional coordinates based on the target parameters includes: when the target parameters include the target orientation, determining an accuracy index of the target orientation; determining a weight coefficient corresponding to the three-dimensional coordinates based on the accuracy index, wherein the accuracy index is proportional to the weight coefficient.

[0047] In this embodiment, the case where the target parameter includes the target orientation of the target object is described. In this case, it is necessary to determine the accuracy index of the target orientation. For example, when the target orientation of the target object is determined to be rightward, the accuracy of the rightward orientation is determined to be how much, and the higher the accuracy index, the more accurate the orientation estimation is. Thus, the weight coefficient corresponding to the three-dimensional coordinates can be determined based on the accuracy index of the target orientation. That is, it is possible to determine the degree to which the three-dimensional coordinates are trustworthy when the target angle is determined based on the three-dimensional coordinates. When the target orientation is more accurate, it means that the position information is more accurate, so the weight coefficient corresponding to the three-dimensional coordinates is increased to illustrate that the three-dimensional coordinates are more trustworthy. When the target orientation is less accurate, it means that the position information is more inaccurate, and the weight coefficient corresponding to the three-dimensional coordinates should be reduced to illustrate that the three-dimensional coordinates are less trustworthy, so that the target angle is judged more based on the initial angle.

[0048] As an optional embodiment, the target angle of the target object relative to the target reference object is determined based on the three-dimensional coordinates, the weight coefficient, and the initial angle, including: obtaining the target inverse tangent value by the x-axis coordinate and the z-axis coordinate in the inverse tangent three-dimensional coordinates; determining the product of the target inverse tangent value and the weight coefficient; and determining the sum of the product and the initial angle as the target angle.

[0049] In this embodiment, a specific method of determining the target angle of the target object relative to the target reference object based on the three-dimensional coordinates, the weight coefficient, and the initial angle is described. The x-axis coordinate and the z-axis coordinate in the inverse tangent Q three-dimensional coordinate can be expressed by the following formula: The product of the target inverse tangent value and the weight coefficient can be expressed by the following formula: Q*alpha. The sum of the product and the initial angle can be expressed by the following formula: This allows for better determination of the target angle.

[0050] As an optional embodiment, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object are determined, including: extracting image features of the target image; determining a target anchor frame for selecting the target object in the target image based on the image features; determining the three-dimensional coordinates corresponding to the center point of the target anchor frame, the initial angle of the center point relative to the target reference object, and the target parameters related to the target object in the target anchor frame.

[0051] In this embodiment, the image features of the target image are extracted. The image features may be features of different dimensions, such as color dimensions, etc., which are not limited here and can be customized according to actual applications and scenarios. Based on the image features, a target anchor frame is determined, and the target anchor frame can determine and select the target object in the target image. By determining the three-dimensional coordinates corresponding to the center point of the target anchor frame, the initial angle of the center point relative to the target reference object, and the target parameters related to the target object in the target anchor frame, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object can be more accurately determined, thereby achieving the effect of accurately determining the data.

[0052] As an optional embodiment, before using the target model to perform the following steps: determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object based on the target image, it also includes: obtaining an initial model and constructing a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, the first loss function term is a loss function term of a three-dimensional coordinate estimation branch, the second loss function term is a loss function term of an angle estimation branch, and the third loss function term is a loss function term of a target parameter estimation branch; based on the loss function, the initial model is trained using sample data to obtain a target model.

[0053] In this embodiment, it is explained that when the execution step: determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system based on the target image, the initial angle of the target object relative to the target reference object, and the target parameters related to the target object are the pre-steps when the target model is used for execution. When the model is used to execute the above steps, the pre-step of training the initial model with sample data is involved. The pre-steps involved in this embodiment also include a loss function. That is, the target model involved in this embodiment is obtained by training the initial model with sample data based on the loss function.

[0054] Since the loss function includes the loss function terms of the three-dimensional coordinate estimation branch, the loss function terms of the angle estimation branch, and the loss function terms of the target parameter estimation branch, it is possible to reduce the error between the predicted three-dimensional coordinates and the true three-dimensional coordinates, reduce the error between the initial angle and the actual angle, and reduce the error between the target parameters and the actual parameters, making the obtained data more accurate.

[0055] As an optional embodiment, before determining the initial angle of the target object relative to the target reference object based on the target image, it also includes: determining the target probabilities corresponding to multiple target angle intervals respectively based on the target image, wherein the corresponding target probability represents the probability that the predicted angle belongs to the corresponding target angle interval, the predicted angle is the angle of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals is 1, and the target probabilities corresponding to the multiple target angle intervals are not 1; determining the initial angle based on the target probabilities corresponding to the multiple target angle intervals.

[0056] In this embodiment, the predicted angle is the angle of the target object relative to the target reference object. In this step, the target probabilities that the predicted angles belong to multiple target angles are determined based on the target image. For example, the predicted angles belong to the target probabilities between 0-10 degrees, 10 degrees-20 degrees...350 degrees-360 degrees. Moreover, the target probability distribution corresponding to the target probabilities belonging to multiple target angles conforms to the Gaussian distribution. It is ensured that the sum of the probabilities belonging to all angle intervals is 1. The problem of inaccurate estimation caused by the sum of the probabilities not being 1 is avoided.

[0057] When determining the initial angle of the target object relative to the target reference object based on the target probabilities that the predicted angles belong to multiple target angles, the initial angle can be determined in a variety of ways. For example, the angle interval corresponding to the maximum probability can be determined from multiple target probabilities, and the final angle can be determined from the angle interval. The final angle can also be obtained by multiplying the multiple target angles with the corresponding probabilities. Specific limitations can be made here based on actual applications and scenarios. Or a trade-off can be made based on the required accuracy and speed. Determine the optimal way to determine the angle for a specific scenario.

[0058] As an optional embodiment, the weight coefficient corresponding to the three-dimensional coordinates is determined based on the target parameters, including: when there are multiple target parameters, determining the sub-weight coefficients corresponding to the multiple target parameters respectively; determining the weight coefficient corresponding to the three-dimensional coordinates based on the multiple target parameters and the sub-weight coefficients corresponding to the multiple target parameters respectively.

[0059] In this embodiment, the case where there are multiple target parameters is described. For example, when the target parameters include the target orientation of the target object, the two-dimensional coordinates of the target object in the target image, and the type of the target object, the sub-weight coefficients corresponding to the target orientation, the two-dimensional coordinates, and the type of the target object can be determined respectively. The weight coefficient corresponding to the three-dimensional coordinate can be determined based on the target orientation, the two-dimensional coordinates, and the type of the target object, and the corresponding weight coefficients.

[0060] It should be noted that when the target parameter includes the target orientation, the more accurate the target orientation is, the higher the weight of the three-dimensional coordinate is. When the target parameter includes the two-dimensional coordinate, the closer the two-dimensional coordinate is to the center of the image, the higher the weight coefficient of the three-dimensional coordinate is. When the target parameter includes the type of the target object, the weight coefficient of the three-dimensional coordinate corresponding to the type can be retrieved. In order to determine the final weight coefficient corresponding to the three-dimensional coordinate based on multiple target parameters.

[0061] By setting multiple target parameters, the weight coefficients of the three-dimensional coordinates are considered in various situations, so that the method provided by the present application is more comprehensive and the target angle finally determined is more accurate.

[0062] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail below.

[0063] In related technologies, the direction angle prediction usually involves first learning the observation angle (alpha) in the network, and then converting it into the final direction angle (Yaw) in combination with the target's position information. Existing observation angle prediction generally uses a direct regression solution, and some use a multi-angle interval multibins classification + regression solution, where multibins divides 360 degrees into multiple bins for observation angle classification, and performs residual angle regression on the bin to which the observation angle belongs. However, when the direction angle is determined using the method in related technologies, there will be a technical problem that the predicted direction angle is inaccurate.

[0064] In view of this, an optional implementation of the present invention provides an angle determination method that can accurately predict the direction angle.

[0065] S1, acquiring a target image, wherein the target image includes a target object, such as a bicycle;

[0066] S2, extracting image features of the target image;

[0067] S3, determining a target anchor frame for selecting the bicycle in the target image based on the image features;

[0068] S4, determining the three-dimensional coordinates corresponding to the center point of the target anchor frame, as well as the initial angle of the center point relative to the ego vehicle camera (such as the ego vehicle camera), and target parameters related to the bicycle in the target anchor frame, such as the target orientation;

[0069] It should be noted that when determining the initial angle of the center point relative to the vehicle camera, it can be determined in the following way:

[0070] S4.1, determining target probabilities corresponding to a plurality of target angle intervals according to the target image, wherein the corresponding target probability represents the probability that the predicted angle belongs to the corresponding target angle interval, the predicted angle is the angle of the bicycle relative to the camera of the vehicle, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not 1;

[0071] S4.2, determining an initial angle according to target probabilities corresponding to a plurality of target angle intervals.

[0072] S5, determine the accurate index of target orientation;

[0073] S6, determining a weight coefficient corresponding to the three-dimensional coordinate according to the accuracy index, wherein the accuracy index is proportional to the weight coefficient;

[0074] S7, the horizontal coordinate and the vertical coordinate in the arctangent three-dimensional coordinate, and obtain the target arctangent value;

[0075] S8, determining the product of the target arctangent value and the weight coefficient;

[0076] It should be noted that, as can be seen here, the method provided in the optional implementation mode of the present invention does not simply couple the predicted observation angle and the target position directly, but allows the network to learn the relationship between the observation angle and the target position at the same time, and assigns a confidence level to the position information predicted by the network to indicate the reliability of the position information predicted by the network, thereby determining the degree of dependence on the predicted position information. The predicted confidence weight coefficient ranges from 0 to 1, and the smaller the weight coefficient, the lower the degree of dependence on the position information. By allowing the network to learn the confidence weight coefficient of the target position information, the network can reduce its dependence on the target position information, thereby reducing the error superposition caused by the position error to the angular prediction. On the other hand, it can also allow the network to better learn the observation angle and adaptively adjust the degree of dependence on the position information.

[0077] S9, determining the sum of the product and the initial angle as the target angle.

[0078] The method provided by the optional implementation mode of the present invention has experimentally obtained the result that the weight of determining the position information can affect the accuracy of the final determined direction angle. The solutions in the above-mentioned related technologies also have a common problem, that is, they do not consider the influence of the position information. If the position prediction of the target is biased, then the prediction of the final direction angle will also produce a large error. The observation angle predicted by the network and the position information are simply coupled, ignoring the implicit relationship between the observation angle and the position. On the one hand, it will introduce errors caused by the position prediction, and on the other hand, it will also affect the learning of the observation angle. These problems will affect the final direction angle prediction.

[0079] It can be seen that the traditional azimuth prediction scheme has the problem of being closely coupled with the target position information. The method provided by the optional implementation of the present invention introduces a confidence weight coefficient, learns the relationship between the observation angle and the position, and allows the network to adaptively learn the degree of dependence on the target position, which can reduce the error caused by the position prediction error on the final azimuth prediction, and also allow the network to better learn the observation angle. Therefore, through the above optional implementation, the beneficial effect of accurately predicting the azimuth can be achieved.

[0080] The experimental data obtained after using the optional embodiment of the present invention is shown in the figure. Figure 2 is a schematic diagram of an angle error indicator based on a wide-angle camera provided in an optional embodiment of the present invention, Figure 3 is a schematic diagram of an angle error index based on a wide-angle camera after introducing a weight coefficient provided in an optional embodiment of the present invention, Figure 4 is a schematic diagram of an AP indicator based on a wide-angle camera provided by an optional embodiment of the present invention, Figure 5 is a schematic diagram of an AP index based on a wide-angle camera after introducing a weight coefficient provided in an optional embodiment of the present invention, Figure 6 is a schematic diagram of an angle error indicator based on a telephoto camera provided in an optional embodiment of the present invention, Figure 7 is a schematic diagram of an angle error index based on a telephoto camera after introducing a weight coefficient provided in an optional embodiment of the present invention, Figure 8 is a schematic diagram of an AP indicator based on a telephoto camera provided by an optional embodiment of the present invention, Fig. 9 is a schematic diagram of the AP index based on the telephoto camera after introducing the weight coefficient provided by an optional embodiment of the present invention. Figure 2-9 As shown, regardless of whether the vehicle camera is a telephoto camera or a wide-angle camera, after using the method provided by the optional embodiment of the present invention, the angle error is significantly reduced, the prediction accuracy is significantly improved, and at least the beneficial effects shown above can be achieved.

[0081] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0083] Example 2

[0084] According to an embodiment of the present invention, a device for implementing the above angle determination method is also provided. Fig.10 is a structural block diagram of an angle determination device according to an embodiment of the present invention. Fig.10 As shown, the device includes: an acquisition module 1002, a first determination module 1004, a second determination module 1006 and a third determination module 1008. The device is described in detail below.

[0085] An acquisition module 1002 is used to acquire a target image, wherein the target image includes a target object; a first determination module 1004 is connected to the above-mentioned acquisition module 1002, and is used to determine, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and target parameters related to the target object, wherein the three-dimensional coordinate system based on the three-dimensional coordinates is a coordinate system constructed with the target reference object as the origin; a second determination module 1006 is connected to the above-mentioned first determination module 1004, and is used to determine, based on the target parameters, a weight coefficient corresponding to the three-dimensional coordinates; a third determination module 1008 is connected to the above-mentioned second determination module 1006, and is used to determine, based on the three-dimensional coordinates, the weight coefficient, and the initial angle, the target angle of the target object relative to the target reference object.

[0086] It should be noted here that the above-mentioned acquisition module 1002, the first determination module 1004, the second determination module 1006 and the third determination module 1008 correspond to steps S102 to S108 in the implementation angle determination method, and the instances and application scenarios implemented by the multiple modules are the same as the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0087] Example 3

[0088] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement any of the above angle determination methods.

[0089] Example 4

[0090] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above angle determination methods.

[0091] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0092] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0095] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0097] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for determining an angle, characterized in that: include: Acquire a target image, wherein the target image includes a target object; Determining, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object, wherein the predetermined three-dimensional coordinate system is a coordinate system constructed with the target reference object as an origin; Determining a weight coefficient corresponding to the three-dimensional coordinate according to the target parameter; A target angle of the target object relative to the target reference object is determined based on the three-dimensional coordinates, the weight coefficient, and the initial angle.

2. The method according to claim 1, characterized in that Determining the weight coefficient corresponding to the three-dimensional coordinate according to the target parameter includes: In the case where the target parameter includes a target orientation, determining an accuracy index of the target orientation; A weight coefficient corresponding to the three-dimensional coordinate is determined according to the accuracy index, wherein the accuracy index is proportional to the weight coefficient.

3. The method according to claim 1, characterized in that Determining the target angle of the target object relative to the target reference object based on the three-dimensional coordinates, the weight coefficient, and the initial angle includes: Inverse tangent the x-axis coordinate and the z-axis coordinate in the three-dimensional coordinate to obtain a target inverse tangent value; Determining the product of the target arctangent value and the weight coefficient; The sum of the product and the initial angle is determined as the target angle.

4. The method according to claim 1, characterized in that: Determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to a target reference object, and target parameters related to the target object based on the target image includes: Extracting image features of the target image; Determining, in the target image, a target anchor frame for selecting the target object according to the image features; Determine the three-dimensional coordinates corresponding to the center point of the target anchor frame, the initial angle of the center point relative to the target reference object, and the target parameters related to the target object in the target anchor frame.

5. The method according to claim 1, characterized in that Before using the target model to perform the following steps: determining the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and target parameters related to the target object based on the target image, the method further includes: Obtain an initial model and construct a loss function for model training, wherein the loss function includes a first loss function term, a second loss function term, and a third loss function term, wherein the first loss function term is a loss function term of a three-dimensional coordinate estimation branch, the second loss function term is a loss function term of an angle estimation branch, and the third loss function term is a loss function term of a target parameter estimation branch; Based on the loss function, the initial model is trained using sample data to obtain the target model.

6. The method according to claim 1, characterized in that Before determining the initial angle of the target object relative to the target reference object according to the target image, the method further includes: Determine, based on the target image, target probabilities corresponding to a plurality of target angle intervals, wherein the corresponding target probabilities represent probabilities that a predicted angle belongs to a corresponding target angle interval, the predicted angle is an angle of the target object relative to the target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not 1; The initial angle is determined according to the target probabilities respectively corresponding to the multiple target angle intervals.

7. The method according to any one of claims 1 to 6, characterized in that Determining the weight coefficient corresponding to the three-dimensional coordinate according to the target parameter includes: In the case where there are multiple target parameters, determining sub-weight coefficients corresponding to the multiple target parameters respectively; The weight coefficient corresponding to the three-dimensional coordinate is determined according to the multiple target parameters and the sub-weight coefficients respectively corresponding to the multiple target parameters.

8. An angle determination device, characterized in that: include: An acquisition module, used for acquiring a target image, wherein the target image includes a target object; A first determination module is used to determine, based on the target image, the three-dimensional coordinates of the target object in a predetermined three-dimensional coordinate system, the initial angle of the target object relative to the target reference object, and target parameters related to the target object, wherein the three-dimensional coordinates are based on a three-dimensional coordinate system that is constructed with the target reference object as the origin; A second determination module, used to determine a weight coefficient corresponding to the three-dimensional coordinate according to the target parameter; The third determination module is used to determine a target angle of the target object relative to the target reference object according to the three-dimensional coordinates, the weight coefficient, and the initial angle.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the angle determination method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the angle determination method as claimed in any one of claims 1 to 7.

Images