Angle value determination method, device and electronic equipment

Abstract

The present invention discloses a method, device and electronic device for determining an angle value. The method comprises: obtaining a target image, wherein the target image includes a target object; determining target probabilities corresponding to a plurality of target angle intervals respectively according to the target image, wherein the corresponding target probability represents the probability that the predicted angle value belongs to the corresponding target angle interval, the predicted angle value is the angle value of the target object relative to a target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not 1; determining the target angle value of the target object relative to the target reference object according to the target probabilities corresponding to the plurality of target angle intervals. The present invention solves the technical problem in the related art that when determining the angle value of an object relative to a reference object, the angle value determined is inaccurate.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method and device for determining an angle value, and an electronic device. Background Art

[0002] In existing unmanned driving technology, the input data of various sensors needs to be processed by the perception module, and the real-time description of the surrounding environment obtained after processing is then input into the planner for behavioral decision-making and planning. Among them, the perception module plays a key role in the unmanned driving system. How to better optimize the algorithm of the perception module has become one of the keys to developing and promoting the progress of unmanned driving technology.

[0003] In the monocular 3D detection task of the perception module, the input is an RGB image. The algorithm needs to detect all vehicles, pedestrians, cyclists and other targets in the image, and the output is the actual position information, size information and direction information of the target. Among them, the direction information plays a very important role in the detection effect of the target. How to improve the prediction effect of the direction angle has become one of the keys to improve the overall detection effect of the monocular 3D detection task.

[0004] Traditional azimuth prediction generally adopts a direct regression solution, which directly inputs the features extracted by the backbone network into a detection head to regress the azimuth of the corresponding target. This direct regression solution has two problems. One is that for deep learning, classification tasks have long been easier than regression tasks, and direct regression of azimuths may increase the difficulty of network learning. The other is that direct regression of azimuths does not take into account other aspects of azimuths, which directly leads to the problem that some special situations are not learned. For example, in some traffic scenes, a certain type of target (such as pedestrians) may be rarer than other types of targets (such as vehicles), which will lead to an imbalance in target distribution, and the network may produce certain errors in target detection in this case.

[0005] Therefore, how to optimize the direction angle prediction in monocular 3D detection tasks is an urgent problem that needs to be solved.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] Embodiments of the present invention provide a method, device and electronic device for determining an angle value, so as to at least solve the technical problem in the related art that when determining the angle value of an object relative to a reference object, the determined angle value is inaccurate.

[0008] According to one aspect of an embodiment of the present invention, there is provided a method for determining an angle value, comprising: acquiring a target image, wherein the target image includes a target object; determining target probabilities corresponding to a plurality of target angle intervals respectively based on the target image, wherein the corresponding target probabilities represent probabilities that a predicted angle value belongs to a corresponding target angle interval, the predicted angle value is an angle value of the target object relative to a target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals respectively is 1, and the target probabilities corresponding to the plurality of target angle intervals respectively are not 1; determining a target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the plurality of target angle intervals respectively.

[0009] Optionally, determining the target probabilities corresponding to multiple target angle intervals respectively based on the target image includes: arranging the multiple target angle intervals in a predetermined order to obtain a predetermined queue; determining a selected angle interval among the multiple target angle intervals, wherein the selected angle interval is an interval with the highest corresponding predicted probability among the multiple target angle intervals; in the predetermined queue, taking the probability corresponding to the selected angle interval as the peak value and taking the probabilities belonging to the multiple target angle intervals respectively conforming to a predetermined distribution as a constraint, determining the target probabilities corresponding to the multiple target angle intervals respectively.

[0010] Optionally, in the predetermined queue, the probability corresponding to the selected angle interval is taken as the peak value, and the probabilities belonging to the multiple target angle intervals respectively conform to the predetermined distribution is constrained to determine the target probabilities corresponding to the multiple target angle intervals respectively, including: in the predetermined queue, the probability corresponding to the selected angle interval is taken as the peak value, and the probabilities belonging to the multiple target angle intervals respectively conform to the predetermined distribution is constrained to determine the initial probabilities corresponding to the multiple target angle intervals respectively; when the sum of the initial probabilities that the predicted angle values ​​respectively belong to the multiple target angle intervals is not 1, the initial probabilities of the multiple target angle intervals are probability offset processed with the selected angle interval as the center to determine the target probabilities corresponding to the multiple target angle intervals respectively.

[0011] Optionally, determining target probabilities corresponding to multiple target angle intervals respectively based on the target image includes: arranging the multiple target angle intervals in a predetermined order to obtain a predetermined queue; determining a target position of a selected angle interval in the predetermined queue; determining a target queue based on the target position of the selected angle interval in the predetermined queue, wherein an adjacent boundary index corresponding to the position of the selected angle interval in the target queue is less than a predetermined threshold; determining target probabilities corresponding to the multiple target angle intervals respectively based on the target queue.

[0012] Optionally, in the case where the target position includes a first target position and a second target position, the target queue is determined based on the target position in the predetermined queue according to the selected angle interval, including: when the selected angle interval is in the first target position of the target queue, changing the arrangement order of the multiple target angle intervals in the target queue to obtain the target queue, wherein the first target position is a position whose corresponding adjacent boundary index is greater than or equal to the predetermined threshold; and / or, when the selected angle interval is in the second target position of the target queue, determining the predetermined queue as the target queue, wherein the second target position is a position whose corresponding adjacent boundary index is less than the predetermined threshold.

[0013] Optionally, determining the target probabilities corresponding to the multiple target angle intervals respectively based on the target queue includes: taking the probability corresponding to the selected angle interval in the target queue as the peak value, and taking the probabilities belonging to the multiple target angle intervals respectively conforming to a predetermined distribution as a constraint, to determine the target probabilities corresponding to the multiple target angle intervals respectively.

[0014] Optionally, determining the target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the multiple target angle intervals includes: determining the target probability distribution of the predicted angle value based on the target position of the selected angle interval in the predetermined queue and the target probabilities corresponding to the multiple target angle intervals; and determining that the expected value corresponding to the target probability distribution is the target angle value.

[0015] Optionally, the target probability distribution of the predicted angle value is determined based on the target position of the selected angle interval in the predetermined queue and the target probabilities corresponding to the multiple target angle intervals, including: when the selected angle interval is in the first target position of the target queue, determining an offset for the angles in the multiple target angle intervals based on the first target position; offsetting the angles in the multiple target angle intervals based on the offset to obtain multiple updated angle intervals; determining the target probability distribution of the predicted angle value based on the target probabilities corresponding to the multiple updated angle intervals; and / or, when the selected angle interval is in the second target position of the target queue, determining the target probability distribution of the predicted angle value based on the target probabilities corresponding to the multiple target angle intervals.

[0016] Optionally, the method also includes: while using the target probability prediction model to perform the following steps: determining the target probabilities corresponding to multiple target angle intervals respectively based on the target image; determining the target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the multiple target angle intervals, training the sample data in the following manner to obtain the target probability prediction model: obtaining an initial model and a loss function for model training, wherein the loss function includes a first loss function term and a second loss function term, the first loss function term represents the degree of difference between the predicted probability distribution and the sample probability distribution, and the second loss function term represents the degree of difference between the predicted probability and the sample probability; based on the loss function, using the sample data to train the initial model to obtain the target probability prediction model.

[0017] According to one aspect of an embodiment of the present invention, there is provided an angle value determination device, comprising: an acquisition module, for acquiring a target image, wherein the target image includes a target object; a first determination module, for determining target probabilities corresponding to a plurality of target angle intervals, respectively, based on the target image, wherein the corresponding target probabilities represent probabilities that a predicted angle value belongs to a corresponding target angle interval, the predicted angle value is an angle value of the target object relative to a target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not all 1; a second determination module, for determining a target angle value of the target object relative to the target reference object, based on the target probabilities corresponding to the plurality of target angle intervals.

[0018] According to one aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the above-mentioned methods for determining an angle value.

[0019] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned methods for determining an angle value.

[0020] In an embodiment of the present invention, a target image including a target object is obtained. Then, based on the target image, the target probabilities corresponding to the multiple target angle intervals are determined, and then based on the target probabilities corresponding to the multiple target angle intervals, the target angle value of the target object relative to the target reference object is determined. Since the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals is 1, which can ensure the stability of the determined target angle value, and the target probabilities corresponding to the multiple target angle intervals are not 1, which can ensure the accuracy of the determined target angle value. When predicting, the problem of predicted probability distribution is considered, and certain constraints are set on the prediction process. Therefore, under the above constraints, the angle value can be predicted more accurately, thereby solving the technical problem of inaccurate angle value determined when determining the angle value of an object relative to a reference object in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 is a flow chart of a method for determining an angle value according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of one-hot encoding provided by an optional embodiment of the present invention;

[0024] Figure 3 is a probability distribution schematic diagram provided by an optional implementation manner of the present invention;

[0025] Figure 4 is another probability distribution schematic diagram provided by an optional embodiment of the present invention;

[0026] Figure 5 is another probability distribution schematic diagram provided by an optional embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of a sudden change in angle of a boundary portion provided by an optional embodiment of the present invention;

[0028] Figure 7 is a schematic diagram of coordinate axis offset provided by an optional embodiment of the present invention;

[0029] Figure 8 4 is a structural block diagram of an angle value determination device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] According to an embodiment of the present invention, an embodiment of a method for determining an angle value is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 1 is a flow chart of a method for determining an angle value according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0035] Step S102, acquiring a target image, wherein the target image includes a target object;

[0036] The method provided in the present application can be applied to multiple scenarios, such as scenarios in the field of vehicle autonomous driving. In this scenario, the direction angle of the target object in the target image is predicted, and the positional relationship between the vehicle and the target object can be accurately determined, thereby achieving the effect of avoiding the target object and preventing traffic accidents.

[0037] When the method provided in this application is used in the field of target detection in autonomous driving, the target object may refer to a person or object that appears next to the vehicle during the driving process, and in a specific solution, may be a specified object. The target image is the image captured by the camera in the vehicle.

[0038] Step S104, determining target probabilities corresponding to the multiple target angle intervals respectively according to the target image, wherein the corresponding target probability represents the probability that the predicted angle value belongs to the corresponding target angle interval, the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals is 1, and the target probabilities corresponding to the multiple target angle intervals are not 1;

[0039] In step S104 provided in the present application, the predicted angle value is the angle value of the target object relative to the target reference object. In the scenario of vehicle automatic driving, the target reference object is the vehicle or a camera in the vehicle. It is used to determine the angle value of the target object relative to the vehicle.

[0040] It should be noted that the angle value facing the front of the vehicle can be set to 0, the angle value facing the rear of the vehicle can be set to 180, etc., and no limitation is made here. The angle standard can be set according to actual applications and scenarios.

[0041] In this step, target probabilities corresponding to multiple target angle intervals are determined based on the target image, and the multiple target probabilities are target probabilities that the predicted angle values ​​belong to the multiple target angle intervals, i.e., for example, the predicted angle values ​​belong to the target probabilities between 0-10 degrees, 10 degrees-20 degrees, ..., 350 degrees-360 degrees.

[0042] Moreover, corresponding restrictions are set for the target probabilities belonging to multiple target angle intervals, respectively. It is ensured that the sum of the probabilities belonging to all angle intervals is 1. The problem of inaccurate estimation caused by the sum of probabilities not being 1 and the problem of low stability of the method provided by the present application caused by misjudgment that is easy to occur when the sum of probabilities is not 1 are avoided.

[0043] Step S106 , determining a target angle value of the target object relative to the target reference object according to the target probabilities corresponding to the plurality of target angle intervals.

[0044] In step S106 provided in the present application, when determining the target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the multiple target angle intervals, the target angle value can be determined in a variety of ways. For example, the angle interval corresponding to the maximum probability can be determined from the multiple target probabilities, and the final angle value can be determined from the angle interval. It is also possible to determine that the sum of the products of multiple target angle intervals and the corresponding probabilities is the final angle value. Specific limitations can be made here according to the actual application and scenario. Or a balance can be made according to the required accuracy and speed. Determine the optimal way to determine the target angle value for a specific scenario.

[0045] Through the above steps S102-S106, a target image including a target object is obtained. Then, based on the target image, the target probabilities corresponding to the multiple target angle intervals are determined, and then based on the target probabilities corresponding to the multiple target angle intervals, the target angle value of the target object relative to the target reference object is determined. Since the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals is 1, which can ensure the stability of the determined target angle value, and the target probabilities corresponding to the multiple target angle intervals are not 1, which can ensure the accuracy of the determined target angle value. When predicting, the problem of predicted probability distribution is considered, and certain constraints are set on the prediction process. Therefore, under the above constraints, the angle value can be predicted more accurately, thereby solving the technical problem of inaccurate angle value determined when determining the angle value of an object relative to a reference object in the related art.

[0046] As an optional embodiment, target probabilities corresponding to multiple target angle intervals are determined based on the target image, including: arranging multiple target angle intervals in a predetermined order to obtain a predetermined queue; determining a selected angle interval among the multiple target angle intervals, wherein the selected angle interval is an interval with the highest corresponding predicted probability among the multiple target angle intervals; in the predetermined queue, taking the probability corresponding to the selected angle interval as the peak value and taking the probabilities belonging to the multiple target angle intervals respectively conforming to the predetermined distribution as a constraint, determining the target probabilities corresponding to the multiple target angle intervals.

[0047] In this embodiment, the steps of determining the target probability based on the predetermined queue are described. First, multiple target angle intervals can be arranged in a predetermined order to obtain a predetermined queue. The predetermined order can be an order of arrangement from a small range to a large range according to the size of the angle interval range. From the arranged predetermined queue, the interval with the highest corresponding probability is determined. Taking the probability corresponding to the selected angle interval as the peak value and the probability of belonging to each of the multiple target angle intervals conforming to the predetermined distribution as a constraint, the target probabilities corresponding to the multiple target angle intervals are determined.

[0048] The predetermined distribution may be a variety of distributions, such as Gaussian distribution, etc. The specific distribution may be customized according to the actual application and scenario. The probability corresponding to the selected angle interval is taken as the peak value, so that the distribution corresponding to the adjacent angle interval can reflect the true probability distribution corresponding to the angle value, thereby ensuring the effectiveness of the subsequent use of the method provided by this application.

[0049] As an optional embodiment, in a predetermined queue, the probability corresponding to the selected angle interval is taken as the peak value, and the probabilities belonging to the multiple target angle intervals respectively conform to the predetermined distribution as a constraint, to determine the target probabilities corresponding to the multiple target angle intervals respectively, including: in a predetermined queue, the probability corresponding to the selected angle interval is taken as the peak value, and the probabilities belonging to the multiple target angle intervals respectively conform to the predetermined distribution as a constraint, to determine the initial probabilities corresponding to the multiple target angle intervals respectively; when the sum of the initial probabilities that the predicted angle values ​​belong to the multiple target angle intervals is not 1, with the selected angle interval as the center, the initial probabilities of the multiple target angle intervals are probability offset processed to determine the target probabilities corresponding to the multiple target angle intervals respectively.

[0050] In this embodiment, the steps of determining the target probabilities corresponding to multiple target angle intervals are defined. In the predetermined queue, when the probability corresponding to the selected angle interval is taken as the peak value and the probability of belonging to multiple target angle intervals conforms to the predetermined distribution as the constraint for probability determination, there will be a special case, that is, if the target angle interval is at the boundary position of the predetermined queue, it may be generated that the sum of the initial probabilities of the predicted angle values ​​belonging to multiple target angle intervals is not 1. For example, if the selected angle interval is at the second positive position and the predetermined distribution is Gaussian distribution, then the probability corresponding to the selected angle interval is taken as the peak value and the probability of belonging to multiple target angle intervals conforms to Gaussian distribution as the constraint for probability prediction, the probability value corresponding to the selected angle interval may be a peak value, the probability corresponding to the first positive position and the third positive position is the same as the secondary peak value, and the probability corresponding to the fourth positive position is a value slightly lower than the probability corresponding to the third positive position. Then, it can be known at this time that since the first positive position is already at the first position, there is no position with the same probability as the fourth positive position. Since it conforms to the Gaussian distribution, the sum of the initial probabilities that the predicted angle values ​​belong to multiple target angle intervals will not be 1.

[0051] In this case, corresponding processing is required. Specifically, the initial probabilities of multiple target angle intervals can be processed with probability offset with the selected angle interval as the center, and the target probabilities corresponding to the multiple target angle intervals can be determined. Since the angle of the circle is 360 degrees, according to the nature of the circle, the penultimate position can be automatically adjusted to be the same as the probability corresponding to the fourth positive position mentioned above. Similarly, the probability corresponding to the fifth positive position is the same as the second to last position, and so on. Through this processing method, it can be made to conform to the Gaussian distribution, and the sum of the target probabilities corresponding to the multiple target angle intervals is 1, and the target probabilities corresponding to the multiple target angle intervals are not 1, avoiding the problem of inaccurate target probability prediction caused by the selected angle interval being at the boundary position.

[0052] As an optional embodiment, target probabilities corresponding to multiple target angle intervals are determined based on the target image, including: arranging multiple target angle intervals in a predetermined order to obtain a predetermined queue; determining a target position of a selected angle interval in the predetermined queue; determining a target queue based on the target position of the selected angle interval in the predetermined queue, wherein the neighboring boundary index corresponding to the position of the selected angle interval in the target queue is less than a predetermined threshold; and determining target probabilities corresponding to multiple target angle intervals based on the target queue.

[0053] In this embodiment, the steps of determining the target probabilities corresponding to the multiple target angle intervals are defined. In this processing method, it can be determined whether the selected angle interval is adjacent to the boundary. Since the above-mentioned inaccurate prediction occurs only when the selected angle interval is at the boundary, it is possible to first determine the position of the selected angle interval in the predetermined queue. At this time, there are two situations, one is the situation where the selected angle interval is adjacent to the boundary. The other is the situation where the selected angle interval is not adjacent to the boundary. Different methods of determining the target queue are determined in different situations.

[0054] Among them, whether the selected angle interval is located at a position adjacent to the boundary can be determined by the adjacent boundary index corresponding to the selected angle interval. That is, after arranging multiple target angle intervals in a predetermined order to obtain a predetermined queue, determine the target position of the selected angle interval in the predetermined queue to determine the adjacent boundary index to determine whether it is adjacent to the boundary. Determine the target queue based on the target position of the selected angle interval in the predetermined queue.

[0055] Since the proximity boundary index corresponding to the position of the selected angle interval in the target queue is less than the predetermined threshold, that is, the position of the selected angle interval in the target queue is not close to the boundary, the target probabilities corresponding to multiple target angle intervals can be determined directly based on the target queue that is not at the boundary, thereby achieving the effect of efficiently determining the target probabilities corresponding to multiple target angle intervals.

[0056] As an optional embodiment, in a case where the target position includes a first target position and a second target position, a target queue is determined based on a target position in a predetermined queue in a selected angle interval, including: in a case where the selected angle interval is in the first target position of the target queue, changing the arrangement order of multiple target angle intervals in the target queue to obtain a target queue, wherein the first target position is a position whose corresponding adjacent boundary index is greater than or equal to a predetermined threshold; and / or, in a case where the selected angle interval is in the second target position of the target queue, determining the predetermined queue as the target queue, wherein the second target position is a position whose corresponding adjacent boundary index is less than a predetermined threshold.

[0057] In this embodiment, a step of determining a target queue is defined based on the target position of a selected angle interval in a predetermined queue. At this time, there are two situations, one is that the selected angle interval is in a position adjacent to the boundary, and the predetermined queue needs to be adjusted to obtain the target queue. The other is that the selected angle interval is not in a position adjacent to the boundary, and the predetermined queue can be directly used as the target queue. Whether the selected angle interval is in a position adjacent to the boundary can be determined by the adjacent boundary index corresponding to the selected angle interval. In this way, it is possible to flexibly determine whether the queue needs to be adjusted, which can not only participate in subsequent operations more accurately, but also speed up the implementation process of the method provided in the present application.

[0058] As an optional embodiment, target probabilities corresponding to multiple target angle intervals are determined based on the target queue, including: taking the probability corresponding to the selected angle interval in the target queue as the peak value, and taking the probabilities belonging to the multiple target angle intervals respectively conforming to a predetermined distribution as a constraint, to determine the target probabilities corresponding to the multiple target angle intervals.

[0059] In this embodiment, the steps of determining the target probability based on the target queue are described. First, multiple target angle intervals can be arranged in a predetermined order to obtain a target queue. The predetermined order can be an order of arrangement from a small range to a large range according to the size of the angle interval range. From the arranged queue, the interval with the highest corresponding predicted probability is determined. Taking the probability corresponding to the selected angle interval as the peak value and the probability of belonging to each of the multiple target angle intervals conforming to the predetermined distribution as a constraint, the target probabilities corresponding to the multiple target angle intervals are determined.

[0060] The predetermined distribution may be a variety of distributions, such as Gaussian distribution, etc. The specific distribution may be customized according to the actual application and scenario. The probability corresponding to the selected angle interval is taken as the peak value, so that the distribution corresponding to the adjacent angle interval can reflect the true probability distribution corresponding to the angle value, thereby ensuring the effectiveness of the subsequent use of the method provided by this application.

[0061] As an optional embodiment, the target angle value of the target object relative to the target reference object is determined based on the target probabilities corresponding to multiple target angle intervals, including: determining the target probability distribution of the predicted angle value based on the target position of the selected angle interval in a predetermined queue and the target probabilities corresponding to multiple target angle intervals; and determining the expected value corresponding to the target probability distribution as the target angle value.

[0062] In this embodiment, a method for determining the target angle value is described. When determining the target angle value, the target probability distribution of the predicted angle value can be determined based on the target probabilities corresponding to the multiple target angle intervals, and the expected value corresponding to the target probability distribution can be determined as the target angle value. Determining the target angle value by calculating the expected value can make the final angle value take into account the possibility in each target angle interval, making the prediction result more accurate. And because the final observation angle is obtained through the expectation corresponding to the predicted probability distribution, the network can predict a continuous observation angle value, which can solve the problem of quantization error.

[0063] As an optional embodiment, the target probability distribution of the predicted angle value is determined according to the target position of the selected angle interval in the predetermined queue and the target probabilities corresponding to the multiple target angle intervals, including: when the selected angle interval is at the first target position of the target queue, the offset amount for offsetting the angles in the multiple target angle intervals is determined according to the first target position; the angles in the multiple target angle intervals are offset according to the offset amount to obtain multiple updated angle intervals; the target probability distribution of the predicted angle value is determined according to the target probabilities corresponding to the multiple updated angle intervals. And / or, when the selected angle interval is at the second target position of the target queue, the target probability distribution of the predicted angle value is determined according to the target probabilities corresponding to the multiple target angle intervals.

[0064] In this embodiment, two situations are described. First, the first situation is described. The first situation defines the step of determining the target probability distribution of the predicted angle value according to the target position in the predetermined queue of the selected angle interval and the target probabilities corresponding to the multiple target angle intervals. For example, when the selected angle interval is 0-10 degrees, the corresponding target probability is up to 0.6, the predicted probability corresponding to the angle interval of 10-20 degrees is 0.2, and the target probability corresponding to the angle interval of 350-360 degrees is 0.2, ..., in this case, if an angle value is directly selected from the angle interval and multiplied by the corresponding target probability, the following result will be obtained, 5*0.6+15*0.2+355*0.2+...≈77, but obviously 77 is unreasonable, because the selected angle interval is 0-10 degrees, the angle value finally determined should also be within the range of 0-10 degrees, or around 0 or 10 degrees, but 77 is obviously beyond the above range, therefore, this is unreasonable, and the reason for this unreasonable phenomenon is that the selected angle interval is at the boundary position, which has a certain deviation.

[0065] Based on the above considerations, it is necessary to determine the position of the selected angle interval and perform corresponding processing according to its position. For example, when the selected angle interval is at the first target position in the target queue, since the first target position is a position where the corresponding adjacent boundary index is greater than or equal to the predetermined threshold, it means that the selected angle interval is at the above boundary position. Therefore, it is necessary to determine the offset amount for offsetting the angles in the multiple target angle intervals based on the first target position, and then offset the angles in the multiple target angle intervals based on the offset amount to obtain multiple updated angle intervals, and determine the target probability distribution of the predicted angle value based on the target probabilities corresponding to the multiple updated angle intervals, so that accurate target angle values ​​can be obtained later.

[0066] It should be noted that, when determining the offset for offsetting the angle in multiple target angle intervals, it is possible to select the selected angle interval with the highest corresponding probability to be offset to the middle position, so that the target angle value can be determined more accurately. As in the above example, when the selected angle interval is 0-10 degrees, the corresponding target probability is up to 0.6, the predicted probability corresponding to the angle interval of 10-20 degrees is 0.2, and the target probability corresponding to the angle interval of 350-360 degrees is 0.2, ..., in this case, 5 degrees are moved to 180 degrees, the offset is 175 degrees, and it is multiplied by the corresponding target probability, then the following result will be obtained, 180*0.5+190*0.2+170*0.2+...≈180, 180 minus the offset, and the target angle value is obtained to be 5 degrees, because 5 degrees are within 0-10 degrees, so it is reasonable to predict the target angle value. Through this embodiment, a more accurate target angle value can be predicted.

[0067] In this embodiment, two situations are described. Now, the second situation is described. The second situation defines the steps of determining the target probability distribution of the predicted angle value based on the target position of the selected angle interval in the predetermined queue and the target probabilities corresponding to multiple target angle intervals. Determine the position of the selected angle interval and perform corresponding processing according to its position. For example, if the second target position is in the target queue in the selected angle interval, since the second target position is a position where the corresponding adjacent boundary index is less than a predetermined threshold, it means that the selected angle interval is not in the above-mentioned boundary position. Determine the target probability distribution of the predicted angle value based on the target probabilities corresponding to multiple target angle intervals. As in the above example, when the selected angle interval is 180-190 degrees, the corresponding target probability is up to 0.6, the corresponding prediction probability of the angle interval of 170-180 degrees is 0.2, and the corresponding target probability of the angle interval of 190-200 degrees is 0.2, ..., in this case, multiplying it with the corresponding target probability, you will get the following result, 185*0.6+195*0.2+175*0.2+...≈185, and get the target angle value of 185 degrees. Since 185 degrees is within 180-190 degrees, it is reasonable to predict the target angle value. Through this embodiment, it is possible to predict a more accurate target angle value.

[0068] As an optional embodiment, the method also includes: while using the target probability prediction model to perform the following steps: determining the target probabilities corresponding to multiple target angle intervals respectively based on the target image; determining the target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the multiple target angle intervals, training the sample data in the following manner to obtain the target probability prediction model: obtaining an initial model and a loss function for model training, wherein the loss function includes a first loss function term and a second loss function term, the first loss function term represents the degree of difference between the predicted probability distribution and the sample probability distribution, and the second loss function term represents the degree of difference between the predicted probability and the sample probability; based on the loss function, using the sample data to train the initial model to obtain the target probability prediction model.

[0069] In this embodiment, the method of training sample data to obtain a target probability prediction model is described. An initial model and a loss function for model training are obtained. The loss function includes a first loss function term and a second loss function term. The first loss function term represents the degree of difference between the predicted probability distribution and the sample probability distribution, and the second loss function term represents the degree of difference between the predicted probability and the sample probability. Therefore, using a loss function composed of the above two loss function terms, the initial model is trained using sample data to obtain a target probability prediction model. The obtained model can minimize the difference between the predicted probability distribution and the sample probability distribution, and can also minimize the difference between the predicted probability and the sample probability. Moreover, by setting the loss function, the differences between the distributions can be learned, which also solves the problem of unfriendly differences between classes.

[0070] Moreover, in this embodiment, it is also explained that a model can be used to perform the above-mentioned prediction step. By using the model, the method provided by the present application can be performed more quickly and accurately. Moreover, since the target probability prediction model is obtained through training with sample data. The sample data may include sample images, and sample angle values ​​of sample objects relative to sample reference objects. Therefore, the trained model can be made to learn the commonalities and differences between the two. The target probabilities whose predicted angle values ​​belong to multiple target angle intervals can be predicted more accurately, so as to more accurately determine the target angle value of the target object relative to the target reference object.

[0071] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail below.

[0072] In the related art, the direction angle prediction usually first learns the observation angle (alpha) in the network, and then converts it into the final direction angle (Yaw) in combination with the position information of the target. The existing observation angle prediction generally adopts the direct regression scheme, which has the problems of difficult network learning and no consideration of the target distribution. There is also a part that adopts the supervised multibins classification + regression scheme, in which multibins divides 360 degrees into multiple bins for observation angle classification, and performs residual angle regression on the range interval bin to which the observation angle belongs. However, this scheme also has some problems. Among them, the classification of 360 degrees into multiple bins will introduce quantization errors and cannot accurately represent the direction angle. At the same time, the cross entropy loss used when multibins is divided into multiple bins for classification is not friendly to the difference between classes. For a direction angle with index index 0 belonging to a real value bin, the network predicts that the classification may be 10 or 20, but the loss corresponding to the cross entropy is the same, but in theory, the corresponding loss penalty for classification as 10 should be smaller. These problems will affect the final direction angle prediction.

[0073] In view of this, an optional implementation of the present invention provides a method for predicting a direction angle. Figure 2 is a schematic diagram of one-hot encoding provided by an optional embodiment of the present invention, Figure 3 is a probability distribution diagram provided by an optional embodiment of the present invention, Figure 4 is another probability distribution diagram provided by an optional embodiment of the present invention, Figure 5 is another probability distribution diagram provided by an optional embodiment of the present invention, Figure 6 is a schematic diagram of a sudden change in angle of a boundary portion provided by an optional embodiment of the present invention, Figure 7 is a schematic diagram of coordinate axis offset provided by an optional embodiment of the present invention, such as Figure 2 , 3 , 4, 5, 6, and 7, the method provided by the optional implementation manner of the present invention is introduced below.

[0074] S1, acquiring a target image, wherein the target image includes a target object;

[0075] S2, using a target probability prediction model to execute a step of determining target probabilities corresponding to a plurality of target angle intervals respectively according to the target image;

[0076] Among them, the target probability prediction model is obtained through training with sample data, and the training process is explained below.

[0077] During the training phase, the network predicts a probability distribution through 12 bins (target angle intervals) to fit the probability distribution of the true value of the observation angle (gt).

[0078] First, based on the target image, a probability distribution that conforms to the Gaussian distribution is determined, and the converted probability distribution is as follows Figure 3 As shown in the figure, due to the high response of Gaussian distribution, the network has a learning preference for the true value, which enables better prediction and increases the accuracy of the prediction.

[0079] However, we need to consider an additional situation, that is, in some cases, the sum of the probabilities of the 12 bins is not 1. In this case, we need to process it and get the above probability distribution. Figure 4 As shown in the figure, gt falls into the bin with index = 0 (referring to the target angle interval with the first position in the arrangement), and Figure 4 It can be seen that the sum of the probabilities is not 1, that is, the sum of the probabilities of the 12 bins is not 1. At the same time, there is another problem, that is, in actual situations, when calculating the observation angle error, the probability corresponding to 0 degrees and the probability corresponding to 360 degrees should be equivalent, but here the probability corresponding to 0 degrees and the probability corresponding to 360 degrees are not equivalent. Therefore, these problems need to be solved.

[0080] In order to solve these problems, a continuous circular probability density distribution is made. Figure 4 The probability distribution shown is transformed into Figure 5 The probability distribution shown is centered on the index corresponding to the high response point (the selected angle interval with the highest corresponding probability), with the index corresponding to the lowest corresponding point as the boundary, and the index beyond the boundary is subjected to a probability offset of length 12 (for example, in the case of a Gaussian distribution with 0 as the high response point, the probability at index = 8 is equal to the probability at index = -4).

[0081] It should also be noted that the losses in the training phase include the KL loss of the supervised multibins distribution (similar to the first loss function term, which represents the degree of difference between the predicted probability distribution and the sample probability distribution) and the L1 loss corresponding to the final observation angle (similar to the second loss function term, which represents the degree of difference between the predicted probability and the sample probability).

[0082] S3, using a target probability prediction model, executing according to target probabilities corresponding to a plurality of target angle intervals, to determine a target angle value of the target object relative to a target reference object.

[0083] It should also be noted that when determining the final target angle value, it is obtained by calculating the expectation of the probability distribution corresponding to the 12 bins output by the model.

[0084] When calculating the expected observation angle, there is a problem of angle mutation at the boundary, such as Figure 6 Assume that the gt of the observation angle is 1°, and after multibins processing it falls into the bin of index = 0, then the probabilities at index = 0 and index = 11 are both very high, and the final calculated probability distribution is expected to be around 180° (179°).

[0085] Therefore, a coordinate axis offset method can be used here, such as Figure 7 As shown in the figure, only the predicted values ​​of the observation angles in the range of 0°-60° and 300°-360° are processed here. For the observation angles falling in these two intervals, a predetermined offset is first made to the angles corresponding to all bins. After processing, the expectation of the probability distribution of all bins is obtained to obtain the shifted observation angle, and then the 180° offset is subtracted to obtain the final observation angle. After this processing, the problem of sudden angle changes in the boundary part can be solved, and the effect of determining the accurate target angle value can be achieved.

[0086] Through the above optional implementation, at least the following beneficial effects can be achieved:

[0087] (1) By setting the loss function and learning the differences between distributions, the problem of unfriendly inter-class differences can be solved.

[0088] (2) Since the final observation angle is obtained through the expectation corresponding to the predicted probability distribution, the network can predict a continuous observation angle value, which can solve the problem of quantization error.

[0089] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0091] Example 2

[0092] According to an embodiment of the present invention, a device for implementing the above angle value determination method is also provided. Figure 8 is a structural block diagram of an angle value determination device according to an embodiment of the present invention. Figure 8 As shown, the device includes: an acquisition module 802, a first determination module 804 and a second determination module 806. The device is described in detail below.

[0093] An acquisition module 802 is used to acquire a target image, wherein the target image includes a target object; a first determination module 804 is connected to the above-mentioned acquisition module 802, and is used to determine, based on the target image, target probabilities corresponding to multiple target angle intervals, wherein the corresponding target probability represents the probability that the predicted angle value belongs to the corresponding target angle interval, the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the multiple target angle intervals is 1, and the target probabilities corresponding to the multiple target angle intervals are not 1; a second determination module 806 is connected to the above-mentioned first determination module 804, and is used to determine the target angle value of the target object relative to the target reference object based on the target probabilities corresponding to the multiple target angle intervals.

[0094] It should be noted here that the above-mentioned acquisition module 802, the first determination module 804 and the second determination module 806 correspond to steps S102 to S106 in the implementation angle value determination method, and the instances and application scenarios implemented by the multiple modules are the same as the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0095] Example 3

[0096] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement any of the above-mentioned methods for determining an angle value.

[0097] Example 4

[0098] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned angle value determination methods.

[0099] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0100] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for determining an angle value, characterized in that: include: Acquire a target image, wherein the target image includes a target object; Determine, according to the target image, target probabilities corresponding to a plurality of target angle intervals, wherein the corresponding target probabilities represent probabilities that the predicted angle value belongs to the corresponding target angle interval, the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not 1; A target angle value of the target object relative to the target reference object is determined according to the target probabilities respectively corresponding to the plurality of target angle intervals.

2. The method according to claim 1, characterized in that The step of determining target probabilities corresponding to a plurality of target angle intervals respectively based on the target image includes: Arrange the multiple target angle intervals in a predetermined order to obtain a predetermined queue; Determining a selected angle interval from the multiple target angle intervals, wherein the selected angle interval is an interval with the highest corresponding prediction probability among the multiple target angle intervals; In the predetermined queue, the probability corresponding to the selected angle interval is taken as the peak value, and the probability that the probabilities respectively belong to the multiple target angle intervals conform to the predetermined distribution is constrained to determine the target probabilities respectively corresponding to the multiple target angle intervals.

3. The method according to claim 2, characterized in that The method of determining target probabilities corresponding to the plurality of target angle intervals in the predetermined queue, taking the probability corresponding to the selected angle interval as a peak value and taking the probabilities respectively belonging to the plurality of target angle intervals conforming to a predetermined distribution as a constraint, comprises: In the predetermined queue, taking the probability corresponding to the selected angle interval as the peak value and taking the probabilities respectively belonging to the plurality of target angle intervals conforming to the predetermined distribution as a constraint, determining initial probabilities respectively corresponding to the plurality of target angle intervals; When the sum of the initial probabilities that the predicted angle values ​​belong to the multiple target angle intervals is not 1, the initial probabilities of the multiple target angle intervals are processed by probability offset with the selected angle interval as the center to determine the target probabilities corresponding to the multiple target angle intervals.

4. The method according to claim 1, characterized in that The step of determining target probabilities corresponding to a plurality of target angle intervals respectively based on the target image includes: Arrange the multiple target angle intervals in a predetermined order to obtain a predetermined queue; Determine that the selected angle interval is at a target position in the predetermined queue; Determining a target queue according to the target position of the selected angle interval in the predetermined queue, wherein the proximity boundary index corresponding to the position of the selected angle interval in the target queue is less than a predetermined threshold; According to the target queue, target probabilities corresponding to the multiple target angle intervals are determined.

5. The method according to claim 4, characterized in that In the case where the target position includes a first target position and a second target position, determining a target queue according to the target position in the selected angle interval being in the predetermined queue comprises: In the case where the selected angle interval is at the first target position of the target queue, changing the arrangement order of the multiple target angle intervals in the target queue to obtain the target queue, wherein the first target position is a position whose corresponding adjacent boundary index is greater than or equal to the predetermined threshold; and / or, In a case where the selected angle interval is at the second target position of the target queue, the predetermined queue is determined to be the target queue, wherein the second target position is a position where the corresponding adjacent boundary index is less than the predetermined threshold.

6. The method according to claim 4, characterized in that The determining, based on the target queue, target probabilities corresponding to the plurality of target angle intervals respectively includes: Taking the probability corresponding to the selected angle interval in the target queue as the peak value, and taking the probabilities respectively belonging to the multiple target angle intervals conforming to a predetermined distribution as a constraint, the target probabilities respectively corresponding to the multiple target angle intervals are determined.

7. The method according to claim 4, characterized in that The determining, based on the target probabilities respectively corresponding to the plurality of target angle intervals, a target angle value of the target object relative to the target reference object comprises: Determining a target probability distribution of the predicted angle value according to the target position of the selected angle interval in the predetermined queue and the target probabilities corresponding to the plurality of target angle intervals respectively; An expected value corresponding to the target probability distribution is determined as the target angle value.

8. The method according to claim 7, characterized in that The step of determining the target probability distribution of the predicted angle value according to the target position in the predetermined queue in the selected angle interval and the target probabilities corresponding to the plurality of target angle intervals respectively includes: In a case where the selected angle interval is at a first target position of the target queue, determining an offset amount for offsetting the angles within the plurality of target angle intervals according to the first target position; offsetting the angles in the plurality of target angle intervals according to the offset to obtain a plurality of updated angle intervals; Determining the target probability distribution of the predicted angle value according to the target probabilities respectively corresponding to the multiple updated angle intervals; and / or, When the selected angle interval is at the second target position of the target queue, the target probability distribution of the predicted angle value is determined according to the target probabilities respectively corresponding to the multiple target angle intervals.

9. An angle value determination device, characterized in that: include: An acquisition module, used for acquiring a target image, wherein the target image includes a target object; A first determination module is used to determine target probabilities corresponding to a plurality of target angle intervals according to the target image, wherein the corresponding target probability represents the probability that the predicted angle value belongs to the corresponding target angle interval, the predicted angle value is the angle value of the target object relative to the target reference object, the sum of the target probabilities corresponding to the plurality of target angle intervals is 1, and the target probabilities corresponding to the plurality of target angle intervals are not 1; The second determination module is used to determine the target angle value of the target object relative to the target reference object according to the target probabilities respectively corresponding to the multiple target angle intervals.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the angle value determination method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the angle value determination method according to any one of claims 1 to 8.

Images