Method, device, controller, vehicle and product for determining the size of an object

Abstract

The invention relates to a method and device for determining the size of an object, a controller, a vehicle and a product. The method includes determining a first size of the object based on a location of a point in the point cloud data of the object. The method further includes determining a second size of the object based on a classification result of the point cloud data. Further, the method includes determining a size of the object based on the first size and a corresponding first weight, and the second size and a corresponding second weight, where the first weight is determined based on a pose of the object. According to the embodiment of the invention, the weight of the size determined based on the point cloud position can be adjusted in combination with the posture of the object, so that the influence degree of the size on the final size is adjusted under the condition that the accuracy of the sizes determined based on the point cloud position is different, and the accuracy of the final size is improved.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of intelligent driving, and more specifically to methods, apparatus, controllers, vehicles, and computer program products for determining the dimensions of an object. Background Technology

[0002] With the improvement of driving technology, various driver assistance systems are widely used in the driving process. For example, through devices such as vehicle radar and vehicle cameras, objects such as vehicles or pedestrians in front can be identified, thereby providing process information and reference information for decision-making such as route planning and driving strategies.

[0003] To improve the accuracy of decision-making during driving, driver assistance systems (ADAS) need to accurately identify objects such as vehicles or pedestrians ahead. Through object recognition in the driving scenario, ADAS can perform functions such as environmental monitoring, hazard warnings, route navigation, and autonomous driving, thereby improving driving comfort and safety. Therefore, accurate object recognition results are crucial for decision-making during driving. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, controller, vehicle, and product for determining the dimensions of an object.

[0005] In a first aspect of this disclosure, a method for determining the size of an object is provided. The method includes determining a first size of the object based on the positions of points in point cloud data of the object. The method also includes determining a second size of the object based on a classification result of the point cloud data. Furthermore, the method includes determining the size of the object based on the first size and a corresponding first weight, and the second size and a corresponding second weight, wherein the first weight is determined based on the pose of the object.

[0006] In a second aspect of this disclosure, an apparatus for determining the size of an object is provided. The apparatus includes a first size determination module configured to determine a first size of the object based on the position of a point in point cloud data of the object. The apparatus also includes a second size determination module configured to determine a second size of the object based on a classification result of the point cloud data. Furthermore, the apparatus includes a size determination module configured to determine the size of the object based on the first size and a corresponding first weight, and the second size and a corresponding second weight, wherein the first weight is determined based on the pose of the object.

[0007] In a third aspect of this disclosure, a controller is provided. The controller includes at least one processor. The controller also includes memory coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the controller to perform the method provided according to the first aspect.

[0008] In a fourth aspect of this disclosure, a vehicle is provided. The vehicle includes radar. The vehicle also includes a controller provided according to a third aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a machine program product is provided, comprising a machine program that is executed by a processor to implement the method provided in the first aspect.

[0010] In a sixth aspect of the disclosure, a machine-readable storage medium is provided. The machine-readable storage medium stores machine-executable instructions, which are executed by a processor to implement the method provided according to a first aspect of this disclosure.

[0011] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1 A schematic diagram of an example environment in which some embodiments of this disclosure may be implemented is shown;

[0014] Figure 2 Flowcharts illustrating methods for determining the dimensions of an object according to some embodiments of this disclosure are shown;

[0015] Figure 3A A schematic diagram of a first scenario based on radar-detected edges and angles, representing some embodiments of this disclosure, is shown;

[0016] Figure 3B A schematic diagram of a second scenario based on radar-detected edges and angles, representing some embodiments of this disclosure, is shown;

[0017] Figure 3C A schematic diagram of a third scenario based on radar-detected edges and angles, representing some embodiments of this disclosure, is shown;

[0018] Figure 4 A schematic diagram illustrating a process for determining the dimensions of an object according to some embodiments of this disclosure is shown;

[0019] Figure 5 A schematic diagram illustrating the correspondence between the weights and confidence levels of objects in some embodiments of this disclosure is shown;

[0020] Figure 6A A schematic diagram illustrating the first correspondence between dimensions of some embodiments of this disclosure and dimensions based on point cloud location and dimensions based on point cloud classification is shown.

[0021] Figure 6B A schematic diagram illustrating a second correspondence between dimensions of some embodiments of this disclosure and dimensions based on point cloud location and dimensions based on point cloud classification is shown.

[0022] Figure 6C A schematic diagram illustrating a third correspondence between dimensions of some embodiments of this disclosure and dimensions based on point cloud location and dimensions based on point cloud classification;

[0023] Figure 7 Block diagrams of apparatus for determining the dimensions of an object, according to some embodiments of the present disclosure, are shown; and

[0024] Figure 8 A schematic block diagram of a controller according to some embodiments of the present disclosure is shown.

[0025] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0026] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0027] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0028] As mentioned above, accurately identifying object information, such as object size, is crucial for decision-making during driving. Due to factors such as the resolution constraints of automotive radar, the point cloud distribution of objects is often inaccurate and incomplete. Therefore, the accuracy of determining object size based solely on the distribution of point cloud positions after obtaining point cloud data through radar detection is low, which in turn affects the subsequent association and update processes during driving.

[0029] Correspondingly, the size is determined by the positional distribution of point cloud data detected by radar, and also by the classification results of the point cloud data detected by radar. The sizes determined by these two methods are then fused with a fixed weight to obtain the final size. However, due to factors such as radar resolution, signal attenuation, or signal interference from multiple paths, point cloud points in the point cloud data may be lost or added. Therefore, the accuracy of the point cloud data and the size determined based on its positional distribution varies depending on the object or the time of the same object. Consequently, the accuracy of the final size obtained by fusing the data with a fixed weight in related technologies is not high.

[0030] Therefore, embodiments of this disclosure provide a method for determining the size of an object. The object's size is determined based on the point cloud position and classification result of the detected object's point cloud data. A weight for the size based on the point cloud position is determined according to the object's current pose. The final size is then determined by fusing the size determined based on the point cloud position and its corresponding weight, as well as the size determined based on the classification result and its corresponding weight. This method allows for adjusting the weight of the size determined based on the point cloud position in conjunction with the object's pose, thereby adjusting the influence of the size on the final size when the accuracy of the size determined based on the point cloud position varies, thus improving the accuracy of the final size.

[0031] Figure 1 A schematic diagram of an example environment 100 in which some embodiments of this disclosure may be implemented is shown. In some embodiments, Figure 1 The illustration shows vehicles 102 and 104 traveling on a road from a bird's-eye view perspective. Vehicle 102 is the current vehicle, and vehicle 104 is the vehicle ahead. Vehicle 104 can be considered an object in this disclosure. It should be understood that although the example environment 100 in this disclosure is shown from a bird's-eye view perspective, and the resulting dimensions are also those from a bird's-eye view perspective, other perspectives and dimensions are also within the scope of this disclosure.

[0032] refer to Figure 1 Vehicle 102 is normally traveling in the right lane, while vehicle 104 is veering out of its left lane. During the movement of vehicle 102, vehicle 104 can be detected by onboard radar to determine its size and adjust the driving strategy accordingly. The driving strategy determined for vehicle 102 will differ depending on whether vehicle 104 is a smaller vehicle (e.g., a sedan, SUV) or a larger vehicle (e.g., a truck, tanker).

[0033] In some embodiments, a controller 106 is provided in the vehicle 102. The controller 106 is used to determine a driving strategy for the vehicle 102 according to the driving environment and control the vehicle 102 to execute the driving strategy, wherein the driving strategy is affected by the size of the vehicle 104. In the process of determining the size of the vehicle 104, the controller 102 can acquire point cloud data 108 of the vehicle 104 detected by radar, and determine the size 110 of the vehicle 104 (which may be referred to as the first size) based on the positional distribution of the point cloud in the point cloud data 108, and determine the size 112 of the vehicle 104 (which may be referred to as the second size) based on the classification result of the point cloud data 108.

[0034] Then, the controller 106 determines the final size 118 based on the weight 114 corresponding to size 110 and the weight 116 corresponding to size 112. The weight 114 is determined based on the attitude of the vehicle 104. It should be understood that the accuracy of the collected point cloud data 108 varies with the attitude of the vehicle 104, which in turn leads to different accuracy of size 110. Therefore, it is necessary to adjust the influence of size 110 on size 118 under different attitudes using the weight 114 for different attitudes. Furthermore, the method for determining weight 116 is not limited in this embodiment; for example, it can be determined based on the accuracy of the classification results.

[0035] In this way, the weights 114 of the size 110 determined based on the point cloud position can be adjusted in combination with the attitude of the vehicle 104. This allows the weights 114 to adjust the influence of the size 110 on the final size 118 when the accuracy of the size 110 determined based on the point cloud position varies, thereby improving the accuracy of the final size 118 and the accuracy of the driving strategy determined based on the size 118.

[0036] It should be understood that the architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and / or functionalities.

[0037] The following will combine Figures 2 to 8 The process according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It should be understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0038] Figure 2 A flowchart illustrating a method 200 for determining the dimensions of an object according to some embodiments of the present disclosure is shown. In some embodiments, in Figure 1In the example environment 100 shown, method 200 can be executed by controller 106. It should be understood that although the following description refers to controller 106 as the executing entity, method 200 can also be executed by other devices. Method 200 may also include additional actions not shown and / or the actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0039] At position 202, the first size of the object is determined based on the location of the midpoint in the object's point cloud data. In some embodiments, in Figure 1 In the example environment 100 shown, the controller 106 determines the position of the point cloud points in the point cloud data 108 based on the point cloud data 108 of the vehicle 104 (which may be referred to as the object) detected by the radar of the vehicle 102, and determines the size 110 (which may be referred to as the first size) based on the distance between the positions of the point cloud points.

[0040] At position 204, based on the classification results of the point cloud data, the second dimension of the object is determined. In some embodiments, in Figure 1 In the example environment 100 shown, the controller 106 classifies the point cloud data 108 of the detected vehicle 104 to obtain the corresponding classification result, and then determines the size 112 (which may be referred to as the second size) corresponding to the classification result. It should be understood that different classification results, such as cars and heavy trucks, will have different corresponding sizes 112.

[0041] At position 206, the size of the object is determined based on a first size and a corresponding first weight, and a second size and a corresponding second weight, wherein the first weight is determined based on the object's pose. In some embodiments, Figure 1 In the example environment 100 shown, after determining size 110 based on the location distribution of point cloud points and size 112 based on classification results, controller 106 determines the final size 118 according to size 110 and its corresponding weight 114, and size 112 and its corresponding weight 116. The weight 114 corresponding to size 110 is determined based on the attitude of vehicle 104.

[0042] In the embodiments of this disclosure, the size of the object is determined based on the point cloud position and classification result of the detected object's point cloud data. A weight for the size based on the point cloud position is determined according to the object's current pose. The final size is then determined by fusing the size determined based on the point cloud position and its corresponding weight, as well as the size determined based on the classification result and its corresponding weight. This approach allows for adjusting the weight of the size determined based on the point cloud position in conjunction with the object's pose. This adjusts the influence of the size on the final size when the accuracy of the size determined based on the point cloud position varies, thereby improving the accuracy of the final size.

[0043] Figures 3A-3C Schematic diagrams of a first scenario 300A, a second scenario 300B, and a third scenario 300C based on radar-detected edges and angles, representing some embodiments of this disclosure, are shown respectively. In some embodiments, reference is made to... Figure 3A When the attitude direction of vehicle 104 is parallel or nearly parallel to the driving direction of vehicle 102 (or the direction detected by radar), controller 106 can determine the detectable edge cd (i.e., visible edge) in vehicle 104 based on the point cloud data 108 collected by radar. At this time, edges ab, bc, and ad are undetectable edges (i.e., invisible edges). Then, controller 106 can determine the angle of vehicle 104 through edge cd, which can be used as the attitude of vehicle 104.

[0044] In some embodiments, the controller 106 can determine the positions of the endpoints c (which may be called the first endpoint) and d (which may be called the second endpoint) of the edge cd, and then determine the direction oc (which may be called the first direction) based on the center point o (which may be called the reference point, such as the radar position point) and the endpoint c, and determine the direction od (which may be called the second direction) based on the center point o and the endpoint d, and then take the angle between the direction oc and the direction od, i.e. ∠cod, as the angle of the vehicle 104.

[0045] refer to Figure 3B When the attitude direction of vehicle 104 forms a certain angle with the driving direction of vehicle 102 (or the direction detected by radar), controller 106 can determine the detectable edges bc and cd in vehicle 104 based on the point cloud data 108 collected by radar. At this time, edges ab and ad are undetectable edges. Then, controller 106 can determine the angle of vehicle 104 through edges bc and / or cd, which can be used as the attitude of vehicle 104. The process of determining the angle of vehicle 104 (i.e., ∠boc and / or ∠cod) through edges bc and / or cd is described in... Figure 3A This has already been explained in detail, so I will not repeat it here.

[0046] refer to Figure 3C When the orientation of vehicle 104 is perpendicular or nearly perpendicular to the driving direction of vehicle 102 (or the direction detected by radar), controller 106 can determine the detectable edge bc in vehicle 104 based on the point cloud data 108 collected by radar. At this time, edges ab, cd, and ad are undetectable edges. Then, controller 106 can determine the angle of vehicle 104 through edge bc, which can be used as the orientation of vehicle 104. The process of determining the angle of vehicle 104 (i.e., ∠boc) through edge bc is described in... Figure 3A This has already been explained in detail, so I will not repeat it here.

[0047] In this way, the angle of vehicle 104 can be determined based on the edge of vehicle 104 that can be detected by radar, thereby obtaining the attitude relationship between vehicle 104 and vehicle 102 (or the radar of vehicle 102), improving the accuracy of the attitude of vehicle 104, and thus improving the accuracy of the size detection of vehicle 104.

[0048] In some embodiments, the final size 118 is determined using a weighted result of the size 110 based on the point cloud location distribution from a bird's-eye view and the size 112 based on the classification result. However, due to the influence of radar active detection performance, the size 110 based on the point cloud location from the bird's-eye view may be missing or expanded due to attenuation of the radar detection signal, multipath interference signals, and other noise. Therefore, if the size 110 based on the point cloud location and the size 112 based on the classification result are directly weighted, missing or expanded information will be indiscriminately introduced into the final size 118, resulting in an error in size 118.

[0049] Therefore, selecting appropriate weights to weight dimensions 112 and 118 is crucial for calculating the final dimension 118. Based on this, to improve the accuracy of the final dimension 118, this disclosure introduces prior information based on a bird's-eye view. An adaptive weight, i.e., weight 110, for dimension 112 is determined using information from the bird's-eye view perspective, thereby improving the accuracy of the weighting of dimension 112.

[0050] Meanwhile, for size 112 based on the classification result, since the classification result depends on the point cloud classification network and requires a large dataset, the final size 118 is adaptively updated based on the prior information and features of the point cloud. In determining the weights 116 corresponding to size 112, the weights of size 112 can be weakened or unreliable size information can be discarded to achieve better robustness. In this way, it is possible to avoid single-signal jumps in the final size 118 caused by the inability to detect single-frame point cloud data and thus determine the classification result.

[0051] Based on this, and according to the aforementioned prior information, the present disclosure adopts... Figures 3A to 3C The angle shown is used to design the attitude information of vehicle 104 based on visible information, namely the visible edges of vehicle 104. Using this prior information, weights 114 can be dynamically updated during the movement of vehicle 102 to obtain an adaptive and accurate size 118. For the weights 116 of size 112, the accuracy of the classification result, i.e., the confidence level, can be used as reference information to calculate the weights 116 of size 112. During the size fusion filtering process, the weights 116 determined based on the confidence level are introduced into the final size 118 calculation process to improve the accuracy of size 118. The following will combine... Figure 4This section describes in detail the process of setting weights 114 and 116 and calculating the final size 118.

[0052] Figure 4 A schematic diagram of a process 400 for determining the dimensions of an object, according to some embodiments of this disclosure, is shown. In some embodiments, in Figure 1 In the example environment 100 shown, process 400 can be executed by controller 106. It should be understood that although the following description refers to controller 106 as the executing entity, process 400 can also be executed by other devices. Process 400 may also include additional actions not shown and / or the actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0053] At 402, controller 102 determines the size based on the point cloud positions in the point cloud data. In some embodiments, controller 106 determines the positions of point cloud points in point cloud data 108 detected by radar of vehicle 104, and determines the size 110 based on the distance between the positions of the point cloud points. The size 110 can be the length or width of vehicle 104, or the length of its diagonal, etc.

[0054] At position 404, controller 102 determines visible edges based on point cloud data. In some embodiments, controller 106 can determine whether the edges of vehicle 104 are visible based on features of the point cloud data 108 detected by radar, for example, by monitoring the edges of vehicle 104 using a free-hit detection algorithm. Edges of the surface of vehicle 104 that can be directly detected by radar can be considered visible edges, while edges that cannot be directly detected by radar can be considered invisible edges. Figures 3A-3C As shown, the solid line marks the visible edge of vehicle 104, while the dashed line marks the invisible edge of vehicle 104.

[0055] At 406, controller 106 determines the angle based on the visible edge. In some embodiments, to eliminate the effects of size asymmetry, controller 106 uses the angle determined based on the visible edge as a reference variable. Controller 106 can define the angle of vehicle 104 in the radar coordinate system by the endpoints of the visible edge, and the angle formed by the left and right endpoints of the visible edge and the radar point is taken as the angle corresponding to vehicle 104. Figures 3A-3C As shown, ∠cod or ∠boc can be taken as the angle of vehicle 104.

[0056] In some embodiments, the controller 106 also needs to determine an angle threshold based on the radar performance, and determine whether the angle is reasonable by comparing the angle with the angle threshold. By comparing the angle of the vehicle 104 with the angle threshold, the controller 106 can determine the weight 114 of the size 110. For example, if the angle is higher than the angle threshold, the controller 106 directly sets the weight 114 to a larger weight value; if the angle is lower than the angle threshold, the controller 106 directly sets the weight 114 to a smaller weight value or directly sets it to 0. In this way, the weight 114 can be quickly determined, thereby improving the efficiency of size detection. A more specific embodiment of how to determine the weight 114 based on the relationship between the angle and the angle threshold will be described below.

[0057] In determining the angle threshold, the controller 106 can determine a lower limit for the angle threshold based on the radar's performance and set the angle threshold to or above this lower limit. In some embodiments, the controller 106 can first determine the resolution of the radar in the vehicle 102 and determine the lower limit based on that resolution. It should be understood that this lower limit is the angle resolution. If the distance between two detection points exceeds the radar's resolution, and the angle formed between these two detection points is lower than the angle resolution, then these two detection points are not detection points on the surface of the vehicle 104. For example, if the radar's angle resolution is set to 5 degrees, then the lower limit can be set to 5 degrees or higher.

[0058] In some embodiments, the controller 106 can determine the maximum value of the detectable edges based on the radar's corresponding scaling parameters and the detected edges, then determine the upper limit of the angle based on the edge with the maximum value, and set the angle threshold to or below this upper limit. It should be understood that the scaling parameters are used to compensate for losses caused by radar accuracy. For example, since the radar's accuracy is 70% acceptable full-size visibility, the controller 106 can determine the upper limit of the angle based on this 70% scaling parameter. In this way, the impact of incomplete size on radar detection results can be reduced in situations where radar visibility or local visibility is low.

[0059] At 408, the controller 106 determines the weights based on the angle. In some embodiments, when the point cloud data 108 consists of point cloud data from different time sequences, two adjacent or close frames of point cloud data are selected. The angle (referred to as the first angle) of the previous frame of point cloud data (referred to as the first frame of point cloud data) and the angle (referred to as the second angle) of the next frame of point cloud data (referred to as the second frame of point cloud data) have a magnitude relationship. Therefore, the weight 114 of the size 110 can be determined by combining the magnitude relationship between the two and the angle threshold.

[0060] Specifically, the controller 106 can determine the relationship between the angles of the previous frame of point cloud data and the angles of the next frame of point cloud data. Based on this relationship, and the relationship between the angles of the previous or next frame of point cloud data and an angle threshold, it determines the weights corresponding to the previous and next frame of point cloud data. By iteratively calculating the angle of each frame of point cloud data in this way, the controller quickly determines the angle and weight for each frame, thereby improving the accuracy of the weights based on the correlation between the point cloud data from consecutive frames.

[0061] In some embodiments, the angle of the previous frame point cloud data is defined as angle. v1 The angle of the next frame of point cloud data is angle. v2 And set a default weight T (which can be called the first reference value). For example, the weight that performs better can be selected as the default weight T based on multiple experiments. At the angle... v1 greater than angle v2 And angle v1 If the angle is greater than the angle threshold, the weight W corresponding to the point cloud data of the previous frame is directly determined. v_1 Assign a default weight T, and determine the weight W corresponding to the point cloud data in the second frame. v_2 The default weights T and angles are given. v1 with angle v2 The product of the ratios. The specific calculation formula is as follows:

[0062] W v_1 =T(if angle) v1 >angle&&angle v1 >angle v2 (1)

[0063]

[0064] At 410, the controller 106 determines the size based on the classification result of the point cloud data. In some embodiments, the controller 106 classifies the point cloud data 108 of the detected vehicle 104 to obtain a corresponding classification result, and then determines the size 112 corresponding to the classification result. It should be understood that different classification results, such as passenger cars and heavy trucks, will correspond to different sizes 112.

[0065] At 412, controller 106 determines the confidence level of the classification result. In some embodiments, controller 106 classifies point cloud data 108 using a neural network model to obtain classification results for the point cloud data and a confidence level corresponding to each classification result. It should be understood that the confidence level of the classification result is the probability that vehicle 104 belongs to that classification result. Then, controller 106 determines the weight 116 corresponding to size 112 based on the confidence level of the classification result. For example, a larger weight value is selected when the confidence level is high, and a lower weight value is selected when the confidence level is low.

[0066] Figure 5 A schematic diagram illustrating the correspondence between weights and confidence levels of objects in some embodiments of this disclosure is shown, where the object may be, for example, a vehicle 104. In some embodiments, the relationship between confidence level C and weight Wc (i.e., weight 116) is as shown by a curve, where weight Wc increases with increasing confidence level C and the rate of change of weight Wc decreases with increasing confidence level C. When confidence level C is high, weight Wc is close to 1 and the rate of change is slow, while when confidence level is low, the rate of change of weight 116 is fast. For example, the correspondence between weight Wc and confidence level C can be set as follows:

[0067]

[0068] In some embodiments, the controller 106 can be configured such that when the confidence level C is greater than the confidence threshold C1 (e.g., 0.5), the correspondence between the weight Wc and the confidence level C is as follows: Figure 5 As shown in the curve, when the confidence level C is less than the confidence threshold C1, the weight Wc is directly set to 0 (which can be called the second reference value). In this way, the classification result can be discarded when it cannot be accurately identified, thereby avoiding the introduction of an incorrect size 116 into the final size 118, avoiding single signal jumps when the classification result cannot be accurately identified, and thus improving the accuracy and stability of the final size 118.

[0069] Return to reference Figure 4 At position 416, controller 106 performs fusion filtering. At position 418, controller 106 outputs the final dimensions. In some embodiments, the dimensions of vehicle 104 include the lengths of its long and wide sides. By adjusting the corresponding weights, the influence of the lengths of the long and wide sides obtained in different ways on the final length and width can be adjusted. The specific expressions are as follows:

[0070]

[0071] Among them, D out_L D out_w These are the lengths of the long and wide sides of the final output, respectively, D. in1_L and D in1_wThese are the lengths of the longer and shorter sides, D, obtained based on the classification results. in2_L and D in2_w Here, W represents the lengths of the long and short sides, respectively, derived from the location distribution of the point cloud. K is the weight corresponding to the confidence score of the point cloud data classification result, and W... V_L The weights W are determined based on the angles corresponding to the visible longer sides. V_w The weights are determined based on the angles corresponding to the visible shorter sides. By fusing and filtering dimensional data obtained through different methods using their respective weights, the accuracy of the final dimensions can be improved.

[0072] Figures 6A-6C Schematic diagrams illustrating the correspondences 600A, 600B, and 600C between dimensions based on point cloud location and dimensions based on point cloud classification, representing some embodiments of this disclosure. In some embodiments, the dimension based on point cloud location is Din1, the dimension based on point cloud classification is Din2, and the final output dimension is Dout. (See reference...) Figure 6A When the confidence level of the point cloud classification result is low, the weight corresponding to size Din2 can be directly set to 0. In this case, size Dout is only affected by size Din1. (Reference) Figure 6B When the weights of both dimensions Din1 and Din2 are not zero, dimension Dout is affected by both dimensions Din1 and Din2. (See reference) Figure 6C When the point cloud position distribution is inaccurate, size Din1 can be discarded, and its corresponding weight is 0. Size Dout is only affected by size Din2.

[0073] Figure 7 A block diagram of an apparatus 700 for determining the dimensions of an object, according to some embodiments of the present disclosure, is shown. (See reference...) Figure 7 The device 700 includes a first size determination module 702, configured to determine a first size of the object based on the position of a midpoint in the object's point cloud data. The device 700 also includes a second size determination module 704, configured to determine a second size of the object based on the classification result of the point cloud data. Furthermore, the device 700 includes a size determination module 706, configured to determine the size of the object based on the first size and its corresponding first weight, and the second size and its corresponding second weight, wherein the first weight is determined based on the object's pose.

[0074] In some embodiments, the attitude includes an angle, and the device 700 further includes an attitude determination module configured to: determine, based on point cloud data, the edges of the object that can be detected by radar, the radar being used to acquire the point cloud data; and determine the angle of the object based on the edges that can be detected by radar.

[0075] In some embodiments, the attitude determination module is further configured to: determine the positions of a first endpoint and a second endpoint of an edge; determine a first direction between the first endpoint and a reference point, and a second direction between the second endpoint and the reference point, wherein the position of the reference point is determined based on the position of the radar; and determine the angle of an object based on the angle between the first direction and the second direction.

[0076] In some embodiments, the apparatus 700 further includes a first weight determination module configured to: determine an angle threshold for the radar; and determine a first weight for a first size based on the relationship between the angle of the object and the angle threshold.

[0077] In some embodiments, the first weight determination module is further configured to: determine a lower limit and / or an upper limit of an angle threshold, the lower limit being determined based on the radar resolution and the upper limit being determined based on the edge and a scaling parameter for the radar; and determine the angle threshold based on the lower limit and / or the upper limit.

[0078] In some embodiments, the angle includes a first angle corresponding to a first frame of point cloud data and a second angle corresponding to a second frame of point cloud data, the first frame of point cloud data precedes the second frame of point cloud data, and the first weight determination module is further configured to: determine the size relationship between the first angle and the second angle; and determine the first weight corresponding to the first frame of point cloud data and the first weight corresponding to the second frame of point cloud data based on the size relationship between the first angle and the second angle, and the size relationship between the first angle or the second angle and the angle threshold.

[0079] In some embodiments, the first weight determination module is further configured to: determine a first weight corresponding to the first frame point cloud data as a first reference value in response to a first angle being greater than a second angle and the first angle being greater than an angle threshold; and determine a first weight corresponding to the second frame point cloud data based on the first reference value, the first angle, and the second angle.

[0080] In some embodiments, the object includes a vehicle, the sides include the long side and / or the wide side of the vehicle, and the dimensions include the length of the long side and / or the wide side.

[0081] In some embodiments, the classification result of the point cloud data is determined by a neural network model, and the apparatus 700 further includes a second weight determination module configured to: determine the confidence level of the classification result of the point cloud data based on the neural network model; and determine a second weight of the second size based on the confidence level.

[0082] In some embodiments, the second weight determination module is further configured to: determine the second weight corresponding to the confidence level based on the correspondence between the second weight and the confidence level when the confidence level is higher than the confidence level threshold, wherein the second weight increases based on the increase of the confidence level and the rate of change of the second weight decreases based on the increase of the confidence level when the confidence level is higher than the confidence level threshold; and determine the second weight as a second reference value when the confidence level is lower than the confidence level threshold.

[0083] It is understood that the apparatus 700 of this disclosure can achieve at least one of the many advantages that the method or process described above can achieve. For example, the apparatus 700 determines the size of the object based on the point cloud position and classification result of the detected object's point cloud data, and determines the weight of the size based on the point cloud position according to the object's current pose. Then, it determines the final size based on the fusion result of the size determined based on the point cloud position and its corresponding weight, and the size determined based on the classification result and its corresponding weight. In this way, the weight of the size determined based on the point cloud position can be adjusted in conjunction with the object's pose, so that the weight can adjust the degree of influence of the size determined based on the point cloud position on the final size under different conditions of varying accuracy, thereby improving the accuracy of the final size.

[0084] Figure 8 A schematic block diagram of a controller according to some embodiments of the present disclosure is shown. Figure 8 A schematic block diagram of a controller 800 that can be used to implement embodiments of the present disclosure is shown. In some embodiments, the controller 800 is used to implement... Figure 1 The controller 106 is shown in the example environment 100. Figure 8 As shown, the controller 800 includes a processor 801, which can perform various appropriate actions and processes based on machine program instructions loaded into random access memory (RAM) 803 according to machine program instructions stored in read-only memory (ROM) 802. The RAM 803 may also store various programs and data required for the operation of the controller 800. The processor 801, ROM 802, and RAM 803 are interconnected via bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0085] The various processes and procedures described above, such as method 200, can be executed by processor 801. For example, in some embodiments, method 200 may be implemented as a machine software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the machine program may be loaded into and / or mounted onto controller 800 via ROM 802. When the machine program is loaded into RAM 803 and executed by processor 801, one or more actions of method 200 described above may be performed.

[0086] This disclosure can be a method, apparatus, system, and / or machine program product. A machine program product may include a machine-readable storage medium loaded with machine-readable program instructions for performing various aspects of this disclosure.

[0087] Machine-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Machine-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of machine-readable storage media include: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and any suitable combination of the foregoing. As used herein, machine-readable storage media is not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0088] The machine-readable program instructions described herein can be downloaded from machine-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to external machines or external storage devices. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway machines, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the machine-readable program instructions from the network and forwards them to the machine-readable storage media within the respective computing / processing device.

[0089] Machine program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The machine-readable program instructions may be executed entirely on the user machine, partially on the user machine, as a stand-alone software package, partially on the user machine and partially on a remote machine, or entirely on a remote machine or server. In cases involving remote machines, the remote machine may be connected to the user machine via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external machine (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the machine-readable program instructions to implement various aspects of this disclosure.

[0090] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and machine program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by machine-readable program instructions.

[0091] These machine-readable program instructions can be provided to the processing unit of a general-purpose machine, a special-purpose machine, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the machine or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These machine-readable program instructions can also be stored in a machine-readable storage medium that causes a machine, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the machine-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0092] Machine-readable program instructions may also be loaded onto a machine, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the machine, other programmable data processing apparatus, or other equipment to produce a machine-implemented process, thereby causing the instructions executed on the machine, other programmable data processing apparatus, or other equipment to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and machine program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and machine instructions.

[0094] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method (200) for determining the dimensions of an object, comprising: Based on the position of the point in the point cloud data of the object, determine (202) the first size of the object; Based on the classification results of the point cloud data, determine (204) the second dimension of the object; as well as The size of the object is determined (206) based on the first size and the corresponding first weight, and the second size and the corresponding second weight, wherein the first weight is determined based on the pose of the object.

2. The method (200) according to claim 1, wherein the posture includes an angle, and the method for determining the posture includes: Based on the point cloud data, the edges of the object that can be detected by radar are determined, and the radar is used to acquire the point cloud data; as well as The angle of the object is determined based on the edge that can be detected by the radar.

3. The method (200) according to claim 2, wherein determining the angle of the object based on the side detectable by the radar comprises: Determine the positions of the first and second endpoints of the edge; Determine a first direction between the first endpoint and the reference point, and a second direction between the second endpoint and the reference point, wherein the position of the reference point is determined based on the position of the radar; and The angle of the object is determined based on the angle between the first direction and the second direction.

4. The method (200) according to claim 2, wherein determining the first weight based on the pose of the object comprises: Determine the angle threshold for the radar; as well as The first weight of the first size is determined based on the relationship between the angle of the object and the angle threshold.

5. The method (200) according to claim 4, wherein determining the angle threshold for the radar comprises: Determine a lower limit and / or an upper limit for the angle threshold, wherein the lower limit is determined based on the resolution of the radar, and the upper limit is determined based on the edge and a scaling parameter for the radar; as well as The angle threshold is determined based on the lower limit and / or the upper limit.

6. The method (200) according to claim 4, wherein the angle includes a first angle corresponding to a first frame of point cloud data and a second angle corresponding to a second frame of point cloud data, the first frame of point cloud data precedes the second frame of point cloud data, and determining the first weight of the first size based on the relationship between the angle of the object and the angle threshold includes: Determine the magnitude relationship between the first angle and the second angle; as well as Based on the relationship between the first angle and the second angle, and the relationship between the first angle or the second angle and the angle threshold, the first weight corresponding to the first frame point cloud data and the first weight corresponding to the second frame point cloud data are determined.

7. The method (200) according to claim 6, wherein determining the first weight corresponding to the first frame point cloud data and the first weight corresponding to the second frame point cloud data based on the magnitude relationship between the first angle and the second angle, and the magnitude relationship between the first angle or the second angle and the angle threshold, includes: In response to the first angle being greater than the second angle and the first angle being greater than the angle threshold, the first weight corresponding to the first frame point cloud data is determined as the first reference value; as well as Based on the first reference value, the first angle, and the second angle, the first weight corresponding to the second frame point cloud data is determined.

8. The method (200) according to any one of claims 2 to 7, wherein the object includes a vehicle, the side includes the long side and / or the wide side of the vehicle, and the dimension includes the length of the long side and / or the wide side.

9. The method (200) according to claim 1, wherein the classification result of the point cloud data is determined by a neural network model, and determining the second weight includes: Based on the neural network model, the confidence level of the classification result of the point cloud data is determined; as well as Based on the confidence level, a second weight for the second size is determined.

10. The method (200) of claim 9, wherein determining the second weight of the second size based on the confidence level comprises: When the confidence level is higher than the confidence level threshold, the second weight corresponding to the confidence level is determined based on the correspondence between the second weight and the confidence level. In the correspondence, the second weight increases with the increase of the confidence level, and the rate of change of the second weight decreases with the increase of the confidence level. as well as If the confidence level is lower than the confidence level threshold, the second weight is determined as the second reference value.

11. An apparatus (700) for determining the dimensions of an object, comprising: The first size determination module (702) is configured to determine the first size of the object based on the position of the midpoint in the point cloud data of the object; The second size determination module (704) is configured to determine the second size of the object based on the classification result of the point cloud data; as well as The size determination module (706) is configured to determine the size of the object based on the first size and the corresponding first weight, and the second size and the corresponding second weight, wherein the first weight is determined based on the pose of the object.

12. A controller (106), comprising: At least one processor; as well as A memory coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the controller to perform the method according to any one of claims 1 to 10.

13. A vehicle (102), comprising: radar; as well as The controller (106) according to claim 12.

14. A machine program product comprising a machine program that is executed by a processor to implement the method according to any one of claims 1 to 10.

Images