Method, device, electronic device and storage medium for determining height of target object

Abstract

The present disclosure provides a method, apparatus, electronic device, and storage medium for determining the height of a target object, relating to the field of artificial intelligence technology, particularly autonomous driving. A specific implementation scheme comprises: determining a first three-dimensional detection frame corresponding to the target object based on current point cloud data of a current frame; determining a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data; and determining an estimated height of the target object based on the first and second three-dimensional detection frames.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of autonomous driving. More specifically, the present disclosure provides a method, device, electronic device, storage medium, and computer program product for determining the height of a target object. Background Art

[0002] When driving, autonomous vehicles use sensors such as lidar and cameras to collect data related to obstacles on the road, then process the data to obtain information such as the shape and height of the obstacles, and make driving decisions such as going straight, braking, and detouring based on the information obtained. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, electronic device, storage medium, and computer program product for determining the height of a target object.

[0004] According to one aspect of the present disclosure, a method for determining the height of a target object is provided, including: determining a first three-dimensional detection frame corresponding to the target object based on current point cloud data of a current frame; determining a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data; and determining an estimated height of the target object based on the first three-dimensional detection frame and the second three-dimensional detection frame.

[0005] According to another aspect of the present disclosure, a device for determining the height of a target object is provided, comprising: a first determination module, a second determination module, and a third determination module. The first determination module is configured to determine a first three-dimensional detection frame corresponding to the target object based on current point cloud data of a current frame; the second determination module is configured to determine a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data; and the third determination module is configured to determine an estimated height of the target object based on the first and second three-dimensional detection frames.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method provided by the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method provided in the present disclosure when executed by a processor.

[0009] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the above-mentioned electronic device.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of an application scenario of the method and apparatus for determining the height of a target object according to an embodiment of the present disclosure;

[0013] Figure 2 is a schematic flow chart of a method for determining the height of a target object according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic flowchart of a method for determining a first three-dimensional detection frame according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic flowchart of a method for determining a second three-dimensional detection frame according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic flow chart of a method for determining an estimated height according to an embodiment of the present disclosure;

[0017] Figure 6 is a schematic flow chart of a method for determining the height of a target object according to another embodiment of the present disclosure;

[0018] Figure 7A is a schematic diagram of a method for determining the height of a target object according to an embodiment of the present disclosure;

[0019] Figure 7B is a schematic diagram of a method for determining the height of a target object according to an embodiment of the present disclosure;

[0020] Figure 8 is a schematic structural block diagram of an apparatus for determining the height of a target object according to an embodiment of the present disclosure; and

[0021] Figure 9 It is a structural block diagram of an electronic device used to implement the method for determining the height of a target object according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] In some embodiments, the current frame point cloud data can be used to determine the estimated height of an obstacle (e.g., a low obstacle such as a tree branch or a plastic bag), and the estimated height is input into a downstream decision module, which then determines a driving decision based on the estimated height. For example, if the estimated height of the obstacle determined based on the current frame point cloud data is relatively high, such as if the estimated height is higher than the actual height of the vehicle chassis, the decision module will determine that the obstacle will collide with the vehicle, and the vehicle will need to circumvent the obstacle. For another example, if the estimated height of the obstacle determined based on the current frame point cloud data is relatively low, such as if the estimated height is lower than the actual height of the vehicle chassis, the decision module will determine that the obstacle will not collide with the vehicle, and the vehicle does not need to circumvent the obstacle, and will make a decision to go straight.

[0024] However, estimating the obstacle height based only on the point cloud data of the current frame has problems with unstable recognition and low accuracy, causing the vehicle to make incorrect driving decisions based on the incorrect estimated height.

[0025] Furthermore, when the vehicle is farther from an obstacle, the estimated obstacle height is lower than the actual height due to factors such as fewer reflective points on low obstacles. This makes it easier for the decision module to proceed straight. For example, if the vehicle's chassis height is 15 cm and the actual height of an obstacle is 17 cm, the estimated height of the obstacle at a distance of 50 meters is determined to be 3 cm. Therefore, the decision module determines that the vehicle will not collide with the obstacle and decides to proceed straight.

[0026] As the distance between the vehicle and the obstacle gradually decreases, the estimated height of the obstacle detected by the vehicle continues to increase, gradually approaching the actual height of the obstacle. This is not until the vehicle is closer to the obstacle, when the estimated height approaches the actual height. For example, the vehicle may only determine that the obstacle's estimated height is 17 centimeters when it is 10 meters away. At this point, the decision module determines that the vehicle will collide with the obstacle if it continues straight ahead, and then takes emergency avoidance decisions such as braking or changing lanes.

[0027] It can be seen that for long-distance obstacles, the estimated height determined based on point cloud data is lower than the actual height, which is not conducive to the downstream decision-making module to make timely avoidance decisions such as braking and detours.

[0028] The embodiments of the present disclosure aim to provide a method for determining the height of a target object, which can improve the accuracy of the determined estimated height.

[0029] The technical solutions provided by the present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Figure 1 : is a schematic diagram of an application scenario of the method and apparatus for determining the height of a target object according to an embodiment of the present disclosure. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include sensors 101, 102, 103, a network 120, a server 130, and a road side unit (RSU) 140. The network 120 is used as a medium for providing communication links between the sensors 101, 102, 103 and the server 130. The network 120 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] The sensors 101 , 102 , 103 may interact with the server 130 via the network 120 to receive or send messages, etc.

[0033] Sensors 101, 102, and 103 may be functional components integrated into vehicle 110, such as infrared sensors, ultrasonic sensors, millimeter-wave radars, image acquisition devices, lidars, inertial measurement units, and the like. Sensors 101, 102, and 103 may be used to collect status data of perceived objects (e.g., pedestrians, vehicles, obstacles, etc.) around vehicle 110, as well as surrounding road data.

[0034] The vehicle 110 can communicate with the roadside unit 140, receive information from the roadside unit 140, or send information to the roadside unit.

[0035] The server 130 may be located at a remote location capable of establishing communication with the vehicle-mounted terminal, and may be implemented as a distributed server cluster consisting of multiple servers, or as a single server.

[0036] Server 130 can be a server that provides various services. For example, mapping applications and data processing applications can be installed on server 130. For example, server 130 running a data processing application can receive point cloud data transmitted from sensors 101, 102, and 103 via network 120. The point cloud data can be used as data to be processed. The data can then be processed to obtain an estimated obstacle height.

[0037] It should be noted that the method for determining the height of a target object provided in the embodiment of the present disclosure can generally be executed by the vehicle 110 or the server 130. Accordingly, the apparatus for determining the height of a target object provided in the embodiment of the present disclosure can also be provided in the vehicle 110 or the server 130.

[0038] I understand. Figure 1 The number of sensors, networks, and servers in the embodiment is only illustrative. Any number of sensors, networks, and servers may be used depending on the implementation requirements.

[0039] Figure 2 is a schematic flowchart of a method for determining the height of a target object according to an embodiment of the present disclosure.

[0040] like Figure 2 As shown, the method 200 for determining the height of a target object may include operations S210 to S230.

[0041] In operation S210 , a first three-dimensional detection box corresponding to a target object is determined based on current point cloud data of a current frame.

[0042] For example, a vehicle is equipped with a laser radar, which collects point cloud data of the surrounding environment at a predetermined period. The current point cloud data of the current frame can represent the most recent frame of point cloud data collected by the laser radar, that is, the last frame of point cloud data collected.

[0043] For example, the target object may be an obstacle, such as a tree branch, a plastic bag, a fallen cone, or other low obstacle.

[0044] For example, the current point cloud data may include multiple sub-point clouds of multiple objects surrounding a vehicle, and the target object may correspond to the first sub-point cloud among the multiple sub-point clouds. Object detection may be performed on the target object to obtain the first sub-point cloud corresponding to the target object, and then a bounding box containing the first sub-point cloud is determined as a first three-dimensional detection box. The first three-dimensional detection box is a 3D detection box with attributes such as length, width, and height.

[0045] In operation S220 , a second three-dimensional detection frame corresponding to the target object is determined based on the current point cloud data and at least one frame of historical point cloud data.

[0046] For example, the at least one frame of historical point cloud data may include point cloud data from the second to last frame to the Nth to last frame collected by the lidar, where N is an integer greater than or equal to 2. The at least one frame of historical point cloud data may be, for example, the most recent four frames of point cloud data collected excluding the current frame.

[0047] For example, if the current point cloud data and at least one frame of historical point cloud data constitute multi-frame point cloud data, a second sub-point cloud corresponding to the target object can be determined for each frame of point cloud data in the multi-frame point cloud data, thereby obtaining multiple second sub-point clouds corresponding to the multi-frame point cloud data. A bounding box containing these multiple second sub-point clouds is then determined as a second three-dimensional detection box. The second three-dimensional detection box is a 3D detection box with attributes such as length, width, and height.

[0048] In operation S230 , an estimated height of the target object is determined based on the first 3D detection frame and the second 3D detection frame.

[0049] For example, the credibility of the first three-dimensional detection frame and the credibility of the second three-dimensional detection frame can be compared first. For example, when the ratio between a certain side length of the second three-dimensional detection frame and the corresponding side length of the first three-dimensional detection frame is less than or equal to a ratio threshold, it can be determined that the second three-dimensional detection frame is credible and the first three-dimensional detection frame is uncredible. When the ratio is greater than the ratio threshold, it can be determined that the second three-dimensional detection frame is uncredible and the first three-dimensional detection frame is credible. Next, the estimated height can be determined based on the credible three-dimensional detection frame. For example, the height of the credible three-dimensional detection frame is determined as the estimated height. In this embodiment, the side length can be at least one of the length, width, and height, and the ratio threshold can be 5.

[0050] For another example, corresponding weights can be set for the first 3D detection frame and the second 3D detection frame, and then the weighted sum of the side lengths of the first 3D detection frame and the second 3D detection frame is calculated to obtain the target 3D detection frame, and the height of the target 3D detection frame is used as the estimated height.

[0051] The disclosed embodiment not only uses the current point cloud data of the current frame to determine the first 3D detection frame, but also uses point cloud data from multiple frames to determine the second 3D detection frame, and then determines the estimated height of the target object based on the first 3D detection frame and the second 3D detection frame. Therefore, compared to technical solutions that only use the current point cloud data of the current frame to determine the estimated height, the disclosed embodiment can use point cloud data from at least one frame of history before the current frame to improve the accuracy of the estimated height, thereby alleviating the problem of inaccurate estimated height caused by unstable detection of the point cloud data of the current frame.

[0052] On this basis, the decision-making module of the autonomous driving vehicle can make driving decisions based on the estimated height with higher accuracy, thereby reducing the frequency of emergency avoidance such as sudden braking, and improving the safety of autonomous driving and the riding experience.

[0053] Figure 3 is a schematic flowchart of a method for determining a first three-dimensional detection frame according to an embodiment of the present disclosure.

[0054] According to another embodiment of the present disclosure, the method 310 for determining a first three-dimensional detection frame corresponding to a target object based on current point cloud data of a current frame may include operations S311 to S313.

[0055] In operation S311 , first sub-point cloud data corresponding to a target object in the current point cloud data is determined.

[0056] In operation S312 , a first main direction of the first sub-point cloud data is determined according to the coordinates of the first sub-point cloud data.

[0057] In operation S313 , a first three-dimensional detection frame is determined according to the coordinates of the first sub-point cloud data and the first main direction.

[0058] For example, the first main direction of the first sub-point cloud data can be calculated using the least squares method. The calculation idea is to determine a first straight line through the least squares method so that the sum of the distances from multiple points in the first sub-point cloud data to the first straight line is minimized.

[0059] In related technologies, the upstream vision module will issue a detection frame, and the detection frame has a corresponding direction, and the direction is determined as the direction of the first sub-point cloud data. However, in actual applications, the direction issued by the upstream vision module does not conform to the distribution of the first sub-point cloud data. For example, the first sub-point cloud data is distributed in the form of long strips, and the direction issued by the vision module has a large angle with the extension direction of the long strip. For example, the direction issued by the vision module is perpendicular to the extension direction of the long strip. Therefore, it is necessary to use a larger first three-dimensional detection frame to include the first sub-point cloud data, which leads to the determined first three-dimensional detection frame being too large in size. A large first three-dimensional detection frame is prone to include point clouds of objects other than the target object, thereby reducing the shape and height accuracy of the detected target object.

[0060] The disclosed embodiment uses the first sub-point cloud data to determine the primary orientation, and uses this orientation to determine the first 3D detection frame. As a result, the resulting first 3D detection frame is smaller and more closely matches the first sub-point cloud data, thereby improving the accuracy of the target object's shape and size. Furthermore, since the primary orientation is determined using the first sub-point cloud data, there's no need for the upstream vision module to transmit direction, enabling decoupling from the upstream vision module.

[0061] Figure 4 is a schematic flowchart of a method for determining a second three-dimensional detection frame according to an embodiment of the present disclosure.

[0062] According to another embodiment of the present disclosure, the method 420 for determining a second three-dimensional detection frame corresponding to a target object based on current point cloud data and at least one frame of historical point cloud data may include operations S421 to S424.

[0063] In operation S421 , first sub-point cloud data corresponding to a target object in the current point cloud data is determined.

[0064] In operation S422 , for each historical point cloud data in at least one frame of historical point cloud data, second sub-point cloud data corresponding to the target object in each historical point cloud data is determined to obtain at least one second sub-point cloud data.

[0065] In operation S423 , the first sub-point cloud data and the at least one second sub-point cloud data are superimposed to obtain third sub-point cloud data.

[0066] For example, affected by factors such as the actual changes in the target object and the accuracy of lidar detection, in two consecutive frames of point cloud data collected, the two frames of point cloud data may include the same points at the same time. In addition, the latter frame of point cloud may have some more points or lack some points compared to the previous frame of point cloud.

[0067] For example, superposition can be to determine the union of the first sub-point cloud data and at least one second sub-point cloud data. For example, the first sub-point cloud data includes the following points: P1, P2, P3, P4, P5, and the second sub-point cloud data includes the following points: P1, P2, P3, P6. After superposition, the two frames of point clouds can include the following points: P1, P2, P3, P4, P5, P6.

[0068] In operation S424 , a second three-dimensional detection box is determined based on the third sub-point cloud data.

[0069] For example, a bounding box containing a plurality of third sub-point cloud data may be determined as the second three-dimensional detection box.

[0070] For another example, the second main direction of the third sub-point cloud data can be determined based on the coordinates of the third sub-point cloud data, and then the second three-dimensional detection frame can be determined based on the coordinates and the second main direction of the third sub-point cloud data. For example, the second main direction can be determined based on the least squares method, and the processing idea is to determine the second straight line so that the sum of the distances from multiple points in the third sub-point cloud data to the second straight line is minimized. Compared with the point cloud direction issued by the upstream vision module, the second three-dimensional detection frame obtained based on the main orientation is smaller in size and fits the third sub-point cloud data better, thereby improving the accuracy of the shape and size of the target object. In addition, it can also be decoupled from the upstream vision module.

[0071] The disclosed embodiment determines multiple sub-point cloud data from the current point cloud data and at least one frame of historical point cloud data, specifically a first sub-point cloud data set and at least one second sub-point cloud data set. These sub-point cloud data sets are then superimposed, and a second 3D detection frame is determined using the superimposed third sub-point cloud data set. As can be seen, since the second 3D detection frame is determined by combining multiple frames of point cloud data, it can more accurately reflect the shape, height, and other attributes of the obstacle.

[0072] Figure 5 is a schematic flowchart of a method for determining an estimated height according to an embodiment of the present disclosure.

[0073] According to another embodiment of the present disclosure, the method 530 for determining the estimated height of the target object based on the first 3D detection frame and the second 3D detection frame may include operations S531 and S532.

[0074] In operation S531 , a reference height of the target object is determined according to a visual detection frame for the target object.

[0075] In one example, the fourth sub-point cloud data in the current visual inspection frame is determined in the current point cloud data, and then the highest height of the fourth sub-point cloud data is determined as the reference height.

[0076] For example, the upstream vision module sends a 2D visual inspection frame, which corresponds to position information. Furthermore, the point cloud side sends the current point cloud data for the current frame, with position information corresponding to each point in the current point cloud data. Based on this position information, the 3D point cloud can be projected into the 2D visual inspection frame, thereby querying the points in the current point cloud data for the current frame that fall within the 2D visual inspection frame, and determining the highest height of the queried points as the reference height.

[0077] This embodiment combines the visual detection frame with the current point cloud data of the current frame, and uses the points in the visual detection frame to determine the reference height, thereby providing a reference for the height of the target object, so as to accurately verify whether the second three-dimensional detection frame is credible.

[0078] In other examples, the height of the visual detection box may be determined as the reference height.

[0079] In operation S532 , an estimated height is determined based on the reference height, the first three-dimensional detection frame, and the second three-dimensional detection frame.

[0080] In one example, the first validity of the second 3D detection frame can be determined based on the reference height and the height of the second 3D detection frame. The second validity of the second 3D detection frame can be determined based on the length of the first 3D detection frame, the width of the first 3D detection frame, the length of the second 3D detection frame, and the width of the second 3D detection frame. An estimated height can then be determined based on the first validity and the second validity.

[0081] For example, if it is determined that the ratio between the height of the second 3D detection frame and the reference height is less than or equal to a first threshold, the first validity of the second 3D detection frame is determined to be valid. The first threshold may be 5.

[0082] For example, the second validity of the second 3D detection frame can be determined to be valid if the ratio of the diagonal size of the second 3D detection frame to the diagonal size of the first 3D detection frame is less than or equal to a second threshold. The diagonal size can be calculated using the Pythagorean theorem, length, and width. The second threshold can be the same as or different from the first threshold, and the second threshold can be 5.

[0083] For example, if the first validity is determined to be valid and the second validity is determined to be valid, and if the shape difference between the single-frame point cloud data (i.e., the current point cloud data of the current frame) and the multi-frame point cloud data (i.e., the current point cloud data of the current frame and at least one frame of historical point cloud data) is small, then the second 3D detection frame can be determined to be credible. Furthermore, the height of the second 3D detection frame can be determined as the estimated height.

[0084] For example, when it is determined that at least one of the first validity and the second validity is invalid, it can be determined that the shapes of the single-frame point cloud data (i.e., the current point cloud data of the current frame) and the multi-frame point cloud data (i.e., the current point cloud data of the current frame and at least one frame of historical point cloud data) are significantly different, and the actual shape of the target object may have changed. At this time, it can be determined that the first three-dimensional detection frame is credible, and then the height of the first three-dimensional detection frame can be determined as the estimated height.

[0085] In other examples, the estimated height may be determined as a weighted sum of the reference height, the height of the first 3D detection frame, and the height of the second 3D detection frame.

[0086] According to the technical solution provided by the embodiments of the present disclosure, since the reference height and the first 3D detection frame are both determined based on the current point cloud data of the current frame, and the second 3D detection frame is determined based on point cloud data of multiple frames, the reference height and the first 3D detection frame can be compared with the second 3D detection frame respectively, and the validity of the second 3D detection frame can be determined through cross-validation, thereby accurately determining whether the second 3D detection frame is credible.

[0087] Figure 6is a schematic flowchart of a method for determining the height of a target object according to another embodiment of the present disclosure.

[0088] like Figure 6 As shown, the method 600 for determining the height of the target object may include operations S610 to S650, wherein operations S610 to S630 may refer to the above operations S210 to S230 and are not described in detail herein.

[0089] In operation S640, a first weight corresponding to the estimated height and a second weight corresponding to the corrected height are determined according to a tracking stability parameter of the target object.

[0090] For example, the tracking stability parameter may include the actual distance between the target vehicle and the target object. The tracking stability may be negatively correlated with the actual distance, such that when the actual distance is greater, the tracking stability is lower, and when the actual distance is closer, the tracking stability is higher.

[0091] For example, the tracking stability parameter may include the number of detected frames for the target object. The detected frame number represents the number of frames of point cloud data containing the target object. The detected frame number may be the number of frames of point cloud data containing the target object detected within a predetermined period of time. Tracking stability may be positively correlated with the number of detected frames. For example, if only two frames are detected, tracking stability is low. If the number of detected frames is greater than 10, tracking stability is high.

[0092] For example, the first weight and the second weight may be variables related to the tracking stability parameter, for example, the first weight is positively correlated with the tracking stability, and the second weight is negatively correlated with the tracking stability.

[0093] For example, the corrected height may be related to the vehicle chassis height. For example, the corrected height may be the sum of the vehicle chassis height and a height threshold, and the height threshold may be 2 centimeters.

[0094] In operation S650, a height of the target object is determined based on the estimated height, the revised height, the first weight, and the second weight.

[0095] For example, the height of the target object can be determined as the weighted sum of the estimated height and the corrected height. For example, the height of the target object can be calculated using the following formula (1):

[0096] h=α*h1+β*h2 Formula (1)

[0097] Wherein, h represents the height of the target object, α represents the first weight, h1 represents the estimated height, β represents the second weight, h2 represents the corrected height, and the sum of the first weight and the second weight may be 1.

[0098] In related technologies, for distant obstacles, the estimated height determined based on point cloud data is lower than the actual height, which is not conducive to the downstream decision-making module making avoidance decisions such as braking and detouring.

[0099] The disclosed embodiment utilizes tracking stability parameters to determine first and second weights, and then determines the target object's height based on the estimated height, corrected height, and their corresponding weights. For low, distant obstacles, the estimated height can easily be lower than the actual height due to factors such as fewer reflective points. Therefore, a smaller first weight and a larger second weight can be used. This allows the determined height of distant obstacles to be higher than the estimated height, facilitating timely avoidance decisions such as braking and detouring by downstream decision-making modules, thereby improving driving safety and the overall riding experience.

[0100] According to another embodiment of the present disclosure, the above-mentioned operation of determining the first weight corresponding to the estimated height and the second weight corresponding to the corrected height based on the tracking stability parameter may include the following operations: in response to detecting that the actual distance between the target vehicle and the target object is less than a distance threshold, and the number of detected frames for the target object is greater than a quantity threshold, the tracking stability is considered to be high, and the first weight may be greater than the second weight. In response to detecting that the actual distance between the target vehicle and the target object is greater than or equal to the distance threshold, or the number of detected frames for the target object is less than or equal to the quantity threshold, the tracking stability is considered to be low, and the first weight may be less than the second weight.

[0101] For example, the distance threshold may be 40 meters, and the quantity threshold may be 10 frames.

[0102] For example, the first weight and the second weight may be fixed values. For example, it is preset that when the first weight is greater than the second weight, the first weight is 0.6 and the second weight is 0.3. For example, it is preset that when the first weight is less than the second weight, the first weight is 0.4 and the second weight is 0.6.

[0103] In other embodiments, for a target object whose actual distance from the vehicle is greater than the target distance and is stably detected, the height value of the target object can be made equal to the height value of the corrected height, thereby avoiding the height of the target object from converging from a smaller value (such as 0) and accelerating the convergence of the height.

[0104] It should be noted that the target distance can be the larger value of the predetermined distance and the braking distance. The predetermined distance can be 40 meters. The braking distance can be calculated based on the current speed of the vehicle and the predetermined acceleration. The predetermined acceleration can be 1.5m / s. 2 As can be seen, when the vehicle speed is low, the braking distance is short, and the target distance is the predetermined distance. When the vehicle speed is low, the braking distance is long, and the target distance is the braking distance.

[0105] For example, the target distance can be calculated using formula (2) and formula (3):

[0106] S=max(S th , S 刹 ) Formula (2)

[0107]

[0108] Among them, S represents the target distance, S th Indicates the predetermined distance, S 刹 represents the braking distance, v represents the current speed of the vehicle, and a represents the predetermined acceleration.

[0109] Figures 7A and 7B is a schematic diagram of a method for determining the height of a target object according to an embodiment of the present disclosure.

[0110] The following combination Figure 7A and Figure 7B , the method for determining the height of the target object provided by the embodiment of the present disclosure is described.

[0111] A first three-dimensional detection frame 704 corresponding to the target object can be determined based on single-frame point cloud data 701 (e.g., current point cloud data of the current frame). For example, first sub-point cloud data 702 corresponding to the target object in single-frame point cloud data 701 can be determined, and then a first principal direction 703 of the first sub-point cloud data 702 can be determined based on the coordinates of the first sub-point cloud data 702. The first three-dimensional detection frame 704 can then be determined based on the coordinates of the first sub-point cloud data 702 and the first principal direction 703.

[0112] A second three-dimensional detection frame 709 corresponding to the target object can be determined based on multiple frames of point cloud data 705 (e.g., the current point cloud data of the current frame and at least one frame of historical point cloud data). For example, first sub-point cloud data 702 corresponding to the target object can be determined in the current point cloud data. For each historical point cloud data in the at least one frame of historical point cloud data, second sub-point cloud data 706 corresponding to the target object in each historical point cloud data is determined, resulting in at least one second sub-point cloud data 706. The first sub-point cloud data 702 and the at least one second sub-point cloud data 706 are superimposed to obtain third sub-point cloud data 707. Based on the coordinates of the third sub-point cloud data 707, a second principal direction 708 of the third sub-point cloud data 707 is determined. Based on the coordinates of the third sub-point cloud data 707 and the second principal direction 708, a second three-dimensional detection frame 709 is determined.

[0113] Next, a reference height 712 of the target object can be determined based on the current visual detection frame 710 for the target object. For example, the fourth sub-point cloud data 711 within the current visual detection frame 710 for the target object in the single-frame point cloud data 701 can be determined, and the highest height of the fourth sub-point cloud data 711 can be determined as the reference height 712.

[0114] Next, cross-validation can be performed to determine whether the second three-dimensional detection frame 709 is credible. For example, based on the reference height 712 and the height of the second three-dimensional detection frame 709, the first validity 713 of the second three-dimensional detection frame 709 is determined. Based on the length of the first three-dimensional detection frame 704, the width of the first three-dimensional detection frame 704, the length of the second three-dimensional detection frame 709, and the width of the second three-dimensional detection frame 709, the second validity 714 of the second three-dimensional detection frame 709 is determined. Then, based on the first validity 713 and the second validity 714, the estimated height 715 is determined. For example, if both the first validity 713 and the second validity 714 are valid, the second three-dimensional detection frame 709 is determined to be credible, otherwise the first three-dimensional detection frame 704 is determined to be credible. The height of the credible three-dimensional detection frame is then determined as the estimated height 715

[0115] Next, a first weight 717 corresponding to the estimated height 715 and a second weight 718 corresponding to the corrected height 719 may be determined based on a tracking stability parameter 716 of the target object. Tracking stability parameter 716 includes at least one of the following: the actual distance between the target vehicle and the target object, and the number of frames detected for the target object.

[0116] Next, the height 720 of the target object may be determined based on the estimated height 715, the revised height 719, the first weight 717, and the second weight 718. For example, the weighted sum of the estimated height 715 and the revised height 719 may be used as the height 720 of the target object.

[0117] Figure 8 4 is a schematic structural block diagram of an apparatus for determining the height of a target object according to an embodiment of the present disclosure.

[0118] like Figure 8 As shown, the device 800 for determining the height of a target object may include: a first determining module 810 , a second determining module 820 and a third determining module 830 .

[0119] The first determination module 810 is used to determine a first three-dimensional detection box corresponding to the target object based on the current point cloud data of the current frame.

[0120] The second determination module 820 is used to determine a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data.

[0121] The third determination module 830 is configured to determine an estimated height of the target object according to the first 3D detection frame and the second 3D detection frame.

[0122] According to another embodiment of the present disclosure, a first determination module includes: a first determination submodule, a second determination submodule, and a third determination submodule. The first determination submodule is configured to determine first sub-point cloud data corresponding to a target object in the current point cloud data. The second determination submodule is configured to determine a first principal direction of the first sub-point cloud data based on the coordinates of the first sub-point cloud data. The third determination submodule is configured to determine a first three-dimensional detection frame based on the coordinates and the first principal direction of the first sub-point cloud data.

[0123] According to another embodiment of the present disclosure, the second determination module includes: a fourth determination submodule, a fifth determination submodule, an overlay submodule, and a sixth determination submodule. The fourth determination submodule is used to determine the first sub-point cloud data corresponding to the target object in the current point cloud data. The fifth determination submodule is used to determine the second sub-point cloud data corresponding to the target object in each historical point cloud data in at least one frame of historical point cloud data, thereby obtaining at least one second sub-point cloud data. The overlay submodule is used to overlay the first sub-point cloud data and the at least one second sub-point cloud data to obtain a third sub-point cloud data. The sixth determination submodule is used to determine the second three-dimensional detection frame based on the third sub-point cloud data.

[0124] According to another embodiment of the present disclosure, the sixth determination submodule includes: a first determination unit and a second determination unit. The first determination unit is configured to determine a second principal direction of the third sub-point cloud data based on the coordinates of the third sub-point cloud data. The second determination unit is configured to determine a second 3D detection frame based on the coordinates of the third sub-point cloud data and the second principal direction.

[0125] According to another embodiment of the present disclosure, the third determination module includes a seventh determination submodule and an eighth determination submodule. The seventh determination submodule is configured to determine a reference height of the target object based on a current visual detection frame for the target object. The eighth determination submodule is configured to determine an estimated height based on the reference height, the first 3D detection frame, and the second 3D detection frame.

[0126] According to another embodiment of the present disclosure, the eighth determination submodule includes: a third determination unit, a fourth determination unit, and a fifth determination unit. The third determination unit is configured to determine a first validity of the second 3D detection frame based on a reference height and a height of the second 3D detection frame. The fourth determination unit is configured to determine a second validity of the second 3D detection frame based on a length of the first 3D detection frame, a width of the first 3D detection frame, a length of the second 3D detection frame, and a width of the second 3D detection frame. The fifth determination unit is configured to determine an estimated height based on the first validity and the second validity.

[0127] According to another embodiment of the present disclosure, the seventh determination submodule includes: a sixth determination unit and a seventh determination unit. The sixth determination unit is configured to determine fourth sub-point cloud data within the current visual inspection frame within the current point cloud data. The seventh determination unit is configured to determine the highest height of the fourth sub-point cloud data as a reference height.

[0128] According to another embodiment of the present disclosure, the above-mentioned device further includes: a fourth determination module and a fifth determination module. The fourth determination module is used to determine, after determining the estimated height of the target object, a first weight corresponding to the estimated height and a second weight corresponding to the corrected height based on the tracking stability parameter of the target object. The fifth determination module is used to determine the height of the target object based on the estimated height, the corrected height, the first weight, and the second weight. The tracking stability parameter includes at least one of the following: the actual distance between the target vehicle and the target object, and the number of detected frames for the target object. The number of detected frames represents the number of frames of point cloud data containing the target object.

[0129] According to another embodiment of the present disclosure, the fourth determination module includes: a ninth determination submodule and a tenth determination submodule. The ninth determination submodule is configured to determine that the first weight is greater than the second weight in response to detecting that the actual distance between the target vehicle and the target object is less than a distance threshold and the number of detected frames for the target object is greater than a quantity threshold. The tenth determination submodule is configured to determine that the first weight is less than the second weight in response to detecting that the actual distance between the target vehicle and the target object is greater than or equal to the distance threshold, or the number of detected frames for the target object is less than or equal to the quantity threshold.

[0130] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0131] In the technical solution disclosed herein, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0132] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned method for determining the height of the target object.

[0133] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned method for determining the height of a target object.

[0134] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, including a computer program, which implements the above-mentioned method for determining the height of a target object when executed by a processor.

[0135] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0136] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0137] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0138] The computing unit 901 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the method for determining the height of a target object. For example, in some embodiments, the method for determining the height of a target object can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method for determining the height of a target object described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the method for determining the height of a target object by any other appropriate means (e.g., by means of firmware).

[0139] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0144] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0145] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0146] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for determining the height of a target object, comprising: Determine a first three-dimensional detection frame corresponding to the target object based on the current point cloud data of the current frame; Determining a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data; Determining fourth sub-point cloud data in the current point cloud data that is within a current visual inspection frame for the target object; Determine the highest height of the fourth sub-point cloud data as a reference height; determining a first validity of the second three-dimensional detection frame according to the reference height and a height of the second three-dimensional detection frame; determining a second validity of the second three-dimensional detection frame according to the length of the first three-dimensional detection frame, the width of the first three-dimensional detection frame, the length of the second three-dimensional detection frame, and the width of the second three-dimensional detection frame; as well as An estimated height of the target object is determined based on the first validity and the second validity.

2. The method according to claim 1, wherein Determining a first three-dimensional detection frame corresponding to the target object based on current point cloud data of the current frame includes: Determining first sub-point cloud data corresponding to the target object in the current point cloud data; determining a first main direction of the first sub-point cloud data according to the coordinates of the first sub-point cloud data; and The first three-dimensional detection frame is determined according to the coordinates of the first sub-point cloud data and the first main direction.

3. The method according to claim 1, wherein Determining a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data includes: Determining first sub-point cloud data corresponding to the target object in the current point cloud data; For each historical point cloud data in the at least one frame of historical point cloud data, determining second sub-point cloud data corresponding to the target object in each historical point cloud data to obtain at least one second sub-point cloud data; superimposing the first sub-point cloud data and the at least one second sub-point cloud data to obtain third sub-point cloud data; and The second three-dimensional detection frame is determined according to the third sub-point cloud data.

4. The method according to claim 3, wherein: Determining the second three-dimensional detection frame according to the third sub-point cloud data includes: determining a second main direction of the third sub-point cloud data according to the coordinates of the third sub-point cloud data; and The second three-dimensional detection frame is determined according to the coordinates of the third sub-point cloud data and the second main direction.

5. The method according to any one of claims 1 to 4, further comprising: After determining the estimated height of the target object, determining, according to a tracking stability parameter of the target object, a first weight corresponding to the estimated height and a second weight corresponding to the corrected height; as well as determining a height of the target object according to the estimated height, the corrected height, the first weight, and the second weight; The tracking stability parameter includes at least one of the following: an actual distance between the target vehicle and the target object, and a number of detected frames for the target object; the number of detected frames represents the number of frames containing point cloud data of the target object.

6. The method according to claim 5, wherein determining a first weight corresponding to the estimated height and a second weight corresponding to the corrected height based on the tracking stability parameter of the target object comprises: In response to detecting that an actual distance between the target vehicle and the target object is less than a distance threshold and the number of detected frames for the target object is greater than a number threshold, determining that the first weight is greater than the second weight; as well as In response to detecting that the actual distance between the target vehicle and the target object is greater than or equal to a distance threshold, or the number of detected frames for the target object is less than or equal to a quantity threshold, determining that the first weight is less than the second weight.

7. A device for determining the height of a target object, comprising: A first determination module is configured to determine a first three-dimensional detection frame corresponding to the target object based on current point cloud data of a current frame; A second determination module is configured to determine a second three-dimensional detection frame corresponding to the target object based on the current point cloud data and at least one frame of historical point cloud data; a sixth determining unit, configured to determine, in the current point cloud data, fourth sub-point cloud data that is within a current visual inspection frame for the target object; a seventh determining unit, configured to determine the highest height of the fourth sub-point cloud data as a reference height; a third determining unit, configured to determine a first validity of the second three-dimensional detection frame according to the reference height and a height of the second three-dimensional detection frame; a fourth determining unit, configured to determine a second validity of the second three-dimensional detection frame based on the length of the first three-dimensional detection frame, the width of the first three-dimensional detection frame, the length of the second three-dimensional detection frame, and the width of the second three-dimensional detection frame; as well as A fifth determining unit is configured to determine an estimated height of the target object according to the first validity and the second validity.

8. The device according to claim 7, wherein The first determining module includes: A first determining submodule, configured to determine first sub-point cloud data corresponding to the target object in the current point cloud data; a second determining submodule, configured to determine a first main direction of the first sub-point cloud data according to the coordinates of the first sub-point cloud data; and The third determination submodule is configured to determine the first three-dimensional detection frame according to the coordinates of the first sub-point cloud data and the first main direction.

9. The device according to claim 7, wherein The second determining module includes: a fourth determining submodule, configured to determine first sub-point cloud data corresponding to the target object in the current point cloud data; a fifth determining submodule, configured to determine, for each historical point cloud data in the at least one frame of historical point cloud data, second sub-point cloud data corresponding to the target object in the each historical point cloud data, to obtain at least one second sub-point cloud data; a superposition submodule, configured to superimpose the first sub-point cloud data and the at least one second sub-point cloud data to obtain third sub-point cloud data; and The sixth determination submodule is configured to determine the second three-dimensional detection frame based on the third sub-point cloud data.

10. The device according to claim 9, wherein The sixth determining submodule includes: a first determining unit, configured to determine a second main direction of the third sub-point cloud data according to the coordinates of the third sub-point cloud data; and The second determining unit is configured to determine the second three-dimensional detection frame according to the coordinates of the third sub-point cloud data and the second main direction.

11. The apparatus according to any one of claims 7 to 10, further comprising: a fourth determining module, configured to, after determining the estimated height of the target object, determine, based on a tracking stability parameter of the target object, a first weight corresponding to the estimated height and a second weight corresponding to the corrected height; as well as a fifth determining module, configured to determine the height of the target object according to the estimated height, the corrected height, the first weight, and the second weight; The tracking stability parameter includes at least one of the following: an actual distance between the target vehicle and the target object, and a number of detected frames for the target object; the number of detected frames represents the number of frames containing point cloud data of the target object.

12. The apparatus according to claim 11, wherein the fourth determining module comprises: a ninth determining submodule, configured to, in response to detecting that an actual distance between the target vehicle and the target object is less than a distance threshold and a number of detected frames for the target object is greater than a number threshold, determine that the first weight is greater than the second weight; as well as A tenth determining submodule is configured to determine that the first weight is less than the second weight in response to detecting that the actual distance between the target vehicle and the target object is greater than or equal to a distance threshold, or the number of detected frames for the target object is less than or equal to a quantity threshold.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

16. An autonomous driving vehicle comprising the electronic device according to claim 13.

Images