Obstacle height estimation method and apparatus, and vehicle

Abstract

The present application provides an obstacle height estimation method and apparatus, and a vehicle. The method comprises: acquiring point cloud data from a sensor; determining a velocity vector of the origin of a sensor coordinate system on the basis of motion state information of a vehicle and calibration information of the sensor, wherein the vehicle carries the sensor, and the motion state information comprises a linear velocity and an angular velocity vector; and determining three-dimensional estimated values of a point cloud on the basis of the point cloud data and the velocity vector, wherein the three-dimensional estimated values include estimated values of pitch angles and / or heights of points in the point cloud. The present application can be applied to intelligent vehicles or electric vehicles, helping improve the accuracy of obstacle height detection results, and avoiding false alarms caused by misdetection.

Description

Obstacle height estimation method, device and carrier

[0001] The present application claims priority to the Chinese patent application No. 202411340506.7, filed on September 24, 2024, and entitled "Obstacle height estimation method, device and carrier", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of intelligent driving, and more particularly, to an obstacle height estimation method, device and carrier. BACKGROUND

[0003] Currently, vehicles in intelligent driving systems are usually equipped with sensors such as millimeter wave radar (Radar) or camera, for perceiving surrounding environment information, including moving targets and stationary targets. In the surrounding environment perceived by the sensors of the vehicle, obtaining the height of the obstacle plays an increasingly important role, typically, such as height limit bar, gantry, toll barrier, vehicle access bar, roadblock, overpass culvert, etc. Due to the lack of height information or inaccurate estimation of height information, the above obstacles are easily identified as ground obstacles, thereby causing false alarm and leading to unexpected braking. SUMMARY

[0004] The present application provides an obstacle height estimation method, device and carrier, which helps to improve the accuracy of the three-dimensional estimation value of the obstacle and avoid false alarm caused by false detection.

[0005] In a first aspect, the present application provides an obstacle height estimation method, comprising: obtaining point cloud data from a sensor; determining a velocity vector of an origin of a sensor coordinate system according to motion state information of a carrier and calibration information of the sensor, the carrier carrying the sensor, the motion state information including linear velocity and angular velocity vector; determining a three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector, the three-dimensional estimation value including an estimation value of the height of a point in the point cloud.

[0006] Based on the above technical solution, according to the point cloud data and the velocity vector of the origin of the sensor coordinate system, the three-dimensional estimation value of the obstacle can be solved, thereby providing data input for passability, helping to improve the accuracy of the three-dimensional estimation value of the obstacle and avoid false alarm caused by false detection. For example, unnecessary braking and safety accidents such as rear-end collision caused thereby can be avoided.

[0007] In some possible implementation manners, the three-dimensional estimation value of the point cloud can further include an estimation value of the pitch angle of the point in the point cloud.

[0008] In some possible implementation manners, the three-dimensional estimation value of the point cloud can be a three-dimensional estimation value of the point cloud in a carrier coordinate system, or can also be a three-dimensional estimation value in a sensor coordinate system. For example, taking the three-dimensional estimation value of the point cloud for regulation and control as an example, the three-dimensional estimation value of the point cloud can be a three-dimensional estimation value of the point cloud in a carrier coordinate system.

[0009] In some possible implementation manners, the three-dimensional estimation value can include not only the pitch angle and / or the height estimation value, but also one or more of the distance, the azimuth angle, or the three-dimensional position (x, y, z), wherein z can be obtained through the pitch angle or the height value.

[0010] In some possible implementation manners, the three-dimensional estimation value can further include a three-dimensional velocity estimation value.

[0011] With reference to the first aspect, in some implementation manners of the first aspect, the point cloud data includes one or more of a measurement value of a position and a radial velocity of a point in the point cloud relative to the sensor. For example, the position of the point in the point cloud relative to the sensor includes a distance and / or an azimuth angle of the point in the point cloud relative to the sensor coordinate system.

[0012] With reference to the first aspect, in some implementation manners of the first aspect, the three-dimensional estimation value of the point cloud is determined according to the point cloud data and the velocity vector, including: obtaining an estimation value of the pitch angle and / or the height h according to the following relationship:

[0013] wherein is the radial velocity measurement data, θ is the azimuth angle measurement data, v x is the radial velocity measurement data, θ is the azimuth angle measurement data, v y is the radial velocity measurement data, θ is the azimuth angle measurement data, v z is a component of the velocity vector of the origin of the sensor coordinate system, and n is a noise or error term.

[0014] With reference to the first aspect, in some implementation manners of the first aspect, the three-dimensional estimation value of the point cloud is determined according to the point cloud data and the velocity vector, including: determining the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and height indication information, the height indication information being an indication of a height of a point in the point cloud relative to the sensor coordinate system.

[0015] Based on the above technical solution, by combining the height indication information output by the sensor, the accuracy of the three-dimensional estimation value of the obstacle can be further improved, and false alarms caused by false detection can be avoided.

[0016] In some possible implementation manners, the point cloud data includes the height indication information.

[0017] ​In some possible implementation manners, the sensor is a radar, and the height indication above can be understood as that the target point is located above an xy plane in a radar coordinate system, the x direction can be a longitudinal direction, and the y direction can be a transverse direction. For example, the x direction is perpendicular to an antenna array plane of the radar.

[0018] For example, the radar can be a laser radar, a millimeter wave radar, or an ultrasonic radar (sonar), or the like, which can be used to acquire distance, azimuth, and radial velocity.

[0019] With reference to the first aspect, in some implementation manners of the first aspect, the determining the three-dimensional estimated value of the point cloud according to the point cloud data and the velocity vector comprises: determining the three-dimensional estimated value of the point cloud according to the point cloud data, the velocity vector, and a calibration parameter of the sensor.

[0020] Based on the technical solution described above, by combining the calibration parameter of the sensor, the accuracy of the three-dimensional estimated value of the obstacle can be further improved, and false alarm caused by false detection can be avoided.

[0021] In some possible implementation manners, the sensor is a radar, and the calibration information of the radar comprises a height translation parameter of the radar.

[0022] With reference to the first aspect, in some implementation manners of the first aspect, the sensor is a radar, and the method further comprises: classifying the obstacle according to the three-dimensional estimated value of the point cloud and time-domain variation information of a radar cross-section (RCS) output by the sensor, to obtain a classification result, the classification result comprising a floating obstacle or a ground obstacle.

[0023] Based on the technical solution described above, by combining the three-dimensional estimated value and the time-domain variation information of the RCS, the classification result of the obstacle can be obtained. In this way, the planning module of the vehicle can be made to know the classification result of the obstacle, so that planning and control can be more accurately performed, and the driving safety of the user can be improved.

[0024] With reference to the first aspect, in some implementation manners of the first aspect, the method further comprises: sending the three-dimensional estimated value of the point cloud to a planning module.

[0025] Based on the technical solution described above, the above method can be performed by a perception module, and after obtaining the three-dimensional estimated value of the point cloud, the perception module can send the three-dimensional estimated value of the point cloud to a planning module, so that the planning module can perform planning and control based on the three-dimensional estimated value of the point cloud.

[0026] Alternatively, the perception module fuses the three-dimensional estimated value of the point cloud with other information of the perception module, and sends the fused information to the planning module, so that the planning module can perform planning and control based on the fused information, wherein the fused information comprises position information.

[0027] With reference to the first aspect, in some implementations of the first aspect, the method further includes controlling the carrier according to the three-dimensional estimate of the point cloud.

[0028] Based on the above technical solutions, taking an example of the method being executed by an intelligent driving system in a vehicle, the intelligent driving system can control the vehicle based on the three-dimensional estimate of the point cloud.

[0029] In a second aspect, the present application provides an obstacle height estimation device, the device comprising: an acquisition unit configured to acquire point cloud data from a sensor; a determination unit configured to determine a velocity vector of an origin of a sensor coordinate system according to motion state information of a carrier and calibration information of the sensor, the carrier carrying the sensor, the motion state information including a linear velocity and an angular velocity vector; and the determination unit is further configured to determine a three-dimensional estimate of the point cloud according to the point cloud data and the velocity vector, the three-dimensional estimate including an estimate of a height of a point in the point cloud.

[0030] With reference to the second aspect, in some implementations of the second aspect, the determination unit is specifically configured to determine the three-dimensional estimate of the point cloud according to the point cloud data, the velocity vector, and height indication information, the height indication information being a height indication of the point in the point cloud relative to the sensor coordinate system.

[0031] With reference to the second aspect, in some implementations of the second aspect, the determination unit is specifically configured to determine the three-dimensional estimate of the point cloud according to the point cloud data, the velocity vector, and calibration parameters of the sensor.

[0032] With reference to the second aspect, in some implementations of the second aspect, the determination unit is specifically configured to determine the three-dimensional estimate of the point cloud according to the point cloud data, the velocity vector, and identification information, the identification information being determined by data collected by other sensors and / or map information.

[0033] With reference to the second aspect, in some implementations of the second aspect, the determination unit is specifically configured to obtain an estimate of the pitch angle and / or the height h according to the following relationship:

[0034] wherein is radial velocity measurement data, θ is azimuth angle measurement data, v x ,v y and v z are components of the velocity vector of the origin of the sensor coordinate system, and n is a noise or error term.

[0035] With reference to the second aspect, in some implementations of the second aspect, the sensor is a radar, and the apparatus further includes an obstacle classification unit configured to classify the obstacle according to the three-dimensional estimate of the point cloud and time-domain variation information of radar cross section (RCS) output by the sensor, to obtain a classification result, the classification result including a floating obstacle or a ground obstacle.

[0036] With reference to the second aspect, in some implementations of the second aspect, the point cloud data includes a measurement of one or more of a distance, an azimuth angle, or a radial velocity of a point in the point cloud relative to the sensor.

[0037] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a sending unit configured to send the three-dimensional estimate of the point cloud to a planning module.

[0038] With reference to the second aspect, in some implementations of the second aspect, the apparatus further includes a control unit configured to control the carrier according to the three-dimensional estimate of the point cloud.

[0039] In a third aspect, the present application provides an obstacle height estimation apparatus, including a memory and a processor, the memory being configured to store a computer program, and the processor being configured to execute the computer program in the memory, so that the obstacle height estimation apparatus can implement the method in the first aspect and any possible implementation manner thereof.

[0040] In a fourth aspect, the present application provides an obstacle height estimation system, including a perception system and the obstacle height estimation apparatus in the second aspect or the third aspect.

[0041] In a fifth aspect, the present application provides a carrier, including the obstacle height estimation apparatus in any of the second aspect to the third aspect, or including the obstacle height estimation system in the fourth aspect.

[0042] With reference to the fifth aspect, in some implementations of the fifth aspect, the carrier is a vehicle.

[0043] The vehicle in the present application is a vehicle in a broad sense, which can be a traffic tool (such as a commercial vehicle, a passenger vehicle, a motorcycle, a flying vehicle, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural device (such as a mower, a harvester, etc.), a recreational device, a toy vehicle, etc. The type of the vehicle is not limited in the embodiments of the present application.

[0044] In a sixth aspect, the present application provides a computer program product, which comprises computer program codes, and when the computer program codes are run on a computer, the computer program codes make the computer execute the method in any possible implementation manner of the first aspect.

[0045] In a seventh aspect, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is run on a computer, the computer program makes the computer execute the method in any possible implementation manner of the first aspect.

[0046] In an eighth aspect, the present application provides a chip, which comprises a circuit for executing the method in any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0047] FIG. 1 is a functional block diagram of a vehicle according to an embodiment of the present application.

[0048] FIG. 2 is a schematic block diagram of an intelligent driving system according to an embodiment of the present application.

[0049] FIG. 3 is measurement data of a target relative to a vehicle-mounted radar detected by the vehicle-mounted radar.

[0050] FIG. 4 is a schematic flowchart of an obstacle height estimation method according to an embodiment of the present application.

[0051] FIG. 5 is a process of matching the height of an obstacle obtained based on the obstacle height estimation method according to an embodiment of the present application with an image.

[0052] FIG. 6 is a schematic diagram of obstacle classification according to an embodiment of the present application.

[0053] FIG. 7 is another schematic diagram of obstacle classification according to an embodiment of the present application.

[0054] FIG. 8 is a schematic block diagram of an obstacle height estimation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; in this document, "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent: only A exists, A and B exist at the same time, and only B exists. "At least one" means one or more. For example, "at least one of A and B" is similar to "A and / or B", which describes the association relationship of the associated objects, which means that there can be three relationships, for example, at least one of A and B, which can represent: only A exists, A and B exist at the same time, and only B exists.

[0056] In the embodiments of the present application, the prefix words such as "first", "second" are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of ordinal words such as prefixes in the embodiments of the present application does not limit the described objects, and the description of the described objects should be referred to the description of the context in the claims or embodiments, and should not be limited by the use of such prefix words. In addition, in the description of the embodiments, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0057] FIG. 1 is a functional block diagram of a vehicle 100 according to an embodiment of the present application. The vehicle 100 can include a perception system 110, a computing platform 120, and a display device 130, wherein the perception system 110 can include one or more sensors that sense information about the environment around the vehicle 100. For example, the perception system 110 can include a positioning system, which can be a global positioning system (GPS), a Beidou system, or other positioning systems. For another example, the perception system 110 can include one or more of an inertial measurement unit (IMU), an acceleration sensor, a laser radar, a millimeter wave radar, an ultrasonic radar (sonar), and a camera.

[0058] Some or all functions of the vehicle 100 can be controlled by the computing platform 120. The computing platform 120 can include one or more processors, such as processors 121 through 12n (n is a positive integer), which are circuits having a processing capability of signals. In one implementation, the processors can be circuits having an instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a kind of microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processors can be circuits having a certain function implemented by a logic relationship of hardware circuits, which is fixed or reconfigurable. For example, the processors can be hardware circuits implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In the reconfigurable hardware circuit, the processor loads a configuration document to implement the hardware circuit configuration. It can be understood that the processor loads instructions to implement the functions of the above units. In addition, the processors can also be hardware circuits designed for artificial intelligence, which can be understood as a kind of ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like. In addition, the computing platform 120 can also include a memory for storing instructions, and some or all of the processors 121 through 12n can call the instructions in the memory to implement corresponding functions.

[0059] The display device 130 in the cabin is mainly divided into two categories, the first category is a vehicle display screen, and the second category is a projection display screen, such as a head up display (HUD). The vehicle display screen is a physical display screen and is an important component of the in-vehicle infotainment system. Multiple display screens can be provided in the cabin, such as a digital instrument display screen, a center control screen, a display screen in front of a passenger (also referred to as a front passenger) at a co-driver position, a display screen in front of a left rear passenger, and a display screen in front of a right rear passenger, or even a vehicle window can be used as a display screen for display. The head up display, also known as a head-up display system, is mainly used for displaying driving information such as speed, navigation, etc. on a display device (such as a windshield) in front of the driver. This reduces the time for the driver to change his line of sight and avoids changes in the pupil caused by the driver changing his line of sight, thereby improving driving safety and comfort. The HUD includes, for example, a combiner-HUD (C-HUD) system, a windshield-HUD (W-HUD) system, and an augmented reality HUD (AR-HUD). It should be understood that other types of systems can also appear as the technology evolves, and the present application does not limit this.

[0060] The display device 130 described above is illustrated by taking the vehicle display screen and the projection display screen as examples, and embodiments of the present application are not limited thereto. For example, the display device 130 can also be a light display screen or a projection screen.

[0061] Optionally, the structure of the vehicle 100 described above is only schematic, and in actual applications, various components in the vehicle 100 described above can be added or deleted according to actual needs.

[0062] The vehicle 100 can include an intelligent driving system, which can include an advanced driving assistant system (ADAS) and an autonomous driving system (ADS). The intelligent driving system uses various sensors (including but not limited to laser radar, millimeter wave radar, camera, ultrasonic sensor, global positioning system, and inertial measurement unit) on the vehicle to obtain information from the surroundings of the vehicle, and analyzes and processes the obtained information to realize functions such as obstacle perception, target recognition, vehicle positioning, path planning, driver monitoring / reminding, etc., thereby improving the safety, automation level, and comfort of vehicle driving.

[0063] For example, FIG. 2 shows a schematic block diagram of an intelligent driving system according to an embodiment of the present application. The intelligent driving system can include three functional modules: a perception module 210, a planning module 220, and a control module 230. The perception module 210 perceives the environment around the vehicle body through sensors and outputs corresponding perception data to the planning module 220. The planning module 220 obtains information of road elements based on the information obtained by the perception module 210. The planning module 220 can determine the physical connectivity of the vehicle from the current position to a sampling point based on the current position of the vehicle and the information of the road elements, and plan a driving trajectory of the vehicle to the sampling point when the vehicle is physically connected from the current position to the sampling point. The planning module 220 can determine a strategy space of the vehicle according to the driving trajectory. The planning module 220 can send the strategy space to the control module 230. The control module 230 can evaluate in the Euclidean space based on the strategy space, thereby making a behavior decision or an interaction decision of the vehicle.

[0064] The above perception module 210, planning module 220, and control module 230 can be located in the computing platform 120.

[0065] An auxiliary driving system (ADAS) or an unmanned driving system is usually configured with a radar or a camera sensor to perceive surrounding environment information, including moving targets such as vehicles and pedestrians, and static targets such as obstacles, guardrails, road edges, lamp poles, surrounding trees, and buildings. A typical vehicle-mounted radar can provide the following measurement data of the target relative to the sensor: (1) distance r; (2) azimuth angle θ; (3) radial velocity (or distance rate) (4) radar cross section (RCS), etc.

[0066] For example, FIG. 3 shows the measurement data of the target detected by the vehicle-mounted radar relative to the vehicle-mounted radar. The vehicle-mounted radar can easily obtain the two-dimensional position and radial velocity measurement data of the target, but it is often difficult to accurately obtain the height or pitch angle measurement data of the target due to the limitation of implementation cost. In the surrounding environment perceived by the vehicle-mounted radar, the height of the obstacle plays an increasingly important role. Typically, the obstacle is, for example, a height-limiting pole, a gantry, a toll barrier, a vehicle passage pole / vehicle blocking pole, an overpass culvert, etc. Due to the lack of height information, the above obstacles are easily identified as ground obstacles, thereby causing false alarms and leading to unexpected braking. In the field of auxiliary driving or autonomous driving, especially in the autonomous emergency braking (AEB) scenario, such false alarms should be minimized to avoid unnecessary braking and the resulting safety problems such as rear-end collision.

[0067] Embodiments of the present application provide a method, device and carrier for obstacle height estimation, which can be based on position and radial velocity measurement data provided by sensors. The method can accurately estimate the three-dimensional value of the obstacle, thereby providing data input for passability. Taking a vehicle as an example, the method can help avoid unnecessary braking and the resulting safety problems such as rear-end collisions, thereby helping to improve the user's driving experience and driving safety.

[0068] The carrier to which embodiments of the present application can be applied includes but is not limited to a vehicle, an aircraft, a satellite, a sensor system or an intelligent agent system. For example, the vehicle includes but is not limited to a vehicle, a motorcycle or a bicycle, etc. For another example, the aircraft includes but is not limited to a drone, a helicopter or a jet. For another example, the satellite includes but is not limited to a satellite. For another example, the intelligent agent system includes but is not limited to a robot system. The perception system in the above carrier includes sensors for environmental perception, such as sensors configured with radars or cameras, etc. These sensors are configured on the carrier and can provide measurement data of moving or stationary targets around the carrier. For example, taking a vehicle as an example, the moving targets can be vehicles and pedestrians, etc. The stationary targets can be obstacles, guardrails, road edges, lamp posts, surrounding trees and buildings, etc. Taking a millimeter wave radar as an example, the measurement data of the millimeter wave radar can include one or more of the distance of the target relative to the millimeter wave radar, the azimuth angle of the target relative to the millimeter wave radar, the radial velocity of the target relative to the millimeter wave radar, and the RCS of the target relative to the millimeter wave radar.

[0069] FIG. 4 shows a schematic flowchart of an obstacle height estimation method 400 provided by embodiments of the present application. The method 400 can be performed by the above carrier (for example, the vehicle 100), or can also be performed by the above computing platform 120; or can also be performed by a processor, a chip or a circuit in the computing platform 120; or can also be performed by the above intelligent driving system; or can also be performed by the above perception module 210. The method 400 includes:

[0070] S410, obtaining point cloud data.

[0071] Optionally, the point cloud data can include measurement data of a point cloud, which can be a stationary point cloud. The stationary point cloud can be understood as a point cloud that is stationary relative to a reference system such as a geodetic coordinate system.

[0072] For example, the point cloud data includes distance r, azimuth angle θ and radial velocity r measurement data of each point in the point cloud relative to the origin of the sensor coordinate system.

[0073] For example, a stationary point cloud can be understood as part or all of the point cloud acquired by a sensor. For instance, a stationary point cloud can be obtained from the point cloud acquired by the sensor using motion clustering or segmentation methods. Motion clustering or segmentation can be achieved through random sample consensus (RANSAC) algorithms, statistical estimation methods, machine learning, deep learning, or neural networks.

[0074] The distance r and azimuth angle θ of each point in the point cloud relative to the origin of the sensor coordinate system can be called the position measurement data of the point cloud.

[0075] S420, based on the motion state information of the carrier and the calibration information of the sensor, determines the velocity vector v at the origin of the sensor coordinate system. s The carrier carries the sensor.

[0076] For example, v s =[v x v y v z ], where v x Let v be the x-axis component of the velocity vector originating at the sensor coordinate system. y The component of the velocity vector along the y-axis at the origin of the sensor coordinate system, v z The component of the velocity vector along the z-axis of the origin of the sensor coordinate system.

[0077] Optionally, determine the velocity vector v s This includes: acquiring motion state information of a carrier (e.g., vehicle 100), including linear velocity v. ego and angular velocity vector ω ego ; Obtain the calibration information of the sensor, which includes the rotation matrix R and translation vector T from the carrier coordinate system (e.g., vehicle coordinate system) to the sensor coordinate system; Obtain the velocity vector v at the origin of the sensor coordinate system according to the following formula (1). s v s =R(v ego +ω ego ×T)(1)

[0078] Where × represents the vector cross product.

[0079] S430, Based on the point cloud data and the velocity vector of the origin of the sensor coordinate system, determine the three-dimensional estimated value of the point cloud.

[0080] For example, the 3D estimate includes the pitch angle of the target point in the point cloud. And / or an estimated value of the altitude h. The target point can be the pitch angle to be determined in the point cloud. And / or the point where the height h is estimated.

[0081] For example, the three-dimensional estimate may include more than just the pitch angle. The estimated value of height h may also include one or more of the following: distance, azimuth, or three-dimensional position (x, y, z), wherein z may be obtained from the pitch angle or height value mentioned above.

[0082] For example, the three-dimensional estimate may also include a three-dimensional velocity estimate.

[0083] Alternatively, the pitch angle can be determined according to the following formula (2). and / or an estimate of the height h:

[0084] in, Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the three components of the velocity vector at the origin of the sensor coordinate system, and n is the measurement noise or error term. Optionally, n can have a mean of 0 or be non-zero, and a variance of σ. 2 If the mean of n is non-zero, for example, b, then the mean of the noise term can be zero by subtracting b from both sides of the above formula (2). Alternatively, as a simplified implementation, n = 0 can be set.

[0085] Furthermore, based on the above formula (2), the pitch angle The estimated value can be shown in formula (3):

[0086] in v h =v x cosθ+v y sinθ and arcsin represent the arcsine function, and arccos represents the arccosine function.

[0087] Alternatively, the estimated value of height h can be as shown in formula (4):

[0088] Where r represents the distance measurement data mentioned above.

[0089] Optionally, the height h and / or pitch angle can be determined according to the following formula (2B). The estimated value:

[0090] in, Here are the radial velocity measurement data, r is the distance measurement data, θ is the azimuth angle measurement data, and v is the radial velocity measurement data. x vy and v z are three components of the velocity vector of the origin of the sensor coordinate system, respectively, and n is a measurement noise or error term.

[0091] Further, by the above formula (2B), an estimated value of the pitch angle can be obtained as shown in formula (3B):

[0092] wherein r is the above distance measurement data.

[0093] Optionally, the three-dimensional estimated value of the point cloud is determined according to the point cloud data and the velocity vector of the origin of the sensor coordinate system, comprising: determining the three-dimensional estimated value of the point cloud according to the point cloud data, the velocity vector of the origin of the sensor coordinate system and height indication information of the point cloud, wherein the height indication information is the height indication of the point in the point cloud relative to the sensor coordinate system.

[0094] For example, if the height indication of the target point relative to the sensor coordinate system is up, the pitch angle and the height h can be determined by the following formula (5): wherein r is the above distance measurement data; and if the height indication of the target point relative to the sensor coordinate system is down, the pitch angle and the height h can be determined by the following formula (6):

[0095] For example, taking a radar as the sensor, the above height indication up can be understood as that the target point in the point cloud is above the xy plane in the radar coordinate system, wherein the x direction can be the longitudinal direction and the y direction can be the lateral direction; optionally, the x direction is perpendicular to the radar antenna array plane. The height indication down can be understood as that the target point is below the xy plane in the radar coordinate system.

[0096] Optionally, the three-dimensional estimated value of the point cloud is determined according to the point cloud data and the velocity vector of the origin of the sensor coordinate system, comprising: determining the three-dimensional estimated value of the point cloud according to the point cloud data, the velocity vector of the origin of the sensor coordinate system and calibration information of the sensor.

[0097] For example, taking a radar as the sensor, the calibration information comprises a radar translation parameter, wherein the height parameter is t z .

[0098] For example, h0 and h1 can be obtained by the following formulas (7) and (8), respectively:

[0099] wherein the height h of the target point can be the value of h0 and h1 that is positive, or the height h of the target point can be the maximum value of h0 and h1.

[0100] Optionally, the three-dimensional estimation of the point cloud is determined according to the point cloud data and the velocity vector of the origin of the sensor coordinate system, including: determining the three-dimensional estimation of the point cloud according to the point cloud data, the velocity vector of the origin of the sensor coordinate system and the identification information.

[0101] For example, the identification information can be identification information from other sensors, such as visual identification information, or map information from the cloud. The above identification information can be used to select the height information that best matches from (7) or (8), or to select the pitch angle information that best matches from (5) or (6).

[0102] Optionally, the method 400 further includes: determining the three-dimensional estimation of the point cloud in the carrier coordinate system according to the calibration parameters of the sensor coordinate system relative to the carrier coordinate system. For example, the calibration parameters of the sensor coordinate system relative to the carrier coordinate system can include a rotation matrix and a translation vector of the sensor coordinate system to the carrier coordinate system.

[0103] The three-dimensional estimation of the point cloud can be converted to the carrier coordinate system according to the rotation matrix and the translation parameter.

[0104] Optionally, the method 400 further includes: classifying the height of the obstacle based on the above three-dimensional estimation and the time domain variation of the RCS, to obtain a classification result. For example, according to the sequence formed by multiple frames of radar measurement data and the associated data points, based on the variation of the corresponding RCS, it can be judged that the height of the obstacle is a suspended object.

[0105] The three-dimensional estimation can be a three-dimensional estimation of the point cloud in the sensor coordinate system or a three-dimensional estimation of the point cloud in the carrier coordinate system.

[0106] Based on the method 400 provided by the embodiments of the present application, the height information of the obstacle, such as the height of the top obstacle in the tunnel, can be accurately obtained.

[0107] Figure 5 shows the process of matching the height of the obstacle obtained by the obstacle height estimation method provided by the embodiments of the present application with the image.

[0108] As shown in (a) of Figure 5, the vehicle can obtain multiple frames of point cloud data collected by the radar, which are ego-frame#8, ego-frame#9 and ego-frame#10, respectively. The x-axis represents the lateral direction (e.g., the lateral direction is perpendicular to the driving direction of the vehicle), and the y-axis represents the longitudinal direction (e.g., the longitudinal direction is the driving direction of the vehicle). The red connecting line represents the corresponding points in the previous frame of point cloud and the next frame of point cloud.

[0109] As shown in (b) of FIG. 5, the vehicle can estimate the height values of the points in the positive direction (or, left side) of the lateral direction in the ego-frame#10 based on the above method 400.

[0110] As shown in (c) of FIG. 5, the vehicle can estimate the height values of the points in the negative direction (or, right side) of the lateral direction in the ego-frame#10 based on the above method 400.

[0111] As shown in (d) of FIG. 5, the height information of the obstacle, such as the height of the top obstacle in the tunnel, can be accurately obtained based on the method 400 provided in the embodiments of the present application. By projecting the obtained height information into the image, it can be seen that the height of the obstacle obtained by the method 400 provided in the embodiments of the present application is consistent with the height of the obstacle shown in the image. In addition, it can be seen that the points on the tunnel wall are consistent with the points in the point cloud measured by the radar.

[0112] For example, FIGS. 6 and 7 show schematic diagrams of the classification of obstacles provided in the embodiments of the present application.

[0113] For example, the method 400 provided in the embodiments of the present application can be used to classify the overhanging objects, such as the top of the tunnel or the gantry or the viaduct. As shown in FIG. 6, based on the method provided in the embodiments of the present application, it can be accurately determined that the obstacle in front of the vehicle is the top of the tunnel. As shown in FIG. 7, based on the method provided in the embodiments of the present application, it can be accurately determined that the obstacle in front of the vehicle is the gantry.

[0114] FIG. 8 shows a schematic block diagram of an obstacle height estimation device 800 provided in the embodiments of the present application. The device 800 comprises: an acquisition unit 810, configured to acquire point cloud data from a sensor; a determination unit 820, configured to determine a velocity vector of an origin of a sensor coordinate system according to motion state information of a carrier and calibration information of the sensor, the carrier carrying the sensor, the motion state information comprising a linear velocity and an angular velocity vector; and the determination unit 820 is further configured to determine a three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector, the three-dimensional estimation value comprising an estimated value of a pitch angle and / or a height of a point in the point cloud.

[0115] Optionally, the determination unit 820 is specifically configured to determine the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and height indication information, the height indication information being a height indication of the point in the point cloud relative to the sensor coordinate system.

[0116] Optionally, the determination unit 820 is specifically configured to determine the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and the calibration parameters of the sensor.

[0117] Optionally, the determining unit 820 is specifically configured to determine the three-dimensional estimation of the point cloud according to the point cloud data, the speed vector, and identification information determined by data collected by other sensors and / or map information.

[0118] Optionally, the determining unit 820 is specifically configured to obtain the estimation of the pitch angle and / or the height h according to the following relationship:

[0119] wherein is the radial velocity measurement data, θ is the azimuth angle measurement data, v x is the radial velocity measurement data, θ is the azimuth angle measurement data, v y is the radial velocity measurement data, θ is the azimuth angle measurement data, v z is the component of the speed vector of the origin of the sensor coordinate system, and n is a noise or error term.

[0120] Optionally, the sensor is a radar, and the apparatus 800 further includes an obstacle classification unit configured to classify obstacles according to the three-dimensional estimation of the point cloud and time-domain variation information of radar cross section (RCS) output by the sensor to obtain a classification result, the classification result including a floating obstacle or a ground obstacle.

[0121] Optionally, the point cloud data includes one or more of the following: a distance, an azimuth angle, or a radial velocity of a point in the point cloud relative to the sensor.

[0122] Optionally, the apparatus 800 further includes a sending unit configured to send the three-dimensional estimation of the point cloud to a planning module.

[0123] Optionally, the apparatus 800 further includes a control unit configured to control the carrier according to the three-dimensional estimation of the point cloud.

[0124] It should be noted that the above-mentioned three-dimensional estimation can not only include the estimation of the pitch angle and / or the height, but also can include one or more of the following: a distance, an azimuth angle, or a three-dimensional position (x, y, z), wherein z can be obtained by the above-mentioned pitch angle or height value.

[0125] In addition, it should be understood that the above-mentioned three-dimensional estimation can also include a three-dimensional speed estimation.

[0126] ​It should be understood that the division of units in the above apparatus is only a logical functional division, and all or part of them can be integrated into a physical entity or physically separated when actually implemented. In addition, the units in the apparatus can be implemented in the form of processor calling software; for example, the apparatus includes a processor connected with a memory, and the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of the units of the apparatus, wherein the processor is, for example, a general processor such as a CPU or a microprocessor, and the memory is an internal memory of the apparatus or an external memory of the apparatus. Alternatively, the units in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units can be realized by the design of the hardware circuit, which can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of part or all of the units are realized by the design of the logical relationship of elements in the circuit; for example, in another implementation, the hardware circuit is a PLD, and taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the units. All units of the above apparatus can be implemented in the form of processor calling software, or all units can be implemented in the form of hardware circuit, or part of the units can be implemented in the form of processor calling software, and the remaining part can be implemented in the form of hardware circuit.

[0127] In the embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as CPU, microprocessor, GPU, or DSP, etc. In another implementation, the processor can realize certain functions through the logical relationship of the hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured, such as ASIC or PLD implemented hardware circuit, such as FPGA. In the reconfigurable hardware circuit, the process of the processor loading the configuration document to realize the configuration of the hardware circuit can be understood as the process of the processor loading the instructions to realize the functions of part or all of the units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as NPU, TPU, DPU, etc.

[0128] It can be seen that each unit in the above apparatus can be one or more processors (or processing circuits) configured to implement the above method, such as CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0129] Furthermore, all or some of the units in the above apparatus can be integrated or can be independent. In one implementation, these units are integrated to be implemented in the form of SoC. The SoC can include at least one processor for implementing the functions of the above methods or the units of the apparatus, and the at least one processor can be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, and the like.

[0130] Embodiments of the present application further provide an obstacle height estimation apparatus, which comprises a processing unit and a storage unit, wherein the storage unit is configured to store instructions, and the processing unit is configured to execute the instructions stored in the storage unit, so that the apparatus executes the method or the steps performed by the above embodiments.

[0131] Optionally, if the obstacle height estimation apparatus is located in a vehicle, the processing unit can be one or more of the processors 121-12n shown in FIG. 1.

[0132] Embodiments of the present application further provide an obstacle height estimation system, which comprises a perception system and a computing platform, and the computing platform comprises the above obstacle height estimation apparatus 800.

[0133] Embodiments of the present application further provide a carrier, which can comprise the above obstacle height estimation apparatus 800 or the above obstacle height estimation system.

[0134] Optionally, the carrier is a vehicle.

[0135] Embodiments of the present application further provide a computer program product, which comprises computer program code, and when the computer program code is executed on a computer, the computer is caused to execute the method in the above embodiments.

[0136] Embodiments of the present application further provide a computer readable medium, which stores program code, and when the computer program code is executed on a computer, the computer is caused to execute the method in the above embodiments.

[0137] Embodiments of the present application further provide a chip, which comprises a circuit, and the circuit is configured to execute the method in the above embodiments.

[0138] In the implementation process, each step of the above method can be completed by integrated logic circuit of hardware in the processor or instruction in the form of software. The method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or combined execution completion by hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0139] It should be understood that in the embodiments of the present application, the memory can include read-only memory and random access memory, and provide instructions and data to the processor.

[0140] It should also be understood that in various embodiments of the present application, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0141] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0143] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and another division mode can be used in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0145] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0146] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0147] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An obstacle height estimation method characterized by, The method comprises: obtaining point cloud data from a sensor; determining a velocity vector of an origin of a sensor coordinate system according to motion state information of a carrier and calibration information of the sensor, the carrier carrying the sensor, the motion state information comprising a linear velocity and an angular velocity vector; determining a three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector, the three-dimensional estimation value comprising an estimation value of a height of a point in the point cloud.

2. The method of claim 1, wherein, The determining of the three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector comprises: determining the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and height indication information, the height indication information being an indication of a height of a point in the point cloud relative to the sensor coordinate system.

3. The method of claim 1, wherein, The determining of the three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector comprises: determining the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and calibration parameters of the sensor.

4. The method of claim 1, wherein, The determining of the three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector comprises: determining the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and identification information, the identification information being determined by data collected by another sensor and / or map information.

5. The method according to any one of claims 1 to 4, characterized in that, The sensor is a radar, and the method further comprises: The pitch angle of the point cloud is determined according to the following formula and / or an estimate of the height h: wherein Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the velocity vector components of the origin of the sensor coordinate system, and n is the noise or error term.

6. The method according to any one of claims 1 to 5, characterized in that, classifying an obstacle according to the three-dimensional estimation value of the point cloud and time-domain variation information of a radar cross section (RCS) output by the sensor, to obtain a classification result, the classification result comprising a suspended obstacle or a ground obstacle. The point cloud data comprises a measurement value of one or more of a distance, an azimuth angle or a radial velocity of a point in the point cloud relative to the sensor.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises:

8. The method according to any one of claims 1 to 7, characterized in that, sending the three-dimensional estimation value of the point cloud to a planning module. The method further comprises:

9. The method according to any one of claims 1 to 7, characterized in that, controlling the carrier according to the three-dimensional estimation value of the point cloud. The method comprises:

10. An obstacle height estimation apparatus characterized by comprising: an obtaining unit configured to obtain point cloud data from a sensor; a determining unit configured to determine a velocity vector of an origin of a sensor coordinate system according to motion state information of a carrier and calibration information of the sensor, the carrier carrying the sensor, the motion state information comprising a linear velocity and an angular velocity vector; the determining unit is further configured to determine a three-dimensional estimation value of the point cloud according to the point cloud data and the velocity vector, the three-dimensional estimation value comprising an estimation value of a height of a point in the point cloud. The determining unit is specifically configured to:

11. The apparatus of claim 10, wherein, determine the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and height indication information, the height indication information being an indication of a height of a point in the point cloud relative to the sensor coordinate system. The determining unit is specifically configured to:

12. The apparatus of claim 10, wherein, determine the three-dimensional estimation value of the point cloud according to the point cloud data, the velocity vector and calibration parameters of the sensor. The determining unit is specifically configured to:

13. The apparatus of claim 10, wherein, ​ According to the point cloud data, the velocity vector, and identification information determined by data collected by other sensors and / or map information, a three-dimensional estimation of the point cloud is determined.

14. The apparatus of any one of claims 10-13, wherein, The determination unit is specifically configured to: The pitch angle of the point cloud is determined according to the following formula and / or an estimate of the height h: wherein Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the velocity vector components of the origin of the sensor coordinate system, and n is the noise or error term.

15. The apparatus of any one of claims 10 to 14, wherein, The sensor is a radar, and the device further comprises: An obstacle classification unit is configured to classify obstacles according to the three-dimensional estimation of the point cloud and time-domain variation information of radar cross section (RCS) output by the sensor, to obtain a classification result, wherein the classification result includes a suspended obstacle or a ground obstacle.

16. The apparatus of any one of claims 10 to 15, wherein, The point cloud data includes one or more of the following: a distance of a point in the point cloud relative to the sensor, an azimuth angle, or a radial velocity.

17. The apparatus of any one of claims 10-16, wherein, The device further comprises: A sending unit is configured to send the three-dimensional estimation of the point cloud to a planning module.

18. The apparatus of any one of claims 10-16, wherein, The device further comprises: A control unit is configured to control the carrier according to the three-dimensional estimation of the point cloud.

19. An obstacle height estimation apparatus characterized by comprising: The device comprises: A processor is configured to execute a computer program stored in a memory, so that the device performs the method according to any one of claims 1 to 9.

20. The apparatus of claim 19, wherein, The device further comprises the memory.

21. An obstacle height estimation system, characterized by, The control system comprises a perception system and a computing platform, and the computing platform comprises the device according to any one of claims 10 to 20.

22. A vector, comprising: The device according to any one of claims 10 to 20, or the system according to claim 21.

23. The carrier of claim 22, wherein, The carrier is a vehicle.

24. A computer-readable storage medium, characterized in that, An instruction is stored on the computer-readable medium, and the instruction is executed by a processor to cause the processor to implement the method according to any one of claims 1 to 9.

25. A computer program product, characterised in that, The computer program product comprises computer program code, and when the computer program code is executed on a computer, the computer is caused to implement the method according to any one of claims 1 to 9.

26. A chip, characterized by The chip comprises a circuit configured to execute the method according to any one of claims 1 to 9.

Images