Obstacle height estimation method and device and carrier

Abstract

The invention provides an obstacle height estimation method and device and a carrier. The method comprises the following steps: acquiring point cloud data from a sensor; according to the motion state information of a carrier and the calibration information of the sensor, the velocity vector of the origin of the sensor coordinate system is determined, the carrier carries the sensor, and the motion state information comprises linear velocity and angular velocity vectors; and according to the point cloud data and the velocity vector, determining a three-dimensional estimated value of the point cloud, the three-dimensional estimated value including an estimated value of a pitch angle and / or a height of a point in the point cloud. The method can be applied to intelligent automobiles or electric automobiles, the accuracy of obstacle height detection results can be improved, and false alarms caused by false detection are avoided.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more specifically, to an obstacle height estimation method, apparatus, and carrier. Background Technology

[0002] Current vehicles typically incorporate sensors such as millimeter-wave radar or cameras in their intelligent driving systems to perceive surrounding environmental information, including moving and stationary targets. Among these sensors, obtaining the height of obstacles plays an increasingly important role. Typical examples include height restriction barriers, gantries, toll booths, vehicle passage barriers, parking barriers, and overpass culverts. Due to a lack of height information or inaccurate height estimation, these obstacles can easily be identified as ground-level obstacles, triggering false alarms and leading to unintended braking. Summary of the Invention

[0003] This application provides an obstacle height estimation method, apparatus, and carrier, which helps improve the accuracy of the three-dimensional estimation of obstacles and avoid false alarms caused by misdetection.

[0004] In a first aspect, this application provides an obstacle height estimation method, which includes: acquiring point cloud data from a sensor; determining the velocity vector of the origin of the sensor coordinate system based on the motion state information of a carrier and the calibration information of the sensor, wherein the carrier carries the sensor and the motion state information includes linear velocity and angular velocity vectors; and determining a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector, wherein the three-dimensional estimate includes an estimate of the height of the points in the point cloud.

[0005] Based on the above technical solution, the three-dimensional estimated value of the obstacle can be obtained from the point cloud data and the velocity vector at the origin of the sensor coordinate system. This provides data input for accessibility assessment, helps improve the accuracy of the obstacle's three-dimensional estimated value, and avoids false alarms caused by misdetection. For example, it can prevent unnecessary braking and the resulting rear-end collisions and other safety accidents.

[0006] In some possible implementations, the 3D estimate of the point cloud may also include an estimate of the pitch angle of the points in the point cloud.

[0007] In some possible implementations, the 3D estimate of the point cloud can be either a 3D estimate of the point cloud in the carrier coordinate system or a 3D estimate of the point cloud in the sensor coordinate system. For example, taking the use of this 3D estimate of the point cloud for planning and control, the 3D estimate of the point cloud can be a 3D estimate of the point cloud in the carrier coordinate system.

[0008] In some possible implementations, the three-dimensional estimate may include not only pitch and / or altitude estimates, but also one or more of distance, azimuth, or three-dimensional position (x, y, z), where z may be obtained from the pitch or altitude values ​​mentioned above.

[0009] In some possible implementations, the three-dimensional estimate may also include a three-dimensional velocity estimate.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, the point cloud data includes measurements of one or more of the position and radial velocity of points in the point cloud relative to the sensor. For example, the position of a point in the point cloud relative to the sensor includes the distance and / or azimuth angle of the point in the point cloud relative to the sensor's coordinate system.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, determining the three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: obtaining the pitch angle according to the following relationship. and / or an estimated value for the height h:

[0012] in Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x ,v y and v z Let n be the component of the velocity vector at the origin of the sensor coordinate system, and n be the noise or error term.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: determining a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and height indication information, wherein the height indication information is a height indication of a point in the point cloud relative to the sensor coordinate system.

[0014] Based on the above technical solution, by combining the height indication information output by the sensor, it is helpful to further improve the accuracy of the three-dimensional estimation of obstacles and avoid false alarms caused by misdetection.

[0015] In some possible implementations, the point cloud data includes the height indication information.

[0016] In some possible implementations, the sensor is a radar. The radar altitude indication can be understood as the target point being located above the xy plane in the radar coordinate system, where the x-direction can be the longitudinal direction and the y-direction can be the lateral direction. For example, the x-direction is perpendicular to the radar antenna array.

[0017] For example, the radar can be a sensor such as lidar, millimeter-wave radar, or ultrasonic radar (sonar) that can be used to obtain distance, azimuth, and radial velocity.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, determining a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: determining a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and the calibration parameters of the sensor.

[0019] Based on the above technical solution, by combining the calibration parameters of the sensor, it is helpful to further improve the accuracy of the three-dimensional estimation of obstacles and avoid false alarms caused by false detection.

[0020] In some possible implementations, the sensor is a radar, and the radar's calibration information includes the radar's altitude translation parameters.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the sensor is a radar, and the method further includes: classifying obstacles based on the three-dimensional estimated value of the point cloud and the temporal variation information of the radar cross-section (RCS) output by the sensor, and obtaining a classification result, which includes suspended obstacles or ground obstacles.

[0022] Based on the above technical solution, by combining the three-dimensional estimated values ​​and the temporal variation information of RCS, obstacle classification results can be obtained. This allows the vehicle's planning module to clearly understand the obstacle classification results, thereby enabling more accurate planning and control, and contributing to improved driving safety for users.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: sending a three-dimensional estimate of the point cloud to the planning module.

[0024] Based on the above technical solution, the above method can be executed by the perception module. After obtaining the three-dimensional estimate of the point cloud, the perception module can send the three-dimensional estimate of the point cloud to the planning module, so that the planning module can perform planning and control based on the three-dimensional estimate of the point cloud.

[0025] Alternatively, the perception module can fuse the 3D estimate of the point cloud with other information from the perception module and send the fused information to the planning module, thereby enabling the planning module to perform planning and control based on the fused information, which includes location information.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: controlling the carrier based on the three-dimensional estimated value of the point cloud.

[0027] Based on the above technical solution, taking the method executed by the intelligent driving system in the vehicle as an example, the intelligent driving system can control the vehicle based on the three-dimensional estimation value of the point cloud.

[0028] Secondly, this application provides an obstacle height estimation device, which includes: an acquisition unit for acquiring point cloud data from a sensor; a determination unit for determining the velocity vector of the origin of the sensor coordinate system based on the motion state information of a carrier and the calibration information of the sensor, wherein the carrier carries the sensor, and the motion state information includes linear velocity and angular velocity vectors; the determination unit is further configured to determine a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector, wherein the three-dimensional estimate includes an estimate of the height of a point in the point cloud.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is specifically used to: determine a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and the height indication information, wherein the height indication information is a height indication of a point in the point cloud relative to the sensor coordinate system.

[0030] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is specifically used to: determine the three-dimensional estimated value of the point cloud based on the point cloud data, the velocity vector, and the calibration parameters of the sensor.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is specifically used to: determine a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and identification information, wherein the identification information is determined from data collected by other sensors and / or map information.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, the determining unit is specifically used to: obtain the pitch angle according to the following relationship. and / or an estimated value for the height h:

[0033] in Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x ,v y and v z Let n be the component of the velocity vector at the origin of the sensor coordinate system, and n be the noise or error term.

[0034] In conjunction with the second aspect, in some implementations of the second aspect, the sensor is a radar, and the device further includes: an obstacle classification unit, used to classify obstacles based on the three-dimensional estimated value of the point cloud and the temporal variation information of the radar cross-section (RCS) output by the sensor, and obtain a classification result, which includes suspended obstacles or ground obstacles.

[0035] In conjunction with the second aspect, in some implementations of the second aspect, the point cloud data includes measurements of one or more of the following: distance, azimuth angle, or radial velocity of points in the point cloud relative to the sensor.

[0036] In conjunction with the second aspect, in some implementations of the second aspect, the device further includes: a transmitting unit for transmitting a three-dimensional estimate of the point cloud to the planning module.

[0037] In conjunction with the second aspect, in some implementations of the second aspect, the device further includes a control unit for controlling the carrier based on a three-dimensional estimate of the point cloud.

[0038] Thirdly, this application provides an obstacle height estimation device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program in the memory, so that the obstacle height estimation device can implement the methods in the first aspect and any possible implementation thereof.

[0039] Fourthly, this application provides an obstacle height estimation system, which includes a sensing system and the obstacle height estimation device described in the second or third aspect above.

[0040] Fifthly, this application provides a carrier that includes an obstacle height estimation device possible in any of the second to third aspects described above, or includes the obstacle height estimation system described in the fourth aspect described above.

[0041] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the carrier is a vehicle.

[0042] The term "vehicle" in this application is used in a broad sense and can refer to means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. The embodiments of this application do not specifically limit the type of vehicle.

[0043] In a sixth aspect, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the method in any possible implementation of the first aspect.

[0044] In a seventh aspect, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method in any possible implementation of the first aspect.

[0045] Eighthly, this application provides a chip including circuitry for performing the method in any possible implementation of the first aspect described above. Attached Figure Description

[0046] Figure 1 This is a functional block diagram of the vehicle provided in the embodiments of this application.

[0047] Figure 2 This is a schematic block diagram of the intelligent driving system provided in the embodiments of this application.

[0048] Figure 3 It is the measurement data of the target relative to the vehicle-mounted radar, obtained by the vehicle-mounted radar.

[0049] Figure 4 This is a schematic flowchart of the obstacle height estimation method provided in the embodiments of this application.

[0050] Figure 5 This is a process of matching the height of an obstacle with an image based on the obstacle height estimation method provided in the embodiments of this application.

[0051] Figure 6 This is a schematic diagram of obstacle classification provided in an embodiment of this application.

[0052] Figure 7 This is another schematic diagram of obstacle classification provided in the embodiments of this application.

[0053] Figure 8 This is a schematic block diagram of the obstacle height estimation device provided in the embodiments of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. "At least one" refers to one or more. For example, "at least one of A and B," similar to "A and / or B," describes the association relationship between related objects, indicating that three relationships can exist. For example, at least one of A and B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0055] The prefixes such as "first" and "second" used in this application embodiment are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not constitute unnecessary restrictions due to the use of such prefixes. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0056] Figure 1 This is a functional block diagram of a vehicle 100 provided in an embodiment of this application. The vehicle 100 may include a sensing system 110, a computing platform 120, and a display device 130. The sensing system 110 may include one or more sensors for sensing information about the environment surrounding the vehicle 100. For example, the sensing system 110 may include a positioning system, which may be a Global Positioning System (GPS), a BeiDou system, or another positioning system. As another example, the sensing system 110 may include one or more of the following: an inertial measurement unit (IMU), an accelerometer, a lidar, a millimeter-wave radar, an ultrasonic radar (sonar), and a camera device.

[0057] Some or all of the functions of vehicle 100 can be controlled by computing platform 120. Computing platform 120 may include one or more processors, such as processors 121 to 12n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 120 may also include a memory for storing instructions. Some or all of the processors 121 to 12n can call the instructions in the memory to implement the corresponding functions.

[0058] The in-cabin display devices 130 are mainly divided into two categories: the first is the in-vehicle display screen; the second is the projection display screen, such as the head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as the digital instrument cluster display, the central control screen, the display screen in front of the front passenger (also known as the front-seat passenger), the display screen in front of the left rear passenger, the display screen in front of the right rear passenger, and even the car window can be used as a display screen. A head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's eye-shift time, avoids pupil changes caused by eye-shifting, and improves driving safety and comfort. Examples of HUDs include combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems. It should be understood that HUDs can also evolve into other types of systems as technology progresses, and this application does not limit them.

[0059] The above description of the display device 130 uses an in-vehicle display screen and a projection display screen as examples, but the embodiments of this application are not limited thereto. For example, the display device 130 can also be a light display screen or a projection screen.

[0060] Optionally, the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.

[0061] Vehicle 100 may include an intelligent driving system, which may include an advanced driving assistant system (ADAS) and an autonomous driving system (ADS). The intelligent driving system uses various sensors on the vehicle (including but not limited to: lidar, millimeter-wave radar, camera devices, ultrasonic sensors, global positioning system, inertial measurement unit) to acquire information from the vehicle's surroundings, and analyzes and processes the acquired information to achieve functions such as obstacle perception, target recognition, vehicle positioning, path planning, and driver monitoring / alerts, thereby improving the safety, automation, and comfort of driving the vehicle.

[0062] For example, Figure 2A schematic block diagram of an intelligent driving system provided in an embodiment of this application is shown. The intelligent driving system may include three functional modules: a perception module 210, a planning module 220, and a control module 230. The perception module 210 perceives the environment surrounding the vehicle through sensors and outputs corresponding perception data to the planning module 220. The planning module 220 obtains road element information based on the information acquired by the perception module 210. Based on the vehicle's current location and the road element information, the planning module 220 determines the physical connectivity of the vehicle from its current location to a sampling point, and plans the vehicle's trajectory to that sampling point when the vehicle is physically connected from its current location to that sampling point. The planning module 220 determines the vehicle's strategy space based on this trajectory. The planning module 220 can send this strategy space to the control module 230. The control module 230 can evaluate the strategy space in Euclidean space to make behavioral or interactive decisions for the vehicle.

[0063] The above-mentioned sensing module 210, planning module 220 and control module 230 can be located in the above-mentioned computing platform 120.

[0064] Advanced Driver Assistance Systems (ADAS) or autonomous driving systems typically incorporate sensors such as radar or cameras to perceive information about the surrounding environment, including moving and stationary targets. Moving targets include vehicles and pedestrians, while stationary targets include obstacles, guardrails, curbs, lampposts, surrounding trees, and buildings. A typical vehicle-mounted radar can provide the following measurement data relative to the sensor: (1) range r; (2) azimuth angle θ; and (3) radial velocity (or range rate). (4) Radar cross section (RCS), etc.

[0065] For example, Figure 3 The diagram illustrates measurement data of a target relative to the vehicle's radar, detected by the vehicle-mounted radar. While vehicle-mounted radar can typically acquire two-dimensional position and radial velocity measurements of a target, it often struggles to accurately determine its height or pitch angle due to cost limitations. In the surrounding environment perceived by vehicle-mounted radar, the height of obstacles plays an increasingly important role. Typical obstacles include height restriction barriers, gantries, toll booths, vehicle access barriers / stops, and overpass culverts. Due to the lack of height information, these obstacles are easily mistaken for ground-level obstacles, triggering false alarms and leading to unintended braking. In the fields of assisted driving or autonomous driving, especially in automatic emergency braking (AEB) scenarios, such false alarms should be minimized to avoid unnecessary braking and subsequent rear-end collisions and other safety issues.

[0066] This application provides a method, apparatus, and carrier for obstacle height estimation, which can be based on position and radial velocity measurement data provided by sensors. This method can accurately estimate the three-dimensional value of the obstacle, thereby providing data input for accessibility. Taking a vehicle as an example, this helps avoid unnecessary braking and the resulting rear-end collisions, thus improving the user's driving experience and safety.

[0067] The carriers applicable to the embodiments of this application include, but are not limited to, vehicle-mounted, airborne, spaceborne, sensor systems, or intelligent agent systems. For example, vehicle-mounted carriers include, but are not limited to, vehicles, motorcycles, or bicycles. Airborne carriers include, but are not limited to, drones, helicopters, or jet aircraft. Spaceborne carriers include, but are not limited to, satellites. Intelligent agent systems include, but are not limited to, robotic systems. The perception system in the aforementioned carriers includes sensors for environmental perception, such as sensors configured with radar or cameras. These sensors are configured on the carrier and can provide measurement data of moving or stationary targets around the carrier. For example, taking a vehicle as an example, moving targets can be vehicles and pedestrians. Stationary targets can be obstacles, guardrails, curbs, lampposts, surrounding trees, and buildings. Taking millimeter-wave radar as an example, the measurement data from the millimeter-wave radar can include one or more of the following: the target's distance relative to the millimeter-wave radar, the target's azimuth angle relative to the millimeter-wave radar, the target's radial velocity relative to the millimeter-wave radar, and the target's RCS relative to the millimeter-wave radar.

[0068] Figure 4 A schematic flowchart of an obstacle height estimation method 400 provided in an embodiment of this application is shown. This method 400 can be executed by the aforementioned carrier (e.g., vehicle 100), or by the aforementioned computing platform 120; or by a processor, chip, or circuit in the computing platform 120; or by the aforementioned intelligent driving system; or by the aforementioned perception module 210. The method 400 includes:

[0069] S410, acquire point cloud data.

[0070] Optionally, the point cloud data may include measurement data of the point cloud, which may be a stationary point cloud. A stationary point cloud can be understood as a point cloud that is stationary relative to a reference frame such as a geodetic coordinate system.

[0071] For example, the point cloud data includes the distance r, azimuth angle θ, and radial velocity of each point in the point cloud relative to the origin of the sensor coordinate system. Measurement data.

[0072] For example, a stationary point cloud can be understood as part or all of the point cloud acquired by a sensor. For instance, a stationary point cloud can be obtained from the point cloud acquired by the sensor using motion clustering or segmentation methods. Motion clustering or segmentation can be achieved through random sample consensus (RANSAC) algorithms, statistical estimation methods, machine learning, deep learning, or neural networks.

[0073] The distance r and azimuth angle θ of each point in the point cloud relative to the origin of the sensor coordinate system can be called the position measurement data of the point cloud.

[0074] S420, based on the motion state information of the carrier and the calibration information of the sensor, determines the velocity vector v at the origin of the sensor coordinate system. s The carrier carries the sensor.

[0075] For example, v s =[v x V y v z ], where v x Let v be the x-axis component of the velocity vector originating at the sensor coordinate system. y V is the component of the velocity vector along the y-axis at the origin of the sensor coordinate system. z The component of the velocity vector along the z-axis of the origin of the sensor coordinate system.

[0076] Optionally, determine the velocity vector V s This includes: acquiring motion state information of a carrier (e.g., vehicle 100), including linear velocity v. ego and angular velocity vector ω ego ; Obtain the calibration information of the sensor, which includes the rotation matrix R and translation vector T from the carrier coordinate system (e.g., vehicle coordinate system) to the sensor coordinate system; Obtain the velocity vector v at the origin of the sensor coordinate system according to the following formula (1). s :

[0077] v s =R(v ego +ω ego ×T) (1)

[0078] Where × represents the vector cross product.

[0079] S430, Based on the point cloud data and the velocity vector of the origin of the sensor coordinate system, determine the three-dimensional estimated value of the point cloud.

[0080] For example, the 3D estimate includes the pitch angle of the target point in the point cloud. And / or an estimated value of the altitude h. The target point can be the pitch angle to be determined in the point cloud. And / or the point where the height h is estimated.

[0081] For example, the three-dimensional estimate may include more than just the pitch angle. The estimated value of height h may also include one or more of the following: distance, azimuth, or three-dimensional position (x, y, z), wherein z may be obtained from the pitch angle or height value mentioned above.

[0082] For example, the three-dimensional estimate may also include a three-dimensional velocity estimate.

[0083] Alternatively, the pitch angle can be determined according to the following formula (2). and / or an estimated value for the height h:

[0084]

[0085] in, Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the three components of the velocity vector at the origin of the sensor coordinate system, and n is the measurement noise or error term. Optionally, n can have a mean of 0 or be non-zero, and a variance of σ. 2 If the mean of n is non-zero, for example, b, then the mean of the noise term can be zero by subtracting b from both sides of the above formula (2). Alternatively, as a simplified implementation, n = 0 can be set.

[0086] Furthermore, based on the above formula (2), the pitch angle The estimated value can be shown in formula (3):

[0087]

[0088] in v h =v x cosθ+v y sinθ and arcsin represent the arcsine function, and arccos represents the arccosine function.

[0089] Alternatively, the estimated value of height h can be as shown in formula (4):

[0090]

[0091] Where r represents the distance measurement data mentioned above.

[0092] Optionally, the height h and / or pitch angle can be determined according to the following formula (2B). The estimated value:

[0093]

[0094] in, Here are the radial velocity measurement data, r is the distance measurement data, θ is the azimuth angle measurement data, and v is the radial velocity measurement data. x v y and v z These are the three components of the velocity vector at the origin of the sensor coordinate system, and n is the measurement noise or error term.

[0095] Furthermore, using the above formula (2B), the pitch angle shown in formula (3B) can be obtained. The estimated value:

[0096]

[0097] Where r represents the distance measurement data mentioned above.

[0098] Optionally, determining the three-dimensional estimated value of the point cloud based on the point cloud data and the velocity vector of the origin of the sensor coordinate system includes: determining the three-dimensional estimated value of the point cloud based on the point cloud data, the velocity vector of the origin of the sensor coordinate system, and height indication information, wherein the height indication information is the height indication of the points in the point cloud relative to the sensor coordinate system.

[0099] For example, if the height indication of the target point relative to the sensor coordinate system is upward, the pitch angle can be determined by the following formula (5). And height h; or, if the height indication of the target point relative to the sensor coordinate system is downward, the pitch angle is determined by the following formula (6). And height h.

[0100]

[0101] For example, taking a radar sensor as an example, the "height indicator above" can be understood as the target point in the point cloud being located above the xy plane in the radar coordinate system, where the x-direction can be the vertical direction and the y-direction can be the horizontal direction; optionally, the x-direction is perpendicular to the radar antenna array. The "height indicator below" can be understood as the target point being located below the xy plane in the radar coordinate system.

[0102] Optionally, the three-dimensional estimated value of the point cloud is determined based on the point cloud data and the velocity vector of the origin of the sensor coordinate system, including: determining the three-dimensional estimated value of the point cloud based on the point cloud data, the velocity vector of the origin of the sensor coordinate system, and the calibration information of the sensor.

[0103] For example, taking a radar sensor as an example, the calibration information includes radar translation parameters, including a height parameter t. z .

[0104] For example, h0 and h1 can be obtained by the following formulas (7) and (8), respectively:

[0105]

[0106] The height h of the target point can be a positive value between h0 and h1, or the height h of the target point can be the maximum value between h0 and h1.

[0107] Optionally, the three-dimensional estimated value of the point cloud is determined based on the point cloud data and the velocity vector of the origin of the sensor coordinate system, including: determining the three-dimensional estimated value of the point cloud based on the point cloud data, the velocity vector of the origin of the sensor coordinate system, and the recognition information.

[0108] For example, the identification information may be identification information from other sensors, such as visual identification information or map information from the cloud. The identification information can be used to select the best matching height information from (7) or (8), or to select the best matching pitch angle information from (5) or (6).

[0109] Optionally, the method 400 further includes: determining the three-dimensional estimated value of the point cloud in the carrier coordinate system based on the calibration parameters of the sensor coordinate system relative to the carrier coordinate system. For example, the calibration parameters of the sensor coordinate system relative to the carrier coordinate system may include a rotation matrix and a translation vector from the sensor coordinate system to the carrier coordinate system.

[0110] The three-dimensional estimated values ​​of the point cloud can be transformed into the carrier coordinate system based on the rotation matrix and translation parameters mentioned above.

[0111] Optionally, the method 400 further includes: classifying the height of the obstacle based on the above three-dimensional estimated value and the time-domain change of RCS, to obtain a classification result. For example, based on the sequence formed by multiple frames of radar measurement data and their associated data points, and based on the corresponding changes in RCS, it can be determined that the height of the obstacle is a suspended object.

[0112] The three-dimensional estimated value can be a three-dimensional estimated value of the point cloud in the sensor coordinate system or a three-dimensional estimated value of the point cloud in the carrier coordinate system.

[0113] Based on the method 400 provided in the embodiments of this application, the height information of obstacles can be accurately obtained, such as the height of obstacles at the top of a tunnel.

[0114] Figure 5 The process of matching the height of an obstacle with an image based on the obstacle height estimation method provided in the embodiments of this application is illustrated.

[0115] like Figure 5As shown in (a), the vehicle can acquire multiple frames of point cloud data collected by the radar, namely ego-frame#8, ego-frame#9, and ego-frame#10. The x-axis represents the lateral direction (e.g., the lateral direction is perpendicular to the vehicle's driving direction), and the y-axis represents the longitudinal direction (e.g., the longitudinal direction is the vehicle's driving direction). The red lines indicate the corresponding points in the previous frame and the next frame of the point cloud.

[0116] like Figure 5 As shown in (b), the vehicle can estimate the height values ​​of each point in the positive direction (or left side) of the ego-frame#10 frame data obtained by the above method 400.

[0117] like Figure 5 As shown in (c), the vehicle can estimate the height values ​​of each point in the negative direction (or right side) of the ego-frame#10 frame data obtained by the above method 400.

[0118] like Figure 5 As shown in (d) of this application, the height information of obstacles, such as the height of obstacles at the top of a tunnel, can be accurately obtained based on the method 400 described in this application embodiment. By projecting the obtained height information into an image, it can be seen that the height of the obstacle obtained by the method 400 described in this application embodiment matches the height of the obstacle shown in the image. In addition, it can be seen that the height of the points on the tunnel wall matches the height of the points in the point cloud measured by radar.

[0119] For example, Figure 6 and Figure 7 A schematic diagram illustrating obstacle classification provided in an embodiment of this application is shown.

[0120] For example, the method 400 provided in the embodiments of this application classifies suspended objects, such as the roof of a tunnel, a gantry, or under a viaduct. Figure 6 As shown, based on the method provided in this application embodiment, it is possible to accurately determine that the obstacle in front of the vehicle is the tunnel ceiling. Figure 7 As shown, based on the method provided in the embodiments of this application, it is possible to accurately determine that the obstacle in front of the vehicle is a gantry.

[0121] Figure 8A schematic block diagram of an obstacle height estimation device 800 provided in an embodiment of this application is shown. The device 800 includes: an acquisition unit 810 for acquiring point cloud data from a sensor; a determination unit 820 for determining the velocity vector of the origin of the sensor coordinate system based on motion state information of a carrier and calibration information of the sensor, wherein the carrier carries the sensor, and the motion state information includes linear velocity and angular velocity vectors; the determination unit 820 is further configured to determine a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector, the three-dimensional estimate including an estimate of the pitch angle and / or height of a point in the point cloud.

[0122] Optionally, the determining unit 820 is specifically used to: determine a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and the height indication information, wherein the height indication information is the height indication of a point in the point cloud relative to the sensor coordinate system.

[0123] Optionally, the determining unit 820 is specifically used to: determine the three-dimensional estimated value of the point cloud based on the point cloud data, the velocity vector, and the calibration parameters of the sensor.

[0124] Optionally, the determining unit 820 is specifically used to: determine a three-dimensional estimate of the point cloud based on the point cloud data, the velocity vector, and the identification information, wherein the identification information is determined by data collected by other sensors and / or map information.

[0125] Optionally, the determining unit 820 is specifically used to: obtain the pitch angle according to the following relationship and / or an estimated value for the height h:

[0126] in Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x ,v y and v z Let n be the component of the velocity vector at the origin of the sensor coordinate system, and n be the noise or error term.

[0127] Optionally, the sensor is a radar, and the device 800 further includes an obstacle classification unit, used to classify obstacles based on the three-dimensional estimated value of the point cloud and the temporal variation information of the radar cross-section (RCS) output by the sensor, and obtain a classification result, which includes suspended obstacles or ground obstacles.

[0128] Optionally, the point cloud data includes measurements of one or more of the following: distance, azimuth angle, or radial velocity of points in the point cloud relative to the sensor.

[0129] Optionally, the device 800 further includes a transmitting unit for transmitting a three-dimensional estimate of the point cloud to the planning module.

[0130] Optionally, the device 800 further includes a control unit for controlling the carrier based on the three-dimensional estimate of the point cloud.

[0131] It should be noted that the three-dimensional estimates mentioned above may include not only pitch angle and / or altitude estimates, but also one or more of distance, azimuth, or three-dimensional position (x, y, z), where z may be obtained through the pitch angle or altitude values ​​mentioned above.

[0132] Furthermore, it should be understood that the three-dimensional estimates described above may also include three-dimensional velocity estimates.

[0133] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0134] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0135] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0136] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a System-on-a-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0137] This application also provides an obstacle height estimation device, which includes a processing unit and a storage unit. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the device to perform the methods or steps described in the above embodiments.

[0138] Optionally, if the obstacle height estimation device is located in a vehicle, the aforementioned processing unit may be... Figure 1 One or more of the processors 121-12n shown.

[0139] This application embodiment also provides an obstacle height estimation system, which includes a sensing system and a computing platform, the computing platform including the obstacle height estimation device 800 described above.

[0140] This application also provides a carrier, which may include the obstacle height estimation device 800 or the obstacle height estimation system described above.

[0141] Alternatively, the carrier may be a vehicle.

[0142] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0143] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0144] This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.

[0145] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0146] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0147] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0153] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for estimating obstacle height, characterized in that, include: Acquire point cloud data from sensors; Based on the motion state information of the carrier and the calibration information of the sensor, the velocity vector of the origin of the sensor coordinate system is determined. The carrier carries the sensor, and the motion state information includes linear velocity and angular velocity vectors. Based on the point cloud data and the velocity vector, a three-dimensional estimate of the point cloud is determined, including an estimate of the height of the points in the point cloud.

2. The method according to claim 1, characterized in that, Determining the three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: Based on the point cloud data, the velocity vector, and the height indication information, a three-dimensional estimate of the point cloud is determined, wherein the height indication information is the height indication of a point in the point cloud relative to the sensor coordinate system.

3. The method according to claim 1, characterized in that, Determining the three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: The three-dimensional estimated value of the point cloud is determined based on the point cloud data, the velocity vector, and the calibration parameters of the sensor.

4. The method according to claim 1, characterized in that, Determining the three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: Based on the point cloud data, the velocity vector, and the identification information, a three-dimensional estimate of the point cloud is determined, wherein the identification information is determined from data collected by other sensors and / or map information.

5. The method according to any one of claims 1 to 4, characterized in that, Determining the three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector includes: The pitch angle of the point cloud is determined according to the following formula. and / or an estimated value for the height h: in, Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the velocity vector components of the origin of the sensor coordinate system, and n is the noise or error term.

6. The method according to any one of claims 1 to 5, characterized in that, The sensor is a radar, and the method further includes: Based on the three-dimensional estimated value of the point cloud and the temporal variation information of the radar cross-section (RCS) output by the sensor, obstacles are classified to obtain classification results, which include suspended obstacles or ground obstacles.

7. The method according to any one of claims 1 to 6, characterized in that, The point cloud data includes measurements of one or more of the following: distance, azimuth angle, or radial velocity of a point in the point cloud relative to the sensor.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Send the three-dimensional estimate of the point cloud to the planning module.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The carrier is controlled based on the three-dimensional estimation values ​​of the point cloud.

10. An obstacle height estimation device, characterized in that, include: Acquisition unit, used to acquire point cloud data from sensors; The determining unit is used to determine the velocity vector of the origin of the sensor coordinate system based on the motion state information of the carrier and the calibration information of the sensor. The carrier carries the sensor, and the motion state information includes linear velocity and angular velocity vectors. The determining unit is further configured to determine a three-dimensional estimate of the point cloud based on the point cloud data and the velocity vector, wherein the three-dimensional estimate includes an estimate of the height of the points in the point cloud.

11. The apparatus according to claim 10, characterized in that, The determining unit is specifically used for: Based on the point cloud data, the velocity vector, and the height indication information, a three-dimensional estimate of the point cloud is determined, wherein the height indication information is the height indication of a point in the point cloud relative to the sensor coordinate system.

12. The apparatus according to claim 10, characterized in that, The determining unit is specifically used for: The three-dimensional estimated value of the point cloud is determined based on the point cloud data, the velocity vector, and the calibration parameters of the sensor.

13. The apparatus according to claim 10, characterized in that, The determining unit is specifically used for: Based on the point cloud data, the velocity vector, and the identification information, a three-dimensional estimate of the point cloud is determined, wherein the identification information is determined from data collected by other sensors and / or map information.

14. The apparatus according to any one of claims 10 to 13, characterized in that, The determining unit is specifically used for: The pitch angle of the point cloud is determined according to the following formula. and / or an estimate of the height h: in, Here are the radial velocity measurement data, θ is the azimuth angle measurement data, and v x v y and v z These are the velocity vector components of the origin of the sensor coordinate system, and n is the noise or error term.

15. The apparatus according to any one of claims 10 to 14, characterized in that, The sensor is a radar, and the device further includes: An obstacle classification unit is used to classify obstacles based on the three-dimensional estimated value of the point cloud and the temporal variation information of the radar cross-section (RCS) output by the sensor, and obtain classification results, which include suspended obstacles or ground obstacles.

16. The apparatus according to any one of claims 10 to 15, characterized in that, The point cloud data includes measurements of one or more of the following: distance, azimuth angle, or radial velocity of a point in the point cloud relative to the sensor.

17. The apparatus according to any one of claims 10 to 16, characterized in that, The device further includes: The sending unit is used to send the three-dimensional estimated value of the point cloud to the planning module.

18. The apparatus according to any one of claims 10 to 16, characterized in that, The device further includes: A control unit is used to control the carrier based on the three-dimensional estimation values ​​of the point cloud.

19. An obstacle height estimation device, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 9.

20. The apparatus according to claim 19, characterized in that, The device also includes the memory.

21. An obstacle height estimation system, characterized in that, The control system includes a sensing system and a computing platform, the computing platform including the apparatus as described in any one of claims 10 to 20.

22. A carrier, characterized in that, Includes the apparatus as described in any one of claims 10 to 20, or includes the system as described in claim 21.

23. The carrier according to claim 22, characterized in that, The carrier is a vehicle.

24. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, cause the processor to implement the method as described in any one of claims 1 to 9.

25. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 9.

26. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 9.

Images