Computer-implemented method and electronic vehicle system with optical sensor system
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- VALEO SCHALTER & SENSOREN GMBH
- Filing Date
- 2026-01-15
- Publication Date
- 2026-07-30
Smart Images
Figure EP2026050961_30072026_PF_FP_ABST
Abstract
Description
[0001] 2023PF02641 1
[0002] COMPUTER-IMPLEMENTED PROCESS AND ELECTRONIC VEHICLE SYSTEM WITH OPTICAL SENSOR SYSTEM
[0003] Technical field
[0004] The application relates to a computer-implemented method for evaluating a point cloud of a detected environment generated by an active optical sensor system, e.g., a lidar system. The application further relates to a method for at least partially autonomous driving of a vehicle depending on the evaluation of the point cloud, as well as an electronic vehicle system comprising a computing unit and the active optical sensor system.
[0005] background
[0006] Modern vehicles (cars, vans, trucks, motorcycles, etc.) are equipped with a multitude of sensor systems whose data serves to inform the driver and / or provide data to driver assistance systems. These sensor systems detect the vehicle's surroundings, including other road users. Based on the collected data, a model of the vehicle's environment can be created, and the system can react to changes in this environment.
[0007] Sensor systems are constantly being developed for various functions, such as environmental sensing in the near and far range of vehicles like passenger cars or commercial vehicles. Sensor systems can also be used for driver assistance systems, particularly those for autonomous or semi-autonomous vehicle control. They can be used specifically for detecting obstacles and / or other road users in the front, rear, or blind spot areas of a vehicle. Sensor systems can be based on various sensor principles, such as radar, ultrasound, optics, etc.
[0008] Lidar technology (Lidar stands for Light Detection and Ranging) is an important sensor principle for environmental perception, based on optical electromagnetic 2023PF02641 2
[0009] This technology is wave-based and used in active optical sensor systems. An active optical sensor system, such as a lidar system, has an optical transmitter and an optical receiver. The transmitter emits light in the form of an optical signal. This optical signal can be pulsed and / or modulated. In a lidar system, laser beams in the ultraviolet, visible, or infrared range can be used as the light source. The receiver picks up the optical signal after reflection from the surroundings of the lidar system. Using the transmitted optical signal, the lidar system's evaluation unit can analyze the received optical signal, for example, using a time-of-flight (TOF) method, and determine the spatial location and distance of the objects from which the reflection occurred.Furthermore, it is possible to determine a relative velocity. In this context, reflection or reflected light is understood to mean any light that is thrown back and is specifically intended to include light thrown back by scattering or absorption-emission.
[0010] Light reflected from the surroundings can be detected in the receiving device by a receiving sensor. Receiving sensors in lidar systems can have multiple receiving elements, called pixels, for opto-electrical conversion. The pixels can be configured to receive optical signals from different angles.
[0011] CN115685237A describes a three-dimensional multimode object detection method and system that combines a lidar system's field of view with a geometric constraint. Three-dimensional objects are detected using a lower boundary region of the field of view.
[0012] Overview
[0013] A point cloud of a detected environment, generated by an active optical sensor system, assigns respective point indices to the respective reflection points. 2023PF02641 3
[0014] The information includes angular information for a given azimuth angle and elevation angle of the respective reflection point.
[0015] A computer-implemented method for determining a plane exhibits:
[0016] • Determining a partial point cloud, wherein the partial point cloud contains point information from reflection points in a region around a predefinable elevation angle,
[0017] • Determining the plane in the environment using the partial point cloud.
[0018] The plane in the surrounding area is determined by analyzing the sub-point cloud. This sub-point cloud is encompassed by the larger point cloud but contains fewer reflection points. This method can be used to identify one or more planes within the point cloud.
[0019] The point cloud is generated by the active optical sensor system, e.g., a lidar system. The points in the point cloud contain point information. This point information depends on the spatial position of the reflection point, the distance of the respective reflection point, and optionally on other properties of the received reflection. The reflection points are the points at which the reflection of the optical signal emitted by the sensor system and received as an echo occurs. The point cloud can be understood as a set of points, where each point has point information with coordinates in a coordinate system, particularly a three-dimensional one.In the case of a three-dimensional point cloud, the three-dimensional coordinates can be determined by the angular information of a light ray reflected at the respective reflection point and the corresponding travel time or radial distance measured for that point. In addition to spatial information, the point cloud's point information can also contain additional information for the individual points, which may depend, for example, on further measurement data of the received echo signal. 2023PF02641 4.
[0020] The angular information includes azimuth and elevation angles. Azimuth and elevation are specified relative to the environment captured by the optical sensor system's field of view. In an Earth-referenced environment, the azimuth corresponds to a lateral angle and the elevation angle to an angle relative to the horizon. For example, when detected by an optical sensor system mounted on a vehicle, the azimuth and elevation can be specified relative to the vehicle's surroundings as captured by the optical sensor system. The field of view of the optical sensor system can be defined by the area covered by the emitted optical signal.
[0021] To transmit the optical signal, the sensor system has an optical transmitter that emits the signal using a light source. The optical signal can be emitted at different elevation and azimuth angles, so that the point cloud has reflection points, each with different azimuth and / or elevation angles.
[0022] To receive the optical signal, the sensor system includes an optical receiver. The optical receiver comprises an optical receiver sensor and a readout device. The receiver sensor can have multiple receiver pixels, each configured to convert an optical signal into an electrical receiver signal. Individual pixels can be configured to receive the echo signal from specific spatial directions, so that the pixel array of the receiver sensor covers the environment detected by the sensor system. These spatial directions can be specified by angular information. The electrical receiver signal can, for example, represent the shape of the echo signal.
[0023] The sub-point cloud comprises reflection points located within the range of the predefined elevation angle. For example, the sub-point cloud can contain one to three layers with varying elevation angles. The advantage is that the evaluation of the sub-point cloud is sufficient to determine the planes, which can save computation time and effort.
[0024] The detection / extraction / segmentation of planes is an important step in the evaluation of point clouds. Determining planes can be part of a 2023PF02641 5
[0025] Segmentation is a step in the evaluation of point clouds. For the segmentation of planes, for example, model-based and / or region-based methods can be used to evaluate point clouds.
[0026] Detected planes provide valuable information for interpreting a scene within the captured environment. Planes can include ground planes, which run in the direction of the ground. Planes can also include transverse planes, which run perpendicular to the ground and / or perpendicular to the optical signal.
[0027] The described use of the partial point cloud within the range of the predefined elevation angle enables the use of layer information for identifying the plane within the surrounding scene. The layer corresponds to the scan by the optical signal at a specific elevation angle. This information about the elevation angle of the respective evaluated layer can be used to further determine the plane. This allows for a more efficient evaluation process. The evaluation is applicable to both dense and sparse point clouds.
[0028] In one embodiment of the method, the point cloud was generated using an optical signal from a scanning active optical sensor system, in which the optical signal undergoes a rotational movement around an axis. The optical signal is emitted in layers around the axis, with each layer having a specific elevation angle. The elevation angle is specified relative to a zero layer, which has an angle of zero and is perpendicular to the axis. The layers located below the zero layer have, for example, negative elevation angles. The layers located above the zero layer have, for example, positive elevation angles. The azimuth angles can, for example, specify the solid angle relative to a plane passing through the axis.If the scanning sensor system uses pulsed optical signals, the timing of the transmission can be coordinated with the angular positions in azimuth and elevation, thus creating targeted positions for reflection points.
[0029] To emit pulsed optical signals, the optical sensor system, e.g., a lidar system, can have an optical transmitter that emits the optical signal in pulses. The pulsed optical signal has short 2023PF02641 6
[0030] Periods of time are recorded during which the optical signal is transmitted. This can be referred to as a pulse. Between pulses, no light is transmitted by the optical transmitter. The reflections of the optical signal pulses at reflection points in the environment are then received by an optical receiver of the sensor system as an echo signal.
[0031] In one embodiment of the method, the point cloud has reflection points at a multitude of elevation angles. Reflection points exhibiting the same elevation angle are referred to as lying in or belonging to a layer. This allows the spatial-geometric information contained in the respective layer to be used to determine the plane, which can improve the efficiency of the algorithm.
[0032] In one embodiment of the method, the scanning motion of the optical signal generating the reflection point forms at least a partial cone, and the plane is determined by identifying intersection points between this partial cone and the plane. This approach models the lidar scan layers as a cone, or at least a partial cone, and uses the elevation and azimuth information to determine the plane. This can improve the efficiency of the algorithm. For elevation angles of 0 degrees, the partial cone can take the form of a surface, since the signal emitted by the optical sensor system is perpendicular to the axis.
[0033] In one embodiment of the method, the respective elevation angle associated with each reflection point, which in turn is associated with each intersection point, is used to determine the plane. Thus, the elevation angle associated with each intersection point is used. In particular, spatial geometric information of the layer associated with the elevation angle can be used.
[0034] In one embodiment of the method, lines are determined from the intersection points to identify the plane. The planes can then be determined using these lines. The lines can be chosen such that they connect the intersection points. 2023PF02641 7
[0035] In one embodiment of the method, the predefinable elevation angle is perpendicular to the axis. For example, the elevation angle of 0 degrees can be assigned to the elevation angle perpendicular to the axis. The assigned layer can be called the zero layer. The zero layer can, in particular, be the layer that is closest to the 0-degree angle. In this embodiment, the partial point cloud is located in a region around the zero layer. By evaluating such a partial point cloud, it is possible, for example, to determine planes in the surrounding area that run along the axis. The partial point cloud can, in particular, contain the zero layer, which has an elevation angle of 0 degrees, or the layer that has the elevation angle closest to zero among the elevation angles available in the point cloud.In another embodiment, the partial point cloud can contain the zero layer, the plus layer above it and / or the minus layer below it.
[0036] In one embodiment of the method, the predefinable elevation angle is negative. In this embodiment, the predefinable negative elevation angle can extend to a boundary region of the point cloud with a large absolute value. The point cloud layer thus extends at a large absolute value relative to the zero plane. In this embodiment, the sub-point cloud is located in a region at the edge of the point cloud.
[0037] By analyzing such a partial point cloud, it is possible, for example, to identify planes in the surrounding area that run along a ground plane. For instance, the identified plane in the surrounding area could correspond to the ground itself.
[0038] In one embodiment, the method further includes: using the determined plane for object detection in the point cloud. The determined plane or planes can, for example, be extracted as a feature from the point cloud and used for object detection. Alternatively or additionally, the determined planes can be used for segmenting the point cloud. In particular, the planes can be used as a property of an object to be detected. 2023PF02641 8
[0039] In a method for at least partially autonomous vehicle operation, a point cloud is generated using an active optical sensor system in the vehicle. The point cloud contains specific point information for each reflection point. This point information includes angular information relative to an i
[0040]
[0041] Azimuth angle and elevation angle of i
[0042]
[0043] The reflection point is determined. The described computer-implemented procedure is carried out using at least one processing unit of the vehicle. Depending on the result of the plane determination, the vehicle is guided at least partially autonomously.
[0044] An electronic vehicle system comprises a processing unit and an active optical sensor system. The active optical sensor system is configured to generate a point cloud of the detected environment. This point cloud contains point information for each reflection point. This point information includes angular information for the respective azimuth and elevation angles of each reflection point. The processing unit is configured to determine a sub-point cloud, where this sub-point cloud contains point information for reflection points within a range around a predefined elevation angle. The processing unit is further configured to determine a plane within the environment using this sub-point cloud.
[0045] In one embodiment of the vehicle system, the active optical sensor system for generating the point cloud comprises a scanning active optical sensor system in which the optical signal undergoes a rotational movement about an axis. The point cloud exhibits reflection points at a multitude of elevation angles. The scanning movement of the respective optical signal generating the reflection point forms at least a partial cone. The processing unit is configured to determine the intersection points between the at least partial cone and the plane in order to ascertain the plane.
[0046] In one embodiment of the vehicle system, the active optical sensor system includes the processing unit. Alternatively, the processing unit can be designed separately from the active optical sensor system and, for example, be a communication device.
[0047] The device has a cation connection to the active optical sensor system. The processing unit can, for example, be located in a central vehicle computer.
[0048] In one embodiment, the vehicle system is configured to guide the vehicle at least partially autonomously, depending on the result of the determination of the plane.
[0049] The vehicle may have the described vehicle system.
[0050] One embodiment of a computer program product includes instructions which, when executed by a computing unit, cause the computing unit to perform the described computer-implemented method for determining the plane.
[0051] One embodiment of the computer program product includes instructions which, when executed by the described electronic vehicle system, cause the electronic vehicle system to perform the described method for at least partially autonomous vehicle control. The processing unit has means to perform the described method for determining a level.
[0052] The described computer program product can be stored on a computer-readable non-volatile data carrier.
[0053] Tour list
[0054] The following section provides further explanation and description of exemplary implementations of this application with reference to the figures. They show
[0055] Fig. 1 schematically shows a vehicle with an optical sensor system and a computing unit,
[0056] Fig. 2 schematically shows a first embodiment of the optical sensor system with several elevation angles,
[0057] Fig. 3 schematically shows a second embodiment of the optical sensor system with multiple elevation angles,
[0058] Fig. 4 schematically shows layers and elevation angles of the first embodiment of the optical sensor system, 2023PF02641 10
[0059] Fig. 5 schematically shows the layers and elevation angles of the second embodiment of the optical sensor system.
[0060] Fig. 6 schematically shows sections of layers with a plane,
[0061] Figs. 7-11 schematically show an embodiment of sections of layers with planes in a surrounding environment,
[0062] Fig. 12 schematically shows the optical sensor system with cone,
[0063] Figs. 13 and 14 schematically show an exemplary embodiment of sections of layers with the soil.
[0064] The same reference symbols are used in the figures for identical or similar elements. Representations in the figures may not be to scale.
[0065] Tour description
[0066] Figure 1 schematically depicts a vehicle 20, for example a passenger car. The vehicle 20 has an active optical sensor system 10, e.g. a lidar system, and a processing unit 24. The optical sensor system 10 is arranged in a front area of the vehicle 20, and the environment 22 it detects is located in front of the vehicle 20 in the direction of travel.
[0067] The active optical sensor system 10 comprises an optical transmitter 12, an optical receiver 14, an optical deflector 16, and an evaluation unit 18. The evaluation unit 18 can include a processor, an FPGA, or similar device for processing data.
[0068] The optical transmitter 12 emits light in the form of an optical signal L. It has a light source for emitting, for example, laser light. The optical receiver 14 receives the optical signal L reflected at reflection points in the environment 22 as an echo signal.
[0069] Optionally, the optical transmitter 12 can transmit the optical signal L in pulses. The pulsed optical signal L has short periods in which the optical signal L is transmitted. This can be referred to as a pulse. Between the pulses, no light is emitted by the optical transmitter 12.
[0070] transmitted. The reflections of the pulses of the optical signal L in the environment 22 are then received by the optical receiving device 14 as an echo signal.
[0071] Preferably, the optical receiving device 14 comprises a receiving sensor that serves as an optoelectronic detector. The receiving sensor can, for example, be a point sensor, line sensor, or area sensor, in particular one or more avalanche photodiodes, photodiode cells, CCD sensors, active pixel sensors, for example CMOS sensors, or the like. The receiving sensor can receive the optical signal L as an echo signal and convert it into electrical receiving signals. The electrical receiving signals can be processed by the evaluation device 18.
[0072] The optical deflection device 16 is configured to deflect the optical signal L transmitted by the optical transmitter 12 into the environment 22 and / or to deflect the optical signal L reflected at reflection points in the environment 22 as an echo signal to the optical receiver 14. The deflection device 16 can be controlled such that the optical signal L performs a scan 26 over the environment 22. For example, the deflection device 16 can include a rotating mirror device that rotates and has an axis A to deflect the optical signal L such that the scan 26 is performed by the optical signal L. During the rotation, the angular position of the deflection device 16 is changed. Scanning the environment 22 can also be referred to as probing. The scan 26 can also be referred to as a probing motion.
[0073] The evaluation unit 18 is configured to control the transmission of the optical signal L, particularly as a function of the angular position of the deflection device 16. The evaluation unit 18 is further configured to evaluate the transmitted optical signal L and the received echo signal. Using the evaluation data generated in this way, a point cloud 28 can be created. The point cloud 28 can be transmitted to a processing unit 24 of the vehicle 20. The processing unit 24 can optionally be part of the optical sensor system 10. In the processing unit 242023PF02641 12
[0074] Point cloud 28 can be further evaluated and / or edited.
[0075] Point cloud 28 can be generated, for example, in the optical sensor system 10 from the evaluation data, or point cloud 28 can be generated, for example, in the processing unit 24 from raw data of the optical sensor system 10. In particular, point cloud 28 can be generated using the evaluation data generated by the evaluation unit 18.
[0076] Point cloud 28 contains the reflection points of the optical signal L in the environment 22. Point information is provided for each point in point cloud 28, which depends on the evaluation data. This point information includes, in particular, three-dimensional spatial coordinates of the reflection points in the environment. The spatial coordinates can include angular information and distances. The angular information can be specified with azimuth and elevation. Point cloud 28 can be used, for example, to detect objects 0, to determine the distance to objects 0, and / or to perform further evaluations.
[0077] In the embodiment shown in Figure 1, the point cloud 28 is generated in the evaluation unit 18 of the optical sensor system 10. In the processing unit 24, a method for determining a plane is performed, in which the point cloud 28 is further evaluated and optionally post-processed. The processing unit 24 can then optionally output the post-processed point cloud 28 as a processed point cloud. The processed point cloud can then be further processed, for example, by other control units of the vehicle 20, e.g., for the execution of driving functions.
[0078] The computing unit 24 can, for example, be configured as the central vehicle computer of the vehicle 20, in which data from several sensor systems of the vehicle 20 can be received, evaluated, and / or further processed. The computing unit 24 can, for example, be used to implement autonomous or semi-autonomous driving functions. The computing unit 24 can further process the processed point cloud and, for example, use it to execute the driving functions.
[0079] The optical sensor system 10 can, for example, be mounted on or integrated into the front of the vehicle 20. Optical sensor systems 2023PF02641 13 are also available.
[0080] 10 for other parts of the vehicle 20 is possible, e.g., for surround-view functions, such as on the sides and / or rear of the vehicle 20. It is also possible to arrange further optical sensor systems 10, such as lidar sensors, and / or other sensor systems such as radar, ultrasound, etc., on the vehicle 20, especially also in corner areas of the vehicle 20.
[0081] The optical sensor system 10 can be used to detect stationary or moving objects O in the environment 22. Such objects O can include items such as vehicles, people, animals, plants, obstacles, road surface irregularities, in particular potholes or stones, road boundaries, traffic signs, open areas, in particular parking lots, precipitation, or the like.
[0082] Figure 2 schematically shows a first embodiment of the optical sensor system 10 with four layers.
[0083] The illustrated embodiment of the optical sensor system 10 scans the environment 22 with a scan movement 26 about an axis A, wherein the scan movement 26 is transverse to the image plane and the axis A lies in the image plane. A multitude of azimuth angles are traversed, which move through the environment 22 in a horizontal direction and perform the scan movement 26 in a horizontal direction.
[0084] The traversal of the multitude of azimuth angles is repeated for four different elevation angles. The scan movement 26, which is traversed at a constant elevation angle, is referred to as a layer. An elevation angle of 0 degrees is shown as a dashed line in Figure 2. It can be seen that the first embodiment of the optical sensor system 10 shown in Figure 2 has four layers, two of which are below the 0-degree line and two of which are above the 0-degree line.
[0085] The layer closest to the 0-degree line can be called the zero layer (N). The layer below it, i.e., closer to the ground, can be called the minus layer (NM), and the layer above it, i.e., farther from the ground, can be called the plus layer (NP). The minus layer (NM) has a negative elevation angle. The plus layer (NP) has a positive elevation angle. 2023PF02641 14
[0086] Figure 3 schematically shows a second embodiment of the optical sensor system 10 with 32 layers.
[0087] The illustrated embodiment of the optical sensor system 10 scans the environment 22 with a scan movement 26 about an axis A, wherein the scan movement 26 is transverse to the image plane and the axis A lies in the image plane. A multitude of azimuth angles are traversed, which move through the environment 22 in a horizontal direction and perform the scan movement 26 in a horizontal direction.
[0088] The traversal of the multitude of azimuth angles is repeated for 32 different elevation angles. The scan movement 26, which is traversed with a constant elevation angle, is referred to as a layer. An elevation angle of 0 degrees is shown as a dashed line in Figure 3. The second embodiment of the optical sensor system 10 shown in Figure 3 has 32 layers, a first number of which lie below the 0-degree line and a second number of which lie above the 0-degree line. In the illustrated example, layer N runs with a 0-degree elevation angle. This is the zero layer N in the illustrated example.
[0089] The layer directly below the zero layer N, i.e., the layer immediately adjacent to the zero layer N and closer to the ground, can be designated as the minus layer NM, and the layer directly above the zero layer N, i.e., the layer further away from the ground, can be designated as the plus layer NP. The minus layer NM has a negative elevation angle. The plus layer NP has a positive elevation angle. In the example shown, the layers above the zero layer N have positive elevation angles. In the example shown, the layers below the zero layer N have negative elevation angles.
[0090] Figure 4 schematically shows the four layers and the associated elevation angles of the first embodiment of the optical sensor system 10 of Figure 2. The layers are plotted along the horizontal axis and the associated elevation angles are plotted along the vertical axis.
[0091] The zero layer N corresponds to layer number 2, as it has an elevation angle closest to 0 degrees. The minus layer NM corresponds to layer number 1, as it lies directly below the zero layer.
[0092] Layer N is located. The plus layer NP corresponds to layer number 3, as it lies directly above the zero layer N.
[0093] Figure 5 schematically shows the 32 layers and the associated elevation angles of the second embodiment of the optical sensor system 10 from Figure 3. The layers are plotted along the horizontal axis and the associated elevation angles are plotted along the vertical axis.
[0094] The zero layer N corresponds to layer number 21, as it has an elevation angle closest to 0 degrees. In the example shown, the elevation angle of layer number 21 is 0 degrees. The minus layer NM corresponds to layer number 20, as it lies directly below the zero layer N. The plus layer NP corresponds to layer number 22, as it lies directly above the zero layer N.
[0095] Fig. 6 schematically shows sections 30. N of the zero layer N with the plane 30. Also shown are sections 30. P of the plus layer NP with the plane 30 and sections 30. M of the minus layer NM with the plane 30.
[0096] In the illustrated embodiment, plane 30 is configured as a plane vertical to the ground or nearly vertical within a small angular range and parallel to or nearly parallel within a small angular range to axis A. The plane can, for example, be a flat object or a planar area of an object located in the environment 22 and detected by the optical sensor system 10 during the generation of the point cloud.
[0097] Such information, from which linear segments 30.N, 30.P, 30.M can be derived as the intersection line or intersection curve of cones and planes 30, may be contained in the point cloud 28. This information can be used in particular to determine planes 30 that run longitudinally, approximately parallel to, or slightly inclined to the axis A.
[0098] The zero layer N, if it has an elevation angle of 0 degrees, scans horizontally around axis A and therefore, in this case, does not form a cone but a layer perpendicular to axis A. The plus layer NP and the minus layer NM are actually very shallow cones with small elevation angles. If a plane 30 is parallel or longitudinal at an angle (not perpendicular) to axis A, it intersects the zero layer N in a line 30. N (if the zero-2023PF02641 16
[0099] layer 0 degrees elevation angle) and two flat cones 30. M, 30. P, which are theoretically hyperbolic, but are mostly approximately straight lines due to the flatness of the cones and the approximation of the cone axis to axis A.
[0100] The sections with plane 30 shown in Figure 6 can result, for example, from a lidar system installed laterally on the vehicle 20. The scan through the zero layer N (with an elevation angle of 0°) forms a horizontal line to axis A, which intersects plane 30 to form a straight line segment 30.N; the scan through the plus layer NP and minus layer NM (e.g., with an elevation angle of 0° = + / -1°) 0 ) forms flat cones 34. They intersect the plane 30 which runs parallel or mostly parallel to the axis A, as a conical curve 30.P, 30.M. Within the operating range of the sensor system 10, these intersection curves 30.P, 30.M are mostly linear and can be approximated linearly to determine the plane 30.
[0101] One embodiment of the method for determining level 30 may include the following steps:
[0102] 1. Receiving a point cloud 28 from an active optical sensor system 10, selecting points from the zero layer N and the plus layer NP and minus layer NM
[0103] 2. Finding line segments 30. N from the zero layer N by Hough transformation.
[0104] 2.1 Creating an image by quantizing the points 30. N in its coordinate range.
[0105] 2.2 Use of the Hough transform for the detection of line segments 30. N for the zero layer N.
[0106] 3. Extend these line segments to the plus layer NP and minus layer NM and group the line segments.
[0107] 4. Determining levels 30 with the points 30. N, 30. P, 30. M on the line segments of the zero layer N, plus layer NP and minus layer NM2023PF02641 17
[0108] 5. Merging similar levels 30 using parameters defined between the levels 30.
[0109] 6. Output of the parameters of the found levels.
[0110] In step 1, planar objects can be identified as planes 30 in the environment 22. Depending on the tilt angle of the sensor system 10, ground planes can also be recognized as planar objects and thus as planes 30. This can occur, for example, during autonomous driving, depending on the road situation, especially in the far front area of the vehicle 20.
[0111] In step 2, the Hough transform is applied to a very small number of 3D points, specifically to a single layer, namely the zero layer. This allows for a cost reduction compared to dense point clouds. This also increases the speed of the process.
[0112] The extension and integration of step 3 can be performed for at least one layer, namely the plus layer NP and / or the minus layer NM (e.g., from zero degrees to a small positive angle, e.g., +1 degree, or zero degrees to a negative angle, e.g., -1 degree, because two lines already define a plane). Additional layers, e.g., + / -2 degrees, + / -3 degrees, could increase the accuracy of determining the planes 30 if there are still intersecting line segments for the additional layers. In step 4, planes 30 can be found not only from the points 30.N, 30.P, 30.M of line segments, but also from the combined line segments.
[0113] Step 5 is useful for small levels 30 that are close to a large level 30, or for a large level 30 that has been divided into many small levels 30 by nearby objects.
[0114] Figure 7 schematically shows an embodiment of sections 30. N of the zero layer N with different planes 30 in the environment 22.
[0115] The point cloud 28 is shown in a perspective view, with sections 30 and N of the zero layer N marked with planes 30 in the surroundings 22. The line segments are reflection points of the optical signal L with, for example, walls in the surroundings 22.2023PF02641 18
[0116] Figure 8 schematically shows the embodiment of Figure 7 of the zero layer N, the plus layer NP and the minus layer NM with different levels 30 in the environment 22.
[0117] The point cloud 28 is shown in a perspective view, with the sections 30. N of the zero layer N with planes 30, sections of the plus layer NP with planes 30, and sections of the minus layer NM marked. The line segments are reflection points of the optical signal L with, for example, walls in the vicinity 22.
[0118] Figure 9 schematically shows an example of a result from step 2, applied to the point cloud 28 from Figure 7. Line segments found from the zero plane N by Hough transformation are shown. The line segments correspond to the intersections 30. N with the planes 30 from Figure 7.
[0119] Figure 10 schematically illustrates the extent from the zero layer N to the plus layer NP and the minus layer NM. Additionally, Figure 10 shows the levels 30 determined from this extent. These levels 30 are the levels 30 marked in Figure 8, i.e., planar objects 0 in the environment 22. These levels 30 can be readily and efficiently determined from the three layers N, NP, and NM.
[0120] Figure 11 shows the top view of these discovered levels 30.
[0121] Fig. 12 schematically shows the optical sensor system 10 with the optical deflection device 16, which has a rotating mirror. The optical signal L scans the scene of the environment 22 and is reflected. The evaluation of the transmitted and reflected signal L can be used to determine the point cloud 28. If the optical signal L strikes the ground 32, the reflection points can form a curvature, which corresponds to an intersection 36 of at least a partial cone of the optical signal L with the ground 32. The intersection 36 corresponds to an ellipse, such as that formed by the intersection of a plane 30, e.g., the ground 32, with the cone 34 of the optical signal.
[0122] The change in angle around axis A can be incremental and corresponds to the respective azimuth angle of the reflection points. A change in angle around 2023PF02641 19
[0123] An additional axis perpendicular to axis A causes a change in the respective elevation angle of the reflection points.
[0124] The described method utilizes the fact that for a point cloud 28, which includes further information such as elevation and azimuth, this information can be used to determine planes 30, e.g., base surfaces such as the ground 32. This makes the determination simpler and / or faster.
[0125] If a ground plane 32 is considered as a geometric plane and a circularly rotating optical signal L, forming a cone 34, sweeps across this geometric plane, the cone 34 intersects the ground 32 in a curve 36, which is an ellipse.
[0126] For layers with large angles to the zero layer N and a sufficiently large plane, complete ellipses appear; for layers with smaller angles to the zero layer N, partial ellipses result, since the ground 32 is only partially intersected. However, if ellipses or partial ellipses are formed, these can be determined and used in determining the plane 30, e.g., the ground plane 32.
[0127] Figure 13 schematically shows an embodiment of sections 36 of layers with the ground 32. Shown are the sections 36 of those layers that have a large negative elevation angle relative to the zero layer N. In the example shown in Figures 3 and 5, these could be layers 1 and 2, whose reflection points lie on the ground plane 32.
[0128] The determination of the ground plane 32 can then be focused on the layers that most frequently intersect the ground 32, i.e., layers with a large negative elevation angle relative to the zero layer N. The cones 34 formed by these two layers intersect the ground as ellipses; the planes 30 on which their points lie lie on the ground 32, and these planes 30 can be considered the ground plane 32. To find a plane from these points of the outer layers of the point cloud 28, the angular information of the layers is used to obtain the points of intersection with the ground plane 32. The Ransac algorithm for detecting planes 30 is then applied to determine the plane equation.
[0129] Figure 14 schematically shows an embodiment for determining the ground plane 32 using the ground-level layers of the point cloud 28. Shown are the planes 30, 32 found for the point cloud 28 from Figure 13 from layers 1 and 2 of the second embodiment shown in Figure 3, viewed from two directions.
[0130] An embodiment of the method for detecting floor surfaces 32 comprises the following steps:
[0131] 1. Extraction of data from at least one layer of the point cloud 28. For this purpose, at least one layer with a large angle of incidence on the ground 32 is preferably selected, i.e., with a large negative elevation angle relative to the zero layer N. This step can include preprocessing such as area limitation and / or filtering for distance discontinuities. A possible distance discontinuity is shown as an example in the left half of Figure 12, where the intersection 36 with the ground 32 is interrupted by a smaller object positioned closer together.
[0132] 2. Determining an initial level:
[0133] a) Applying a Ransac algorithm to find a plane equation of the initial plane with output of the plane equation and inlier points
[0134] b) Performing a level adjustment using the least squares mean method with the Inlier points
[0135] 3. Transform the Inlier points (3D points) for the adapted plane into a plane with the normal [0, 0, 1], so that the 3D points are mapped to 2D points.
[0136] 4. Fitting an ellipse to these 2D points
[0137] 5. Search for ellipse points that correspond to the transformed points from the inlier points of the layer under consideration.
[0138] 5.1. Creating guide points with angles as parameters
[0139] 5.2. Aligning the 2D transformed reflection points of the layer with these guide points 2023PF02641 21
[0140] 5.3. Determining the angle parameters for the aligned 2D reflection points
[0141] 5.4. Determining the corresponding ellipse points for the angle parameters
[0142] 6. Determining the distances between 3D inlier points of the layer and the corresponding 3D ellipse points after the inverse transformation from 2D.
[0143] 7. Filtering the 3D reflection points with the calculated distances
[0144] 8. Determining an improved plane by adjusting to the filtered 3D reflection points
[0145] 9. Output of plane parameters: Using the determined plane parameters, points that lie outside a certain distance from this plane are removed. The output then shows the points that belong to the determined plane 30, e.g., the ground plane 32.
[0146] The Random Sample Consensus (Ransac) soil segmentation algorithm determines the plane parameters iteratively. The described procedure introduces the ellipse to check whether the intersection points 36 lie on the ellipse or not. The procedure for determining the soil plane 32 can therefore be more efficient and achieve more accurate results, especially when using an iterative method such as the Ransac algorithm. 2023PF02641
[0147] Reference mark
[0148] 10 Optical sensor system 12 Optical transmitter 14 Optical receiver 16 Optical deflection device 18 Evaluation device
[0149] 20 vehicles
[0150] 22 surroundings
[0151] 24 computing units
[0152] 26 Scan movement
[0153] 28 point cloud
[0154] Level 30
[0155] 30. N Cut Zero Layer / Level 30. P Cut Plus Layer / Level 30. M Cut Minus Layer / Level 32 Floor
[0156] 34 cones
[0157] 36 Cut bottom / cone
[0158] Axis
[0159] N zero layer
[0160] NP Plus layer
[0161] NM Minus layer
[0162] L optical signal
[0163] O object
Claims
2023PF02641 23 REQUIREMENTS 1. Computer-implemented method for determining a plane (30) in a point cloud (28) of a detected environment (22) generated by means of an active optical sensor system (10), wherein the point cloud (28) has point information for each reflection point, wherein the point information includes angular information for each azimuth angle and elevation angle of the respective reflection point, wherein the method comprises: Determining a partial point cloud, wherein the partial point cloud contains point information from reflection points in a region around a predefinable elevation angle, Determining the plane (30) in the environment (22) using the partial point cloud.
2. Method according to claim 1, wherein the point cloud (28) was generated by means of an optical signal (L) of a scanning active optical sensor system (10) in which the optical signal (L) performs a rotational movement about an axis (A).
3. Method according to claim 2, wherein the point cloud (28) has reflection points at a plurality of elevation angles, wherein the scan movement (26) of the respective optical signal (L) generating the reflection point forms an at least partial cone (34), wherein, in order to determine the plane (30), points of intersection (30. N, 30. P, 30. M, 36) between the at least partial cone (34) and the plane (30) are determined.
4. Method according to claim 3, wherein the respective elevation angle is used to determine the plane (30) which is assigned to the respective reflection point which is assigned to the respective intersection point (30. N, 30. P, 30. M, 36).
5. Method according to claim 3 or 4, wherein lines are determined from the intersection points (30. N, 30. P, 30. M, 36) to determine the plane (30). 2023PF02641 24 6. Method according to any one of claims 2 to 5, wherein the predefinable elevation angle is perpendicular to the axis (A).
7. Method according to one of claims 2 to 5, wherein the predefinable elevation angle is negative and is in particular assigned to a boundary region of the point cloud (28) with a large negative elevation angle in absolute value.
8. Method according to claim 7, wherein the determined plane (30) in the environment (22) corresponds to the ground (32).
9. Method according to any one of the preceding claims, further comprising: Using the determined plane (30) for object detection in the point cloud (28).
10. Method for at least partially autonomous driving of a vehicle (20), wherein a point cloud (28) is generated by means of an active optical sensor system (10) of the vehicle (20), wherein the point cloud (28) has respective point information for respective reflection points, wherein the point information has angular information for a respective azimuth angle and elevation angle of the respective reflection point, wherein a computer-implemented method for determining a plane (30) according to one of the preceding claims is carried out by means of at least one computing unit (24) of the vehicle (20), and the vehicle (20) is guided at least partially autonomously depending on a result of the determination.
11. Electronic vehicle system comprising a computing unit (24) and an active optical sensor system (10), wherein the active optical sensor system (10) is configured to generate a point cloud (28) of a detected environment (22), wherein the point cloud (28) contains point information for each reflection point, wherein the point information includes angular information for each azimuth angle and elevation angle of the respective reflection point, wherein the computing unit (24) is configured to to determine a partial point cloud, wherein the partial point cloud contains point information from reflection points in a region around a predefinable elevation angle, to determine a plane (30) in the environment (22) using the partial point cloud.
12. Vehicle system according to claim 11, wherein the active optical sensor system (10) for generating the point cloud (28) comprises a scanning active optical sensor system (10) in which the optical signal (L) performs a rotational movement about an axis (A), wherein the point cloud (28) has reflection points at a plurality of elevation angles, wherein the scanning movement (26) of the respective optical signal (L) generating the reflection point forms an at least partial cone (34), wherein the computing unit (24) is configured to determine the intersection points between the at least partial cone (34) and the plane (30) for determining the plane (30).
13. Vehicle system according to claim 11 or 12, wherein the active optical sensor system (10) comprises the computing unit (24).
14. Vehicle system according to one of claims 11 to 13, wherein the vehicle system is configured to guide the vehicle (20) at least partially autonomously depending on a result of the determination of the plane (30).
15. Vehicle (20) comprising a vehicle system according to any one of claims 11 to 14.
16. Computer program product with commands that when executed by a computing unit (24), cause the computing unit (24) to carry out a computer-implemented method according to one of claims 1 to 9, or When implemented by an electronic vehicle system according to one of claims 11 to 14, cause the electronic vehicle system to perform a method according to claim 10. 2023PF02641 26 17. Computing unit (24) comprising means for carrying out the method according to any one of claims 1 to 9.
18. Computer-readable non-volatile data carrier on which the computer program product according to claim 16 is stored.