Computer-implemented method, electronic vehicle system, and computer program product

Abstract

The invention relates to a computer-implemented method for determining object information in a point cloud (28) generated by means of an active optical sensor system (10), wherein the point cloud (28) comprises point information for reflection points, the point information being dependent on the corresponding distance (di) of the reflection point, the method comprising: • determining an object (O) in the point cloud (28), • determining a ghost object (GO) associated with the object (O) in the point cloud (28), and • determining object information relating to the object (O) using the ghost object (GO). The invention also relates to: a method for at least partially autonomous driving of a vehicle (20); an electronic vehicle system; a vehicle (20); a computer program product; a processing unit (24); and a data carrier.

Description

[0001] 2023PF01552 1

[0002] COMPUTER-IMPLEMENTED PROCESS, ELECTRONIC VEHICLE SYSTEM AND COMPUTER PROGRAM PRODUCT

[0003] Technical field

[0004] The application relates to a computer-implemented method, an electronic vehicle system, a vehicle, and a method for at least partially automated vehicle operation. The application further relates to a computer program product, a computing unit, and a non-volatile, computer-readable data carrier. The electronic vehicle system comprises the computing unit and an active optical sensor system, e.g., a lidar system. The optical sensor system is configured to generate a point cloud.

[0005] background

[0006] Modern vehicles (cars, vans, trucks, motorcycles, etc.) are equipped with a multitude of sensor systems whose data serves to inform the driver and / or is provided to driver assistance systems. These sensor systems detect the vehicle's surroundings, including other road users. Based on the collected data, a model of the vehicle's environment can be created, and the system can react to changes in this environment.

[0007] Sensor systems are constantly being developed for various functions, such as environmental sensing in the near and far range of vehicles, including people, cars, and commercial vehicles. Sensor systems can also be used for driver assistance systems, particularly those for autonomous or semi-autonomous vehicle control. They can be used specifically for detecting obstacles and / or other road users in the front, rear, or blind spot areas of a vehicle. Sensor systems can be based on various sensor principles, such as radar, ultrasound, optics, etc.

[0008] Lidar technology (Lidar stands for Light Detection and Ranging) is an important sensor principle for environmental perception, based on optical electromagnetic 2023PF01552 2

[0009] Lidar technology is wave-based and used in active optical sensor systems. A lidar system comprises an optical transmitter and an optical receiver. The transmitter emits light in the form of an optical signal. This optical signal can be pulsed and / or modulated. In a lidar system, laser beams in the ultraviolet, visible, or infrared range can be used as the light source. The receiver picks up the optical signal after reflection from the surroundings of the lidar system. Using the transmitted optical signal, the lidar system's evaluation unit can analyze the received optical signal, for example, using a time-of-flight (TOF) method. This allows the spatial location and distance of the objects from which the reflection occurred to be determined. Furthermore, the relative velocity can be calculated.In this context, reflection or reflected light is understood to mean any light that is thrown back and is intended in particular to include light thrown back by scattering or absorption-emission.

[0010] Light reflected from the surroundings can be detected in the receiving device by a receiving sensor. Receiving sensors in lidar systems can have multiple receiving elements, called pixels, for opto-electrical conversion. The pixels can be configured to receive optical signals from different angles.

[0011] KR20170140005A describes an object detection device that uses a laser scanner with ghost object detection. The laser scanner unit comprises two rotating optical modules that emit laser light and receive reflected light from an object on the optical plane as they rotate around the axis of rotation, forming an optical plane orthogonal to the axis of rotation. A timing unit measures the time between a first time point at which each of the rotating optical modules emits the laser light and a second time point at which the reflected light is received. A rotation angle calculation unit calculates a rotation angle at the second time point on the optical plane.

[0012] The basis for a predetermined starting position for each of the rotating optical modules. A control unit determines the existence of an object based on triangulation.

[0013] Overview

[0014] A point cloud generated by an active optical sensor system contains point information for each reflection point, which depends on the distance of that reflection point. A computer-implemented method for determining object information in such a point cloud exhibits:

[0015] • Identifying an object in the point cloud,

[0016] • Identifying a ghost object belonging to the object in the point cloud,

[0017] • Determining object information about the object using the ghost object.

[0018] The point cloud is generated by the active optical sensor system, e.g., a lidar system. The points in the point cloud contain point information. This point information depends on the spatial location and the distance of the reflection points in the environment. The reflection points are the points at which the reflection of the optical signal emitted and received by the sensor system occurs. The point cloud can be understood as a set of points, where each point has point information with coordinates in a coordinate system, particularly a three-dimensional one. In the case of a three-dimensional point cloud, the three-dimensional coordinates can be determined, for example, by the direction of incidence, i.e., the solid angle, of a light ray reflected at the respective reflection point and the corresponding travel time or radial distance measured for that point. In addition to the spatial information, namely, for example, theIn addition to three-dimensional coordinates, which depend on the distance to the reflection point, the point cloud's point information can also contain additional information or measurement data for individual points, which may depend, for example, on the intensity of the received light. 2023PF01552 4.

[0019] The computer-implemented method for determining object information uses the point cloud to extract this information. The object information can optionally be output and / or optionally stored in the point cloud. This method for determining object information can, for example, be part of an object detection algorithm and / or be placed before and / or after it.

[0020] The three-dimensional coordinate system of the point cloud can be a three-dimensional polar coordinate system, which is based, for example, on solid angles. However, the spatial information can also be specified for the points in other coordinate systems, such as Cartesian coordinates. A conversion can then take place.

[0021] A ghost object appears in the point cloud as an object, but has no corresponding actual physical object in its environment. The ghost object in the point cloud therefore has no real-world counterpart, but can nevertheless contain information from which inferences can be drawn about real objects, especially the object to which the ghost object is associated. This relationship between the object and its associated ghost object can be used by the proposed method to determine the object information.

[0022] A ghost object can be caused in particular by multiple reflections, where the optical signal is reflected multiple times by the object in the environment before being received by the sensor system. Such multiple reflections can also be referred to as echoes.

[0023] Ghost objects can be generated in particular by highly reflective objects, which have a very high reflectivity and therefore reflect a large part of the optical signal emitted by the active optical sensor system.

[0024] In one embodiment of the method, the ghost object is determined using the object's distance.

[0025] If the optical signal is reflected multiple times from the object, the corresponding ghost object will appear in the point cloud at a greater distance than the actual object. At the same time, the received reflections originate from the real object (2023PF01552 5) and can therefore contain information that can be used to determine object information.

[0026] In particular, the ghost object can be identified using the object's distance by searching the point cloud in a specific direction along a solid angle for objects that are an integer multiple of the object's distance. Such objects with integer multiples, e.g., twice the distance, can be identified as ghost objects. The distance twice the object's distance is particularly relevant because the optical signal strength can be highest at this point.

[0027] In one embodiment of the method, the point cloud exhibits an optical artifact in the area of ​​the object. Ghost objects frequently occur with highly reflective objects. These reflect such a large proportion of the received optical signal that a receiving sensor of the optical sensor system can saturate when it receives the reflected optical signal. An optical receiving sensor in saturation can lead to artifacts, such as crosstalk.

[0028] Highly reflective objects are strongly reflective objects, such as retroreflective objects like traffic signs. In the receiving sensor, such objects can cause problems with the dynamics of the detected optical signal. Pixels of the optical receiving sensor, which receive optical signals with particularly high energy, reach saturation. Saturation means that more light reaches the pixels than they are capable of converting into an electrical received signal. Optical signals reflected by such highly reflective objects can exhibit crosstalk to other pixels of the receiving sensor. Crosstalk is an undesirable optical artifact. The received signal of the other pixels is distorted by the crosstalk, leading to undesirable effects that can impair environmental perception, object detection, and / or the acquisition of object information.Crosstalk can also be referred to by the English terms crosstalk or blooming. The energy of the optical signal is related to its intensity and amplitude. Therefore, possible characteristic values ​​for the detection of crosstalk include, for example, the energy, amplitude, and / or intensity of the optical signal.

[0029] This method makes it possible to supplement or replace object information that cannot be determined, or can only be determined poorly, by the optical artifact with object information determined using the corresponding ghost object. This can improve environmental perception and / or object detection.

[0030] In one embodiment of the method, the ghost object is identified using the optical artifact. This method takes advantage of the fact that, for example, highly reflective objects exhibit an artifact, such as crosstalk, in the point cloud. These objects can also exhibit a ghost object due to their high reflectivity. If an artifact, such as crosstalk, is identified in the point cloud, the next step involves specifically searching for the corresponding ghost object for the object for which the artifact was identified. Starting from the object with the artifact, the search can be conducted for ghost objects at multiple integer distances. If at least one ghost object is identified, it is analyzed to determine object information about the object.

[0031] In one embodiment of the method, the optical artifact can be corrected using the acquired object information. This allows information lost due to the artifact to be added to the point cloud. Alternatively or additionally, the correction can be output as supplementary information and / or stored in the point cloud.

[0032] In one embodiment of the method, the point information depends on the position of the reflection point in a two-dimensional field of view and on the optical intensity of the respective received reflection, wherein the field of view has vertical slits relative to the detected environment. The method features:

[0033] • Column-wise evaluation of the point information,

[0034] • Column-by-column identification of the optical artifact, particularly depending on the result of the evaluation of a respective preceding column. 2023PF01552 7

[0035] Columns in a point cloud can refer, in particular, to an arrangement of the points in the point cloud in the form of a field. The columns of points in the point cloud correspond to the columns of pixels in a receiving sensor, which generated the point information. The pixels of the receiving sensor are also arranged in a field. The columns can extend, in particular, in the direction in which the corresponding pixels of the receiving sensor are contiguous and electrically connected to each other. Optical artifacts, such as crosstalk, are especially noticeable in this direction. The direction perpendicular to the columns is then referred to as the row of points or pixels.

[0036] The pixels of the optical sensor's receiver can be arranged in a field with rows and columns. Individual pixels are configured to receive the optical signal from specific spatial directions, so that the receiver's pixel field covers the environment detected by the sensor system. In some embodiments, the point information can be read out column by column, where the term "column" refers to a vertical direction relative to the detected environment. In particular, the sensor system can scan the environment using an optical signal in the form of a vertical, linear light beam. In this case, the column evaluation corresponds to the form factor of the scan.

[0037] The optical artifact is then evaluated, for example, in relation to the preceding column. Thus, if crosstalk is detected between the pixels of a column, this can optionally be taken into account when evaluating the following column, where the crosstalk may still be present. This additional information can further improve the evaluation.

[0038] If the optical artifact, e.g. crosstalk, is identified using this method, this information can be used to identify at least one ghost object, e.g. at twice the distance, and the object information can be determined from the ghost object.

[0039] The object may exhibit at least one ghost object, i.e., one or more ghost objects. This ghost object may, in particular, have been generated by multiple reflections between the active optical sensor system and the object. The ghost object thus appears in the point cloud at integer multiples of the object's distance. Due to the multiple reflections, the energy of the received optical signal is lower at the ghost object than at the object itself. The artifact, e.g., crosstalk, may therefore be reduced or absent at the ghost object. This can be used to determine object information using the ghost object.

[0040] In one embodiment of the method, the object has multiple reflection points, and the object's distance corresponds to the respective distances of these reflection points. The object has an area encompassing multiple reflection points. During object detection, a group of reflection points can be assigned to the respective object. The object's distance then corresponds to the group of distances of the object's reflection points. This group of reflection points can then be located at the ghost object and used to determine object information.

[0041] In one embodiment of the method, the object information includes the object's shape. The object's shape can play an important role in environment detection and interpretation. Therefore, the object's shape information can further improve environment detection.

[0042] For example, traffic signs are often highly reflective and can cause crosstalk. Furthermore, due to their high reflectivity, traffic signs often exhibit at least one ghost image. At the same time, the shape of the traffic sign is important for its interpretation and meaning. Therefore, the shape can improve the recognition of the meaning of traffic signs.

[0043] In particular, the shape of the object can be determined using the shape of the ghost object. Both reflections, the one to the object and the one to the ghost object, originated from the real object. The shape of the real object is therefore reflected in the ghost object. This is particularly advantageous when the point cloud for the object no longer contains any information about the object's shape due to the artifact. This shape can then be determined from the identified ghost object and its shape.

[0044] Using the described algorithm, it is possible to extract object information about objects from ghost objects that are assigned to those objects. This can be particularly advantageous in further processing, e.g., for autonomous or semi-autonomous driving in vehicles.

[0045] In a method for at least partially autonomous vehicle control, a point cloud is generated using an active optical sensor system in the vehicle. To generate the point cloud, the active optical sensor system emits an optical signal, which is reflected in the vehicle's surroundings and evaluated by the optical sensor system. The point cloud contains the corresponding distances to reflection points in the vehicle's environment. The described computer-implemented method for determining object information is carried out using at least one processing unit in the vehicle. Depending on the result of this determination, the vehicle is driven at least partially automatically.

[0046] An electronic vehicle system comprises a processing unit and an active optical sensor system. The active optical sensor system is configured to generate a point cloud, where the point cloud contains the respective distances of the received reflections for reflection points in the vehicle's environment. The processing unit is configured to

[0047] • to identify at least one object in the point cloud,

[0048] • to identify a ghost object belonging to the object in the point cloud and

[0049] • To determine object information about the object using the ghost object.

[0050] In some embodiments, the active optical sensor system can include the processing unit. In other embodiments, the processing unit can be implemented separately from the active optical sensor system and, for example, be integrated into a central vehicle computer (e.g., 2023PF01552 10), which performs vehicle functions, particularly using data from other sensor systems in the vehicle.

[0051] In one embodiment, the vehicle system is configured to guide the vehicle at least partially autonomously, depending on the object information obtained.

[0052] A vehicle can have the described vehicle system.

[0053] A computer program product contains commands that

[0054] • when executed by a computing unit, cause the computing unit to perform the described computer-implemented procedure for determining object information, or

[0055] • when executed by the described electronic vehicle system, cause the electronic vehicle system to carry out the described procedure for at least partially autonomous driving of a vehicle.

[0056] The computing unit has the means to carry out the described computer-implemented procedure for determining object information.

[0057] The described computer program product can be stored on a non-volatile, computer-readable data carrier.

[0058] Tour list

[0059] The following section provides further explanation and description of exemplary implementations of this application with reference to the figures. They show

[0060] Fig. 1 schematically shows a vehicle with an active optical sensor system and a computing unit,

[0061] Fig. 2 schematically shows a method for determining object information,

[0062] Fig. 3 shows an example of a reflection received by the sensor system.

[0063] Fig. 4 shows an example of an environment detected by the sensor system.

[0064] Fig. 5 shows the captured environment as a grayscale image, 2023PF01552 11

[0065] Fig. 6 shows a highly reflective object detected in the environment with an associated ghost object.

[0066] Fig. 7 Point information of the point cloud for the object from Figure 6,

[0067] Fig. 8 Point cloud information for the ghost object belonging to the object,

[0068] Fig. 9 shows the extraction of object information using the ghost object.

[0069] Fig. 10 schematically shows a field of view in grid form with rows and columns,

[0070] Fig. 11 schematically shows a deleted object with a ghost object.

[0071] The same reference symbols are used in the figures for identical or similar elements. Representations in the figures may not be to scale.

[0072] Fiourenbebeschreibunguno

[0073] Figure 1 schematically depicts a vehicle 20, for example a passenger car. The vehicle 20 has an active optical sensor system 10, e.g. a lidar system. The optical sensor system 10 is arranged in a front area of ​​the vehicle 20 and the environment 22 it detects is located in front of the vehicle 20 in the direction of travel.

[0074] The active optical sensor system 10 comprises an optical transmitter 12, an optical receiver 14, an optical deflector 16, and an evaluation unit 18. The evaluation unit 18 can include a processor, an FPGA, or similar device for processing data.

[0075] The optical transmitter 12 emits light in the form of an optical signal L. It has a light source for emitting, for example, laser light. The optical receiver 14 receives the optical signal L reflected at reflection points in the environment 22.

[0076] Optionally, the optical transmitter 12 can transmit the optical signal L in pulses. The pulsed optical signal L has short periods during which the optical signal L is transmitted. This can be referred to as a pulse. Between the pulses, no light is transmitted by the optical transmitter 12. The reflections of the pulses of the optical signal L in the environment 22 are then received by the optical receiver 14.

[0077] Preferably, the optical receiving device 14 comprises a receiving sensor that serves as an optoelectronic detector. The receiving sensor can, for example, be a point sensor, line sensor, or area sensor, in particular an avalanche photodiode, a photodiode cell, a CCD sensor, an active pixel sensor, for example a CMOS sensor, or the like. The receiving sensor can receive the optical signal L and convert it into electrical signals. The electrical signals can be processed by the evaluation device 18.

[0078] The optical deflection device 16 is configured to deflect the optical signal L transmitted by the optical transmitter 12 into the environment 22 and / or to deflect the optical signal L reflected at reflection points in the environment 22 to the optical receiver 14. The deflection device 16 can be controlled such that the optical signal L performs a scanning motion 26 across the environment 22. For example, the deflection device 16 can include a rotating mirror device that performs a rotational movement to deflect the optical signal L such that the scanning motion 26 is carried out by the optical signal L. During the rotational movement, the angular position of the deflection device 16 is changed.

[0079] The evaluation unit 18 is configured to control the transmission of the optical signal L, particularly as a function of the angular position of the deflection device 16. The evaluation unit 18 is further configured to evaluate the transmitted and received optical signal L. Using the evaluation data generated in this way, a point cloud 28 can be created. The point cloud 28 can be transmitted to a processing unit 24 of the vehicle 20. The processing unit 24 can optionally be part of the optical sensor system 10.

[0080] The point cloud 28 can be generated, for example, in the optical sensor system 10 from the evaluation data, or the point cloud 28 can be generated, for example, in the computing unit 24 from raw data of the optical sensor system 10.

[0081] The evaluation data generated by the evaluation unit 18 can be used to create the point cloud 28, which represents the reflection points of the optical 2023PF01552 13.

[0082] Signals L are present in the environment 22. Point information is provided for each point in the point cloud 28, which depends on the evaluation data, such as distance information. The distance information depends on the distance of the sensor system 10 to the reflection point. The point cloud 28 can be used, for example, to detect objects 0, to determine the distance to objects 0, and / or to perform further evaluations.

[0083] In the embodiment shown in Figure 1, the point cloud 28 is generated in the evaluation unit 18 of the optical sensor system 10. In the processing unit 24, a method for determining object information is carried out, in which the point cloud 28 is post-processed. The processing unit 24 can then optionally output the post-processed point cloud 28 as a processed point cloud 28'. The processed point cloud 28' can then be further processed, for example, by other control units of the vehicle 20, e.g., for the execution of driving functions.

[0084] The computing unit 24 can, for example, be configured as the central vehicle computer of the vehicle 20, in which data from several sensor systems of the vehicle 20 can be received, evaluated, and / or further processed. The computing unit 24 can, for example, be used to implement autonomous or semi-autonomous driving functions. The computing unit 24 can further process the processed point cloud 28' and use it, for example, to execute the driving functions.

[0085] The optical sensor system 10 can, for example, be mounted or integrated at the front of the vehicle 20. Optical sensor systems 10 are also possible for other parts of the vehicle 20, e.g., for surround-view functions, such as on the sides and / or rear of the vehicle 20. It is also possible to arrange further optical sensor systems 10, such as lidar sensors, and / or other sensor systems such as radar, ultrasound, etc., on the vehicle 20, particularly in corner areas of the vehicle 20.

[0086] The optical sensor system 10 can be used to detect stationary or moving objects O in the environment 22. Such objects O can be items such as vehicles, people, animals, plants, obstacles, 2023PF01552 14

[0087] This includes road surface irregularities, in particular potholes or stones, road boundaries, traffic signs, open areas, in particular parking lots, precipitation or the like.

[0088] Figure 2 schematically illustrates a procedure for determining object information, as performed, for example, by the computing unit 24.

[0089] The procedure uses the point cloud 28 generated by the active optical sensor system 10, e.g., a lidar system. In 210, object 0 is identified in the point cloud 28. For this purpose, an object detection algorithm can be used, for example. In 220, a ghost object GO belonging to object O is identified in the point cloud 28.

[0090] To identify the ghost object, the distance di of the object, which is contained in the point information, can be used, for example. The ghost object GO is then identified by a targeted search for the ghost object at integer multiples of the distance di, e.g., twice the distance 2di.

[0091] Alternatively or additionally, the ghost object can be identified by detecting an optical artifact, such as crosstalk, in the point cloud 28. Based on the detected crosstalk, the ghost object GO can then be identified by a targeted search within the range of integer multiples of the crosstalk distance di, e.g., twice the crosstalk distance 2di. Here, the property of highly reflective objects O, such as traffic signs, can be used, as these often exhibit ghost objects GO due to their high reflectivity.

[0092] In 230, object information for object O is determined from point cloud 28 using the ghost object GO. This takes advantage of the fact that the ghost object GO originated from reflections off the same object O, but the reflections received for the ghost object GO have a lower intensity, for example due to multiple reflections, and therefore exhibit no or fewer artifacts. The data in point cloud 28 can thus contain usable information, such as the shape of object O. This information about the ghost object GO is then assigned to object O, which belongs to the ghost object GO. 2023PF01552 15

[0093] In step 240, the object information determined in step 230 is further processed. This further processing can include, for example, editing, which involves saving the object information in the point information for object 0 in point cloud 28. The editing can also include, for example, correcting the artifact in the point information for object 0 in point cloud 28. During correction, the object information is used to correct any artifact that may be present in the point cloud for object 0. The edited point cloud 28' is generated through editing, for example, by adding and / or correcting. The edited point cloud 28' can contain the object information and / or the correction of the artifact in the point information for object 0 in point cloud 28.

[0094] In 240, the processed point cloud 28' can also be output, e.g., via an output interface of the computing unit 24.

[0095] Figure 3 shows an example of the reflection of the optical signal L received by the sensor system for a specific solid angle. The vertical axis represents the area under the received light pulse. This area is a measure of the received intensity or strength of the optical signal L. The horizontal axis represents the distance at which the reflection of the optical signal L occurs.

[0096] An internal IR reflection can be observed at very close range. This corresponds to the reflection of the optical signal L within the optical sensor system 10.

[0097] The object O at distance di is also visible. The shape of the displayed area signal shows that the receiving sensor went into saturation when receiving the optical signal L reflected by object O. Accordingly, the point cloud may exhibit crosstalk artifacts in the area of ​​the object.

[0098] The ghost object GO can also be seen at twice the distance 2di of object O. Ghost object GO was created in point cloud 28 by a double reflection of the optical signal L at object O. The light of the optical signal L therefore traveled twice the distance 2di of the distance di to object O, since the light traversed the path twice. The signal waveform for ghost object GO shows that the receiving sensor did not reach saturation when receiving the 2023PF01552 16 optical signal L, so there is presumably no crosstalk. Due to the longer path, the light was attenuated more. This means that the signal for ghost object GO may contain information about object O, which can be evaluated to determine object information.

[0099] Figure 4 shows an exemplary environment 22 detected by the sensor system 10 in the form of the point cloud 28.

[0100] The point cloud 28 contains the point information for the individual points of the point cloud 28. The point information includes, in particular, information about the spatial location in a field of view 50 of the optical sensor system 10 and the distance of the reflection points in the environment 22. The reflection points are those points in the environment 22 at which the reflection of the optical signal L emitted and received by the sensor system 10 occurs.

[0101] Figure 4 visualizes an example of the environment 22 of vehicle 20 as a point cloud 28. Object O, e.g., a highly reflective traffic sign, is detected and marked. Simultaneously, the corresponding ghost object GO is detected and marked at twice the distance 2di. Due to crosstalk, the shape of object O is not recognizable in point cloud 28. However, the corresponding ghost object GO has the shape of object O, and this shape—a rectangle—is also recognizable. The shape of ghost object GO can therefore be determined as object information for object O. The determined object information for object O can, for example, be stored at the points in point cloud 28 belonging to object O, and / or the crosstalk can be corrected in point cloud 28. Processing point cloud 28 then results in the processed point cloud 28'.

[0102] Figure 5 shows an example of an environment 22 detected by the sensor system 20 as a grayscale image. A "grayscale image" is a two-dimensional image in which brightness values ​​for the pixels are represented as shades of gray between black (no brightness) and white (full brightness). In the scene shown in Figure 5, for example, an object O can be seen, which is a "Caution" traffic sign. The "Caution" traffic sign is triangular in shape and displays an exclamation mark. 2023PF01552 17

[0103] Figure 6 depicts the scene from Figure 5 as the environment 22 captured by the optical sensor system 10. The point cloud 28 is visualized in Figure 6. The traffic sign, captured as object 0, is visible. The traffic sign is highly reflective, which is why the point cloud shows the artifact of crosstalk, or blooming, in the area of ​​the captured highly reflective object 0. At the same time, the point cloud 28 also contains the ghost object GO associated with the object.

[0104] The crosstalk of object 0, i.e., the traffic sign, is clearly visible. The object dimensions are no longer correctly identifiable in point cloud 28. However, the shape of the highly reflective object 0 is correctly recognizable in the ghost object GO at twice the distance. This property of the ghost object GO can be used in the execution of the procedure.

[0105] Figure 7 shows point information from point cloud 28 for object 0 from Figure 6.

[0106] Figure 7a) presents information about object 0 in textual form. The distance di to object 0 can be determined from this.

[0107] In Figure 7 b), the points from point cloud 28 are shown at a distance di from object 0. The crosstalk of object 0 is evident, meaning that the information regarding the extent and dimensions of object 0 is distorted by this crosstalk artifact.

[0108] Figure 7c) shows a section of Figure 7b). Here, the crosstalk artifact in the area of ​​object 0 is illustrated.

[0109] Figure 8 shows point information from point cloud 28 for the ghost object GO belonging to object 0 from Figure 7.

[0110] Figure 8a) presents information about the ghost object GO in textual form. The distance 2di to the ghost object GO can be determined, which is twice the distance di to object 0.

[0111] Figure 8b) shows the points from point cloud 28 at a distance 2di from the ghost object GO. Information on the extent and dimensions of the ghost object GO, which is actually object 0, can be obtained from point cloud 28, since the artifact crosstalk is significantly lower in the area of ​​the ghost object GO. 2023PF01552 18

[0112] Figure 8c) shows a section of Figure 8b). Here it can be seen that the information on the extent and / or dimensions of object GO is not, or less, distorted than the information on object 0 in Figure 7c). Since the ghost object GO was created by object 0, the information, especially regarding its extent and / or dimensions, can be used as object information for object 0.

[0113] Figure 9 illustrates the extraction of object information using the ghost object GO. Figure 9a) shows the ghost object GO from Figure 8. The shape of object 0, the shape of the "Caution" traffic sign with a rectangular supplementary sign below it, is recognizable. Figure 9b) shows the recognition of the ghost object GO and its shape, which corresponds to the shape of object 0.

[0114] Figure 10 schematically depicts the field of view 50 with rows ZE and columns SP. In the point cloud, the field of view 50 corresponds to the two-dimensional plane that covers the environment 22. Each point of the field of view 50 can have multiple reflection points in the point cloud, each at a different distance.

[0115] The optical receiving device 14 comprises the optical receiving sensor and a readout device. The optical receiving sensor has several pixels, each configured to convert an optical signal into an electrical received signal. The readout device activates the respective pixels by reading the respective electrical received signal. From the electrical received signal, intensity values ​​and / or related values ​​of the received optical signal L can be determined, for example, by the evaluation device 18.

[0116] The pixels of the optical receiver sensor contain light-sensitive components that capture light and convert it into an electrical signal, which can then be read out by the readout device and made available for further processing. These light-sensitive components can be photodiodes, charge-coupled devices (CCDs), CMOS sensors, photomultiplier tubes, SPADs, or similar devices. 2023PF01552 19

[0117] In embodiments, at least one superpixel, comprising a plurality of pixels, can be used to detect the optical signal. In a superpixel, several pixels can be used together to detect the optical signal, and the electrical output signal can be evaluated together as an electrical receive signal. A pixel or superpixel can be assigned to a specific area of ​​the field of view 50 and be designed to detect that area of ​​the field of view 50.

[0118] The pixels or superpixels of the receiving sensor can form an array of pixels or superpixels. In particular, the division of the field of view 50 can correspond to the arrangement of the pixels or superpixels of the receiving sensor. The pixels of the receiving sensor can thus be assigned to areas of the field of view 50. The field of view 50 can be divided accordingly – for example, as shown in Figure 10 – into rows ZE and columns SP. The points of the field of view 50 of the point cloud 28 can be evaluated in patterns, e.g., column by column.

[0119] In a scanning sensor system 10, in which the environment 22 is scanned, for example, by a vertical linear optical signal L in a horizontal direction, the corresponding field of view 50 in the point cloud 28 can optionally be evaluated analogously, i.e., column by column.

[0120] If crosstalk is detected in column SP, it is likely that crosstalk will also occur in the subsequent column(s). This can be taken into account during evaluation, and, for example, a highly reflective object O can be identified based on the column-wise analysis.

[0121] After detecting the highly reflective object O, the corresponding ghost object GO can be identified using the distance di of the highly reflective object O in integer multiples of di, e.g., twice the distance 2di. Its shape can then be used—as described—to determine the shape of object O. In general, the ghost object GO can be used to obtain object information about object O.

[0122] Figure 11 schematically depicts a deleted object O with a ghost object GO. 2023PF01552 20

[0123] In the upper part of Figure 11, the received optical signal L is shown with the respective area under the received pulse. It can be seen that the reflection from object 0 leads to saturation of the receiving sensor and crosstalk. The ghost object GO is detectable at twice the distance due to the high strength of the received signal. It can also be seen that the ghost object GO does not saturate the receiving sensor.

[0124] The lower part of Figure 11 shows point cloud 28, which contains the corresponding object 0 and ghost object GO. The information for object 0 in point cloud 28 is faulty because it has been corrupted by the artifact crosstalk.

[0125] However, it is possible to recognize the shape and, to some extent, the content of the ghost object GO. This information, captured from the ghost object GO, can be used to obtain object information for object 0.

[0126] 2023PF01552

[0127] Reference mark

[0128] 10 optical sensor system

[0129] 12 optical transmitting device

[0130] 14 optical receiving device

[0131] 16 optical deflection devices

[0132] 18 Evaluation unit

[0133] 20 vehicles

[0134] 22 surroundings

[0135] 24 computing units

[0136] 26 scanning movement

[0137] 28 point cloud

[0138] 28' processed point cloud

[0139] 50 field of view

[0140] 210-240 process steps

[0141] IR internal reflection

[0142] L optical signal

[0143] O object

[0144] GO Ghost object di Distance to the object

[0145] 2di Distance to the ghost object

[0146] SP column

[0147] ZE line

Claims

2023PF01552 22 REQUIREMENTS 1. Computer-implemented method for determining object information in a point cloud (28) generated by means of an active optical sensor system (10), wherein the point cloud (28) has point information for each reflection point which depends on a respective distance (di) of the respective reflection point, wherein the method comprises: Identifying an object (0) in the point cloud (28), Identifying a ghost object (GO) belonging to the object (O) in the point cloud (28), Retrieving object information about object (0) using the ghost object (GO).

2. The method of claim 1, wherein the ghost object (GO) is determined using the distance (di) of the object (0).

3. Method according to claim 1 or 2, wherein the point cloud (28) has an optical artifact in the region of the object (0).

4. The method of claim 3, wherein the ghost object (GO) is determined using the optical artifact.

5. Method according to claim 3 or 4, wherein the optical artifact is corrected using the determined object information.

6. Method according to any one of claims 3 to 5, wherein the point information depends on the position of the respective reflection point in a two-dimensional field of view (50) and on the optical intensity of the respective received reflection, wherein the field of view (50) has vertical slits (SP) in relation to the detected environment (22), wherein the method comprises: Column-wise evaluation of the point information, Column-by-column identification of the optical artifact, particularly depending on the result of the evaluation of a respective preceding column (SP). 2023PF01552 23 7. Method according to any of the preceding claims, wherein the object (0) has at least one ghost object (GO), wherein the at least one ghost object (GO) was generated by multiple reflections between the active optical sensor system (10) and the object (0).

8. Method according to one of the preceding claims, wherein the object (0) has multiple reflection points and the distance (di) of the object (0) corresponds to the respective distances of the reflection points of the object (0).

9. Method according to one of the preceding claims, wherein the object (0) has a high reflectivity and the point cloud (28) in the area of ​​the object (0) has an optical artifact produced by crosstalk.

10. Method according to any of the preceding claims, wherein the ghost object (GO) has twice the distance (2di) of the object (0).

11. Method according to any of the preceding claims, wherein the object information comprises a form of the object (0).

12. Method according to claim 11, wherein the shape of the object (0) is determined using the shape of the ghost object (GO).

13. Method for at least partially autonomous driving of a vehicle (20), wherein a point cloud (28) is generated by means of an active optical sensor system (10) of the vehicle (20), wherein the point cloud (28) has respective distances for reflection points in an environment (22) of the vehicle (20), wherein a computer-implemented method for determining object information according to one of the preceding claims is carried out by means of at least one computing unit (24) of the vehicle (20), and the vehicle (20) is driven at least partially autonomously depending on a result of the determination. 2023PF01552 24 14. Electronic vehicle system comprising a computing unit (24) and an active optical sensor system (10) configured to generate a point cloud (28), wherein the point cloud (28) has respective distances of the received reflections for reflection points in an environment (22) of the vehicle (20), wherein the computing unit (24) is configured to identify at least one object (0) in the point cloud (28), to identify a ghost object (GO) belonging to the object (O) in the point cloud (28) and To determine object information for object (0) using the ghost object (GO).

15. Vehicle system according to claim 14, wherein the active optical sensor system (10) comprises the computing unit (24).

16. Vehicle system according to claim 14 or 15, wherein the vehicle system is configured to guide the vehicle (20) at least partially autonomously depending on the object information obtained.

17. Vehicle comprising a vehicle system according to one of claims 14 to 16.

18. Computer program product comprising instructions which, when executed by a computing unit (24), cause the computing unit (24) to perform a computer-implemented method according to any one of claims 1 to 12, or, when executed by an electronic vehicle system according to any one of claims 14 to 16, cause the electronic vehicle system to perform a method according to claim 13.

19. Computing unit (24) comprising means for carrying out the method according to any one of claims 1 to 12.

20. Computer-readable non-volatile data carrier on which the computer program product according to claim 18 is stored.

Images