Reconstructing a scan of an object by an active optical sensor system from an object representation and at least partially automatically guiding a motor vehicle

Abstract

In order to reconstruct a scan of an object (5) by an active optical sensor system (3) from an object representation of the object (5) that includes a first curve (14, 14a, 14b, 14c, 14d) and a second curve (15) located on an approximated surface of the object (5) in three-dimensional space, a plurality of scan points (16) on the approximated surface of the object (5) are determined in a manner depending on the first curve (14, 14a, 14b, 14c, 14d) and the second curve (15). A response function (17) describing a respective response to a virtual laser pulse is generated for each of the scan points (16). A superposition (18) of the generated response functions (17) is determined, and a reconstructed measure relating to a light energy incident on a detector array (10) of the active optical sensor system (3) is determined on a detector array (10) of the active optical sensor system (3) in a manner depending on the superposition (18), wherein the detector array (10) has a plurality of detector pixels.

Description

[0001] 2023PF01453

[0002] 1

[0003] Reconstruction of a scan of an object with an active optical sensor system from an object representation and at least partially automated driving of a motor vehicle.

[0004] The present invention relates to a computer-implemented method for reconstructing or partially reconstructing a scan of an object, in particular a motor vehicle, from an object representation of the object using an active optical sensor system, and to a method for at least partially automating the driving of a motor vehicle, wherein such a computer-implemented method is carried out. The invention further relates to a data processing system for carrying out such a computer-implemented method, an electronic vehicle guidance system comprising such a data processing system, and corresponding computer program products.

[0005] Active optical sensor systems, particularly lidar systems such as laser scanners or flash lidar systems, can be mounted on motor vehicles and used to implement a wide variety of driver assistance functions and / or other functions for partially automated driving. In particular, distances to objects in the vehicle's vicinity can be determined, for example, by measuring the time of flight of light using active optical sensor systems. Furthermore, the light energy arriving at the detector array of the active optical sensor system can be quantified by one or more parameters, such as the maximum amplitude of a signal pulse, the width of the signal pulse, the area under the signal pulse, or similar measures.The distances and / or the key figures or data derived therefrom can be used for driver assistance or for other functions for at least partially automatic driving of the motor vehicle.

[0006] The amount of data generated per frame by modern active optical sensor systems, whose detector array may have hundreds of rows and hundreds of columns, is very high. This places correspondingly high demands on the computing and storage resources required for storing, transmitting, and / or processing this data. To counteract this, abstracted object representations of objects scanned by the active optical sensor system can be generated. Such object representations may be sufficient for various applications, but in other applications it is desirable or

[0007] 2

[0008] It is necessary to have more information from the original scan by the active optical sensor system available in order to increase the reliability and / or accuracy of the functions for at least partially automatic driving of the motor vehicle.

[0009] Document DE 10 2019 120777 A1 describes a computer-implemented simulation method for simulating the scanning of an object with an active optical sensor system, where a description of the object's surface is specified. Based on this description, a processing unit generates a set of discrete points, and for each point, a response function is generated. A superposition of these generated response functions is then determined. The processing unit calculates a virtual radiant power on an active surface of the sensor system, depending on this superposition.

[0010] It is an object of the present invention to provide a possibility for the reconstruction or partial reconstruction of a scan of an object with an active optical sensor system from an object representation of the object.

[0011] This problem is solved by the subject matter of the independent claim.

[0012] Advantageous further developments and preferred embodiments as well as advantageous applications are the subject of the dependent claims.

[0013] The invention is based on the idea of ​​determining sampling points on the approximate surface, starting from an object representation that includes at least two curves in three-dimensional space, each of which lies on an approximate surface of the object, and determining the overall response to virtual laser pulses for these sampling points in order to reconstruct a measure relating to light energy arriving at a detector array of the active optical sensor system.

[0014] According to one aspect of the invention, a computer-implemented method for reconstructing or partially reconstructing a scan of an object with an active optical sensor system, in particular a motor vehicle, from an object representation of the object is provided. The object representation has a first curve in three-dimensional space, which lies on an approximate surface of the object, and a second curve in three-dimensional space, which lies on the approximate surface of the object. Depending on the first curve and the second curve, a 2023PF01453

[0015] 3

[0016] A multitude of sampling points are determined on the approximate surface of the object. For each of these sampling points, a response function is generated, describing the respective response to a virtual laser pulse. A superposition of the generated response functions is determined. A reconstructed measure of the light energy arriving at a detector array of the active optical sensor system is determined as a function of this superposition, where the detector array comprises, in particular, a multitude of detector pixels.

[0017] Unless otherwise specified, all steps of the computer-implemented method can be performed by a data processing system comprising at least one data processing device, in particular by a data processing system of the vehicle. Specifically, the at least one data processing device is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the at least one data processing device may, for example, store a computer program containing instructions which, when executed by the at least one data processing device, cause it to execute the computer-implemented method. The terms "data processing system" and "at least one data processing device" may be used interchangeably.

[0018] All data processing devices of the at least one data processing device can be part of the motor vehicle. However, it is also possible that all data processing devices of the at least one data processing device are part of an external computing system outside the motor vehicle, for example, a mobile electronic device, a backend server, or a cloud computing system. It is also possible that the at least one data processing device comprises both at least one vehicle data processing device of the motor vehicle and at least one external data processing device of the external computing system.The at least one vehicle data processing device may, for example, comprise one or more electronic control units (ECUs), and / or one or more zone control units (ZCUs), and / or one or more domain control units (DCUs) of the motor vehicle and / or the active optical sensor system.

[0019] In the event that the at least one data processing device includes two or more data processing devices, certain steps that are performed using the at least one data processing device can be, for example, so2023PF01453

[0020] 4

[0021] It should be understood that different data processing devices perform different steps or different parts of a step. In particular, it is not necessary for each data processing device to perform the steps completely. In other words, the execution of the steps can be distributed across two or more data processing devices.

[0022] The object representation was created, in particular, based on data, especially one or more images, generated by the active optical sensor system depending on actual measurements. Performing the measurements or acquiring the data, as well as creating the object representation, are not necessarily parts of the computer-implemented method according to the invention. In particular, the object representation is provided to the data processing system for carrying out the computer-implemented method according to the invention, for example, stored on a storage device, especially of the data processing system.

[0023] In some embodiments, however, creating the object representation is part of the computer-implemented method according to the invention. Even in such embodiments, which can also be purely computer-implemented, performing the measurements or acquiring the data is not necessarily part of the computer-implemented method according to the invention. However, from each embodiment of the computer-implemented method, a corresponding embodiment of a method that is not purely computer-implemented can be derived by including corresponding steps for performing the measurements or acquiring the data using the active optical sensor system.

[0024] An active optical sensor system, by definition, has a light source for emitting light or light pulses. The light source can be, in particular, a light-emitting diode (LED) or a laser, for example, an infrared LED or infrared laser. Furthermore, an active optical sensor system, in this case the detector array, by definition has at least one optical detector to detect the emitted light or parts thereof. The active optical sensor system is specifically configured to generate and process or output one or more detector signals based on the detected components of the light.

[0025] 5

[0026] Here and in the following, the term "light" can be understood to encompass electromagnetic waves in the visible, infrared, and / or ultraviolet ranges. Accordingly, the term "optical" can also be understood to refer to light as defined in this way.

[0027] A well-known type of lidar system is the so-called laser scanner, in which a laser beam is deflected by a light deflection device, allowing for various deflection angles of the laser beam. The light deflection device can, for example, contain a rotatably mounted mirror. Alternatively, the light deflection device can have a mirror element with a tiltable and / or swiveling surface. The mirror element can, for example, be designed as a microelectromechanical system (MEMS). In the environment, the emitted laser beams can be partially reflected, and the reflected portions can then strike the laser scanner, specifically the light deflection device, which can direct them onto a detector array of the laser scanner.

[0028] Each optical detector in the detector array generates a corresponding detector signal based on the components it detects. The spatial arrangement of each detector, together with the current position of the light deflection device, particularly its rotational, tilting, and / or swiveling position, allows the system to determine the direction of incidence of the detected reflected components. An evaluation unit can also perform, for example, a time-of-flight measurement to determine the radial distance of the reflecting object. Alternatively or additionally, a method can be used to determine the distance by evaluating the phase difference between emitted and detected light.

[0029] Other types of lidar systems are flash lidar systems. These are non-scanning systems that do not require such a light deflection arrangement. Instead, the laser light generated by the light source is scattered by an optical element, so that it is emitted in a single flash over a wide angle.

[0030] It should be noted that a single detector pixel of the detector array does not necessarily consist of a single optical detector. Rather, it is also possible for a group of several adjacent optical detectors to form a detector pixel. The latter is particularly relevant when using single-photon detectors.

[0031] 6

[0032] Avalanche diodes, SPADs (single photon avalanche diodes), are possible. In other embodiments, however, it is also possible for a pixel to consist of exactly one optical detector, for example a single photodiode or avalanche photodiode, APD.

[0033] In some embodiments, the active optical sensor system is a lidar sensor system.

[0034] The active optical sensor system can, for example, be mounted on the vehicle or on an infrastructure device to detect the vehicle's surroundings.

[0035] That the first and second curves lie on the approximated surface follows by definition from the construction of the object representation. How exactly this approximated surface corresponds to the actual surface of the object is not crucial for the computer-implemented method according to the invention. Rather, by carrying out the computer-implemented method according to the invention, it is assumed that the first and second curves approximate the corresponding areas or contours of the actual surface. In particular, the first curve approximates a first edge or first contour on the actual surface of the object, and the second curve approximates a second edge or second contour on the actual surface of the object.

[0036] The first curve and the second curve are not identical; in particular, they are not parallel straight lines and / or do not lie in a common plane. For example, the first curve and the second curve are not closed curves.

[0037] The object representation can consist of the first curve and the second curve, or the object representation can include one or more additional curves that lie on the approximated surface.

[0038] The first and second curves are curves in three-dimensional space; therefore, they do not necessarily lie in a particular plane, and in particular, they do not necessarily lie in a plane parallel to an image plane defined by the detector array. This is made possible by the fact that the active optical sensor system can, by design, generate a 2.5-dimensional or three-dimensional representation of its environment, especially the object, and the object representation is particularly 2023PF01453

[0039] 7

[0040] can be generated based on the 2.5-dimensional or three-dimensional representation.

[0041] Typically, data generated by an active optical sensor system can be presented as a point cloud, which is understood as a set of points, each identified by corresponding coordinates in a two- or three-dimensional coordinate system. In the case of a three-dimensional point cloud, the three-dimensional coordinates can be determined, for example, by the direction of incidence of the reflected light and the corresponding light travel time or the radial distance measured for that particular point. However, the information can also be preprocessed to obtain three-dimensional Cartesian coordinates for each point. Generally, the points in a point cloud can be presented in an unordered or unsorted manner, unlike, for example, a camera image.

[0042] In addition to spatial information, namely the two- or three-dimensional coordinates, the point cloud can also contain additional information or measured values ​​for the individual points, such as a parameter relating to the energy of incident light, for example, an echo pulse width (EPW) of a pulse of the respective detector signal, a maximum pulse amplitude, an area or partial area under the pulse, and so on. The fact that a parameter relates to the energy of the incident light does not necessarily mean that the parameter is a value of energy itself; rather, it could also be a value derived from the energy, such as light intensity or luminous flux, and so forth.

[0043] The data generated by the active optical sensor system during a frame can be interpreted as a 2.5-dimensional point cloud, provided the radial spacing is appropriately determined. This means that while a three-dimensional position is stored for each point in the point cloud, the entire three-dimensional space is not represented within the active optical sensor system's field of view. This is because the emitted light pulses only reach the side of objects in the environment facing the active optical sensor system. Therefore, parts of the objects behind this side, or other objects or parts thereof obscured by the objects, are not represented by the point cloud, similar to a camera image. 2023PF01453

[0044] 8

[0045] The view of data generated by an active optical sensor system as a point cloud stems, among other things, from the fact that the detector arrays of earlier active optical sensor systems had only a very small number of detector pixels, for example, two to four detector pixels arranged in a row. In modern active optical sensor systems, however, this is sometimes different, as the detector pixels are arranged in a large number of rows and columns, for example, in 100 to 1,000 rows and / or 100 to 1,000 columns. As a result, a point cloud of a frame can also be viewed or displayed as an image, where the number of image pixels corresponds to the number of detector pixels, and the spatial two-dimensional arrangement of the detector pixels defines the corresponding image plane and directly corresponds to the arrangement of the image pixels.The individual pixel values ​​of the image can be given by the radial distance of the reflecting point, measured by the corresponding detector pixel, or by the corresponding depth, which is the perpendicular distance of the reflecting point from the detector pixel and can be calculated from the radial distance. The image can then be referred to as a depth image, in particular a monochromatic depth image.

[0046] However, it is also possible, for example, to generate corresponding images, especially monochromatic images, based on other quantities instead of depth, for example based on the EPW or other parameters of the corresponding reflected components captured by the detector pixel.

[0047] The detector pixels are arranged in a two-dimensional plane, for example as a multitude of rows and a multitude of columns, which corresponds to the image plane of the depth image.

[0048] In some embodiments, filtering of the data generated by the detector array can also be carried out to create the depth image, for example noise filtering and / or filtering to remove certain objects or other components of the data that are not considered relevant.

[0049] If the image plane of the depth image is defined by a transverse direction y and a vertical direction z perpendicular to it, then the depth is defined along a longitudinal direction x perpendicular to y and z. The rows of the detector array and the depth image are then arranged side by side parallel to the vertical direction z, and the columns of the detector array and the depth image are arranged side by side parallel to the transverse direction y.

[0050] 9

[0051] The three-dimensional space is defined by x, y, and z. As already mentioned, the first edge and the first curve lie in three-dimensional space, and therefore not necessarily in a plane parallel to the plane of the detector array.

[0052] The reconstructed measure of light energy incident on a detector array of the active optical sensor system, hereinafter also referred to simply as the reconstructed measure, is in particular a quantity that quantifies the incident light energy or a quantity derivable from or dependent on it, such as luminous power, luminous intensity, or the like. The reconstructed measure can also be understood as a virtual measure, for example, as virtual light energy, virtual light power, virtual light intensity, and so on.

[0053] The virtual laser pulse can be understood, for example, as a laser pulse such as could be emitted by the active optical sensor system. In particular, it is not an actually existing laser pulse. The virtual laser pulse can be defined, for example, by a wavelength, a wavelength spectrum, one or more linewidths, a pulse shape, a maximum intensity, a temporal intensity profile, an emission direction, and / or other parameters. The wavelength of the virtual laser pulse could, for example, be a peak wavelength of the corresponding wavelength spectrum.

[0054] The virtual laser pulse is defined, for example, by storing the relevant parameters defining the laser pulse on the data carrier and / or by a user transferring them to the processing unit via a user interface. The response functions can then be generated based on the virtual laser pulse, for example, on the parameters defining it.

[0055] The multitude of sampling points comprises three or more sampling points. In various embodiments, the multitude of sampling points comprises a number of sampling points on the order of one hundred, several hundred, one thousand, or several thousand sampling points. In particular, the multitude of sampling points can be understood as an object point cloud.

[0056] The response function for a sampling point corresponds, in particular, to an approximate or assumed response of the object at that point to an incoming laser beam, especially a virtual laser beam. The response function describes 2023PF01453

[0057] 10

[0058] in particular an electromagnetic wave, for example light, that would be emitted from the point on the object if a laser beam corresponding to the virtual laser beam were to hit that point.

[0059] The response functions are, in particular, location-dependent functions. Generating the response function for a sampling point involves, in particular, calculating the response function on the detector array, or on a sub-area, or at least on a single point of the detector array.

[0060] This means, in particular, that generating the response function takes into account or includes the propagation of the corresponding electromagnetic wave from the respective point on the object surface through space and, if applicable, through components of the active optical sensor system up to the active surface of the active optical sensor system.

[0061] The response functions, or rather the superposition of the response functions, can be understood in particular as virtual electromagnetic radiation. This virtual electromagnetic radiation specifically describes a virtual reflection of the virtual laser pulse from the approaching object surface.

[0062] Depending on the orientation of the virtual laser pulse, i.e., its propagation direction and, consequently, on which part of the object's surface the virtual laser pulse would strike, the various response functions can differ. Different response functions can, for example, exhibit different amplitudes and / or phases.

[0063] The amplitude of the respective response function can depend, in particular, on the distance between the active optical sensor system and the corresponding point on the approaching object surface, as well as on the reflectivity of the approaching object surface at that point. This information or these parameters are, for example, contained in the object representation or provided in addition to it.

[0064] Generating the response functions includes, in particular, calculating the response functions, or at least calculating the response functions on the 2023PF01453

[0065] 11

[0066] detector array, the sub-area or at least one point of the detector array, and the storage of the corresponding calculation result.

[0067] The reconstructed measure can, for example, be a position-dependent function on the detector array. For example, the reconstructed measure can be a discrete function on the detector array.

[0068] The reconstructed measure can, for example, be a discrete function, where a corresponding value of the reconstructed measure can be given for each individual active surface of the detectors.

[0069] The determination of the reconstructed measure can be understood as a result of the reconstruction. In particular, a detector response, for example, of one or more detector signals, of the detector array is uniquely defined by the reconstructed measure.

[0070] The reconstructed measure can be used, for example, to simulate or reconstruct a point or a multitude of points within a reconstructed point cloud. This can be achieved by taking into account factors such as the travel time of the virtual laser pulse, the response functions, and / or a virtual emission direction of the virtual laser pulse.

[0071] By modeling the object surface using discrete sampling points and calculating the response functions, as well as superimposing them, the object surface is modeled, for example, as a collection of free or quasi-free electric charges. These are excited by the virtual laser pulse and subsequently emit electromagnetic waves, the wavelength of which is determined, in particular, by the wavelength of the virtual laser pulse, for example, equal to or approximately equal to the wavelength of the virtual laser pulse. The electromagnetic waves emitted by the charges can be approximated by different types of waves. For example, the response functions can each be determined as spherical waves.

[0072] As comparisons with real systems demonstrate, this makes it possible to achieve a realistic reconstruction or partial reconstruction of the sampling. 2023PF01453

[0073] 12

[0074] The reconstruction or partial reconstruction of the scan can be understood, in particular, as meaning that the computer-implemented method according to the invention reconstructs at least some of the information that was explicitly or implicitly obtained through the measurements or data acquisition by the active optical sensor system, but is no longer explicitly included in the object representation, in particular the reconstructed dimension. Determining the reconstructed dimension thus already constitutes at least a partial reconstruction of the scan. In some embodiments, further information or data can be reconstructed.

[0075] The invention particularly utilizes the fact that the first and second curves lie on the approximate surface of the object to determine the reconstructed dimension. How exactly the object representation or the first and second curves were originally generated is generally irrelevant for the computer-implemented method according to the invention; it is only necessary that the first and second curves lie on the approximate surface of the object. In further embodiments of the computer-implemented method according to the invention, additional information, such as how the object representation or the first and second curves were originally generated, can also be used to further improve and / or complete the reconstruction.

[0076] According to at least one embodiment, the response functions are identical to each other except for a respective scaling factor, wherein the scaling factor depends on the reflectivity of the object surface at the respective point and / or on the distance of the respective point from the sensor system.

[0077] The object representation contains, for example, respective values ​​for reflectivity and / or distance for each of the points, or this information is provided in addition to the object representation.

[0078] According to at least one embodiment, the intensity of the virtual laser pulse is specified. The response functions are generated as a function of the intensity of the virtual laser pulse. 2023PF01453

[0079] 13

[0080] According to at least one embodiment, the emission direction of the virtual laser pulse is predetermined. The response functions are generated depending on the emission direction.

[0081] This allows, especially in combination with the location-dependent reconstructed measure and the runtime, a three-dimensional or 2.5-dimensional sampling to be simulated.

[0082] According to at least one embodiment, a first part of the sampling points lies on the first curve and / or a second part of the sampling points lies on the second curve.

[0083] The number of sampling points in the first part of the sampling depends, for example, on the length of the first curve and on a correspondingly predefined first sampling rate. For instance, the sampling points of the first part can be uniformly distributed along the first curve, and the distance between any two sampling points of the first part is determined by the first sampling rate. It is also possible that specific anchor points are defined, such that, for example, a sampling point of the first part is located at each endpoint of the first curve. In this case, it might be intended that the distance between any two sampling points of the first part deviates from the distance determined by the first sampling rate, for example, to ensure the uniform distribution of the sampling points of the first part.Alternatively, it can be provided that the uniform distribution is only intended for all sampling points of the first part except the two endpoints, and not for the two endpoints and their direct neighbors.

[0084] The number of sampling points for the second part of the sampling depends, for example, on the length of the second curve and on a correspondingly defined second sampling rate. For instance, the sampling points of the second part can be uniformly distributed along the second curve, with the distance between any two sampling points being determined by the second sampling rate. It is also possible to define specific anchor points, such that, for example, a sampling point of the second part is located at each endpoint of the second curve. In this case, it might be intended that the distance between any two sampling points of the second part deviates from the distance defined by the second sampling rate, for example, to ensure the uniform distribution of the sampling points of the second part.Alternatively, it can be provided that the uniform distribution applies only to all sampling points of the second part except the two endpoints, and not to the two endpoints and their direct neighbors. 2023PF01453.

[0085] 14

[0086] The first and / or second sampling rate can also depend on the object's distance, for example, a minimum distance between the object and the active optical sensor system. In this case, the object representation includes this distance.

[0087] Since the first curve and the second curve lie on the approximate surface of the object by means of the construction of the object representation, such embodiments result in a particularly high accuracy and / or reliability of the reconstruction.

[0088] For example, the first curve and the second curve may intersect or touch. In particular, an endpoint of the second curve may lie on the first curve. It is therefore also possible that a sampling point is contained in both the first part of the sampling points and the second part of the sampling points, especially if an endpoint of the second curve lies on the first curve and corresponds to a sampling point.

[0089] According to at least one embodiment, the first curve is an elliptical arc, in particular a semi-ellipse, and / or the second curve is an elliptical arc, in particular a semi-ellipse.

[0090] This includes, in particular, the limiting cases of a straight line, which corresponds to a semi-ellipse with a first semi-axis length equal to half the length of the line and a second semi-axis length equal to zero, and a semicircle, which corresponds to a semi-ellipse with two equal semi-axis lengths. This can be interpreted, in particular, as meaning that the curves are generally defined as semi-ellipses, but when adapting them to the object, one of the two limiting cases may prove to be optimal.

[0091] By using an elliptical arc or a semi-ellipse, a good approximation to the actual surface of the object can be achieved with comparatively low complexity of the curves.

[0092] According to at least one embodiment, a mirrored first curve is generated by reflecting the first curve across a mirror plane that is perpendicular to a connecting line, in particular a straight connecting line, between two endpoints of the second curve and bisects the connecting line. A third part of the sampling points lies on the mirrored first curve. 2023PF01453

[0093] 15

[0094] The number of sampling points of the third part of the sampling depends, for example, on the length of the third curve and on a correspondingly defined third sampling rate. For instance, the sampling points of the third part can be uniformly distributed along the third curve, and the distance between any two sampling points of the third part is determined by the third sampling rate. It is also possible that specific anchor points are defined, such that, for example, a sampling point of the third part is located at each endpoint of the third curve. In this case, it might be stipulated that the distance between any two sampling points of the third part deviates from the distance defined by the third sampling rate, for example, to ensure the uniform distribution of the sampling points of the third part.Alternatively, it can be provided that the uniform distribution is only intended for all sampling points of the third part except the two endpoints, and not for the two endpoints and their direct neighbors.

[0095] In particular, the third sampling rate can be the same as the first sampling rate.

[0096] For example, the second curve is also mirror-symmetric with respect to the mirror plane. In such embodiments, it is assumed, for example, that the object is approximately mirror-symmetric with respect to the mirror plane, which is often a good approximation insofar as the second curve lies on the object's surface, the second curve is mirror-symmetric with respect to the mirror plane, and the first curve and its reflection pass through the endpoints of the second curve.

[0097] In such embodiments, a larger part of the approximated surface is modeled by the multitude of sampling points, leading to a further increase in the accuracy and / or reliability of the reconstruction.

[0098] For example, the reflected first curve and the second curve may intersect or touch. In particular, an endpoint of the second curve may lie on the reflected first curve. It is therefore also possible that a sampling point is contained in both the third part of the sampling points and the second part of the sampling points, especially if an endpoint of the second curve lies on the reflected first curve and corresponds to a sampling point.

[0099] According to at least one embodiment, the third part of the sampling points corresponds to the sampling points of the first part of the sampling points reflected across the mirror plane. 2023PF01453

[0100] 16

[0101] In other words, for every sampling point of the first part of the sampling points, there exists a sampling point reflected across the mirror plane that is contained in the third part of the sampling points, and vice versa.

[0102] According to at least one embodiment, scaling the second curve produces a scaled second curve which has a first endpoint that is a sampling point of the first part of the sampling points, and a second endpoint that is a sampling point of the second part of the sampling points and corresponds to the first endpoint of the scaled second curve reflected across the mirror plane.

[0103] Accordingly, the second endpoint of the scaled second curve lies in particular on the mirrored first curve and is contained, for example, by the third part of the sampling points.

[0104] In particular, a quarter of the sample points lie on the scaled second curve. Specifically, the first endpoint of the scaled second curve is contained by both the fourth part of the sample points and the first part of the sample points.

[0105] For example, the second endpoint of the scaled second curve is contained by both the fourth part of the sample points and the third part of the sample points.

[0106] In such embodiments, a larger part of the approximated surface is modeled by the multitude of sampling points, leading to a further increase in the accuracy and / or reliability of the reconstruction.

[0107] As described regarding the scaled second curve, for further or all pairs of sample points of the first part and the second part of the sample points that are reflected across the mirror plane, a further scaled second curve can be generated by appropriately scaling the second curve, and further parts of the sample points can lie on the respective further scaled second curves. A first endpoint of the respective further scaled second curve is then a sample point of the first part of the sample points, and a second endpoint is given by the respective first endpoint of the further scaled second curve reflected across the mirror plane and is contained in particular in the third part of the sample points. This allows the accuracy and / or reliability of the reconstruction to be further increased. 2023PF01453

[0108] 17

[0109] The scaling of the second curve is carried out in such a way that the shape of the second curve is preserved and only the curve length is adjusted accordingly, so that the endpoints are reached as described above.

[0110] In particular, the second curve is scaled by a scaling factor that is given by the ratio of the distance between the first and second endpoints of the scaled second curves to the distance between the endpoints of the second curve. It should be noted that the first and second endpoints of the scaled second curve are predefined as described above, and the scaled second curve can therefore be generated by appropriately scaling the second curve.

[0111] For example, if the second curve is given in a parametric representation by

[0112]

[0113] For example, the scaled second curve is given by

[0114] v'2(t) = K v2(t)>t = 0 ... T,

[0115] where K denotes the scaling factor. In the special case of a semi-ellipse in the xy-plane as the second curve, for example, the following results:

[0116]

[0117] where a and b correspond to the semi-axis lengths of the semi-ellipse.

[0118] The scaling factor K is given in particular by d' / d, where d is the distance between the endpoints of the second curve, in the case of a semi-ellipse as above this is 2a, and d' is the distance between the endpoints of the scaled second curve. In the case of the semi-ellipse, this results in 2023PF01453

[0119]

[0120] According to at least one embodiment, the computer-implemented method according to the invention includes generating the object representation, in particular by means of the data processing system or by means of a further data processing system. For example, the object representation is generated and stored, and then, in particular at a later time, the reconstruction is carried out as described above and below.

[0121] According to at least one embodiment, to generate the object representation, an image of the environment with a plurality of image pixels is obtained using the detector array, wherein each image pixel corresponds to one of the detector pixels. A pixel value of the respective image pixel corresponds to a depth determined by the corresponding detector pixel in a direction perpendicular to the detector array, or to a characteristic value relating to the energy of incident light determined by the corresponding detector pixel. The first curve is determined depending on the image such that the first curve approximates a first edge of the object.

[0122] If the pixel value corresponds to the depth, the image can also be called a depth image. The depth can, for example, correspond to the radial distance determined, in particular by measuring the time of flight of light, or to a distance along a longitudinal axis of a Cartesian coordinate system, which can be calculated from the radial distance.

[0123] The first edge is defined, in particular, by the two-dimensional position of the image pixels belonging to the first edge in the image plane of the image, in combination with their pixel values, which correspond to the depth. The first curve is then determined such that it approximates the first edge. The first curve can be determined, for example, by defining a basic shape or type of first curve, such as an elliptical arc, and adjusting the parameters that remain free, such as the semi-axes lengths of the elliptical arc and / or its position in space, so that this first curve approximates the first edge, for example, by regression or optimization in a known manner. The first edge itself can be determined, in particular, by applying a method, especially one that is 2023PF01453

[0124] 19

[0125] Known edge detection algorithms, such as threshold-based edge detection algorithms, can be used to identify edges. This may also apply analogously to other edges of the object and corresponding curves.

[0126] According to at least one embodiment, the first curve lies in a first curve plane which forms an angle in the range [0°, 180°[ with the image plane of the image.

[0127] In other words, the first curve can be considered a plane curve in three-dimensional space. The value of the angle depends on the path of the first edge in three-dimensional space.

[0128] In this way, the complexity of the first curve and the number of parameters required to describe the first curve remain comparatively low, which further reduces the storage and computing effort.

[0129] According to at least one embodiment, the first curve lies in the first curve plane, and to generate the object representation, a second curve plane is constructed that intersects the first curve plane. For each image pixel, a further pixel value is determined, which is a predetermined first binary value if a position defined by the respective image pixel in three-dimensional space lies on a first side of the second curve plane, and a predetermined second binary value if the position defined by the respective image pixel in three-dimensional space lies on a second side of the second curve plane opposite the first side. Based on the determined further pixel values, a binary image is generated, wherein one image plane of the binary image is, in particular, parallel to the second curve plane. Based on the binary image, the second curve is determined such that the second curve approximates the second edge of the object.

[0130] For example, the first binary value is equal to one and the second binary value is equal to zero, or vice versa.

[0131] The second edge can also be interpreted as the contour of the object in the binary image. The second curve is therefore a planar curve in three-dimensional space that lies in the second curve plane. The second curve plane can, for example, be perpendicular to the first curve plane. 2023PF01453

[0132] 20

[0133] The position defined by an image pixel in three-dimensional space is given, in particular, by the position of the image pixel in the image plane and the corresponding pixel value of the image, i.e., specifically the corresponding depth. For image pixels whose position in three-dimensional space lies exactly on the second curve plane, various predefined approaches can be used. For example, the next pixel value in this case can, by definition, be the first binary value or the second binary value.

[0134] The object representation is generated and stored in such a way that the first curve and the second curve are assigned to each other. If a further first curve is defined for each additional first edge of the object, then in various embodiments, a corresponding further second curve can be generated for each additional first curve analogously to the first and second curves described above.

[0135] By representing the object using the curve pair determined as described, consisting of the first curve and the second curve, the object can be represented particularly accurately without drastically increasing the required storage and computing effort for processing the object representation.

[0136] According to at least one embodiment, the first curve is mirror-symmetric and the second curve plane is constructed as a mirror plane of the first curve.

[0137] The second curve plane therefore passes, in particular, through a midpoint of the first curve. If the first curve is a semi-ellipse, the second curve plane passes, in particular, through a vertex of the semi-ellipse.

[0138] This enables a particularly simple construction of the second curve plane and achieves a particularly meaningful representation of the object.

[0139] According to at least one embodiment, the first edge is determined using an edge detection algorithm, for example a threshold-based edge detection algorithm.

[0140] This applies analogously in various embodiments to the further first curves, the second curve, and / or, if applicable, further second curves. 2023PF01453

[0141] 21

[0142] According to at least one embodiment, the first edge is approximated by a polyline and the polyline is approximated by the first curve.

[0143] In other words, starting from, for example, a result of the edge detection algorithm, the polyline that approximates the first edge is determined, and then the first curve is determined so that it approximates the polyline.

[0144] A polyline can be understood as two or more line segments, each connected at one end. The polyline can, for example, be a polynomial chain, also known as a spline. In some embodiments, the individual line segments can be straight lines, i.e., first-degree polynomial segments or linear polynomial segments. In this case, the polyline can also be called a traverse. The term traverse does not imply, in particular, that the polyline is a closed polyline.

[0145] For example, the first curve can be a smooth curve, meaning it can be differentiated any number of times except at its endpoints. This simplifies the processing of the object representation. On the other hand, the polyline, especially when used as line segments with straight lines, is generally not differentiable at the connection points. However, the polyline may allow for a more precise approximation of the first edge.

[0146] The same applies in various embodiments to the further first curves, the second curve, and / or possibly further second curves.

[0147] According to at least one embodiment, the first edge is approximated by two or more line segments, and the two or more line segments are approximated by the first curve.

[0148] The two or more line segments differ from a polyline in that they are not necessarily connected to each other.

[0149] For example, the two or more line segments can each be polynomial segments, in particular straight line segments. The above explanations and embodiments regarding the polyline can be applied analogously to corresponding embodiments with the two or more line segments. 2023PF01453

[0150] 22

[0151] The same applies in various embodiments to the further first curves, the second curve, and / or possibly further second curves.

[0152] According to at least one embodiment, the first curve and the second curve do not lie in a common plane.

[0153] This can increase the accuracy of the approximated object surface.

[0154] According to at least one embodiment, the first curve lies in the first curve plane and the second curve lies in the second curve plane, which intersects the first curve plane.

[0155] This can increase the accuracy of the approximated object surface, especially with regard to the three-dimensionality of the object surface.

[0156] According to at least one embodiment, the reconstructed measure of the incident light energy on a surface of the detector array is determined spatially depending on the superposition. Based on this spatially dependent reconstructed measure of the incident light energy, a reconstructed detector signal is generated for each detector pixel of the plurality of detector pixels.

[0157] Accordingly, the generation of the virtual detector signal, especially taking into account parameters such as characteristic values ​​or characteristic curves of the individual optical detectors, enables a more realistic and therefore more accurate reconstruction.

[0158] In particular, if the detectors of the receiving unit are configured as APDs, they can each have an output voltage that depends on, or is proportional to, the respective radiated power on the corresponding individual active surface. The reconstructed detector signal can then, for example, include at least one virtual radiated power-dependent output voltage of the detectors.

[0159] For SPADs or other photon-counting detectors, the reconstructed detector signal can, for example, represent an approximation of a corresponding histogram of the counting events or the like. 2023PF01453

[0160] 23

[0161] According to at least one embodiment, the object representation includes at least one distance value of the object from the active optical sensor system, and depending on the at least one distance value and the reconstructed measure relating to the incoming light energy, at least one point of a reconstructed point cloud is generated.

[0162] According to at least one embodiment, the active optical sensor system has a transmitting unit, for example with a laser source, to emit light, in particular laser beams, with different emission angles within a transmission plane of the sensor system.

[0163] According to at least one embodiment, the active optical sensor system includes a receiving unit. The receiving unit is configured to detect reflected components of the light emitted by the transmitting unit and, based on this, to generate at least one detector signal.

[0164] According to at least one embodiment, the receiving unit includes at least one, for example three or more, optical detectors, which in particular each include one or more photodiodes, APDs or SPADs.

[0165] In various embodiments, the active optical sensor system comprises a processing unit coupled to the receiving unit, particularly to the optical detectors, to receive the at least one detector signal. The processing unit is configured to generate one or more sampling points depending on the at least one detector signal, specifically depending on the light transit time and, in particular, depending on the direction of emission of the light from the transmitting unit. For this purpose, the processing unit can, for example, calculate the light transit time.

[0166] According to at least one embodiment, the active optical sensor system has a shaft that is rotatably mounted, in particular about an axis of rotation that is perpendicular to the transmitting plane.

[0167] According to at least one embodiment, the active optical sensor system includes a mirror that is connected to the shaft and rotatable about the axis of rotation. 2023PF01453

[0168] 24

[0169] In particular, the transmitter unit can generate the laser pulse, which is then directed onto the mirror. Depending on the mirror's rotational position, an outgoing angle or direction of the laser pulse, or a propagation direction, is defined.

[0170] According to at least one embodiment, the active optical sensor system has a rotary encoder that is coupled to a shaft to determine a rotational position of the mirror.

[0171] According to at least one embodiment, the active optical sensor system comprises an optical unit, which in particular includes at least one lens. The at least one lens is arranged in a beam path between the transmitting unit and the mirror.

[0172] According to at least one embodiment, at least one geometric parameter is specified to describe the active optical sensor system and / or the object. The superposition is generated depending on this at least one geometric parameter.

[0173] At least one geometric parameter includes, for example, a first parameter that describes the structure of the active optical sensor system.

[0174] The at least one geometric parameter includes, for example, at least one second parameter that describes a pose of the active optical sensor system or of one or more components of the sensor system, for example, one or more detectors, the optical unit, the lens, the mirror, or another component.

[0175] Here and in the following, "pose" can be understood as position, orientation, or a combination of position and orientation.

[0176] The at least one geometric parameter can, for example, include at least one third parameter that describes a pose of the object.

[0177] The at least one geometric parameter for describing the active optical sensor system and / or the at least one geometric parameter of the object can, for example, be relevant distances or optical path lengths from the object to 2023PF01453

[0178] 25

[0179] to or define the active optical sensor system, particularly the detector array. Accordingly, the response functions depend, for example, on the geometric parameters.

[0180] By taking the geometric parameter(s) into account, the accuracy of the reconstruction can be improved.

[0181] According to at least one embodiment, at least one material parameter is specified to describe the active optical sensor system. The superposition is determined depending on this at least one material parameter.

[0182] The at least one material parameter may, for example, include a refractive index, in particular a complex refractive index or a real part of the refractive index, an absorption index or any other relevant optical parameter of the active optical sensor system, in particular of one or more components of the active optical sensor system, for example the mirror, the optical unit and / or the at least one lens.

[0183] At least one material parameter influences, for example, the optical path length from the object surface to the detector array. Furthermore, the phase of the corresponding response function at the detector array can be influenced by at least one material parameter.

[0184] Since the response functions are superimposed on the surface of the detector array, taking at least one material parameter into account can improve the accuracy of the reconstruction.

[0185] According to at least one embodiment, the response functions are generated as waves, i.e., calculated, where one wavelength of the waves is given by a predetermined wavelength of the virtual laser pulse.

[0186] According to at least one embodiment, the waves are spherical waves. That is, for a given sampling point from the plurality of sampling points, the response function is a spherical wave.

[0187] According to a further aspect of the invention, a method for at least partially automatic driving of a motor vehicle is specified, wherein a 2023PF01453

[0188] 26

[0189] The computer-implemented method according to the invention is carried out. Based on data which depend on the reconstructed measure of the incoming light energy, at least one control signal is generated for at least partially automatic driving of the motor vehicle and / or assistance information is generated to support a person driving the motor vehicle.

[0190] The data that depend on the reconstructed measure of incoming light energy may, for example, be a reconstructed point cloud or other data determined based on the reconstructed measure.

[0191] The at least one control signal can be provided, for example, to one or more actuators of the motor vehicle, including, for example, one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle. Based on the at least one control signal, the one or more actuators can influence the longitudinal and / or lateral steering of the vehicle in order to steer the motor vehicle at least partially automatically.

[0192] The assistance information can be output via a vehicle output device, such as a display and / or an audio output system and / or a haptic output system.

[0193] Further embodiments of the method according to the invention follow directly from the various configurations of the computer-implemented method according to the invention, and vice versa. In particular, individual features and corresponding explanations as well as advantages relating to the various configurations of the computer-implemented method according to the invention can be transferred analogously to corresponding configurations of the method according to the invention.

[0194] According to another aspect of the invention, a data processing system is provided which is configured to carry out a computer-implemented method according to the invention.

[0195] The terms "data processing system" and "at least one data processing device" may be used interchangeably within the scope of this disclosure. In2023PF01453

[0196] 27

[0197] In the present disclosure, a data processing device can be understood, for example, as a device with processing circuits for processing data. A data processing device can therefore perform arithmetic operations to process data. Indexed access to a data structure, such as a lookup table (LUT) or a database, can also be considered an arithmetic operation. Data processing that is partially or fully implemented in hardware can also be considered an arithmetic operation.

[0198] A data processing device may, in particular, comprise one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more systems-on-a-chip (SoCs). A data processing device may also comprise one or more processors, for example, one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The data processing device may also comprise a physical or virtual cluster of computers or other devices of the aforementioned type.

[0199] A data processing device may also include one or more hardware and / or software interfaces, for example for receiving and / or providing data.

[0200] A data processing device may also include one or more storage devices. A storage device may be implemented as volatile memory, such as dynamic random access memory (DRAM) or static random access memory (SRAM), or as non-volatile data storage, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or flash EEPROM, ferroelectric random access memory (FRAM), magnetoresistive random access memory (MRAM), or phase-change random access memory (PCRAM).

[0201] According to a further aspect of the invention, an electronic vehicle guidance system is provided which has a data processing system according to the invention and a control system which is configured to, based on data which are derived from the reconstructed measure relating to the arriving 2023PF01453

[0202] 28

[0203] Light energy is used to generate at least a control signal for at least partially automatic driving of the motor vehicle and / or to generate assistance information to support a person driving the motor vehicle.

[0204] An electronic vehicle control system (EVS) can be understood as an electronic system designed to control a vehicle fully automatically or autonomously, in particular without requiring any intervention from a driver. The vehicle automatically performs all necessary functions, such as steering, braking, and / or acceleration maneuvers, monitoring and recording road traffic, and reacting accordingly. Specifically, the EVS can implement a fully automatic or fully autonomous driving mode of the vehicle according to Level 5 of the SAE J3016 classification. An EVS can also be understood as an advanced driver assistance system (ADAS), which supports the driver during partially automated or semi-autonomous driving.In particular, the electronic vehicle guidance system can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and in the following, "SAE J3016" refers to the corresponding standard in the April 2021 version.

[0205] At least partially automated vehicle control can therefore include driving the vehicle in accordance with a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. At least partially automated vehicle control can also include driving the vehicle in accordance with a partially automated or semi-autonomous driving mode according to Levels 1 to 4 of SAE J3016.

[0206] According to at least one embodiment, the electronic vehicle guidance system includes the active optical sensor system.

[0207] Further embodiments of the electronic vehicle guidance system according to the invention follow directly from the various configurations of the computer-implemented method according to the invention and vice versa.

[0208] In particular, individual features and corresponding explanations as well as advantages regarding the various embodiments of the computer-implemented method according to the invention can be applied analogously to corresponding embodiments of the 2023PF01453

[0209] 29

[0210] The electronic vehicle guidance system according to the invention is transmitted. In particular, the electronic vehicle guidance system according to the invention is designed or programmed to carry out a method according to the invention for at least partially automatic driving of a motor vehicle. In particular, the electronic vehicle guidance system according to the invention carries out the method according to the invention for at least partially automatic driving of a motor vehicle.

[0211] According to another aspect of the invention, a computer program with instructions is specified. When the instructions are executed by a data processing system, the instructions cause the data processing system to carry out a computer-implemented method according to the invention.

[0212] The instructions can be provided, for example, as program code. This program code can be provided, for example, as binary code or assembly language, and / or as source code in a programming language such as C, and / or as a program script, such as Python.

[0213] According to a further aspect of the invention, a further computer program with additional commands is specified. When the additional commands are executed by an electronic vehicle guidance system according to the invention, in particular by the data processing system of the electronic vehicle guidance system, the commands cause the electronic vehicle guidance system to carry out a method according to the invention for at least partially automatic guidance of a motor vehicle.

[0214] The additional instructions can be provided, for example, as program code. This program code can be provided, for example, as binary code or assembly language, and / or as source code in a programming language such as C, and / or as a program script, such as Python.

[0215] According to another aspect of the invention, a computer-readable storage medium is specified which stores a computer program according to the invention and / or a further computer program according to the invention.

[0216] The computer program, the further computer program, and the computer-readable storage medium are each computer program products with the commands and / or the further commands. 2023PF01453

[0217] 30

[0218] Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those mentioned below in the description of the figures and / or illustrated in the figures, may be encompassed by the invention not only in the combinations specified, but also in other combinations. In particular, embodiments and combinations of features that do not include all the features of an originally formulated claim may also be encompassed by the invention. Furthermore, embodiments and combinations of features that go beyond or deviate from the combinations of features mentioned in the claims may also include the invention.

[0219] The invention is explained in more detail below with reference to specific exemplary embodiments and corresponding schematic drawings. Identical or functionally equivalent elements in the drawings may be provided with the same reference numerals. The description of identical or functionally equivalent elements is not necessarily repeated with respect to the different figures.

[0220] The figures show

[0221] Fig. 1 shows a schematic representation of a motor vehicle with an exemplary embodiment of an electronic vehicle guidance system according to the invention;

[0222] Fig. 2 shows a schematic representation of an active optical sensor system of another exemplary embodiment of an electronic vehicle guidance system according to the invention;

[0223] Fig. 3 shows a schematic flowchart of an exemplary embodiment of a computer-implemented method according to the invention;

[0224] Fig. 4 shows a schematic flowchart of part of a further exemplary embodiment of a computer-implemented method according to the invention; 2023PF01453

[0225] 31

[0226] Fig. 5 schematically shows a depth image corresponding to a further exemplary embodiment of a computer-implemented method according to the invention;

[0227] Fig. 6 schematically shows a depth image and first curves of an object representation according to another exemplary embodiment of a computer-implemented method according to the invention;

[0228] Fig. 7 schematically shows a depth image, a first curve and a second curve of an object representation according to a further exemplary embodiment of a computer-implemented method according to the invention; and

[0229] Fig. 8 schematic curve shapes for use in another exemplary embodiment of a computer-implemented method according to the invention.

[0230] Figure 1 schematically shows a motor vehicle 1 with an exemplary embodiment of an electronic vehicle guidance system 2 according to the invention. The electronic vehicle guidance system 2 comprises a data processing system 4 and an active optical sensor system 3.

[0231] An exemplary embodiment of the active optical sensor system 3 is shown schematically in Fig. 2. The active optical sensor system 3 comprises a housing 7 and an emitter unit 9 for emitting light 6a into the external environment of the active optical sensor system 3. The emitted light 6a passes, for example, through a window 8 of the housing 7. The active optical sensor system 3 has a control unit 11, which is configured to control a deflection device 13, for example, a rotatable mirror, of the active optical sensor system 3, to deflect the light 6a in different directions, and thereby to scan the environment of the active optical sensor system 3.

[0232] The active optical sensor system 3 has a detector unit with a two-dimensional detector array 10, which has a plurality of detector pixels arranged, for example, in a plurality of rows and a plurality of columns. If fractions 6b of the emitted light 6a are reflected by an object 5 in the environment, these fractions 6b can be directed towards the active optical sensor system 32023PF01453

[0233] 32

[0234] are reflected back and, for example, pass through window 8 again, where they are directed by the deflection device 13 onto the detector array 10 and detected by it. Depending on the corresponding detector signals, the control unit 11 can then calculate, for example by measuring the time of flight of light, a radial distance of the object 5 from the active optical sensor system 3.

[0235] The active optical sensor system 3 is designed, in particular, as a lidar sensor system in the form of a laser scanner. The emitter unit 9 thus comprises one or more laser light sources, in particular laser diodes, for generating the light 6a. The detector pixels each comprise one or more photodetectors, which can be designed, for example, as photodiodes, APDs, or SPADs. Possible optical components in the optical path are represented in Fig. 2 by a lens 12.

[0236] In particular, a computer-implemented method according to the invention for reconstructing or partially reconstructing a scan of the object 5 with the active optical sensor system 3 from an object representation of the object 5 can be carried out using the data processing system 4. A schematic flowchart of an exemplary embodiment of such a computer-implemented method is shown in Fig. 3.

[0237] The object representation has a first curve 14, 14a, 14b, 14c, 14d in three-dimensional space, which lies on an approximate surface of the object 5, and a second curve 15 in three-dimensional space, which lies on the approximate surface of the object 5.

[0238] It should be noted that the active optical sensor system 3 is generally not required to carry out the computer-implemented method according to the invention. However, the object representation may have been generated based on measurements from the active optical sensor system 3. The object representation is obtained by the data processing system 4 in step 300.

[0239] Depending on the first curve 14, 14a, 14b, 14c, 14d and the second curve 15, a plurality of sampling points 16 are determined on the approximate surface of the object 5 in step 320. For each of the sampling points 16, a response function 17 is generated in step 330, which describes a respective response to a virtual laser pulse, and a superposition 18 of the generated response functions 172023PF01453 is performed.

[0240] 33

[0241] In step 340, a reconstructed measure concerning the light energy arriving at a detector array 10 of the active optical sensor system 3 is determined as a function of the superposition 18.

[0242] For example, the reconstructed measure of the incident light energy on a surface of the detector array 10 can be determined spatially depending on the superposition 18. Depending on the spatially dependent reconstructed measure of the incident light energy, a reconstructed detector signal can be generated for each detector pixel of the plurality of detector pixels.

[0243] For example, the object representation includes at least one distance value of the object 5 from the active optical sensor system 3, and depending on the at least one distance value and the location-dependent reconstructed measure concerning the incoming light energy, a reconstructed point cloud is generated.

[0244] Based on the reconstructed point cloud, various functions and procedures can be performed that can be used for at least partially automated driving of the motor vehicle 1. In particular, the data processing system 4 can generate at least one control signal for at least partially automated driving of the motor vehicle and / or assistance information to support a person driving the motor vehicle 1 in driving the motor vehicle 1, depending on the reconstructed point cloud.

[0245] The at least one control signal can, for example, be provided to one or more actuators of the motor vehicle 1, including, for example, one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle 1. The one or more actuators can influence longitudinal and / or lateral steering of the motor vehicle in order to steer the motor vehicle 1 at least partially automatically.

[0246] The assistance information can be output via an output device of the vehicle 1, for example a display and / or an audio output system and / or a haptic output system.

[0247] Figure 4 shows further details of another exemplary embodiment of the computer-implemented method according to the invention, which is based on the embodiment of Figure 3. 2023PF01453

[0248] 34

[0249] For the sake of clarity, the process is shown only for a first curve 14. The first curve 14 and the second curve 15 are, for example, each semi-ellipses 22, which also includes the limiting case of straight lines, as schematically shown in Fig. 8 and in the simplified example of Fig. 6. The limiting case of a semicircle is also included. An endpoint of the second curve 15 lies, for example, at the center of the first curve 14. In particular, the first curve 14 lies in a first curve plane, here parallel to the yz-plane, and the second curve 15 lies in a second curve plane, here parallel to the xy-plane, which is perpendicular to the first curve plane.

[0250] For example, the endpoints of the first curve 14 lie in the second curve plane and the endpoints of the second curve 15 lie in the first curve plane.

[0251] Starting from the object representation, in step 305 a reflected first curve 14' is generated by reflecting the first curve 14 across a mirror plane that is perpendicular to a connecting line between two endpoints of the second curve (15) and bisects the connecting line. In this case, the mirror plane is parallel to the xz-plane.

[0252] In step 310, a scaled second curve 15' is generated by scaling the second curve 15. This second curve has a first endpoint that lies on the first curve 14 and a second endpoint that lies on the mirrored first curve 14'. To scale the second curve 15, it is scaled, specifically multiplied, by a scaling factor, which is given by the ratio of the distance between the endpoints of the scaled second curve 15' to the distance between the endpoints of the second curve 15.

[0253] In particular, a first part of the sampling points 16 lies on the first curve 14, and a second part of the sampling points 16 lies on the second curve 15. A third part of the sampling points 16 lies on the mirrored first curve 14. Specifically, the third part of the sampling points 16 corresponds to the sampling points 16 of the first part of the sampling points 16 reflected across the mirror plane. In other words, for example, the endpoints of the second curve 15 are encompassed by both the second part of the sampling points 16 and the first part of the sampling points 16 (left endpoint in Fig. 4), as well as by the third part of the sampling points 16 (right endpoint in Fig. 4). The endpoints of the second curve 15 are thus their respective mirror images with respect to the mirror plane. Likewise, for example, the endpoints of the scaled second curve 15' are their respective mirror images with respect to the mirror plane. 2023PF01453

[0254] 35

[0255] A fourth part of the sampling points 16 lies on the scaled second curve 15'. In other words, for example, the endpoints of the scaled second curve 15' are encompassed by both the fourth part of the sampling points 16 and by the first part of the sampling points 16 (left endpoint in Fig. 4) or by the third part of the sampling points 16 (right endpoint in Fig. 4).

[0256] Similarly, further scaled second curves can be constructed for each further pair of a sampling point 16 on the first curve 14 and a corresponding mirror image with respect to the mirror plane, which then corresponds to a sampling point 16 on the mirrored first curve 14', and respective further parts of the sampling points 16 can then lie on the further scaled second curves.

[0257] Figures 5 to 7 describe how, for example, the object representation can be generated, which in some embodiments is part of the computer-implemented method according to the invention.

[0258] For example, a depth image 20 of the environment of the active optical sensor system 3 is generated using the detector array 10. This image has a multitude of image pixels, where each image pixel corresponds to one of the detector pixels, and a pixel value of the respective image pixel corresponds to a depth determined by the corresponding detector pixel in a direction perpendicular to the detector array 10, i.e., for example, in the x-direction. In Figures 5 to 7, the depth image 20 is schematically represented as a binary image for the sake of simplicity, where the hatched area corresponds to a background 21, particularly at infinity, and the unhatched area corresponds to the object 5. In the general case, the depth image would be a monochromatic image, for example, a grayscale image.An image plane of the depth image 20 is defined by a transverse direction y and a vertical direction z; a longitudinal direction x is oriented perpendicular to the transverse direction y and the vertical direction z. In the simplified example of Fig. 5, the object 5 has a flat rectangular surface on the side facing the active optical sensor system 3.

[0259] Based on the depth image 20, a first curve 14a, 14b, 14c, 14d is determined in three-dimensional space, which approximates a first edge of the object 5. This is shown schematically in Fig. 6 for the depth image 20 from Fig. 5 for four first curves 14a, 14b, 14c, 14d. In this simplifying example, the first curves 14a, 14b, 14c, 14d lie in the yz-plane, which, however, is generally not necessarily the case.

[0260] 36

[0261] This is the case. In general, the first curve 14a, 14b, 14c, 14d, for example, lies in a first curve plane which encloses an angle in the range [0, 180°[ with an image plane of the depth image 20.

[0262] In particular, based on the depth image 20, it is also possible to distinguish between edges of different objects, although it may be that these are not distinguishable in the image plane of the depth image 20 if the objects partially overlap or are directly adjacent to each other.

[0263] The first curves 14a, 14b, 14c, 14d are determined, for example, as respective semi-ellipses 19, which in particular also includes the limiting case of straight lines shown in Fig. 6 and Fig. 7.

[0264] The object representation contains the first curve 14a, 14b, 14c, 14d, or rather all first curves 14a, 14b, 14c, 14d generated for object 5. For each first curve 14a, 14b, 14c, 14d, the object representation contains a corresponding second curve 15, as shown in Fig. 7 for the first curve 14a. The second curve 15 can also be defined as a semi-ellipse.

[0265] To determine the second curve 15, a second curve plane is constructed which intersects the first curve plane. For example, the second curve plane is perpendicular to the first curve plane. For example, the second curve plane is a mirror plane with respect to a mirror symmetry of the first curve 14a, 14b, 14c, 14d.

[0266] The second curve plane divides the three-dimensional space into two halves, defined by a region on the first side of the second curve plane and a region on the opposite side of the second curve plane. For each image pixel of the depth image 20, for example, a further pixel value is determined. This first binary value is determined if a position in three-dimensional space defined by the respective image pixel lies on the first side of the second curve plane, and a second binary value is determined if the position in three-dimensional space defined by the respective image pixel lies on the second side of the second curve plane. A binary image is generated based on these determined pixel values. Based on this binary image, the second curve 15 is determined, which approximates a second edge of the object 5.

Claims

2023PF01453 37 Patent claims 1. Computer-implemented method for reconstructing or partially reconstructing a scan of an object (5) with an active optical sensor system (3) from an object representation of the object (5), wherein the object representation has a first curve (14, 14a, 14b, 14c, 14d) in three-dimensional space which lies on an approximate surface of the object (5), and a second curve (15) in three-dimensional space which lies on the approximate surface of the object (5); depending on the first curve (14, 14a, 14b, 14c, 14d) and the second curve (15) a plurality of sampling points (16) on the approximate surface of the object (5) is determined; for each of the sampling points (16) a response function (17) is generated which describes a respective response to a virtual laser pulse; a superposition (18) of the generated response functions (17) is determined; and a reconstructed measure of light energy arriving at a detector array (10) of the active optical sensor system (3) is determined as a function of the superposition (18), wherein the detector array (10) has a plurality of detector pixels.

2. Computer-implemented method according to claim 1, wherein a first part of the sampling points (16) lies on the first curve (14, 14a, 14b, 14c, 14d) and / or a second part of the sampling points (16) lies on the second curve (15).

3. Computer-implemented method according to one of the preceding claims, wherein the first curve (14, 14a, 14b, 14c, 14d) and the second curve (15) are each elliptical arcs or semi-ellipses.

4. Computer-implemented method according to one of the preceding claims, wherein 2023PF01453 38 a reflected first curve (14') is generated by reflecting the first curve (14, 14a, 14b, 14c, 14d) across a mirror plane that is perpendicular to a line connecting two endpoints of the second curve (15) and bisects the line connecting; and a third part of the sampling points (16) lies on the mirrored first curve (14').

5. Computer-implemented method according to claim 4, wherein the third part of the sampling points (16) corresponds to the sampling points (16) of the first part of the sampling points (16) reflected in the mirror plane.

6. Computer-implemented method according to claim 5, wherein by scaling the second curve (15) a scaled second curve (15') is generated which has a first endpoint that is a sampling point of the first part of the sampling points (16), and a second endpoint that is a sampling point of the second part of the sampling points (16) and that corresponds to the first endpoint reflected across the mirror plane; and a fourth part of the sampling points (16) lies on the scaled second curve (15').

7. Computer-implemented method according to claim 6, wherein, to scale the second curve (15), the second curve (15) is scaled by a scaling factor which is given by a ratio of a distance of the first endpoint from the second endpoint to a distance between the endpoints of the second curve (15).

8. Computer-implemented method according to any one of the preceding claims, wherein an image (20) of the environment with a plurality of image pixels is obtained using the detector array (10), wherein each image pixel corresponds to one of the detector pixels; a pixel value of the respective image pixel corresponds to a depth determined by means of the corresponding detector pixel in a direction perpendicular to the detector array (10) or to a characteristic value relating to an energy of incident light determined by means of the corresponding detector pixel;2023PF01453 39 the first curve (14, 14a, 14b, 14c, 14d) is determined depending on the image (20), such that the first curve (14, 14a, 14b, 14c, 14d) approximates a first edge of the object (5).

9. Computer-implemented method according to claim 8, wherein the first curve (14, 14a, 14b, 14c, 14d) lies in a first curve plane and a second curve plane is constructed which intersects the first curve plane; for each image pixel of the image (20) a further pixel value is determined, which is a predetermined first binary value if a position defined by the respective image pixel in three-dimensional space lies on a first side of the second curve plane, and a predetermined second binary value if the position defined by the respective image pixel in three-dimensional space lies on a second side of the second curve plane opposite the first side; based on the determined further pixel values, a binary image is generated; based on the binary image, the second curve (15) is determined such that the second curve (15) approximates a second edge of the object (5).

10. Computer-implemented method according to any one of claims 1 to 8, wherein the first curve (14, 14a, 14b, 14c, 14d) and the second curve (15) do not lie in a common plane; or where the first curve (14, 14a, 14b, 14c, 14d) lies in a first curve plane and the second curve (15) lies in a second curve plane which intersects the first curve plane.

11. Computer-implemented method according to one of the preceding claims, wherein at least one reconstructed detector signal is generated from the reconstructed measure relating to the incoming light energy.

12. Computer-implemented method according to any one of claims 1 to 10, wherein the reconstructed measure of the incident light energy on a surface of the detector array (10) is determined as a function of the superposition (18); and Depending on the location-dependent reconstructed measure of the incoming light energy, a reconstructed detector signal is generated for each detector pixel of the multitude of detector pixels. 2023PF01453 40 13. Computer-implemented method according to one of the preceding claims, wherein the object representation includes at least one distance value of the object (5) from the active optical sensor system (3) and, depending on the at least one distance value and the reconstructed measure relating to the incoming light energy, at least one point of a reconstructed point cloud is generated.

14. Computer-implemented method according to one of the preceding claims, wherein the response functions (17) are generated as waves, wherein a wavelength of the waves is given by a predetermined wavelength of the virtual laser pulse.

15. Method for at least partially automatic driving of a motor vehicle (1), wherein a computer-implemented method according to one of the preceding claims is carried out, and at least one control signal for at least partially automatic driving of the motor vehicle (1) is generated based on data which depend on the reconstructed measure of the incoming light energy; and / or Assistance information is generated to support a person driving the motor vehicle (1) in driving the motor vehicle (1).

16. Data processing system (4) configured to carry out a computer-implemented method according to any one of claims 1 to 14.

17. Electronic vehicle guidance system (2) comprising a data processing system (4) according to claim 16 and a control system configured to operate based on data which depend on the reconstructed measure relating to the incoming light energy to generate at least one control signal for at least partially automatic operation of the motor vehicle (1); and / or To generate assistance information to support a person driving the motor vehicle (1) in driving the motor vehicle (1). 2023PF01453 41 18. Electronic vehicle guidance system (2) according to claim 17, comprising the active optical sensor system (3).

19. computer program product - Commands which, when executed by a data processing system (4), cause the data processing system (4) to perform a computer-implemented method according to any one of claims 1 to 14; and / or further commands which, when executed by an electronic vehicle guidance system (2) according to any one of claims 17 or 18, cause the electronic vehicle guidance system (2) to perform a method according to claim 15.

Images