Method for operating a sensor carrier arranged on a vehicle and sensor arrangement

Abstract

Method for operating a sensor carrier (2) arranged on a vehicle (1), which has at least one sensor device (3) that is calibrated to a sensor carrier coordinate system (27) of the sensor carrier (2), wherein at least in a partial area (11) of a vehicle surface (12) of the vehicle (1) that lies in a detection range (10) of the at least one sensor device (3), at least three marking elements (16) are arranged on the vehicle surface (12), and for each of the marking elements (16) of a control device (7), three-dimensional coordinate data (33) are provided, which describe a three-dimensional coordinate of the marking element (16) in a vehicle coordinate system (25) of the vehicle (1), wherein the method comprises the following steps: - for each of the at least three marking elements (16): • detecting (S3) marking element data (34) describing the marking element (16) by means of the at least one sensor device (3); • Determining (S4) marker element coordinate data (36) which describe a two-dimensional coordinate of a center point of the marker element (16) in the sensor carrier coordinate system (27) by applying a center point determination criterion (35) to the acquired marker element data (34) by means of the control device (7); - determining (S5) at least one orientation parameter (46) which describes an orientation of the sensor carrier coordinate system (27) relative to the vehicle coordinate system (25), by applying a spatial backward step (44) to the provided three-dimensional coordinate data (33) and the determined marker element coordinate data (36) by means of the control device (7); and - operating (S6) the sensor carrier (2) taking into account the determined at least one orientation parameter (46) by means of the control device (7); wherein the following steps are carried out when applying the center point determination criterion (35) for each marking element (16): - determining a center of gravity (37) of the marker element (16) in a binary image representation (38) of the marker element data (34); - creating a plurality of grey value profiles (39) in a grey value representation (40) of the marking element data (34) which originate from the determined centre of gravity (37); - determining an edge point (41) of the marking element (16) to the vehicle surface (12) in each of the gray value profiles (39) with subpixel accuracy; - determining a coherent edge boundary (42) of the marking element (16), in particular an elliptical edge boundary (42), by performing an edge shape adjustment, in particular an ellipse adjustment, over all determined edge points (41); and - determining an edge boundary center point (43) of the edge boundary (42), in particular an ellipse center point; and - Providing the determined edge boundary center point (43) as marking element coordinate data (36).

Description

The invention relates to a method for operating a sensor carrier arranged on a vehicle. The invention also relates to a sensor arrangement comprising a vehicle, a control device and a sensor carrier arranged on the vehicle.A vehicle, such as a motor vehicle, for example, can have at least one sensor device, such as a camera and / or a lidar sensor, for example. For a validation of a function of the sensor device and / or a checking of a function of the vehicle, which is based on sensor data of the sensor device, a sensor arrangement external to the vehicle can be arranged at least temporarily, for example during a development process of a driver assistance system for the motor vehicle. The sensor arrangement can be designed as a sensor carrier which is fastened, for example, on a roof of the vehicle. The sensor carrier can have, for example, a plurality of sensor devices, for example at least one camera, at least one lidar sensor, an inertial measurement unit and / or a position determination unit. The sensor carrier can alternatively be referred to as a multisensor system. The driver assistance system can be designed to assist in a parking process for static and / or dynamic obstacle detection and / or for lane detection.DE 10 2004 056 669 A1 shows a device for calibrating an image sensor system in a motor vehicle. This includes an on-board calibration object, which is arranged, for example, on the engine hood of the motor vehicle.DE 102 46 067 A1 discloses a method and a device for calibrating at least one image sensor system, which is located on and / or in and / or on a motor vehicle, by means of at least one calibration object. For calibration, the alignment of the image sensor system with respect to the geometric travel axis of the motor vehicle is determined.It is the object of the invention to provide a solution by means of which a sensor carrier arranged on a vehicle can be operated reliably and accurately.The object is achieved by the subject matters of the independent claims. Advantageous embodiments with expedient and non-trivial developments of the invention are specified in the dependent claims, the following description and the figures.One aspect of the invention relates to a method for operating a sensor carrier arranged on a vehicle. The vehicle can be, for example, a motor vehicle, in particular a passenger car, lorry, bus and / or motorcycle. The vehicle can alternatively or additionally be a watercraft, an aircraft and / or a robot, in particular a mobile robot.The sensor carrier has at least one sensor device. The sensor carrier preferably has a plurality of sensor devices, for example a plurality of cameras and / or lidar sensors, each directed in different directions. Furthermore, the sensor carrier can have a position determination device. The position determination device can determine a position of the sensor carrier, for example, on the basis of a global navigation satellite system (GNSS). The inertial measurement unit (IMU) has a combination of a plurality of inertial sensors such as acceleration sensors and rotation rate sensors. The at least one sensor device is calibrated to a sensor carrier coordinate system of the sensor carrier. This means that all sensor devices of the sensor carrier are related to a common sensor carrier coordinate system, i.e. their respective sensor data are available in relation to the sensor carrier coordinate system. The sensor carrier coordinate system can define, for example, a longitudinal direction, a transverse direction and a vertical direction of the sensor carrier. The sensor carrier coordinate system is thus a three-dimensional coordinate system with three axes arranged perpendicular to one another. The sensor carrier coordinate system can be placed in one of the sensor devices, so that this sensor device represents a reference sensor of the sensor carrier. The reference sensor is, for example, the camera whose detection range is arranged to look back in the longitudinal direction.The invention is based on the finding that the sensor devices of the sensor carrier are calibrated to the common sensor carrier coordinate system. However, sensor devices of the vehicle are calibrated to a common vehicle coordinate system of the vehicle. Data provided by the sensor carrier is thus not directly comparable to data provided by the sensor devices of the vehicle, since the data were acquired with respect to different coordinate systems. If the sensor carrier is positioned on the vehicle, an orientation of the sensor carrier coordinate system with respect to the vehicle coordinate system should therefore be determined. In particular, if the sensor carrier is to be used on a plurality of different vehicles, it is expedient that a method is provided by means of which the orientation of the sensor carrier coordinate system with respect to the vehicle coordinate system can be determined quickly and easily and can then be taken into account. Such a method also enables mechanically flexible arrangements, in particular alignments and positions, of the sensor carrier on the vehicle. For this reason, orientation parameters are determined within the scope of the method according to the invention, by means of which orientation parameters the orientation of the sensor carrier relative to the vehicle can be taken into account.It is preferably provided that at least one camera and one lidar sensor are arranged on each side in the longitudinal direction and in the transverse direction of the sensor carrier, so that a 360-degree detection of the environment of the sensor carrier is possible in each case by means of the cameras and the lidar sensors. The environment, which is detected by means of the respective sensor device, is spatially limited to a detection range of the respective sensor device. Conventional calibration methods can be used for calibrating the individual sensor devices on the sensor carrier coordinate system.At least three marking elements are arranged on the vehicle surface at least in a partial area of a vehicle surface of the vehicle. The partial region lies in a detection region of the at least one sensor device. In other words, a region of the vehicle surface on which the at least three marking elements are arranged can be detected with a sensor device of the sensor carrier. A marking element can be, for example, a circular or quadrangular two-dimensional marking on the vehicle surface. The marking element can be fastened to the vehicle surface, for example, by means of an adhesive film. It is provided that the at least three marking elements are arranged spatially separated from one another. In other words, the at least three marking elements are arranged distributed over the partial region of the vehicle surface which lies in the detection region of the sensor device. The at least three marking elements can each be configured differently or have a common design, that is to say a uniform size and / or configuration.Three-dimensional coordinate data are provided for each of the marking elements, i.e. here for the at least three marking elements, in a control device. The three-dimensional coordinate data describe three-dimensional coordinates of the marking element in a vehicle coordinate system of the motor vehicle. It is therefore assumed that before carrying out the method, the three-dimensional positions of the plurality of marking elements on the vehicle surface have already been measured and stored as three-dimensional coordinate data, so that they are provided to the control device.The control device may be a component of the sensor carrier or of the vehicle. If the control device is the component of the vehicle, it can be designed, for example, as an onboard computer. The control device can alternatively or additionally be designed as an external control device, which can be arranged externally of the sensor carrier and of the vehicle. The external control device may be positioned, for example, in a trunk of the vehicle while performing the described method. The control device can be a computing device, that is to say it can have at least one microprocessor and / or microcontroller and thus ultimately represent a computer.The vehicle coordinate system of the vehicle can be defined, for example, in the case of a motor vehicle as a vehicle as a coordinate system in which a longitudinal axis (x-axis) is oriented parallel to a longitudinal direction of the vehicle, a transverse axis (y-axis) is oriented parallel to a transverse direction of the vehicle and a vertical axis (z-axis) is oriented parallel to a vertical direction of the vehicle. The transverse axis may be a projection of a rear wheel axis of the vehicle onto a travel floor on which the motor vehicle is standing. The longitudinal axis preferably intersects the transverse axis centrally in the transverse direction of the motor vehicle. Alternatively, a differently defined vehicle coordinate system can be assumed. The individual sensor devices that the vehicle optionally has are each calibrated relative to the vehicle coordinate system, that is to say they indicate their respective sensor data relative to the vehicle coordinate system.The method according to the invention has the following steps: for each of the at least three marking elements, acquisition of marking element data describing the marking element. The marker element data are recorded by means of the at least one sensor device. The at least one sensor device is preferably one of the cameras of the sensor carrier. The at least three marking elements are thus described by means of the captured marking element data. These can be image data, i.e. camera data, for example. The image data can describe static and / or moving images. It is important here that the entire partial region which lies in the detection region of the sensor device is detected, so that each of the at least three marking elements is described by the marking element data.In addition, for each of the at least three marking elements, a determination of marking element coordinate data takes place. The marker coordinate data describes a two-dimensional coordinate of the center of the marker in the sensor carrier coordinate system. The marker coordinate data is obtained by applying a center point determination criterion to the acquired marker data by the control device. The center point determination criterion is an algorithm or a rule, on the basis of which the center point of the respective marking element determines and indicates the two-dimensional coordinate thereof relative to the sensor device, that is to say relative to the sensor carrier coordinate system. The midpoint determination criterion is stored in the control device for this purpose, that is to say is stored therein, for example. Finally, the respective marking elements are initially detected by means of the sensor device and the respective position of the center point of the respective marking element is subsequently provided in the sensor carrier coordinate system.In a further method step, at least one orientation parameter is determined by means of the control device. The at least one orientation parameter describes an orientation of the sensor carrier coordinate system with respect to the vehicle coordinate system. In other words, the at least one orientation parameter can describe the orientation of the sensor device in the vehicle coordinate system. On the basis of the orientation parameter, it is thus possible to contribute at least to a conversion from the sensor carrier coordinate system to the vehicle coordinate system and / or to carry out the conversion. Preferably, a set of orientation parameters is determined which describes a pose of the sensor device with six degrees of freedom (three position information and three direction information). The at least one orientation parameter is determined using a spatial backward step, wherein the spatial backward step is applied to the provided three-dimensional coordinate data and the determined marker element coordinate data. Here, the detected marker element coordinate data is adopted as measured values and the three-dimensional coordinate data is adopted as fixed invariable quantities. The spatial backward step is an orientation method known from photogrammetry, on which general photogrammetric imaging equations, as are known from camera calibration or from imaging equations, are based. For the spatial back section, the parameters of an internal orientation (intrinsic) of the camera used can be provided as known variables, which parameters were previously obtained via a camera calibration. The sensor carrier coordinates in the two-dimensional image can then be supplemented by a z-coordinate, i.e. by a focal length c, to form three-dimensional coordinates. It is provided that only already known, i.e. initial and thus approximately present, orientation parameters are improved, i.e. an optimization process is carried out within the scope of the spatial backward step in order to adapt currently provided orientation parameters to the current arrangement of the sensor carrier to the vehicle coordinate system.In the backward step, it may be the case that for a pose with six degrees of freedom, the angles cannot be directly determined, since no suitable approximations are known, whereas the nine parameters of a rotation matrix can be directly determined. Among the nine parameters of the rotation matrix, six conditions are taken into account which define a unique rotation matrix. Thus, the nine parameters are reduced to three degrees of freedom again. The background to this effort is that, in contrast to the direct use of the angles which are linked to one another via sine and cosine terms and whose determination is therefore nonlinear, a quasi-linear relationship is present and therefore a rough approximation suffices.The operation of the sensor carrier can then take place by means of the control device taking into account the determined at least one orientation parameter. From this point in time on, the sensor carrier therefore calculates how the sensor carrier coordinate system is oriented to the vehicle coordinate system according to the orientation parameter determined. This has the effect that, for example, sensor data of the individual sensor devices of the vehicle can be directly compared with sensor data of the at least one sensor device of the sensor carrier. If, for example, the sensor carrier now has a sensor device which can record the environment of the vehicle with a higher accuracy than the sensor devices of the vehicle, it is possible, for example, to check sensor data recorded by means of the sensor device of the vehicle on the basis of comparable but more accurate sensor data recorded by means of the sensor device of the sensor carrier. Ultimately, therefore, the sensor carrier can be operated reliably and accurately, since the currently valid at least one orientation parameter is taken into account. The respective orientation parameter is preferably determined for all coordinate axes of the sensor carrier coordinate system and taken into account when operating the sensor carrier.According to the invention, it is provided that the following steps are carried out for each marker element when applying the center point determination criterion: a center of gravity of the marker element is determined in a binary image representation of the marker element data. This can be done, for example, by using a centroid method, which is often referred to as a center-of-gravity method. In other words, the respective center point of the respective marking element, which is referred to here as the center point, is first roughly determined by the recognition according to the center point method in a binary image that describes the marking element. The binary image or binary image representation is a digital raster graphic, the pixels of which can assume only the two colors black and white. Each pixel can therefore assume either the value 0 for black or the value 1 for white. The value for black or white can alternatively be chosen inversely.A plurality of gray value profiles are then produced in a gray value representation of the marking element data, wherein the gray value profiles originate from the ascertained center of gravity. The gray scale representation into a gray scale image that describes the respective marking element and in which each pixel has a gray scale value that can lie between 1 and 256, for example. For example, star-shaped gray value profiles are created in the gray value image based on the positions of the determined centroids. The gray value profiles correspond to a cross section starting from the ascertained center of gravity through a point of the region spatially delimited by the marking element. Preferably, a gray value profile is created and evaluated for each point of the spatially delimiting region. This enables a subpixel-accurate determination of an edge point of the marking element for the two-dimensional image of the vehicle surface in each of the gray value profiles. The point of the spatially delimiting region is therefore the edge point of the marking element with respect to the vehicle surface. Marking elements with small marking elements compared to the vehicle surface in the detection area should be used, which are bonded planar to the vehicle surface in order to be able to determine the edge points as accurately as possible. It is thus finally determined where the gray value in the gray value profile changes between the marking element itself, which may be white and / or black, for example, and the vehicle surface surrounding the marking element, which may have a higher gray value, for example. At this location, the edge point determined for the respective gray value profile is located. In the case of the white and / or black marking element, the image of the marking element can be occupied by a natural noise.A cohesive edge boundary of the marking element is then ascertained by carrying out an edge shape adaptation over all ascertained edge points. If the marking element is a circular marking element, which can be referred to as a circular mark, for example, the edge boundary is depicted in an elliptical shape in the corresponding image. That is, ellipse matching, which may be referred to as ellipse fitting, is performed. Thus, ultimately the edge and thus the boundary line between the marking element and the vehicle surface surrounding the marking element is determined with subpixel accuracy, since the individual edge points that form this edge boundary have been determined beforehand with subpixel accuracy. In connection with the circular marking element, in particular the elliptical edge boundary is determined by carrying out the ellipse adaptation. Strictly speaking, the ellipse adaptation is determined only discretely with subpixel precision by the gray value profiles. After the adaptation, the edge points of the edge boundary are determined indirectly with subpixel accuracy, i.e. via the ellipse parameters in the case of the ellipse adaptation, for example, so that ultimately the edge of the marking element, which is elliptical, for example, is determined.An edge boundary center point of the edge boundary is then ascertained. Thus, in particular, the ellipse center point of the elliptical edge boundary is determined. In the case of the circular marking element, in addition to the ellipse center point, the two half axes of the elliptical image of the circular marking element can be determined and, if necessary, further taken into account. The edge boundary center corresponds to the center of the marking element, wherein the marking element is determined here to be subpixel accurate. The determined edge boundary center point is then provided as marking element coordinate data. The edge boundary center is therefore assumed as a measured value, based on which the spatial backward step is carried out. Finally, as new parameters, the ellipse center (edge boundary center) in an x- and y-direction in the image, the lengths of both half axes and a direction of the ellipse are determined. By means of the described procedure, the edge boundary center point, i.e. the marking element coordinate data, can be provided particularly reliably and accurately.The invention also includes embodiments which provide additional advantages.One embodiment of the invention provides that the method is carried out for at least four marking elements. In particular, it can be provided that the method is carried out for at least seven marking elements. The use of only three marking elements and their marking element data for ascertaining the marking element coordinate data and for later ascertaining the orientation parameter has the disadvantage that in the case of an unfavorable constellation of the marking elements in space, that is to say in the partial region of the vehicle surface, the calculation becomes inaccurate since three points are always located on one plane. However, overdetermination would be achieved at at least four points, i.e. at least four marker elements, so that there is a redundancy of greater than or equal to 1. This makes it possible to calculate standard deviations for the parameter variables to be determined, that is to say, for example, for the determination of the marker element coordinate data, so that a precision of the method can be estimated. However, if only three marking elements are taken into account, for example, it is theoretically possible to achieve a clearly closed solution, but no overdetermination is achieved in this case, which means that in this case the position indication which can be specified for the specific data is to be evaluated as inaccurate, in particular as not sufficiently accurate. In the case of exactly three marking elements each having two image coordinate values as two-dimensional coordinates of the center point, a total of six measured values are present in the space for determining a pose having six degrees of freedom and thus six unknowns. The redundancy, i.e. the overdetermination, is therefore zero in this case, since the redundancy is calculated from the difference between the number of measured values and the number of degrees of freedom. For this reason, it is particularly advantageous to take at least four marking elements into account. It has also been found that data are obtained from at least seven marking elements, the error ranges of which are already in an acceptable order of magnitude, so that at least seven marking elements are particularly preferably selected.Furthermore, it is provided in one embodiment that at least one piece of calibration information relating to the at least one sensor device is taken into account when applying the spatial backward step. The calibration information can relate in particular to an internal orientation of the at least one sensor device. The calibration information may be provided in the form of calibration information data. The calibration information relates to the sensor device of the sensor carrier that has determined the marking element data. The inner orientation can alternatively be referred to as an intrinsic nature of the sensor device. The inner orientation can describe, for example, a pose, i.e. a position and orientation, of components of the sensor device relative to one another. The internal orientation thus describes the sensor conditions within the sensor device, such as, for example, a measurement origin and / or the imaging conditions of the sensor device. For example, a deviation of an objective of the sensor device from an optical axis of the sensor device to its sensor element (CCD array, CCD for charge-coupled device) can be included in the calibration information. Alternatively or additionally thereto, distortion parameters, which relate in particular to a detection edge region of the sensor device, can be described by the calibration information. Alternatively or additionally, a camera constant may be described by the calibration information. The camera constant can describe a mathematically currently set focal length of the camera as a sensor device, which corresponds to a z component of the internal orientation. As a result, the spatial backward step can ultimately be carried out particularly reliably.According to an additional embodiment, it is provided that the three-dimensional coordinate data are determined by means of a photogrammetric measurement process. The photogrammetric measurement process is carried out by means of a photogrammetric measurement camera and an evaluation device for evaluating measurement data of the measurement camera. The photogrammetric measurement camera is, for example, a camera, the measurement data of which are suitable for digital photogrammetric evaluation, wherein the measurement camera has, for this purpose, an objective with a quality which is better than the quality of a camera which is typically used for a sensor device of the vehicle. The measurement camera can have a fixed focus objective. For carrying out the photogrammetric measurement process, it can be provided that a plurality of marking elements are likewise arranged outside the partial region of the vehicle surface, such that data of, for example, a plurality of different surfaces of the vehicle are present. In the context of the measurement process, an environment of the vehicle may also be provided with marking elements, so that the environment of the vehicle is also taken into account. Finally, three-dimensional coordinates can be determined for various points on the vehicle surface of the vehicle and in the environment of the vehicle.In the photogrammetric measurement process, in particular, at least one scale in space can be used, on the basis of which actual distances between individual marking elements can be determined. Preferably, a plurality of scales are taken into account in a spatially distributed manner during the photogrammetric measurement process. For this purpose, the scale can be arranged in the environment of the vehicle, for example on a travel floor on which the vehicle is arranged. The evaluation device can be a component of the control device. Alternatively or additionally, the evaluation device can be included in the measurement camera. Alternatively or additionally, the evaluation device can be designed as an independent device, for example as an independent computer. The independent device is a device external to the vehicle and external to the sensor carrier.An additional embodiment provides that for the photogrammetric measurement process at least one marking element is arranged at a location of the vehicle, through which at least one axis of the vehicle coordinate system runs, wherein a location can be meant here, to which an axis of the vehicle coordinate system runs at least parallel. The location is in particular a wheel hub of a wheel of the vehicle, an adapter on the wheel hub and / or an outer edge of a rim of the wheel. The wheel is in particular a rear wheel of the vehicle. The adapter can be, for example, a cap which can be fastened to the wheel hub of the wheel and on which at least one marking element is arranged. This adapter can alternatively be arranged as a hand chuck directly on the wheel hub. It is relevant here that a sufficient linking and connection of the individual marking elements to this adapter or the marking element is provided at the location of the wheel hub and / or the edge of the rim. In addition, marking elements should be applied to the ground in order to be able to perform the projection of the vehicle coordinate system onto the travel ground. Finally, typical methods for determining a vehicle geometry based on, for example, photogrammetric markers can be used. As a result, the three-dimensional coordinates can be provided particularly reliably, in particular for a motor vehicle as a vehicle.A further aspect of the invention relates to a sensor arrangement comprising a vehicle, a control device and a sensor carrier arranged on the vehicle. The sensor carrier has at least one sensor device calibrated to a sensor carrier coordinate system of the sensor carrier, wherein at least three marking elements are arranged on the vehicle surface at least in a partial region of a vehicle surface of the vehicle, which lies in a detection region of the at least one sensor device. Three-dimensional coordinate data describing a three-dimensional coordinate of the marker in a vehicle coordinate system of the vehicle is provided to the control device for each of the marker. At least one sensor device is designed to record, for each of the at least three marking elements, marking element data describing the marking element.The control device is configured to also determine the captured marker element data for each of the at least three marker elements marker element coordinates that describe a two-dimensional coordinate of a center point of the marker element in the sensor carrier coordinate system by applying a center point determination criterion. The control device is also configured to ascertain at least one orientation parameter, which describes an orientation of the sensor carrier coordinate system with respect to the vehicle coordinate system, by applying a spatial step-back to the provided three-dimensional coordinate data and the ascertained marking element coordinate data. The control device is also designed to operate the sensor carrier taking into account the determined at least one orientation parameter. The advantageous embodiments described in connection with the method according to the invention and the advantages thereof apply, if applicable, in each case individually and in combination with one another also to the sensor arrangement according to the invention. Analogously to this, the embodiments of the sensor arrangement described below, each individually and in combination with one another, also apply to the method described above.An advantageous embodiment of the sensor arrangement provides that the respective marking element is designed as a circular marker, encoded marker and / or magnetic marker. The circular marker is, for example, a circular element which is held in white, for example. The circular marker may alternatively be referred to as a circular marker. The encoded marker comprises, for example, a circular central element with at least one additional geometric element in the vicinity of the central element. The marking element is encoded by a shape, number and / or arrangement of the at least one geometric element, i.e. it can be identified unambiguously. The individual coded markers thus each have a distinguishing feature which distinguishes them from one another. The magnetic marker is, for example, a unambiguously signaled marker which can be recognized as being immediately, for example on the basis of its configuration and arrangement. The marking element is suitable for marking particularly relevant locations of the vehicle surface, such as the wheel hub, the adapter on the wheel hub and / or the outer edge of the rim of the wheel. The various markers may be combined with each other. This ultimately achieves the result that individual locations on the vehicle can be identified particularly reliably by means of the marking element and can be detected, for example, in the marking element data.A further embodiment provides that the sensor carrier is arranged on a roof of the vehicle. For this purpose, the sensor carrier can have, for example, a holding unit, by means of which it can be fastened, for example, to a roof carrier on the roof of the vehicle. The at least one sensor device is a camera directed rearward relative to a longitudinal direction of the vehicle. The vehicle surface portion is a roof portion disposed rearward in the longitudinal direction. The portion should comprise at least the rear roof portion of the roof of the vehicle. This makes it possible that a plurality of marking elements can actually be positioned on the vehicle surface in the detection region of the sensor device at a spatial distance from one another, since the typically large-area rear roof region is provided for this purpose. The rear-facing camera is calibrated to the sensor carrier coordinate system and preferably one of a plurality of cameras of the sensor carrier. In other words, the sensor carrier coordinate system may be defined via the rear-facing camera. Furthermore, marking elements on the roof region are not visually perceived immediately by a viewer of the vehicle and they can therefore be integrated particularly non-significantly into a vehicle actually used in road traffic, regardless of whether said vehicle is driving with or without a sensor carrier.The individual marking elements are preferably fastened directly on the vehicle surface of the roof of the vehicle by means of an adhesive film. They can be unreruptably removable from the vehicle surface, for example by being able to be pulled off the vehicle surface again.According to a further embodiment, it is provided that the subregion additionally comprises an outer tailgate wall of a tailgate of the vehicle. The tailgate is in an open position at least for ascertaining the at least one orientation parameter. In the open position, the tailgate is removed from the rest of the motor vehicle, thus allowing access to a trunk, for example, which is delimited from the outside world by the tailgate in a closed position of the tailgate. As an alternative to the outer trunk lid wall, the partial area can additionally comprise an outer trunk lid if no trunk lid but only a trunk with an outer trunk lid wall should be provided in the vehicle.In principle, it is sufficient that the marking elements are arranged only on the roof region. As a result, for example, a spatial precision of up to 2 centimeters with respect to an object at a distance of 50 meters from the vehicle can be achieved. In the case that the marking elements are included on the tailgate outer wall, a spatial precision of less than 5 millimeters with respect to an object at a distance of 50 meters can even be achieved. However, the opening level of the tailgate of the vehicle may then have an effect on the measured values, wherein the opening level may be dependent on an aging process of shock absorbers or an automatic opening device of the tailgate. In order to be able to achieve reliable results over the long term, it may therefore be expedient to dispense with the incorporation of the marking elements on the tailgate outer wall. If the partial region extends over the outer wall of the tailgate, it is assumed that at least one of the marking elements is located on the latter. However, this then presupposes that the determination of the orientation parameter can only be carried out when the tailgate is open and thus preferably when the vehicle is at a standstill. If only the partial area in the roof area is taken into account, the orientation parameter can be determined, for example, during a trip of the motor vehicle.The invention also includes the control device. The control device has a processor device which is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (field programmable gate array) and / or at least one DSP (digital signal processor). Furthermore, the processor device can have program code which is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device.The invention also encompasses the combinations of the described embodiments.Exemplary embodiments of the invention are described below. The following shows: FIG. 1 shows a schematic illustration of a vehicle having a sensor carrier; and FIG. 2 shows a schematic illustration of a signal flow graph of a method for operating a sensor carrier arranged on a vehicle.The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that are to be considered independently of one another and that develop the invention in each case also independently of one another and are therefore also to be considered as part of the invention individually or in a combination other than the combination shown. Furthermore, the described embodiments can also be supplemented by further features of the invention that have already been described.In the figures, elements having the same function are each provided with the same reference numerals.FIG. 1 outlines a vehicle 1, which is here exemplarily designed as a motor vehicle. Alternatively, the vehicle 1 can be a watercraft, an aircraft and / or a robot. The robot is in particular a mobile robot, that is to say a robot which can move translationally in space.A sensor carrier 2 is arranged on the vehicle 1. The sensor carrier 2 has a plurality of sensor devices 3. As sensor device 3, it has, for example, a plurality of cameras 4, a plurality of lidar sensors 5 and a position determination unit 6. The sensor carrier can alternatively or additionally have an inertial measurement unit. The positioning unit 6 may be based on a global navigation satellite system (GNSS).The sensor carrier 2 has a control device 7. The control device 7 can alternatively or additionally be a component of the vehicle 1 and / or a component external to the sensor and the vehicle. The control device 7 is, for example, a computer, i.e., a computing device.The sensor carrier 2 is positioned here on a roof carrier 8 of the vehicle 1, wherein it is fastened on the roof carrier 8 by means of a plurality of holding units 9 of the sensor carrier 2.The camera 4 of the sensor carrier 2 which is particularly relevant for the method described below is a camera 4 which is directed rearwards here in a longitudinal direction (x direction) of the vehicle 1 and for which a detection region 10 is outlined by means of dashed lines. In the detection region 10, a partial region 11 lies within a vehicle surface 12 of the vehicle 1. the vehicle surface 12 is here a rear region of a roof 13 of the vehicle 1, on which the roof carrier 8 is arranged. Furthermore, a tailgate 14 with a tailgate outer wall 15 lies in the detection region 10.A plurality of marking elements 16 are arranged both on the rear roof region of the roof 13 and on the tailgate outer wall 15, that is to say distributed on the vehicle surface 12 in the subregion 11. Here, by way of example, seven marking elements 16 are outlined in the sub-region 11. In principle, it is assumed that at least three, preferably at least four and in particular at least seven marking elements 16 are arranged in the detection region 10 and thus in the partial region 11.A side wall 17 of the vehicle 1 is also shown, on which further marking elements 16 are arranged. Individual ones of the marking elements 16 have a coding 18, that is to say they are individually marked, so that these marking elements 16 can be unambiguously identified.In FIG. 1, a wheel 19, which is here a rear wheel of the vehicle 1, is furthermore emphasized. On this, at least one marking element 16 is likewise positioned on a wheel hub 20 or an adapter 21 on the wheel hub 20 and on an outer edge 23 of a rim 22 of the wheel 19. The wheel 19 also includes a tire 24 disposed about the rim 22. The marking elements 16 on the wheel 19 are arranged there for a preliminary method, which is referred to below as photogrametric measurement process. The photogrammetric measurement process is carried out in order to be able to determine the exact position of the individual marking elements 16 with respect to a vehicle coordinate system 25 of the vehicle 1. A z direction of the vehicle coordinate system 25 corresponds to a height direction of the vehicle 1. a y direction, that is, a transverse direction, is parallel to the rear wheel axis of the rear wheel but projected onto a travel floor 26 on which the vehicle 1 is positioned. An x direction, that is, a longitudinal direction intersects the y axis centrally in the lateral direction of the vehicle 1.FIG. 1 also outlines a sensor carrier coordinate system 27 of the sensor carrier 2, to which all sensor devices 3 of the sensor carrier 2 are calibrated.The vehicle 1 further includes a vehicle camera 28 that is a sensor device of the vehicle 1. Additionally or alternatively, the vehicle 1 can have a plurality of further sensor devices, for example lidar sensors, further cameras, radar devices, infrared sensors and / or other measuring devices. The vehicle camera 28 is calibrated relative to the vehicle coordinate system 25.FIG. 2 outlines a method for operating a sensor carrier 2 arranged on a vehicle 1. It is assumed that the sensor carrier 2 has the at least one sensor device 3, wherein the rear-facing camera 4 is assumed below as the sensor device 3. In a method step S 1, a first part of the photogrammetric measurement process can first be carried out, in which the vehicle 1 with the numerous marking elements 16 is detected by means of a photogrammetric measurement camera 30. In a further method step S 2, which is likewise part of the photogrammetric measurement process, the measurement data 31 of the measurement camera 30 can be evaluated by means of an evaluation device 32 in order to determine three-dimensional coordinate data 33 for each of the marking elements 16. The determined three-dimensional coordinate data 33 are then provided to the control device 7 for the sensor carrier 2. For this purpose, they can be stored, for example, in a storage device of the control device 7. The three-dimensional coordinate data 33 describe three-dimensional coordinates of the respective marking element 16 in the vehicle coordinate system 25 and are present for the at least three and in particular four marking elements 16 in the subregion 11.In a method step S 3, for each of the at least three marking elements 16, the sensor device 3 is used to record marking element data 34 describing the marking element 16. The marking element data 34 are, for example, static and / or moving image data of the camera 4. a method step S 4 is then carried out, during the scope of which a center point determination criterion 35 is applied to the captured marking element data 34 by means of the control device 7, in order to thereby determine marking element coordinate data 36. The marking element coordinate data 36 describe a two-dimensional coordinate of a center point of the respective marking element 16. In detail, a center of gravity 37 of the respective marking element 16 is first determined in a binary image representation 38 of the marking element data 34. In this case, a centroid determination method, which is often referred to as a center-of-gravity method, can be used. A plurality of gray value profiles 39 are then produced in a gray value representation 40 of the marking element data 34, wherein the plurality of gray value profiles 39 each originate from the ascertained center of gravity 37. The individual gray value profiles 39 are preferably arranged in a star shape around the center of gravity 37. An edge point 41 can then be determined in the gray value profiles 39 with subpixel accuracy, which edge point represents a transition from the marking element 16 to the vehicle surface 12 in the two-dimensional image of the camera for each of the gray value profiles 39. A contiguous edge boundary 42 of the marking element 16 is then determined by carrying out an edge shape adaptation over all the determined edge points 41. in the case of a circular marking element 16, which can also be referred to as a circular marker, an elliptical edge boundary 42 is determined by carrying out an ellipse adaptation. An edge boundary center 43 of the edge boundary 42 is then determined, which is in particular an ellipse center. The determined edge boundary center point 43 or the determined ellipse center point is provided as marking element coordinate data 36. The individual method steps of method step S 4 described so far are carried out by means of the control device 7.As an additional method step, it can be provided that a correction algorithm for correcting a distortion is applied to the provided marker element coordinate data 36. The correction algorithm can have a rule, in the case of which a radially symmetrical and / or a tangential distortion, an affinity and / or a shear can be corrected when applied to the marking element coordinate data 36. The correction algorithm can be based on the model of Brown and Conrady for the objective distortion of an objective of a camera 4. By applying the correction algorithm to the marker coordinate data 36 in the x and y directions of the image described by the marker data 34, corrected data for the x and y directions can be obtained, respectively. For this purpose, for example, the marking element coordinate data 36 in the x direction can be reduced by a main point position in the x direction and the sum of all distortions in the x direction. Analogously, the corrected data for the y-direction can be determined. The main point position describes a point of penetration of an optical axis of the objective of the camera 4 through a camera lens system of the camera 4 onto its image sensor, wherein the actual optical axis deviates from an ideal central axis of the lens system of the camera 4 and thus the point of penetration on the image sensor also deviates from an image center point of the image captured by means of the camera 4. In order to be able to provide corrected three-dimensional coordinates in the sensor carrier coordinate system 27 for the respective edge boundary center point 43, the corrected data in the x- and y-direction and a predefined camera constant of the camera 4 in the z-direction are assumed.In a method step S 5, at least one orientation parameter 46 is determined. this is carried out by applying a spatial backward step 44 to the provided three-dimensional coordinate data 33 and the determined marker element coordinate data 36. The orientation parameter 46 describes an orientation of the sensor carrier coordinate system 27 to the vehicle coordinate system 25. The orientation parameter 46 is determined by means of the control device 7. Furthermore, at least one piece of calibration information 45 can be taken into account, which relates to the at least one sensor device 3, in particular to an inner orientation of the at least one sensor device 3. The calibration information 45 is taken into account when applying the spatial backward step 44.In a method step S 6, the sensor carrier 2 can then be operated by means of the control device 7 taking into account the determined at least one orientation parameter 46, that is to say the sensor data provided by the sensor carrier 2 take into account the orientation of the sensor carrier coordinate system 27 present according to the orientation parameter 46 with respect to the vehicle coordinate system 25 and can thus be specified, for example, in the vehicle coordinate system 25 and can thereby be directly compared or at least related to data of the sensor device of the vehicle 1, such as the vehicle camera 28.The method described is preferably used for at least four marking elements 16, in particular for at least seven marking elements 16. It is sufficient that only the rear roof region of the roof 13 lies in the partial region 11 and is taken into account. In order to achieve higher precision, that is to say a smaller measurement error for the orientation parameters 46, the marking elements 16 on the tailgate outer wall 15 can also be taken into account if the tailgate 14 is arranged in the open position outlined in FIG. 1.Overall, the examples show a realignment of a sensor system within a vehicle coordinate system 25. For the vehicle 1 to be equipped with the sensor carrier 2, special permanent photogrammetric markers, referred to here as marking elements 16, are glued onto the rear vehicle roof, which can be seen by a rear-looking camera 4 as sensor device 3 of the sensor carrier 2. These marking elements 16 are measured photogrammetrically only once for each vehicle 1 in the vehicle coordinate system 25 via special adapters 21 on the wheel hub 20 of the rear axle of the vehicle 1 in a single process. Since these points are now individually known for each vehicle 1 in the respective vehicle coordinate system 25, the orientation in the vehicle coordinate system 25 can be determined by means of a special photogrammetric process for that camera 4 on the sensor carrier 2 which can record the points, that is to say the marking elements 16 there. The determination of a camera pose of the camera 4 can be determined accurately with respect to a position of the camera 4 with respect to the vehicle coordinate system 25 to a few hundredths of a millimeter, for example to 1 / 10 millimeter to 1 / 100 millimeter, and with respect to an orientation of the camera 4 with respect to the vehicle coordinate system 25 to a few milligrades. Since the reference to the other sensor devices 3 of the sensor carrier 2 is known, since it is predetermined relative to the sensor carrier coordinate system 27, all further sensor devices 3 of the sensor carrier 2 can thus also be transferred via transformation rules into the vehicle coordinate system 25, wherein these transformation rules are known by the orientation parameters 46 determined. The method also represents a general possibility of orienting any desired sensor carrier 2 or a sensor system, which is configured at least with a camera 4, in the vehicle coordinate system 25 or any desired coordinate system with an unpreceived precision. The method can also be applied to the calibration of sensor devices for mass-produced vehicles.In summary, the method is divided into the following sequences: signaling by means of photogrametric markers on the vehicle 1 (method steps S 1 and S 2); one-time measurement of roof and tailgate measurement points using the measurement camera 30 in the vehicle coordinate system 25 (method steps S 1 and S 2); detection of the marking elements 16 using the rear-oriented overview camera, that is to say the camera 4 as a sensor device 3 having the detection range 10 drawn in FIG. 1 (method step S 3); and orientation of the sensor device 3 in the vehicle coordinate system 25 by means of a spatial rearward step (method steps S 4 and S 5).

Claims

Method for operating a sensor carrier (2) arranged on a vehicle (1), which has at least one sensor device (3) calibrated to a sensor carrier coordinate system (27) of the sensor carrier (2), wherein at least three marking elements (16) are arranged on the vehicle surface (12) at least in a partial region (11) of a vehicle surface (12) of the vehicle (1) which lies in a detection region (10) of the at least one sensor device (3), and three-dimensional coordinate data (33) are provided for each of the marking elements (16) of a control device (7) which describe a three-dimensional coordinate of the marking element (16) in a vehicle coordinate system (25) of the vehicle (1), wherein the method has the following steps: - for each of the at least three marking elements (16): • detecting (S3) marking element data (34) which describe the marking element (16), by means of the at least one sensor device (3); • ascertaining (S4) marking element coordinate data (36), which describe a two-dimensional coordinate of a center point of the marking element (16) in the sensor carrier coordinate system (27), by applying a center point determination criterion (35) to the captured marking element data (34) by means of the control device (7); - ascertaining (S5) at least one orientation parameter (46), which describes an orientation of the sensor carrier coordinate system (27) with respect to the vehicle coordinate system (25), by applying a spatial backward step (44) to the provided three-dimensional coordinate data (33) and the ascertained marking element coordinate data (36) by means of the control device (7); and - operating (S6) the sensor carrier (2) by means of the control device (7) taking into account the ascertained at least one orientation parameter (46); wherein the following steps are carried out for each marking element (16) when applying the center point determination criterion (35): - determining a center of gravity (37) of the marking element (16) in a binary image representation (38) of the marking element data (34); - creating a plurality of gray value profiles (39) in a gray value representation (40) of the marking element data (34), which originate from the determined center of gravity (37); - determining an edge point (41) of the marking element (16) to the vehicle surface (12) in each of the gray value profiles (39) in a subpixel-accurate manner; - determining a contiguous edge boundary (42) of the marking element (16), in particular an elliptical edge boundary (42), by carrying out an edge shape adaptation, in particular an ellipse adaptation, over all the determined edge points (41); and - determining an edge boundary center point (43) of the edge boundary (42), in particular an ellipse center point; and - providing the determined edge boundary center point (43) as marking element coordinate data (36).Method according to claim 1, wherein the method is carried out for at least four marking elements (16), in particular for at least seven marking elements (16).Method according to one of the preceding claims, wherein, when applying the spatial reverse step (44), at least one piece of calibration information (45) relating to the at least one sensor device (3), in particular an inner orientation of the at least one sensor device (3), is taken into account.Method according to one of the preceding claims, wherein the three-dimensional coordinate data (33) are determined by means of a photogenetic measurement process (S1, S2) which was carried out by means of a photogenetic measurement camera (30) and an evaluation device (32) for evaluating measurement data (31) of the measurement camera (30).Method according to claim 5, wherein for the photogrametic measurement process at least one marking element (16) is arranged at a location of the vehicle (1), through which at least one axis of the vehicle coordinate system (25) runs, wherein the location is in particular a wheel hub (20) of a wheel (19) of the vehicle (1), an adapter (21) at the wheel hub (20) and / or an outer edge (23) of a rim (22) of the wheel (19), wherein the wheel (19) is in particular a rear wheel of the vehicle (1).Sensor arrangement comprising a vehicle (1), a control device (7) and a sensor carrier (2) arranged on the vehicle (1), which sensor carrier has at least one sensor device (3) calibrated to a sensor carrier coordinate system (27) of the sensor carrier (2), wherein at least three marking elements (16) are arranged on the vehicle surface (12) at least in a partial region (11) of a vehicle surface (12) of the vehicle (1) which is situated in a detection region (10) of the at least one sensor device (3), and three-dimensional coordinate data (33) are provided to the control device (7) for each of the marking elements (16), which three-dimensional coordinate data describe a three-dimensional coordinate of the marking element (16) in a vehicle coordinate system (25) of the vehicle (1), wherein - the at least one sensor device (3) is designed to provide marking element data (34) for each of the at least three marking elements (16), the control device (7) is configured to • determine, for each of the at least three marking elements (16), marking element coordinate data (36) describing a two-dimensional coordinate of a center point of the marking element (16) in the sensor carrier coordinate system (27) by applying a center point determination criterion (35) to the detected marking element data (34); • determine at least one orientation parameter (46) describing an orientation of the sensor carrier coordinate system (27) to the vehicle coordinate system (25) by applying a spatial backward step (44) to the provided three-dimensional coordinate data (33) and the determined marking element coordinate data (36); and • operate the sensor carrier (2) taking into account the determined at least one orientation parameter (46); wherein the control device (7) is configured to perform the following steps for each marking element (16) when applying the center point determination criterion (35): - determining a center of gravity (37) of the marking element (16) in a binary image representation (38) of the marking element data (34); - creating a plurality of gray value profiles (39) in a gray value representation (40) of the marking element data (34), which originate from the determined center of gravity (37); - determining an edge point (41) of the marking element (16) to the vehicle surface (12) in each of the gray value profiles (39) in a subpixel-accurate manner; - determining a contiguous edge boundary (42) of the marking element (16), in particular an elliptical edge boundary (42), by performing an edge shape adaptation, in particular an ellipse adaptation, over all the determined edge points (41); and - determining an edge boundary center point (43) of the edge boundary (42), in particular an ellipse center point; and - providing the determined edge boundary center point (43) as marking element coordinate data (36).Sensor arrangement according to Claim 6, wherein the respective marking element (16) is designed as a circular marker, coded marker and / or magnetic marker.Sensor arrangement according to either of Claims 6 and 7, wherein the sensor carrier (2) is arranged on a roof of the vehicle (1), the at least one sensor device (3) is a camera (4) which is directed rearwards relative to a longitudinal direction of the vehicle (1), and the partial region (11) of the vehicle surface (12) is at least one roof region which is arranged rearwards in the longitudinal direction.Sensor arrangement according to Claim 8, wherein the sub-region (11) additionally comprises a tailgate outer wall (15) of a tailgate (14) of the vehicle (1), wherein the tailgate (14) is in an open position at least for ascertaining the at least one orientation parameter (46).

Images