Method for calibrating a camera system and camera system with processor unit

Abstract

The invention relates to a method for calibrating cameras of a camera system that are stationary relative to one another, wherein the camera system is characterized by a speed (313) that is not equal to zero and a change in the direction of the speed (313). For at least two of the cameras, pairs of image points are determined from a first image of the camera at a first point in time and a second image of the camera at a second point in time, which pairs each describe the same object point that is stationary in an environment in the first image or in the second image. The method comprises determining the movement of the at least two cameras on the basis of the pairs of image points using epipolar geometry, and determining the relative orientation of the cameras to one another using the determined movements of the cameras.

Description

The invention relates to a method for calibrating cameras of a camera system that are stationary relative to one another, to a camera system having a processor unit for calibrating the cameras, and to a vehicle having the camera system and a processor unit.The camera system comprises a plurality of cameras, which can be fastened on a common body and whose positions and orientations are fixed with respect to one another, in particular during the calibration of the camera system. These cameras thus have the same positions and orientations relative to one another at any point in time, in particular during the calibration of the camera system. For example, a vehicle can comprise the camera system. In particular in applications in which the vehicle is the common body and at least the image area of an image of a camera is related to the position of the vehicle in 3D space, a knowledge about the relative orientation of the cameras with respect to one another and / or with respect to the vehicle is important. This can be the case, for example, in the case of image-based object recognition and / or image-based 3D object reconstruction, in particular in the context of a driver assistance system. Additionally or alternatively, in applications where the images of multiple cameras are needed, knowing the relative positions and orientations of the cameras to each other may be important. An example of this is a composition of the images of a plurality of cameras, for example in order to be able to cover a larger image region.A determination of the relative positions and / or orientations of the cameras, also referred to below as "calibration of the camera system", is also described in the publication T. Ruland, H. Loose, T. Pajdla and L. Krüger, "Hand-eye autocalibration of camera position on vehicles," 13th International IEEE Conference on Intelligent Transportation Systems, Funchal, Portugal, 2010, pp. 367-372, doi: 10.1109 / ITSC.2010.5625279. This discloses an algorithm for estimating the X and Z position of a camera whose image images the environment from the bird's eye view. However, a disadvantage of this method is that a "flat world" is assumed around the vehicle, as a result of which the environment in which the camera system can be calibrated is greatly restricted.WO 2020 / 006 378 A1 discloses a method for calibrating a camera system. In this case, points describing the same object point in images of two cameras with at least partially overlapping capture regions are compared with one another, such that a correction for a relative alignment of the cameras with respect to one another can be determined by means of epipolar geometry.DE 10 2017 123 228 A1 describes a method in which it is determined from an image sequence of an imaging sensor device of the motor vehicle whether an object in the environment of the motor vehicle is static or dynamic.The invention is based on the object of providing a method for calibrating the cameras of the camera system, in which no further infrastructure for calibration and / or additional infrastructure and / or predefinable calibration environment and / or no calibration object and / or no device which is not part of the camera system or of the processor unit for calibration has to be used.The object is achieved by the subject matters of the independent claims. Advantageous further developments of the invention are described by the dependent patent claims, the following description and the figures.The invention is based on the finding that the relative orientations, i.e. the relative angular positions of the alignments of the cameras with respect to one another, can be determined by knowing movements (translation and / or rotation movements) of the cameras, between which the relative positions are known.As mentioned, the invention relates to a method for calibrating cameras of a camera system that are stationary relative to one another. The camera system moves relative to an environment or environment, wherein a translation component of the speed describing the movement of the camera system has a value that is not zero and a direction of the movement changes, i.e. for example a curved movement or generally a curved movement path is present for example on account of a changing gradient. Additionally or alternatively, the movement of the camera system is two rotations of the camera system about two axes which are not parallel to one another and are preferably perpendicular to one another. Additionally or alternatively, the movement of the camera system is two translations along two axes which are not parallel to one another and are preferably perpendicular to one another. Additionally or alternatively, the movement is translation along a first axis and rotation along a second axis that is not parallel to the first axis. The axes are in particular virtual axes and / or axes of a predeterminable coordinate system and / or direction information. The movement of the cameras of the camera system corresponds to the movement of the camera system, that is to say the cameras execute the same at least one rotation about the same at least one axis and / or the same translation along the same at least one axis as the camera system. The camera system moves relative to an environment, wherein the cameras, in particular each camera of the camera system, capture parts of the environment in the image area. The movement of the camera system can be a speed which is constant in terms of amount or changes in terms of amount and which differs from the speed of the environment. For example, the camera system may be integrated into a vehicle that travels on a road at a non-zero speed, simultaneously traveling along a changing slope and / or curve.Since all cameras are immovable with respect to one another, their movements (translation and / or rotation movements) are related. By knowing the movements of the individual cameras with respect to the environment, the relative orientation of the cameras with respect to one another can be determined with a translation movement whose direction does not change, except for one degree of freedom. The lack of freedom, the relative orientations of the cameras with respect to one another about the axis or line which points in the direction of movement can be determined by a translation movement of the camera system along and / or a rotation movement about an axis which is not parallel to the direction of movement. The rotation can be effected by changing the direction of movement.For at least two of the cameras, one pixel each is determined from a first image of the camera at a first point in time and a second image of the same camera at a second point in time, so that a respective pixel pair results. The first time and / or the second time of a camera can differ from the first time or second time of another camera with respect to a predefinable clock. This means that each of the at least two cameras makes an image at two different points in time (a first point in time and a second point in time) which is designated as a first image or a second image in each case corresponding to its point in time. Each pair of pixels comprises a first pixel and a second pixel; the first pixel has a first image position in the first image and the second pixel has a second image position in the second image. The first and second images are captured by the same camera.The pixels of the pixel pair each describe the same object point which is a spatially invariable point in the environment, that is to say a point which does not move with respect to the environment and which is mapped both on the first image and on the second image. The position at which the object point is respectively mapped on the first image and on the second image is respectively referred to as the first image position and the second image position. Pixels of different pixel pairs each describe different object points, i.e. different positions in the environment. For example, image point pairs can be identified by means of at least one of the two following methods, which are known from the prior art and which are used, for example, for the so-called stitching or the optical flow of two images.For the at least two cameras of the camera system, the movement (translation and / or rotation movement) of the respective camera is determined in each case from the image point pairs by means of epipolar geometry. Epipolar geometry describes possible positions of the second image position as a function of the first image position and the relative positions and orientations of the first and the second camera with respect to one another. In particular, the relative position of the first camera to the second camera comprises only relative distances, but no absolute distances. This means that no scale (scale) is known. For determining the movement of the cameras, it can be used that a pair of image points respectively describes the same object point recorded by the same camera at two different times and consequently at two different recording locations linked by the movement of the camera. The image point pairs are each also referred to as temporal correspondences (temporal correspondences).The prior art discloses, for example, a method in which the relative orientations and positions between two cameras can be determined by means of at least five image point pairs and the epipolar geometry at different locations at which the image is recorded at the same time from two different perspectives. This can be used within the scope of the method according to the invention in that the image point pairs of two different cameras at two different locations of the method known from the prior art are replaced by image point pairs of the same camera at two different locations, i.e. at two different times. From the relative positions and orientations of the camera determined from the epipolar geometry at the two different times, the movement (translation and rotation movement) of the camera can be deduced. Correspondingly, it is also possible to infer the relative orientation of the camera with respect to the direction of movement and / or the axis of rotation.Since the cameras are immovable relative to one another, the movements of the cameras of the camera system are linked to one another. Consequently, from the knowledge of the movements of the individual cameras and the orientations of the cameras with respect to their respective direction of movement, in particular with knowledge of the relative positions of the cameras with respect to one another, it is possible to infer the relative orientations of the cameras with respect to one another. In other words, a relative orientation of the at least two cameras with respect to one another is determined from the determined movements of the at least two cameras. This means that by linking the translation and rotation movement of the cameras determined from the epipolar geometry with the movement of the camera system, the relative orientation of the cameras with respect to one another can be determined. For this purpose, a reference camera can be selected from the cameras of the camera system, wherein the selection of the reference camera can change and the relative orientations of the cameras with respect to the orientation of the reference camera are determined. For example, the reference camera is a camera in which the orientation, in particular the orientation with respect to its common carrier body, for example a vehicle, is known.It can be provided that the camera system is calibrated at predeterminable intervals. In particular, if it is determined that the relative orientation of more than half of the cameras of the camera system, preferably at least 90% of the cameras of the camera system, in particular of all cameras except for the reference camera of the camera system, has changed with respect to the reference camera compared to the last calibration, a change of the reference camera can be provided, that is to say another camera of the camera system is selected as the reference camera. This can be useful because the probability that the orientation of the reference camera has changed with respect to a predefinable coordinate system is then greater than the probability that the orientation of all cameras whose orientation has changed with respect to the reference camera has also changed with respect to the predefinable coordinate system. Thus, there is an effective reaction to a change in the orientation of the reference camera with respect to the predeterminable coordinate system. The predeterminable coordinate system can be fixedly connected to a carrier body and / or has the same movement as the camera system and in particular does not change with respect to the camera system. The change in orientation of a camera of the camera system and / or of the reference camera can take place, for example, when the camera system is integrated in a vehicle and the vehicle has been in contact with the environment at the location of a camera of the camera system and / or of the reference camera, in particular between two calibrations, and a deformation of the vehicle occurred, in particular, at least at the location of a camera of the camera system and / or of the reference camera. In the case of the camera system integrated in the vehicle, the predeterminable coordinate system can be a coordinate system invariable with respect to the vehicle.Several advantages result from the method:• The cameras can be calibrated without using additional devices and / or additional infrastructure objects for calibrating and / or additional infrastructure and / or a predeterminable calibration environment and / or additional calibration objects. This means that the calibration is carried out only by means of the images of the cameras and software, which enables a cost-effective implementation.• Immediate calibration of the camera system is possible. This is particularly useful when a wrong position of a camera of the camera system is detected. For example, the camera system can be integrated in a vehicle. If the vehicle comes into contact with the environment, for example, an immediate calibration of the camera system can take place, as a result of which reliable functioning of at least one, in particular of all driver assistance systems and / or of the semi-autonomous and / or autonomous driving can be ensured, for example, even after the contact. This can be based on the fact that at least one driver assistance system and / or the semi-autonomous and / or autonomous driving is based on the image data of at least one camera of the camera system and the calibration of the cameras is required for reliable functioning of the at least one driver assistance system and / or semi-autonomous and / or autonomous driving. This can be caused, for example, by the fact that the 2D image of the camera must be classified into the 3D environment of the vehicle, which can assume calibration. Additionally or alternatively, the driver assistance system and / or the semi-autonomous and / or autonomous driving can be based on image data of a plurality of cameras of the camera system, which presupposes the knowledge of the relative orientations of the cameras when these image data are combined.• Unlike a method in which overlaps of the image areas of at least two cameras are used to determine the relative orientations of the two cameras, in this method no overlaps of the image areas are assumed and are therefore not present (overlap-free image capture areas of the cameras) in one embodiment. This means that not only can cameras be calibrated relative to one another, the image areas of which are at least partially identical. As a result, for example, in the case of a vehicle, a calibration of the front and rear camera is also possible.• In contrast to other methods, it is not necessary for the floor of the environment, in particular the road, to be imaged by the cameras when the camera system is installed in a vehicle, i.e. in one embodiment no floor is imaged in at least one camera image. In particular, the other methods function via prominent points on the road, for example roadway textures, in particular corner-like objects such as corners of lane markings, for example, whereby the speed range is limited by the fact that the individual roadway textures still have to be recognized by the camera. In contrast, the method described here offers the advantage that, due to the fact that arbitrary object points can be used, it can be carried out at arbitrary speeds, for example by the object points being selected accordingly depending on the speed. For example, object points may be selected at higher speeds of the vehicle that are farther from the vehicle than at lower speeds of the vehicle where object points closer to the vehicle may be selected. A distance estimate can be taken from the prior art, e.g. a parallax evaluation.The invention also comprises developments, by means of which additional advantages result.One development comprises a calibration, in particular a further calibration, of the camera system being carried out, method steps based on a Kalman filter being used, an extended Kalman filter in particular being used for solving nonlinear problems. The use of the Kalman filter offers the advantage over a method based on a graph that substantially less hardware is required for calculating the relative orientations of the cameras with respect to one another. Thus, the method based on a graph is based on first collecting the measurement points and subsequently determining the calibration in a calculation including all measurement points. A Kalman filter is a sequential and / or filter-based method, that is to say the new measured value is added in each case to the result in one step, resulting in a new result as an input variable for a next step. This means in particular that the measured value is not stored. As a result, the overall required computing effort is distributed over a greater period of time, as a result of which overall less computing power has to be available at one point in time than, for example, in the case of a graph-based method. This allows more cost-effective and space-saving hardware to be used, which is advantageous in particular when used in a vehicle in which the space available and / or the production costs are limited. The graph-based method and / or other optimization methods and / or variants of the Kalman filter are likewise considered to be part of the invention.For the Kalman filter, the equations known from the prior art can be used, wherein the notation described below can be selected. It is noted that other notations are possible as well as combinations of several sizes. The notation provided is for clarity and traceability purposes only and is not intended to be limiting in nature. In the specific case, the state comprises the orientations of the cameras.Initialization: Initialization:x: the average value assumed by the state (state vector)P: Covariance matrix of the statePrediction:x k|k-1= f (x k-1|k-1, u k): prediction of the state from the previous state (estimated state vector) with the deterministic disturbance (control vector) u k prediction of the covariance matrix via the transition matrix F k and the process noise Q kCorrection:y k= y(x k|k-1, z k) = z k- h(xk|k-1). Innovation with observation vector z k Innovation covariance with observation matrix H k and variance R kKalman-Gain Matrixx k|k= x k|k-1+ K k y k: corrected average statecorrected covariance matrixThe meaning of the specified variables corresponds to that known from the prior art which uses the invention in a novel manner, as is described in more detail below.The first step comprises at least one state vector comprising, for at least two cameras, in each case angles for describing the orientation in comparison with a predeterminable coordinate system. The coordinate system is in particular coupled to the camera system; in particular, the coordinate system describes the reference system of the camera system. The state vector is a vector describing the state x. By way of example, the roll angle, the pitch angle and the yaw angle, preferably in the form of the Euler angles, can be used as angles for describing the relative orientation of the camera to the predeterminable coordinate system. In addition to the orientations of the cameras (relative to the predeterminable coordinate system), the state vector can comprise the angular speed ω of the camera system and / or the speed v of the camera system. An example of a state vector of a camera system having n cameras, wherein the orientation of the ithcamera is described by the angles ψ i, θ i and φ i taking into account the optionally added speed and angular speed is given below:For initialization, the relative orientations of the cameras, which were obtained via the epipolar geometry, can be assigned to the state vector. Alternatively, predefinable initial values can be selected.As a second step, the method includes a repeatable Kalman cycle. The Kalman cycle includes a prediction step including determining an estimation state vector based on the state vector. This means that the basis for the prediction step is the state vector-preferably either the state vector obtained during the initialization or the state vector from the previous Kalman cycle. On the basis of this, the estimated state vector x k|k-1 can be subsequently determined in the prediction step, which indicates the state to a step k if the state vector indicates the state in the step k-1.The prediction step can be implemented, for example, by the angular velocity and the velocity being assumed to be constant and their uncertainty increasing by the process noise Q k being included in the prediction step. The changes in speed and angular speed can be taken into account via a correction step. This can implement that the speed and the angular speed change from one step k to the next step k+1, wherein the change does not have to be predefined by a model which, for example, also includes control variables of a vehicle and / or variables apart from the speed and / or angular speed. This enables a variety of possible uses, in particular also the possibility of using the camera system in a vehicle, in which the angular speed and / or the speed cannot be described by a predefinable model. In the example, movements of the camera system, in particular in the form of speed and angular speed, can be included as observations.Since the camera orientations are assumed to be constant and the movement of the camera system, i.e. the speed and the angular speed, enters via the process noise Q k the transition matrix F k is simplified to a unit matrix in the example. This advantageously results in a particularly simple calculation. This advantage can likewise result in the selection u k= 0 as is the case in the example given. u k describes a deterministic disturbance (control vector) which is not present with respect to the relative orientations of the cameras with respect to one another by the cameras of the camera system which are immovable with respect to one another. This means that the deterministic disturbance u k used in the Kalman filter can be set to zero on account of the immobility of the cameras of the camera system with respect to one another.In addition to the prediction step, the Kalman cycle comprises at least a first correction step. The first correction step comprises the determination of at least one correction point pair comprising a first correction point in a first correction image of a first camera and a second correction point in a second correction image of a second camera, wherein the first correction point and the second correction point describe the same object point, that is to say the same object point represented on the first correction image is referred to as a first correction point and represented on the second correction image is referred to as a second correction point. The identification of a correction point pair can be effected by means of a method which is known from the prior art.The first camera and the second camera can be the same camera which generates the first correction image at a first correction time and generates the second correction image at a second correction time which differs from the first correction time and is preferably after the first correction time as seen in time. Alternatively, the first camera and the second camera may be two different cameras, wherein the first correction image and the second correction image are created at the same time. In particular, the same object point is depicted on the first correction image and on the second correction image. The case where the first camera and the second camera are the same camera that respectively generates images at a first correction time and a second correction time different from the first correction time will be described below.The first correction step further comprises that for at least one pair of correction points, in particular from two images recorded by the same camera at different times, in each case an innovation distance between a prediction line, which can have the shape of a circle or a straight line, for example, and the second correction point is determined, wherein the prediction line is determined by determining, via the estimation state vector, the relative position and orientation of a virtual second estimation camera (model prediction from the prediction step of the second camera), which is assigned the second correction point and, on the basis of the position and the orientation of the virtual second estimation camera relative to the position and orientation of the first camera and, on the basis of the first correction point, the prediction line is determined by means of epipolar geometry. In other words, the estimation state vector from the prediction step is used to estimate therefrom the relative orientation and relative position of the virtual second estimation camera (and thus of the second camera) with respect to the first camera. This means in particular that it is estimated based on the prediction step how the orientation and position of a camera at the second correction time (given by the virtual second estimation camera) is relative to the first correction time (given by the first camera). In particular, since the relative orientation and position of the virtual second estimation camera with respect to the first camera determined from the state of estimation vector does not have to correspond to the relative orientation and relative position of the second camera with respect to the first camera present in the camera system (the state of estimation vector and thus of the virtual second estimation camera is a description from a model), the virtual second estimation camera is introduced. The virtual second estimation camera can be viewed as a model prediction (from the prediction step) of the second camera, wherein the second camera is in particular part of the camera system. In particular, since the orientation and position of the first camera is used and not only model prediction of the orientation and position of the first camera is used, introduction of a first estimation camera can be omitted. If correctly estimated, the relative orientation and position of the virtual second estimation camera to the first camera would correspond to the relative orientation and position of the second camera to the first camera.From the relative orientation and position of the virtual second estimation camera with respect to the first camera and the position of the first correction point in the first correction image, the prediction line can be determined by means of a method known from the prior art, in particular via epipolar geometry, which prediction line indicates a virtual second correction point range, i.e. indicates where a virtual second correction point is located in the virtual second correction image. The virtual second correction image is that image which would be recorded by the virtual second estimation camera, that is to say that image which would be recorded by a camera whose relative orientation and position to the first camera corresponds to the relative orientation and position determined from the estimation state vector. The virtual second correction point can be found at the location in the virtual second correction image at which the object point, which is described by the first correction point on the first correction image, is located in the virtual second correction image. In other words, the virtual second correction point and the virtual second correction image correspond to the second correction point and the second correction image, but from the viewpoint of the virtual second estimation camera instead of the second camera. Since the position of the virtual second correction point cannot be narrowed down to a single position by means of epipolar geometry, but only to a region which is referred to here as virtual second correction region, this model is referred to as "error model" instead of the observation model which is usual in the case of the Kalman filter and is based on measurements. The shortest distance between the prediction line and the second correction point can be considered as the innovation distance, in particular on a predeterminable surface, for example on a predeterminable plane and / or a predeterminable spherical surface.The innovation distance corresponds to the innovation y k of the first correction step of the Kalman filter. In particular, the observation matrix H k can be determined via the Jakobi matrix of the innovation: the state vector for the next step can be determined from the estimated state vector by means of the innovation. In particular, the state vector for the next step may be the state vector that serves as a basis for the next Kalman cycle.The first correction step can be performed in different ways: 1. all the pairs of correction points are used in the same first correction step to determine the innovation distance and thus the innovation. This means that the innovation determined in a single first correction step is based on all the determined pairs of correction points. This makes the calculation computationally expensive, since correspondingly large matrices are used and inverted according to the number of correction point pairs. A stable convergence behavior, in particular of the state vector, is obtained as an advantage. 2. a separate first correction step is used for each pair of correction points. This means that, in the case of n pairs of correction points, there are n first correction steps per Kalman cycle and in each first correction step only one pair of correction points is taken into account in order to determine the innovation. The advantage here is that, in contrast to the procedure set forth in (1), the calculation is substantially less computation-intensive, since no matrices are used. On the other hand, by distributing the corrections over the pairs of correction points to several corrections, the relationship between the individual corrections is lost. The division of the correction point pairs into different first correction steps leads to poorer convergence behavior. Additionally or alternatively, using fewer pairs of points, wherein by little may be meant in this case less than 10, in particular less than 5, may result in an underdetermined system. This means that when using only one pair of correction points for each first correction step, there is an underdetermined system as is known from the prior art, which can lead to poor convergence behavior. 3. a combination of options (1) and (2): The n pairs of correction points are divided into m (m<n) groups, so that altogether m first correction steps are carried out. The number of correction point pairs in a group m i may be n i where n i may differ from group to group. In a correction step assigned to the group m i n i correction point pairs are used to determine the innovation. This results in the advantage that the number of correction point pairs used per first correction step can be adapted to the available computing power, but at the same time as good a convergence behavior as possible can be achieved.The use of the Kalman filter in the described manner results in the advantage that a more precise estimation is possible solely by means of epipolar geometry, in particular by repeatedly carrying out a correction on the estimation of the relative orientations of the cameras with respect to one another.A development comprises that the prediction step comprises a noise in a value of the relative orientation of the cameras with respect to one another and / or with respect to a predeterminable coordinate system, so that during the prediction step of the Kalman filter, an orientation change of the cameras is permitted. In other words, the relative orientations of the cameras with respect to each other are assumed to be constant with a noise in the prediction step. The noise is given in that in the prediction step the relative orientations of the cameras with respect to one another are not kept constant, but are variable (variable). This noise can be realized by corresponding entries in the matrix Q k describing the process noise.Even if this is in complete contrast to the fact that the relative orientations of the cameras of the camera system with respect to one another do not change, this procedure has the advantage that it prevents a convergence to an incorrect value for the estimation of the relative orientations of the cameras with respect to one another. The method of the Kalman filter is based in particular on the uncertainty of the estimation being increased in the prediction step, while it is reduced in the correction step. If the relative orientation of the cameras is assumed to be constant in the prediction step, therefore no noise is added in the relative orientations of the cameras with respect to one another, the uncertainty in the prediction step remains constant, while the uncertainty in the correction step decreases. As a result, the uncertainty decreases further in each Kalman cycle. If the uncertainty is less than a value, in particular less than a predefinable value, the state of estimation vector is not adapted in the correction step, so that the state of estimation vector corresponds to the state vector for the next step. However, this does not involve any new pairs of correction points in the Kalman cycle. A convergence to incorrect values at constant relative orientations of the cameras with respect to one another in the prediction step was detected by calculations. The convergence to false values can be prevented by introducing the noise.A development comprises that a second correction step comprises the translation and / or rotation speed, which characterizes the movement of the cameras and in particular the movement of the camera system. In other words, a second correction step is carried out in addition to the at least one first correction step. In particular in the case that the camera system is integrated in a vehicle, the speed of the vehicle is included by the correction step. The speed is particularly preferably viewed only in one direction, in the case of the vehicle in the direction of travel. It is particularly preferred to choose the coordinate system such that the camera system moves in the x-direction of the coordinate system or points the coordinate system with the x-axis in the direction of travel, in particular of the vehicle. In particular, only the translation speed is considered.In the present example of a speed in the x direction, the function of the correction step h(x|k-1) can be selected to be equal to the speed in the x direction, whereby for the Jakobi matrix this function, which yields the observation matrix H k follows for the example that all entries are zero, except for the entry which is assigned to the speed in the x direction (in the example, this would be the 4th entry from above of the vector H k), which assumes the value 1. A predefinable value can be selected as variance R k in particular at the point assigned to the speed in the x direction.The advantage of the speed being included in the second correction step is that all six degrees of freedom of movement of the camera system can be determined. This means that a scale can be determined, i.e. not only relative but also absolute details can be made. Additionally or alternatively, the advantage results that the model, i.e. the application of the Kalman filter to the camera system, is simplified, in particular since this second correction step represents a simple possibility for taking into account the speed. This can simplify the calculation, which can reduce the costs for the required hardware.A development comprises that the at least one first correction step comprises a check for outliers of the first type, wherein the outlier of the first type is identified as a pair of correction points, but does not describe the same object point with a predeterminable probability, and the check comprises that, in the case of an innovation distance of a pair of correction points which is greater than a predeterminable threshold value, this pair of correction points is identified as outliers of the first type and is not used as a pair of correction points in the at least one first correction step. This means that a first correction point and a second correction point are recognized as a pair of correction points, but are not used in the determination of the innovation distance. The innovation can serve as a basis for this. For example, a pair of correction points may be discarded if Jakobi is the matrix of transforming the second correction point from the original image into homogeneous coordinates. Over τ, which is preferably indicated in pixels, the probability with which a correction point pair is discarded can be controlled: the higher τ, the fewer correction point pairs are discarded. On the basis of calculations, τ=3 px has proven to be advantageous, wherein τ is preferably between 1 px and 5 px. τ can have a predefinable value. It is possible to preset over τ with which probability correction point pairs are identified as correction point pairs, that is to say the correction points describe the same object point.The advantage of detecting outliers of the first type is that undesired behavior and / or incorrect convergence behavior is prevented. The Kalman filter has the property that new observations are always integrated into the result comprising all observations before. Consequently, in the case of using two points as the correction point pair, although the two points do not describe the same object point, wrong observation has influence not only on a single result but also on subsequent results. Consequently, a distance and thus also a detection for outliers is important in particular in filter methods such as the Kalman filter. Additionally or alternatively, this procedure offers a simple and low-computational possibility (and thus easy and cost-effective to implement on the hardware side) of ascertaining outliers, that is to say points determined as correction point pairs, which, however, do not describe the same object point.A development comprises that a check is made for outliers of the second type which are identified as a pair of image points and / or a pair of correction points but do not describe the same object point with a predeterminable probability, and the check is made by the movement of the camera system being captured, a valid region being determined on the basis of the movement and the first image point and / or correction point, and a pair of image points and / or correction point pair being discarded as outliers of the second type if the second image point and / or the second correction point is located outside the valid region. The probability can be predetermined by defining the size of the valid range. In particular, the check for outliers of the second type takes place before the first Kalman cycle, in particular before each Kalman cycle.The advantage of this is an effective identification of outliers of the second type, which also involves the movement of the camera system and is thus less prone to errors.As a further aspect, the invention comprises a camera system having a processor unit, wherein the processor unit is configured to carry out the method according to the invention for calibrating the camera system.As a further aspect, the invention comprises a vehicle having a camera system according to the invention comprising a processor unit according to the invention.As a further aspect, the invention comprises a vehicle according to the invention, in which the cameras of the camera system are used for at least one driver assistance system and / or semi-autonomous and / or autonomous driving and reliable functioning of the at least one driver assistance system and / or of semi-autonomous and / or autonomous driving is ensured by the calibration. At least one driver assistance system and / or semi-autonomous and / or autonomous driving can access image data from at least one camera. The driver assistance system can be, for example, at least one of the following driver assistance systems:• Cruise Control CC (Adaptive Cruise Control ACC and / or Dynamic Cruise Control DCC)• Parking Assistant (Automatically Finding Parking and Unparking, Parking Spaces)• Detection of vehicles and / or traffic signs and / or persons• Lane keeping assistant and / or lane departure warning• Congestion AssistIn at least some of the driver assistance systems and / or in semi-autonomous and / or autonomous driving, at least classification of the camera image into the 3D space in which the vehicle is located is important. Additionally or alternatively, the cooperation of a plurality of cameras may be necessary, in particular if the driver assistance system and / or the semi-autonomous and / or autonomous driving accesses the image data of a plurality of cameras as input data. In order for the driver assistance system and / or the semi-autonomous and / or autonomous driving to function reliably, in particular when a plurality of cameras cooperate and / or when the image needs to be arranged in the 3D space, a calibration of the cameras is essential. In particular, in the case of incorrect calibration, for example because the vehicle has contacted its environment and / or deformation of at least part of the vehicle has occurred, malfunctions of at least one driver assistance system and / or of the semi-autonomous and / or autonomous driving can occur. This can lead to undesired reactions of the vehicle, for example to a misestimate of the distance to an object of the environment of the vehicle, as a result of which an accident can occur, for example.The camera system according to the invention with the processor unit and the vehicle according to the invention can comprise developments which have been described as developments of the method according to the invention. For this reason, the corresponding developments of the camera system according to the invention with the processor unit and the vehicle according to the invention are not described again here.For use cases or application situations which can arise in the method and which are not explicitly described here, provision can be made for an error message and / or a request for inputting a user feedback to be output and / or for a default setting and / or a predetermined initial state to be set according to the method.The invention also includes the control device for the vehicle. The control device can have a data processing device or a processor device which is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (field programmable gate array) and / or at least one DSP (digital signal processor). In particular, a CPU (central processing unit), a GPU (graphic processing unit) or an NPU (neural processing unit) can be used as the microprocessor in each case. Furthermore, the processor device can have program code which is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device. The processor device can be based on at least one circuit board and / or on at least one SoC (system on chip), for example.The vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.As a further solution, the invention also comprises a computer-readable storage medium comprising program code which, when executed by a computer or a computer cluster, causes the latter to execute an embodiment of the method according to the invention. The storage medium may be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD-solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM-random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated, for example, as a so-called store server and / or cloud server on the Internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor. The program code can be provided as binary code and / or as assembler code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).The invention also includes the combinations of the features of the described embodiments. The invention therefore also comprises implementations which each have a combination of the features of a plurality of the described embodiments, provided that the embodiments have not been described as mutually exclusive.Exemplary embodiments of the invention are described below. The following shows: FIG. 1 shows a flow chart of an embodiment of the method according to the invention.The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that are to be considered independently of one another and that also develop the invention independently of one another. Therefore, the disclosure is intended to include combinations of the features of the embodiments other than those illustrated. Furthermore, the described embodiments can also be supplemented by further features of the invention that have already been described.In the figures, identical reference numerals designate functionally identical elements.FIG. 1 describes a flow chart which describes an embodiment of the method according to the invention. This method can be carried out, for example, within a vehicle 100 comprising at least one camera system.As a first calibration step S 200, the method comprises a calibration of cameras of a camera system. In particular, the calibration comprises a determination of the relative orientations of the cameras of the camera system relative to one another. For this purpose, the positions of the cameras with respect to one another are preferably known, wherein position is to be understood in particular as the relative position with respect to at least one other camera without specifying a scale. For this purpose, image point pairs are used for each camera, which image point pairs comprise a first image point on a first image and a second image point on a second image, wherein the second image point describes the first image point on the second image, which was recorded by the same camera at a different, in particular a later, time. The movement of the respective camera is determined via the image point pairs, which are in particular from the same camera on images recorded at different points in time. That is, it may be determined how the position of the camera has changed between capturing the first image (occurring at a first time) and capturing the second image (occurring at a second time). In addition, it is possible to determine the relative orientation to the direction of movement and / or the movement path of the camera and / or between the camera at the first point in time and the camera at the second point in time. In particular, no statement about a scale, in particular the relative positions of the cameras, at different points in time with respect to one another is possible here.The determination of the movement, in particular the direction of movement, and the orientation of the respective camera at the second point in time relative to the first point in time takes place by means of epipolar geometry. In particular, at least 5 image point pairs are required for each camera. Epipolar geometry can indicate a possible second image position of the second image point in the second image, in particular depending on the first image position of the first image point in the first image as well as the relative position and orientation of the camera at the second point in time relative to the first point in time. The second image position can be limited by means of epipolar geometry to a line, as is the case in particular with a perspective camera, or to a large circle, as is the case in particular with the use of fisheye cameras.The determination of the movement (translation and / or rotation movement) of the camera and the determination of the orientation of the camera at the second point in time relative to the first point in time in particular by means of the calibration of the cameras take place for at least two, preferably all, cameras of the camera system. Those cameras for which the movement and the orientation were determined at the second point in time relative to the first point in time are referred to as specific cameras. Since all cameras of the camera system are immovable with respect to one another and their movements are coupled to the movements of the camera system, the movements of the cameras are also coupled to one another. By coupling the movement of the cameras to one another and knowing the orientations and positions of the cameras at the second point in time relative to the first point in time and knowing the movement for the particular cameras, the relative orientation of the particular cameras with respect to one another is determined. For this purpose, for example, a translation movement of the cameras and thus of the camera system at a speed 313 of not zero is necessary, as a result of which two degrees of freedom of the relative orientations of the specific cameras with respect to one another can be determined. In order to determine the last degree of freedom of the relative orientations of the specific cameras with respect to one another, the relative orientation of the specific cameras about the axis which points in the direction of the translation movement, a change in the translation movement direction may be necessary. Additionally or alternatively, the three degrees of freedom of the relative orientations of the cameras with respect to one another can be determined if a translation and / or rotation of the camera system takes place along two and / or about two axes (straight lines) running non-parallel to one another.By means of a camera system moving at a speed 313 of not zero and a changing direction of movement, a calibration of the cameras, in particular of the specific cameras, of the camera system can take place in the first calibration step S 200.As a second calibration step S300, the method can provide a further calibration of the at least two cameras of the camera system. For this purpose, in particular a procedure as is also provided for a Kalman filter S 340 can be used.The input data 310 for the second calibration step S 300 comprise in particular correction point pairs 311 in each case for the individual cameras of the camera system and / or the speed 313 of the camera system and / or the yaw speed 312 of the camera system. A correction point pair 311 can each be a first correction point in a first correction image and a second correction point in a second correction image, wherein the first correction point and the second correction point describe the same object point which is in particular immovable with respect to an environment not surrounding the camera system and / or the vehicle 100, wherein the second correction image has been recorded by the same camera at another, in particular a later, time.By means of the input data 310 comprising the correction point pairs 311 and the speed 313 or the correction point pairs 311 and the speed 313 and the yaw speed 312 of the camera system, a check for outliers of the second type S 320 can be carried out. An exemplary embodiment for checking for outliers of the second type S 320 is given below, the basic idea of which is based on the object point having a minimum and a maximum distance from the camera (distance factor S 321) and the change in position of which from the first correction image to the second correction image is restricted by the movement of the camera system (movement restriction S322).The distance inclusion S 321 can take place via a predefinable minimum distance d min and a predefinable maximum distance d max. Preferably, the value for d min is from 1m to 10m, more preferably from 1m to 5m, and the value for d max is from 100m to 500m, more preferably from 100m to 300m. In particular, the positions of the first correction point and the second correction point of each correction point pair 311 are present in normalized camera space (normalized coordinates).For the distance inclusion S 321 a line may be drawn starting from the camera at a first correction time to the object point observed at a correction point pair 311. Subsequently, by means of a determined movement of the camera system, the position of the camera can be determined at a second correction time. The movement of the camera system can comprise in particular the speed 313 and / or the yaw speed 312 of the camera system. Based on the position of the camera at the second correction time relative to the first correction time, a valid range may be determined. For this purpose, two circles each - one with radius d min and one with radius d max- whose intersection region is the valid region, can be determined around the positions of the camera at the first point in time and at the second point in time. That part of the line starting from the camera at the first correction time to the object point which runs in the valid range is referred to as a valid line.For the movement restriction S322, the valid line can be transferred into the image, in particular into the registered image or into the raw data format of the image (raw image) of the camera at the second correction time, whereby a transferred valid line is determined. The image in which the correction points are detected can be a registered image. A recorded image is in particular an image in which the geometric aberration of optical systems known from the prior art as distortion has not been corrected. The distortion can occur due to the lens of the camera and / or due to the orientation between the lens and the image sensor of the camera. Registered images may be images in raw data format. A valid circle can be determined around the transmitted valid line, the center point of which circle is preferably situated centrally between the end points of the transmitted valid line and on the transmitted valid line. The radius corresponds in particular to the length of the valid line or a predeterminable length. Preferably, the predeterminable length can depend on the distance of the object point from the camera system.If the second correction point is located within or on the line of the valid circle, the correction point pair 311 comprising the first correction point and the second correction point can be retained. If the second correction point is outside the valid circle, the correction point pair 311 can be discarded and not taken into account in the Kalman cycle S 342.In a normalization step S 330, the correction point pairs 311 can be normalized in undistorted (distortion-free) normalized coordinates (distorted normalized camera points). For this purpose, the intrinsic parameters of the respective camera can be used. The advantage of this is that the properties of the correction point pairs 311 are subsequently no longer dependent on the properties of the respective camera, so that it is not necessary to take into account in further calculations with which camera the images were recorded with the respective correction points of the respective correction point pair 311.Based on a Kalman filter S 340, a further calibration of the at least two cameras of the camera system can take place. For this purpose, an initialization S 341 of a state vector can be provided. After initialization S 341, the state vector preferably comprises the relative orientations of the cameras of the camera system with respect to one another determined in the first calibration step S 200. Additionally or alternatively, standard values for the orientations of the cameras and / or a predefinable value, for example zero or a current measured value, can be used for the speed and / or angular speed. In the initialization S 341, using the state vector (camera system with n cameras) mentioned as an example in the description, the following matrix can be selected as covariance matrix: with the unit matrix of the corresponding size and the uncertainties σω, σ v and σψ, respectively for the angular velocity ω, the velocity v and the relative orientations ψ. By trial, the following values have proven to be particularly advantageous: σωfrom 80 deg / s to 100 deg / s, σ v from 50 km / h to 250 km / h, in particular between 150 km / h and 250 km / h, and σψbetween 1 deg and 5 deg.The state vector obtained at the initialization S 341 may be used as a basis for the first Kalman cycle S 342. For the j-th (j from the natural numbers without zero and one) Kalman cycle S 342, the state vector which is obtained as a result from the (j-1)-th Kalman cycle S 342 can be used in each case.The Kalman cycle S 342 can comprise a prediction step S 343 in which an estimation state vector can be determined which can form the basis for an update step S 344. Preferably, for the process noise in the example given in the description, the following matrix is used: Q k preferably with σαfrom 40 deg / s 2 to 60 deg / s 2, σα,xfrom 5 m / s 2 to 15 m / s 2, σαfrom 1⁄2 m / s 2 to 3 m / s 2 and σωbetween 0 deg / s and 1 / 10 deg / s. Δt corresponds to the time difference between two time steps, i.e. between the recording times of the first and second correction images. In particular, by introducing an uncertainty, i.e. a noise, in the relative orientation of the cameras with respect to one another, incorrect convergence of the Kalman filter can be prevented. The uncertainty in the relative orientation of the cameras with respect to one another would correspond to the fact that the relative orientation of the cameras with respect to one another can change during the prediction step S 343, which is not the case in particular in the camera system.The update step 344 comprises at least one correction step S345. An update step 344 can comprise a predefinable number of correction steps S345, wherein a correction step S345 preferably uses the results of the previous step-either prediction step S343 or correction step S345-as a basis.An example of a correction step S 334 is a first correction step S 334. The first correction step S346 preferably comprises a check for outliers of the first type. Additionally or alternatively, the first correction step S346 comprises the determination of an innovation distance by means of at least one correction point pair 311, in that a virtual second estimation camera is assumed, which has a position and orientation relative to the position and orientation of the first camera, which position and orientation is given by the estimation state vector determined in the prediction step S343. By means of the relative position and orientation of the first camera to the virtual second estimation camera, which can represent a model prediction by means of the prediction step of the first camera at a later point in time, and by means of the first correction point, a prediction line can be determined by means of epipolar geometry, which can be used for determining the innovation. The prediction line and the innovation can be determined, for example, in one of the following ways:• For a perspective camera, the prediction line may correspond to the epipolar line. The determination of the epipolar line based on the relative orientations and positions of the first camera and the virtual second estimation camera and the first correction point can be taken from the prior art. The shortest distance between the second correction point and the epipolar line can be considered as an innovation.• For a fisheye camera, the prediction line may correspond to the epipolar large circle (English). The reason for this is that in fisheye cameras the projection surface, in contrast to a perspective camera, is not a plane but a hemisphere with center in the respective camera. Accordingly, the epipolar circle is defined as a line at which the hemisphere and the epipolar plane intersect. The shortest distance along the surface of the hemisphere between the second correction point and the epipolar circle can be considered as an innovation.• If a straight line exists which intersects the position of the first camera, the first correction point and the position of the virtual second estimation camera, this straight line may correspond to the prediction line. The shortest distance in the image plane of the second virtual estimation camera between the prediction line and the second correction point can be considered as an innovation. Alternatively, in this case, the pair of correction points may be ignored in the correction step S 345.The variance R k required for the first correction step S 334 can be determined, for example, by the following formula, which advantageously includes the fact that the detection takes place in the recorded image, for example in the image in the raw data format, but the calculation takes place in undistorted and / or unsigned normalized coordinates, since this in particular prevents a computationally expensive image equalization and the determination of the innovation is less computationally expensive. The reason for the lower computational effort in determining the innovation is that the properties of the camera by which the image was generated have already been removed, so that the properties of the points imaged on the image are independent of the camera with which the image was recorded. The removal of the distortion results in at least one prediction line for which efficient distance calculation is possible. Additionally or alternatively, it must be taken into account that the variance is dependent on both the first and the second correction point, since the innovation determined via the prediction line is dependent on both points. This also takes into consideration the following formula for the variance in an advantageous manner.Here, z is an observation vector containing the coordinates of the first correction point and the second correction point in undistorted (unsigned) normalized coordinates, z' is the corresponding observation vector containing the coordinates of the first correction point and the second correction point in the recorded image, for example in raw data format, y is the innovation and a diagonal matrix whose values all correspond to the value which represents the uncertainty in the detection in the image, i.e. how exactly the coordinates of a point in the image can be specified. Preferably, σ z, is set to a value of 1 px to 5 px (px = pixel).When ascertaining the innovation in the first correction step S 334, the innovation can be ascertained on the basis of all correction point pairs 311 ascertained in the first and second correction images or only a real part of all correction point pairs 311 ascertained in the first and second correction images. If not all correction point pairs 311 determined from the first and second correction images are used for determining the innovation, it can be provided that the update step S 344 comprises, as a correction step S 334, a first correction step S 334, in which a first part of the determined correction point pairs 311 are used for determining the innovation, and, as at least one further correction step S 334, a first correction step S 334, in which a second part of the determined correction point pairs 311 (the correction point pairs 311 included in the second part are not included in the first part) is used for determining an innovation and which is based on the result of the previous correction step S 334. This means that the correction point pairs 311 are divided into a predefinable number n (in the case of N correction point pairs 311, n can go from 0 to N) of groups, wherein preferably no correction point pair 311 can be found in two groups. The innovation is determined for each of these groups.In addition or as an alternative to at least one first correction step S346, the update step S344 can comprise a second correction step S347, in which the speed 313 of the camera system is included.The Kalman cycle S 342 can be repeated in particular according to a predefinable number, that is to say the Kalman cycle S 342 can be repeated as many times as a predefinable number indicates.The result that the Kalman filter S 340 outputs can comprise, in particular, a state vector which comprises the relative orientations of at least a part of, preferably all, the cameras of the camera system with respect to one another. These relative orientations of the cameras of the camera system with respect to one another can be output in an output step S350.The calibration of the cameras of the camera system can be used, for example, in a vehicle 100 for realizing and / or ensuring the mode of operation of driver assistance systems and / or autonomous driving. This is the case in particular when images of different cameras are composed, for example for the purpose of displaying a corresponding image, and / or when a plurality of cameras have to cooperate for the purpose of realizing the driver assistance system and / or autonomous driving, for example in order to capture a corresponding solid angle range and / or in order to combine the advantages of different cameras with different properties.Application examples for driver assistance systems and / or semi-autonomous and / or autonomous driving are to be mentioned below, for which a calibration of cameras is a prerequisite:• Vehicle environment display: If a larger solid angle range is to be displayed than is possible by the cameras present in a camera system of a vehicle 100, and / or a display takes place from a predeterminable perspective, the images of the various cameras must be put together. For this purpose, a calibration of the cameras of the camera system is necessary so that the images of the various cameras are correctly composed and image areas are not doubled and / or not shown. For example, for top view or bird's eye view, as is known from the prior art (view from above), images of different cameras of the camera system are composed and (virtually) projected onto a (flat) ground. The resulting image can be displayed in the vehicle 100, for example, and preferably allows a better overview of the vehicle environment than if only the individual images of the individual cameras are displayed. Additionally or alternatively, larger image areas can be displayed, for example a surrounding view around the vehicle or at least a part of a surrounding view around the vehicle, whereby the vehicle environment can be better detected by the driver, for example.• Collision Avoidance Assistant: In order to avoid and / or warn about collisions, the risks in front of and / or behind and / or to the side of the vehicle 100 must be detected and assessed by cameras. For this purpose, it is necessary to cooperate different cameras with different solid angle ranges, which are captured by the cameras, and / or cameras with different properties, for example a combination of larger and smaller captured solid angle ranges.• Object tracking: If stationary and / or moving objects move with respect to the vehicle 100 in such a way that they transition from the detection range of one camera of the camera system to the detection range of another camera of the camera system, a calibration of the cameras is necessary for successful tracking of the object and detection of at least one possible risk.• Environment capture: 3D reconstruction of the environment of the vehicle 100 in the 3D reconstruction at least one 3-dimensional object is reconstructed from 2D images and / or a 3D reconstruction of a point, i.e. a classification of a point into a 3D space, can take place. In this case, images of a plurality of cameras are linked to one another via triangulation known from the prior art. An example of this is the stereo camera when using images of two cameras.• Self-localization with the aid of a map, for example for navigation: by means of a map in which objects stationary with respect to an environment, for example buildings, signs, roadway markings and / or landmarks, are contained, the vehicle can determine its location exactly, for example up to 10 cm, by comparing the images of the cameras, in particular the objects detected on the images of the cameras, with the objects contained in the map. Knowledge of the orientation of the cameras of the camera system of the vehicle 100 with respect to one another is necessary for this purpose. In particular, using multiple cameras increases the accuracy of self-localization.Additionally or alternatively, costs may be saved in manufacturing the vehicle by allowing multiple cameras whose images are composed to replace at least one camera with other characteristics, for example, capturing a larger solid angle range.Overall, the examples show how a movement-based calibration of cameras can be provided by means of cameras.

Claims

Method for calibrating cameras of a camera system that are not moving relative to one another, characterized in that • the camera system moves relative to an environment, wherein the movement of the camera system is characterized in that o has a translation component of the speed (313) describing the movement of a magnitude of non-zero and changes a direction of the movement, and / or o the movement is two rotations about two axes that are not parallel to one another, and / or o the movement is two translations about two axes that are not parallel to one another, and / or o the movement is a translation about a first axis and a rotation about a second axis that is not parallel to the first axis, • For at least two of the cameras, in each case image point pairs are determined from a first image of the camera at a first point in time and a second image of the camera at a second point in time, which image point pairs are characterized in that o for each image point pair in each case a first image point of the image point pair describes a first image position on the first image and a second image point of the image point pair describes a second image position on the second image, o for each image point pair the image points of the image point pair each describe a same object point, ▪ wherein the object point describes a point in the environment that is non-movable in position, and o different image point pairs describe different object points, • for the at least two cameras, the movement of the respective camera is determined in each case from the image point pairs by means of epipolar geometry, and • a relative orientation of the at least two cameras with respect to one another is determined from the determined movements of the at least two cameras.Method according to Claim 1, wherein the at least two cameras are calibrated with respect to one another, wherein method steps are used on the basis of a Kalman filter (S340), comprising • at least one state vector comprising angles for the at least two cameras in each case for describing the orientation of the camera in comparison with a predeterminable coordinate system, and • a repeatable Kalman cycle (S342), comprising o a prediction step (S343) comprising the determination of an estimation state vector on the basis of the state vector and o at least one first correction step (S346) comprising ▪ the determination of at least one correction point pair (311) comprising a first correction point on a first correction image of a first camera and a second correction point on a second correction image of a second camera, wherein a. the first correction point and the second correction point describe the same object point, b. the first camera and the second camera are the same camera which generates the first correction image at a first correction time and generates the second correction image at a second correction time which differs from the first correction time, or the first camera and the second camera are different cameras, and the first correction image and the second correction image are generated at the same time, ▪ for at least one pair of correction points (311) an innovation distance between a prediction line and the second correction point is determined in each case, wherein the determination of the prediction line is carried out by a. determining the relative position and orientation of a virtual second estimation camera via the estimation state vector, and, to which the second correction point is assigned and b. on the basis of the position and the orientation of the virtual second estimation camera relative to the position and orientation of the first camera and on the basis of the first correction point the prediction line is determined by means of epipolar geometry ▪ and the innovation distance corresponds to the innovation of the first correction step (S 346) of the Kalman filter (S 340).The method according to claim 2, wherein the prediction step (S343) comprises a noise in a value of the relative orientation of the cameras to each other, such that during the prediction step (S343) of the Kalman filter (S340), an orientation change of the cameras is allowed.Method according to claim 2 or 3, wherein a second correction step (S347) comprises the speed (313) characterizing the movement of the cameras.Method according to one of Claims 2 to 4, wherein the at least one first correction step (S346) comprises a check for outliers of the first type, wherein the outlier of the first type is identified as a correction point pair (311), but does not describe the same object point with a predeterminable probability, and the check comprises that, in the case of an innovation distance of a correction point pair (311) which is greater than a predeterminable threshold value, this correction point pair (311) is identified as an outlier of the first type and is not used as a correction point pair (311) in the at least one first correction step (S346).Method according to one of the preceding claims, wherein a check is made for outliers of the second type (S320) which are identified as a pair of image points and / or pair of correction points (311), but do not describe the same object point with a predeterminable probability, and the check is made by the movement of the camera system being captured, a valid region being determined on the basis of the movement and the first image point and / or correction point, and a pair of image points and / or pair of correction points (311) being discarded as outliers of the second type (S320) if the second image point and / or the second correction point is located outside the valid region.Camera system comprising a processor unit, wherein the processor unit is configured to perform a method for the camera system according to one of the preceding claims.Vehicle (100) having a camera system comprising a processor unit according to claim 7.The vehicle (100) according to claim 8, wherein the cameras of the camera system are used for at least one driver assistance system and / or semi-autonomous and / or autonomous driving and reliable functioning of the at least one driver assistance system and / or of the semi-autonomous and / or autonomous driving is ensured by the calibration.

Images