Method for operating a camera of a vehicle, camera system and vehicle

Abstract

The invention relates to a method for operating a camera (3) of a vehicle (1), wherein the camera (3) comprises an image sensor (13), a support structure (14) and an actuator (15), wherein the image sensor (13) is connected to the support structure (14) by way of the actuator (15) and is able to be moved relative to said support structure along one or more degrees of freedom, wherein the relative position of the image sensor (13) in relation to the support structure (14) is set depending on positioning information stored in a storage device (20), wherein the positioning information is updated at one or more points in time depending on at least one orientation state of the image sensor (13), based on operating time information describing an operating time of the camera (3), depending on a distance from an object in the surroundings, depending on at least one measured or predicted driving state criterion and / or depending on a surroundings model comprising multiple parameters, wherein the relative position of the image sensor (13) is set depending on the updated positioning information.

Description

[0001]202304350 1 Description Method for operating a camera of a vehicle, camera system and vehicle The invention relates to a method for operating a camera of a vehicle, wherein the camera comprises an image sensor, a support structure and an actuator, wherein the image sensor is connected to the support structure via the actuator and is movable relative to the latter along one or more degrees of freedom. The invention further relates to a camera system and a vehicle. Cameras are used in modern vehicles, for example, to detect objects in the vehicle's surroundings. Individual cameras can, for example, detect an area in front of the vehicle, next to the vehicle, behind the vehicle and / or underneath the vehicle. The images generated by one or more of the cameras and / or a view of the surroundings derived from these images can be displayed to a driver of the vehicle, for example to facilitate maneuvering in tight spaces or parking.Furthermore, such camera images can also be processed using a computing device, for example, to provide a driver assistance function based on the vehicle's surroundings. To capture images, cameras typically include an image sensor that can detect light. In order to move the image sensor relative to other components of the camera, it is known for some types of cameras to couple the image sensor to an actuator. US 2021 / 0354979 A1 describes a camera with an image sensor that can be adjusted relative to a camera lens via an actuator. The image sensor can be moved along the camera's optical axis to achieve focusing depending on a temperature. 202304350 2 WO 2022 / 115932 A1 describes an actuator for the image sensor of a camera, which enables movement of the image sensor with up to five degrees of freedom. With the help of the actuator, an autofocus function can beand an increase in the resolution of the image sensor can be achieved. In cameras mounted in vehicles, external influences (e.g. temperature, humidity, etc.) can lead to a misalignment or an undesired offset of camera components. The probability of such a misalignment can increase with increasing operating time of the camera or the vehicle and, for example, can cause a reduction in the image quality of an image generated by the camera. Furthermore, different conditions or circumstances in the surroundings of the vehicle can also influence the imaging by means of the camera. There is therefore a need for methods for calibrating cameras in order to at least partially compensate for the effects of such effects. The object of the invention is to provide an improved method for operating a camera of a vehicle, which in particular enables a calibration of the camera or, by means ofwhich can at least partially compensate for effects occurring during operation and influencing the image quality of the camera. This object is achieved according to the invention in a method of the type mentioned at the outset in that the relative position of the image sensor to the support structure is adjusted as a function of positioning information stored in a memory device, wherein the positioning information is updated at one or more times as a function of at least one orientation state of the image sensor, based on operating time information describing an operating time of the camera, as a function of a distance to an environmental object, as a function of at least one measured or predicted driving state criterion and / or as a function of an environmental model comprising several parameters, wherein the 202304350 3 relative position of the image sensor is adjusted as a function of the updated positioning informationis adjusted. The image sensor of the camera can be moved relative to the support structure via the actuator. The image sensor is used in particular to receive electromagnetic radiation, in particular visible light and / or infrared radiation, and to generate a camera image of at least part of the vehicle's surroundings. The captured part of the vehicle's surroundings depends in particular on the arrangement position of the camera on the vehicle. Other properties of the camera, for example a type of lens used in the camera, can also influence the field of view. The support structure can be, for example, a circuit board or another component that is mounted immovably with respect to a camera housing. Because the image sensor is connected to the actuator and is movable or adjustable relative to the support structure, the relative positioning of the image sensor to the support structure and / or to other components of theCamera, for example, to a lens of the camera that is stationary in relation to the support structure, can be changed via the actuator. The actuator enables the image sensor to be moved relative to the support structure along one or more degrees of freedom. The degrees of freedom can be translational degrees of freedom, in particular along a direction of the optical axis of the camera as well as along two directions that are orthogonal to one another and orthogonal to the optical axis. It is also possible for one or more of the degrees of freedom to be rotational degrees of freedom, which in particular enable rotation about the optical axis and / or about one or more spatial directions that are orthogonal to the optical axis of the camera. Depending on the degrees of freedom available for the movement of the image sensor relative to the support structure, in particular the alignment and / or orientation of the image sensor relative to the support structure202304350 4 or relative to a further component of the camera that is stationary with respect to the support structure. This makes it possible to at least partially compensate for different forms of misalignment between the image sensor and at least one further component of the camera. The adjustment of the image sensor relative to the support structure via the actuator is carried out depending on the positioning information, wherein the positioning information describes, in particular, a positioning of the image sensor relative to the support structure that is to be adjusted via the actuator. The positioning information is stored in a storage device, wherein the storage device can be located in the vehicle or at an external location. Initial positioning information can, for example, be determined during calibration of the camera or the vehicle at the end of its production and stored in the storage device. This advantageously allows complexAdjustments during camera installation can be avoided, since the basic alignment of the image sensor inside the camera can be individually adjusted for each of the sensors and saved as individual positioning information. In order to compensate for aging effects during operation of the camera or the vehicle, the positioning information is updated according to the invention at one or more points in time. The update takes place as a function of at least one orientation state of the image sensor, based on operating time information describing an operating time of the camera, as a function of a distance to an environmental object, as a function of at least one measured or predicted driving state criterion and / or as a function of an environmental model comprising several parameters. After updating the positioning information, the relative position of the image sensor is then adjusted as a function of theupdated positioning information. Older positioning information can either be discarded or stored as historical positioning data for later evaluation or analysis. 202304350 5 The method according to the invention makes it possible to adapt the positioning of the image sensor relative to the support structure, for example during operation of the vehicle, depending on different update parameters, so that misalignments or incorrect positions of the image sensor can be compensated. Taking into account an orientation state of the image sensor as an update parameter makes it possible, for example, to correct a detected incorrect position of the image sensor in relation to the support structure or other components of the camera, such as a lens. Additionally or alternatively, the operating time of the camera can also be used as an update parameter for updating the positioning information.Operating time information can be taken into account, which can be understood as a measure of the aging effects accumulated so far. The operating time information can, in particular, describe a total operating time of the camera. Additionally or alternatively, the operating time information can also contain information on the operating time of the camera in different climatic conditions. For this purpose, the operating time information can, for example, be determined during operation of the vehicle as a function of position data and / or weather data and stored, for example, in the vehicle. The updated positioning information can, according to the invention, also be determined as a function of at least one measured or predicted driving state criterion. The driving state criterion can describe a specific driving state of the vehicle, wherein, in particular, when the driving state criterion is fulfilled, aupdated positioning information is determined and used for a movement of the image sensor. It is possible that, in addition or as an alternative, the distance to an environmental object is also taken into account. The environmental object can be, for example, another road user and / or an infrastructure object. By adapting the image sensor in 202304350 6 dependence on the distance to the environmental object, an improved representation of the environmental object in the camera images can be achieved. Furthermore, it is possible that the updated positioning information is determined on the basis of the environmental model comprising several parameters. By taking the environmental model into account, it is possible to compensate for or even compensate for temporary misalignments that are attributable to changing environmental influences. The respective times at which an update of thePositioning information can differ for the different update parameters, i.e. for the orientation state of the image sensor, the operating time, the distance to the surrounding object, the driving state criterion and the surrounding model, so that, depending on these update parameters, updated positioning information can be determined at different times or a previously determined positioning information stored in the memory device can be updated accordingly. For example, an update of the position can occur significantly more frequently depending on the distance of the surrounding object and / or depending on the surrounding model than an adjustment depending on the operating time. The updating of the positioning information as well as the adjustment of the relative position of the image sensor depending on the updated positioning information can be carried out when the vehicle is stationary orin a moving vehicle. The camera can, in particular, be a camera arranged on the outside of the vehicle and / or a camera capturing a partial area of ​​the vehicle's surroundings outside the vehicle. The camera can, for example, be a front camera, a side camera, and / or a rear-view camera. It is possible for the vehicle to have several such cameras, which, for example, form a surround-view system. The vehicle can, in particular, comprise at least one 202304350 7 display device on which images generated by the camera and / or a view of the surroundings derived from the images of one or more cameras of the vehicle can be presented to a driver of the vehicle. The method according to the invention advantageously makes it possible to compensate for different effects that affect the image quality of an image generated by the camera. In particular,This makes it possible to continuously adjust the orientation of the image sensor over the lifetime of the camera or vehicle. Particularly for cameras located on the outside of the vehicle and continuously exposed to environmental influences and / or mechanical stress, high image quality can be achieved throughout the entire lifetime of the camera or vehicle. This also applies analogously to cameras located in the interior of the vehicle, which may, for example, be exposed to significant temperature influences. In a preferred embodiment of the invention, it can be provided that an actuator is used, by means of which the image sensor can be moved at least along the optical axis of the camera. This makes it possible, in particular, to readjust the positioning of the image sensor in relation to a focal point of a camera lens, so that sharper images are obtained compared to a non-compensated state.can be obtained. According to the invention, it can be provided that an actuator is used by which the image sensor is movable along at least two translational degrees of freedom and / or at least two rotational degrees of freedom relative to the support structure. In addition or alternatively to a mobility of the image sensor along an optical axis, the image sensor can in particular also be moved along two axes that are orthogonal to the optical axis and orthogonal to one another. In this way, for example, a centering of the image sensor in relation to a lens of the camera can be corrected or adjusted. 202304350 8 Additionally or alternatively, a rotational movement of the image sensor, i.e. a rotation of the image sensor about two or more axes, is also possible via the actuator. For this purpose, the image sensor can, for example, be tilted about one or more axes that are orthogonal to the optical axis of the camera, whereby a parallelism of theImage sensor can be corrected or readjusted to an image plane of the camera. Rotatability of the image sensor about the optical axis also enables correction of the centering of the image sensor. According to the invention, an actuator can be used which comprises a stationary first part, which is fastened to the support structure, and a second part, which is movable relative to the first part and is fastened to the image sensor. The first part, which can also be referred to as a stator, is in particular fixedly fastened to the support structure and can, for example, have a frame-like shape. The second part, which can accordingly also be referred to as a rotor, is in particular firmly connected to the image sensor, wherein the second part is movable relative to the first part in a translational and / or rotational manner along one or more degrees of freedom. For this purpose, the second part can, for example, be at least partially detachable from the frame-shaped first part.be enclosed. According to the invention, the actuator can be a MEMS actuator (MEMS = micro-electro-mechanical system), which enables precise movement of the image sensor, in particular on the order of magnitude of the pixels of the image sensor or even smaller. As a result, even small differences in the orientation of the image sensor can be compensated, so that high image quality can be achieved. According to the invention, the at least one orientation state can describe a parallelism of the image sensor to an image plane of a lens of the camera, an alignment of the image sensor to an optical axis of the camera and / or an alignment of the image sensor to a focal point of the camera. The parallelism can be described, for example, by means of an angle between the image plane and a recording surface of the image sensor or by means of an angle between a 202304350 9 normal of the recording surface of the image sensor and the optical axis. The alignment of the image sensor to theoptical axis or to a focal point can be described, for example, by a translational and / or rotational offset between the image sensor, for example a center point of a recording surface of the image sensor, and the optical axis or the focal point. According to the invention, it can be provided that the orientation state of the image sensor is determined as a function of at least one image sensor measured value. In other words, the orientation state can be determined by the image sensor or by an evaluation of one or more camera images generated with the image sensor. The image sensor measured value can be determined, for example, by means of an image data evaluation, i.e., by means of the evaluation of the content of a camera image recorded with the image sensor. The image sensor measured value can, for example, describe image sharpness, image parallelism, and / or image centering. The image sensor measured value can, for example, be determined in aTrial-and-error methods can be used by gradually moving the image sensor via the actuator along at least one degree of freedom to find a better positioning. In this way, for example, an image sharpness or a relative image sharpness for the positioning of the image sensor can be determined according to the current positioning information and compared with an image sharpness at other positions of the image sensor. A non-optimal positioning of the image sensor can then be corrected accordingly by updating the positioning information. Analogously, this method can also be carried out for other image sensor measured values, in particular with regard to a centering of the image sensor and / or to a parallelism of the image sensor. According to the invention, the operating time information can also be stored in the memory device and, for example, represent a temporal operating time of the camera and / or the vehicle.By taking into account the operating time, characteristic aging effects, which for example statistically occur after a certain number of operating hours, can be compensated for in a simple manner. In a preferred embodiment of the invention, the distance to the surrounding object can be determined by means of a distance sensor of the vehicle. The distance sensor can be, for example, an ultrasonic sensor, a radar sensor, a lidar sensor and / or another camera of the vehicle. In order to adapt the camera to the state of a current vehicle environment, it can be provided according to the invention that a temperature, an air pressure and / or a humidity of the environment of the vehicle is used as a parameter of the environment model. According to the invention, a current value of at least one parameter of the environment model can be determined using at least one environment sensor of the vehicle, in particular using atemperature sensor, a humidity sensor and / or an air pressure sensor. The determination of the parameter by the environmental sensor can in particular take place continuously. The environmental model can in particular model the influence of the vehicle's surroundings on the imaging by means of the camera, so that effects dependent on the temperature, the air pressure and / or the air humidity, for example on a beam path within the camera or the like, can be at least partially compensated for. According to the invention, it can be provided that the positioning information and / or the relative position are determined using a calculation rule and / or a lookup table. The calculation rule and / or the lookup table can, for example, be stored in the memory device and specify how the updated positioning information is to be determined depending on the respective update parameters.202304350 11 In order to protect the camera from damage, the invention can provide for the image sensor to be moved into a safety position by the actuator if at least one measured or predicted driving state criterion describes the action or impending action of mechanical forces (above a limit value) on the image sensor. The driving state criterion can, for example, describe vibrations occurring currently or in the near future. Currently occurring vibrations can, for example, be measured via a sensor of the vehicle, for example a sensor that detects chassis deflection. Vibrations that will occur in the near future can, for example, be detected by detecting unevenness in the ground such as potholes and the like, wherein such unevenness in the ground can, for example, be detected via a front camera of the vehicle. Additionally or alternatively, imminentVibrations can be predicted, for example, by detecting a sudden steering intervention and / or a heavy braking operation with an impending intervention of an anti-lock braking system. According to the invention, it can be provided that the positioning information is updated as a function of the orientation state of the image sensor and / or based on the operating time information at the beginning of operation of the vehicle and / or during configuration operation of the camera, and / or that the positioning information is updated as a function of the distance to the surrounding object, as a function of the measured or predicted driving state criterion, and / or as a function of the surrounding model during driving operation of the vehicle. In this way, the respective time scales in which the respective update parameters influence the imaging with the camera can advantageously be taken into account. 202304350 12 The invention further relatesa camera system comprising at least one camera and a control device, wherein the camera comprises an image sensor, a support structure and an actuator, wherein the image sensor is connected to the support structure via the actuator and is movable relative thereto along one or more degrees of freedom, wherein the control device is configured to adjust the relative position of the image sensor to the support structure as a function of a value stored in a memory device, wherein the control device is further configured to update the positioning information at one or more points in time as a function of at least one orientation state of the image sensor, based on operating time information describing an operating time of the camera, as a function of a distance to an environmental object, as a function of a measured or predicted driving state criterion and / or as a function of an environmental model comprising several parameters, and theThe relative position of the image sensor can be adjusted depending on the updated positioning information. The control unit can be a control unit that is separate from the camera or a control unit that is integrated into the camera or at least one of the cameras, for example a system-on-a-chip. A vehicle according to the invention is provided that it comprises a camera system according to the invention. All advantages and embodiments described above in relation to the method according to the invention also apply analogously to the camera system according to the invention and to the vehicle according to the invention, and vice versa. Further advantages and embodiments of the invention emerge from the exemplary embodiments described below and from the drawings. These are schematic representations and show: 202304350 13 Fig. 1 shows an exemplary embodiment of a vehicle according to the invention with an exemplary embodiment of an exemplary embodiment of the invention.Camera system, Fig. 2 shows a detailed view of a camera of the vehicle with the image sensor of the camera in a first relative position, Fig. 3 shows a detailed view of the camera with the image sensor in a second relative position, and Fig. 4 shows a flowchart of an embodiment of a method according to the invention. Fig. 1 shows an embodiment of a vehicle 1. The vehicle 1 can be, for example, a motor vehicle, in particular a passenger car, a truck, or another type of commercial vehicle. It is also possible for the vehicle 1 to be a non-motorized vehicle such as a trailer or for it to be a combination of a towing vehicle and a trailer. The vehicle 1 can also be a rail-bound vehicle, such as a tram or the like, or a robot. The vehicle 1 comprises an embodiment of a camera system 2, which comprises one or more cameras 3 and a control device 4. InIn the present exemplary embodiment, the vehicle 1 comprises four cameras 3, which form a camera arrangement designed as a surround view system. A first camera 5 is arranged as a front camera of the vehicle 1, a second camera 6 as a rear view camera of the vehicle 1, and a third camera 7 and a fourth camera 8 are each arranged as a side camera of the vehicle 1. The control device 4 can be designed, for example, as a microcontroller, as a processor, or as another type of computing device. One or more computing rules or algorithms can be implemented in the control device 4, which can implement one or more functions, 202304350 14 in particular driver assistance functions, for the vehicle 1 by evaluating the camera images of the cameras 3. The vehicle 1 further comprises a display device 9, via which graphic information can be presented to a driver or a user of the vehicle 1.The display device 9 can, for example, be one or more screens or displays arranged in an interior of the vehicle 1. By means of the cameras 3, image data can be generated from a partial area of ​​the surroundings of the vehicle 1. The cameras 3 are connected to the control device 4 via a communication link 10. The communication link 10 can, for example, comprise a plurality of point-to-point connections or be a bus connection such as a CAN bus or the like. Camera images or image data are transmitted, in particular continuously, from the cameras 3 to the control device 4 via the communication link 10. At least some of the camera images and / or a view of the surroundings generated from several camera images can then be displayed on the display device 9 to a driver of the vehicle 1. The vehicle 1 further comprises at least one distance sensor 11, which can be designed, for example, as an ultrasonic sensor,as a radar sensor, as a lidar sensor and / or as another camera. In addition, the vehicle 1 has at least one environmental sensor 12, wherein the environmental sensor 12 is, for example, a temperature sensor, a humidity sensor and / or an air pressure sensor. Fig. 2 shows a detailed view of a camera 3 of the vehicle 1. The camera 3 comprises an image sensor 13, a support structure 14 and an actuator 15, wherein the image sensor 13 is coupled to the support structure 14 via the actuator 15. The image sensor 13 comprises a recording surface 16 which extends in the x-direction and y-direction, or in an xy-plane. The optical axis 17 of the camera 3 runs orthogonal to the recording surface in the z-direction. The support structure 14 can be 202304350 15 for example as a circuit board or as another component of the camera 3, which is arranged stationary with respect to a housing of the camera 3 or with respect to the vehicle 1. The image sensor 13 serves forReception of electromagnetic radiation in a predetermined frequency band, in particular visible light and / or infrared radiation, and for generating an image of at least part of the surroundings of the vehicle 1. The image sensor 13 can, for example, be embodied as a semiconductor chip and connected to other electrical components of the camera 3 and / or to the support structure 14 via one or more flexible electrical connections, such as cables, spring contacts, bonding wires, or the like. The actuator 15 enables the image sensor 13 to be moved relative to the support structure 14 along one or more degrees of freedom. In Fig. 2, the image sensor 13 is in a first relative position relative to the support structure 15. Depending on the design of the actuator 15, the degrees of freedom can be translational degrees of freedom, which enable a displacement of the image sensor 13 in the x-direction, y-direction, and / or z-direction. Additionallyor alternatively, the actuator 15 can also enable a rotational movement, i.e. a rotation of the image sensor 13, about the x-axis, the y-axis and / or the z-axis. Combined movements consisting of a translational movement in the direction of one or more of the axes and / or a rotational movement about one or more of the axes are also possible. In the present exemplary embodiment, the actuator 15 is designed as a MEMS actuator which comprises a stationary first part 18, which is fastened to the support structure 14, and a second part 19, which is movable relative to the first part 18 and is fastened to the image sensor 13. For example, by applying one or more electrical voltages to the stationary first part 18 of the actuator 15, the relative position of the second part 19 of the actuator 15 and thus also of the image sensor 13 can be changed or adjusted. 202304350 16 In Fig.3 the camera 3 is in a second relative positionmoving image sensor 13 is shown. The image sensor 13 was moved translationally along the x-axis and the y-axis and rotated about the z-axis to reach the second relative position. In addition to this exemplary second relative position, further relative positions are also possible, which result from the degrees of freedom for the movement of the image sensor 13 enabled by the actuator 15. The control device 4 is configured to carry out a method for operating one or more of the cameras 3 of the vehicle 1. In this case, the relative position of the image sensor 3 to the support structure 14 of one or more of the cameras 3 is set as a function of positioning information stored in a storage device 20. In this case, separate positioning information can preferably be stored in the storage device 20 for each of the cameras 3. The storage device 20 can, for example, be part of the control device 4. Alternatively, it can be aControl device 4 can be a separate memory device 20 of the vehicle, or a vehicle-external memory device can be used, to which the control device 4 has access, in particular via a wireless communication connection. The positioning information is updated, in particular during operation of the vehicle, at one or more times as a function of at least one orientation state of the image sensor 13, based on operating time information describing an operating time of the camera 3, as a function of a distance to an environmental object, and / or as a function of an environmental model comprising several parameters. Subsequently, the relative position of the image sensor 13 is adjusted as a function of the updated positioning information. Fig. 4 shows a block diagram of an embodiment of the method for operating the at least one camera 3 of the vehicle 1. 202304350 17 In a step S1, whichFor example, when the vehicle 1 is started up, the image sensor 13 is adjusted depending on positioning information stored in the memory device 20. The positioning information specifies how the image sensor 13 of the respective camera 3 must be adjusted in order to obtain optimal image quality. In particular, the positioning information specifies the alignment and orientation of the image sensor 13 to the optical axis 17 or to another reference point of the camera 3. The positioning information can, for example, have been stored during a previous driving operation or during a special configuration operation of the vehicle 1. Initially, positioning information determined after a calibration carried out during production of the vehicle 1 or the cameras 3 can be stored in the memory device 20. The initially determined and storedPositioning information can be optimized with regard to the image quality to be achieved with the image sensor 13 or the camera 3 and, if necessary, compensate for any misalignment of the image sensor 13 with other components of the camera 3 and / or the vehicle 1 that may occur after assembly of the camera components. In order to compensate for various effects that may have a detrimental effect on the image quality to be achieved by means of the camera 3, the positioning information is updated at various times depending on different update parameters. The updates of the positioning information are described with reference to steps S2 to S6, wherein it is particularly possible for the respective updates of the positioning information to be carried out parallel to the times suitable for the respective update parameters, as described below. In step S2, thePositioning information is set as a function of at least one orientation state of the image sensor 13. The orientation state 202304350 18 of the image sensor 13 can, for example, be a parallelism of the image sensor 13 or the recording surface 16 of the image sensor 13 in relation to an image plane of a lens of the camera 3. If it is determined that the orientation state of the image sensor 13 is not optimal, ie, if, for example, a parallelism of the image sensor 13 or the recording surface 16 to the image plane is no longer present, a correction can be made via the actuator 15. A correction of the parallelism of the image sensor 13 is possible, for example, by a rotational movement of the image sensor 13 about the x-axis and / or the y-axis. For this purpose, updated positioning information for the image sensor 13 is determined as a function of the orientation state, which describes a relative position of the image sensor 13 in relation to the support structure 14, in whichthe image sensor 13 is oriented parallel, or at least with better parallelism, i.e., a smaller deviation from a completely parallel state, to the image plane of the camera. Subsequently, in a step S7, the positioning information stored in the memory device 20 is replaced by the updated positioning information. Replaced, older positioning information can either be discarded or stored as historical positioning data for later evaluation. In a step S8, the relative position of the image sensor 13 is then adjusted depending on the updated positioning information. Steps S7 and S8 can be carried out in any order. Adjusting the updated relative position of the image sensor 13 according to the updated positioning information can be implemented by controlling the actuator 15 by the control device 4, wherein aThe position currently set by the actuator 15 can be evaluated to determine suitable control signals. Depending on the type of actuator, the current position of the actuator 15 can be read out, for example, capacitively by the control device 4. 202304350 19 Updating the positioning information and correcting the relative position of the image sensor 13 makes it possible, for example, when the vehicle 1 is started up again, to adjust the image sensor 3 according to the stored positioning information. Permanent effects, which are attributable, for example, to a misalignment as a result of aging of the camera 3, can thus be compensated and balanced out over the service life of the camera 3 or the vehicle 1. Additionally or alternatively, the orientation state can also be an alignment of the image sensor 13 to the optical axis 17 of the camera 3 and / or an alignment of the image sensor 13 toa focal point of the camera 3. Such an alignment of the image sensor 13 can be achieved, for example, by a translational movement of the image sensor 13 along the x-axis, the y-axis, and / or the z-axis, wherein, analogously to the previous description, updated positioning information can be determined and used to correct the alignment of the image sensor 13. Preferably, the orientation state of the image sensor 13 is determined as a function of at least one image sensor measured value. The image sensor measured value can, for example, be obtained from an evaluation of the image content of a camera image recorded via the image sensor 13. This makes it possible to determine an incorrect position or misalignment of the image sensor 13 by the image sensor 13 itself or by a camera image recorded by the image sensor 13. The evaluation of the camera image can, for example, be carried out by the control device 4. TheThe orientation state can be determined, for example, by evaluating the sharpness of the camera image or by checking for an offset of the camera image relative to a projection area of ​​the optics of the camera 3. As updated positioning information, a position of the image sensor 13 can be determined, for example, in which a camera image generated via the image sensor 13 has the best sharpness, for example, the highest sharpness in the 202304350 20 corners and / or the highest sharpness in the center of the recording surface 16. To determine this position, the image sensor 13 can, for example, be moved successively by one or more increments along the degrees of freedom available for the movement of the image sensor 13, wherein, for each relative position of the image sensor, an evaluation of a camera image recorded in this position is performed with regard to an image property to be optimized, such as the aforementioned image sharpness or the like.is carried out. Additionally or alternatively, in a step S3, updated positioning information can also be determined based on operating time information describing the operating time of the camera 3. The operating time information can, for example, also be stored in the storage device 20 and reflect how long the camera 3 has currently been in operation. The operating time information can be continuously updated so that it always indicates the current operating time of the camera 3. In addition to a total operating time of the camera 3, the operating time information can also contain information on the operating time of the camera 3 under different climatic conditions. For this purpose, the operating time information can, for example, be determined during operation of the vehicle 1 depending on position data and / or weather data and stored, for example in the vehicle 1. Taking into account theOperating time information makes it possible to take into account certain aging effects that occur stochastically after a certain operating time. In a simple manner, only the operating time of the camera 3 needs to be taken into account, whereby, for example, the operating time information can be compared with an assignment table in order to determine current positioning information for the image sensor 13 that is suitable for the operating time. Subsequently, as described above, the updated positioning information can be stored and the relative position of the image sensor 13 can be set in steps S7 and S8. 202304350 21 The updated positioning information can also be determined in a parallel or alternatively performed step S4 as a function of a distance to an environmental object. The environmental object can, for example, be another road user whoin front of, next to or behind the vehicle and which is in particular in the detection range of the corresponding camera 3, whose image sensor 13 is to be adjusted. The distance to the surrounding object can be determined in particular via the distance sensor 11 of the vehicle 1. The adjustment of the relative position of the image sensor 13 can in particular be carried out such that the image sensor 13 is located in the correct focal point along the optical axis 17 of the camera 3 depending on the distance to the surrounding object. The image sensor 13 can for this purpose be moved translationally in particular parallel to the optical axis 17, i.e. along the z-direction. Subsequently, the relative position of the image sensor 13 can also be adjusted in step S8 according to the updated positioning information. The storage of the updated positioning information in the memory device 20 can be carried out at a depending on the distance to theThe determination of the current positioning information determined for the surrounding object can be carried out or optionally also omitted. In step S5, updated positioning information is determined as a function of an environmental model comprising several parameters. A temperature, an air pressure and / or an air humidity of the environment of the vehicle 1 can be used as parameters of the environmental model. The environmental model can in particular describe the influence of the parameters on the beam path inside the camera 3, so that influences of the environmental state on the imaging of the camera 3 can be compensated by positioning the image sensor 3 according to the updated positioning information determined using the environmental model. Preferably, a current value of at least one parameter of the environmental model is determined using the at least one environmental sensor 12 of the vehicle 1. 202304350 22 Subsequently, theThe relative position of the image sensor 13 is adjusted in step S8 according to the updated positioning information. The storage of the updated positioning information can be carried out with a current positioning information determined as a function of the distance to the surrounding object according to step S7 or optionally also omitted. In step S6, the determination of the updated positioning information takes place as a function of at least one measured or predicted driving state criterion. Preferably, the control device 4 is designed to move the image sensor 13 into a safety position by the actuator 14 if at least one measured or predicted state criterion describes the action or the impending action of mechanical forces above a limit value on the image sensor 13. The safety position can, for example, be a relative position of the image sensor 13 in which the actuator 15 is in aor several degrees of freedom up to a stop, so that it is at least partially supported and is itself exposed to no or at least only minor vibrations when vibrations act on the vehicle 1. As a result, the image sensor 13 or the respective camera 3 can be advantageously protected from mechanical damage. Updated positioning information can be used, in particular, for setting a new relative position of the image sensor 13 when one or more of the driving state criteria are met. The fulfillment of at least one state criterion can be determined by the control device 4. For this purpose, the control device 4 can, for example, evaluate camera images from the cameras 3 and / or sensor data from one or more sensors of the vehicle 1 in order to determine whether one or more of the, in particular predefined, state criteria are met. The action of above a limit valueThe mechanical forces acting on the vehicle 1 can be measured, for example, by an acceleration sensor (not shown) of the vehicle 1 or a sensor (not shown) of the vehicle 1 measuring the deflection of a chassis of the vehicle 1. A predicted driving state criterion can be determined, for example, by evaluating the camera images of one of the cameras 3, in particular the front camera 5, wherein the driving state criterion indicates whether potholes, curbs, or similar structures are detected in front of the vehicle, which, when driven over, will cause vibrations of the vehicle 1 and thus mechanical forces acting on the image sensor 13, which are expected to be above a limit value. Subsequently, the relative position of the image sensor 13 can also be adjusted in step S8 according to the updated positioning information. The storage of the updated positioning information can be carried out at a time depending on the distance toThe current positioning information determined from the surrounding object according to step S7 can be carried out or optionally omitted. Steps S2 to S6 can be carried out in any combination. It is possible in each case for the positioning information and / or the relative position of the image sensor 13 for the different update parameters to be determined using a calculation rule and / or a lookup table. The calculation rule and / or the lookup table can be stored, for example, in the control device 4 or in the memory device 20. A combination of calculation rule and lookup table can be implemented, for example, as a weighted combination. The updating of the positioning information depending on the orientation state of the image sensor 13 according to step S2 and / or using the operating time information according to step S3 can, for example, be carried out during aConfiguration mode of the camera 3. The configuration mode can be started, for example, when the vehicle 3 is started up and / or after a user of the vehicle 1 has selected a configuration mode, so that, in particular, long-term and gradually occurring changes in the relative arrangement of the image sensor 13 in the camera 3 can be compensated for. The cause of such slow changes can be, for example, aging effects, such as deformations due to forces acting during operation, the shrinkage of adhesive bonds, or the like. The updating of the positioning information as a function of the distance to the surrounding object according to step S4, as a function of the surrounding model according to step S5, and / or as a function of the at least one measured or predicted driving state criterion according to step S6, can preferably take place during driving of the vehicle. ThisThis allows the position of the image sensor 13 in the camera 3 to be adjusted according to the current conditions of the vehicle's surroundings. This allows, for example, effects due to temperature and other environmental variables, as well as the current traffic environment of the vehicle 1, to be taken into account.

Claims

202304350 25 claims 1. Method for operating a camera (3) of a vehicle (1), wherein the camera (3) comprises an image sensor (13), a support structure (14) and an actuator (15), wherein the image sensor (13) is connected to the support structure (14) via the actuator (15) and is movable relative thereto along one or more degrees of freedom, wherein the relative position of the image sensor (13) to the support structure (14) is set as a function of positioning information stored in a storage device (20), wherein the positioning information is determined at one or more times as a function of at least one orientation state of the image sensor (13), on the basis of operating time information describing an operating time of the camera (3), as a function of a distance to an object in the surroundings,is updated as a function of at least one measured or predicted driving state criterion and / or as a function of an environmental model comprising a plurality of parameters, wherein the relative position of the image sensor (13) is adjusted as a function of the updated positioning information.

2. Method according to claim 1, characterized in that an actuator is used by which the image sensor (13) is movable at least along the optical axis of the camera (3).

3. Method according to claim 1 or 2, characterized in that an actuator (15) is used by which the image sensor (13) is movable along at least two translational degrees of freedom and / or at least two rotational degrees of freedom relative to the support structure (14).

4. Method according to one of the preceding claims, characterized in that an actuator (15), in particular a MEMS actuator, is used which has a stationary first part (18),which is attached to the support structure (14), and a second part (19) which is movable relative to the first part (18) and is attached to the image sensor (13). 202304350 26 5. Method according to one of the preceding claims, characterized in that the at least one orientation state describes a parallelism of the image sensor (13) to an image plane of a lens of the camera (3), an alignment of the image sensor (13) to an optical axis (17) of the camera (3) and / or an alignment of the image sensor (13) to a focal point of the camera (3).

6. Method according to one of the preceding claims, characterized in that the orientation state of the image sensor (13) is determined as a function of at least one image sensor measured value.

7. Method according to one of the preceding claims, characterized in that the distance to the surrounding object is determined by means of a distance sensor (11) of the vehicle (1). 8.Method according to one of the preceding claims, characterized in that a distance to an environmental object, a temperature, an air pressure and / or an air humidity of the environment of the vehicle (1) is used as a parameter of the environmental model.

9. Method according to one of the preceding claims, characterized in that a current value of at least one parameter of the environmental model is determined using at least one environmental sensor (12) of the vehicle (1), in particular using a temperature sensor, a humidity sensor and / or an air pressure sensor.

10. Method according to one of the preceding claims, characterized in that the positioning information and / or the relative position are determined using a calculation rule and / or a lookup table.

11. Method according to one of the preceding claims, characterized in that the image sensor (13) is converted into a... 202304350 27 safety position is moved if at least one measured or predicted driving condition criterion describes the action or the impending action of mechanical forces above a limit value on the image sensor (13).

12. Method according to one of the preceding claims, characterized in that the positioning information is updated as a function of the orientation state of the image sensor (13) and / or based on the operating time information at the beginning of operation of the vehicle (1) and / or during configuration operation of the camera (3), and / or that the positioning information is updated as a function of the distance to the surrounding object, as a function of at least one measured or predicted driving condition criterion, and / or as a function of the surrounding model during operation of the vehicle (1).

13. Camera system comprising at least one camera (3) and a control unit (4),wherein the camera (3) comprises an image sensor (13), a support structure (14) and an actuator (15), wherein the image sensor (13) is connected to the support structure (14) via the actuator (15) and is movable relative thereto along one or more degrees of freedom, wherein the control device (4) is configured to adjust the relative position of the image sensor (13) to the support structure (14) as a function of positioning information stored in a memory device (20), wherein the control device (4) is further configured to adjust the positioning information at one or more points in time as a function of at least one orientation state of the image sensor (13), based on operating time information describing an operating time of the camera (3), as a function of a distance to an object in the surroundings,to update as a function of a measured or predicted driving condition criterion and / or as a function of an environmental model comprising several parameters and to adjust the relative position of the image sensor (13) as a function of the updated positioning information. 202304350 28 14. A vehicle comprising a camera system according to claim 13.

Images